Compare commits

..

85 Commits

Author SHA1 Message Date
Paul Hernandez b5f13d6903 Update README.md
add devin deep wiki badge for weekly updates

Signed-off-by: Paul Hernandez <60959+phernandez@users.noreply.github.com>
2026-06-14 11:08:45 -05:00
Drew Cain 6d06c4a4e7 docs(ci): fix stale basicmachines.co post-release step
The marketing site moved to basicmemory.com (repo basicmachines-co/basicmemory.com,
now Astro + React) and no longer renders a hardcoded version number anywhere in
its UI — src/components/sections/hero.tsx has no version string. The post-release
instruction to bump the version there is obsolete. Replace it across the runbook
(release.md), AGENTS.md post-release tasks, and the release/beta justfile reminders
with the current reality: no version bump; optionally add a dated blog post under
src/content/blog/ for significant releases; skip for patches.

Signed-off-by: Drew Cain <groksrc@gmail.com>
2026-06-13 14:53:52 -05:00
Drew Cain 16869867da fix(core): gate hybrid FTS relaxation with the service's eligibility rules
Addresses Codex review on #994: the hybrid FTS branch opted into
OR-relaxation for every query shape, but SearchService's relaxed FTS
path deliberately rejects short queries and numeric identifiers because
OR-relaxing them over-broadens — and in hybrid the relaxed FTS-only rows
normalize to 1.0 and can outrank the vector result the user wanted
(e.g. "SPEC 16", "root note 1", "New Feature").

relaxed_query_words now enforces the same eligibility as
SearchService._is_relaxed_fts_fallback_eligible: tokenize on
[A-Za-z0-9]+ and return None when there are fewer than three tokens or
any token is a pure digit (in addition to the existing quoted/boolean
guards). Both the SQLite and Postgres relaxed retries route through this
helper, so the hybrid path now relaxes exactly the query shapes the
service does.

Tests updated for the tightened eligibility (short + numeric queries no
longer relax); SQLite + Postgres relaxation suites green, full
repository/search-service/link-resolver suite green (442 passed), ty
clean.

Signed-off-by: Drew Cain <groksrc@gmail.com>
2026-06-13 14:50:15 -05:00
Drew Cain 18da6194f9 fix(core): Postgres parity for FTS punctuation/relaxation (CI green)
CI Postgres shard caught two issues invisible to the local SQLite suite:

1. Postgres _prepare_single_term regression: the new edge-punctuation
   strip ran after special-character cleaning, so an all-special-char
   term ("()&!:") collapsed to empty and skipped the existing
   NOSPECIALCHARS:* guard, emitting a malformed ":*". Folded the strip
   into the word handlers so every guard survives, and added a
   single-word empty guard.

2. Backend-specific test assumptions. Four tests in
   test_search_repository.py (run under both backends via the
   search_repository fixture) asserted SQLite FTS5 syntax and
   SQLite-only strict-miss behavior. Postgres to_tsquery('english', ...)
   auto-strips stopwords, so "When did Melanie paint a sunrise?" already
   matches under strict AND. Made the four tests backend-aware via the
   existing is_postgres_backend() helper, and switched the relaxation
   integration test to a query with a word absent from the doc
   ("hiking") so the strict miss holds on both backends.

Reproduced and fixed against real Postgres (testcontainers): full
search test surface green on both backends (53 passed Postgres,
2968 SQLite), ruff + ty clean.

Signed-off-by: Drew Cain <groksrc@gmail.com>
2026-06-13 14:50:15 -05:00
Drew Cain a6d0784335 fix(core): repair FTS half of hybrid search for natural-language queries
Hybrid search was silently running vector-only on natural-language
queries — the FTS branch contributed zero candidates. Two causes in the
SQLite (and parallel Postgres) FTS query preparation:

1. Sentence punctuation forced phrase matching. A question like
   "When did Melanie paint a sunrise?" reached FTS5 as the exact phrase
   '"When did Melanie paint a sunrise?"*', which matches no document.
   The FTS5 tokenizer ignores this punctuation in the index, so
   stripping it from word edges loses nothing — but leaving it disabled
   the entire FTS contribution. _prepare_single_term now strips
   ?!.,;: from word edges of multi-word queries (interior characters —
   hyphens, slashes in permalinks/paths — untouched).

2. No relaxation when strict all-terms-AND matched nothing. Questions
   rarely have every word in one document, so even after (1) the
   strict AND returned zero rows. The hybrid path now retries once with
   an OR-joined, stopword-filtered, content-term query when the strict
   query is empty. bm25/ts_rank still rank multi-term matches first, and
   fusion with the vector branch keeps relaxed lexical candidates from
   dominating precision.

The relaxation is gated behind a new allow_relaxed=False parameter on
SearchRepositoryBase.search; only _search_hybrid opts in. Strict FTS
behavior (search_type=text, title, permalink, link resolution) is
unchanged — the service layer keeps its own conservative fallback.
No config flag, default-safe.

Discovered via the benchmark harness: two different fusion algorithms
produced byte-identical rankings across 1,986 queries (impossible with
two live sources), and instrumentation confirmed fts=0 on 40/40
sampled LoCoMo queries.

Benchmark impact (corrected LoCoMo, 1,986 queries, same index,
retrieval metrics — every category improves, no regression):
  recall@5  0.745 -> 0.823  (+7.9)
  MRR       0.618 -> 0.718  (+10.0)
  headline  r5 0.734 -> 0.801, MRR 0.621 -> 0.706
Largest gains on open_domain (+0.10 r5) and adversarial (+0.12 r5);
smallest on temporal (+0.003 r5 / +0.02 MRR).

Tests: punctuation no longer phrase-quotes; relaxation builds the
expected OR query and respects boolean/quoted/short-query intent; the
hybrid opt-in surfaces a partial-overlap document while the default
strict path still returns empty. Parallel coverage for Postgres. Full
SQLite unit suite green (2968 passed); ty + ruff clean.

Signed-off-by: Drew Cain <groksrc@gmail.com>
2026-06-13 14:50:15 -05:00
Drew Cain 232f469065 chore: update version to 0.22.1 for v0.22.1 release
Signed-off-by: Drew Cain <groksrc@gmail.com>
2026-06-12 22:34:04 -05:00
Drew Cain 5a08cfd9ac docs(core): add v0.22.1 changelog entry
Signed-off-by: Drew Cain <groksrc@gmail.com>
2026-06-12 22:33:34 -05:00
Drew Cain b997d858cd fix(sync): also exclude orphan DB projects absent from config (#949)
Address Codex review feedback. The previous path-only filter dropped an
implicit protection: get_project_mode() defaults projects missing from
config to CLOUD, so the old mode-based guard skipped stale DB rows that
had been removed from config. With a path-only check, an orphan row with
an absolute path would pass and background sync/watch could still mutate
a directory the user already removed from config (config is the source of
truth) if reconciliation was skipped or failed.

Introduce BasicMemoryConfig.is_locally_syncable(name, path), which
requires both config membership and an absolute path, and use it from
both the background sync selection and the watch cycle so the two paths
cannot diverge. Add direct unit tests for the helper plus an orphan-row
regression test for the watch selection.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Drew Cain <groksrc@gmail.com>
2026-06-12 18:02:18 -05:00
Drew Cain 598965c389 test(sync): use OS-absolute paths in watch selection tests
The absolute-path sync guard from the previous commit now checks every
project's path, not just cloud-mode ones. Three existing watch-selection
tests hardcoded POSIX paths like "/tmp/alpha", which are absolute on
Linux/macOS but not on Windows (no drive letter), so the guard filtered
them out and the tests failed on windows-latest.

Build the project paths from the tmp_path fixture so they are absolute on
every platform. Production paths are unaffected: real local projects are
always resolved to OS-absolute paths at creation.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Drew Cain <groksrc@gmail.com>
2026-06-12 18:02:18 -05:00
Drew Cain 8dd6451dfe fix(sync): skip projects without an absolute local path (#949)
A project entry in config.json with an empty path (e.g. `{"path": ""}`)
caused background sync and the watch service to adopt the process cwd as
the project root, injecting Basic Memory frontmatter into unrelated
markdown files.

The existing guards only skipped a project when get_project_mode()
returned CLOUD. But ProjectEntry.mode defaults to LOCAL, so an empty- or
relative-path entry without an explicit mode slipped through, and
Path("") resolves against the current working directory.

Gate local sync and watching on the path itself: any project whose path
is not absolute is excluded, regardless of mode. Legitimate local
projects are always resolved to absolute paths at creation, and cloud
projects with a real local bisync copy keep their absolute path and are
still synced/watched.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Drew Cain <groksrc@gmail.com>
2026-06-12 18:02:18 -05:00
Drew Cain d46c68806e test(mcp): add workspace to write_note expected signature contract
The signature contract test pins exact tool parameters; update it for
the new workspace parameter.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Drew Cain <groksrc@gmail.com>
2026-06-12 12:19:56 -05:00
Drew Cain 1d9ff3c94f feat(mcp): add workspace parameter to write_note for parity with edit_note
write_note now accepts workspace= alongside project= and project_id=,
matching the parameter surface and docstring guidance already present in
edit_note. The _compose_workspace_project_route helper is added locally
(mirroring the edit_note pattern) so agents can route writes to same-named
projects across workspaces using workspace/project qualified syntax.

Closes #882

Other write-path tools that share the same gap (move_note, delete_note)
are noted here but left for a separate PR to keep this change focused.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Drew Cain <groksrc@gmail.com>
2026-06-12 12:19:56 -05:00
phernandez 515b2c8365 fix(api): point fresh installs at project setup when resolve finds an empty projects table
On a brand-new config dir, model_post_init bootstraps a 'main' default in
config.json but the one-shot CLI path never runs the server-lifespan
reconciliation that would create its database row, so the first read fails
with a bare "Project not found: 'main'" — which reads as a broken install
rather than a missing first-run step. When resolution misses and the
projects table is empty, the 404 now names the setup command.

Follow-up to #974/#985/#987 (which repair the default during project add
but cannot help when the first action is a read). Refs #974.

Signed-off-by: phernandez <paul@basicmachines.co>
2026-06-12 10:27:16 -05:00
phernandez 32a1c208b3 fix(core): preserve an existing database default when repairing a missing config default
Follow-up to #985 (the #974 fix by @rudi193-cmd), addressing the codex P2
raised on #987: when config's default_project has no database row but the
database still holds a valid default of its own, promoting the just-added
project would silently steal that default. The repair now repoints config
at a surviving database default — matching synchronize_projects, which
treats the database default as authoritative — and promotes the added
project only when no usable database default exists (no default at all,
or one unknown to config, which set_default_project rejects and
reconciliation deletes).

Also removes an unused ProjectConfig import from the #985 regression test
(strict ruff failure; CI's --fix lint masked it).

Refs #974

Signed-off-by: phernandez <paul@basicmachines.co>
2026-06-12 09:44:21 -05:00
rudi193-cmd 25732b2fe2 fix: return promoted default state from project create API
When add_project promotes a new project because the configured default is missing from the database, return the persisted default flag instead of echoing the request flag.

Signed-off-by: rudi193-cmd <rudi193@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-12 09:03:36 -05:00
rudi193-cmd b92b0340d5 fix: promote first project when config default is missing from DB
Fresh CLI installs auto-create main in config but never sync it to the
projects table; the first project add now becomes default when the
configured default has no DB row. Fixes #974.

Signed-off-by: rudi193-cmd <rudi193@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-12 09:03:36 -05:00
phernandez 0247ef0ead fix(cli): defer FastAPI and app imports out of CLI startup
Every basic-memory CLI invocation paid roughly 2 seconds of module-import
cost before any work started, which blew the Claude Code plugin's
SessionStart hook budget on cold machines (#886). The cost came from
module-level imports that pulled the entire server stack into CLI startup:

- mcp/async_client.py imported FastAPI at module level, so every consumer
  of get_client() loaded FastAPI even for cloud-routed or help-only paths.
- mcp/clients/*.py imported call_* helpers from basic_memory.mcp.tools.utils,
  which executes the whole tools package __init__ — every MCP tool module
  plus fastmcp and the mcp SDK.
- mcp/project_context.py imported fastmcp.Context and ToolError eagerly.
- CLI command modules (tool, ci, schema) imported MCP tool functions at
  module level; db and the import_* commands pulled SQLAlchemy/Alembic and
  the markdown/file-service stack; status/doctor/orphans/command_utils
  imported ToolError (the mcp SDK) and basic_memory.db.
- schemas/base.py imported dateparser (~0.13s) for one helper function.

The fix only defers imports to the point of use (no behavior changes):
FastAPI now loads inside _resolve_local_asgi_database alongside the
existing lazy api.app import, so it is only paid when a request actually
routes through the in-process ASGI transport; the typed clients import
call_* per method; project_context uses PEP 563 annotations with Context
under TYPE_CHECKING; the CLI command modules import their heavy
dependencies inside the command bodies. Tests that patched the old
module-level aliases now patch the source modules instead.

Measured on a warm cache (python -X importtime / wall time):
- import basic_memory.cli.main: 1.92s -> 0.45s
- bm --help: 2.40s -> 0.52s
- bm tool search-notes --help: 2.40s -> 0.86s

A regression test asserts that importing the CLI entry module with full
command registration leaves fastapi, sqlalchemy, alembic, fastmcp, mcp,
basic_memory.api.app, basic_memory.db, basic_memory.markdown,
basic_memory.mcp.tools, and basic_memory.services out of sys.modules.

Fixes #886

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: phernandez <paul@basicmachines.co>
2026-06-12 09:03:08 -05:00
phernandez 253e240d68 fix(core): use (type, id) keys in vector search hydration to prevent id collisions
Root cause: entity, observation, and relation rows in search_index carry
ids from independent auto-increment sequences, so rows of different types
routinely share the same numeric id (guaranteed in young databases).
_search_vector_only parsed each vector hit's chunk_key (e.g. 'entity:4:0')
but discarded the type, and _fetch_search_index_rows_by_ids keyed its
result dict by bare row.id with no type discrimination. Whichever row the
database returned last clobbered the other in the dict; the clobbered hit
then hydrated against the wrong row or found None and was silently
dropped from results. The FTS-filter branch already guarded this with
(id, type) tuples, but the primary vector lookup path and the hybrid
fusion maps missed the same treatment.

Fix: introduce a SearchIndexKey = tuple[str, int] alias and key every map
in the vector/hybrid retrieval path by (type, id) — the similarity and
chunk maps in _search_vector_only, the _fetch_search_index_rows_by_ids
result, the FTS-filter allowed keys, and the rows/fts/vec/fused score
maps in _search_hybrid. The SQL stays unchanged; bare ids are deduped
before the IN query and rows are discriminated by row.type when building
dict keys.

Tests: end-to-end SQLite regression test indexes an entity row and a
relation row sharing id 7, syncs vectors for both, and asserts vector
search returns both rows (and that the entity survives a
search_item_types filter); a hybrid fusion unit test asserts an entity
and relation sharing id 1 stay distinct with single-source scores. Both
fail without the fix. Existing mocked vector tests updated for tuple keys.

Fixes #982

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: phernandez <paul@basicmachines.co>
2026-06-12 09:03:04 -05:00
phernandez b3bdd5914f fix(cli): write config.json atomically and isolate auto-update in mcp routing tests
Root cause of the test_mcp_sse_forces_local KeyError('FORCE_LOCAL') flake
(#940, Python 3.14 leg): the stdio variants of the mcp routing tests invoke
`bm mcp`, which starts a real background auto-update daemon thread before
mcp_server.run (the tests only mock run). That thread hits PyPI and then
rewrites config.json (auto_update_last_checked_at) with an in-place
Path.write_text, which truncates the file before writing. If that write lands
while a later test's CLI invocation is reading config.json (the app callback's
CliContainer.create()), load_config() sees empty/partial JSON and raises
SystemExit. CliRunner.invoke swallows it, the mocked mcp_server.run never
executes, and the test dies with KeyError on env_at_run['FORCE_LOCAL'] —
exactly the observed CI failure shape. Nothing is 3.14-specific; that leg
only shifted the timing.

The same torn write is user-visible in production: the MCP stdio server
re-reads config.json on mtime change and load_config() exits the process on
invalid JSON if it races a CLI save.

Fix: save_basic_memory_config writes a per-process/per-thread sibling temp
file and publishes it with os.replace, so readers always observe either the
old or the new complete document. The regression test injects an interrupted
write and asserts the published config stays untouched; it fails against the
old in-place write.

Test hardening: the stdio routing tests stub run_auto_update so no PyPI call
or config write leaks across tests, and all four transport tests now assert
result.exit_code == 0 so a future pre-run failure surfaces its real error
instead of a KeyError.

Refs #940

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: phernandez <paul@basicmachines.co>
2026-06-12 09:03:00 -05:00
phernandez 85c701b8c2 fix(sync): serialize in-memory SQLite sessions so concurrent rollbacks cannot destroy writes
Root cause of the test_sync_entity_circular_relations CI failure (#940,
len(entity_b.outgoing_relations) == 0): the in-memory SQLite URL
(sqlite+aiosqlite://) falls back to SQLAlchemy's StaticPool, which hands the
same DBAPI connection to every concurrently checked-out session. Concurrent
asyncio tasks therefore share one SQLite transaction scope. A rollback issued
through one session — scoped_session's exception handler, or the pool's
reset-on-return at connection checkin (~40 real ROLLBACKs per sync, measured)
— also rolls back any other task's executed-but-uncommitted statements.

During batch indexing, per-file tasks run concurrently (asyncio.gather bounded
by index_entity_max_concurrent). If a sibling session's checkin ROLLBACK lands
between one task's relation INSERT and its COMMIT, the relation row is silently
erased: no error is raised, the sync reports success. The final sync-level
resolve_relations() pass cannot heal this because it only re-queries rows with
to_id IS NULL — the destroyed INSERT leaves no row at all.

Fix: give MEMORY-type engines a single-connection AsyncAdaptedQueuePool
(pool_size=1, max_overflow=0). The lone connection keeps the in-memory
database alive for the engine's lifetime (the reason StaticPool was used),
while the blocking checkout serializes sessions at transaction granularity,
restoring the isolation the repositories assume. File-based SQLite and
Postgres engines are unchanged; production never uses MEMORY engines.

The regression test pins the invariant directly: a session that rolls back in
one task must never destroy another task's uncommitted writes. It fails
deterministically against StaticPool and passes with the serialized pool.

tests/repository/test_entity_repository.py held a scoped session open while
calling a repository method that opens its own session — tolerated on a shared
connection, a deadlock under a serialized pool — so the nested call moved out
of the session block.

Refs #940

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: phernandez <paul@basicmachines.co>
2026-06-12 09:03:00 -05:00
Paul Hernandez 1ad3a350ad fix(mcp): close out the #952 manual verification findings (#981)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-06-11 17:59:35 -05:00
Paul Hernandez a148e72f56 feat(plugins): manual-pages flow — manpage seed schema, flow docs, verification fixes (#971)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-06-11 16:27:30 -05:00
Drew Cain 33e741fd29 fix(mcp): resolve workspace display names and tenant ids in qualified project routes
The edit_note docstring (and write_note in PR #964) advertise that the
workspace segment of a "workspace/project" route may be a slug, name, or
tenant_id, but resolve_workspace_project_identifier() only matched against
WorkspaceInfo.slug. Cloud routes using a display name or tenant UUID failed
with "Workspace ... was not found" despite the documented contract.

Extend the first-segment matching (only the matching logic; the overall
resolution flow is unchanged) to honor, in priority order:

1. slug (casefold) — unchanged, checked first so working routes keep meaning
2. tenant_id — exact match on the opaque id
3. display name (casefold) — fails fast on collisions, listing candidate slugs

A name that collides with another workspace's slug resolves to the slug owner.
Unknown identifiers raise a not-found error naming the forms that were tried.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Drew Cain <groksrc@gmail.com>
2026-06-11 15:28:19 -05:00
Drew Cain c9770375e9 feat(mcp): add title and tags annotations to all MCP tool decorators (issue #826 phase 1)
Adds human-readable `title` and categorization `tags` to all ~24 @mcp.tool
decorators across the MCP tools package. FastMCP 3.3.1 (pinned in pyproject.toml)
supports both fields natively. Tags used: notes, search, projects, cloud, schema,
navigation, canvas, ui.

Extends test_tool_contracts.py with an async test that asserts every registered
tool has a non-empty title and at least one tag to prevent future regressions.

output_schema is explicitly deferred as a follow-up (phase 2): it requires
per-tool design decisions about which tools reliably return structured JSON and
how to handle tools that return str|dict depending on output_format.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Drew Cain <groksrc@gmail.com>
2026-06-11 15:21:25 -05:00
Drew Cain 0811c48252 fix(mcp): normalize note_types/entity_types/categories on direct call path and reject non-string list elements
Codex review of PR #962 identified two real issues:

1. CLI bypass: the BeforeValidator(parse_str_list) on note_types, entity_types, and
   categories only fires through MCP/Pydantic validation. The CLI path in
   cli/commands/tool.py calls search_notes() directly, so `bm tool search-notes
   --type note,task` arrived as note_types=["note,task"] and matched nothing.
   Fix: add in-body parse_str_list() normalization for all three params (mirroring
   the existing parse_tags() call for tags on the same code path).

2. Silent stringify: parse_str_list used str(raw) in the list branch, so [42] became
   ["42"] before Pydantic saw it, accepting invalid input as a no-result search
   instead of rejecting it. Fix: guard against non-string list elements and return
   the original value unchanged so Pydantic rejects it with a clear error.

Tests added: annotation-level split tests for note_types/entity_types/categories,
non-string-element rejection tests, async direct-call regression for note_types,
and unit-level parse_str_list non-string list tests in test_coerce.py.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Drew Cain <groksrc@gmail.com>
2026-06-11 14:38:24 -05:00
Drew Cain 747e64e5e9 feat(mcp): comma-split note_types/entity_types/categories in search_notes (#930)
Add `parse_str_list` to utils.py — like `parse_tags` but without stripping
'#' — and wire it as the BeforeValidator for note_types, entity_types, and
categories in search_notes. This makes passing "note,task" or
'["note","task"]' work correctly instead of being wrapped as a single literal
value by coerce_list.

coerce_list is left unchanged; canvas and other callers that depend on its
wrap-single-string behaviour are unaffected.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Drew Cain <groksrc@gmail.com>
2026-06-11 14:38:24 -05:00
Drew Cain be00df27c7 fix(api): allow setting default project when none is currently set
The v2 set_default_project_by_id handler fetched the current default
solely to echo it as old_project in the response, and raised 404
"No default project is currently set" when none existed. That guard
(marked # pragma: no cover, never tested) made the bootstrap/recovery
case impossible: with no default row in the DB, `bm project default
<name>` always failed -- yet that is exactly the command you reach for
when no default is set.

ProjectStatusResponse.old_project is already Optional, so the guard
served no schema requirement. Remove it and build old_project only
when a previous default exists, else pass None.

Add a regression test that clears is_default, sets a default via the
endpoint, and asserts 200, old_project is None, the new project is
default, and a follow-up read-back returns it.

Closes #975

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Drew Cain <groksrc@gmail.com>
2026-06-11 14:17:35 -05:00
Drew Cain 3ce42de57e test(mcp): narrow read_note return before string assertions for ty
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Drew Cain <groksrc@gmail.com>
2026-06-11 13:30:16 -05:00
Drew Cain a143072d35 fix(core): require line-anchored frontmatter fences in file_utils
has_frontmatter(), parse_frontmatter(), and remove_frontmatter() detected
frontmatter by substring/split (`content.startswith("---")` plus
`content.split("---", 2)`) rather than by line-anchored fences. A single-line
string that merely starts with `---` — e.g. `---\nstatus: active\n---\nBody`
where `\n` are literal backslash-n characters, a common CLI/agent input shape —
was misread as frontmatter: yaml parsed `\nstatus` as a key that got merged into
the note's YAML on disk, and the body was silently transformed.

Introduce a shared `_split_frontmatter()` helper that anchors both fences to
their own lines (`^---[ \t]*$`), tolerating leading blank lines so dedented
heredoc-style content still parses. All three public helpers now delegate to it,
preserving existing behavior for valid frontmatter (BOM stripping, empty -> {},
non-dict -> ParseError, and the existing ParseError messages callers assert on).

Closes #972

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Drew Cain <groksrc@gmail.com>
2026-06-11 13:30:16 -05:00
Drew Cain 497a4e0a43 fix(cli): preload deferred rich/typer modules before in-place upgrade
brew upgrade / uv tool upgrade delete the running install's files while the old process is still alive. rich defers importing rich._emoji_codes until print time and typer defers typer.rich_utils until excepthook time, so printing the post-upgrade status message crashed the exiting process with ModuleNotFoundError (and the traceback renderer crashed too). Import both before launching the upgrade subprocess, while the files still exist.

Signed-off-by: Drew Cain <groksrc@gmail.com>
2026-06-11 12:55:04 -05:00
Drew Cain ec5fac8d76 chore(ci): wait for the release PR merge to land before tagging
Addresses Codex review on PR #961: gh pr merge may not complete synchronously if merge gates exist (or while GitHub computes mergeability), so the recipes now fall back to queueing auto-merge and poll origin/main for the rebased bump commit (up to 5 minutes) before tagging, with manual finish instructions on timeout.

Signed-off-by: Drew Cain <groksrc@gmail.com>
2026-06-11 01:35:34 -05:00
Drew Cain 05bfd0f04e chore(ci): remove user-specific absolute paths from release runbook
Signed-off-by: Drew Cain <groksrc@gmail.com>
2026-06-11 01:35:34 -05:00
Drew Cain 338f357f55 chore(ci): route release recipes through PRs and refresh release runbook
main's ruleset now rejects direct pushes, which broke 'just release' mid-flight for v0.22.0. The release/beta recipes now land the version bump via a rebase-merged release PR and tag the rebased commit (found by commit subject). Adds a changelog-on-main pre-flight check, and updates AGENTS.md and the release runbook to match, including the docs.basicmemory.com steps which still described the old src/pages site.

Signed-off-by: Drew Cain <groksrc@gmail.com>
2026-06-11 01:35:34 -05:00
Drew Cain 8e7825ba01 chore: update version to 0.22.0 for v0.22.0 release
Signed-off-by: Drew Cain <groksrc@gmail.com>
2026-06-11 00:53:13 -05:00
Drew Cain dc29ba2a00 docs(core): add v0.22.0 changelog entry
Signed-off-by: Drew Cain <groksrc@gmail.com>
2026-06-11 00:53:13 -05:00
Drew Cain fe9b2e9c95 Add glama.json to claim the Glama MCP directory listing (#953) 2026-06-11 00:02:21 -05:00
Paul Hernandez 650f88a2c5 docs(cli): document personal-vs-team cloud sync semantics (#947)
Folds in the intent of #857 reconciled with the post-#920 push/pull reality. Closes #851.

Co-authored-by: Drew Cain <groksrc@users.noreply.github.com>
Signed-off-by: phernandez <paul@basicmemory.com>
2026-06-10 14:50:28 -05:00
Paul Hernandez 2f7ef136de perf(ci): lean CI — testmon-select branches, shard postgres, drop depot, remove bossbot (#945)
Branch builds testmon-select against cached baselines; 3-shard Postgres on 3.14 only; free runners; 120s hang ceiling; non-code changes skip the matrix; BM Bossbot and infographic machinery removed.

Signed-off-by: phernandez <paul@basicmemory.com>
2026-06-10 14:21:57 -05:00
Drew Cain 0a3a6bbd96 test(mcp): integration coverage for colliding observation permalinks (#926)
Refs #909. Integration coverage over the #931 fix.

Signed-off-by: phernandez <paul@basicmemory.com>
2026-06-10 14:05:05 -05:00
Drew Cain 7bb7664fae fix(core): use Observation.permalink in build_context to match the search index (#946)
Closes #929

Signed-off-by: phernandez <paul@basicmemory.com>
2026-06-10 14:05:02 -05:00
Paul Hernandez db578ccfdb fix(mcp): recover edit_note when file exists on disk but is not indexed (#934)
Closes #581

Signed-off-by: phernandez <paul@basicmemory.com>
2026-06-10 14:04:59 -05:00
Paul Hernandez df485aa5a4 fix(mcp): tighten search_notes tags input and normalize for direct callers (#941)
Refs #910. Follow-up to #932.

Signed-off-by: phernandez <paul@basicmemory.com>
2026-06-10 14:04:56 -05:00
K Jagadeeswara Reddy 44ecec2917 test(mcp): assert tags param and tag: shorthand comma equivalence
Integration regression test for #910: the tags= parameter and the
tag:alpha,beta query shorthand must return the same results for the
same comma string.

Cherry-picked from #918. The validator swap itself (coerce_list ->
parse_tags) landed separately via #932; this carries the remaining
test-only piece with original authorship.

Signed-off-by: K Jagadeeswara Reddy <social@jagadeeswar.com>
2026-06-10 12:04:07 -05:00
Paul Hernandez 49041a5168 docs(skills): fix npx skill install docs and ignore docs assets (#927)
Refs #930 docs cleanup; adds docs/assets to gitignore.

Signed-off-by: phernandez <paul@basicmemory.com>
2026-06-10 01:30:50 -05:00
Paul Hernandez 8cbe1634b6 fix(cli): don't let bm cloud setup overwrite an existing rclone remote (#923)
Closes #922

Signed-off-by: phernandez <paul@basicmemory.com>
2026-06-10 01:02:43 -05:00
Paul Hernandez c4b651f5b0 fix(mcp): accept page/page_size in read_note for parity with sibling tools (#933)
Closes #883

Refs #882

Signed-off-by: phernandez <paul@basicmemory.com>
2026-06-10 00:03:22 -05:00
Paul Hernandez ca9a4d9c12 fix(mcp): split comma-separated tags in search_notes tags param (#932)
Closes #910

Signed-off-by: phernandez <paul@basicmemory.com>
2026-06-09 22:23:26 -05:00
Paul Hernandez aa9594d82a perf(ci): disable semantic search in default test fixtures (#938)
Disable the silently-on semantic embedding stack in default test fixtures and deselect on-demand benchmarks from CI int jobs. int SQLite 337s -> 110s, Postgres unit ~25min -> ~13min.

Signed-off-by: phernandez <paul@basicmemory.com>
2026-06-09 21:55:21 -05:00
Paul Hernandez 93494b8c13 ci(ci): gate bossbot on passing tests (#937)
Signed-off-by: phernandez <paul@basicmachines.co>
2026-06-09 20:47:34 -05:00
Paul Hernandez 93ed34001b fix(core): disambiguate truncated observation permalinks to prevent index collisions (#931)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 20:20:46 -05:00
Paul Hernandez ec94feb6a4 Merge branch 'main' into codex/fix-bossbot-assets-publish 2026-06-09 19:23:27 -05:00
Paul Hernandez 831b9141a5 ci(ci): use pytestmon and Depot runners (#928)
Signed-off-by: phernandez <paul@basicmachines.co>
2026-06-09 19:20:08 -05:00
phernandez 2de19713f6 fix(ci): make BM Bossbot asset cleanup idempotent
Signed-off-by: phernandez <paul@basicmachines.co>
2026-06-09 19:14:13 -05:00
phernandez 79fcfbce6a fix(ci): restrict BM Bossbot to trusted PR authors
Signed-off-by: phernandez <paul@basicmachines.co>
2026-06-09 17:16:49 -05:00
phernandez 3d22ba3004 fix(ci): harden BM Bossbot finalization
Signed-off-by: phernandez <paul@basicmachines.co>
2026-06-09 16:32:06 -05:00
phernandez 62229d9d0a fix(ci): address BM Bossbot PR feedback
Signed-off-by: phernandez <paul@basicmachines.co>
2026-06-09 15:55:36 -05:00
phernandez 03ba268cb1 feat(ci): add BM Bossbot PR gate
Signed-off-by: phernandez <paul@basicmachines.co>
2026-06-09 15:18:40 -05:00
Paul Hernandez de53e0ecc5 feat(cli): per-workspace rclone remotes for Team push/pull (#920)
Signed-off-by: phernandez <paul@basicmachines.co>
2026-06-08 17:39:37 -05:00
Paul Hernandez 4128cac9ab docs(core): add Basic Machines agent style guidance (#921)
Signed-off-by: phernandez <paul@basicmachines.co>
2026-06-08 17:09:50 -05:00
Paul Hernandez 9b53d7863f feat(cli): add Team-safe cloud push/pull and gate sync to Personal workspaces (#917)
Adds additive, git-style `bm cloud push`/`pull` that are safe on shared Team workspaces (never delete on the destination; conflicts abort by default with `--on-conflict {fail|keep-local|keep-cloud|keep-both}`), and gates the destructive `bm cloud sync`/`bisync` mirrors to Personal workspaces. Closes #858. Longer-term Team-safe reconciler tracked in #862; workspace-scoped mount info (Codex P1) tracked as a follow-up.
2026-06-08 14:08:49 -05:00
Paul Hernandez a8d034b940 fix(mcp): point move mismatch guidance at landing path (#916)
Signed-off-by: phernandez <paul@basicmachines.co>
2026-06-08 10:42:38 -05:00
Paul Hernandez 7c0937f658 fix(mcp): validate navigation pagination and fix recent_activity project display (#915)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 23:03:00 -05:00
Paul Hernandez 8570d96bad fix(cli): align bm tool commands with MCP (error exits, overwrite, category, default) (#913)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 22:56:43 -05:00
Paul Hernandez 480a2d9468 fix(mcp): resolve memory:// in move_note and stop false cross-project rejections (#914)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 22:44:08 -05:00
Paul Hernandez df7452e3ba fix(core): make note_types search filter case-insensitive (#912)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 22:44:04 -05:00
Paul Hernandez 85a8e59d0f fix(core): split comma-separated tags in parse_tags (#911)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 22:44:01 -05:00
Paul Hernandez 07cb7a606b docs(core): mark LiteLLM provider experimental (#899)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 20:13:54 -05:00
Paul Hernandez 0ef03edde6 feat(cli): add --type to the write-note tool command (#907)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 20:06:44 -05:00
Paul Hernandez efe43a10ea feat(cli): add --wait and --timeout to bm status (#906)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 19:58:02 -05:00
Paul Hernandez 4fe6fe09c8 feat(core): add observation category filter to search (#908)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 19:57:19 -05:00
Paul Hernandez 8acdb49a41 fix(core): reuse a single embedding provider per process (#903)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 19:14:07 -05:00
Paul Hernandez f7304bf553 feat(cli): improve workspace and cloud bisync command discoverability (#905)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 18:18:17 -05:00
Paul Hernandez 5b034f081d fix(core): self-heal corrupt FastEmbed model cache (#900)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 17:37:30 -05:00
Paul Hernandez 271c883ea8 fix(core): load sqlite-vec for embedding-status query (#901)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 17:37:27 -05:00
Paul Hernandez 816ee85fb9 fix(core): prevent asyncpg engine-dispose crash on Postgres backend (#902)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 17:37:23 -05:00
Paul Hernandez 3ba3a9504d fix(mcp): stop move_note reporting false success across boundaries (#904)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 17:37:20 -05:00
Paul Hernandez 1667cdc000 test(ci): normalize setup overwrite assertion (#898)
Signed-off-by: phernandez <paul@basicmachines.co>
2026-06-06 23:20:06 -05:00
Sourish Chakraborty f916662ff3 fix(core): allow cross-project context traversal 2026-06-06 21:43:21 -05:00
DoubleDeeRuffy 0c9800cd3b fix(sync): use strict deferred relation resolution 2026-06-06 21:43:09 -05:00
Adit Karode 476239d878 feat(cli): add delete-note tool command 2026-06-06 21:41:46 -05:00
DoubleDeeRuffy 20bb19f4cd fix(mcp): resolve write-note overwrite conflicts 2026-06-06 21:41:36 -05:00
tk f6565b9d23 fix(core): L2-normalize FastEmbed vectors (#843)
L2-normalizes FastEmbed output vectors at the provider boundary so SQLite vector scoring keeps its unit-vector contract for custom FastEmbed models such as multilingual MiniLM variants.

Zero vectors are preserved as-is to avoid division errors, and the provider tests cover both non-unit vectors and zero-vector behavior.

Verification:
- uv run pytest tests/repository/test_fastembed_provider.py -q
- uv run ruff check src/basic_memory/repository/fastembed_provider.py tests/repository/test_fastembed_provider.py
- uv run ruff format --check src/basic_memory/repository/fastembed_provider.py tests/repository/test_fastembed_provider.py

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: tk-pkm111 <133480534+tk-pkm111@users.noreply.github.com>
Signed-off-by: phernandez <paul@basicmachines.co>
2026-06-06 17:15:19 -05:00
Aarish Alam b6e8c636ce feat(core): add LiteLLM embedding provider (#809)
Adds LiteLLM as a semantic embedding provider, including provider configuration, vector normalization, live-provider evaluation tooling, and documentation for OpenAI, Cohere, Azure Foundry, Azure OpenAI, and NVIDIA NIM-style cases.

Maintainer follow-up on this PR added provider hardening, asymmetric document/query embedding support, dimension-forwarding controls, SQLite/Postgres vector invalidation coverage, and the repeatable live LiteLLM harness.

Verification:
- Full base-repo Tests workflow passed for 3d4e092ceb: https://github.com/basicmachines-co/basic-memory/actions/runs/27072071785
- Live LiteLLM harness passed locally for OpenAI, Cohere, and Azure Foundry.

Co-authored-by: Aarish Alam <arishalam121@gmail.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Signed-off-by: RheagalFire <arishalam121@gmail.com>
Signed-off-by: phernandez <paul@basicmachines.co>
2026-06-06 17:13:09 -05:00
Paul Hernandez 442fd1523c fix(plugins): include codex in shared version bump (#897)
Signed-off-by: phernandez <paul@basicmachines.co>
2026-06-05 18:34:57 -05:00
Paul Hernandez ce4b5d47a0 fix(skills): reject invalid frontmatter YAML (#896)
Signed-off-by: phernandez <paul@basicmachines.co>
2026-06-05 13:14:27 -05:00
229 changed files with 17708 additions and 1662 deletions
+68
View File
@@ -0,0 +1,68 @@
---
name: code-review
description: Use when reviewing Basic Machines code for house style, architecture risk, pre-merge hardening, or whether a change fits basic-memory/basic-memory-cloud conventions.
license: MIT
---
# Basic Machines Review
Use this skill for repo-local review passes where ordinary code review needs Basic Machines
house style and architecture judgment. Report findings only; do not edit code unless the user
asks you to fix specific findings.
## Scope
Review the current diff or named files against:
- The repo's `AGENTS.md` / `CLAUDE.md`
- `docs/ENGINEERING_STYLE.md`
- The touched code paths and tests
Apply only the guidance for the active repo. In `basic-memory`, prioritize local-first
file/database/MCP boundaries. In `basic-memory-cloud`, prioritize tenant/workspace isolation,
cloud worker behavior, and web-v2 state/runtime boundaries.
## Review Rubric
Report only concrete, falsifiable risks:
- **Cognitive load:** Is the change harder to understand than the problem requires?
- **Change propagation:** Will one product change force edits across unrelated layers?
- **Knowledge duplication:** Is the same rule encoded in multiple places that can drift?
- **Accidental complexity:** Did the change add abstractions, fallbacks, or state without need?
- **Dependency direction:** Are API/MCP/CLI, services, repositories, and UI stores respecting
their intended boundaries?
- **Domain model distortion:** Do names and types still match the product concept, or did a
transport/storage detail leak into the domain?
- **Test oracle quality:** Would the tests fail for the bug or regression the change claims to
protect against?
## House Rules To Check Explicitly
- No speculative `getattr(obj, "attr", default)` for unknown model shapes.
- No broad exception swallowing, warning-only failure paths, or hidden fallback behavior.
- No casts or `Any` that hide an unclear type relationship.
- Dataclasses for internal value/result objects; Pydantic at validation/serialization
boundaries.
- Narrow `Protocol`s when only a capability is needed.
- Explicit async/resource ownership, cancellation, and cleanup.
- Meaningful regression tests or verification for risky changes.
- Comments explain why, not what.
## Reporting Format
Lead with findings ordered by severity. Each finding should include:
| Severity | Use for |
| -------- | ------- |
| `high` | A likely correctness, security, data-loss, or tenant/workspace isolation failure |
| `medium` | A concrete maintainability or boundary risk that can cause future defects |
| `low` | A minor consistency issue, ambiguous guidance, or review-only cleanup |
```text
severity | file:line | risk category | claim
Why: concrete behavior or code path that proves the risk.
Fix: smallest practical change, or "none obvious" if the risk needs product input.
```
If there are no findings, say so and note any verification gaps that remain.
+48
View File
@@ -0,0 +1,48 @@
---
name: fix-pr-issues
description: Use when addressing Basic Memory pull request feedback, failed checks, or BM Bossbot blockers from Codex.
---
# Fix Basic Memory PR Issues
Resolve PR feedback and failed checks, then wait for BM Bossbot to approve the
new head SHA. This skill never merges a PR.
## Gather
1. Identify the PR:
- `gh pr view --json number,url,headRefOid,mergeStateStatus,statusCheckRollup`
2. Collect feedback:
- PR comments and review summaries
- inline review comments and unresolved review threads
- failed GitHub Actions jobs and relevant logs
- the managed `BM_BOSSBOT_SUMMARY` block in the PR body
3. Build a short issue ledger:
- source
- concrete problem
- expected fix
- verification needed
## Fix
1. Address one ledger item at a time.
2. Read each file in full before editing it.
3. Keep diffs narrow and preserve unrelated user changes.
4. Run the smallest meaningful verification first, then widen as needed.
5. Commit with `git commit -s` when code or docs changed.
## Push And Recheck
1. Push the branch.
2. Watch checks for the new `headRefOid`.
3. Wait for the required `BM Bossbot Approval` status to pass on that exact SHA.
4. If BM Bossbot reviews an older SHA, treat the approval as stale and keep
waiting for the current one.
## Reply
For each addressed comment or blocker, reply with the fix commit, verification
run, and current BM Bossbot status. Do not resolve or dismiss substantive
feedback without evidence.
@@ -0,0 +1,7 @@
interface:
display_name: "Fix PR Issues"
short_description: "Address PR feedback and BM Bossbot blockers"
icon_small: "./assets/icon.svg"
icon_large: "./assets/icon.svg"
brand_color: "#2563EB"
default_prompt: "Use $fix-pr-issues to address PR feedback and wait for BM Bossbot Approval on the latest head SHA."
@@ -0,0 +1,5 @@
<svg xmlns="http://www.w3.org/2000/svg" width="128" height="128" viewBox="0 0 24 24" fill="none" stroke="#111827" stroke-width="1.8" stroke-linecap="round" stroke-linejoin="round">
<path d="M3 12h4l2-6 4 12 2-6h6"/>
<path d="M4 20h16"/>
</svg>

After

Width:  |  Height:  |  Size: 249 B

+247
View File
@@ -0,0 +1,247 @@
---
name: infographics
description: Use when generating Basic Memory PR, changelog, release, or weekly images from Codex.
---
# Basic Memory Images
Generate repository visuals with evidence-grounded content and canonical output
paths. The file and marker names still say "infographic" for compatibility, but
PR generation is image-first: scene, poster, painting, photograph, cover,
tableau, staged artifact, or another editorial visual moment that describes the
intent of the PR. PR images are non-gating BM Bossbot artifacts; changelog and
release-summary images are manual evidence-pack workflows.
## Output Contract
- Base output directory: `docs/assets/infographics/`
- PR image: `docs/assets/infographics/pr-<number>.webp`
- Changelog image: `docs/assets/infographics/changelog.webp`
- Weekly image:
- This is always a 2-Week Retro window: previous ISO week through current ISO
week (`start-week = current-week - 1`, `end-week = current-week`).
- Same year window: `docs/assets/infographics/<year>-w<start-week>-w<end-week>.webp`
- Cross-year window:
`docs/assets/infographics/<start-year>-w<start-week>-<end-year>-w<end-week>.webp`
## PR Mode
PR mode uses the BM Bossbot summary block as source material. Do not hand-write
claims that are not present in the PR body.
1. Fetch the PR body:
```bash
gh pr view <number> --json body --jq '.body // ""' > /tmp/bm-pr-body.md
```
2. Generate the canonical asset:
```bash
uv run --script scripts/generate_pr_infographic.py \
--pr-number <number> \
--pr-body-file /tmp/bm-pr-body.md \
--theme "<optional visual theme>" \
--provenance-output /tmp/bm-infographic-provenance.md \
--output docs/assets/infographics/pr-<number>.webp
```
If the PR body contains a managed image theme block, the script reads it
automatically:
```markdown
<!-- BM_INFOGRAPHIC_THEME:start -->
<theme>
<!-- BM_INFOGRAPHIC_THEME:end -->
```
Before spending an image call, test the prompt path locally:
```bash
uv run --script scripts/generate_pr_infographic.py \
--pr-number <number> \
--pr-body-file /tmp/bm-pr-body.md \
--theme "<optional visual theme>" \
--output docs/assets/infographics/pr-<number>.webp \
--print-prompt
```
`--dry-run` is an alias for `--print-prompt`; both print the final prompt and
exit without calling OpenAI.
When no theme is supplied, the script selects a deterministic BM visual
direction from the style pool below based on the PR number and Bossbot summary.
This keeps repeated PR images from collapsing into the same generic visual.
When the image is generated, also write provenance with
`--provenance-output <path>`. BM Bossbot publishes that managed block into the
PR body with these markers:
```markdown
<!-- BM_INFOGRAPHIC_PROVENANCE:start -->
...
<!-- BM_INFOGRAPHIC_PROVENANCE:end -->
```
The provenance block records the generated asset path, image model, size,
quality, image mode, theme source, and selected visual direction. It
intentionally does not dump the full generated prompt into the PR body. Treat
this block as debugging and creative provenance only; it is not a merge gate.
The PR image is visual support only. The authoritative merge gate is the
GitHub commit status named `BM Bossbot Approval`.
## Changelog Mode
Build an evidence pack before writing a prompt:
- diff truth source: merged PR diffs, merge commits, or local reconstructed diffs
- changed-file orientation: `git diff --stat` plus key file reads
- impact ledger: before/after outcomes tied to actual changes
- discard list: misleading titles, reverted work, rename-only churn, speculative TODOs
- chosen image form: poster, scene, tableau, cover, painting, photograph,
staged artifact, or another editorial visual moment
- chosen BM style category: exactly one category from the selection pool below
Read these references before drafting the prompt:
- `references/prompt-blueprint.md`
- `references/style-balance.md`
Read the current `CHANGELOG.md` entries and include the latest meaningful
changes.
## Style And Category Selection
Select exactly one BM style category per image based on semantic fit. The
visual language should be recognizable and tasteful, while staying
business-readable.
Create an image-first visual form that communicates the change: poster, scene,
tableau, cover image, painting, photograph, staged artifact, or another
editorial visual moment. Maps, diagrams, dossiers, charts, and labels can appear
as props inside the scene, but do not make a text-heavy infographic.
BM category pool:
- computer science college textbooks: SICP-style diagrams, algorithms lectures,
compiler pipelines, automata, database systems, type theory, operating systems
- classic literature subjects: sea voyages, gothic manors, Dickensian city maps,
Austen social graphs, library marginalia, travel journals
- fantasy/D&D-inspired: quest maps, dungeon keys, guild ledgers, spellbooks,
bestiaries, tavern notice boards; no copyrighted settings
- Music: Metal, Hard Rock, Punk, techno, soul, reggae bands; no pop music, no
direct band logos, album covers, or musician likenesses
- sci-fi: Star Wars inspired knockoff, Spaceballs-adjacent space opera, fleet
routes, mission consoles, contraband manifests; avoid copyrighted characters,
logos, or named fictional universes
- Conan the barbarian-inspired sword-and-sorcery: ruined temples, desert routes,
battle standards, ancient maps; no named character likenesses
- Comic books: issue covers, splash pages, action-panel maps, caption boxes,
halftone energy, clean sound-effect typography
- French new wave movies: poster style, stark typography, city route maps,
jump-cut sequencing, high-contrast editorial photography cues
- WWII propaganda posters: home-front public-information poster language,
logistics arrows, ration charts, mobilization maps, bold simplified figures;
no real-world party symbols, hate imagery, dehumanizing slogans, or false
historical claims
- Italian movie posters: hand-painted drama, bold credits, expressive color,
route-map collage, 1960s or 1970s cinema energy; no direct film titles or
actor likenesses
- Shakespeare: stage maps, acts and scenes, dramatis personae, royal courts,
backstage cue sheets
- Greek mythology: temple diagrams, constellation routes, hero's journey maps,
oracle tablets, labyrinths, ship routes
- noir detective boards: case files, red-string maps, typed evidence labels,
precinct wall charts
- NASA mission-control dashboards: launch timelines, telemetry maps, orbital
routes, status boards
- space exploration and astronomy: celestial atlases, observatory charts,
star-field maps, orbital mechanics diagrams, planetary survey routes,
telescope annotations, mission trajectories, deep-space timelines
- paintings: abstract painting, classical landscape, Remington-inspired western
action painting, Rembrandt-inspired chiaroscuro, historical mural, stormy
seascape, allegorical editorial painting
- classic black-and-white photography: documentary field report, newsroom
archive print, editorial photo essay, street photography, high-contrast
darkroom print, contact sheet, civic infrastructure photograph
- 80's action movies: practical explosions, smoky backlit warehouses, neon city
streets, helicopter searchlights, mission dossiers, heroic silhouettes,
high-stakes countdowns, painted ensemble posters; no direct actor likenesses,
real film titles, franchise marks, or catchphrases
- alchemy manuscripts: transformation diagrams, annotated symbols, recipe-like
process maps, illuminated margins
- brutalist civic planning: transit maps, concrete signage, zoning blocks,
infrastructure diagrams
Selection rules:
- Pick one category only; do not create mixed mashups.
- Pick the most appropriate image form. Prefer an actual scene, poster,
painting, photograph, tableau, or cover over a text-heavy infographic.
- Match metaphor to content, but do not overthink it. The category is a creative
catalyst, not a semantic constraint.
- Use a polished editorial rendering direction: smooth anti-aliased
text, high contrast, clean edges, readable labels.
- Make the category drive the composition through a readable staged moment,
editorial composition, symbolic environment, route, artifact, or visual
metaphor.
- Keep the structure literal enough to aid understanding, but not so heavy that
it obscures engineering meaning.
- Give the image generator creative latitude on layout, structure, color palette,
and visual metaphors. Be precise about what content to show, loose about how
to show it.
- Do not use copyrighted characters, logos, or named fictional universes. Use
genre cues, knockoffs, and original compositions instead.
## Content-First Aesthetic Contract
The meaning must be readable and clearly hierarchical. Everything else is
creative territory: image form, layout, visual metaphors, decorative elements,
color choices, and category-specific visual language.
Hierarchy:
1. Meaning: what shipped, what changed, and why it matters must be clear.
2. If the image uses text, labels, sections, or evidence bullets, they must be
legible.
3. The selected category's visual DNA should drive the composition as a poster,
scene, painting, photograph, tableau, cover, or symbolic object arrangement.
4. Do not play it safe. A visually striking image that someone wants to look at
beats a correct but boring one.
Hard rules:
- Content sections and labels must be readable when present. Text cannot be
obscured by decorations.
- Do not use lore-heavy copy that competes with engineering or business meaning.
- Every prompt must include a clear image-first composition cue: a staged scene,
poster composition, painting, photograph, symbolic tableau, hero object,
mission room, dossier, artifact, route, or visual metaphor.
- Do not over-prescribe exact coordinates or panel geometry; give a composition
backbone and let the model compose around it.
## Generation
1. Write the final prompt to a temporary markdown file.
2. Generate with the shared image helper:
```bash
uv run --script scripts/generate_infographic.py \
--prompt-file /tmp/bm-infographic-prompt.md \
--output docs/assets/infographics/<name>.webp
```
3. Verify the image exists and is readable before reporting success.
## Quality Bar
- Tell a concrete before/after value story, not vague improvement claims.
- Stay understandable for both engineers and non-technical stakeholders.
- Use plain-language section titles and labels when text is present.
- Include clear visual hierarchy: title, staged focal point, symbolic scene,
evidence props, or hero object.
- Avoid invented facts; only use provided source material.
- Favor shipped outcomes over intermediate or reverted work.
- Preserve readability with high contrast, non-tiny labels, and uncluttered
layout.
@@ -0,0 +1,7 @@
interface:
display_name: "Infographics"
short_description: "Generate Basic Memory repo infographics"
icon_small: "./assets/icon.svg"
icon_large: "./assets/icon.svg"
brand_color: "#2563EB"
default_prompt: "Use $infographics to generate a Basic Memory PR or changelog infographic with canonical output paths."
@@ -0,0 +1,5 @@
<svg xmlns="http://www.w3.org/2000/svg" width="128" height="128" viewBox="0 0 24 24" fill="none" stroke="#111827" stroke-width="1.8" stroke-linecap="round" stroke-linejoin="round">
<path d="M3 12h4l2-6 4 12 2-6h6"/>
<path d="M4 20h16"/>
</svg>

After

Width:  |  Height:  |  Size: 249 B

@@ -0,0 +1,72 @@
# Prompt Blueprint
Convert an evidence pack into a final visual prompt. Be precise about the
content and loose about visual execution.
## Required Inputs
- Diff truth source summary
- Changed-file orientation summary
- Impact ledger with before/after outcomes
- Discard list for excluded noise
- Chosen image form
- Chosen BM style category
## Prompt Shape
```text
Create a polished Basic Memory editorial image inspired by
<BM_STYLE_CATEGORY>. Use a poster, scene, tableau, painting, photograph, cover
image, staged artifact, or another image-first form that best communicates the
intent. Use HD editorial rendering with smooth anti-aliased text when text is
present. Go bold and let the selected category drive the visual language through
original, non-infringing cues.
TITLE:
- "<clear title>"
- "<scope subtitle>"
COMPOSITION:
- Recreate a clear staged moment or symbolic image that describes the PR
intent.
- Maps, diagrams, dossiers, route lines, labels, and artifacts can appear as
props inside the scene, but the output should read as an image rather than a
dense infographic.
- Take creative liberty with layout and styling.
- The hard rule: the meaning must be readable and clearly hierarchical.
- Keep labels plain-language and technical when labels are used.
CONTENT:
1. "<section>"
- <evidence-grounded outcome>
- <evidence-grounded outcome>
2. "<section>"
- <evidence-grounded outcome>
- <evidence-grounded outcome>
METRICS:
- <metric>
- <metric>
STYLE DIRECTION:
- Upscaled editorial, high contrast, anti-aliased text, smooth edges.
- Let the category's visual DNA drive the composition.
- Use genre/category cues only; do not use copyrighted characters, logos, named
fictional universes, direct band logos, album art, or celebrity likenesses.
DO NOT:
- Make text unreadable or let decoration obscure content.
- Render a text-heavy infographic, dashboard, flowchart, timeline strip,
checklist, bullet-list panel, or dense explanatory diagram.
- Use crunchy low-resolution pixel art.
- Invent facts not present in the evidence pack.
```
## Writing Rules
- Keep each bullet specific and evidence-grounded.
- Prefer outcome language over implementation trivia.
- Default to three or four sections; never exceed five.
- Give proportionally more space to dominant changes.
- Keep the final prompt short, energetic, and readable.
@@ -0,0 +1,42 @@
# Style Balance Rubric
## Core Principle
Be bold, not confusing. The selected BM style category should structure the
visual through a readable image-first composition, not decorate a generic grid.
Use an editorial scene, poster, painting, photograph, cover, staged artifact, or
tableau that turns the PR intent into a visual moment.
## Required Traits
- Anti-aliased typography
- Smooth edges
- High contrast between text and background
- Plain-language section labels
- Clear composition backbone: staged scene, editorial poster, painting,
photograph, symbolic tableau, hero artifact, dossier, mission room, or route
embodied as part of the scene
- A single coherent BM style category, expressed through original visual cues
## Reject Or Rewrite If
- Content text is unreadable.
- The prompt lacks a composition backbone.
- The prompt over-prescribes exact panel positions or a rigid grid.
- The style leans into crunchy low-resolution pixelation.
- Copy uses lore-heavy references instead of engineering meaning.
- The prompt uses copyrighted characters, logos, named fictional universes,
direct band logos, album art, or celebrity likenesses.
## Creative Integration Patterns
- Use category-native map details to organize content: textbook diagrams,
literary journeys, quest maps, tour posters, mission-control routes, stage
blocking, mythic constellations, star charts, mission trajectories, case
boards, or civic plans as props inside the image.
- Recreate a scene, editorial poster, painting, photograph, cover, artifact, or
tableau instead of sectioned bullets.
- Map engineering metrics to visual counters, route progress, or status boards
only when they naturally belong in the scene.
- Let headers and accents borrow from the selected style.
- Keep atmospheric details behind or around content, never over it.
+114
View File
@@ -0,0 +1,114 @@
---
name: pr-create
description: Use when creating or updating a Basic Memory pull request from Codex with BM Bossbot merge-gate monitoring.
---
# Create A Basic Memory PR
Create or update a pull request for the current branch, then wait for BM
Bossbot to approve the latest head SHA. This skill never merges a PR.
## Inputs
- Optional `<theme>`: free-form visual direction for the non-gating PR
image. Example: `$pr-create "Italian movie poster"`.
- Treat `<theme>` as style guidance only. It must not affect PR readiness,
BM Bossbot review, status checks, or merge behavior.
## How To Use
Ask Codex to use the skill from a feature branch:
```text
$pr-create
$pr-create "Italian movie poster"
$pr-create "80's action movies"
```
Use the plain form when you only want the PR workflow. Pass a theme when you
want the non-gating image to lean toward a particular visual direction. The
theme can be specific ("Rembrandt-inspired approval scene") or broad ("let the
model choose from BM categories").
## What Happens
1. Codex checks the branch, local verification, GitHub auth, commit sign-offs,
and semantic PR title shape.
2. Codex pushes the branch, creates or reuses the PR, and adds the optional
`BM_INFOGRAPHIC_THEME` block when a theme was supplied.
3. BM Bossbot runs from trusted base code, reviews sanitized PR metadata and
diff context, and sets the required `BM Bossbot Approval` status for the
exact head SHA.
4. If approval succeeds, BM Bossbot may publish a non-gating image block and a
provenance block:
```markdown
<!-- BM_INFOGRAPHIC_PROVENANCE:start -->
...
<!-- BM_INFOGRAPHIC_PROVENANCE:end -->
```
The provenance records the image mode, theme source, selected visual
direction, and image settings. It is for review/debugging context only.
5. Codex reports the PR URL, head SHA, checks watched, verification run, and BM
Bossbot verdict.
The skill never merges, never enables auto-merge, and never treats the image or
provenance block as a gate. The only required merge signal is the
`BM Bossbot Approval` status on the current PR head SHA.
## Preflight
1. Confirm the repo and branch:
- `git status --short --branch`
- stop if detached or on `main`
- keep unrelated user changes intact
2. Confirm GitHub access:
- `gh auth status`
- `gh repo view --json nameWithOwner,defaultBranchRef,url`
3. Check PR readiness:
- commits are signed off with `git commit -s`
- title uses the repo semantic format
- local verification appropriate to the change has run
## Create Or Reuse
1. Push the branch:
- `git push -u origin HEAD`
2. Check for an existing PR:
- `gh pr view --json number,url,headRefOid,mergeStateStatus,statusCheckRollup`
3. If no PR exists, create one:
- `gh pr create --fill`
- adjust the title if it does not satisfy the semantic PR title workflow
4. If `<theme>` is provided, add or update this managed block in the PR body:
```markdown
<!-- BM_INFOGRAPHIC_THEME:start -->
<theme>
<!-- BM_INFOGRAPHIC_THEME:end -->
```
Keep the rest of the PR body intact. The theme is non-gating image guidance
only.
5. Do not merge. Do not enable auto-merge.
## Watch The Gate
1. Trigger or wait for `.github/workflows/bm-bossbot.yml`.
2. Watch the required commit status named `BM Bossbot Approval`.
3. Treat approval as valid only when it is green for the current `headRefOid`.
4. If the branch changes after approval, wait for BM Bossbot to review the new
head SHA.
5. If BM Bossbot fails or requests changes, use `$fix-pr-issues`.
## Report
Return the PR URL, current head SHA, checks watched, verification run, and the
BM Bossbot verdict. Include the image `<theme>` if one was supplied. Be
explicit when any check is still pending.
@@ -0,0 +1,7 @@
interface:
display_name: "PR Create"
short_description: "Create PRs and wait for BM Bossbot"
icon_small: "./assets/icon.svg"
icon_large: "./assets/icon.svg"
brand_color: "#2563EB"
default_prompt: "Use $pr-create to create or update this Basic Memory PR and wait for BM Bossbot Approval."
+5
View File
@@ -0,0 +1,5 @@
<svg xmlns="http://www.w3.org/2000/svg" width="128" height="128" viewBox="0 0 24 24" fill="none" stroke="#111827" stroke-width="1.8" stroke-linecap="round" stroke-linejoin="round">
<path d="M3 12h4l2-6 4 12 2-6h6"/>
<path d="M4 20h16"/>
</svg>

After

Width:  |  Height:  |  Size: 249 B

+2 -2
View File
@@ -6,14 +6,14 @@
},
"metadata": {
"description": "Official Basic Memory plugins from the canonical basic-memory repository",
"version": "0.21.6"
"version": "0.22.1"
},
"plugins": [
{
"name": "basic-memory",
"source": "./plugins/claude-code",
"description": "The bridge between Claude's working memory and Basic Memory's durable knowledge graph \u2014 session briefings, pre-compaction checkpoints, and capture reflexes",
"version": "0.21.6",
"version": "0.22.1",
"author": {
"name": "Basic Machines"
},
+4 -3
View File
@@ -30,7 +30,8 @@ The justfile target handles:
- ✅ Beta version format validation (supports b1, b2, rc1, etc.)
- ✅ Git status and branch checks
- ✅ Quality checks (`just check` - lint, format, type-check, tests)
- ✅ Version update in `src/basic_memory/__init__.py`
- ✅ Version update across all consolidated manifests via `just set-version` (Python
package + Claude Code plugin/marketplaces + Codex plugin + Hermes + OpenClaw)
- ✅ Automatic commit with proper message
- ✅ Tag creation and pushing to GitHub
- ✅ Beta release workflow trigger
@@ -90,6 +91,6 @@ Monitor release: https://github.com/basicmachines-co/basic-memory/actions
- Beta releases are pre-releases for testing new features
- Automatically published to PyPI with pre-release flag
- Uses the automated justfile target for consistency
- Version is automatically updated in `__init__.py`
- Version is automatically updated across all consolidated manifests via `just set-version`
- Ideal for validating changes before stable release
- Supports both beta (b1, b2) and release candidate (rc1, rc2) versions
- Supports both beta (b1, b2) and release candidate (rc1, rc2) versions
+56 -40
View File
@@ -28,7 +28,10 @@ You are an expert release manager for the Basic Memory project. When the user ru
#### Documentation Validation
1. **Changelog Check**
- CHANGELOG.md contains entry for target version
- CHANGELOG.md contains entry for target version **already landed on `main`**
(main only accepts changes via PR, so the changelog entry must go through
its own PR before running the release; the recipe pre-flight-checks for a
`## vX.Y.Z` heading)
- Entry includes all major features and fixes
- Breaking changes are documented
@@ -41,10 +44,15 @@ just release <version>
The justfile target handles:
- ✅ Version format validation
- ✅ Git status and branch checks
-Quality checks (`just check` - lint, format, type-check, tests)
-Version update across all consolidated manifests via `just set-version` (Python package + Claude Code plugin/marketplaces + Hermes + OpenClaw)
-Automatic commit with proper message
- ✅ Tag creation and pushing to GitHub
-Changelog entry check (must already be on `main`)
-Quality checks (`just lint` + `just typecheck`)
-Version update across all consolidated manifests via `just set-version` (Python
package + Claude Code plugin/marketplaces + Codex plugin + Hermes + OpenClaw)
- ✅ Release PR: commits the bump on a `release/vX.Y.Z` branch, opens a PR
(`chore(core): release vX.Y.Z`), and rebase-merges it — the `main` ruleset
rejects direct pushes and the repo disallows merge commits
- ✅ Tags the rebased bump commit on `main` (found by commit subject, since
the rebase rewrites the SHA) and pushes the tag
- ✅ Release workflow trigger (automatic on tag push)
The GitHub Actions workflow (`.github/workflows/release.yml`) then:
@@ -88,7 +96,7 @@ After PyPI release is published, update the MCP registry:
2. **Publish to MCP Registry**
```bash
cd /Users/drew/code/basic-memory
# from the basic-memory repo root
mcp-publisher publish
```
@@ -108,43 +116,50 @@ After PyPI release is published, update the MCP registry:
#### Website Updates
**1. basicmachines.co** (`/Users/drew/code/basicmachines.co`)
- **Goal**: Update version number displayed on the homepage
- **Location**: Search for "Basic Memory v0." in the codebase to find version displays
- **What to update**:
- Hero section heading that shows "Basic Memory v{VERSION}"
- "What's New in v{VERSION}" section heading
- Feature highlights array (look for array of features with title/description)
- **Process**:
**1. basicmemory.com** (sibling `basicmemory.com` repo —
`basicmachines-co/basicmemory.com`, formerly `basicmachines.co`)
- **No version bump needed.** The marketing site is an Astro + React app and
carries **no hardcoded Basic Memory version number** anywhere in its UI
(`hero.tsx` and the rest of the site have no version string). The old
instruction to bump `src/components/sections/hero.tsx` is obsolete — that
file no longer holds a version. Release announcements are dated blog posts,
not an in-place edit.
- **Skip entirely for patch releases.**
- **Significant releases only — optional announcement post**:
1. Pull latest from GitHub: `git pull origin main`
2. Create release branch: `git checkout -b release/v{VERSION}`
3. Search codebase for current version number (e.g., "v0.16.1")
4. Update version numbers to new release version
5. Update feature highlights with 3-5 key features from this release (extract from CHANGELOG.md)
6. Commit changes: `git commit -m "chore: update to v{VERSION}"`
7. Push branch: `git push origin release/v{VERSION}`
- **Deploy**: Follow deployment process for basicmachines.co
3. Add a dated post under `src/content/blog/` modeled on an existing
release post (e.g. `basic-memory-v0-19-0-release.md`), summarizing 35
headline features from `CHANGELOG.md`
4. Commit (`git commit -s -m "..."`), push, and open a PR against
`basicmachines-co/basicmemory.com`
- **Deploy**: follow that repo's deployment process.
**2. docs.basicmemory.com** (`/Users/drew/code/docs.basicmemory.com`)
- **Goal**: Add new release notes section to the latest-releases page
- **File**: `src/pages/latest-releases.mdx`
**2. docs.basicmemory.com** (sibling `docs.basicmemory.com` repo)
- **Goal**: Add a What's New page for the release and bump the homepage badge
- **Site shape**: Nuxt/Docus content site. The changelog page
(`content/2.whats-new/*.changelog.md`) auto-fetches GitHub releases — no
manual changelog update needed. See that repo's CLAUDE.md "Version Bump
Checklist".
- **What to do**:
1. Pull latest from GitHub: `git pull origin main`
2. Create release branch: `git checkout -b release/v{VERSION}`
3. Read the existing file to understand the format and structure
4. Read `/Users/drew/code/basic-memory/CHANGELOG.md` to get release content
5. Add new release section **at the top** (after MDX imports, before other releases)
6. Follow the existing pattern:
- Heading: `## [v{VERSION}](github-link) — YYYY-MM-DD`
- Focus statement if applicable
- `<Info>` block with highlights (3-5 key items)
- Sections for Features, Bug Fixes, Breaking Changes, etc.
- Link to full changelog at the end
- Separator `---` between releases
7. Commit changes: `git commit -m "docs: add v{VERSION} release notes"`
8. Push branch: `git push origin release/v{VERSION}`
- **Source content**: Extract and format sections from CHANGELOG.md for this version
- **Deploy**: Follow deployment process for docs.basicmemory.com
3. Read `CHANGELOG.md` in the `basic-memory` repo to get release content
4. **New minor/major release**: add `content/2.whats-new/1.v{VERSION}.md`
modeled on the previous version page (frontmatter title/description,
headline feature first, then sections, then an Upgrading note) and
renumber the existing what's-new pages down one slot (URLs don't
change — Nuxt strips the numeric prefixes)
5. **Patch release**: append a short note to the current version's page
instead of creating a new one
6. Update the homepage version badge in `content/index.md` (the
`v0.XX →` button text and its `to: /whats-new/v{VERSION}` link)
7. If the release adds user-facing features, update the relevant guide
and reference pages (`content/3.cloud/`, `content/9.reference/`)
8. Commit: `git commit -s -m "docs: add v{VERSION} release notes"`
9. Push branch and open a PR; merge after the release is tagged
- **Deploy**: push to main auto-deploys to development; production requires
manual workflow dispatch via GitHub Actions
**4. Announce Release**
- Post to Discord community if significant changes
@@ -194,12 +209,13 @@ Users can now upgrade:
- Version is automatically updated across **all** consolidated manifests via
`just set-version <version>` (which calls `scripts/update_versions.py`): the
Python package (`__init__.py`, `server.json`) **and** the plugin/agent artifacts
(Claude Code `plugin.json` + root/local marketplaces, Hermes `plugin.yaml` +
`__init__.py`, OpenClaw `package.json`). To bump only the plugin/agent artifacts
(Claude Code `plugin.json` + root/local marketplaces, Codex `plugin.json`,
Hermes `plugin.yaml` + `__init__.py`, OpenClaw `package.json`). To bump only
the plugin/agent artifacts
out of band, use `just set-packages-version <version>` (preview with
`just set-packages-version-dry-run <version>`).
- Triggers automated GitHub release with changelog
- Package is published to PyPI for `pip` and `uv` users
- Homebrew formula is automatically updated for stable releases
- MCP Registry is updated manually via `mcp-publisher publish`
- Supports multiple installation methods (uv, pip, Homebrew)
- Supports multiple installation methods (uv, pip, Homebrew)
+1
View File
@@ -0,0 +1 @@
../../.agents/skills/basic-machines-review
+15 -16
View File
@@ -1,26 +1,18 @@
name: Claude Code Review
on:
pull_request:
types: [opened, synchronize]
# Optional: Only run on specific file changes
# paths:
# - "src/**/*.ts"
# - "src/**/*.tsx"
# - "src/**/*.js"
# - "src/**/*.jsx"
"on":
workflow_dispatch:
inputs:
pr_number:
description: Pull request number to review manually
required: true
env:
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true"
jobs:
claude-review:
# Only run for organization members and collaborators
if: |
github.event.pull_request.author_association == 'OWNER' ||
github.event.pull_request.author_association == 'MEMBER' ||
github.event.pull_request.author_association == 'COLLABORATOR'
if: inputs.pr_number != ''
runs-on: ubuntu-latest
permissions:
contents: read
@@ -43,7 +35,14 @@ jobs:
track_progress: true # Enable visual progress tracking
allowed_bots: '*'
prompt: |
Review this Basic Memory PR against our team checklist:
Review Basic Memory PR #${{ inputs.pr_number }} as an advisory manual review.
Use `gh pr view ${{ inputs.pr_number }}` and related `gh pr`/`gh api`
commands to inspect the pull request. Do not merge the PR and do not
treat this advisory review as the required merge gate. BM Bossbot owns
the required `BM Bossbot Approval` status.
Review the PR against our team checklist:
## Code Quality & Standards
- [ ] Follows Basic Memory's coding conventions in CLAUDE.md
+6 -8
View File
@@ -13,9 +13,10 @@ env:
jobs:
docker:
runs-on: ubuntu-latest
runs-on: depot-ubuntu-24.04
permissions:
contents: read
id-token: write
packages: write
steps:
@@ -24,10 +25,8 @@ jobs:
with:
fetch-depth: 0
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v4
with:
platforms: linux/amd64,linux/arm64
- name: Set up Depot
uses: depot/setup-action@v1
- name: Log in to GitHub Container Registry
uses: docker/login-action@v4
@@ -49,13 +48,12 @@ jobs:
type=raw,value=latest,enable={{is_default_branch}}
- name: Build and push Docker image
uses: docker/build-push-action@v7
uses: depot/build-push-action@v1
with:
project: ${{ vars.DEPOT_BASIC_MEMORY_PROJECT_ID || vars.DEPOT_PROJECT_ID }}
context: .
file: ./Dockerfile
platforms: linux/amd64,linux/arm64
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
cache-from: type=gha
cache-to: type=gha,mode=max
+123 -10
View File
@@ -13,9 +13,43 @@ on:
env:
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true"
# Branch builds (PRs arrive as push events — this workflow has no
# pull_request trigger) select only impacted tests from the cached testmon
# baseline (branch cache falling back to main's full-run recording). Pushes
# to main run the full suite with --testmon-noselect to refresh the baseline.
BASIC_MEMORY_TESTMON_FLAGS: ${{ github.ref_name == 'main' && '--testmon-noselect' || '--testmon --testmon-forceselect' }}
jobs:
changes:
# Docs/workflow-only changes skip the entire test matrix while the workflow
# still concludes successfully, so the BM Bossbot gate (workflow_run on
# Tests success) keeps firing and the PR stays mergeable.
name: Detect code changes
runs-on: ubuntu-latest
outputs:
code: ${{ steps.filter.outputs.code }}
steps:
- uses: actions/checkout@v6
- id: filter
uses: dorny/paths-filter@v3
with:
# Tests only runs on push events; for branch pushes compare against
# main (merge-base), for main pushes dorny diffs the push range.
base: main
filters: |
code:
- 'src/**'
- 'tests/**'
- 'test-int/**'
- 'alembic/**'
- 'pyproject.toml'
- 'uv.lock'
- 'justfile'
- '.github/workflows/test.yml'
static-checks:
needs: changes
if: needs.changes.outputs.code == 'true'
name: Static Checks (Python 3.12)
timeout-minutes: 20
runs-on: ubuntu-latest
@@ -54,6 +88,8 @@ jobs:
just lint
test-sqlite-unit:
needs: changes
if: needs.changes.outputs.code == 'true'
name: Test SQLite Unit (${{ matrix.os }}, Python ${{ matrix.python-version }})
timeout-minutes: 45
strategy:
@@ -64,6 +100,8 @@ jobs:
python-version: "3.12"
- os: ubuntu-latest
python-version: "3.13"
# Python 3.14 unit tests are the longest full-suite slice; keep this
# one on GitHub-hosted runners after Depot terminated it mid-suite.
- os: ubuntu-latest
python-version: "3.14"
- os: windows-latest
@@ -87,6 +125,19 @@ jobs:
- uses: extractions/setup-just@v4
- name: Cache pytest-testmon results
uses: actions/cache@v4
with:
path: |
.testmondata
.testmondata-shm
.testmondata-wal
key: ${{ runner.os }}-testmon-sqlite-unit-py${{ matrix.python-version }}-${{ github.ref_name }}-${{ github.run_id }}
restore-keys: |
${{ runner.os }}-testmon-sqlite-unit-py${{ matrix.python-version }}-${{ github.ref_name }}-
${{ runner.os }}-testmon-sqlite-unit-py${{ matrix.python-version }}-main-
${{ runner.os }}-testmon-sqlite-unit-py${{ matrix.python-version }}-
- name: Create virtual env
run: |
uv venv
@@ -100,6 +151,8 @@ jobs:
just test-unit-sqlite
test-sqlite-integration:
needs: changes
if: needs.changes.outputs.code == 'true'
name: Test SQLite Integration (${{ matrix.os }}, Python ${{ matrix.python-version }})
timeout-minutes: 45
strategy:
@@ -133,6 +186,19 @@ jobs:
- uses: extractions/setup-just@v4
- name: Cache pytest-testmon results
uses: actions/cache@v4
with:
path: |
.testmondata
.testmondata-shm
.testmondata-wal
key: ${{ runner.os }}-testmon-sqlite-integration-py${{ matrix.python-version }}-${{ github.ref_name }}-${{ github.run_id }}
restore-keys: |
${{ runner.os }}-testmon-sqlite-integration-py${{ matrix.python-version }}-${{ github.ref_name }}-
${{ runner.os }}-testmon-sqlite-integration-py${{ matrix.python-version }}-main-
${{ runner.os }}-testmon-sqlite-integration-py${{ matrix.python-version }}-
- name: Create virtual env
run: |
uv venv
@@ -146,15 +212,19 @@ jobs:
just test-int-sqlite
test-postgres-unit:
name: Test Postgres Unit (Python ${{ matrix.python-version }})
needs: changes
if: needs.changes.outputs.code == 'true'
name: Test Postgres Unit (Python ${{ matrix.python-version }}, shard ${{ matrix.group }}/3)
timeout-minutes: 60
strategy:
fail-fast: false
matrix:
include:
- python-version: "3.12"
- python-version: "3.13"
- python-version: "3.14"
# Shard the largest suite across parallel jobs: each shard is a full job
# with its own Postgres service running 1/3 of the collection.
# Postgres runs on the latest Python only — the SQLite matrix carries
# Python-version coverage; Postgres carries backend coverage.
group: [1, 2, 3]
python-version: ["3.14"]
runs-on: ubuntu-latest
services:
postgres:
@@ -190,6 +260,20 @@ jobs:
- uses: extractions/setup-just@v4
- name: Cache pytest-testmon results
uses: actions/cache@v4
with:
path: |
.testmondata
.testmondata-shm
.testmondata-wal
key: ${{ runner.os }}-testmon-postgres-unit-py${{ matrix.python-version }}-g${{ matrix.group }}-${{ github.ref_name }}-${{ github.run_id }}
restore-keys: |
${{ runner.os }}-testmon-postgres-unit-py${{ matrix.python-version }}-g${{ matrix.group }}-${{ github.ref_name }}-
${{ runner.os }}-testmon-postgres-unit-py${{ matrix.python-version }}-g${{ matrix.group }}-main-
${{ runner.os }}-testmon-postgres-unit-py${{ matrix.python-version }}-main-
${{ runner.os }}-testmon-postgres-unit-py${{ matrix.python-version }}-
- name: Create virtual env
run: |
uv venv
@@ -200,18 +284,19 @@ jobs:
- name: Run tests
run: |
just test-unit-postgres
BASIC_MEMORY_PYTEST_SPLIT_FLAGS="--splits 3 --group ${{ matrix.group }}" just test-unit-postgres
test-postgres-integration:
needs: changes
if: needs.changes.outputs.code == 'true'
name: Test Postgres Integration (Python ${{ matrix.python-version }})
timeout-minutes: 45
strategy:
fail-fast: false
matrix:
include:
- python-version: "3.12"
- python-version: "3.13"
- python-version: "3.14"
# Latest Python only: SQLite carries version coverage, Postgres carries
# backend coverage.
python-version: ["3.14"]
runs-on: ubuntu-latest
services:
postgres:
@@ -247,6 +332,19 @@ jobs:
- uses: extractions/setup-just@v4
- name: Cache pytest-testmon results
uses: actions/cache@v4
with:
path: |
.testmondata
.testmondata-shm
.testmondata-wal
key: ${{ runner.os }}-testmon-postgres-integration-py${{ matrix.python-version }}-${{ github.ref_name }}-${{ github.run_id }}
restore-keys: |
${{ runner.os }}-testmon-postgres-integration-py${{ matrix.python-version }}-${{ github.ref_name }}-
${{ runner.os }}-testmon-postgres-integration-py${{ matrix.python-version }}-main-
${{ runner.os }}-testmon-postgres-integration-py${{ matrix.python-version }}-
- name: Create virtual env
run: |
uv venv
@@ -260,6 +358,8 @@ jobs:
just test-int-postgres
test-semantic:
needs: changes
if: needs.changes.outputs.code == 'true'
name: Test Semantic (Python 3.12)
timeout-minutes: 45
runs-on: ubuntu-latest
@@ -281,6 +381,19 @@ jobs:
- uses: extractions/setup-just@v4
- name: Cache pytest-testmon results
uses: actions/cache@v4
with:
path: |
.testmondata
.testmondata-shm
.testmondata-wal
key: ${{ runner.os }}-testmon-semantic-py3.12-${{ github.ref_name }}-${{ github.run_id }}
restore-keys: |
${{ runner.os }}-testmon-semantic-py3.12-${{ github.ref_name }}-
${{ runner.os }}-testmon-semantic-py3.12-main-
${{ runner.os }}-testmon-semantic-py3.12-
- name: Create virtual env
run: |
uv venv
+1 -1
View File
@@ -49,7 +49,7 @@ ENV/
/docs/.obsidian/
/examples/.obsidian/
/examples/.basic-memory/
/docs/assets
# claude action
claude-output
+42 -7
View File
@@ -108,13 +108,34 @@ Before opening or updating a PR, run the checks that mirror the common required
- Follow the repository pattern for data access
- Tools communicate to api routers via the httpx ASGI client (in process)
### Programming Style
See [docs/ENGINEERING_STYLE.md](docs/ENGINEERING_STYLE.md) for the fuller house style. The
short version for agents:
- Prefer type-safe, explicit designs over object-heavy indirection. Use Python 3.12 `type`
aliases, full annotations, and narrow `Protocol`s when a caller only needs a capability.
- Use dataclasses for internal value objects and operation results; use Pydantic v2 at API,
CLI, MCP, and persistence boundaries where validation and serialization matter.
- Keep async boundaries obvious. Resource-owning code should use context managers, propagate
cancellation, and avoid hidden background work unless the lifecycle is explicit.
- Fail fast. Do not add silent fallback logic, broad exception swallowing, speculative
`getattr`, or casts that hide an unclear model shape.
- Keep control flow simple and local. Push branching decisions up, keep leaf helpers focused,
and name values after the domain concept they carry.
- Use evidence-first testing. Add or update meaningful regression tests for bugs and risky
behavior, prefer real code paths over mocks, and run the narrowest command that proves the
change before widening verification.
- Comments should explain why a branch, invariant, or constraint exists. Avoid comments that
merely narrate obvious code.
### Code Change Guidelines
- **Full file read before edits**: Before editing any file, read it in full first to ensure complete context; partial reads lead to corrupted edits
- **Minimize diffs**: Prefer the smallest change that satisfies the request. Avoid unrelated refactors or style rewrites unless necessary for correctness
- **No speculative getattr**: Never use `getattr(obj, "attr", default)` when unsure about attribute names. Check the class definition or source code first
- **Fail fast**: Write code with fail-fast logic by default. Do not swallow exceptions with errors or warnings
- **No fallback logic**: Do not add fallback logic unless explicitly told to and agreed with the user
- **House style is canonical**: Follow the Programming Style section above for type-safe,
fail-fast code; do not hide unclear models with speculative attributes, broad exception
handling, casts, or unapproved fallback logic
- **No guessing**: Do not say "The issue is..." before you actually know what the issue is. Investigate first.
### Literate Programming Style
@@ -279,7 +300,9 @@ See SPEC-16 for full context manager refactor details.
### Release Process
Releases are driven by `just release` / `just beta` — never by a bare `git tag`. The recipes bump version metadata, run pre-flight checks, commit, tag, and push. GitHub Actions then publishes to PyPI and updates the Homebrew formula.
Releases are driven by `just release` / `just beta` — never by a bare `git tag`. The recipes bump version metadata, run pre-flight checks, land the bump on `main` through a release PR, tag, and push the tag. GitHub Actions then publishes to PyPI and updates the Homebrew formula.
**Main requires PRs.** The `main` ruleset rejects direct pushes ("Changes must be made through a pull request") and the repo disallows merge commits, so the recipes push a `release/vX.Y.Z` branch, open a PR titled `chore(core): release vX.Y.Z`, rebase-merge it with `gh pr merge --rebase`, then tag the rebased bump commit on `main` (located by its commit subject, since rebasing rewrites the SHA) and push the tag. The CHANGELOG entry for the version must already be on `main` — land it via a normal PR before running the recipe (it pre-flight-checks for a `## vX.Y.Z` heading).
**Stable release:**
@@ -287,7 +310,7 @@ Releases are driven by `just release` / `just beta` — never by a bare `git tag
just release v0.21.3
```
The recipe runs `just lint` + `just typecheck`, then updates every release manifest through `scripts/update_versions.py`: `src/basic_memory/__init__.py`, `server.json`, the root Claude marketplace, the Claude Code plugin manifest and local marketplace, the Hermes `plugin.yaml`, and the OpenClaw `package.json`. It commits as `chore: update version to X.Y.Z for vX.Y.Z release`, creates the `vX.Y.Z` tag, and pushes both the commit and the tag to `origin/main`. After the tag lands, the `Release` workflow builds the Python package, publishes to PyPI, creates the GitHub release with auto-generated notes, publishes the OpenClaw npm package, and updates the Homebrew formula. The recipe finishes by printing the post-release tasks the workflow doesn't cover.
The recipe runs `just lint` + `just typecheck`, then updates every release manifest through `scripts/update_versions.py`: `src/basic_memory/__init__.py`, `server.json`, the root Claude marketplace, the Claude Code plugin manifest and local marketplace, the Hermes `plugin.yaml`, and the OpenClaw `package.json`. It commits as `chore: update version to X.Y.Z for vX.Y.Z release` on a `release/vX.Y.Z` branch, lands it on `main` via a rebase-merged PR, then tags the rebased commit and pushes the tag. After the tag lands, the `Release` workflow builds the Python package, publishes to PyPI, creates the GitHub release with auto-generated notes, publishes the OpenClaw npm package, and updates the Homebrew formula. The recipe finishes by printing the post-release tasks the workflow doesn't cover.
**Beta release:** `just beta v0.21.3b1` — same flow with a beta-suffixed tag. PyPI consumers install with `pip install basic-memory --pre`.
@@ -298,8 +321,13 @@ The recipe runs `just lint` + `just typecheck`, then updates every release manif
**Do not tag releases by hand.** A bare `git tag vX.Y.Z` skips the in-code version bump. Package metadata is still correct (uv-dynamic-versioning derives it from the git tag) but `basic-memory --version` reports the previous release, which is what happened with v0.21.2 → v0.21.3.
**Post-release tasks** the recipe surfaces but doesn't run:
- `docs.basicmemory.com` — add notes to `src/pages/latest-releases.mdx`
- `basicmachines.co`bump version in `src/components/sections/hero.tsx`
- `docs.basicmemory.com` — add a What's New page under `content/2.whats-new/` and bump the version badge in `content/index.md` (the changelog page auto-fetches GitHub releases; see that repo's CLAUDE.md version-bump checklist)
- `basicmemory.com`the marketing site (Astro + React, repo
`basicmachines-co/basicmemory.com`, formerly `basicmachines.co`) carries **no
hardcoded version number** in its UI, so there is nothing to bump. For a
significant release, optionally add a dated announcement post under
`src/content/blog/` (model it on an existing `basic-memory-vX-Y-Z-release.md`).
Skip entirely for routine patch releases.
- MCP Registry — `mcp-publisher publish` from the repo root
See `.claude/commands/release/release.md` (and `beta.md`, `release-check.md`, `changelog.md` alongside it) for the full release + post-release runbook, including the slash commands.
@@ -348,6 +376,13 @@ See `.claude/commands/release/release.md` (and `beta.md`, `release-check.md`, `c
- Manage snapshots: `basic-memory cloud snapshot [create|list|delete|show|browse]`
- Restore from snapshot: `basic-memory cloud restore <path> --snapshot <id>`
**Cloud Sync Commands (Personal and Team workspaces):**
- Fetch cloud changes (cloud -> local): `basic-memory cloud pull --name "name"` (Team-safe; additive, never deletes local)
- Upload local changes (local -> cloud): `basic-memory cloud push --name "name"` (Team-safe; additive, never deletes cloud)
- Resolve conflicts on push/pull: `--on-conflict [fail|keep-local|keep-cloud|keep-both]` (default `fail` lists conflicts and aborts, git-style)
- One-way mirror (local -> cloud): `basic-memory cloud sync --name "name"` (Personal workspaces only; deletes cloud files missing locally)
- Two-way mirror (local <-> cloud): `basic-memory cloud bisync --name "name"` (Personal workspaces only)
### MCP Capabilities
- Basic Memory exposes these MCP tools to LLMs:
+109
View File
@@ -1,5 +1,114 @@
# CHANGELOG
## v0.22.1 (2026-06-12)
Follow-up patch to v0.22.0. Fixes project and default-project resolution on
fresh installs, MCP workspace routing, sync project selection, and CLI startup
latency, plus a few MCP parity additions.
### Features
- Added a `workspace` parameter to `write_note` for parity with `edit_note`.
- **#826**: Added `title` and `tags` annotations to all MCP tool decorators
(phase 1).
- **#930**: `search_notes` now comma-splits `note_types`, `entity_types`, and
`categories`.
- **#971**: Added the manual-pages flow — manpage seed schema, flow docs, and
verification fixes.
### Bug Fixes
- Fresh installs no longer fail when the projects table is empty: resolve now
points them at project setup, the first project is promoted to default when
the config default is missing from the database, the promoted default state
is returned from the project-create API, and a default can be set when none
is currently set. An existing database default is preserved when repairing a
missing config default.
- **#949**: Sync skips projects without an absolute local path and excludes
orphan DB projects that are absent from config.
- **#952 / #981**: Resolved workspace display names and tenant ids in qualified
project routes, closing out the manual verification findings.
- `note_types`/`entity_types`/`categories` are normalized on the direct-call
path, with non-string list elements rejected.
- Vector-search hydration keys on `(type, id)` to prevent id collisions.
- `file_utils` requires line-anchored frontmatter fences.
- CLI startup is faster: FastAPI and app imports are deferred out of CLI
startup, and rich/typer modules are preloaded before an in-place upgrade.
- `config.json` is written atomically.
- In-memory SQLite sessions are serialized so concurrent rollbacks cannot
destroy writes.
### Maintenance
- Release recipes route through PRs and wait for the release PR merge to land
before tagging; the release runbook is refreshed and stripped of
user-specific absolute paths.
## v0.22.0 (2026-06-11)
Team-safe cloud sync. New additive `bm cloud push` and `bm cloud pull`
commands work safely on shared Team workspaces, while the destructive mirror
commands are gated to Personal workspaces. Also: a large batch of MCP tool
fixes, search improvements, and embedding reliability work.
### Features
- **#917**: Added Team-safe `bm cloud push` / `bm cloud pull`. Both are
additive (they never delete on the destination) and abort on conflicts by
default, git-style, with `--on-conflict {fail|keep-local|keep-cloud|keep-both}`.
The destructive `bm cloud sync` / `bm cloud bisync` mirrors are now gated
to Personal workspaces.
- **#920**: Team push/pull uses per-workspace rclone remotes, so remotes and
credentials stay scoped to each workspace.
- **#908**: Search supports an observation category filter.
- **#809**: Added an experimental LiteLLM embedding provider for semantic
search (marked experimental, see **#899**).
- **#907**: `bm tool write-note` accepts `--type`.
- **#906**: `bm status` accepts `--wait` and `--timeout`.
- Added the `bm tool delete-note` command.
- **#905**: Improved workspace and cloud bisync command discoverability.
### Bug Fixes
- **#931 / #946**: Truncated observation permalinks are disambiguated to
prevent search-index collisions, and `build_context` resolves observations
by the same permalink the search index uses (**#909**, **#929**).
- **#934**: `edit_note` recovers when the file exists on disk but is not yet
indexed (**#581**).
- **#911 / #932 / #941**: Comma-separated tags are split consistently in
`parse_tags` and the `search_notes` tags parameter, with input normalized
for direct callers (**#910**).
- **#933**: `read_note` accepts `page`/`page_size` for parity with sibling
tools (**#883**).
- **#914 / #904 / #916**: `move_note` resolves `memory://` URLs, stops
falsely rejecting same-project moves as cross-project, no longer reports
false success across project boundaries, and mismatch guidance points at
the landing path.
- **#915**: Navigation pagination is validated, and `recent_activity` shows
the correct project.
- **#913**: `bm tool` commands align with MCP behavior (error exit codes,
overwrite handling, category support, defaults).
- **#923**: `bm cloud setup` no longer overwrites an existing rclone remote
(**#922**).
- **#912**: The `note_types` search filter is case-insensitive.
- `build_context` allows cross-project context traversal, `write_note`
resolves overwrite conflicts, and sync uses strict deferred relation
resolution.
- Embedding reliability: FastEmbed vectors are L2-normalized (**#843**),
corrupt FastEmbed model caches self-heal (**#900**), a single embedding
provider is reused per process (**#903**), `sqlite-vec` loads for the
embedding-status query (**#901**), and engine disposal no longer crashes
on the Postgres backend (**#902**).
### Maintenance
- Leaner, faster CI: testmon-selected branch builds, sharded Postgres jobs,
and faster default test fixtures (**#928**, **#938**, **#945**).
- Documented personal-vs-team cloud sync semantics (**#947**, closes
**#851**).
- Fixed npx skill install docs (**#927**).
- Added `glama.json` to claim the Glama MCP directory listing (**#953**).
## v0.21.6 (2026-06-04)
Monorepo consolidation plus a redesigned Claude Code plugin. The satellite
+6 -5
View File
@@ -6,6 +6,7 @@
[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)
![](https://badge.mcpx.dev?type=server 'MCP Server')
![](https://badge.mcpx.dev?type=dev 'MCP Dev')
[![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/basicmachines-co/basic-memory)
## Skip the install — try Basic Memory in the cloud
@@ -216,14 +217,14 @@ Source: [`plugins/claude-code`](plugins/claude-code).
### Shared skills
Framework-agnostic `SKILL.md` files live in [`skills/`](skills). If your
Skills CLI supports subpath installs:
Skills CLI supports repository subdirectory sources:
```bash
npx skills add basicmachines-co/basic-memory --path skills
npx skills add basicmachines-co/basic-memory/skills
```
If it does not, copy the `memory-*` directories from `skills/` into your
agent's skills directory as a temporary Phase 1 install path.
If your installed Skills CLI cannot load that source, update the CLI or copy
the `memory-*` directories from `skills/` into your agent's skills directory.
### Hermes
@@ -590,7 +591,7 @@ retention).
| `BASIC_MEMORY_IMPORT_UPLOAD_MAX_BYTES` | `104857600` | Max uploaded import size |
```bash
BASIC_MEMORY_LOG_LEVEL=DEBUG basic-memory sync
BASIC_MEMORY_LOG_LEVEL=DEBUG basic-memory reindex
tail -f ~/.basic-memory/basic-memory.log
```
+3 -3
View File
@@ -111,7 +111,7 @@ You can run Basic Memory CLI commands inside the container using `docker exec`:
docker exec basic-memory-server basic-memory status
# Sync files
docker exec basic-memory-server basic-memory sync
docker exec basic-memory-server basic-memory reindex
# Show help
docker exec basic-memory-server basic-memory --help
@@ -137,7 +137,7 @@ When using Docker volumes, you'll need to configure projects to point to your mo
3. **Sync the new project:**
```bash
docker exec basic-memory-server basic-memory sync
docker exec basic-memory-server basic-memory reindex
```
### Example: Setting up an Obsidian Vault
@@ -157,7 +157,7 @@ docker exec basic-memory-server basic-memory project create obsidian /app/data
docker exec basic-memory-server basic-memory project set-default obsidian
# Sync to index all files
docker exec basic-memory-server basic-memory sync
docker exec basic-memory-server basic-memory reindex
```
### Environment Variables
+64
View File
@@ -0,0 +1,64 @@
# Basic Memory Engineering Style
Style is how we make code easier to verify. Prefer explicit, typed, local-first code that
preserves the file system as the source of truth while keeping the database, API, and MCP
surfaces in sync.
## Design Center
- Basic Memory is local-first. Markdown files are the durable source; SQLite/Postgres indexes
are derived state that should be rebuilt or reconciled from files when needed.
- Keep the existing boundary order: CLI/MCP/API entrypoints compose dependencies, services own
business behavior, repositories own database access, and file services own filesystem writes.
- MCP tools should remain atomic and composable. They should call API routers through typed MCP
clients, not reach around into services.
- Prefer small, explicit abstractions that match a real domain boundary. Avoid object
hierarchies when a function, dataclass, type alias, or protocol describes the concept better.
## Types And Data
- Use full type annotations and Python 3.12 syntax. Introduce `type` aliases for repeated
structured shapes, callback signatures, or domain concepts that would otherwise become
anonymous `dict[str, Any]` values.
- Use dataclasses for internal values, operation inputs, and service results. Prefer
`frozen=True` when the value should not change and `slots=True` when identity/dynamic
attributes are not needed.
- Use Pydantic v2 at boundaries that validate, serialize, or deserialize data: API payloads,
CLI/MCP schemas, configuration, and persistence-adjacent schemas.
- Use narrow `Protocol`s when a caller needs a capability rather than a concrete repository or
service. Keep protocols small enough that fake implementations in tests are obvious.
- Avoid speculative `getattr`, broad casts, or `Any` as a way to paper over uncertainty. Read
the model or schema definition and make the type relationship explicit.
## Control Flow And Resources
- Fail fast when an invariant is broken. Do not swallow exceptions, add warning-only error
handling, or introduce fallback behavior unless the user explicitly agrees to that behavior.
- Keep control flow simple and close to the domain decision. Push `if` statements up into the
function that owns orchestration; keep leaf helpers focused on computation or one side effect.
- Make async/resource boundaries visible with context managers and explicit lifecycles. Do not
start background work without a clear owner, cancellation story, and verification path.
- Keep file mutations centralized through the existing file utilities/services so checksum,
atomic write, and index synchronization behavior stays coherent.
## Testing And Verification
- Use evidence-first testing, not mechanical TDD. For bugs and risky behavior, add or update a
regression test that would catch the failure. For small documentation-only edits, use the
relevant doc/repo hygiene checks.
- Prefer tests that exercise real code paths. Use mocks, doubles, or `monkeypatch` only when
the external boundary would be slow, nondeterministic, or impossible to trigger directly.
- Keep coverage at 100% for new code. Use `# pragma: no cover` only for code that would require
disproportionate mocking and is covered through an integration or runtime path.
- Start with targeted commands, then widen as risk grows: focused pytest, `just fast-check`,
`just doctor`, package checks for agent packaging changes, and full SQLite/Postgres gates
when behavior crosses shared boundaries.
## Comments And Names
- Name values after the domain concept they carry: project, entity, permalink, tenant, route,
checksum, observation, relation, batch, or index state.
- Comments should say why a branch, invariant, retry, lifecycle, or compatibility constraint
exists. Section headers are useful when a function or file has clear phases.
- Avoid comments that restate the code. If a comment cannot explain a decision, simplify the
code or improve the name instead.
+2 -2
View File
@@ -184,8 +184,8 @@ finance/ (lowercase f)
Use Basic Memory's built-in conflict detection:
```bash
# Sync will report conflicts
basic-memory sync
# Index local file changes (conflicts are handled during the scan)
basic-memory reindex
# Check sync status for warnings
basic-memory status
+175 -55
View File
@@ -8,9 +8,25 @@ The cloud CLI enables you to:
- **Authenticate cloud access** - OAuth/API key credentials are stored locally for cloud operations
- **Project-scoped sync** - Each project independently manages its sync configuration
- **Explicit operations** - Sync only what you want, when you want
- **Bidirectional sync** - Keep local and cloud in sync with rclone bisync
- **Team-safe push/pull** - Additive, git-style transfers that work on shared Team workspaces
- **Bidirectional sync** - Keep local and cloud in sync with rclone bisync (Personal workspaces)
- **Offline access** - Work locally, sync when ready
### Personal vs Team workspaces
The transfer commands fall into two groups:
| Command | Direction | Behavior | Personal | Team |
|---|---|---|---|---|
| `bm cloud pull` | cloud → local | **additive** — never deletes local | ✅ | ✅ |
| `bm cloud push` | local → cloud | **additive** — never deletes cloud | ✅ | ✅ |
| `bm cloud sync` | local → cloud | **mirror** — deletes cloud files missing locally | ✅ | ❌ |
| `bm cloud bisync` | local ↔ cloud | **mirror** — two-way, deletes on both sides | ✅ | ❌ |
`sync` and `bisync` are mirror operations: one local tree becomes authoritative and files missing on the other side get deleted. That is correct for a Personal workspace (one user, one source of truth) but unsafe on a shared Team bucket, where it could delete a teammate's files. On Team workspaces these commands exit early with a clear error and point you at `push`/`pull`.
`push` and `pull` are additive (they use `rclone copy`, which never deletes on the destination), so they are safe on both Personal and Team workspaces.
## Prerequisites
Before using Basic Memory Cloud, you need:
@@ -55,8 +71,8 @@ bm project add work --cloud --local-path ~/work-notes
bm project add temp --cloud # No local sync
# Now you can sync individually (after initial --resync):
bm project bisync --name research
bm project bisync --name work
bm cloud bisync --name research
bm cloud bisync --name work
# temp stays cloud-only
```
@@ -137,10 +153,10 @@ Establish the initial sync baseline. **Best practice:** Always preview with `--d
```bash
# Step 1: Preview the initial sync (recommended)
bm project bisync --name research --resync --dry-run
bm cloud bisync --name research --resync --dry-run
# Step 2: If all looks good, run the actual sync
bm project bisync --name research --resync
bm cloud bisync --name research --resync
```
**What happens under the covers:**
@@ -167,7 +183,7 @@ This will effectively make both Path1 and Path2 filesystems contain a matching s
After the first sync, just run bisync without `--resync`:
```bash
bm project bisync --name research
bm cloud bisync --name research
```
**What happens:**
@@ -235,7 +251,7 @@ bm project add research --cloud --local-path ~/Documents/research
- Stores sync config in `~/.basic-memory/config.json`
- Prepares for bisync (but doesn't sync yet)
**Result:** Project ready to sync. Run `bm project bisync --name research --resync` to establish baseline.
**Result:** Project ready to sync. Run `bm cloud bisync --name research --resync` to establish baseline.
**Use case 3: Add sync to existing cloud project**
@@ -294,18 +310,98 @@ For MCP stdio, routing is always local.
### Understanding the Sync Commands
**There are three sync-related commands:**
**There are five sync-related commands:**
1. `bm project sync` - One-way: local → cloud (make cloud match local)
2. `bm project bisync` - Two-way: local ↔ cloud (recommended)
3. `bm project check` - Verify files match (no changes)
| Command | Direction | Workspace | Summary |
|---|---|---|---|
| `bm cloud pull` | cloud → local | Personal + Team | Fetch cloud changes, additively (git-style) |
| `bm cloud push` | local → cloud | Personal + Team | Upload local changes, additively (git-style) |
| `bm cloud sync` | local → cloud | Personal only | One-way mirror (cloud becomes identical to local) |
| `bm cloud bisync` | local ↔ cloud | Personal only | Two-way mirror (recommended for solo use) |
| `bm cloud check` | — | Personal only | Verify mirror integrity (no changes) |
### One-Way Sync: Local → Cloud
If you collaborate on a shared Team workspace, use **`push`/`pull`** (see [Team Workspaces](#team-workspaces-push--pull-additive-git-style)). If you are the only writer (a Personal workspace), the mirror commands `sync`/`bisync` give you a single source of truth.
### Team Workspaces: push / pull (additive, git-style)
`push` and `pull` are the Team-safe transfer commands. They model `git push` / `git pull`:
- **`bm cloud pull`** fetches changes from the cloud into your local directory.
- **`bm cloud push`** uploads your local changes to the cloud.
Both use `rclone copy`, so they are **additive — they never delete on the destination**. A conflict (a file that differs on both sides) is never resolved silently: by default the command aborts and lists the conflicting files, exactly like git refusing to clobber your changes.
#### Pull: fetch cloud changes
```bash
# Preview first (recommended)
bm cloud pull --name research --dry-run
# Fetch new/changed cloud files into local
bm cloud pull --name research
```
**What happens:**
1. Compares cloud and local with `rclone check`
2. Downloads files that are new or changed on the cloud
3. Leaves your local-only files untouched (never deletes local)
4. If any file differs on both sides, aborts and lists the conflicts (unless you pass `--on-conflict`)
#### Push: upload local changes
```bash
bm cloud push --name research --dry-run
bm cloud push --name research
```
**What happens:**
1. Compares local and cloud with `rclone check`
2. Uploads files that are new or changed locally
3. Leaves cloud-only files untouched (never deletes cloud)
4. If any file differs on both sides, aborts and lists the conflicts — pull first, like a rejected `git push`
#### Resolving conflicts
When `push`/`pull` reports conflicts, re-run with `--on-conflict` to choose how differing files are handled. The value names exactly what survives, so it reads the same in both directions:
| `--on-conflict` | Behavior |
|---|---|
| `fail` *(default)* | List the conflicting files and exit without transferring anything |
| `keep-cloud` | Take the cloud version (pull: overwrite local; push: skip those files) |
| `keep-local` | Keep the local version (pull: skip those files; push: overwrite cloud) |
| `keep-both` | Keep both — write the incoming version beside the existing one as `name.conflict-<date>.md` |
```bash
# A teammate edited notes you also changed locally — pull reports a conflict:
bm cloud pull --name research
# pull aborted: 1 file(s) differ between local and cloud.
# * notes/decisions.md
# Re-run with one of: --on-conflict keep-cloud | keep-local | keep-both
# Take the cloud copy:
bm cloud pull --name research --on-conflict keep-cloud
# Or keep both versions to merge by hand:
bm cloud pull --name research --on-conflict keep-both
```
#### Limitations
`push`/`pull` are deliberately simple, conflict-aware byte transfers — not a full reconciler. Without a sync baseline:
- **Deletions are not propagated.** A note deleted on one side is not removed from the other (we cannot tell an intentional delete from a file the other side never had). This is surfaced in the command output.
- **Every divergence is treated as a conflict.** We cannot tell a teammate's edit from your stale copy, so any differing file prompts a decision rather than auto-resolving.
For conflict-aware *editing*, write through the MCP/API tools (which merge at the note level). A Team-safe bidirectional reconciler with a real baseline is tracked in [issue #862](https://github.com/basicmachines-co/basic-memory/issues/862).
### One-Way Sync: Local → Cloud (Personal only)
**Use case:** You made changes locally and want to push to cloud (overwrite cloud).
> **Personal workspaces only.** `sync` is a destructive mirror — it deletes cloud files that are not present locally. On a Team workspace it would delete a teammate's files, so it is blocked there. Use `bm cloud push` (additive) on Team workspaces.
```bash
bm project sync --name research
bm cloud sync --name research
```
**What happens:**
@@ -321,16 +417,18 @@ bm project sync --name research
- You want to force cloud to match local
- You don't care about cloud changes
### Two-Way Sync: Local ↔ Cloud (Recommended)
### Two-Way Sync: Local ↔ Cloud (Personal only, recommended for solo use)
**Use case:** You edit files both locally and in cloud UI, want both to stay in sync.
> **Personal workspaces only.** `bisync` is a two-way mirror that can delete and overwrite on both sides. It is blocked on Team workspaces — use `bm cloud pull` then `bm cloud push` there. A Team-safe bidirectional reconciler is tracked separately ([issue #862](https://github.com/basicmachines-co/basic-memory/issues/862)).
```bash
# First time - establish baseline
bm project bisync --name research --resync
bm cloud bisync --name research --resync
# Subsequent syncs
bm project bisync --name research
bm cloud bisync --name research
```
**What happens:**
@@ -349,7 +447,7 @@ echo "Local change" > ~/Documents/research/notes.md
# Cloud now has: "Cloud change"
# Run bisync
bm project bisync --name research
bm cloud bisync --name research
# Result: Newer file wins (based on modification time)
# If cloud was more recent, cloud version kept
@@ -361,12 +459,14 @@ bm project bisync --name research
- You edit in multiple places
- You want automatic conflict resolution
### Verify Sync Integrity
### Verify Sync Integrity (Personal only)
**Use case:** Check if local and cloud match without making changes.
> **Personal workspaces only.** `check` compares against the Personal workspace mirror remote, like `sync`/`bisync`. On Team workspaces use `bm cloud pull --dry-run` / `bm cloud push --dry-run` to preview differences instead.
```bash
bm project check --name research
bm cloud check --name research
```
**What happens:**
@@ -378,7 +478,7 @@ bm project check --name research
```bash
# One-way check (faster)
bm project check --name research --one-way
bm cloud check --name research --one-way
```
### Preview Changes (Dry Run)
@@ -386,7 +486,7 @@ bm project check --name research --one-way
**Use case:** See what would change without actually syncing.
```bash
bm project bisync --name research --dry-run
bm cloud bisync --name research --dry-run
```
**What happens:**
@@ -432,20 +532,20 @@ bm project add work --cloud --local-path ~/work-notes
bm project add personal --cloud --local-path ~/personal
# Establish baselines
bm project bisync --name research --resync
bm project bisync --name work --resync
bm project bisync --name personal --resync
bm cloud bisync --name research --resync
bm cloud bisync --name work --resync
bm cloud bisync --name personal --resync
# Daily workflow: sync everything
bm project bisync --name research
bm project bisync --name work
bm project bisync --name personal
bm cloud bisync --name research
bm cloud bisync --name work
bm cloud bisync --name personal
```
**Future:** `--all` flag will sync all configured projects:
```bash
bm project bisync --all # Coming soon
bm cloud bisync --all # Coming soon
```
### Mixed Usage
@@ -462,8 +562,8 @@ bm project add archive --cloud
bm project add temp-notes --cloud
# Sync only the configured ones
bm project bisync --name research
bm project bisync --name work
bm cloud bisync --name research
bm cloud bisync --name work
# Archive and temp-notes stay cloud-only
```
@@ -661,7 +761,7 @@ code ~/.basic-memory/.bmignore
echo "*.tmp" >> ~/.basic-memory/.bmignore
# Next sync uses updated patterns
bm project bisync --name research
bm cloud bisync --name research
```
## Troubleshooting
@@ -724,7 +824,7 @@ bm cloud login
**Solution:**
```bash
bm project bisync --name research --resync
bm cloud bisync --name research --resync
```
**What this does:**
@@ -747,7 +847,7 @@ bm project bisync --name research --resync
echo "# Research Notes" > ~/Documents/research/README.md
# Now run bisync
bm project bisync --name research --resync
bm cloud bisync --name research --resync
```
**Why this happens:** Bisync creates listing files that track the state of each side. When both directories are completely empty, these listing files are considered invalid by rclone.
@@ -764,10 +864,10 @@ bm project bisync --name research --resync
```bash
# Clear bisync state
bm project bisync-reset research
bm cloud bisync-reset research
# Re-establish baseline
bm project bisync --name research --resync
bm cloud bisync --name research --resync
```
**What this does:**
@@ -787,16 +887,16 @@ bm project bisync --name research --resync
```bash
# Check what would be deleted
bm project bisync --name research --dry-run
bm cloud bisync --name research --dry-run
# If correct, establish new baseline
bm project bisync --name research --resync
bm cloud bisync --name research --resync
```
**Solution 2:** Use one-way sync if you know local is correct:
```bash
bm project sync --name research
bm cloud sync --name research
```
### Project Not Configured for Sync
@@ -809,7 +909,7 @@ bm project sync --name research
```bash
bm cloud sync-setup research ~/Documents/research
bm project bisync --name research --resync
bm cloud bisync --name research --resync
```
### Connection Issues
@@ -880,20 +980,30 @@ bm project set-local <name> # Revert project to local mode
### File Synchronization
```bash
# One-way sync (local → cloud)
bm project sync --name <project>
bm project sync --name <project> --dry-run
bm project sync --name <project> --verbose
# Pull: fetch cloud changes (cloud → local) - Personal + Team, additive
bm cloud pull --name <project>
bm cloud pull --name <project> --dry-run
bm cloud pull --name <project> --on-conflict [fail|keep-local|keep-cloud|keep-both]
# Two-way sync (local cloud) - Recommended
bm project bisync --name <project> # After first --resync
bm project bisync --name <project> --resync # First time / force baseline
bm project bisync --name <project> --dry-run
bm project bisync --name <project> --verbose
# Push: upload local changes (local cloud) - Personal + Team, additive
bm cloud push --name <project>
bm cloud push --name <project> --dry-run
bm cloud push --name <project> --on-conflict [fail|keep-local|keep-cloud|keep-both]
# Integrity check
bm project check --name <project>
bm project check --name <project> --one-way
# One-way mirror (local → cloud) - Personal workspaces only
bm cloud sync --name <project>
bm cloud sync --name <project> --dry-run
bm cloud sync --name <project> --verbose
# Two-way mirror (local ↔ cloud) - Personal workspaces only
bm cloud bisync --name <project> # After first --resync
bm cloud bisync --name <project> --resync # First time / force baseline
bm cloud bisync --name <project> --dry-run
bm cloud bisync --name <project> --verbose
# Integrity check - Personal workspaces only
bm cloud check --name <project>
bm cloud check --name <project> --one-way
# List project files by route
bm project ls --name <project> # Default target: local
@@ -909,15 +1019,25 @@ bm project ls --name <project> --cloud --path <subpath>
1. **Authenticate cloud access** - `bm cloud login`
2. **Install rclone** - `bm cloud setup`
3. **Add projects with sync** - `bm project add research --cloud --local-path ~/Documents/research`
4. **Preview first sync** - `bm project bisync --name research --resync --dry-run`
5. **Establish baseline** - `bm project bisync --name research --resync`
6. **Daily workflow** - `bm project bisync --name research`
**Personal workspace (solo, mirror) workflow:**
4. **Preview first sync** - `bm cloud bisync --name research --resync --dry-run`
5. **Establish baseline** - `bm cloud bisync --name research --resync`
6. **Daily workflow** - `bm cloud bisync --name research`
**Team workspace (shared, additive) workflow:**
4. **Fetch teammates' changes** - `bm cloud pull --name research`
5. **Upload your changes** - `bm cloud push --name research`
6. **Resolve conflicts explicitly** - re-run with `--on-conflict keep-cloud|keep-local|keep-both`
**Key benefits:**
- ✅ Each project independently syncs (or doesn't)
- ✅ Projects can live anywhere on disk
- ✅ Explicit sync operations (no magic)
- ✅ Safe by design (max delete limits, conflict resolution)
- ✅ Team-safe push/pull that never delete on the destination
- ✅ Safe by design (max delete limits, conflict resolution, git-style conflict aborts)
- ✅ Full offline access (work locally, sync when ready)
**Future enhancements:**
+300
View File
@@ -0,0 +1,300 @@
# LiteLLM Provider
Basic Memory can use the LiteLLM SDK for semantic search embeddings. This lets you
keep Basic Memory's vector indexing and search behavior while routing embedding calls
to OpenAI-compatible and provider-specific backends such as OpenAI, Azure OpenAI,
Cohere, Bedrock, NVIDIA NIM, and other LiteLLM-supported embedding providers.
Use this page when you want to try a non-default embedding model, validate a provider,
or tune LiteLLM-specific settings.
> **Experimental — advanced users only.** The LiteLLM provider is experimental and
> intended for users who are comfortable operating remote embedding backends. It makes
> paid, networked API calls, requires per-model dimension and input-role configuration,
> and reindexing a real corpus can be slow and spend provider quota (see
> [Reindexing with a remote provider](#reindexing-with-a-remote-provider)). For most
> users, the default local **FastEmbed** provider is the recommended choice. Use LiteLLM
> only if you know what you're doing.
## Quick Start
The default LiteLLM model is OpenAI `text-embedding-3-small` through the LiteLLM
model string `openai/text-embedding-3-small`.
```bash
export BASIC_MEMORY_SEMANTIC_SEARCH_ENABLED=true
export BASIC_MEMORY_SEMANTIC_EMBEDDING_PROVIDER=litellm
export OPENAI_API_KEY=sk-...
bm reindex --embeddings
```
Then use vector or hybrid search:
```python
search_notes("login token flow", search_type="hybrid")
```
## Basic Memory Options
All options can be set in config or as environment variables.
| Config Field | Env Var | Default | Notes |
|---|---|---|---|
| `semantic_search_enabled` | `BASIC_MEMORY_SEMANTIC_SEARCH_ENABLED` | Auto | Set to `true` to force vector/hybrid support on. |
| `semantic_embedding_provider` | `BASIC_MEMORY_SEMANTIC_EMBEDDING_PROVIDER` | `fastembed` | Set to `litellm` for the LiteLLM provider. |
| `semantic_embedding_model` | `BASIC_MEMORY_SEMANTIC_EMBEDDING_MODEL` | `bge-small-en-v1.5` | With `litellm`, the default is remapped to `openai/text-embedding-3-small`. |
| `semantic_embedding_dimensions` | `BASIC_MEMORY_SEMANTIC_EMBEDDING_DIMENSIONS` | Provider default | Required for non-default LiteLLM models because vector tables are dimensioned before the first API call. |
| `semantic_embedding_forward_dimensions` | `BASIC_MEMORY_SEMANTIC_EMBEDDING_FORWARD_DIMENSIONS` | Auto | Sends `dimensions` to LiteLLM only when supported. Auto is enabled for `text-embedding-3` model strings. |
| `semantic_embedding_document_input_type` | `BASIC_MEMORY_SEMANTIC_EMBEDDING_DOCUMENT_INPUT_TYPE` | Auto | LiteLLM `input_type` for indexed notes/passages. |
| `semantic_embedding_query_input_type` | `BASIC_MEMORY_SEMANTIC_EMBEDDING_QUERY_INPUT_TYPE` | Auto | LiteLLM `input_type` for search queries. |
| `semantic_embedding_batch_size` | `BASIC_MEMORY_SEMANTIC_EMBEDDING_BATCH_SIZE` | `2` | Number of text chunks per provider request. |
| `semantic_embedding_request_concurrency` | `BASIC_MEMORY_SEMANTIC_EMBEDDING_REQUEST_CONCURRENCY` | `4` | Maximum concurrent LiteLLM embedding requests. |
| `semantic_embedding_sync_batch_size` | `BASIC_MEMORY_SEMANTIC_EMBEDDING_SYNC_BATCH_SIZE` | `2` | Number of prepared vector jobs flushed through the sync pipeline together. |
## Dimensions
Basic Memory needs the vector dimension before it can create SQLite or Postgres
vector tables. The OpenAI default is known, so this works without an explicit
dimension:
```bash
export BASIC_MEMORY_SEMANTIC_EMBEDDING_PROVIDER=litellm
export BASIC_MEMORY_SEMANTIC_EMBEDDING_MODEL=openai/text-embedding-3-small
```
For every other LiteLLM model, set the dimension explicitly:
```bash
export BASIC_MEMORY_SEMANTIC_EMBEDDING_PROVIDER=litellm
export BASIC_MEMORY_SEMANTIC_EMBEDDING_MODEL=cohere/embed-english-v3.0
export BASIC_MEMORY_SEMANTIC_EMBEDDING_DIMENSIONS=1024
```
For fixed-size models, `semantic_embedding_dimensions` is Basic Memory's local
schema and validation size. For OpenAI/Azure `text-embedding-3` models, LiteLLM
can also forward `dimensions` as a provider-side reduced-output request. Basic
Memory enables that automatically when the model string contains `text-embedding-3`.
If you use an Azure deployment alias such as `azure/<deployment-name>`, the model
string may not reveal that the underlying model supports reduced output dimensions.
Set this only when your deployment supports it:
```bash
export BASIC_MEMORY_SEMANTIC_EMBEDDING_FORWARD_DIMENSIONS=true
```
## Asymmetric Models
Some embedding models use different request roles for indexed documents and
search queries. Basic Memory automatically sets these for known LiteLLM families:
| Model Family | Document `input_type` | Query `input_type` |
|---|---|---|
| Cohere v3 embeddings | `search_document` | `search_query` |
| NVIDIA NIM retrieval embeddings | `passage` | `query` |
For any other asymmetric model, configure both roles explicitly:
```bash
export BASIC_MEMORY_SEMANTIC_EMBEDDING_DOCUMENT_INPUT_TYPE=passage
export BASIC_MEMORY_SEMANTIC_EMBEDDING_QUERY_INPUT_TYPE=query
```
Changing provider, model, dimensions, dimension-forwarding, or document/query
roles changes the meaning of stored vectors. Rebuild embeddings after any of
those changes:
```bash
bm reindex --embeddings
```
## Reindexing with a remote provider
Embedding a real corpus through a network API is far slower than local FastEmbed, and
the defaults are tuned for the local case. Two things to know before you run a full
reindex.
**Raise the sync batch size.** `semantic_embedding_sync_batch_size` defaults to `2`, and
it — not `semantic_embedding_batch_size` — governs throughput on the sync pipeline. With
the default, a full reindex can take tens of seconds *per note* against a remote provider.
Raising both to a larger value turns a multi-minute (or longer) reindex into well under a
minute for the same corpus:
```bash
export BASIC_MEMORY_SEMANTIC_EMBEDDING_SYNC_BATCH_SIZE=32
export BASIC_MEMORY_SEMANTIC_EMBEDDING_BATCH_SIZE=64
```
Stay within the provider's per-request size and rate limits — Cohere v3, for example,
accepts up to 96 inputs per embedding request.
**Changing dimensions requires recreating the vector table.** Basic Memory dimensions the
vector table on first index and refuses to mix sizes. Switching to a model with a
different dimension (for example FastEmbed 384 → OpenAI 1536 → Cohere 1024) makes a plain
`bm reindex` raise an `Embedding dimension mismatch` error. Recreate the table with a full
rebuild — files are the source of truth, so this re-indexes from disk and re-embeds
everything:
```bash
bm reset --reindex
```
To trial a provider without disturbing your existing index, point Basic Memory at a
throwaway config + database instead:
```bash
export BASIC_MEMORY_CONFIG_DIR=/tmp/bm-litellm-trial
```
## Provider Setup Examples
LiteLLM reads provider credentials from the environment. These are the examples
covered by Basic Memory's live validation harness.
### OpenAI Through LiteLLM
```bash
export OPENAI_API_KEY=sk-...
export BASIC_MEMORY_SEMANTIC_SEARCH_ENABLED=true
export BASIC_MEMORY_SEMANTIC_EMBEDDING_PROVIDER=litellm
export BASIC_MEMORY_SEMANTIC_EMBEDDING_MODEL=openai/text-embedding-3-small
```
### Cohere v3
```bash
export COHERE_API_KEY=...
export BASIC_MEMORY_SEMANTIC_SEARCH_ENABLED=true
export BASIC_MEMORY_SEMANTIC_EMBEDDING_PROVIDER=litellm
export BASIC_MEMORY_SEMANTIC_EMBEDDING_MODEL=cohere/embed-english-v3.0
export BASIC_MEMORY_SEMANTIC_EMBEDDING_DIMENSIONS=1024
```
The provider auto-selects `search_document` for indexed chunks and `search_query`
for search queries.
### Azure OpenAI
```bash
export AZURE_API_KEY=...
export AZURE_API_BASE=https://<resource-name>.openai.azure.com
export AZURE_API_VERSION=2024-02-01
export BASIC_MEMORY_SEMANTIC_SEARCH_ENABLED=true
export BASIC_MEMORY_SEMANTIC_EMBEDDING_PROVIDER=litellm
export BASIC_MEMORY_SEMANTIC_EMBEDDING_MODEL=azure/<deployment-name>
export BASIC_MEMORY_SEMANTIC_EMBEDDING_DIMENSIONS=1536
```
If your Azure deployment is a reduced-dimension `text-embedding-3` deployment,
set the dimension you want and enable forwarding:
```bash
export BASIC_MEMORY_SEMANTIC_EMBEDDING_DIMENSIONS=512
export BASIC_MEMORY_SEMANTIC_EMBEDDING_FORWARD_DIMENSIONS=true
```
### NVIDIA NIM
```bash
export NVIDIA_NIM_API_KEY=...
# Optional when using a custom or self-hosted NIM endpoint:
export NVIDIA_NIM_API_BASE=https://integrate.api.nvidia.com/v1
export BASIC_MEMORY_SEMANTIC_SEARCH_ENABLED=true
export BASIC_MEMORY_SEMANTIC_EMBEDDING_PROVIDER=litellm
export BASIC_MEMORY_SEMANTIC_EMBEDDING_MODEL=nvidia_nim/nvidia/embed-qa-4
export BASIC_MEMORY_SEMANTIC_EMBEDDING_DIMENSIONS=1024
```
The provider auto-selects `passage` for indexed chunks and `query` for search
queries.
## Testing LiteLLM Providers
Run the non-live LiteLLM unit and harness tests first:
```bash
uv run pytest tests/repository/test_litellm_provider.py \
test-int/semantic/test_litellm_live_harness.py -q
```
Run the SQLite and Postgres vector identity regressions when changing model
identity, role, or vector sync behavior:
```bash
uv run pytest \
tests/repository/test_sqlite_vector_search_repository.py::test_sqlite_embedding_model_key_includes_litellm_role_settings \
-q
BASIC_MEMORY_TEST_POSTGRES=1 uv run pytest \
tests/repository/test_postgres_search_repository.py::test_postgres_litellm_role_change_reembeds_existing_chunks \
-q
```
The Postgres command uses testcontainers, so Docker must be running.
## Live Provider Harness
The live harness makes real LiteLLM API calls and spends provider quota. It is
opt-in by design:
```bash
export OPENAI_API_KEY=sk-...
export COHERE_API_KEY=...
just test-litellm-live
```
Built-in cases run when their API keys are present:
| Case | Required Env Var | Validates |
|---|---|---|
| `openai-text-embedding-3-small` | `OPENAI_API_KEY` | OpenAI via LiteLLM, 1536 dimensions, normalized vectors, ranking sanity. |
| `cohere-embed-english-v3` | `COHERE_API_KEY` | Cohere v3 role handling, 1024 dimensions, normalized vectors, ranking sanity. |
Add provider aliases or new backends with a custom cases file:
```bash
cat > /tmp/litellm-cases.json <<'JSON'
[
{
"name": "azure-text-embedding-3-small-512",
"model": "azure/<deployment-name>",
"dimensions": 512,
"api_key_env": "AZURE_API_KEY",
"forward_dimensions": true
},
{
"name": "nvidia-embed-qa-4",
"model": "nvidia_nim/nvidia/embed-qa-4",
"dimensions": 1024,
"api_key_env": "NVIDIA_NIM_API_KEY",
"document_input_type": "passage",
"query_input_type": "query"
}
]
JSON
just test-litellm-live --cases-file /tmp/litellm-cases.json
```
For CI-style output:
```bash
just test-litellm-live --cases-file /tmp/litellm-cases.json --json
```
The harness embeds two documents and one query, validates dimension and vector
normalization, checks that the authentication query ranks the authentication
document above a distractor, and reports latency plus role/dimension settings.
## Provider Reference
LiteLLM's own provider and embedding docs are the source of truth for current
model strings and credential names:
- [LiteLLM embedding models](https://docs.litellm.ai/docs/embedding/supported_embedding)
- [LiteLLM Azure OpenAI provider](https://docs.litellm.ai/docs/providers/azure)
- [LiteLLM NVIDIA NIM provider](https://docs.litellm.ai/docs/providers/nvidia_nim)
+183
View File
@@ -0,0 +1,183 @@
# Manual Pages
Basic Memory's manual is written in the style of Unix man pages — and
implemented as Basic Memory notes ([#952](https://github.com/basicmachines-co/basic-memory/issues/952)).
Every page is a markdown note conforming to the `Manpage` schema, `SEE ALSO`
entries are real knowledge-graph relations, and every example on every page
was executed against a live project before the page shipped. The manual
documents the tools; the tools verify the manual.
## Where it lives
The canonical manual is the **`manual` project in the Basic Memory team
workspace** (cloud, shared). Anyone can build their own: the schema ships as
an opt-in seed at `plugins/claude-code/schemas/manpage.md` — copy it into any
project's folder and start writing pages against it.
Layout:
```
manual/
├── schemas/Manpage.md # the manpage schema (type: schema)
├── man1/ # CLI commands bm(1), bm-status(1), ...
├── man3/ # MCP tools write-note(3), search-notes(3), ...
├── man5/ # file formats bm-note(5), bm-observation(5), ...
├── man7/ # concepts basic-memory(7), semantic-memory(7), ...
├── playground/ # scratch notes for destructive examples
└── diagrams/ # canvas visualizations of the manual graph
```
### Why "man1", "man3", "man5"?
The folder names are Unix's, unchanged since 1971. The manual is divided
into numbered **sections**, pages physically live in directories named
after them (`/usr/share/man/man1`, `man5`, ...), and the number tells you
what *kind* of thing is documented — not importance, not reading order:
- **1** — user commands (`ls`, `grep`)
- **2** — system calls
- **3** — library functions / APIs (`printf(3)`)
- **4** — devices
- **5** — file formats and config files (`crontab(5)`, `passwd(5)`)
- **6** — games (really)
- **7** — miscellanea: concepts, conventions, overviews (`regex(7)`, `signal(7)`)
- **8** — system administration
That's also why man page names carry the parenthesized number —
`crontab(1)` is the command, `crontab(5)` is the file format, same name in
two sections. `man 5 crontab` picks the section explicitly.
This manual copies that layout with the sections that have a Basic Memory
analog:
- **man1/**`bm` CLI commands → `bm-status(1)`
- **man3/** — MCP tools, our equivalent of the "library API" section → `write-note(3)`
- **man5/** — file formats: note syntax, observations, relations, schemas → `bm-note(5)`
- **man7/** — concepts → `basic-memory(7)`, `semantic-memory(7)`
- **8** is reserved for admin/cloud operations but has no pages yet; 2, 4,
and 6 have no analog (no system calls, no devices, and no games — yet)
When a page says `see_also [[bm-note(5)]]`, the `(5)` reads "the
file-format page," exactly the way a Unix manual cross-references — except
here it's a traversable relation in the graph instead of a typographic
convention. The manual explains its own conventions in `man-pages(7)`
fittingly, the same page name Linux uses for this, and that almost nobody
ever reads.
## Page anatomy
Pages use the classic headers where applicable: `NAME`, `SYNOPSIS`,
`DESCRIPTION`, `PARAMETERS`, `MCP USAGE`, `CLI EQUIVALENT`, `EXAMPLES`,
`GOTCHAS`, `SEE ALSO`. Frontmatter (validated by the schema):
```yaml
type: manpage
section: 3 # 1 | 3 | 5 | 7 | 8
name: write-note # page name without section suffix
summary: create or overwrite a markdown note in the knowledge base
generated: hand # hand | registry | typer (regeneration ownership)
tool: write_note # section-3 pages: the MCP tool documented
command: basic-memory status # section-1 pages: the CLI command documented
verified: 0.21.6 mcp+cli # version + path(s) that proved the page
```
Field knowledge accumulates as observations — `[gotcha]`, `[bug]` (with issue
links), `[pattern]` — and `SEE ALSO` entries are `see_also` relations, so the
manual is a navigable graph, not a folder of files.
## How to use it
Man-style reads (any MCP client or the CLI):
```bash
# read a page
bm tool read-note "man3/write-note-3" --project manual
# apropos — find pages by section, tool, or text
bm tool search-notes --project manual # then filter, or via MCP:
# search_notes(project="manual", metadata_filters={"type": "manpage", "section": 3})
# search_notes(project="manual", metadata_filters={"type": "manpage", "tool": "write_note"})
# traverse SEE ALSO from any page
# build_context(url="man3/write-note-3", project="manual")
```
A future `bm man <topic>` command is thin sugar over exactly these calls.
And for the real thing — `man bm` in an actual terminal:
```bash
bm man install # copies bundled groff pages to ~/.local/share/man
man bm # the overview page, rendered by man(1)
man basic-memory # same page via its alias
```
`bm man install` warns with a one-line `MANPATH` fix if the install root
isn't searched by your `man`. Agents with shell access can use `man bm` as
an offline quick reference; the full per-tool detail stays in the manual
project's section-3 pages.
## The verification discipline
Two rules make the manual trustworthy:
1. **Examples must have run.** An `EXAMPLES` (or `MCP USAGE` / `CLI
EQUIVALENT`) block contains only commands that actually executed against
the manual project. Destructive operations (`delete_note`, `move_note`,
destructive `edit_note`) run only against `playground/` notes — never
against pages. The `verified:` field records the version and which path
proved the page: `mcp` (live service), `cli` (dev checkout), or both.
2. **The schema is the linter.** Validate the whole manual any time:
```bash
bm tool schema-validate manpage --project manual
# → {"total_notes": 38, "valid_count": 38, "warning_count": 0, ...}
```
`bm orphans --project manual` confirms every page is connected to the
graph, and `schema_diff`/`schema_infer` report drift between the schema
and how pages are actually written.
Because verification exercises real tool calls against the live service,
building the manual doubles as an end-to-end smoke test. The initial build
found six bugs in one pass (#954#959) — including the verification rule
catching a test that asserted a bug as expected output (#958).
## Adding or updating a page
1. Run the commands you intend to document; keep the actual output.
2. Write the page with `write_note`, passing frontmatter through the
`metadata` parameter (nested YAML in content frontmatter is unreliable on
some clients):
```
write_note(title="my-tool(3)", directory="man3", project="manual",
note_type="manpage",
metadata={"section": 3, "name": "my-tool",
"summary": "...", "generated": "hand",
"tool": "my_tool", "verified": "<version> mcp"})
```
3. Link related pages in `SEE ALSO` with `see_also [[other-page(3)]]`.
Forward references to pages that don't exist yet are fine — they resolve
automatically when the target is written.
4. Validate: `bm tool schema-validate manpage --project manual`.
For mechanical updates to generated sections, prefer `edit_note` with
`replace_section` / `insert_after_section` so curated content (EXAMPLES,
GOTCHAS, SEE ALSO, observations) survives — that ownership split is what the
`generated:` field declares.
## Roadmap
- **Registry generator** — section-3 SYNOPSIS/PARAMETERS generated from the
MCP tool registry (docstrings + pydantic schemas), section-1 from Typer
help; the hand-written corpus is the template spec. Regenerate-and-diff in
CI becomes the drift gate.
- **`bm man <topic>`** — CLI sugar over `read_note` + metadata search.
(`bm man install` + a hand-written `bm.1` already ship — the first slice
of [#610](https://github.com/basicmachines-co/basic-memory/issues/610);
the generator will produce per-command pages from the same extraction.)
- **Docs site** — the notes remain canonical for sections 5 and 7, code is
canonical for 1 and 3; both render to the hosted docs site.
+116 -5
View File
@@ -99,10 +99,13 @@ All settings are fields on `BasicMemoryConfig` and can be set via environment va
| Config Field | Env Var | Default | Description |
|---|---|---|---|
| `semantic_search_enabled` | `BASIC_MEMORY_SEMANTIC_SEARCH_ENABLED` | Auto (`true` when semantic deps are available) | Enable semantic search. Required before vector/hybrid modes work. |
| `semantic_embedding_provider` | `BASIC_MEMORY_SEMANTIC_EMBEDDING_PROVIDER` | `"fastembed"` | Embedding provider: `"fastembed"` (local) or `"openai"` (API). |
| `semantic_embedding_provider` | `BASIC_MEMORY_SEMANTIC_EMBEDDING_PROVIDER` | `"fastembed"` | Embedding provider: `"fastembed"` (local), `"openai"` (API), or `"litellm"` (multi-provider API, **experimental** — advanced users only). |
| `semantic_embedding_model` | `BASIC_MEMORY_SEMANTIC_EMBEDDING_MODEL` | `"bge-small-en-v1.5"` | Model identifier. Auto-adjusted per provider if left at default. |
| `semantic_embedding_dimensions` | `BASIC_MEMORY_SEMANTIC_EMBEDDING_DIMENSIONS` | Auto-detected | Vector dimensions. 384 for FastEmbed, 1536 for OpenAI. Override only if using a non-default model. |
| `semantic_embedding_batch_size` | `BASIC_MEMORY_SEMANTIC_EMBEDDING_BATCH_SIZE` | `64` | Number of texts to embed per batch. |
| `semantic_embedding_dimensions` | `BASIC_MEMORY_SEMANTIC_EMBEDDING_DIMENSIONS` | Provider default | Vector dimensions. 384 for FastEmbed, 1536 for OpenAI/LiteLLM OpenAI. Required when using a non-default LiteLLM model. |
| `semantic_embedding_forward_dimensions` | `BASIC_MEMORY_SEMANTIC_EMBEDDING_FORWARD_DIMENSIONS` | Auto | LiteLLM-only override for whether configured dimensions are sent as a provider-side output-size request. |
| `semantic_embedding_batch_size` | `BASIC_MEMORY_SEMANTIC_EMBEDDING_BATCH_SIZE` | `2` | Number of texts to embed per batch. |
| `semantic_embedding_document_input_type` | `BASIC_MEMORY_SEMANTIC_EMBEDDING_DOCUMENT_INPUT_TYPE` | Auto for known LiteLLM models | Optional LiteLLM `input_type` for indexed document/passages. |
| `semantic_embedding_query_input_type` | `BASIC_MEMORY_SEMANTIC_EMBEDDING_QUERY_INPUT_TYPE` | Auto for known LiteLLM models | Optional LiteLLM `input_type` for search queries. |
| `semantic_vector_k` | `BASIC_MEMORY_SEMANTIC_VECTOR_K` | `100` | Candidate count for vector nearest-neighbour retrieval. Higher values improve recall at the cost of latency. |
## Embedding Providers
@@ -135,7 +138,114 @@ export BASIC_MEMORY_SEMANTIC_EMBEDDING_PROVIDER=openai
export OPENAI_API_KEY=sk-...
```
When switching from FastEmbed to OpenAI (or vice versa), you must rebuild embeddings since the vector dimensions differ:
### LiteLLM
> **Experimental — advanced users only.** The LiteLLM provider is experimental and aimed at users comfortable operating remote embedding backends: paid API calls, per-model dimension and input-role configuration, and slower reindexing of large corpora. For most users, FastEmbed (local, default) is recommended. See [LiteLLM Provider](litellm-provider.md) for the caveats and tuning.
Uses the LiteLLM SDK to call embedding models from providers such as OpenAI, Cohere, Azure, Bedrock, NVIDIA NIM, and other LiteLLM-supported backends. Requires the provider's API credentials.
For the full option reference, provider setup examples, and live validation harness, see [LiteLLM Provider](litellm-provider.md).
```bash
export BASIC_MEMORY_SEMANTIC_SEARCH_ENABLED=true
export BASIC_MEMORY_SEMANTIC_EMBEDDING_PROVIDER=litellm
export BASIC_MEMORY_SEMANTIC_EMBEDDING_MODEL=cohere/embed-english-v3.0
export BASIC_MEMORY_SEMANTIC_EMBEDDING_DIMENSIONS=1024
export COHERE_API_KEY=...
```
Basic Memory creates vector tables before the first embedding call, so non-default LiteLLM models must set `BASIC_MEMORY_SEMANTIC_EMBEDDING_DIMENSIONS`. The LiteLLM OpenAI default (`openai/text-embedding-3-small`) uses 1536 dimensions automatically.
For fixed-size LiteLLM models, dimensions are used as Basic Memory's local vector schema and
validation size. Basic Memory automatically sends dimensions as a provider-side output-size
request for `text-embedding-3` model strings, where LiteLLM/OpenAI support reduced output
dimensions. If an Azure/OpenAI deployment uses an arbitrary LiteLLM model string such as
`azure/<deployment-name>` and the underlying model supports reduced dimensions, set
`BASIC_MEMORY_SEMANTIC_EMBEDDING_FORWARD_DIMENSIONS=true`.
Some retrieval models are asymmetric: indexed passages and search queries must be embedded with different provider parameters. Basic Memory automatically sets LiteLLM `input_type` for known asymmetric model families:
- Cohere v3: documents use `search_document`, queries use `search_query`
- NVIDIA NIM retrieval models: documents use `passage`, queries use `query`
For other asymmetric LiteLLM models, set the input types explicitly:
```bash
export BASIC_MEMORY_SEMANTIC_EMBEDDING_DOCUMENT_INPUT_TYPE=passage
export BASIC_MEMORY_SEMANTIC_EMBEDDING_QUERY_INPUT_TYPE=query
```
#### Live LiteLLM Validation
Provider APIs differ in subtle ways: some accept `dimensions`, some require separate
document/query roles, and some route through deployment aliases that do not reveal the
underlying model name. Before adding or changing LiteLLM model support, run the opt-in live
evaluation harness:
```bash
export OPENAI_API_KEY=sk-...
export COHERE_API_KEY=...
just test-litellm-live
```
The built-in live cases cover:
| Case | Required key | What it validates |
|---|---|---|
| `openai/text-embedding-3-small` | `OPENAI_API_KEY` | Standard LiteLLM OpenAI embedding calls and normalized 1536-dimensional output. |
| `cohere/embed-english-v3.0` | `COHERE_API_KEY` | Cohere v3 asymmetric `search_document` / `search_query` handling and fixed 1024-dimensional output. |
The harness embeds two documents and one query, checks vector dimensions and normalization,
then verifies the authentication query ranks the authentication document above the distractor.
It prints a table with per-model scores, norms, latency, role settings, and dimension-forwarding
mode.
To validate provider aliases or additional LiteLLM backends, save custom JSON cases:
```bash
export AZURE_API_KEY=...
export AZURE_API_BASE=https://example.openai.azure.com
export AZURE_API_VERSION=2024-02-01
cat > /tmp/litellm-azure-cases.json <<'JSON'
[
{
"name": "azure-text-embedding-3-small-512",
"model": "azure/<deployment-name>",
"dimensions": 512,
"api_key_env": "AZURE_API_KEY",
"forward_dimensions": true
}
]
JSON
just test-litellm-live --cases-file /tmp/litellm-azure-cases.json
```
NVIDIA NIM retrieval models can be checked the same way:
```bash
export NVIDIA_NIM_API_KEY=...
cat > /tmp/litellm-nvidia-cases.json <<'JSON'
[
{
"name": "nvidia-embed-qa-4",
"model": "nvidia_nim/nvidia/embed-qa-4",
"dimensions": 1024,
"api_key_env": "NVIDIA_NIM_API_KEY",
"document_input_type": "passage",
"query_input_type": "query"
}
]
JSON
just test-litellm-live --cases-file /tmp/litellm-nvidia-cases.json
```
For repeatable local runs, put the same JSON array in a file and pass
`just test-litellm-live --cases-file path/to/litellm-cases.json`.
When switching providers, models, dimensions, or LiteLLM document/query input types, rebuild embeddings:
```bash
bm reindex --embeddings
@@ -203,9 +313,10 @@ bm reindex -p my-project
- **Upgrade note**: Migration now performs a one-time automatic embedding backfill on upgrade.
- **Manual enable case**: If you explicitly had `semantic_search_enabled=false` and then turn it on
- **Provider change**: After switching between `fastembed` and `openai`
- **Provider change**: After switching between `fastembed`, `openai`, and `litellm`
- **Model change**: After changing `semantic_embedding_model`
- **Dimension change**: After changing `semantic_embedding_dimensions`
- **LiteLLM role change**: After changing `semantic_embedding_document_input_type` or `semantic_embedding_query_input_type`
The reindex command shows progress with embedded/skipped/error counts:
+7
View File
@@ -0,0 +1,7 @@
{
"$schema": "https://glama.ai/mcp/schemas/server.json",
"maintainers": [
"phernandez",
"groksrc"
]
}
+1 -1
View File
@@ -38,7 +38,7 @@ from typing import Any, Callable
from agent.memory_provider import MemoryProvider
from tools.registry import tool_error
__version__ = "0.21.6"
__version__ = "0.22.1"
logger = logging.getLogger("hermes.memory.basic-memory")
+1 -1
View File
@@ -1,5 +1,5 @@
name: basic-memory
version: 0.21.6
version: 0.22.1
description: "Basic Memory — persistent knowledge graph backed by the basic-memory MCP server"
pip_dependencies:
- mcp
+1 -1
View File
@@ -150,7 +150,7 @@ This plugin ships with workflow-oriented skills that are automatically loaded wh
No manual installation needed. To update skills or install new ones as they become available:
```bash
npx skills add basicmachines-co/basic-memory --path skills --agent openclaw
npx skills add basicmachines-co/basic-memory/skills --agent openclaw
```
See the canonical source at [`basic-memory/skills`](../../skills).
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@basicmemory/openclaw-basic-memory",
"version": "0.21.6",
"version": "0.22.1",
"type": "module",
"main": "./dist/index.js",
"types": "./dist/index.d.ts",
+176 -51
View File
@@ -1,5 +1,12 @@
# Basic Memory - Modern Command Runner
TESTMON_FLAGS := env_var_or_default("BASIC_MEMORY_TESTMON_FLAGS", "--testmon-noselect")
TESTMON_SELECT_FLAGS := env_var_or_default("BASIC_MEMORY_TESTMON_SELECT_FLAGS", "--testmon --testmon-forceselect")
TESTMON_REFRESH_FLAGS := env_var_or_default("BASIC_MEMORY_TESTMON_REFRESH_FLAGS", "--testmon-noselect")
# CI shards the Postgres unit suite across parallel jobs via pytest-split
# (e.g. "--splits 3 --group 2"). Empty locally.
PYTEST_SPLIT_FLAGS := env_var_or_default("BASIC_MEMORY_PYTEST_SPLIT_FLAGS", "")
# Install dependencies
install:
uv sync
@@ -35,40 +42,60 @@ test-sqlite: test-unit-sqlite test-int-sqlite
test-postgres: test-unit-postgres test-int-postgres
# Run unit tests against SQLite
test-unit-sqlite:
BASIC_MEMORY_ENV=test uv run pytest -p pytest_mock -v --no-cov tests
test-unit-sqlite: testmon-seed
BASIC_MEMORY_ENV=test uv run pytest -p pytest_mock -v --no-cov {{TESTMON_FLAGS}} --testmon-env=unit-sqlite tests
# Run unit tests against Postgres
test-unit-postgres:
BASIC_MEMORY_ENV=test BASIC_MEMORY_TEST_POSTGRES=1 uv run pytest -p pytest_mock -v --no-cov tests
# Exit code 5 (no tests collected) is success: a testmon-selected PR build can
# leave a pytest-split shard empty.
test-unit-postgres: testmon-seed
#!/usr/bin/env bash
set -euo pipefail
BASIC_MEMORY_ENV=test BASIC_MEMORY_TEST_POSTGRES=1 uv run pytest -p pytest_mock -v --no-cov {{TESTMON_FLAGS}} {{PYTEST_SPLIT_FLAGS}} --testmon-env=unit-postgres tests || test $? -eq 5
# Run integration tests against SQLite (excludes semantic benchmarks — use just test-semantic)
test-int-sqlite:
BASIC_MEMORY_ENV=test uv run pytest -p pytest_mock -v --no-cov -m "not semantic" test-int
# Run integration tests against SQLite (excludes semantic tests and on-demand benchmarks —
# use just test-semantic / run benchmark files explicitly)
test-int-sqlite: testmon-seed
BASIC_MEMORY_ENV=test uv run pytest -p pytest_mock -v --no-cov {{TESTMON_FLAGS}} --testmon-env=int-sqlite -m "not semantic and not benchmark" test-int
# Run integration tests against Postgres
# Note: Uses timeout due to FastMCP Client + asyncpg cleanup hang (tests pass, process hangs on exit)
# See: https://github.com/jlowin/fastmcp/issues/1311
test-int-postgres:
test-int-postgres: testmon-seed
#!/usr/bin/env bash
set -euo pipefail
# Use gtimeout (macOS/Homebrew) or timeout (Linux)
TIMEOUT_CMD=$(command -v gtimeout || command -v timeout || echo "")
if [[ -n "$TIMEOUT_CMD" ]]; then
$TIMEOUT_CMD --signal=KILL 600 bash -c 'BASIC_MEMORY_ENV=test BASIC_MEMORY_TEST_POSTGRES=1 uv run pytest -p pytest_mock -v --no-cov -m "not semantic" test-int' || test $? -eq 137
$TIMEOUT_CMD --signal=KILL 600 bash -c 'BASIC_MEMORY_ENV=test BASIC_MEMORY_TEST_POSTGRES=1 uv run pytest -p pytest_mock -v --no-cov {{TESTMON_FLAGS}} --testmon-env=int-postgres -m "not semantic and not benchmark" test-int' || test $? -eq 137
else
echo "⚠️ No timeout command found, running without timeout..."
BASIC_MEMORY_ENV=test BASIC_MEMORY_TEST_POSTGRES=1 uv run pytest -p pytest_mock -v --no-cov -m "not semantic" test-int
BASIC_MEMORY_ENV=test BASIC_MEMORY_TEST_POSTGRES=1 uv run pytest -p pytest_mock -v --no-cov {{TESTMON_FLAGS}} --testmon-env=int-postgres -m "not semantic and not benchmark" test-int
fi
# Run tests impacted by recent changes (requires pytest-testmon)
# Pass paths or node ids after `just testmon` to limit the candidate set further.
testmon *args:
BASIC_MEMORY_ENV=test uv run pytest -p pytest_mock -v --no-cov --testmon {{args}}
testmon *args: testmon-seed
BASIC_MEMORY_ENV=test uv run pytest -p pytest_mock -v --no-cov {{TESTMON_SELECT_FLAGS}} --testmon-env=local {{args}}
# Seed pytest-testmon data into this worktree from the shared Git cache.
testmon-seed:
uv run python scripts/testmon_cache.py seed
# Refresh the shared pytest-testmon cache from a full backend test run.
testmon-refresh:
#!/usr/bin/env bash
set -euo pipefail
BASIC_MEMORY_TESTMON_FLAGS="{{TESTMON_REFRESH_FLAGS}}" just test
uv run python scripts/testmon_cache.py refresh
# Show local and shared pytest-testmon cache locations.
testmon-status:
uv run python scripts/testmon_cache.py status
# Run MCP smoke test (fast end-to-end loop)
test-smoke:
BASIC_MEMORY_ENV=test uv run pytest -p pytest_mock -v --no-cov -m smoke test-int/mcp/test_smoke_integration.py
test-smoke: testmon-seed
BASIC_MEMORY_ENV=test uv run pytest -p pytest_mock -v --no-cov {{TESTMON_FLAGS}} --testmon-env=smoke -m smoke test-int/mcp/test_smoke_integration.py
# Fast local loop: lint, format, typecheck, impacted tests via pytest-testmon
fast-check:
@@ -97,27 +124,31 @@ postgres-migrate:
# Run Windows-specific tests only (only works on Windows platform)
# These tests verify Windows-specific database optimizations (locking mode, NullPool)
# Will be skipped automatically on non-Windows platforms
test-windows:
BASIC_MEMORY_ENV=test uv run pytest -p pytest_mock -v --no-cov -m windows tests test-int
test-windows: testmon-seed
BASIC_MEMORY_ENV=test uv run pytest -p pytest_mock -v --no-cov {{TESTMON_FLAGS}} --testmon-env=windows -m windows tests test-int
# Run benchmark tests only (performance testing)
# These are slow tests that measure sync performance with various file counts
# Excluded from default test runs to keep CI fast
test-benchmark:
BASIC_MEMORY_ENV=test uv run pytest -p pytest_mock -v --no-cov -m benchmark tests test-int
test-benchmark: testmon-seed
BASIC_MEMORY_ENV=test uv run pytest -p pytest_mock -v --no-cov {{TESTMON_FLAGS}} --testmon-env=benchmark -m benchmark tests test-int
# Run semantic search quality benchmarks (all combos)
test-semantic:
BASIC_MEMORY_ENV=test uv run pytest -p pytest_mock -v --no-cov -m semantic test-int/semantic/
test-semantic: testmon-seed
BASIC_MEMORY_ENV=test uv run pytest -p pytest_mock -v --no-cov {{TESTMON_FLAGS}} --testmon-env=semantic -m semantic test-int/semantic/
# Run semantic benchmarks with JSON artifact output, then show report
test-semantic-report:
BASIC_MEMORY_ENV=test BASIC_MEMORY_BENCHMARK_OUTPUT=.benchmarks/semantic-quality.jsonl uv run pytest -p pytest_mock -v -s --no-cov -m semantic test-int/semantic/
uv run python test-int/semantic/report.py .benchmarks/semantic-quality.jsonl
# Run opt-in live LiteLLM provider checks against configured external APIs
test-litellm-live *args:
BASIC_MEMORY_ENV=test BASIC_MEMORY_RUN_LITELLM_INTEGRATION=1 PYTHONPATH=test-int:src uv run python -m semantic.litellm_live_harness {{args}}
# Run semantic benchmarks (Postgres combos only)
test-semantic-postgres:
BASIC_MEMORY_ENV=test uv run pytest -p pytest_mock -v --no-cov -m semantic -k postgres test-int/semantic/
test-semantic-postgres: testmon-seed
BASIC_MEMORY_ENV=test uv run pytest -p pytest_mock -v --no-cov {{TESTMON_FLAGS}} --testmon-env=semantic-postgres -m semantic -k postgres test-int/semantic/
# View semantic benchmark results (rich formatted table)
# Usage: just semantic-report [--filter-combo sqlite] [--filter-suite paraphrase] [--sort-by avg_latency_ms]
@@ -133,8 +164,8 @@ benchmark-compare baseline candidate *args:
# Run all tests including Windows, Postgres, and Benchmarks (for CI/comprehensive testing)
# Use this before releasing to ensure everything works across all backends and platforms
test-all:
BASIC_MEMORY_ENV=test uv run pytest -p pytest_mock -v --no-cov tests test-int
test-all: testmon-seed
BASIC_MEMORY_ENV=test uv run pytest -p pytest_mock -v --no-cov {{TESTMON_FLAGS}} --testmon-env=all tests test-int
# Generate HTML coverage report
coverage:
@@ -307,7 +338,7 @@ set-version version scope="all":
set-version-dry-run version scope="all":
python3 scripts/update_versions.py "{{version}}" --scope "{{scope}}" --dry-run
# Set the version for just the plugin/agent artifacts (plugin, marketplaces, Hermes, OpenClaw)
# Set the version for just the plugin/agent artifacts (plugins, marketplaces, Hermes, OpenClaw)
set-packages-version version:
just set-version "{{version}}" packages
@@ -352,43 +383,93 @@ release version:
echo "❌ Tag {{version}} already exists"
exit 1
fi
# Changelog must already be on main (land it via a normal PR first)
if ! grep -q "^## {{version}} " CHANGELOG.md; then
echo "❌ CHANGELOG.md has no entry for {{version}}. Land one via PR first."
exit 1
fi
# Run quality checks
echo "🔍 Running lint checks..."
just lint
just typecheck
# Update all package manifests to the one Basic Memory product version.
echo "📝 Updating consolidated package versions..."
just set-version "{{version}}"
# Commit version update
# Trigger: main's ruleset rejects direct pushes ("Changes must be made
# through a pull request").
# Why: the version bump must land on main before the tag is cut, so it
# rides a release PR that is rebase-merged (the repo disallows merge
# commits).
# Outcome: the bump commit gets a new SHA on main; the tag is created on
# that rebased commit, found by its commit subject.
COMMIT_SUBJECT="chore: update version to $VERSION_NUM for {{version}} release"
git checkout -b "release/{{version}}"
git add \
src/basic_memory/__init__.py \
server.json \
.claude-plugin/marketplace.json \
plugins/claude-code/.claude-plugin/plugin.json \
plugins/claude-code/.claude-plugin/marketplace.json \
plugins/codex/.codex-plugin/plugin.json \
integrations/hermes/plugin.yaml \
integrations/hermes/__init__.py \
integrations/openclaw/package.json
git commit -s -m "chore: update version to $VERSION_NUM for {{version}} release"
# Create and push tag
echo "🏷️ Creating tag {{version}}..."
git tag "{{version}}"
echo "📤 Pushing to GitHub..."
git push origin main
git commit -s -m "$COMMIT_SUBJECT"
echo "📤 Opening release PR..."
git push -u origin "release/{{version}}"
gh pr create --title "chore(core): release {{version}}" \
--body "Version bump for {{version}}. See CHANGELOG.md for release notes."
# Trigger: the PR may not be mergeable synchronously (merge gates,
# required checks added later, or GitHub still computing mergeability).
# Why: the tag must point at the bump commit on main, so the recipe
# cannot tag until the merge has actually landed.
# Outcome: try a direct rebase-merge, fall back to queueing auto-merge,
# then poll main for the rebased bump commit before tagging.
if ! gh pr merge "release/{{version}}" --rebase --delete-branch; then
echo "⚠️ Direct merge did not complete (merge gates pending?). Queueing auto-merge..."
gh pr merge "release/{{version}}" --rebase --delete-branch --auto
fi
echo "⏳ Waiting for the bump commit to land on main..."
TAG_COMMIT=""
for _ in $(seq 1 60); do
git fetch origin main --quiet
TAG_COMMIT=$(git log FETCH_HEAD --fixed-strings --grep "$COMMIT_SUBJECT" --format='%H' -1)
[[ -n "$TAG_COMMIT" ]] && break
sleep 5
done
if [[ -z "$TAG_COMMIT" ]]; then
echo "❌ Bump commit not on main after 5 minutes (merge still pending?)."
echo " Once the release PR merges, finish the release manually:"
echo " git fetch origin main"
echo " git tag {{version}} \$(git log FETCH_HEAD --fixed-strings --grep \"$COMMIT_SUBJECT\" --format='%H' -1)"
echo " git push origin {{version}}"
exit 1
fi
git checkout main
git pull --ff-only origin main
git branch -D "release/{{version}}" 2>/dev/null || true
echo "🏷️ Creating tag {{version}} at $TAG_COMMIT..."
git tag "{{version}}" "$TAG_COMMIT"
git push origin "{{version}}"
echo "✅ Release {{version}} created successfully!"
echo "📦 GitHub Actions will build and publish to PyPI"
echo "🔗 Monitor at: https://github.com/basicmachines-co/basic-memory/actions"
echo ""
echo "📝 REMINDER: Post-release tasks:"
echo " 1. docs.basicmemory.com - Add release notes to src/pages/latest-releases.mdx"
echo " 2. basicmachines.co - Update version in src/components/sections/hero.tsx"
echo " 1. docs.basicmemory.com - Add a What's New page under content/2.whats-new/"
echo " and bump the badge in content/index.md (see that repo's CLAUDE.md)"
echo " 2. basicmemory.com - No version number in the site UI; for a significant"
echo " release optionally add a post under src/content/blog/. Skip for patches."
echo " 3. MCP Registry - Run: mcp-publisher publish"
echo " See: .claude/commands/release/release.md for detailed instructions"
@@ -435,34 +516,78 @@ beta version:
echo "📝 Updating consolidated package versions..."
just set-version "{{version}}"
# Commit version update
# Trigger: main's ruleset rejects direct pushes ("Changes must be made
# through a pull request").
# Why: the version bump must land on main before the tag is cut, so it
# rides a release PR that is rebase-merged (the repo disallows merge
# commits).
# Outcome: the bump commit gets a new SHA on main; the tag is created on
# that rebased commit, found by its commit subject.
COMMIT_SUBJECT="chore: update version to $VERSION_NUM for {{version}} beta release"
git checkout -b "release/{{version}}"
git add \
src/basic_memory/__init__.py \
server.json \
.claude-plugin/marketplace.json \
plugins/claude-code/.claude-plugin/plugin.json \
plugins/claude-code/.claude-plugin/marketplace.json \
plugins/codex/.codex-plugin/plugin.json \
integrations/hermes/plugin.yaml \
integrations/hermes/__init__.py \
integrations/openclaw/package.json
git commit -s -m "chore: update version to $VERSION_NUM for {{version}} beta release"
# Create and push tag
echo "🏷️ Creating tag {{version}}..."
git tag "{{version}}"
echo "📤 Pushing to GitHub..."
git push origin main
git commit -s -m "$COMMIT_SUBJECT"
echo "📤 Opening release PR..."
git push -u origin "release/{{version}}"
gh pr create --title "chore(core): release {{version}}" \
--body "Version bump for {{version}} beta."
# Trigger: the PR may not be mergeable synchronously (merge gates,
# required checks added later, or GitHub still computing mergeability).
# Why: the tag must point at the bump commit on main, so the recipe
# cannot tag until the merge has actually landed.
# Outcome: try a direct rebase-merge, fall back to queueing auto-merge,
# then poll main for the rebased bump commit before tagging.
if ! gh pr merge "release/{{version}}" --rebase --delete-branch; then
echo "⚠️ Direct merge did not complete (merge gates pending?). Queueing auto-merge..."
gh pr merge "release/{{version}}" --rebase --delete-branch --auto
fi
echo "⏳ Waiting for the bump commit to land on main..."
TAG_COMMIT=""
for _ in $(seq 1 60); do
git fetch origin main --quiet
TAG_COMMIT=$(git log FETCH_HEAD --fixed-strings --grep "$COMMIT_SUBJECT" --format='%H' -1)
[[ -n "$TAG_COMMIT" ]] && break
sleep 5
done
if [[ -z "$TAG_COMMIT" ]]; then
echo "❌ Bump commit not on main after 5 minutes (merge still pending?)."
echo " Once the release PR merges, finish the release manually:"
echo " git fetch origin main"
echo " git tag {{version}} \$(git log FETCH_HEAD --fixed-strings --grep \"$COMMIT_SUBJECT\" --format='%H' -1)"
echo " git push origin {{version}}"
exit 1
fi
git checkout main
git pull --ff-only origin main
git branch -D "release/{{version}}" 2>/dev/null || true
echo "🏷️ Creating tag {{version}} at $TAG_COMMIT..."
git tag "{{version}}" "$TAG_COMMIT"
git push origin "{{version}}"
echo "✅ Beta release {{version}} created successfully!"
echo "📦 GitHub Actions will build and publish to PyPI as pre-release"
echo "🔗 Monitor at: https://github.com/basicmachines-co/basic-memory/actions"
echo "📥 Install with: uv tool install basic-memory --pre"
echo ""
echo "📝 REMINDER: For stable releases, update documentation sites:"
echo " 1. docs.basicmemory.com - Add release notes to src/pages/latest-releases.mdx"
echo " 2. basicmachines.co - Update version in src/components/sections/hero.tsx"
echo " 1. docs.basicmemory.com - Add a What's New page under content/2.whats-new/"
echo " and bump the badge in content/index.md (see that repo's CLAUDE.md)"
echo " 2. basicmemory.com - No version number in the site UI; for a significant"
echo " release optionally add a post under src/content/blog/. Skip for patches."
echo " See: .claude/commands/release/release.md for detailed instructions"
# List all available recipes
@@ -6,14 +6,14 @@
},
"metadata": {
"description": "Official plugins from Basic Machines for knowledge management and AI-assisted development",
"version": "0.21.6"
"version": "0.22.1"
},
"plugins": [
{
"name": "basic-memory",
"source": "./",
"description": "The bridge between Claude's working memory and Basic Memory's durable knowledge graph \u2014 session briefings, pre-compaction checkpoints, and capture reflexes",
"version": "0.21.6",
"version": "0.22.1",
"author": {
"name": "Basic Machines"
},
@@ -1,7 +1,7 @@
{
"name": "basic-memory",
"description": "The bridge between Claude's working memory and Basic Memory's durable knowledge graph \u2014 session briefings, pre-compaction checkpoints, and capture reflexes",
"version": "0.21.6",
"version": "0.22.1",
"author": {
"name": "Basic Machines"
},
+1 -1
View File
@@ -24,7 +24,7 @@ Memory's durable graph**, rather than a memory layer of its own. See
configures the project for the plugin: maps it to a Basic Memory project (picking
an existing one or creating a new one), seeds the `session`/`decision`/`task`
schemas into the project, installs the shared `memory-*` skills via
`npx skills add basicmachines-co/basic-memory --path skills` (the plugin doesn't
`npx skills add basicmachines-co/basic-memory/skills` (the plugin doesn't
vendor its own copies — `skills/` is the single source of truth, shared with
OpenClaw), optionally learns the project's placement conventions, and enables the
capture reflexes. Writes the `basicMemory` block to
+55
View File
@@ -0,0 +1,55 @@
---
title: Manpage
type: schema
entity: Manpage
version: 1
schema:
gotcha?(array): string, sharp edges and surprising behavior learned from live verification
example?(array): string, worked examples beyond the generated synopsis
pattern?(array): string, recommended idioms and usage patterns
bug?(array): string, known defects affecting this surface, with issue links
see_also?(array): Entity, related manual pages — the SEE ALSO graph
settings:
validation: warn
frontmatter:
section(enum, Unix manual section number): [1, 3, 5, 7, 8]
name: string, page name without section suffix (e.g. write-note)
summary: string, one-line NAME description
generated?(enum, who owns the mechanical sections): [registry, typer, hand]
tool?: string, MCP tool this page documents (section 3 pages)
command?: string, CLI command this page documents (section 1 pages)
verified?: string, version and path that verified this page (e.g. 0.21.6 mcp+cli)
since?: string, version this surface first appeared
---
# Manpage
A **ManpageNote** is one page of a Unix-style manual implemented as Basic
Memory notes (issue #952): commands in section 1, MCP tools in section 3,
file formats in section 5, concepts in section 7, admin in section 8. The
manual becomes a knowledge graph — `SEE ALSO` entries are typed relations,
and pages are found by structured recall:
`search_notes(metadata_filters={"type": "manpage", "section": 3})`.
This schema is an opt-in seed for documentation projects; the canonical
manual lives in the Basic Memory team workspace `manual` project.
## What makes a good ManpageNote
- **NAME / SYNOPSIS / DESCRIPTION** — classic man-page structure, with
PARAMETERS, MCP USAGE, CLI EQUIVALENT, EXAMPLES, GOTCHAS, SEE ALSO where
applicable.
- **Verified examples** — EXAMPLES contain only commands that actually ran;
the `verified` field records the version and path (mcp, cli, or both).
- **generated** — declares regeneration ownership: `registry` (from the MCP
tool registry) and `typer` (from CLI help) pages get mechanical sections
rewritten; curated sections (EXAMPLES, GOTCHAS, SEE ALSO, observations)
are never overwritten.
- **gotcha / bug observations** — field knowledge accumulates on pages
without being clobbered by regeneration; bugs link their tracking issues.
## Frontmatter
`type: manpage` plus `section` makes the manual queryable like `man -k`:
by section, by `tool`, by `command`, or by missing/stale `verified` stamps.
Validation is `warn`, never blocking.
+1 -1
View File
@@ -156,7 +156,7 @@ git ls-files skills/ | grep -q memory- && echo "source repo - skip install"
Otherwise, run from the project root:
```
npx skills add basicmachines-co/basic-memory --path skills
npx skills add basicmachines-co/basic-memory/skills
```
This installs the canonical `memory-*` skills into the user's skills directory — the
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "codex",
"version": "0.1.0+codex.20260604201213",
"version": "0.22.1",
"description": "A Codex-native bridge to Basic Memory for durable engineering context, decisions, and resumable checkpoints.",
"author": {
"name": "Basic Machines",
+12
View File
@@ -48,8 +48,13 @@ dependencies = [
"fastembed>=0.7.4",
"sqlite-vec>=0.1.6",
"openai>=1.100.2",
"litellm>=1.60.0,<2.0.0",
"logfire>=4.19.0",
"psutil>=5.9.0",
# uvloop's C event loop has no self._ready.popleft() codepath, so the
# asyncpg engine-dispose race ("IndexError: pop from an empty deque") that
# crashes the Postgres backend cannot fire under it. Not available on Windows.
"uvloop>=0.21.0; sys_platform != 'win32'",
]
[project.urls]
@@ -71,6 +76,10 @@ addopts = "--cov=basic_memory --cov-report term-missing"
testpaths = ["tests", "test-int"]
asyncio_mode = "strict"
asyncio_default_fixture_loop_scope = "function"
# Any test hanging >120s fails with a stack dump instead of stalling the CI job
# until the runner times out (the FastMCP/asyncpg cleanup-hang family).
timeout = 120
timeout_method = "thread"
filterwarnings = [
"ignore:The @wait_container_is_ready decorator is deprecated.*:DeprecationWarning:testcontainers\\.core\\.waiting_utils",
"ignore:The default datetime adapter is deprecated as of Python 3\\.12.*:DeprecationWarning:aiosqlite\\.core",
@@ -84,6 +93,7 @@ markers = [
"windows: Windows-specific tests (deselect with '-m \"not windows\"')",
"smoke: Fast end-to-end smoke tests for MCP flows",
"semantic: Tests requiring semantic dependencies (fastembed, sqlite-vec, openai)",
"live: Tests that call external provider APIs and require explicit opt-in",
]
[tool.ruff]
@@ -109,6 +119,8 @@ dev = [
"ty>=0.0.18",
"cst-lsp>=0.1.3",
"libcst>=1.8.6",
"pytest-timeout>=2.4.0",
"pytest-split>=0.11.0",
]
[tool.hatch.version]
+225
View File
@@ -0,0 +1,225 @@
#!/usr/bin/env python3
"""Seed and refresh shared pytest-testmon data for Git worktrees."""
from __future__ import annotations
import argparse
import os
import shutil
import subprocess
import sys
import tempfile
from pathlib import Path
from typing import NamedTuple
TESTMON_FILENAMES = (".testmondata", ".testmondata-shm", ".testmondata-wal")
TESTMON_CACHE_ENV = "BM_TESTMON_CACHE_DIR"
class TestmonCacheResult(NamedTuple):
status: str
source_dir: Path
destination_dir: Path
copied: tuple[Path, ...]
def _run_git(args: list[str], cwd: Path) -> str:
return subprocess.check_output(["git", *args], cwd=cwd, text=True).strip()
def resolve_repo_root(repo_root: Path | None = None) -> Path:
if repo_root is not None:
return repo_root.expanduser().resolve()
return Path(_run_git(["rev-parse", "--show-toplevel"], Path.cwd())).resolve()
def resolve_cache_dir(repo_root: Path, cache_dir: Path | None = None) -> Path:
if cache_dir is not None:
return cache_dir.expanduser().resolve()
if env_cache_dir := os.environ.get(TESTMON_CACHE_ENV):
return Path(env_cache_dir).expanduser().resolve()
git_common_dir = Path(_run_git(["rev-parse", "--git-common-dir"], repo_root))
if not git_common_dir.is_absolute():
git_common_dir = repo_root / git_common_dir
return git_common_dir.resolve() / "testmon-cache" / "main"
def _testmon_datafile(directory: Path) -> Path:
return directory / ".testmondata"
def _testmon_files(directory: Path) -> list[Path]:
return [
directory / filename for filename in TESTMON_FILENAMES if (directory / filename).is_file()
]
def _remove_path(path: Path) -> None:
if path.is_dir():
shutil.rmtree(path)
elif path.exists():
path.unlink()
def _copy_testmon_files(source_dir: Path, destination_dir: Path) -> tuple[Path, ...]:
destination_dir.mkdir(parents=True, exist_ok=True)
copied: list[Path] = []
for source in _testmon_files(source_dir):
destination = destination_dir / source.name
shutil.copy2(source, destination)
copied.append(destination)
return tuple(copied)
def seed_testmon_data(repo_root: Path, cache_dir: Path) -> TestmonCacheResult:
local_datafile = _testmon_datafile(repo_root)
shared_datafile = _testmon_datafile(cache_dir)
if local_datafile.exists():
return TestmonCacheResult(
status="exists",
source_dir=cache_dir,
destination_dir=repo_root,
copied=(),
)
if not shared_datafile.exists():
return TestmonCacheResult(
status="missing",
source_dir=cache_dir,
destination_dir=repo_root,
copied=(),
)
# A worktree with sidecars but no main database is stale; replace the set
# together so SQLite never sees a mixed local/cache snapshot.
for filename in TESTMON_FILENAMES:
_remove_path(repo_root / filename)
copied = _copy_testmon_files(cache_dir, repo_root)
return TestmonCacheResult(
status="seeded",
source_dir=cache_dir,
destination_dir=repo_root,
copied=copied,
)
def refresh_testmon_data(repo_root: Path, cache_dir: Path) -> TestmonCacheResult:
local_datafile = _testmon_datafile(repo_root)
if not local_datafile.exists():
raise FileNotFoundError(
f"No local pytest-testmon data at {local_datafile}; run tests first."
)
cache_parent = cache_dir.parent
cache_parent.mkdir(parents=True, exist_ok=True)
temp_dir = Path(tempfile.mkdtemp(prefix=f".{cache_dir.name}.", dir=cache_parent))
backup_dir = cache_parent / f".{cache_dir.name}.previous-{os.getpid()}"
copied: tuple[Path, ...] = ()
try:
copied = _copy_testmon_files(repo_root, temp_dir)
_remove_path(backup_dir)
if cache_dir.exists():
cache_dir.rename(backup_dir)
try:
temp_dir.rename(cache_dir)
except Exception:
if backup_dir.exists() and not cache_dir.exists():
backup_dir.rename(cache_dir)
raise
finally:
_remove_path(temp_dir)
_remove_path(backup_dir)
return TestmonCacheResult(
status="refreshed",
source_dir=repo_root,
destination_dir=cache_dir,
copied=tuple(cache_dir / path.name for path in copied),
)
def _print_seed_result(result: TestmonCacheResult) -> None:
if result.status == "seeded":
print(f"Seeded pytest-testmon data from {result.source_dir} into {result.destination_dir}")
elif result.status == "exists":
print(f"Local pytest-testmon data already exists at {result.destination_dir}")
elif result.status == "missing":
print(
f"No shared pytest-testmon baseline at {result.source_dir}; "
"run `just testmon-refresh` after a full backend test run to create one."
)
else:
raise ValueError(f"Unexpected seed result: {result.status}")
def _print_refresh_result(result: TestmonCacheResult) -> None:
print(f"Published pytest-testmon data from {result.source_dir} to {result.destination_dir}")
def _print_status(repo_root: Path, cache_dir: Path) -> None:
print(f"Repo root: {repo_root}")
print(f"Worktree data: {_testmon_datafile(repo_root)}")
print(f"Shared cache: {_testmon_datafile(cache_dir)}")
print(f"Worktree ready: {_testmon_datafile(repo_root).exists()}")
print(f"Cache ready: {_testmon_datafile(cache_dir).exists()}")
def build_parser() -> argparse.ArgumentParser:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument(
"--repo-root",
type=Path,
help="Repository root to operate on (default: git rev-parse --show-toplevel)",
)
parser.add_argument(
"--cache-dir",
type=Path,
help=(
"Shared testmon cache directory "
f"(default: ${TESTMON_CACHE_ENV} or <git-common-dir>/testmon-cache/main)"
),
)
subparsers = parser.add_subparsers(dest="command", required=True)
subparsers.add_parser("seed", help="Copy shared testmon data into this worktree if missing")
subparsers.add_parser("refresh", help="Publish this worktree's testmon data to the cache")
subparsers.add_parser("status", help="Show local and shared testmon data paths")
return parser
def main(argv: list[str] | None = None) -> int:
parser = build_parser()
args = parser.parse_args(argv)
repo_root = resolve_repo_root(args.repo_root)
cache_dir = resolve_cache_dir(repo_root, args.cache_dir)
if args.command == "seed":
_print_seed_result(seed_testmon_data(repo_root=repo_root, cache_dir=cache_dir))
return 0
if args.command == "refresh":
_print_refresh_result(refresh_testmon_data(repo_root=repo_root, cache_dir=cache_dir))
return 0
if args.command == "status":
_print_status(repo_root=repo_root, cache_dir=cache_dir)
return 0
parser.error(f"Unknown command: {args.command}")
return 2
if __name__ == "__main__":
sys.exit(main())
+7 -2
View File
@@ -99,7 +99,7 @@ def set_package_version(data: dict[str, Any], version: str) -> None:
# Version scopes. The two groups map to the two distribution tracks:
# core — the Python package and its MCP registry manifest
# packages — the host-native agent artifacts (Claude Code plugin + marketplaces,
# Hermes, OpenClaw). These are the "plugin/agent artifacts."
# Codex plugin, Hermes, OpenClaw). These are the "plugin/agent artifacts."
# `all` writes both. Lockstep releases use `all`; targeted fixes can use one group.
SCOPES = ("all", "core", "packages")
@@ -134,6 +134,11 @@ def _update_packages(version: str, *, dry_run: bool) -> None:
lambda data: set_claude_marketplace_version(data, version),
dry_run=dry_run,
)
update_json(
"plugins/codex/.codex-plugin/plugin.json",
lambda data: set_package_version(data, npm_package_version(version)),
dry_run=dry_run,
)
update_text(
"integrations/hermes/plugin.yaml",
r"^version:\s*.*$",
@@ -174,7 +179,7 @@ def main() -> None:
choices=SCOPES,
default="all",
help="Which artifacts to update: all (default), core (Python + server.json), "
"or packages (Claude Code plugin, marketplaces, Hermes, OpenClaw)",
"or packages (Claude Code plugin, Codex plugin, marketplaces, Hermes, OpenClaw)",
)
parser.add_argument("--dry-run", action="store_true", help="Preview changes without writing")
args = parser.parse_args()
+29 -3
View File
@@ -4,9 +4,31 @@
from __future__ import annotations
import argparse
import re
from pathlib import Path
PLAIN_SCALAR_MAPPING_VALUE = re.compile(r":(?:\s|$)")
def strip_matching_quotes(value: str) -> str:
if len(value) >= 2 and value[0] == value[-1] and value[0] in {'"', "'"}:
return value[1:-1]
return value
def validate_plain_scalar(path: Path, line_number: int, key: str, value: str) -> None:
"""Catch invalid plain-scalar YAML that Codex rejects while loading skills."""
stripped = value.strip()
if not stripped or stripped[0] in {'"', "'"} or stripped in {"|", ">", "|-", ">-", "|+", ">+"}:
return
if PLAIN_SCALAR_MAPPING_VALUE.search(stripped):
raise SystemExit(
f"{path}:{line_number}: invalid YAML frontmatter for {key!r}: "
"unquoted ':' followed by whitespace; quote the value"
)
def parse_frontmatter(path: Path) -> dict[str, str]:
"""Extract top-level frontmatter keys from a Markdown file.
@@ -15,14 +37,15 @@ def parse_frontmatter(path: Path) -> dict[str, str]:
skipped, so nested blocks (a schema note's `schema:`/`settings:` children) can't
overwrite a top-level key like `type` or `entity` via last-write-wins. It does
not interpret block scalars or multi-line values; callers rely on single-line
top-level fields (name, description, type, entity).
top-level fields (name, description, type, entity). Keep the Codex-facing YAML
guard here dependency-free so package checks work under bare `python3`.
"""
lines = path.read_text().splitlines()
if not lines or lines[0] != "---":
raise SystemExit(f"{path}: missing YAML frontmatter")
frontmatter: dict[str, str] = {}
for line in lines[1:]:
for line_number, line in enumerate(lines[1:], start=2):
if line == "---":
break
if line[:1] in (" ", "\t"): # nested key — not a top-level field
@@ -30,7 +53,10 @@ def parse_frontmatter(path: Path) -> dict[str, str]:
if ":" not in line:
continue
key, value = line.split(":", 1)
frontmatter[key.strip()] = value.strip().strip('"')
key = key.strip()
value = value.strip()
validate_plain_scalar(path, line_number, key, value)
frontmatter[key] = strip_matching_quotes(value)
else:
raise SystemExit(f"{path}: unclosed YAML frontmatter")
+2 -2
View File
@@ -6,12 +6,12 @@
"url": "https://github.com/basicmachines-co/basic-memory.git",
"source": "github"
},
"version": "0.21.6",
"version": "0.22.1",
"packages": [
{
"registryType": "pypi",
"identifier": "basic-memory",
"version": "0.21.6",
"version": "0.22.1",
"runtimeHint": "uvx",
"runtimeArguments": [
{
+3 -3
View File
@@ -54,13 +54,13 @@ Users can install or update skills with the [Skills CLI](https://github.com/verc
```bash
# Install all skills
npx skills add basicmachines-co/basic-memory --path skills
npx skills add basicmachines-co/basic-memory/skills
# Install a specific skill
npx skills add basicmachines-co/basic-memory --path skills --skill memory-tasks
npx skills add basicmachines-co/basic-memory/skills --skill memory-tasks
# Install for a specific agent
npx skills add basicmachines-co/basic-memory --path skills --agent claude
npx skills add basicmachines-co/basic-memory/skills --agent claude
```
## Adding a New Skill
+5 -5
View File
@@ -47,16 +47,16 @@ Install or update skills using the [Skills CLI](https://github.com/vercel-labs/s
```bash
# Install all skills
npx skills add basicmachines-co/basic-memory --path skills
npx skills add basicmachines-co/basic-memory/skills
# Install a specific skill
npx skills add basicmachines-co/basic-memory --path skills --skill memory-tasks
npx skills add basicmachines-co/basic-memory/skills --skill memory-tasks
# Install all skills for a specific agent
npx skills add basicmachines-co/basic-memory --path skills --agent claude
npx skills add basicmachines-co/basic-memory/skills --agent claude
# List available skills without installing
npx skills add basicmachines-co/basic-memory --path skills --list
npx skills add basicmachines-co/basic-memory/skills --list
# Check for updates
npx skills check
@@ -67,7 +67,7 @@ npx skills update
Skills are installed to your agent's skills directory (e.g., `~/.claude/skills/` for Claude Code global, or `.claude/skills/` for project-scoped).
If your installed Skills CLI does not support `--path`, copy the `memory-*` directories manually for now. Phase 2 will add a first-class Codex/package install path.
If your installed Skills CLI cannot load `basicmachines-co/basic-memory/skills`, update the CLI or copy the `memory-*` directories manually.
### Claude Desktop (claude.ai)
+1 -1
View File
@@ -1,7 +1,7 @@
"""basic-memory - Local-first knowledge management combining Zettelkasten with knowledge graphs"""
# Package version - updated by release automation
__version__ = "0.21.6"
__version__ = "0.22.1"
# API version for FastAPI - independent of package version
__api_version__ = "v0"
+21 -4
View File
@@ -2,8 +2,11 @@
import asyncio
import os
from contextlib import suppress
from logging.config import fileConfig
from loguru import logger
# Allow nested event loops (needed for pytest-asyncio and other async contexts)
# Note: nest_asyncio doesn't work with uvloop or Python 3.14+, so we handle those cases separately
import sys
@@ -13,9 +16,15 @@ if sys.version_info < (3, 14):
import nest_asyncio
nest_asyncio.apply()
except (ImportError, ValueError):
# nest_asyncio not available or can't patch this loop type (e.g., uvloop)
pass
except (ImportError, ValueError) as exc:
# Trigger: nest_asyncio is absent (ImportError) or refuses to patch the
# running loop (ValueError, e.g. the uvloop policy installed for the
# Postgres backend - #831/#877).
# Why: the uvloop ValueError is now an *expected* path on every Postgres
# startup, so swallowing it silently hides a routine branch.
# Outcome: log at DEBUG (observable, not noisy) and fall through to the
# thread-based migration fallback.
logger.debug(f"nest_asyncio not applied ({exc!r}); using thread-based migration fallback")
# For Python 3.14+, we rely on the thread-based fallback in run_migrations_online()
from sqlalchemy import engine_from_config, pool
@@ -115,7 +124,15 @@ async def run_async_migrations(connectable):
"""Run migrations asynchronously with AsyncEngine."""
async with connectable.connect() as connection:
await connection.run_sync(do_run_migrations)
await connectable.dispose()
# Trigger: startup migrations on the asyncpg backend dispose the engine
# while the event loop may be tearing down around them.
# Why: that race surfaces "IndexError: pop from an empty deque" from
# base_events._run_once (#831/#877); shielding lets dispose finish atomically
# and suppressing CancelledError keeps a cancelled teardown from re-raising it.
# Outcome: the migration engine always disposes cleanly. (uvloop is the
# structural fix for the race; this hardens the teardown path.)
with suppress(asyncio.CancelledError):
await asyncio.shield(connectable.dispose())
def _run_async_migrations_with_asyncio_run(connectable) -> None:
@@ -10,10 +10,18 @@ Key improvements:
- Simplified caching strategies
"""
import os
import pathlib
from fastapi import APIRouter, HTTPException, Response, Path
from loguru import logger
import logfire
from basic_memory.ignore_utils import (
IGNORED_PATH_REJECTION_DETAIL,
load_gitignore_patterns,
should_ignore_path,
)
from basic_memory.deps import (
EntityServiceV2ExternalDep,
SearchServiceV2ExternalDep,
@@ -24,6 +32,7 @@ from basic_memory.deps import (
EntityRepositoryV2ExternalDep,
RelationRepositoryV2ExternalDep,
ProjectExternalIdPathDep,
SyncServiceV2ExternalDep,
TaskSchedulerDep,
)
from basic_memory.schemas import DeleteEntitiesResponse
@@ -40,8 +49,10 @@ from basic_memory.schemas.v2 import (
MoveDirectoryRequestV2,
DeleteDirectoryRequestV2,
OrphanEntitiesResponse,
SyncFileRequest,
)
from basic_memory.schemas.response import DirectoryMoveResult, DirectoryDeleteResult
from basic_memory.utils import validate_project_path
router = APIRouter(prefix="/knowledge", tags=["knowledge-v2"])
@@ -236,6 +247,187 @@ async def resolve_identifier(
return result
## Single-file sync endpoint
def _canonical_file_path(home: pathlib.Path, segments: list[str]) -> str | None:
"""Resolve the actual on-disk casing of a file path under the project home.
Trigger: case-insensitive filesystems (macOS/Windows) pass existence checks for
wrong-cased paths like 'notes/Disk-Note.md' when the file is 'notes/disk-note.md'.
Why: indexing the caller-supplied casing misses the existing DB row keyed by the
on-disk path and inserts a duplicate entity under the wrong-cased path.
Outcome: each segment is matched against real directory entries exact name first
(so distinct case-variant files on case-sensitive filesystems stay distinct),
then a unique case-insensitive match. Returns None when any segment cannot be
matched to exactly one entry, including missing files. Traversal stops at the
project boundary: a directory whose resolved path escapes the project home is
never scanned.
"""
resolved_home = home.resolve()
current = home
canonical_segments: list[str] = []
for segment in segments:
# Trigger: a previously matched segment may be a symlink whose target lies
# outside the project root (e.g. wrong-cased 'LINK' matched the on-disk
# 'link' -> /tmp/outside on a case-sensitive filesystem).
# Why: os.scandir follows symlinked directories, so continuing would read
# directory contents outside the project boundary even though the
# post-canonicalization containment check rejects the request later.
# Outcome: bail before scanning the moment resolution escapes the home.
if not current.resolve().is_relative_to(resolved_home):
return None
try:
with os.scandir(current) as entries_iter:
entries = [entry.name for entry in entries_iter]
except OSError:
# A parent segment resolved to a non-directory (or vanished): no canonical
# path exists for the remaining segments.
return None
if segment in entries:
matched = segment
else:
matches = [entry for entry in entries if entry.lower() == segment.lower()]
if len(matches) != 1:
return None
matched = matches[0]
canonical_segments.append(matched)
current = current / matched
return "/".join(canonical_segments)
@router.post("/sync-file", response_model=EntityResponseV2)
async def sync_file(
data: SyncFileRequest,
project_id: ProjectExternalIdPathDep,
sync_service: SyncServiceV2ExternalDep,
project_config: ProjectConfigV2ExternalDep,
search_service: SearchServiceV2ExternalDep,
app_config: AppConfigDep,
) -> EntityResponseV2:
"""Index a single markdown file that exists on disk but is not indexed yet.
Recovery path for files written directly to disk before the watcher indexed
them (#581): callers such as edit_note can index the exact file and retry
identifier resolution without running a full project sync.
Args:
data: Request containing the markdown file path relative to project root
Returns:
The indexed entity
Raises:
HTTPException: 400 if the path escapes the project root, contains
non-normalized segments, matches the project ignore rules, or is
not markdown, 404 if the file does not exist on disk
"""
with logfire.span(
"api.request.knowledge.sync_file",
entrypoint="api",
domain="knowledge",
action="sync_file",
):
logger.info(f"API v2 request: sync_file file_path='{data.file_path}'")
if not validate_project_path(data.file_path, project_config.home):
raise HTTPException(
status_code=400,
detail=f"File path '{data.file_path}' is not allowed - "
"paths must stay within project boundaries",
)
# Trigger: segments like './' or '//' survive the traversal check above
# Why: a non-normalized path would index under a non-canonical DB key
# Outcome: reject fail-fast instead of guessing the canonical form
segments = data.file_path.replace("\\", "/").split("/")
if any(segment in ("", ".") for segment in segments):
raise HTTPException(
status_code=400,
detail=f"File path '{data.file_path}' is not normalized - "
"segments like './' or '//' are not allowed",
)
# Canonicalize to the actual on-disk casing so the DB lookup below hits the
# row keyed by the real path instead of inserting a wrong-cased duplicate.
file_path = _canonical_file_path(project_config.home, segments)
if file_path is None:
raise HTTPException(
status_code=404, detail=f"File not found on disk: '{data.file_path}'"
)
# Trigger: canonicalization rewrote a segment to its on-disk form, and that
# segment may be a symlink. The pre-check above validated the ORIGINAL
# request path — on a case-sensitive filesystem 'LINK/secret.md' does not
# exist, so resolve() cannot follow the real 'link' symlink and the check
# passes even when 'link' points outside the project root.
# Why: indexing through an escaping symlink would read and index content
# outside the project boundary — and even an is_file() existence probe on
# the joined path would follow the symlink and stat its target, so
# containment must hold BEFORE any filesystem probe that follows symlinks.
# Path.resolve() only walks symlink names (readlink); it never opens or
# stats the final target, so it is safe to run pre-containment.
# Outcome: the canonical path is re-validated and the fully-resolved absolute
# target must stay inside the resolved project home; escapes get a 400
# before the file-existence probe below ever touches the target.
resolved_target = (project_config.home / file_path).resolve()
if not validate_project_path(file_path, project_config.home) or not (
resolved_target.is_relative_to(project_config.home.resolve())
):
raise HTTPException(
status_code=400,
detail=f"File path '{data.file_path}' is not allowed - "
"paths must stay within project boundaries",
)
# Containment holds, so probing the resolved target cannot leave the project.
if not resolved_target.is_file():
raise HTTPException(
status_code=404, detail=f"File not found on disk: '{data.file_path}'"
)
# Trigger: the canonical path matches the .bmignore / project .gitignore rules
# Why: scan and watch flows filter ignored files before they ever reach the
# indexer; indexing one here would bypass the ignored-file contract and
# make hidden or gitignored content searchable
# Outcome: the same should_ignore_path() rules apply to single-file sync
ignore_patterns = load_gitignore_patterns(project_config.home)
if should_ignore_path(
project_config.home / file_path, project_config.home, ignore_patterns
):
raise HTTPException(
status_code=400,
detail=f"File path '{data.file_path}' {IGNORED_PATH_REJECTION_DETAIL} "
"and cannot be indexed",
)
if not sync_service.file_service.is_markdown(file_path):
raise HTTPException(
status_code=400,
detail=f"Only markdown files can be indexed: '{data.file_path}'",
)
# Trigger: the file may already have a DB row (e.g. modified on disk after indexing)
# Why: the indexer needs to know whether to insert or update the entity
# Outcome: new is computed from the database instead of assumed by the caller
existing = await sync_service.entity_repository.get_by_file_path(file_path)
synced = await sync_service.sync_one_markdown_file(
file_path, new=existing is None, index_search=True
)
# Trigger: semantic search is enabled and the entity index was just refreshed
# Why: the project sync flow awaits sync_entity_vectors_batch() inline after
# indexing changed files (SyncService.sync); without the single-entity
# equivalent, a note recovered via sync-file stays missing or stale in
# semantic search until a later edit or full project sync
# Outcome: vectors refresh synchronously before the response returns,
# mirroring the sync flow instead of the out-of-band scheduler
if app_config.semantic_search_enabled:
await search_service.sync_entity_vectors_batch([synced.entity.id])
result = EntityResponseV2.model_validate(synced.entity)
logger.info(
f"API v2 response: sync_file file_path='{file_path}' external_id={result.external_id}"
)
return result
## Read endpoints
@@ -193,7 +193,7 @@ async def add_project(
return ProjectStatusResponse( # pyright: ignore [reportCallIssue]
message=f"Project '{new_project.name}' added successfully",
status="success",
default=project_data.set_default,
default=new_project.is_default or False,
new_project=ProjectItem(
id=new_project.id,
external_id=new_project.external_id,
@@ -320,7 +320,20 @@ async def resolve_project_identifier(
)
if not project:
raise HTTPException(status_code=404, detail=f"Project not found: '{data.identifier}'")
detail = f"Project not found: '{data.identifier}'"
# Trigger: resolution missed and the projects table is empty.
# Why: a fresh install bootstraps config.json's default project before any
# reconciliation has created database rows (the one-shot CLI never runs
# the server lifespan), so the first read fails on the configured
# default and the bare not-found message reads as a broken install
# rather than a missing first-run step (#974 follow-up).
# Outcome: the error names the setup command instead.
if not await project_repository.find_all(limit=1, use_load_options=False):
detail = (
f"{detail}. No projects are set up yet — run "
"'basic-memory project add <name> <path>' to create one."
)
raise HTTPException(status_code=404, detail=detail)
return ProjectResolveResponse(
external_id=project.external_id,
@@ -559,12 +572,10 @@ async def set_default_project_by_id(
logger.info(f"API v2 request: set_default_project_by_id for project_id={project_id}")
try:
# Get the old default project from database
# Get the old default project from database. It may be absent during
# bootstrap/recovery (no default row yet); that is a valid state, not an
# error, so we only echo it back when one exists.
default_project = await project_repository.get_default_project()
if not default_project:
raise HTTPException( # pragma: no cover
status_code=404, detail="No default project is currently set"
)
# Get the new default project by external_id
new_default_project = await project_repository.get_by_external_id(project_id)
@@ -576,17 +587,27 @@ async def set_default_project_by_id(
# Set as default using project name (service layer still uses names internally)
await project_service.set_default_project(new_default_project.name)
return ProjectStatusResponse(
message=f"Project '{new_default_project.name}' set as default successfully",
status="success",
default=True,
old_project=ProjectItem(
# Trigger: a previous default existed
# Why: ProjectStatusResponse.old_project is Optional; the no-default
# bootstrap case must succeed with old_project=None
# Outcome: response echoes the prior default only when there was one
old_project = (
ProjectItem(
id=default_project.id,
external_id=default_project.external_id,
name=default_project.name,
path=default_project.path,
is_default=False,
),
)
if default_project
else None
)
return ProjectStatusResponse(
message=f"Project '{new_default_project.name}' set as default successfully",
status="success",
default=True,
old_project=old_project,
new_project=ProjectItem(
id=new_default_project.id,
external_id=new_default_project.external_id,
@@ -64,7 +64,9 @@ async def search(
or query.permalink
or query.permalink_match
),
has_filters=bool(query.note_types or query.entity_types or query.metadata_filters),
has_filters=bool(
query.note_types or query.entity_types or query.categories or query.metadata_filters
),
):
offset = (page - 1) * page_size
exact_count_available = query.retrieval_mode == SearchRetrievalMode.FTS
+5 -2
View File
@@ -18,7 +18,9 @@ from basic_memory.services.context_service import (
class EntityBatchLookup(Protocol):
async def find_by_ids_for_hydration(self, ids: List[int]) -> Sequence[Any]: ...
async def find_by_ids_for_hydration(
self, ids: List[int], *, include_cross_project: bool = False
) -> Sequence[Any]: ...
class EntityServiceBatchLookup(Protocol):
@@ -88,7 +90,7 @@ async def to_graph_context(
result_count=len(entity_ids_needed),
):
entities = await entity_repository.find_by_ids_for_hydration(
list(entity_ids_needed)
list(entity_ids_needed), include_cross_project=True
)
for e in entities:
entity_title_lookup[e.id] = e.title
@@ -145,6 +147,7 @@ async def to_graph_context(
from_entity_id=item.from_id,
from_entity_external_id=from_ext_id,
to_entity=to_title,
to_name=item.to_name,
to_entity_id=item.to_id,
to_entity_external_id=to_ext_id,
created_at=item.created_at,
+12
View File
@@ -57,6 +57,14 @@ def app_callback(
container = CliContainer.create()
set_container(container)
# Trigger: Postgres backend resolved at CLI startup, before any asyncio.run().
# Why: uvloop must own the event-loop policy before the loop is created so the
# asyncpg engine-dispose race (#831/#877) cannot fire. No-op for SQLite.
# Outcome: subsequent asyncio.run() calls in CLI commands use uvloop on Postgres.
from basic_memory.db import maybe_install_uvloop
maybe_install_uvloop(container.config)
# Trigger: first-run init confirmation before command output.
# Why: informational "initialized" message belongs above command results, not in the upsell panel.
# Outcome: one-time plain line printed before the subcommand runs.
@@ -75,8 +83,11 @@ def app_callback(
# Skip for 'mcp' command - it has its own lifespan that handles initialization
# Skip for API-using commands (status, sync, etc.) - they handle initialization via deps.py
# Skip for 'reset' command - it manages its own database lifecycle
# Skip for 'man' - it only copies packaged files; a broken local database
# must not block installing the offline docs
skip_init_commands = {
"doctor",
"man",
"mcp",
"status",
"sync",
@@ -86,6 +97,7 @@ def app_callback(
"reindex",
"update",
"watch",
"workspace",
}
if (
not version
+15
View File
@@ -168,6 +168,20 @@ def _manual_update_hint(source: InstallSource) -> str:
)
def _preload_lazy_console_modules() -> None:
"""Import modules the post-upgrade output path defers until print time.
Trigger: an in-place upgrade is about to replace this installation on disk.
Why: rich and typer defer some imports until print/excepthook time; once
`brew upgrade` / `uv tool upgrade` removes the running version's files,
those imports raise ModuleNotFoundError and the final status message
crashes the exiting process.
Outcome: import the deferred modules now, while the files still exist.
"""
import rich._emoji_codes # noqa: F401
import typer.rich_utils # noqa: F401
def _save_last_checked_timestamp(config_manager: ConfigManager, checked_at: datetime) -> None:
"""Persist the timestamp for the most recent attempted update check."""
config = config_manager.load_config()
@@ -288,6 +302,7 @@ def run_auto_update(
else BREW_UPGRADE_TIMEOUT_SECONDS
)
_preload_lazy_console_modules()
install_result = _run_subprocess(
command,
timeout_seconds=timeout,
@@ -4,11 +4,13 @@ from . import ci, status, db, doctor, import_memory_json, mcp, import_claude_con
from . import (
import_claude_projects,
import_chatgpt,
man,
tool,
project,
format,
schema,
update,
workspace,
)
__all__ = [
@@ -27,4 +29,6 @@ __all__ = [
"format",
"schema",
"update",
"workspace",
"man",
]
+12 -2
View File
@@ -34,8 +34,10 @@ from basic_memory.ci.project_updates import (
from basic_memory.cli.app import app
from basic_memory.cli.commands.command_utils import run_with_cleanup
from basic_memory.cli.commands.routing import force_routing, validate_routing_flags
from basic_memory.mcp.tools import search_notes as mcp_search_notes
from basic_memory.mcp.tools import write_note as mcp_write_note
# MCP tool functions are imported inside the async helpers below: importing
# basic_memory.mcp.tools loads the entire tool stack (fastmcp, mcp SDK,
# SQLAlchemy), which would slow every CLI invocation, including --help (#886).
console = Console()
@@ -252,6 +254,10 @@ async def seed_project_update_schemas(
refresh: bool = False,
) -> list[str]:
"""Seed Auto BM schema notes without overwriting customized schemas."""
# Deferred: loading the MCP tool stack at module import slows CLI startup (#886).
from basic_memory.mcp.tools import search_notes as mcp_search_notes
from basic_memory.mcp.tools import write_note as mcp_write_note
seeded: list[str] = []
routed_project = _routed_project(project=project, project_id=project_id, workspace=workspace)
for spec in schema_seed_specs():
@@ -295,6 +301,10 @@ async def publish_project_update_note(
note: ProjectUpdateNote,
) -> dict[str, Any]:
"""Search by idempotency key and then upsert the deterministic note path."""
# Deferred: loading the MCP tool stack at module import slows CLI startup (#886).
from basic_memory.mcp.tools import search_notes as mcp_search_notes
from basic_memory.mcp.tools import write_note as mcp_write_note
routed_project = _routed_project(
project=config.project,
project_id=config.project_id,
@@ -32,14 +32,33 @@ def _rclone_exclude_filters(pattern: str) -> list[str]:
return [f"- {path_pattern}", f"- {path_pattern}/**"]
async def get_mount_info() -> TenantMountInfo:
"""Get current tenant information from cloud API."""
def _workspace_id_header(workspace_id: str | None) -> dict[str, str]:
"""Header that routes a /tenant/mount/* request to a specific tenant.
The mount endpoints resolve the workspace from X-Workspace-ID (validating
membership + subscription) and fall back to the user's default tenant when
it is absent so omitting it preserves the original default-tenant behavior.
"""
return {"X-Workspace-ID": workspace_id} if workspace_id else {}
async def get_mount_info(*, workspace_id: str | None = None) -> TenantMountInfo:
"""Get tenant mount info (bucket name + tenant id) from the cloud API.
Args:
workspace_id: Tenant id of the target workspace. When omitted, the API
uses the authenticated user's default tenant.
"""
try:
config_manager = ConfigManager()
config = config_manager.config
host_url = config.cloud_host.rstrip("/")
response = await make_api_request(method="GET", url=f"{host_url}/tenant/mount/info")
response = await make_api_request(
method="GET",
url=f"{host_url}/tenant/mount/info",
headers=_workspace_id_header(workspace_id),
)
return TenantMountInfo.model_validate(response.json())
except Exception as e:
@@ -47,13 +66,24 @@ async def get_mount_info() -> TenantMountInfo:
async def generate_mount_credentials(tenant_id: str) -> MountCredentials:
"""Generate scoped credentials for syncing."""
"""Generate scoped S3 credentials for syncing a specific tenant's bucket.
Args:
tenant_id: Tenant id whose bucket-scoped credentials to mint. Routed via
X-Workspace-ID so team workspaces get their own bucket's credentials.
"""
try:
config_manager = ConfigManager()
config = config_manager.config
host_url = config.cloud_host.rstrip("/")
response = await make_api_request(method="POST", url=f"{host_url}/tenant/mount/credentials")
# The mount endpoints resolve X-Workspace-ID by matching the workspace's
# tenant_id, so passing a tenant_id here is the correct routing key.
response = await make_api_request(
method="POST",
url=f"{host_url}/tenant/mount/credentials",
headers=_workspace_id_header(tenant_id),
)
return MountCredentials.model_validate(response.json())
except Exception as e:
@@ -26,15 +26,52 @@ from basic_memory.cli.commands.cloud.bisync_commands import (
generate_mount_credentials,
get_mount_info,
)
from basic_memory.cli.commands.cloud.rclone_config import configure_rclone_remote
from basic_memory.cli.commands.cloud.rclone_config import (
configure_rclone_remote,
rclone_remote_exists,
remote_name_for_workspace,
)
from basic_memory.cli.commands.cloud.rclone_installer import (
RcloneInstallError,
install_rclone,
)
from basic_memory.mcp.project_context import get_available_workspaces
from basic_memory.schemas.cloud import (
WorkspaceInfo,
format_workspace_choices,
format_workspace_selection_choices,
workspace_matches_exact_identifier,
)
console = Console()
def _resolve_setup_workspace(identifier: str) -> WorkspaceInfo:
"""Resolve a workspace identifier (slug, name, or tenant_id) for setup.
Errors with copyable choices when the identifier matches zero or multiple
workspaces, so the user can disambiguate.
"""
workspaces = run_with_cleanup(get_available_workspaces())
if not workspaces:
console.print("[red]No accessible cloud workspaces found for this account[/red]")
raise typer.Exit(1)
matches = [ws for ws in workspaces if workspace_matches_exact_identifier(ws, identifier)]
if len(matches) == 1:
return matches[0]
if not matches:
console.print(f"[red]No workspace matches '{identifier}'[/red]")
console.print("\nAvailable workspaces:")
console.print(format_workspace_choices(workspaces))
else:
console.print(f"[red]'{identifier}' matches multiple workspaces[/red]")
console.print("\nDisambiguate with the workspace slug or tenant_id:")
console.print(format_workspace_selection_choices(matches))
raise typer.Exit(1)
@cloud_app.command()
def login():
"""Authenticate with WorkOS using OAuth Device Authorization flow."""
@@ -164,13 +201,28 @@ def status() -> None:
@cloud_app.command("setup")
def setup() -> None:
def setup(
workspace: str | None = typer.Option(
None,
"--workspace",
help="Set up sync for a specific workspace (slug, name, or tenant_id). "
"Omit for your default workspace.",
),
force: bool = typer.Option(
False,
"--force",
help="Reconfigure an rclone remote that already exists (mints new credentials).",
),
) -> None:
"""Set up cloud sync by installing rclone and configuring credentials.
After setup, use project commands for syncing:
bm project add <name> --cloud --local-path ~/projects/<name>
bm project bisync --name <name> --resync # First time
bm project bisync --name <name> # Subsequent syncs
Run once per workspace you sync. The default workspace uses the
'basic-memory-cloud' remote; other (e.g. Team) workspaces each get their own
tenant-scoped remote, since Tigris credentials are bucket-scoped.
After setup, use the cloud sync commands:
bm cloud pull --name <name> # fetch cloud changes (Team-safe)
bm cloud push --name <name> # upload local changes (Team-safe)
"""
console.print("[bold blue]Basic Memory Cloud Setup[/bold blue]")
console.print("Setting up cloud sync with rclone...\n")
@@ -180,35 +232,60 @@ def setup() -> None:
console.print("[blue]Step 1: Installing rclone...[/blue]")
install_rclone()
# Step 2: Get tenant info
# --- Resolve target workspace ---
# Trigger: --workspace given. Why: Tigris keys are tenant-scoped, so a
# non-default workspace needs its own bucket + remote. Outcome: scope the
# mount-info/credentials calls and name the remote after the workspace.
if workspace is not None:
target = _resolve_setup_workspace(workspace)
workspace_id: str | None = target.tenant_id
remote_name = remote_name_for_workspace(target.slug, is_default=target.is_default)
console.print(f"[dim]Workspace: {target.name} ({target.slug})[/dim]")
else:
workspace_id = None # default tenant
remote_name = remote_name_for_workspace(None, is_default=True)
# Trigger: the target rclone remote already exists.
# Why: re-running setup mints new credentials and overwrites the remote,
# which would silently repoint it — e.g. clobbering the shared
# basic-memory-cloud remote that served another tenant. Checked BEFORE
# minting so an abort wastes no credentials.
# Outcome: stop unless the user explicitly opts in with --force.
if rclone_remote_exists(remote_name) and not force:
console.print(f"[red]rclone remote '{remote_name}' is already configured.[/red]")
console.print(
"Re-running setup mints new credentials and overwrites it. "
"Pass --force to reconfigure."
)
raise typer.Exit(1)
# Step 2: Get tenant info (scoped to the target workspace when given)
console.print("\n[blue]Step 2: Getting tenant information...[/blue]")
tenant_info = run_with_cleanup(get_mount_info())
tenant_info = run_with_cleanup(get_mount_info(workspace_id=workspace_id))
console.print(f"[green]Found tenant: {tenant_info.tenant_id}[/green]")
# Step 3: Generate credentials
# Step 3: Generate credentials for that tenant's bucket
console.print("\n[blue]Step 3: Generating sync credentials...[/blue]")
creds = run_with_cleanup(generate_mount_credentials(tenant_info.tenant_id))
console.print("[green]Generated secure credentials[/green]")
# Step 4: Configure rclone remote
# Step 4: Configure the tenant's rclone remote
console.print("\n[blue]Step 4: Configuring rclone remote...[/blue]")
configure_rclone_remote(
access_key=creds.access_key,
secret_key=creds.secret_key,
remote_name=remote_name,
)
console.print("\n[bold green]Cloud setup completed successfully![/bold green]")
console.print("\n[bold]Next steps:[/bold]")
console.print("1. Add a project with local sync path:")
console.print(" bm project add research --cloud --local-path ~/Documents/research")
console.print("\n Or configure sync for an existing project:")
console.print("1. Configure sync for a project:")
console.print(" bm cloud sync-setup research ~/Documents/research")
console.print("\n2. Preview the initial sync (recommended):")
console.print(" bm project bisync --name research --resync --dry-run")
console.print("\n3. If all looks good, run the actual sync:")
console.print(" bm project bisync --name research --resync")
console.print("\n4. Subsequent syncs (no --resync needed):")
console.print(" bm project bisync --name research")
console.print("\n2. Preview a pull (recommended):")
console.print(" bm cloud pull --name research --dry-run")
console.print("\n3. Fetch cloud changes / upload local changes:")
console.print(" bm cloud pull --name research")
console.print(" bm cloud push --name research")
console.print(
"\n[dim]Tip: Always use --dry-run first to preview changes before syncing[/dim]"
)
@@ -216,6 +293,8 @@ def setup() -> None:
except (RcloneInstallError, BisyncError, CloudAPIError) as e:
console.print(f"\n[red]Setup failed: {e}[/red]")
raise typer.Exit(1)
except typer.Exit:
raise
except Exception as e:
console.print(f"\n[red]Unexpected error during setup: {e}[/red]")
raise typer.Exit(1)
@@ -7,6 +7,7 @@ they are cloud-specific operations.
import os
from datetime import datetime
from enum import Enum
import typer
from rich.console import Console
@@ -16,10 +17,19 @@ from basic_memory.cli.commands.cloud.bisync_commands import get_mount_info
from basic_memory.cli.commands.cloud.rclone_commands import (
RcloneError,
SyncProject,
TransferDirection,
TransferPlan,
get_project_bisync_state,
project_bisync,
project_check,
project_diff,
project_sync,
project_transfer,
)
from basic_memory.cli.commands.cloud.rclone_config import (
DEFAULT_RCLONE_REMOTE,
rclone_remote_exists,
remote_name_for_workspace,
)
from basic_memory.cli.commands.command_utils import run_with_cleanup
from basic_memory.cli.commands.routing import force_routing
@@ -27,7 +37,12 @@ from basic_memory.config import BasicMemoryConfig, ConfigManager, ProjectEntry
from basic_memory.mcp.async_client import get_client
from basic_memory.mcp.clients import ProjectClient
from basic_memory.mcp.project_context import get_available_workspaces
from basic_memory.schemas.cloud import WorkspaceInfo
from basic_memory.schemas.cloud import (
WorkspaceInfo,
format_workspace_choices,
format_workspace_selection_choices,
workspace_matches_exact_identifier,
)
from basic_memory.schemas.project_info import ProjectItem
from basic_memory.utils import generate_permalink, normalize_project_path
@@ -35,9 +50,34 @@ console = Console()
TEAM_WORKSPACE_BISYNC_UNSUPPORTED = (
"The bisync operation is only supported on Personal workspaces.\n"
"Use `bm cloud sync --name {name}` instead."
"Use `bm cloud pull --name {name}` / `bm cloud push --name {name}` instead."
)
TEAM_WORKSPACE_SYNC_UNSUPPORTED = (
"The sync operation mirrors local onto the shared bucket and can delete a "
"teammate's files, so it is only supported on Personal workspaces.\n"
"Use `bm cloud pull --name {name}` (fetch) / `bm cloud push --name {name}` "
"(additive upload) instead."
)
class ConflictStrategy(str, Enum):
"""How push/pull resolves files that differ on both sides.
Default is ``fail``: surface the conflicts and abort before transferring,
leaving the user to re-run with an explicit resolution like git refusing
to clobber local changes.
This is the Typer-facing enum; the engine in ``rclone_commands`` accepts the
same values as a ``ConflictStrategy`` Literal. ``_run_directional_transfer``
bridges the two by passing ``on_conflict.value``. Keep the values in sync.
"""
fail = "fail"
keep_local = "keep-local"
keep_cloud = "keep-cloud"
keep_both = "keep-both"
# --- Shared helpers ---
@@ -59,12 +99,41 @@ def _require_cloud_credentials(config: BasicMemoryConfig) -> None:
raise typer.Exit(1)
async def _get_workspace_for_project(name: str, config: BasicMemoryConfig) -> WorkspaceInfo:
"""Resolve the cloud workspace targeted by a project-scoped sync command."""
async def _get_workspace_for_project(
name: str,
config: BasicMemoryConfig,
*,
workspace_override: str | None = None,
) -> WorkspaceInfo:
"""Resolve the cloud workspace targeted by a project-scoped sync command.
``workspace_override`` (a slug, name, or tenant_id, e.g. from ``--workspace``)
takes precedence over config, letting the user disambiguate a project name
that exists in more than one workspace.
"""
workspaces = await get_available_workspaces()
if not workspaces:
raise ValueError("No accessible cloud workspaces found for this account")
# An explicit override wins over config — this is how the user disambiguates.
if workspace_override is not None:
matches = [
item
for item in workspaces
if workspace_matches_exact_identifier(item, workspace_override)
]
if len(matches) == 1:
return matches[0]
if not matches:
raise ValueError(
f"No accessible workspace matches '{workspace_override}'.\n"
f"{format_workspace_choices(workspaces)}"
)
raise ValueError(
f"'{workspace_override}' matches multiple workspaces; use a slug or tenant_id:\n"
f"{format_workspace_selection_choices(matches)}"
)
entry = config.projects.get(name)
workspace_id = entry.workspace_id if entry and entry.workspace_id else config.default_workspace
if workspace_id:
@@ -92,8 +161,18 @@ async def _get_workspace_for_project(name: str, config: BasicMemoryConfig) -> Wo
)
def _require_personal_workspace(name: str, config: BasicMemoryConfig) -> WorkspaceInfo:
"""Exit before bisync work when the target workspace is not personal."""
def _require_personal_workspace(
name: str,
config: BasicMemoryConfig,
*,
unsupported_message: str = TEAM_WORKSPACE_BISYNC_UNSUPPORTED,
) -> WorkspaceInfo:
"""Exit before mirror work when the target workspace is not personal.
Used to gate the destructive mirror operations (`sync`, `bisync`) to
Personal workspaces. ``unsupported_message`` lets each command point Team
users at the right Team-safe alternative.
"""
try:
workspace = run_with_cleanup(_get_workspace_for_project(name, config))
except Exception as exc:
@@ -101,15 +180,21 @@ def _require_personal_workspace(name: str, config: BasicMemoryConfig) -> Workspa
raise typer.Exit(1)
if workspace.workspace_type != "personal":
console.print(f"[red]{TEAM_WORKSPACE_BISYNC_UNSUPPORTED.format(name=name)}[/red]")
console.print(f"[red]{unsupported_message.format(name=name)}[/red]")
raise typer.Exit(1)
return workspace
async def _get_cloud_project(name: str) -> ProjectItem | None:
"""Fetch a project by name from the cloud API."""
async with get_client(project_name=name) as client:
async def _get_cloud_project(name: str, *, workspace_id: str | None = None) -> ProjectItem | None:
"""Fetch a project by name from the cloud API.
``workspace_id`` routes the lookup to a specific tenant so the project
metadata comes from the same workspace the transfer targets (otherwise
get_client would resolve the workspace from config/default and could read a
different tenant see #920 review).
"""
async with get_client(project_name=name, workspace=workspace_id) as client:
projects_list = await ProjectClient(client).list_projects()
for proj in projects_list.projects:
if generate_permalink(proj.name) == generate_permalink(name):
@@ -118,10 +203,17 @@ async def _get_cloud_project(name: str) -> ProjectItem | None:
def _get_sync_project(
name: str, config: BasicMemoryConfig, project_data: ProjectItem
name: str,
config: BasicMemoryConfig,
project_data: ProjectItem,
*,
remote_name: str = DEFAULT_RCLONE_REMOTE,
) -> tuple[SyncProject, str | None]:
"""Build a SyncProject and resolve local_sync_path from config.
``remote_name`` selects which tenant-scoped rclone remote the project routes
through (default tenant vs a team workspace remote).
Returns (sync_project, local_sync_path). Exits if no local_sync_path configured.
"""
sync_entry = config.projects.get(name)
@@ -137,6 +229,7 @@ def _get_sync_project(
name=project_data.name,
path=normalize_project_path(project_data.path),
local_sync_path=local_sync_path,
remote_name=remote_name,
)
return sync_project, local_sync_path
@@ -146,11 +239,15 @@ def _get_sync_project(
@cloud_app.command("sync")
def sync_project_command(
name: str = typer.Option(..., "--name", help="Project name to sync"),
name: str = typer.Option(..., "--name", "--project", help="Project name to sync"),
dry_run: bool = typer.Option(False, "--dry-run", help="Preview changes without syncing"),
verbose: bool = typer.Option(False, "--verbose", "-v", help="Show detailed output"),
) -> None:
"""One-way sync: local -> cloud (make cloud identical to local).
"""One-way mirror: local -> cloud (make cloud identical to local).
Personal workspaces only. This deletes cloud files not present locally, so
on Team workspaces use `bm cloud push` (additive upload) / `bm cloud pull`
(fetch) instead.
Example:
bm cloud sync --name research
@@ -158,9 +255,13 @@ def sync_project_command(
"""
config = ConfigManager().config
_require_cloud_credentials(config)
_require_personal_workspace(name, config, unsupported_message=TEAM_WORKSPACE_SYNC_UNSUPPORTED)
try:
# Get tenant info for bucket name
# Get tenant info for bucket name.
# TODO(#919): scope to the project's workspace like push/pull. Safe for now
# because these mirror commands are gated to the (default-tenant) Personal
# workspace, so the default mount info is correct.
tenant_info = run_with_cleanup(get_mount_info())
bucket_name = tenant_info.bucket_name
@@ -191,14 +292,231 @@ def sync_project_command(
raise typer.Exit(1)
def _print_conflict_abort(name: str, direction: TransferDirection, plan: TransferPlan) -> None:
"""Explain a conflict abort and how to resolve it (git-pull style)."""
console.print(
f"[red]{direction.capitalize()} aborted: {len(plan.conflicts)} file(s) differ between "
f"local and cloud.[/red]"
)
for path in plan.conflicts:
console.print(f" [yellow]*[/yellow] {path}")
console.print("\nRe-run with one of:")
console.print(" [dim]--on-conflict keep-cloud[/dim] take the cloud version")
console.print(" [dim]--on-conflict keep-local[/dim] keep your local version")
console.print(
" [dim]--on-conflict keep-both[/dim] keep both (writes <name>.conflict-<date>)"
)
def _run_directional_transfer(
name: str,
direction: TransferDirection,
*,
on_conflict: ConflictStrategy,
dry_run: bool,
verbose: bool,
workspace: str | None = None,
) -> None:
"""Shared orchestration for `bm cloud push` / `bm cloud pull`.
Detects conflicts first, then aborts (the default) or applies the chosen
resolution. Uses additive `rclone copy`, so it never deletes on the
destination safe for Team workspaces and therefore not gated.
Routes through the resolved workspace's own tenant-scoped rclone remote, so a
Team project reads/writes the right bucket (see #919).
"""
config = ConfigManager().config
_require_cloud_credentials(config)
try:
# --- Resolve the target workspace and its tenant-scoped remote ---
# Tigris credentials are bucket/tenant-scoped, so each workspace has its
# own rclone remote. Resolve which workspace this project belongs to
# (config or --workspace override) before touching any bucket.
try:
target_workspace = run_with_cleanup(
_get_workspace_for_project(name, config, workspace_override=workspace)
)
except Exception as exc:
console.print(f"[red]Error resolving workspace for project '{name}': {exc}[/red]")
raise typer.Exit(1)
remote_name = remote_name_for_workspace(
target_workspace.slug, is_default=target_workspace.is_default
)
# Trigger: the workspace's remote has not been configured yet.
# Why: provisioning mints tenant-scoped credentials and must be explicit
# (no surprise key generation); push/pull only transfer.
# Outcome: stop with the exact setup command for this workspace.
if not rclone_remote_exists(remote_name):
setup_target = (
"" if target_workspace.is_default else f" --workspace {target_workspace.slug}"
)
console.print(f"[red]Workspace '{target_workspace.slug}' is not set up for sync.[/red]")
console.print(f"\nRun: bm cloud setup{setup_target}")
raise typer.Exit(1)
# Get tenant info for bucket name, scoped to the resolved workspace
tenant_info = run_with_cleanup(get_mount_info(workspace_id=target_workspace.tenant_id))
bucket_name = tenant_info.bucket_name
# Get project info from the same workspace we resolved above, so the
# project path and the bucket/remote all refer to one tenant.
with force_routing(cloud=True):
project_data = run_with_cleanup(
_get_cloud_project(name, workspace_id=target_workspace.tenant_id)
)
if not project_data:
console.print(f"[red]Error: Project '{name}' not found[/red]")
raise typer.Exit(1)
sync_project, _ = _get_sync_project(name, config, project_data, remote_name=remote_name)
# --- Detect before transferring ---
plan = project_diff(sync_project, bucket_name, direction)
# Trigger: rclone could not read/hash some files.
# Why: comparing is the whole basis for a safe transfer — never guess.
# Outcome: abort before moving any bytes.
if plan.errors:
console.print(
f"[red]{direction.capitalize()} aborted: rclone could not compare "
f"{len(plan.errors)} file(s)[/red]"
)
for path in plan.errors:
console.print(f" [red]![/red] {path}")
raise typer.Exit(1)
# Trigger: files differ on both sides and the user chose no resolution.
# Why: "no surprises" — never silently pick a winner.
# Outcome: list the conflicts and exit, like git refusing to clobber.
if plan.conflicts and on_conflict is ConflictStrategy.fail:
_print_conflict_abort(name, direction, plan)
raise typer.Exit(1)
# --- Transfer ---
arrow = "cloud -> local" if direction == "pull" else "local -> cloud"
console.print(f"[blue]{direction.capitalize()} {name} ({arrow})...[/blue]")
conflict_suffix = datetime.now().strftime("%Y%m%d-%H%M%S")
success = project_transfer(
sync_project,
bucket_name,
direction,
plan,
strategy=on_conflict.value,
conflict_suffix=conflict_suffix,
dry_run=dry_run,
verbose=verbose,
)
if not success:
console.print(f"[red]{name} {direction} failed[/red]")
raise typer.Exit(1)
console.print(f"[green]{name} {direction} completed successfully[/green]")
# Without a sync baseline (see #862) we cannot tell an intentional delete
# from a file the other side simply never had, so deletions never sync.
if plan.dest_only:
kept_on = "local" if direction == "pull" else "cloud"
console.print(
f"[dim]{len(plan.dest_only)} file(s) exist only on {kept_on} and were left "
"untouched (deletions are not propagated).[/dim]"
)
except RcloneError as e:
console.print(f"[red]{direction.capitalize()} error: {e}[/red]")
raise typer.Exit(1)
except typer.Exit:
# Already-handled exits (not found, conflicts, errors) propagate cleanly.
raise
except Exception as e:
console.print(f"[red]Error: {e}[/red]")
raise typer.Exit(1)
@cloud_app.command("pull")
def pull_project_command(
name: str = typer.Option(..., "--name", "--project", help="Project name to pull"),
on_conflict: ConflictStrategy = typer.Option(
ConflictStrategy.fail,
"--on-conflict",
help="Resolve files that differ on both sides (default: fail and list them)",
),
workspace: str | None = typer.Option(
None,
"--workspace",
help="Workspace (slug, name, or tenant_id) when the project name is ambiguous",
),
dry_run: bool = typer.Option(False, "--dry-run", help="Preview changes without pulling"),
verbose: bool = typer.Option(False, "--verbose", "-v", help="Show detailed output"),
) -> None:
"""Fetch cloud changes into local (cloud -> local), git-pull style.
Additive and Team-safe: downloads new/changed cloud files and never deletes
local files. A file that differs on both sides is a conflict; by default
pull aborts and lists them. Deletions are not propagated (see #862).
Examples:
bm cloud pull --name research
bm cloud pull --name research --dry-run
bm cloud pull --name research --on-conflict keep-cloud
bm cloud pull --name research --workspace acme
"""
_run_directional_transfer(
name, "pull", on_conflict=on_conflict, dry_run=dry_run, verbose=verbose, workspace=workspace
)
@cloud_app.command("push")
def push_project_command(
name: str = typer.Option(..., "--name", "--project", help="Project name to push"),
on_conflict: ConflictStrategy = typer.Option(
ConflictStrategy.fail,
"--on-conflict",
help="Resolve files that differ on both sides (default: fail and list them)",
),
workspace: str | None = typer.Option(
None,
"--workspace",
help="Workspace (slug, name, or tenant_id) when the project name is ambiguous",
),
dry_run: bool = typer.Option(False, "--dry-run", help="Preview changes without pushing"),
verbose: bool = typer.Option(False, "--verbose", "-v", help="Show detailed output"),
) -> None:
"""Upload local changes to cloud (local -> cloud), additive and Team-safe.
Uploads new/changed local files and never deletes cloud files. A file that
differs on both sides is a conflict; by default push aborts and lists them
(like git rejecting a push when the remote is ahead pull first). Deletions
are not propagated (see #862).
Examples:
bm cloud push --name research
bm cloud push --name research --dry-run
bm cloud push --name research --on-conflict keep-local
bm cloud push --name research --workspace acme
"""
_run_directional_transfer(
name, "push", on_conflict=on_conflict, dry_run=dry_run, verbose=verbose, workspace=workspace
)
@cloud_app.command("bisync")
def bisync_project_command(
name: str = typer.Option(..., "--name", help="Project name to bisync"),
name: str = typer.Option(..., "--name", "--project", help="Project name to bisync"),
dry_run: bool = typer.Option(False, "--dry-run", help="Preview changes without syncing"),
resync: bool = typer.Option(False, "--resync", help="Force new baseline"),
verbose: bool = typer.Option(False, "--verbose", "-v", help="Show detailed output"),
) -> None:
"""Two-way sync: local <-> cloud (bidirectional sync).
"""Two-way mirror: local <-> cloud (bidirectional sync).
Personal workspaces only. This mirror can delete and overwrite files on both
sides, so on Team workspaces use `bm cloud pull` (fetch) / `bm cloud push`
(additive upload) instead.
Examples:
bm cloud bisync --name research --resync # First time
@@ -210,7 +528,10 @@ def bisync_project_command(
_require_personal_workspace(name, config)
try:
# Get tenant info for bucket name
# Get tenant info for bucket name.
# TODO(#919): scope to the project's workspace like push/pull. Safe for now
# because these mirror commands are gated to the (default-tenant) Personal
# workspace, so the default mount info is correct.
tenant_info = run_with_cleanup(get_mount_info())
bucket_name = tenant_info.bucket_name
@@ -256,10 +577,14 @@ def bisync_project_command(
@cloud_app.command("check")
def check_project_command(
name: str = typer.Option(..., "--name", help="Project name to check"),
name: str = typer.Option(..., "--name", "--project", help="Project name to check"),
one_way: bool = typer.Option(False, "--one-way", help="Check one direction only (faster)"),
) -> None:
"""Verify file integrity between local and cloud.
"""Verify file integrity between local and cloud (no changes made).
Personal workspaces only: check compares against the Personal workspace
mirror remote. On Team workspaces use `bm cloud pull --dry-run` /
`bm cloud push --dry-run` to preview differences instead.
Example:
bm cloud check --name research
@@ -268,7 +593,10 @@ def check_project_command(
_require_cloud_credentials(config)
try:
# Get tenant info for bucket name
# Get tenant info for bucket name.
# TODO(#919): scope to the project's workspace like push/pull. Safe for now
# because these mirror commands are gated to the (default-tenant) Personal
# workspace, so the default mount info is correct.
tenant_info = run_with_cleanup(get_mount_info())
bucket_name = tenant_info.bucket_name
@@ -304,6 +632,9 @@ def bisync_reset(
) -> None:
"""Clear bisync state for a project.
Personal workspaces only (bisync is a Personal-workspace mirror; on Team
workspaces use `bm cloud pull` / `bm cloud push` instead).
This removes the bisync metadata files, forcing a fresh --resync on next bisync.
Useful when bisync gets into an inconsistent state or when remote path changes.
"""
@@ -395,9 +726,15 @@ def setup_project_sync(
console.print(f"[green]Sync configured for project '{name}'[/green]")
console.print(f"\nLocal sync path: {resolved_path}")
# Lead with the Team-safe additive commands (work on any workspace); the
# `sync`/`bisync` mirrors are Personal-workspace-only.
console.print("\nNext steps:")
console.print(f" 1. Preview: bm cloud sync --name {name} --dry-run")
console.print(f" 2. Sync: bm cloud sync --name {name}")
console.print(f" 1. Preview a pull: bm cloud pull --name {name} --dry-run")
console.print(f" 2. Fetch from cloud: bm cloud pull --name {name}")
console.print(f" 3. Upload local changes: bm cloud push --name {name}")
console.print(
f" Personal workspaces can also mirror with: bm cloud bisync --name {name} --resync"
)
except Exception as e:
console.print(f"[red]Error configuring sync: {str(e)}[/red]")
raise typer.Exit(1)
@@ -2,7 +2,8 @@
This module provides simplified, project-scoped rclone operations:
- Each project syncs independently
- Uses single "basic-memory-cloud" remote (not tenant-specific)
- Routes through the project's tenant-scoped remote (SyncProject.remote_name);
the default tenant keeps "basic-memory-cloud", others use their own (see #919)
- Balanced defaults from SPEC-8 Phase 4 testing
- Per-project bisync state tracking
@@ -11,10 +12,10 @@ Replaces tenant-wide sync with project-scoped workflows.
import re
import subprocess
from dataclasses import dataclass
from dataclasses import dataclass, field
from functools import lru_cache
from pathlib import Path
from typing import Callable, Optional, Protocol
from pathlib import Path, PurePosixPath
from typing import Callable, Literal, Optional, Protocol
from loguru import logger
from rich.console import Console
@@ -42,6 +43,7 @@ TIGRIS_CONSISTENCY_HEADERS = [
class RunResult(Protocol):
returncode: int
stdout: str
stderr: str
RunFunc = Callable[..., RunResult]
@@ -112,11 +114,15 @@ class SyncProject:
name: Project name
path: Cloud path (e.g., "app/data/research")
local_sync_path: Local directory for syncing (optional)
remote_name: rclone remote serving this project's tenant bucket. Defaults
to the legacy single remote; team/non-default workspaces use their own
(see remote_name_for_workspace).
"""
name: str
path: str
local_sync_path: Optional[str] = None
remote_name: str = "basic-memory-cloud"
def get_bmignore_filter_path() -> Path:
@@ -178,10 +184,98 @@ def get_project_remote(project: SyncProject, bucket_name: str) -> str:
The API returns paths like "/app/data/basic-memory-llc" because the S3 bucket
is mounted at /app/data on the fly machine. We need to strip the /app/data/
prefix to get the actual S3 path within the bucket.
The remote name comes from the project so non-default/team workspaces route
through their own tenant-scoped remote (see #919).
"""
# Normalize path to strip /app/data/ mount point prefix
cloud_path = normalize_project_path(project.path).lstrip("/")
return f"basic-memory-cloud:{bucket_name}/{cloud_path}"
return f"{project.remote_name}:{bucket_name}/{cloud_path}"
# --- Directional transfer primitives (push / pull) ---
#
# These power the Team-safe `bm cloud push` / `bm cloud pull` commands. Unlike
# the mirror operations (`sync`/`bisync`), they use `rclone copy` so they never
# delete on the destination, and conflicts are surfaced to the caller rather
# than silently resolved. See issue #858 for the full design rationale.
# push = local -> cloud, pull = cloud -> local.
TransferDirection = Literal["push", "pull"]
# How a directional transfer treats files that differ on both sides. "fail" is
# the safe default: the caller is expected to abort before any transfer runs.
ConflictStrategy = Literal["fail", "keep-local", "keep-cloud", "keep-both"]
@dataclass
class TransferPlan:
"""Classification of how local and cloud differ for a directional transfer.
Built from ``rclone check --combined``. Paths are relative to the project
root. ``conflicts`` are files present on both sides with differing content
without a sync baseline (see #862) every divergence is a conflict, because
we cannot tell a teammate's edit from a stale local copy.
"""
new: list[str] = field(default_factory=list) # only on source → safe to bring over
conflicts: list[str] = field(default_factory=list) # differ on both sides
dest_only: list[str] = field(default_factory=list) # only on destination → left untouched
errors: list[str] = field(default_factory=list) # rclone could not read/hash
def _transfer_endpoints(project: SyncProject, bucket_name: str) -> tuple[str, str]:
"""Return (local_path, remote_path) strings for a project's transfer.
Raises:
RcloneError: If the project has no local_sync_path configured.
"""
if not project.local_sync_path:
raise RcloneError(f"Project {project.name} has no local_sync_path configured")
local_path = str(Path(project.local_sync_path).expanduser())
remote_path = get_project_remote(project, bucket_name)
return local_path, remote_path
def _build_transfer_cmd(
operation: str,
source: str,
dest: str,
*,
filter_path: Path,
dry_run: bool,
verbose: bool,
extra_flags: tuple[str, ...] = (),
) -> list[str]:
"""Build an rclone sync/copy command with the shared Basic Memory flags.
All directional transfers share the same tail: Tigris consistency headers,
the .bmignore filter, and --local-no-preallocate (a no-op when local is the
source, required when local is the destination on pull see rclone#6801).
"""
cmd = [
"rclone",
operation,
source,
dest,
*TIGRIS_CONSISTENCY_HEADERS,
"--filter-from",
str(filter_path),
# Prevent NUL byte padding on virtual filesystems (e.g. Google Drive File Stream)
# See: rclone/rclone#6801
"--local-no-preallocate",
*extra_flags,
]
if verbose:
cmd.append("--verbose")
else:
cmd.append("--progress")
if dry_run:
cmd.append("--dry-run")
return cmd
def project_sync(
@@ -219,24 +313,199 @@ def project_sync(
remote_path = get_project_remote(project, bucket_name)
filter_path = filter_path or get_bmignore_filter_path()
cmd = [
"rclone",
cmd = _build_transfer_cmd(
"sync",
str(local_path),
remote_path,
filter_path=filter_path,
dry_run=dry_run,
verbose=verbose,
)
result = run(cmd, text=True)
return result.returncode == 0
def _parse_check_combined(output: str) -> TransferPlan:
"""Parse ``rclone check --combined`` output into a TransferPlan.
rclone emits one prefixed line per path (src is the transfer source):
``=`` identical, ``+`` only on src, ``-`` only on dst, ``*`` differ,
``!`` error reading/hashing. We ignore identical files.
"""
plan = TransferPlan()
for line in output.splitlines():
symbol, _, path = line.partition(" ")
path = path.strip()
if not path:
continue
if symbol == "+":
plan.new.append(path)
elif symbol == "*":
plan.conflicts.append(path)
elif symbol == "-":
plan.dest_only.append(path)
elif symbol == "!":
plan.errors.append(path)
# "=" (identical) is intentionally dropped.
return plan
def project_diff(
project: SyncProject,
bucket_name: str,
direction: TransferDirection,
*,
run: RunFunc = subprocess.run,
is_installed: IsInstalledFunc = is_rclone_installed,
filter_path: Path | None = None,
) -> TransferPlan:
"""Classify how local and cloud differ for a push/pull, without transferring.
Uses ``rclone check`` (content comparison) so the caller can surface
conflicts before any data moves. The source side depends on direction:
pull compares cloudlocal, push compares localcloud.
Raises:
RcloneError: If project has no local_sync_path configured or rclone not installed
"""
check_rclone_installed(is_installed=is_installed)
local_path, remote_path = _transfer_endpoints(project, bucket_name)
filter_path = filter_path or get_bmignore_filter_path()
# Source/dest order matters: rclone check reports "+" for files only on the
# source, which is what we want to bring over.
source, dest = (remote_path, local_path) if direction == "pull" else (local_path, remote_path)
cmd = [
"rclone",
"check",
source,
dest,
*TIGRIS_CONSISTENCY_HEADERS,
"--filter-from",
str(filter_path),
# Prevent NUL byte padding on virtual filesystems (e.g. Google Drive File Stream)
# See: rclone/rclone#6801
"--local-no-preallocate",
"--combined",
"-",
]
# rclone check exits non-zero when files differ — that's expected here, so we
# parse the combined listing rather than trusting the return code.
result = run(cmd, capture_output=True, text=True)
plan = _parse_check_combined(result.stdout)
# Trigger: non-zero exit AND the combined listing produced no entries at all.
# Why: a difference always yields +/-/*/! lines, so an empty listing on a
# non-zero exit means the check itself failed (auth, missing remote, network,
# bad filter) rather than finding zero differences. Without this guard the
# caller would see an empty plan, transfer nothing, and report success.
# Outcome: fail fast with rclone's stderr instead of a silent no-op.
if result.returncode != 0 and not (plan.new or plan.conflicts or plan.dest_only or plan.errors):
detail = result.stderr.strip() or f"rclone check exited with code {result.returncode}"
raise RcloneError(f"Failed to compare {project.name} with cloud: {detail}")
return plan
def project_copy(
project: SyncProject,
bucket_name: str,
direction: TransferDirection,
*,
overwrite: bool,
dry_run: bool = False,
verbose: bool = False,
run: RunFunc = subprocess.run,
is_installed: IsInstalledFunc = is_rclone_installed,
filter_path: Path | None = None,
) -> bool:
"""Additive transfer via ``rclone copy`` — never deletes on the destination.
Trigger: ``overwrite=False`` adds ``--ignore-existing`` so files already on
the destination are left as-is (used when the destination side wins a
conflict, and for the no-conflict fast path).
Why: keeps the loser's bytes intact unless the caller explicitly chose to
overwrite, matching the "no surprises" contract.
Raises:
RcloneError: If project has no local_sync_path configured or rclone not installed
"""
check_rclone_installed(is_installed=is_installed)
local_path, remote_path = _transfer_endpoints(project, bucket_name)
filter_path = filter_path or get_bmignore_filter_path()
source, dest = (remote_path, local_path) if direction == "pull" else (local_path, remote_path)
# Overwrite mode compares by checksum so the transfer decision matches
# project_diff's content-based conflict detection (rclone check). Without
# --checksum, copy's default size+modtime comparison could skip a file the
# diff flagged as a conflict (same size, destination not older) — silently
# ignoring the user's explicit keep-cloud/keep-local choice. New-only mode
# uses --ignore-existing, which skips by existence so the comparison basis
# does not matter.
extra_flags = ("--checksum",) if overwrite else ("--ignore-existing",)
cmd = _build_transfer_cmd(
"copy",
source,
dest,
filter_path=filter_path,
dry_run=dry_run,
verbose=verbose,
extra_flags=extra_flags,
)
result = run(cmd, text=True)
return result.returncode == 0
def _conflict_copy_name(rel_path: str, suffix: str) -> str:
"""Insert a ``.conflict-<suffix>`` marker before the extension of a rel path."""
p = PurePosixPath(rel_path)
return str(p.with_name(f"{p.stem}.conflict-{suffix}{p.suffix}"))
def project_copy_file(
project: SyncProject,
bucket_name: str,
direction: TransferDirection,
source_rel_path: str,
dest_rel_path: str,
*,
dry_run: bool = False,
verbose: bool = False,
run: RunFunc = subprocess.run,
is_installed: IsInstalledFunc = is_rclone_installed,
) -> bool:
"""Copy a single file from source to destination under a (possibly renamed) path.
Used for the ``keep-both`` strategy: the incoming version is written beside
the destination's own copy as ``name.conflict-<date>`` so nothing is lost.
Raises:
RcloneError: If project has no local_sync_path configured or rclone not installed
"""
check_rclone_installed(is_installed=is_installed)
local_path, remote_path = _transfer_endpoints(project, bucket_name)
source_root, dest_root = (
(remote_path, local_path) if direction == "pull" else (local_path, remote_path)
)
cmd = [
"rclone",
"copyto",
f"{source_root}/{source_rel_path}",
f"{dest_root}/{dest_rel_path}",
*TIGRIS_CONSISTENCY_HEADERS,
# Matches _build_transfer_cmd: on pull this writes the conflict copy to
# the local filesystem, where this prevents NUL byte padding on virtual
# filesystems (e.g. Google Drive File Stream). See rclone/rclone#6801.
"--local-no-preallocate",
]
if verbose:
cmd.append("--verbose")
else:
cmd.append("--progress")
if dry_run:
cmd.append("--dry-run")
@@ -244,6 +513,72 @@ def project_sync(
return result.returncode == 0
def _strategy_overwrites_dest(direction: TransferDirection, strategy: ConflictStrategy) -> bool:
"""True when the strategy lets the source side overwrite the destination.
The source side is cloud on pull, local on push. "keep-cloud" wins on pull,
"keep-local" wins on push; otherwise the destination is preserved.
"""
if strategy == "keep-cloud":
return direction == "pull"
if strategy == "keep-local":
return direction == "push"
return False # "fail" (no conflicts) and "keep-both" never overwrite existing dest files
def project_transfer(
project: SyncProject,
bucket_name: str,
direction: TransferDirection,
plan: TransferPlan,
*,
strategy: ConflictStrategy = "fail",
conflict_suffix: str = "",
dry_run: bool = False,
verbose: bool = False,
run: RunFunc = subprocess.run,
is_installed: IsInstalledFunc = is_rclone_installed,
filter_path: Path | None = None,
) -> bool:
"""Execute a directional transfer for the chosen conflict strategy.
Callers detect conflicts with ``project_diff`` first and abort when
``strategy == "fail"`` and conflicts exist; this function assumes that gate
has already passed and applies the resolution.
"""
# keep-both: preserve the destination's version and drop the incoming one
# beside it as a conflict copy, then do an additive (new-only) pass.
if strategy == "keep-both":
for rel_path in plan.conflicts:
dest_rel = _conflict_copy_name(rel_path, conflict_suffix)
copied = project_copy_file(
project,
bucket_name,
direction,
rel_path,
dest_rel,
dry_run=dry_run,
verbose=verbose,
run=run,
is_installed=is_installed,
)
if not copied:
return False
overwrite = _strategy_overwrites_dest(direction, strategy)
return project_copy(
project,
bucket_name,
direction,
overwrite=overwrite,
dry_run=dry_run,
verbose=verbose,
run=run,
is_installed=is_installed,
filter_path=filter_path,
)
def project_bisync(
project: SyncProject,
bucket_name: str,
@@ -1,11 +1,14 @@
"""rclone configuration management for Basic Memory Cloud.
This module provides simplified rclone configuration for SPEC-20.
Uses a single "basic-memory-cloud" remote for all operations.
This module owns rclone remote configuration and naming. The default tenant uses
the "basic-memory-cloud" remote (from SPEC-20); non-default/team workspaces each
get their own tenant-scoped remote via remote_name_for_workspace (see #919),
since Tigris credentials are bucket-scoped.
"""
import configparser
import os
import re
import shutil
from pathlib import Path
from typing import Optional
@@ -61,25 +64,65 @@ def save_rclone_config(config: configparser.ConfigParser) -> None:
console.print(f"[dim]Updated rclone config: {config_path}[/dim]")
# The default remote serves the account's default tenant (back-compat with SPEC-20,
# which used a single "basic-memory-cloud" remote). Non-default workspaces each get
# their own remote, since Tigris credentials are bucket/tenant-scoped (see #919).
DEFAULT_RCLONE_REMOTE = "basic-memory-cloud"
# rclone remote section names allow letters, digits, hyphens, and underscores.
# The slug comes from the cloud API (a trust boundary), so validate it before
# splicing it into a remote name to avoid a broken/unusable rclone.conf section.
_SAFE_SLUG = re.compile(r"^[A-Za-z0-9_-]+$")
def remote_name_for_workspace(slug: str | None, *, is_default: bool) -> str:
"""Return the rclone remote name for a workspace.
The default workspace keeps the legacy ``basic-memory-cloud`` remote so
existing setups keep working; other workspaces get ``basic-memory-cloud-<slug>``.
Raises:
RcloneConfigError: If a non-default workspace slug contains characters
that are not valid in an rclone remote name.
"""
if is_default or not slug:
return DEFAULT_RCLONE_REMOTE
if not _SAFE_SLUG.match(slug):
raise RcloneConfigError(
f"Workspace slug '{slug}' cannot be used as an rclone remote name "
"(allowed: letters, digits, hyphens, underscores)."
)
return f"{DEFAULT_RCLONE_REMOTE}-{slug}"
def rclone_remote_exists(remote_name: str) -> bool:
"""Return whether an rclone remote section is already configured."""
return load_rclone_config().has_section(remote_name)
def configure_rclone_remote(
access_key: str,
secret_key: str,
endpoint: str = "https://fly.storage.tigris.dev",
region: str = "auto",
remote_name: str = DEFAULT_RCLONE_REMOTE,
) -> str:
"""Configure single rclone remote named 'basic-memory-cloud'.
"""Configure an rclone remote for one tenant's bucket.
This is the simplified approach from SPEC-20 that uses one remote
for all Basic Memory cloud operations (not tenant-specific).
Each tenant (personal or team) has its own bucket-scoped credentials, so a
remote maps 1:1 to a tenant. The default tenant keeps ``basic-memory-cloud``;
other workspaces pass ``remote_name`` from :func:`remote_name_for_workspace`.
Args:
access_key: S3 access key ID
secret_key: S3 secret access key
endpoint: S3-compatible endpoint URL
region: S3 region (default: auto)
remote_name: rclone remote section name to write
Returns:
The remote name: "basic-memory-cloud"
The remote name that was configured
"""
# Backup existing config
backup_rclone_config()
@@ -87,24 +130,21 @@ def configure_rclone_remote(
# Load existing config
config = load_rclone_config()
# Single remote name (not tenant-specific)
REMOTE_NAME = "basic-memory-cloud"
# Add/update the remote section
if not config.has_section(REMOTE_NAME):
config.add_section(REMOTE_NAME)
if not config.has_section(remote_name):
config.add_section(remote_name)
config.set(REMOTE_NAME, "type", "s3")
config.set(REMOTE_NAME, "provider", "Other")
config.set(REMOTE_NAME, "access_key_id", access_key)
config.set(REMOTE_NAME, "secret_access_key", secret_key)
config.set(REMOTE_NAME, "endpoint", endpoint)
config.set(REMOTE_NAME, "region", region)
config.set(remote_name, "type", "s3")
config.set(remote_name, "provider", "Other")
config.set(remote_name, "access_key_id", access_key)
config.set(remote_name, "secret_access_key", secret_key)
config.set(remote_name, "endpoint", endpoint)
config.set(remote_name, "region", region)
# Prevent unnecessary encoding of filenames (only encode slashes and invalid UTF-8)
# This prevents files with spaces like "Hello World.md" from being quoted
config.set(REMOTE_NAME, "encoding", "Slash,InvalidUtf8")
config.set(remote_name, "encoding", "Slash,InvalidUtf8")
# Save updated config
save_rclone_config(config)
console.print(f"[green]Configured rclone remote: {REMOTE_NAME}[/green]")
return REMOTE_NAME
console.print(f"[green]Configured rclone remote: {remote_name}[/green]")
return remote_name
@@ -3,12 +3,10 @@
import asyncio
from typing import Optional, TypeVar, Coroutine, Any
from mcp.server.fastmcp.exceptions import ToolError
import typer
from rich.console import Console
from basic_memory import db
from basic_memory.config import ConfigManager
from basic_memory.mcp.async_client import get_client
from basic_memory.mcp.clients import ProjectClient
@@ -31,6 +29,9 @@ def run_with_cleanup(coro: Coroutine[Any, Any, T]) -> T:
Returns:
The result of the coroutine
"""
# Deferred: basic_memory.db pulls SQLAlchemy + Alembic, which must not load
# at CLI import time — only when a command actually runs (#886).
from basic_memory import db
async def _with_cleanup() -> T:
try:
@@ -53,6 +54,8 @@ async def run_sync(
force_full: If True, force a full scan bypassing watermark optimization
run_in_background: If True, return immediately; if False, wait for completion
"""
# Deferred: ToolError lives in the mcp SDK, which must not load at CLI startup (#886).
from mcp.server.fastmcp.exceptions import ToolError
# Resolve default project so get_client() can route per-project
project = project or ConfigManager().default_project
@@ -86,6 +89,9 @@ async def run_sync(
async def get_project_info(project: str):
"""Get project information via API endpoint."""
# Deferred: ToolError lives in the mcp SDK, which must not load at CLI startup (#886).
from mcp.server.fastmcp.exceptions import ToolError
try:
async with get_client(project_name=project) as client:
project_item = await get_active_project(client, project, None)
+28 -7
View File
@@ -1,24 +1,27 @@
"""Database management commands."""
# PEP 563 lazy annotations let signatures reference IndexProgress without importing
# the indexing stack at module load; reset/reindex import their heavy database and
# sync dependencies at call time so CLI startup stays fast (#886).
from __future__ import annotations
import os
from dataclasses import dataclass
from pathlib import Path, PurePosixPath, PureWindowsPath
from typing import TYPE_CHECKING
import psutil
import typer
from loguru import logger
from rich.console import Console
from rich.progress import Progress, SpinnerColumn, TextColumn, BarColumn, TaskProgressColumn
from sqlalchemy.exc import OperationalError
from basic_memory import db
from basic_memory.cli.app import app
from basic_memory.cli.commands.command_utils import run_with_cleanup
from basic_memory.config import ConfigManager, ProjectMode
from basic_memory.indexing import IndexProgress
from basic_memory.repository import ProjectRepository
from basic_memory.services.initialization import reconcile_projects_with_config
from basic_memory.sync.sync_service import get_sync_service
if TYPE_CHECKING:
from basic_memory.indexing import IndexProgress
console = Console()
@@ -159,6 +162,13 @@ async def _reindex_projects(app_config):
This ensures all database operations use the same event loop,
and proper cleanup happens when the function completes.
"""
# Deferred: SQLAlchemy, repositories, and the sync stack load only when a
# reindex actually runs, not on every CLI start (#886).
from basic_memory import db
from basic_memory.repository import ProjectRepository
from basic_memory.services.initialization import reconcile_projects_with_config
from basic_memory.sync.sync_service import get_sync_service
try:
await reconcile_projects_with_config(app_config)
@@ -197,6 +207,12 @@ def reset(
),
): # pragma: no cover
"""Reset database (drop all tables and recreate)."""
# Deferred: SQLAlchemy and the db module load only when a reset actually
# runs, not on every CLI start (#886).
from sqlalchemy.exc import OperationalError
from basic_memory import db
console.print(
"[yellow]Note:[/yellow] This only deletes the index database. "
"Your markdown note files will not be affected.\n"
@@ -320,10 +336,15 @@ async def _reindex(
project: str | None,
):
"""Run reindex operations."""
from basic_memory.repository import EntityRepository
# Deferred: SQLAlchemy, repositories, and the sync stack load only when a
# reindex actually runs, not on every CLI start (#886).
from basic_memory import db
from basic_memory.repository import EntityRepository, ProjectRepository
from basic_memory.repository.search_repository import create_search_repository
from basic_memory.services.initialization import reconcile_projects_with_config
from basic_memory.services.search_service import SearchService
from basic_memory.services.file_service import FileService
from basic_memory.sync.sync_service import get_sync_service
from basic_memory.markdown.markdown_processor import MarkdownProcessor
from basic_memory.markdown.entity_parser import EntityParser
+9 -4
View File
@@ -7,16 +7,12 @@ import uuid
from pathlib import Path
from loguru import logger
from mcp.server.fastmcp.exceptions import ToolError
from rich.console import Console
import typer
from basic_memory.cli.app import app
from basic_memory.cli.commands.command_utils import run_with_cleanup
from basic_memory.cli.commands.routing import force_routing, validate_routing_flags
from basic_memory.markdown.entity_parser import EntityParser
from basic_memory.markdown.markdown_processor import MarkdownProcessor
from basic_memory.markdown.schemas import EntityFrontmatter, EntityMarkdown
from basic_memory.mcp.async_client import get_client
from basic_memory.mcp.clients import KnowledgeClient, ProjectClient, SearchClient
from basic_memory.schemas.base import Entity
@@ -29,6 +25,12 @@ console = Console()
async def run_doctor() -> None:
"""Run local consistency checks for file <-> database flows."""
# Deferred: the markdown parsing stack is only needed while the checks run,
# and importing it at module level slows every CLI invocation (#886).
from basic_memory.markdown.entity_parser import EntityParser
from basic_memory.markdown.markdown_processor import MarkdownProcessor
from basic_memory.markdown.schemas import EntityFrontmatter, EntityMarkdown
console.print("[blue]Running Basic Memory doctor checks...[/blue]")
project_name = f"doctor-{uuid.uuid4().hex[:8]}"
@@ -140,6 +142,9 @@ def doctor(
cloud: bool = typer.Option(False, "--cloud", help="Force cloud API routing"),
) -> None:
"""Run local consistency checks to verify file/database sync."""
# Deferred: ToolError lives in the mcp SDK, which must not load at CLI startup (#886).
from mcp.server.fastmcp.exceptions import ToolError
try:
validate_routing_flags(local, cloud)
# Doctor runs local filesystem checks — always default to local routing
@@ -1,25 +1,34 @@
"""Import command for ChatGPT conversations."""
# PEP 563 lazy annotations keep heavy importer types out of module import (#886).
from __future__ import annotations
import json
from pathlib import Path
from typing import Annotated, Tuple
from typing import TYPE_CHECKING, Annotated, Tuple
import typer
from basic_memory.cli.app import import_app
from basic_memory.cli.commands.command_utils import run_with_cleanup
from basic_memory.config import ConfigManager, get_project_config
from basic_memory.importers import ChatGPTImporter
from basic_memory.markdown import EntityParser, MarkdownProcessor
from basic_memory.services.file_service import FileService
from loguru import logger
from rich.console import Console
from rich.panel import Panel
if TYPE_CHECKING:
from basic_memory.markdown import MarkdownProcessor
from basic_memory.services.file_service import FileService
console = Console()
async def get_importer_dependencies() -> Tuple[MarkdownProcessor, FileService]:
"""Get MarkdownProcessor and FileService instances for importers."""
# Deferred: the markdown/file-service stack pulls SQLAlchemy and must load
# only when an import actually runs, not on every CLI start (#886).
from basic_memory.markdown import EntityParser, MarkdownProcessor
from basic_memory.services.file_service import FileService
config = get_project_config()
app_config = ConfigManager().config
entity_parser = EntityParser(config.home)
@@ -60,6 +69,9 @@ def import_chatgpt(
console.print(f"\nImporting chats from {conversations_json}...writing to {base_path}")
# Create importer and run import
# Deferred: importer stack loads at import-command run time only (#886).
from basic_memory.importers import ChatGPTImporter
importer = ChatGPTImporter(
config.home, markdown_processor, file_service, project_name=config.name
)
@@ -1,25 +1,34 @@
"""Import command for basic-memory CLI to import chat data from conversations2.json format."""
# PEP 563 lazy annotations keep heavy importer types out of module import (#886).
from __future__ import annotations
import json
from pathlib import Path
from typing import Annotated, Tuple
from typing import TYPE_CHECKING, Annotated, Tuple
import typer
from basic_memory.cli.app import claude_app
from basic_memory.cli.commands.command_utils import run_with_cleanup
from basic_memory.config import ConfigManager, get_project_config
from basic_memory.importers.claude_conversations_importer import ClaudeConversationsImporter
from basic_memory.markdown import EntityParser, MarkdownProcessor
from basic_memory.services.file_service import FileService
from loguru import logger
from rich.console import Console
from rich.panel import Panel
if TYPE_CHECKING:
from basic_memory.markdown import MarkdownProcessor
from basic_memory.services.file_service import FileService
console = Console()
async def get_importer_dependencies() -> Tuple[MarkdownProcessor, FileService]:
"""Get MarkdownProcessor and FileService instances for importers."""
# Deferred: the markdown/file-service stack pulls SQLAlchemy and must load
# only when an import actually runs, not on every CLI start (#886).
from basic_memory.markdown import EntityParser, MarkdownProcessor
from basic_memory.services.file_service import FileService
config = get_project_config()
app_config = ConfigManager().config
entity_parser = EntityParser(config.home)
@@ -57,6 +66,9 @@ def import_claude(
markdown_processor, file_service = run_with_cleanup(get_importer_dependencies())
# Create the importer
# Deferred: importer stack loads at import-command run time only (#886).
from basic_memory.importers.claude_conversations_importer import ClaudeConversationsImporter
importer = ClaudeConversationsImporter(
config.home, markdown_processor, file_service, project_name=config.name
)
@@ -1,25 +1,34 @@
"""Import command for basic-memory CLI to import project data from Claude.ai."""
# PEP 563 lazy annotations keep heavy importer types out of module import (#886).
from __future__ import annotations
import json
from pathlib import Path
from typing import Annotated, Tuple
from typing import TYPE_CHECKING, Annotated, Tuple
import typer
from basic_memory.cli.app import claude_app
from basic_memory.cli.commands.command_utils import run_with_cleanup
from basic_memory.config import ConfigManager, get_project_config
from basic_memory.importers.claude_projects_importer import ClaudeProjectsImporter
from basic_memory.markdown import EntityParser, MarkdownProcessor
from basic_memory.services.file_service import FileService
from loguru import logger
from rich.console import Console
from rich.panel import Panel
if TYPE_CHECKING:
from basic_memory.markdown import MarkdownProcessor
from basic_memory.services.file_service import FileService
console = Console()
async def get_importer_dependencies() -> Tuple[MarkdownProcessor, FileService]:
"""Get MarkdownProcessor and FileService instances for importers."""
# Deferred: the markdown/file-service stack pulls SQLAlchemy and must load
# only when an import actually runs, not on every CLI start (#886).
from basic_memory.markdown import EntityParser, MarkdownProcessor
from basic_memory.services.file_service import FileService
config = get_project_config()
app_config = ConfigManager().config
entity_parser = EntityParser(config.home)
@@ -56,6 +65,9 @@ def import_projects(
markdown_processor, file_service = run_with_cleanup(get_importer_dependencies())
# Create the importer
# Deferred: importer stack loads at import-command run time only (#886).
from basic_memory.importers.claude_projects_importer import ClaudeProjectsImporter
importer = ClaudeProjectsImporter(
config.home, markdown_processor, file_service, project_name=config.name
)
@@ -1,25 +1,34 @@
"""Import command for basic-memory CLI to import from JSON memory format."""
# PEP 563 lazy annotations keep heavy importer types out of module import (#886).
from __future__ import annotations
import json
from pathlib import Path
from typing import Annotated, Tuple
from typing import TYPE_CHECKING, Annotated, Tuple
import typer
from basic_memory.cli.app import import_app
from basic_memory.cli.commands.command_utils import run_with_cleanup
from basic_memory.config import ConfigManager, get_project_config
from basic_memory.importers.memory_json_importer import MemoryJsonImporter
from basic_memory.markdown import EntityParser, MarkdownProcessor
from basic_memory.services.file_service import FileService
from loguru import logger
from rich.console import Console
from rich.panel import Panel
if TYPE_CHECKING:
from basic_memory.markdown import MarkdownProcessor
from basic_memory.services.file_service import FileService
console = Console()
async def get_importer_dependencies() -> Tuple[MarkdownProcessor, FileService]:
"""Get MarkdownProcessor and FileService instances for importers."""
# Deferred: the markdown/file-service stack pulls SQLAlchemy and must load
# only when an import actually runs, not on every CLI start (#886).
from basic_memory.markdown import EntityParser, MarkdownProcessor
from basic_memory.services.file_service import FileService
config = get_project_config()
app_config = ConfigManager().config
entity_parser = EntityParser(config.home)
@@ -55,6 +64,9 @@ def memory_json(
markdown_processor, file_service = run_with_cleanup(get_importer_dependencies())
# Create the importer
# Deferred: importer stack loads at import-command run time only (#886).
from basic_memory.importers.memory_json_importer import MemoryJsonImporter
importer = MemoryJsonImporter(
config.home, markdown_processor, file_service, project_name=config.name
)
+76
View File
@@ -0,0 +1,76 @@
"""Install the bundled man pages so `man bm` works."""
import shutil
import subprocess
from pathlib import Path
from typing import Annotated, Optional
import typer
from rich.console import Console
from basic_memory.cli.app import app
console = Console()
man_app = typer.Typer(help="Manage the bm man pages.")
app.add_typer(man_app, name="man")
# Bundled groff sources ship inside the package (src/basic_memory/man).
_MAN_SOURCE_DIR = Path(__file__).parent.parent.parent / "man"
def _default_man_root() -> Path:
# Why ~/.local/share/man: manpath(1) derives man directories from PATH
# entries on both man-db (Linux) and BSD man (macOS), so ~/.local/bin on
# PATH — the pipx/uv tool layout — makes this root searchable without any
# MANPATH configuration.
return Path.home() / ".local" / "share" / "man"
def _man_root_on_manpath(man_root: Path) -> Optional[bool]:
"""Best-effort check whether man(1) will search man_root; None if unknown."""
try:
result = subprocess.run(["manpath"], capture_output=True, text=True, timeout=5)
except (FileNotFoundError, subprocess.TimeoutExpired):
return None
if result.returncode != 0:
return None
paths = [entry.rstrip("/") for entry in result.stdout.strip().split(":") if entry]
return str(man_root).rstrip("/") in paths
@man_app.command()
def install(
directory: Annotated[
Optional[Path],
typer.Option(
"--dir",
help="Man root to install into (default: ~/.local/share/man)",
),
] = None,
) -> None:
"""Install the bm man pages, then try `man bm`."""
man_root = (directory or _default_man_root()).expanduser()
man1 = man_root / "man1"
man1.mkdir(parents=True, exist_ok=True)
pages = sorted(_MAN_SOURCE_DIR.glob("*.1"))
if not pages: # pragma: no cover - broken packaging, not a runtime state
console.print("[red]No bundled man pages found — broken installation[/red]")
raise typer.Exit(1)
for page in pages:
shutil.copyfile(page, man1 / page.name)
console.print(f"installed {man1 / page.name}")
# Trigger: the chosen root is provably absent from manpath output.
# Why: a silent install into an unsearched directory looks like success
# but `man bm` still fails; say so and hand over the one-line fix.
# Outcome: actionable hint; unknown (None) stays quiet to avoid false alarms.
if _man_root_on_manpath(man_root) is False:
console.print(
f"\n[yellow]{man_root} is not on your manpath.[/yellow] Add it with:\n"
f' export MANPATH="{man_root}:$MANPATH"'
)
console.print("\nTry: [bold]man bm[/bold]")
+12
View File
@@ -98,6 +98,18 @@ def mcp(
if transport == "stdio":
threading.Thread(target=_run_background_auto_update, daemon=True).start()
# Trigger: MCP server startup on the Postgres backend, before the transport
# creates its event loop.
# Why: the watcher/lifespan path runs startup migrations + engine.dispose() on
# asyncpg, which races stdlib asyncio teardown and crashes the container loop
# (#831/#877). uvloop's C scheduler structurally avoids that race and must own
# the loop policy before the loop is created. No-op for SQLite.
# Outcome: `basic-memory mcp` on Postgres runs on uvloop. (The CLI callback also
# installs it; this keeps the server startup seam explicit and self-contained.)
from basic_memory.db import maybe_install_uvloop
maybe_install_uvloop(ConfigManager().config)
# Run the MCP server (blocks)
# Lifespan handles: initialization, migrations, file sync, cleanup
logger.info(f"Starting MCP server with {transport.upper()} transport")
+3 -1
View File
@@ -5,7 +5,6 @@ from typing import Annotated, Optional
import typer
from loguru import logger
from mcp.server.fastmcp.exceptions import ToolError
from rich.console import Console
from rich.table import Table
@@ -50,6 +49,9 @@ def orphans(
"""
from basic_memory.cli.commands.command_utils import run_with_cleanup
# Deferred: ToolError lives in the mcp SDK, which must not load at CLI startup (#886).
from mcp.server.fastmcp.exceptions import ToolError
try:
validate_routing_flags(local, cloud)
with force_routing(local=local, cloud=cloud):
+13 -3
View File
@@ -20,9 +20,10 @@ from basic_memory.cli.app import app
from basic_memory.cli.commands.command_utils import run_with_cleanup
from basic_memory.cli.commands.routing import force_routing, validate_routing_flags
from basic_memory.config import ConfigManager
from basic_memory.mcp.tools import schema_diff as mcp_schema_diff
from basic_memory.mcp.tools import schema_infer as mcp_schema_infer
from basic_memory.mcp.tools import schema_validate as mcp_schema_validate
# MCP tool functions are imported inside each command: importing
# basic_memory.mcp.tools loads the entire tool stack (fastmcp, mcp SDK,
# SQLAlchemy), which would slow every CLI invocation, including --help (#886).
console = Console()
@@ -189,6 +190,9 @@ def validate(
Use --local to force local routing when cloud mode is enabled.
Use --cloud to force cloud routing when cloud mode is disabled.
"""
# Deferred: loading the MCP tool stack at module import slows CLI startup (#886).
from basic_memory.mcp.tools import schema_validate as mcp_schema_validate
try:
validate_routing_flags(local, cloud)
project_name = _resolve_project_name(project)
@@ -272,6 +276,9 @@ def infer(
Use --local to force local routing when cloud mode is enabled.
Use --cloud to force cloud routing when cloud mode is disabled.
"""
# Deferred: loading the MCP tool stack at module import slows CLI startup (#886).
from basic_memory.mcp.tools import schema_infer as mcp_schema_infer
try:
validate_routing_flags(local, cloud)
project_name = _resolve_project_name(project)
@@ -352,6 +359,9 @@ def diff(
Use --local to force local routing when cloud mode is enabled.
Use --cloud to force cloud routing when cloud mode is disabled.
"""
# Deferred: loading the MCP tool stack at module import slows CLI startup (#886).
from basic_memory.mcp.tools import schema_diff as mcp_schema_diff
try:
validate_routing_flags(local, cloud)
project_name = _resolve_project_name(project)
+80 -6
View File
@@ -1,10 +1,10 @@
"""Status command for basic-memory CLI."""
import asyncio
import json
from typing import Set, Dict
from typing import Annotated, Optional
import time
from typing import Annotated, Dict, Optional, Set
from mcp.server.fastmcp.exceptions import ToolError
import typer
from loguru import logger
from rich.console import Console
@@ -142,20 +142,65 @@ def display_changes(
console.print(Panel(tree, expand=False))
class StatusTimeout(Exception):
"""Raised when --wait does not reach a synced state before the deadline."""
async def run_status(
project: Optional[str] = None,
wait: bool = False,
timeout: float = 30.0,
poll_interval: float = 0.5,
) -> tuple[str, SyncReportResponse]:
"""Fetch sync status of files vs database.
When ``wait`` is False this performs a single live disk-vs-DB scan and
returns immediately. When ``wait`` is True it polls until the project has
no pending changes (``sync_report.total == 0``) or the timeout elapses.
Returns (project_name, sync_report) for the caller to render.
Raises:
StatusTimeout: If ``wait`` is True and the deadline passes before the
project reaches a synced state.
"""
# Resolve default project so get_client() can route per-project
project = project or ConfigManager().default_project
# Reuse a single client/context across polls so we don't reconnect each loop.
async with get_client(project_name=project) as client:
project_item = await get_active_project(client, project, None)
sync_report = await ProjectClient(client).get_status(project_item.external_id)
return project_item.name, sync_report
project_client = ProjectClient(client)
# Trigger: caller did not request --wait
# Why: preserve the original single-scan behavior for the common case
# Outcome: one status scan, returned as-is
if not wait:
sync_report = await project_client.get_status(project_item.external_id)
return project_item.name, sync_report
# Trigger: --wait requested
# Why: callers (bulk imports, benchmarks, tests) need to block until the
# index has caught up instead of polling externally
# Outcome: poll get_status until total == 0 or the deadline is reached
deadline = time.monotonic() + timeout
while True:
sync_report = await project_client.get_status(project_item.external_id)
if sync_report.total == 0:
return project_item.name, sync_report
if time.monotonic() >= deadline:
# Why the hint: indexing is done by the sync coordinator, which
# only runs inside a live server (bm mcp / hosted API). In a
# CLI-only session nothing will ever drain the pending count,
# so this wait cannot succeed — point at the command that
# actually indexes (#959).
raise StatusTimeout(
f"Timed out after {timeout:g}s waiting for '{project_item.name}' "
f"to finish indexing ({sync_report.total} pending change(s) remaining). "
f"If no Basic Memory server is running, pending changes are never "
f"indexed — run 'bm reindex --project {project_item.name}' instead."
)
await asyncio.sleep(poll_interval)
@app.command()
@@ -166,6 +211,10 @@ def status(
] = None,
verbose: bool = typer.Option(False, "--verbose", "-v", help="Show detailed file information"),
json_output: bool = typer.Option(False, "--json", help="Output in JSON format"),
wait: bool = typer.Option(
False, "--wait", help="Block until indexing is complete (no pending changes)"
),
timeout: float = typer.Option(30.0, "--timeout", help="Max seconds to wait when --wait is set"),
local: bool = typer.Option(
False, "--local", help="Force local API routing (ignore cloud mode)"
),
@@ -174,11 +223,24 @@ def status(
"""Show sync status between files and database.
Use --json for machine-readable output.
Use --wait to block until indexing is complete (e.g. after a bulk import);
combine with --timeout to bound the wait. On timeout the command exits 1.
Use --local to force local routing when cloud mode is enabled.
Use --cloud to force cloud routing when cloud mode is disabled.
"""
from basic_memory.cli.commands.command_utils import run_with_cleanup
# Deferred: ToolError lives in the mcp SDK, which must not load at CLI startup (#886).
from mcp.server.fastmcp.exceptions import ToolError
# Trigger: --wait with a negative --timeout
# Why: a negative deadline times out on the very first poll, producing a confusing
# "Timed out after -5s" message instead of flagging the bad input. Raised
# before the try/except so typer renders a clean usage error (exit 2).
# Outcome: reject it up front with a clear parameter error.
if wait and timeout < 0:
raise typer.BadParameter("--timeout must be >= 0", param_hint="'--timeout'")
try:
validate_routing_flags(local, cloud)
# Trigger: no explicit routing flag provided
@@ -189,12 +251,24 @@ def status(
if not local and not cloud:
local = True
with force_routing(local=local, cloud=cloud):
project_name, sync_report = run_with_cleanup(run_status(project))
project_name, sync_report = run_with_cleanup(
run_status(project, wait=wait, timeout=timeout)
)
if json_output:
print(json.dumps(sync_report.model_dump(mode="json"), indent=2, default=str))
else:
display_changes(project_name, "Status", sync_report, verbose)
except StatusTimeout as e:
# Trigger: --wait deadline passed before the project finished indexing
# Why: callers depend on exit code 1 to detect that indexing did not
# complete in time, while still getting a clear machine/human message
# Outcome: emit the timeout message (JSON-shaped under --json) and exit 1
if json_output:
print(json.dumps({"error": str(e)}, indent=2))
else:
console.print(f"[red]Error: {e}[/red]")
raise typer.Exit(code=1)
except (ValueError, ToolError) as e:
if json_output:
print(json.dumps({"error": str(e)}, indent=2))
+179 -12
View File
@@ -14,17 +14,10 @@ from loguru import logger
from basic_memory.cli.app import app
from basic_memory.cli.commands.command_utils import run_with_cleanup
from basic_memory.cli.commands.routing import force_routing, validate_routing_flags
from basic_memory.mcp.tools import build_context as mcp_build_context
from basic_memory.mcp.tools import edit_note as mcp_edit_note
from basic_memory.mcp.tools import list_memory_projects as mcp_list_projects
from basic_memory.mcp.tools import list_workspaces as mcp_list_workspaces
from basic_memory.mcp.tools import read_note as mcp_read_note
from basic_memory.mcp.tools import recent_activity as mcp_recent_activity
from basic_memory.mcp.tools import schema_diff as mcp_schema_diff
from basic_memory.mcp.tools import schema_infer as mcp_schema_infer
from basic_memory.mcp.tools import schema_validate as mcp_schema_validate
from basic_memory.mcp.tools import search_notes as mcp_search
from basic_memory.mcp.tools import write_note as mcp_write_note
# MCP tool functions are imported inside each command: importing
# basic_memory.mcp.tools loads the entire tool stack (fastmcp, mcp SDK,
# SQLAlchemy), which would slow every CLI invocation, including --help (#886).
tool_app = typer.Typer()
app.add_typer(tool_app, name="tool", help="Access to MCP tools via CLI")
@@ -40,6 +33,26 @@ def _print_json(result: Any) -> None:
print(json.dumps(result, indent=2, ensure_ascii=True, default=str))
def _delete_note_failure_message(result: dict[str, Any]) -> str | None:
"""Return the CLI failure message for delete-note JSON results, if any."""
error = result.get("error")
if error:
return str(error)
failed_deletes = result.get("failed_deletes")
# Trigger: directory deletion can partially fail without raising from the service.
# Why: cleanup scripts need a non-zero exit when files remain undeleted.
# Outcome: the CLI fails even if older MCP JSON did not include an error field.
if (
result.get("is_directory") is True
and isinstance(failed_deletes, int)
and failed_deletes > 0
):
return f"Directory delete incomplete: {failed_deletes} file(s) failed"
return None
# --- Commands ---
@@ -56,6 +69,16 @@ def write_note(
tags: Annotated[
Optional[List[str]], typer.Option(help="A list of tags to apply to the note")
] = None,
note_type: Annotated[
str,
typer.Option(
"--type",
help=(
"Note type stored in frontmatter (e.g. 'guide', 'report'). "
"A 'type:' in the note's own content frontmatter takes precedence."
),
),
] = "note",
project: Annotated[
Optional[str],
typer.Option(
@@ -69,6 +92,11 @@ def write_note(
help="Project external_id (UUID). Takes precedence over --project; use to disambiguate same-named projects across cloud workspaces.",
),
] = None,
overwrite: bool = typer.Option(
False,
"--overwrite",
help="Replace an existing note on conflict (matches MCP write_note overwrite=True)",
),
local: bool = typer.Option(
False, "--local", help="Force local API routing (ignore cloud mode)"
),
@@ -79,9 +107,14 @@ def write_note(
Examples:
bm tool write-note --title "My Note" --folder "notes" --content "Note content"
bm tool write-note --title "My Guide" --folder "notes" --content "..." --type guide
echo "content" | bm tool write-note --title "My Note" --folder "notes"
bm tool write-note --title "My Note" --folder "notes" --overwrite
bm tool write-note --title "My Note" --folder "notes" --local
"""
# Deferred: loading the MCP tool stack at module import slows CLI startup (#886).
from basic_memory.mcp.tools import write_note as mcp_write_note
try:
validate_routing_flags(local, cloud)
@@ -111,9 +144,23 @@ def write_note(
project=project,
project_id=project_id,
tags=tags,
note_type=note_type,
overwrite=overwrite,
output_format="json",
)
)
# MCP tool returns an error field on failure in JSON mode (e.g.
# NOTE_ALREADY_EXISTS on a blocked overwrite, SECURITY_VALIDATION_ERROR).
# Trigger: result carries a non-empty `error`.
# Why: parity with delete-note/edit-note/search-notes so exit-code-driven
# scripts detect a failed/blocked write instead of seeing exit 0.
# Outcome: print the error to stderr and exit non-zero.
if isinstance(result, dict) and result.get("error"):
typer.echo(f"Error: {result['error']}", err=True)
_print_json(result)
raise typer.Exit(1)
_print_json(result)
except ValueError as e:
typer.echo(f"Error: {e}", err=True)
@@ -154,6 +201,9 @@ def read_note(
bm tool read-note my-note
bm tool read-note my-note --include-frontmatter
"""
# Deferred: loading the MCP tool stack at module import slows CLI startup (#886).
from basic_memory.mcp.tools import read_note as mcp_read_note
try:
validate_routing_flags(local, cloud)
@@ -167,6 +217,19 @@ def read_note(
output_format="json",
)
)
# MCP tool returns an error field on failure in JSON mode (e.g.
# SECURITY_VALIDATION_ERROR on a path-traversal identifier). A genuine
# not-found returns null fields with no `error` key, so it still exits 0.
# Trigger: result carries a non-empty `error`.
# Why: parity with edit-note/delete-note/search-notes so a blocked read
# surfaces a non-zero exit instead of looking like success.
# Outcome: print the error to stderr and exit non-zero.
if isinstance(result, dict) and result.get("error"):
typer.echo(f"Error: {result['error']}", err=True)
_print_json(result)
raise typer.Exit(1)
_print_json(result)
except ValueError as e:
typer.echo(f"Error: {e}", err=True)
@@ -178,6 +241,69 @@ def read_note(
raise
@tool_app.command("delete-note")
def delete_note(
identifier: str,
is_directory: bool = typer.Option(
False, "--is-directory", help="Delete a directory instead of a single note"
),
project: Annotated[
Optional[str],
typer.Option(help="The project to use. If not provided, the default project will be used."),
] = None,
project_id: Annotated[
Optional[str],
typer.Option(
"--project-id",
help="Project external_id (UUID). Takes precedence over --project; use to disambiguate same-named projects across cloud workspaces.",
),
] = None,
local: bool = typer.Option(
False, "--local", help="Force local API routing (ignore cloud mode)"
),
cloud: bool = typer.Option(False, "--cloud", help="Force cloud API routing"),
) -> None:
"""Delete a note or directory from the knowledge base.
Examples:
bm tool delete-note notes/old-draft
bm tool delete-note docs/archive --is-directory
"""
# Deferred: loading the MCP tool stack at module import slows CLI startup (#886).
from basic_memory.mcp.tools import delete_note as mcp_delete_note
try:
validate_routing_flags(local, cloud)
with force_routing(local=local, cloud=cloud):
result = run_with_cleanup(
mcp_delete_note(
identifier=identifier,
is_directory=is_directory,
project=project,
project_id=project_id,
output_format="json",
)
)
if isinstance(result, dict):
failure_message = _delete_note_failure_message(result)
if failure_message:
typer.echo(f"Error: {failure_message}", err=True)
raise typer.Exit(1)
_print_json(result)
except ValueError as e:
typer.echo(f"Error: {e}", err=True)
raise typer.Exit(1)
except Exception as e: # pragma: no cover
if not isinstance(e, typer.Exit):
typer.echo(f"Error during delete_note: {e}", err=True)
raise typer.Exit(1)
raise
@tool_app.command()
def edit_note(
identifier: str,
@@ -221,6 +347,9 @@ def edit_note(
bm tool edit-note my-note --operation find_replace --find-text "old" --content "new"
bm tool edit-note my-note --operation replace_section --section "## Notes" --content "updated"
"""
# Deferred: loading the MCP tool stack at module import slows CLI startup (#886).
from basic_memory.mcp.tools import edit_note as mcp_edit_note
try:
validate_routing_flags(local, cloud)
@@ -288,6 +417,9 @@ def build_context(
bm tool build-context memory://specs/search
bm tool build-context specs/search --depth 2 --timeframe 30d
"""
# Deferred: loading the MCP tool stack at module import slows CLI startup (#886).
from basic_memory.mcp.tools import build_context as mcp_build_context
try:
validate_routing_flags(local, cloud)
@@ -324,7 +456,9 @@ def recent_activity(
"7d", "--timeframe", help="Timeframe filter (e.g., '7d', '1 week')"
),
page: int = typer.Option(1, "--page", help="Page number for pagination"),
page_size: int = typer.Option(50, "--page-size", help="Number of results per page"),
# Match the MCP recent_activity default (page_size=10) so identical default
# invocations return the same number of rows from CLI and MCP.
page_size: int = typer.Option(10, "--page-size", help="Number of results per page"),
project: Annotated[
Optional[str],
typer.Option(help="The project to use. If not provided, the default project will be used."),
@@ -349,6 +483,9 @@ def recent_activity(
bm tool recent-activity --timeframe 30d --page-size 20
bm tool recent-activity --type entity --type observation
"""
# Deferred: loading the MCP tool stack at module import slows CLI startup (#886).
from basic_memory.mcp.tools import recent_activity as mcp_recent_activity
try:
validate_routing_flags(local, cloud)
@@ -409,6 +546,16 @@ def search_notes(
help="Filter by search item type: entity, observation, relation (repeatable)",
),
] = None,
categories: Annotated[
Optional[List[str]],
typer.Option(
"--category",
help=(
"Filter observation results to exact categories (repeatable); "
"pair with --entity-type observation"
),
),
] = None,
meta: Annotated[
Optional[List[str]],
typer.Option("--meta", help="Filter by frontmatter key=value (repeatable)"),
@@ -443,7 +590,11 @@ def search_notes(
bm tool search-notes --permalink "specs/*"
bm tool search-notes --tag python --tag async
bm tool search-notes --meta status=draft
bm tool search-notes "auth" --entity-type observation --category requirement
"""
# Deferred: loading the MCP tool stack at module import slows CLI startup (#886).
from basic_memory.mcp.tools import search_notes as mcp_search
try:
validate_routing_flags(local, cloud)
@@ -508,6 +659,7 @@ def search_notes(
page_size=page_size,
note_types=note_types,
entity_types=entity_types,
categories=categories,
metadata_filters=metadata_filters,
tags=tags,
status=status,
@@ -548,6 +700,9 @@ def list_projects(
bm tool list-projects
bm tool list-projects --local
"""
# Deferred: loading the MCP tool stack at module import slows CLI startup (#886).
from basic_memory.mcp.tools import list_memory_projects as mcp_list_projects
try:
validate_routing_flags(local, cloud)
@@ -581,6 +736,9 @@ def list_workspaces(
bm tool list-workspaces
bm tool list-workspaces --cloud
"""
# Deferred: loading the MCP tool stack at module import slows CLI startup (#886).
from basic_memory.mcp.tools import list_workspaces as mcp_list_workspaces
try:
validate_routing_flags(local, cloud)
@@ -633,6 +791,9 @@ def schema_validate(
bm tool schema-validate people/ada-lovelace.md
bm tool schema-validate --project research
"""
# Deferred: loading the MCP tool stack at module import slows CLI startup (#886).
from basic_memory.mcp.tools import schema_validate as mcp_schema_validate
try:
validate_routing_flags(local, cloud)
@@ -701,6 +862,9 @@ def schema_infer(
bm tool schema-infer meeting --threshold 0.5
bm tool schema-infer person --project research
"""
# Deferred: loading the MCP tool stack at module import slows CLI startup (#886).
from basic_memory.mcp.tools import schema_infer as mcp_schema_infer
try:
validate_routing_flags(local, cloud)
@@ -757,6 +921,9 @@ def schema_diff(
bm tool schema-diff person
bm tool schema-diff person --project research
"""
# Deferred: loading the MCP tool stack at module import slows CLI startup (#886).
from basic_memory.mcp.tools import schema_diff as mcp_schema_diff
try:
validate_routing_flags(local, cloud)
@@ -0,0 +1,23 @@
"""Top-level `workspace` stub that redirects users to `bm cloud workspace`."""
import typer
from basic_memory.cli.app import app
# Trigger: user runs `bm workspace`, `bm workspace list`, etc.
# Why: workspace verbs live under `bm cloud workspace`; a bare top-level miss
# only emits Typer's terse "No such command 'workspace'." with exit 2.
# Outcome: allow_extra_args + ignore_unknown_options absorb any trailing tokens
# (e.g. `list`, `set-default foo`) so every invocation reaches this body and
# prints actionable guidance instead of being rejected as bad usage.
@app.command(
"workspace",
context_settings={"allow_extra_args": True, "ignore_unknown_options": True},
)
def workspace_stub(ctx: typer.Context) -> None:
"""Point users to the real workspace verbs under `bm cloud workspace`."""
typer.echo("'bm workspace' is not a command. Workspace verbs live under 'bm cloud workspace':")
typer.echo(" bm cloud workspace list")
typer.echo(" bm cloud workspace set-default <name>")
raise typer.Exit(1)
+2
View File
@@ -24,6 +24,7 @@ if not _version_only_invocation(sys.argv[1:]):
import_claude_conversations,
import_claude_projects,
import_memory_json,
man,
mcp,
orphans,
project,
@@ -31,6 +32,7 @@ if not _version_only_invocation(sys.argv[1:]):
status,
tool,
update,
workspace,
)
warnings.filterwarnings("ignore") # pragma: no cover
+63 -3
View File
@@ -4,6 +4,7 @@ import importlib.util
import json
import os
import shutil
import threading
from dataclasses import dataclass
from datetime import datetime
from enum import Enum
@@ -248,7 +249,19 @@ class BasicMemoryConfig(BaseSettings):
)
semantic_embedding_dimensions: int | None = Field(
default=None,
description="Embedding vector dimensions. Auto-detected from provider if not set (384 for FastEmbed, 1536 for OpenAI).",
description=(
"Embedding vector dimensions. Uses provider defaults when unset "
"(384 for FastEmbed, 1536 for OpenAI and LiteLLM OpenAI default); "
"required for custom LiteLLM models."
),
)
semantic_embedding_forward_dimensions: bool | None = Field(
default=None,
description=(
"LiteLLM-only override for sending semantic_embedding_dimensions as a "
"provider-side output-size request parameter. When unset, Basic Memory "
"auto-detects model strings such as text-embedding-3."
),
)
# Trigger: full local rebuilds spend most of their time waiting behind shared
# embed flushes, not constructing vectors themselves.
@@ -266,6 +279,20 @@ class BasicMemoryConfig(BaseSettings):
description="Maximum number of concurrent provider requests for batched embedding generation when the active provider supports request-level concurrency.",
gt=0,
)
semantic_embedding_document_input_type: str | None = Field(
default=None,
description=(
"Optional LiteLLM input_type for indexed document/passages. "
"Use with asymmetric embedding models such as Cohere or NVIDIA retrieval models."
),
)
semantic_embedding_query_input_type: str | None = Field(
default=None,
description=(
"Optional LiteLLM input_type for search queries. "
"Use with asymmetric embedding models such as Cohere or NVIDIA retrieval models."
),
)
semantic_embedding_sync_batch_size: int = Field(
default=2,
description="Batch size for vector sync orchestration flushes.",
@@ -631,6 +658,26 @@ class BasicMemoryConfig(BaseSettings):
entry = self.projects.get(project_name)
return entry.mode if entry else ProjectMode.CLOUD
def is_locally_syncable(self, project_name: str, project_path: str) -> bool:
"""Whether a project should be synced/watched on the local filesystem.
Both conditions are required (issue #949):
* The project is present in config. Config is the source of truth, so a
stale database row that was removed from config but whose deletion
has not yet been reconciled, or whose reconciliation failed must
not be synced even though it still has a real directory on disk.
* Its path is absolute. An empty or relative path resolves against the
process cwd, so syncing it would adopt whatever directory the server
was launched from as the project root and mutate unrelated files.
Cloud-only projects (empty/slug path) and cloud projects with a real
local bisync copy (absolute path) are handled correctly by these two
conditions, so no separate mode check is needed.
"""
entry = self.projects.get(project_name)
return entry is not None and Path(project_path).is_absolute()
def set_project_mode(self, project_name: str, mode: ProjectMode) -> None:
"""Set the routing mode for a project.
@@ -1073,8 +1120,21 @@ def save_basic_memory_config(file_path: Path, config: BasicMemoryConfig) -> None
_secure_config_dir(file_path.parent)
# Use model_dump with mode='json' to serialize datetime objects properly
config_dict = config.model_dump(mode="json")
file_path.write_text(json.dumps(config_dict, indent=2))
_secure_config_file(file_path)
# Trigger: long-lived readers (MCP stdio server config reload, background
# auto-update threads) re-read config.json whenever its mtime changes,
# concurrently with CLI commands saving it.
# Why: writing the destination in place truncates it first, so a concurrent
# reader can observe empty/partial JSON and load_config() exits the process.
# Outcome: write a sibling temp file (unique per process/thread so parallel
# savers cannot interleave) and publish atomically via os.replace — readers
# always see either the old or the new complete document. (#940)
tmp_path = file_path.parent / f"{file_path.name}.{os.getpid()}.{threading.get_ident()}.tmp"
try:
tmp_path.write_text(json.dumps(config_dict, indent=2))
_secure_config_file(tmp_path)
os.replace(tmp_path, file_path)
finally:
tmp_path.unlink(missing_ok=True)
except Exception as e: # pragma: no cover
logger.error(f"Failed to save config: {e}")
+88 -19
View File
@@ -1,7 +1,7 @@
import asyncio
import os
import sys
from contextlib import asynccontextmanager
from contextlib import asynccontextmanager, suppress
from enum import Enum, auto
from pathlib import Path
from typing import AsyncGenerator, Optional
@@ -19,7 +19,7 @@ from sqlalchemy.ext.asyncio import (
AsyncEngine,
async_scoped_session,
)
from sqlalchemy.pool import NullPool
from sqlalchemy.pool import AsyncAdaptedQueuePool, NullPool
from basic_memory.repository.postgres_search_repository import PostgresSearchRepository
from basic_memory.repository.sqlite_search_repository import SQLiteSearchRepository
@@ -39,6 +39,44 @@ from basic_memory.repository.sqlite_search_repository import SQLiteSearchReposit
if sys.platform == "win32": # pragma: no cover
asyncio.set_event_loop_policy(asyncio.WindowsSelectorEventLoopPolicy())
def maybe_install_uvloop(config: BasicMemoryConfig) -> bool:
"""Install the uvloop event-loop policy for the Postgres backend.
Trigger: process entrypoint starting with database_backend == postgres,
uvloop importable, and a non-Windows platform.
Why: asyncpg engine teardown (engine.dispose()) races the stdlib asyncio
loop shutdown and surfaces "IndexError: pop from an empty deque" from
base_events._run_once (see #831/#877). uvloop's C scheduler has no
self._ready.popleft() codepath, so that class of crash cannot fire under it.
Outcome: Postgres deployments run on uvloop; SQLite users keep the default
loop (no behavior change, smaller blast radius). Must run before the event
loop is created, i.e. before asyncio.run().
Returns:
True if the uvloop policy was installed, False otherwise.
"""
# uvloop is not available on Windows; the default loop already differs there.
if sys.platform == "win32": # pragma: no cover
return False
# Limit the change to the backend that actually hits the asyncpg dispose race.
if config.database_backend != DatabaseBackend.POSTGRES:
return False
# Deferred import: uvloop is an optional, platform-gated dependency and the
# default (SQLite) path must not require it to be installed.
try:
import uvloop
except ImportError: # pragma: no cover
logger.warning("uvloop not available - using default event loop for Postgres backend")
return False
asyncio.set_event_loop_policy(uvloop.EventLoopPolicy())
logger.info("Installed uvloop event-loop policy for Postgres backend")
return True
# Module level state
_engine: Optional[AsyncEngine] = None
_session_maker: Optional[async_sessionmaker[AsyncSession]] = None
@@ -178,19 +216,31 @@ def _create_sqlite_engine(db_url: str, db_type: DatabaseType) -> AsyncEngine:
"isolation_level": None, # Use autocommit mode
}
)
if db_type == DatabaseType.MEMORY:
# Trigger: an in-memory SQLite URL would default to StaticPool, which hands the
# same DBAPI connection to every concurrently checked-out session.
# Why: concurrent asyncio tasks then share one transaction scope — a rollback
# issued by one session (scoped_session exception handling or the pool's
# reset-on-return) silently destroys another session's uncommitted writes (#940).
# Outcome: a single-connection blocking queue pool keeps the in-memory database
# alive for the engine's lifetime while serializing sessions at transaction
# granularity, restoring the isolation the repositories assume.
engine = create_async_engine(
db_url,
connect_args=connect_args,
poolclass=AsyncAdaptedQueuePool,
pool_size=1,
max_overflow=0,
)
elif os.name == "nt":
# Use NullPool for Windows filesystem databases to avoid connection pooling issues
# Important: Do NOT use NullPool for in-memory databases as it will destroy the database
# between connections
if db_type == DatabaseType.FILESYSTEM:
engine = create_async_engine(
db_url,
connect_args=connect_args,
poolclass=NullPool, # Disable connection pooling on Windows
echo=False,
)
else:
# In-memory databases need connection pooling to maintain state
engine = create_async_engine(db_url, connect_args=connect_args)
engine = create_async_engine(
db_url,
connect_args=connect_args,
poolclass=NullPool, # Disable connection pooling on Windows
echo=False,
)
else:
engine = create_async_engine(db_url, connect_args=connect_args)
@@ -320,7 +370,15 @@ async def shutdown_db() -> None: # pragma: no cover
global _engine, _session_maker
if _engine:
await _engine.dispose()
# Trigger: teardown can run while the surrounding task is being cancelled
# (e.g. lifespan shutdown, unshielded CLI cleanup).
# Why: a cancellation landing mid-dispose surfaces the asyncpg
# "IndexError: pop from an empty deque" race (#831/#877); shielding lets
# dispose finish atomically, and suppressing CancelledError keeps a
# cancelled shutdown from re-raising the underlying race.
# Outcome: connections always close cleanly even under cancellation.
with suppress(asyncio.CancelledError):
await asyncio.shield(_engine.dispose())
_engine = None
_session_maker = None
@@ -364,7 +422,14 @@ async def engine_session_factory(
yield created_engine, created_session_maker
finally:
await created_engine.dispose()
# Trigger: context-manager teardown can run while the surrounding task is
# being cancelled (e.g. a test aborting mid-fixture).
# Why: on the asyncpg backend a cancellation landing mid-dispose surfaces
# the "IndexError: pop from an empty deque" race (#831/#877); shield the
# dispose and suppress CancelledError to match the other dispose seams.
# Outcome: the per-context engine always disposes cleanly under cancellation.
with suppress(asyncio.CancelledError):
await asyncio.shield(created_engine.dispose())
# Only clear module-level globals if they still point to this context's
# engine/session. This avoids clobbering newer globals from other callers.
@@ -433,7 +498,11 @@ async def run_migrations(
# Trigger: run_migrations() created a temporary engine while module-level
# session maker was not initialized.
# Why: temporary aiosqlite worker threads can outlive CLI command execution
# and block process shutdown if the engine is not disposed.
# Outcome: always dispose temporary engines after migration work completes.
# and block process shutdown if the engine is not disposed. On the asyncpg
# backend a cancellation landing mid-dispose surfaces the same "IndexError:
# pop from an empty deque" race as the other dispose seams (#831/#877), so
# shield the dispose and suppress CancelledError to match them.
# Outcome: always dispose temporary engines cleanly, even under cancellation.
if temp_engine is not None:
await temp_engine.dispose()
with suppress(asyncio.CancelledError):
await asyncio.shield(temp_engine.dispose())
+68 -22
View File
@@ -302,26 +302,76 @@ async def format_file(
return None
# A frontmatter fence is a line containing exactly `---`, optionally followed by
# trailing horizontal whitespace. Anchoring to a full line (rather than a bare
# substring/`startswith`) prevents single-line content like
# `---\nstatus: active\n---\nBody` — where `\n` is a literal backslash-n, not a
# newline — from being misread as frontmatter. See issue #972.
_FENCE_RE = re.compile(r"^---[ \t]*$")
def _split_frontmatter(content: str) -> Optional[tuple[str, str]]:
"""Split content into (yaml_block, body) when it opens with a line-anchored fence.
The opening fence must be the very first line and the closing fence must be a
later line, each matching exactly `---` (with optional trailing whitespace).
Returns:
A `(yaml_block, body)` tuple when a complete fenced block is present, or
``None`` when the content does not open with a frontmatter fence.
Raises:
ParseError: If the content opens with a fence but has no closing fence.
"""
lines = content.splitlines(keepends=True)
# Skip leading blank lines: a document may begin with whitespace before the
# opening fence (e.g. a heredoc/dedented string starting with a newline). This
# does NOT relax line-anchoring — the opening fence must still be the first
# non-blank line, all on its own, so a single-line `---\\nstatus...` (literal
# backslash-n) is still rejected. See issue #972.
start = 0
while start < len(lines) and lines[start].strip() == "":
start += 1
if start >= len(lines) or not _FENCE_RE.match(lines[start].rstrip("\r\n")):
return None
# Find the closing fence on its own line somewhere after the opening fence.
for index in range(start + 1, len(lines)):
if _FENCE_RE.match(lines[index].rstrip("\r\n")):
yaml_block = "".join(lines[start + 1 : index])
body = "".join(lines[index + 1 :])
return yaml_block, body
raise ParseError("Invalid frontmatter format")
def has_frontmatter(content: str) -> bool:
"""
Check if content contains valid YAML frontmatter.
Frontmatter requires `---` fences on their own lines; an inline `---` (such as
a single-line string that merely starts with the characters `---`) is not
frontmatter.
Args:
content: Content to check
Returns:
True if content has valid frontmatter markers (---), False otherwise
True if content has line-anchored frontmatter fences, False otherwise
"""
if not content:
return False
# Strip BOM before checking for frontmatter markers
content = strip_bom(content).strip()
if not content.startswith("---"):
content = strip_bom(content)
try:
return _split_frontmatter(content) is not None
except ParseError:
# An opening fence with no closing fence is not usable frontmatter.
return False
return "---" in content[3:]
def parse_frontmatter(content: str) -> Dict[str, Any]:
"""
@@ -339,17 +389,14 @@ def parse_frontmatter(content: str) -> Dict[str, Any]:
try:
# Strip BOM before parsing frontmatter
content = strip_bom(content)
if not content.strip().startswith("---"):
split = _split_frontmatter(content)
if split is None:
raise ParseError("Content has no frontmatter")
# Split on first two occurrences of ---
parts = content.split("---", 2)
if len(parts) < 3:
raise ParseError("Invalid frontmatter format")
yaml_block, _ = split
# Parse YAML
try:
frontmatter = yaml.safe_load(parts[1])
frontmatter = yaml.safe_load(yaml_block)
# Handle empty frontmatter (None from yaml.safe_load)
if frontmatter is None:
return {}
@@ -381,18 +428,17 @@ def remove_frontmatter(content: str) -> str:
ParseError: If content starts with frontmatter marker but is malformed
"""
# Strip BOM before processing
content = strip_bom(content).strip()
content = strip_bom(content)
# Return as-is if no frontmatter marker
if not content.startswith("---"):
return content
split = _split_frontmatter(content)
# Trigger: content does not open with a line-anchored fence
# Why: inline `---` is ordinary content, not frontmatter (issue #972)
# Outcome: return the content untouched (stripped to preserve prior behavior)
if split is None:
return content.strip()
# Split on first two occurrences of ---
parts = content.split("---", 2)
if len(parts) < 3:
raise ParseError("Invalid frontmatter format")
return parts[2].strip()
_, body = split
return body.strip()
def dump_frontmatter(post: frontmatter.Post) -> str:
+9
View File
@@ -7,6 +7,15 @@ from typing import Set
from basic_memory.config import resolve_data_dir
# Marker shared by the API ignored-path rejection detail and MCP-side error handling.
# The sync-file endpoint embeds it in its 400 detail and edit_note's disk recovery
# matches on it, so "exists but ignored" stays distinguishable from generic rejections
# without duplicating message text across layers.
IGNORED_PATH_REJECTION_DETAIL = (
"matches Basic Memory ignore rules (.bmignore or project .gitignore)"
)
# Common directories and patterns to ignore by default
# These are used as fallback if .bmignore doesn't exist
DEFAULT_IGNORE_PATTERNS = {
+7 -1
View File
@@ -493,8 +493,14 @@ class BatchIndexer:
async def resolve_relation(relation: Relation) -> int:
async with semaphore:
try:
# strict=True for deferred resolution: only fill in to_id on an
# exact permalink/title/file_path match. Fuzzy fallback would silently
# resolve ambiguous links to whichever entity shares tokens with the
# link text, mismatching this with the sync_service forward-reference
# path and producing confidently-wrong graph edges. See
# sync_service.resolve_forward_references for the same change.
resolved_entity = await self.entity_service.link_resolver.resolve_link(
relation.to_name
relation.to_name, strict=True
)
if resolved_entity is None or resolved_entity.id == relation.from_id:
return 0
+1
View File
@@ -0,0 +1 @@
.so man1/bm.1
+134
View File
@@ -0,0 +1,134 @@
.TH BM 1 "2026-06-11" "basic-memory" "Basic Memory Manual"
.SH NAME
bm \- local-first knowledge base for humans and AI agents
.SH SYNOPSIS
.B bm
.I COMMAND
.RI [ ARGS ]...
.br
.B basic-memory
.I COMMAND
.RI [ ARGS ]...
.SH DESCRIPTION
.B bm
manages Basic Memory projects: plain markdown files that form a knowledge
graph. Files are the source of truth; SQLite provides indexing and
full-text search; the same operations are exposed to AI agents over the
Model Context Protocol (MCP) and to humans and scripts through this CLI.
.PP
Notes use semantic markdown: observations
.RB ( "\- [category] text #tag" )
and relations
.RB ( "\- relation_type [[Target]]" )
become queryable graph structure. Projects route independently to the
local API or to Basic Memory Cloud.
.SH COMMANDS
Knowledge operations:
.TP
.B bm tool
CLI access to the MCP tools (write-note, read-note, search-notes,
build-context, ...). These emit JSON and are the scriptable surface.
.TP
.B bm status
Show sync status between files and the database.
.TP
.B bm reindex
Index local file changes and rebuild search/embeddings. This is the manual
sync trigger when no MCP server is running.
.TP
.B bm doctor
Run end-to-end file/database consistency checks.
.TP
.B bm orphans
List entities with no relations in the knowledge graph.
.TP
.B bm format
Run configured formatters over note files.
.PP
Projects and schemas:
.TP
.B bm project
Add, remove, list projects; set the default; flip a project between local
and cloud routing.
.TP
.B bm schema
List, validate, infer, and drift-check Picoschema note-type contracts.
.PP
Data and cloud:
.TP
.B bm import
Import from ChatGPT, Claude, or memory.json exports. Imports write files;
run
.B bm reindex
afterwards.
.TP
.B bm cloud
Authenticate, sync (push/pull/sync/bisync), snapshots, and team workspace
administration.
.PP
Infrastructure:
.TP
.B bm mcp
Run the MCP server (hosts the live file watcher).
.TP
.B bm man
Manage these man pages
.RB ( "bm man install" ).
.TP
.B bm reset
Drop and recreate the database (destructive).
.PP
Most commands accept
.B \-\-project
.I NAME
to target a project and
.BR \-\-local / \-\-cloud
to override routing.
.SH EXAMPLES
Write and find a note from the shell:
.PP
.nf
.RS
echo "# Standup notes" | bm tool write-note \\
\-\-title "Standup" \-\-folder notes
bm tool search-notes "standup"
.RE
.fi
.PP
Pick up files created outside the tools:
.PP
.nf
.RS
bm status # shows pending changes
bm reindex # indexes them
.RE
.fi
.SH FILES
.TP
.I ~/.basic-memory/config.json
Projects, default project, per-project routing modes, cloud settings.
.TP
.I ~/.basic-memory/memory.db
SQLite index (derived; safe to rebuild with bm reindex).
.SH ENVIRONMENT
.TP
.B BASIC_MEMORY_FORCE_LOCAL
Force local routing regardless of cloud mode.
.TP
.B BASIC_MEMORY_LOG_LEVEL
Logging verbosity (e.g. DEBUG).
.SH SEE ALSO
Full manual (machine-readable, agent-traversable): the
.I manual
Basic Memory project \(em see docs/manual-pages.md in the repository.
Documentation: https://docs.basicmemory.com
.PP
For AI agents: the complete tool reference lives in the manual project as
section\-3 pages (write-note(3), search-notes(3), ...), queryable via
.B search_notes
with
.BR "metadata_filters={\(dqtype\(dq: \(dqmanpage\(dq}" .
.SH BUGS
https://github.com/basicmachines-co/basic-memory/issues
.SH AUTHORS
Basic Machines (https://basicmachines.co)
+38 -13
View File
@@ -5,14 +5,16 @@ from dataclasses import dataclass
from threading import RLock
from typing import TYPE_CHECKING, Annotated, Any, AsyncIterator, Callable, Optional
from fastapi import Depends, FastAPI, Request
from httpx import ASGITransport, AsyncClient, Timeout
from loguru import logger
import logfire
from basic_memory.config import ConfigManager, ProjectMode
from basic_memory.config import ConfigManager, ProjectMode, has_cloud_credentials
if TYPE_CHECKING:
# FastAPI is only needed when a request routes through the local ASGI
# transport; importing it at module level costs ~0.1s on every CLI start (#886).
from fastapi import FastAPI
from sqlalchemy.ext.asyncio import AsyncEngine, AsyncSession, async_sessionmaker
LocalDatabaseState = tuple["AsyncEngine", "async_sessionmaker[AsyncSession]"]
@@ -28,8 +30,8 @@ class _PreparedLocalAsgiDatabase:
_prepared_local_asgi_database_lock = RLock()
_prepared_local_asgi_database_prepare_locks: dict[FastAPI, Lock] = {}
_prepared_local_asgi_databases: dict[FastAPI, _PreparedLocalAsgiDatabase] = {}
_prepared_local_asgi_database_prepare_locks: dict["FastAPI", Lock] = {}
_prepared_local_asgi_databases: dict["FastAPI", _PreparedLocalAsgiDatabase] = {}
def _force_local_mode() -> bool:
@@ -57,7 +59,7 @@ def _build_timeout() -> Timeout:
)
def _build_asgi_client(app: FastAPI, timeout: Timeout) -> AsyncClient:
def _build_asgi_client(app: "FastAPI", timeout: Timeout) -> AsyncClient:
"""Create a local ASGI client for an already-prepared FastAPI app."""
from basic_memory.workspace_context import workspace_permalink_headers
@@ -71,7 +73,7 @@ def _build_asgi_client(app: FastAPI, timeout: Timeout) -> AsyncClient:
)
def _get_prepared_local_asgi_database_prepare_lock(app: FastAPI) -> Lock:
def _get_prepared_local_asgi_database_prepare_lock(app: "FastAPI") -> Lock:
"""Get the async lock that serializes first-time DB preparation for an app."""
with _prepared_local_asgi_database_lock:
prepare_lock = _prepared_local_asgi_database_prepare_locks.get(app)
@@ -82,8 +84,10 @@ def _get_prepared_local_asgi_database_prepare_lock(app: FastAPI) -> Lock:
@asynccontextmanager
async def _resolve_local_asgi_database(app: FastAPI) -> AsyncIterator[LocalDatabaseState]:
async def _resolve_local_asgi_database(app: "FastAPI") -> AsyncIterator[LocalDatabaseState]:
"""Resolve database state for a local ASGI request."""
# Imported on first local-ASGI use so CLI startup never pays for FastAPI (#886).
from fastapi import Depends, Request
from fastapi.dependencies.utils import get_dependant, solve_dependencies
from basic_memory.deps import get_engine_factory
@@ -127,7 +131,7 @@ async def _resolve_local_asgi_database(app: FastAPI) -> AsyncIterator[LocalDatab
yield await resolve_database_state(**solved.values)
def _retain_prepared_local_asgi_database(app: FastAPI) -> bool:
def _retain_prepared_local_asgi_database(app: "FastAPI") -> bool:
"""Retain an active local ASGI database preparation if one exists."""
with _prepared_local_asgi_database_lock:
active = _prepared_local_asgi_databases.get(app)
@@ -139,7 +143,7 @@ def _retain_prepared_local_asgi_database(app: FastAPI) -> bool:
def _install_prepared_local_asgi_database(
app: FastAPI,
app: "FastAPI",
database_state: LocalDatabaseState,
dependency_context: AbstractAsyncContextManager[LocalDatabaseState],
) -> None:
@@ -163,7 +167,7 @@ def _install_prepared_local_asgi_database(
)
def _restore_local_asgi_state_attribute(app: FastAPI, name: str, previous_value: object) -> None:
def _restore_local_asgi_state_attribute(app: "FastAPI", name: str, previous_value: object) -> None:
"""Restore a FastAPI app.state attribute captured before local ASGI preparation."""
if previous_value is _MISSING_STATE_VALUE:
if hasattr(app.state, name):
@@ -173,7 +177,7 @@ def _restore_local_asgi_state_attribute(app: FastAPI, name: str, previous_value:
def _release_prepared_local_asgi_database(
app: FastAPI,
app: "FastAPI",
) -> AbstractAsyncContextManager[LocalDatabaseState] | None:
"""Release local ASGI database state after a client context exits."""
with _prepared_local_asgi_database_lock:
@@ -196,7 +200,7 @@ def _release_prepared_local_asgi_database(
@asynccontextmanager
async def _prepared_local_asgi_database(app: FastAPI) -> AsyncIterator[None]:
async def _prepared_local_asgi_database(app: "FastAPI") -> AsyncIterator[None]:
"""Initialize local ASGI database state before the first request."""
prepare_lock = _get_prepared_local_asgi_database_prepare_lock(app)
async with prepare_lock:
@@ -368,7 +372,8 @@ async def get_client(
1. Factory injection.
2. Explicit routing flags (--local/--cloud).
3. Per-project mode routing when project_name is provided.
4. Local ASGI transport by default.
4. Cloud routing when a workspace selector is provided.
5. Local ASGI transport by default.
"""
if _client_factory:
async with _client_factory(workspace=workspace) as client:
@@ -428,6 +433,26 @@ async def get_client(
yield client
return
# --- Workspace-selector routing ---
# Trigger: caller passed a cloud workspace selector and nothing above routed.
# Why: a workspace names a cloud tenant — silently serving the request from
# the local ASGI app sent writes to the wrong destination (#954: a cloud
# project create either failed on the cloud-style path or landed locally).
# Outcome: route to the cloud proxy when credentials exist; without
# credentials fail fast instead of pretending the operation succeeded.
if workspace is not None:
if not has_cloud_credentials(config):
raise RuntimeError(
f"A cloud workspace was requested ('{workspace}') but no cloud "
"credentials were found. Run 'bm cloud login' or "
"'bm cloud set-key <key>' first, or omit the workspace selector "
"for a local operation."
)
logger.debug(f"Workspace selector '{workspace}' provided - using cloud proxy client")
async with _cloud_client(config, timeout, workspace=workspace) as client:
yield client
return
# --- Default fallback ---
logger.debug("Default routing - using ASGI client for local Basic Memory API")
async with _asgi_client(timeout) as client:
+5 -1
View File
@@ -7,7 +7,9 @@ from typing import Optional, Any
from httpx import AsyncClient
from basic_memory.mcp.tools.utils import call_get
# call_* helpers live in basic_memory.mcp.tools.utils; importing that at module
# level executes the whole tools package (fastmcp + mcp SDK) during CLI startup,
# so each method defers the import to call time instead (#886).
class DirectoryClient:
@@ -55,6 +57,8 @@ class DirectoryClient:
Raises:
ToolError: If the request fails
"""
from basic_memory.mcp.tools.utils import call_get
params: dict = {
"dir_name": dir_name,
"depth": depth,
+55 -1
View File
@@ -8,7 +8,10 @@ from typing import Any
from httpx import AsyncClient
import logfire
from basic_memory.mcp.tools.utils import call_get, call_post, call_put, call_patch, call_delete
# call_* helpers live in basic_memory.mcp.tools.utils; importing that at module
# level executes the whole tools package (fastmcp + mcp SDK) during CLI startup,
# so each method defers the import to call time instead (#886).
from basic_memory.schemas.response import (
EntityResponse,
DeleteEntitiesResponse,
@@ -57,6 +60,8 @@ class KnowledgeClient:
Raises:
ToolError: If the request fails
"""
from basic_memory.mcp.tools.utils import call_post
with logfire.span(
"mcp.client.knowledge.create_entity",
client_name="knowledge",
@@ -89,6 +94,8 @@ class KnowledgeClient:
Raises:
ToolError: If the request fails
"""
from basic_memory.mcp.tools.utils import call_put
with logfire.span(
"mcp.client.knowledge.update_entity",
client_name="knowledge",
@@ -116,6 +123,8 @@ class KnowledgeClient:
Raises:
ToolError: If the entity is not found or request fails
"""
from basic_memory.mcp.tools.utils import call_get
with logfire.span(
"mcp.client.knowledge.get_entity",
client_name="knowledge",
@@ -147,6 +156,8 @@ class KnowledgeClient:
Raises:
ToolError: If the request fails
"""
from basic_memory.mcp.tools.utils import call_patch
with logfire.span(
"mcp.client.knowledge.patch_entity",
client_name="knowledge",
@@ -174,6 +185,8 @@ class KnowledgeClient:
Raises:
ToolError: If the entity is not found or request fails
"""
from basic_memory.mcp.tools.utils import call_delete
with logfire.span(
"mcp.client.knowledge.delete_entity",
client_name="knowledge",
@@ -201,6 +214,8 @@ class KnowledgeClient:
Raises:
ToolError: If the request fails
"""
from basic_memory.mcp.tools.utils import call_put
with logfire.span(
"mcp.client.knowledge.move_entity",
client_name="knowledge",
@@ -231,6 +246,8 @@ class KnowledgeClient:
Raises:
ToolError: If the request fails
"""
from basic_memory.mcp.tools.utils import call_post
with logfire.span(
"mcp.client.knowledge.move_directory",
client_name="knowledge",
@@ -261,6 +278,8 @@ class KnowledgeClient:
Raises:
ToolError: If the request fails
"""
from basic_memory.mcp.tools.utils import call_post
with logfire.span(
"mcp.client.knowledge.delete_directory",
client_name="knowledge",
@@ -276,10 +295,43 @@ class KnowledgeClient:
)
return DirectoryDeleteResult.model_validate(response.json())
# --- Single-file sync ---
async def sync_file(self, file_path: str) -> EntityResponse:
"""Index a markdown file that exists on disk but is not indexed yet.
Args:
file_path: Markdown file path relative to the project root
Returns:
EntityResponse for the indexed entity
Raises:
ToolError: If the file does not exist on disk or indexing fails
"""
from basic_memory.mcp.tools.utils import call_post
with logfire.span(
"mcp.client.knowledge.sync_file",
client_name="knowledge",
operation="sync_file",
):
response = await call_post(
self.http_client,
f"{self._base_path}/sync-file",
json={"file_path": file_path},
client_name="knowledge",
operation="sync_file",
path_template="/v2/projects/{project_id}/knowledge/sync-file",
)
return EntityResponse.model_validate(response.json())
# --- Orphan detection ---
async def get_orphans(self) -> list[GraphNode]:
"""Get entities that have no incoming or outgoing relations."""
from basic_memory.mcp.tools.utils import call_get
with logfire.span(
"mcp.client.knowledge.get_orphans",
client_name="knowledge",
@@ -309,6 +361,8 @@ class KnowledgeClient:
Raises:
ToolError: If the identifier cannot be resolved
"""
from basic_memory.mcp.tools.utils import call_post
with logfire.span(
"mcp.client.knowledge.resolve_entity",
client_name="knowledge",
+8 -1
View File
@@ -8,7 +8,10 @@ from typing import Optional
from httpx import AsyncClient
import logfire
from basic_memory.mcp.tools.utils import call_get
# call_* helpers live in basic_memory.mcp.tools.utils; importing that at module
# level executes the whole tools package (fastmcp + mcp SDK) during CLI startup,
# so each method defers the import to call time instead (#886).
from basic_memory.schemas.memory import GraphContext
@@ -63,6 +66,8 @@ class MemoryClient:
Raises:
ToolError: If the request fails
"""
from basic_memory.mcp.tools.utils import call_get
params: dict = {
"depth": depth,
"page": page,
@@ -113,6 +118,8 @@ class MemoryClient:
Raises:
ToolError: If the request fails
"""
from basic_memory.mcp.tools.utils import call_get
params: dict = {
"timeframe": timeframe,
"depth": depth,
+21 -7
View File
@@ -7,13 +7,9 @@ from typing import Any
from httpx import AsyncClient
from basic_memory.mcp.tools.utils import (
call_delete,
call_get,
call_patch,
call_post,
call_put,
)
# call_* helpers live in basic_memory.mcp.tools.utils; importing that at module
# level executes the whole tools package (fastmcp + mcp SDK) during CLI startup,
# so each method defers the import to call time instead (#886).
from basic_memory.schemas import ProjectInfoResponse, SyncReportResponse
from basic_memory.schemas.project_info import ProjectList, ProjectStatusResponse
from basic_memory.schemas.v2 import ProjectResolveResponse
@@ -53,6 +49,8 @@ class ProjectClient:
Raises:
ToolError: If the request fails
"""
from basic_memory.mcp.tools.utils import call_get
response = await call_get(
self.http_client,
"/v2/projects/",
@@ -71,6 +69,8 @@ class ProjectClient:
Raises:
ToolError: If the request fails
"""
from basic_memory.mcp.tools.utils import call_post
response = await call_post(
self.http_client,
"/v2/projects/",
@@ -93,6 +93,8 @@ class ProjectClient:
Raises:
ToolError: If the request fails
"""
from basic_memory.mcp.tools.utils import call_delete
url = f"/v2/projects/{project_external_id}"
if delete_notes:
url += "?delete_notes=true"
@@ -114,6 +116,8 @@ class ProjectClient:
Raises:
ToolError: If the request fails
"""
from basic_memory.mcp.tools.utils import call_post
response = await call_post(
self.http_client,
"/v2/projects/resolve",
@@ -133,6 +137,8 @@ class ProjectClient:
Raises:
ToolError: If the request fails
"""
from basic_memory.mcp.tools.utils import call_put
response = await call_put(
self.http_client,
f"/v2/projects/{project_external_id}/default",
@@ -154,6 +160,8 @@ class ProjectClient:
Raises:
ToolError: If the request fails
"""
from basic_memory.mcp.tools.utils import call_patch
response = await call_patch(
self.http_client,
f"/v2/projects/{project_external_id}",
@@ -181,6 +189,8 @@ class ProjectClient:
Raises:
ToolError: If the request fails
"""
from basic_memory.mcp.tools.utils import call_post
url = f"/v2/projects/{project_external_id}/sync"
params = []
if force_full:
@@ -204,6 +214,8 @@ class ProjectClient:
Raises:
ToolError: If the request fails
"""
from basic_memory.mcp.tools.utils import call_post
response = await call_post(
self.http_client,
f"/v2/projects/{project_external_id}/status",
@@ -222,6 +234,8 @@ class ProjectClient:
Raises:
ToolError: If the request fails
"""
from basic_memory.mcp.tools.utils import call_get
response = await call_get(
self.http_client,
f"/v2/projects/{project_external_id}/info",
+5 -1
View File
@@ -6,7 +6,9 @@ Encapsulates all /v2/projects/{project_id}/resource/* endpoints.
from httpx import AsyncClient, Response
import logfire
from basic_memory.mcp.tools.utils import call_get
# call_* helpers live in basic_memory.mcp.tools.utils; importing that at module
# level executes the whole tools package (fastmcp + mcp SDK) during CLI startup,
# so each method defers the import to call time instead (#886).
class ResourceClient:
@@ -49,6 +51,8 @@ class ResourceClient:
Raises:
ToolError: If the resource is not found or request fails
"""
from basic_memory.mcp.tools.utils import call_get
with logfire.span(
"mcp.client.resource.read",
client_name="resource",
+9 -1
View File
@@ -5,7 +5,9 @@ Encapsulates all /v2/projects/{project_id}/schema/* endpoints.
from httpx import AsyncClient
from basic_memory.mcp.tools.utils import call_post, call_get
# call_* helpers live in basic_memory.mcp.tools.utils; importing that at module
# level executes the whole tools package (fastmcp + mcp SDK) during CLI startup,
# so each method defers the import to call time instead (#886).
from basic_memory.schemas.schema import (
ValidationReport,
InferenceReport,
@@ -56,6 +58,8 @@ class SchemaClient:
Raises:
ToolError: If the request fails
"""
from basic_memory.mcp.tools.utils import call_post
params: dict[str, str] = {}
if note_type:
params["note_type"] = note_type
@@ -87,6 +91,8 @@ class SchemaClient:
Raises:
ToolError: If the request fails
"""
from basic_memory.mcp.tools.utils import call_post
response = await call_post(
self.http_client,
f"{self._base_path}/infer",
@@ -106,6 +112,8 @@ class SchemaClient:
Raises:
ToolError: If the request fails
"""
from basic_memory.mcp.tools.utils import call_get
response = await call_get(
self.http_client,
f"{self._base_path}/diff/{note_type}",
+6 -1
View File
@@ -8,7 +8,10 @@ from typing import Any
from httpx import AsyncClient
import logfire
from basic_memory.mcp.tools.utils import call_post
# call_* helpers live in basic_memory.mcp.tools.utils; importing that at module
# level executes the whole tools package (fastmcp + mcp SDK) during CLI startup,
# so each method defers the import to call time instead (#886).
from basic_memory.schemas.search import SearchResponse
@@ -57,6 +60,8 @@ class SearchClient:
Raises:
ToolError: If the request fails
"""
from basic_memory.mcp.tools.utils import call_post
with logfire.span(
"mcp.client.search.search",
client_name="search",
+164 -23
View File
@@ -8,10 +8,24 @@ The resolve_project_parameter function is a thin wrapper for backwards
compatibility with existing MCP tools.
"""
# PEP 563 lazy annotations keep `Context` usable in signatures without importing
# fastmcp at module load — the fastmcp/mcp stack costs ~0.5s of CLI startup (#886).
from __future__ import annotations
import asyncio
from contextlib import asynccontextmanager, nullcontext
from typing import (
TYPE_CHECKING,
AsyncIterator,
Awaitable,
Callable,
List,
Optional,
Sequence,
Tuple,
cast,
)
from dataclasses import dataclass, field
from typing import AsyncIterator, Awaitable, Callable, List, Optional, Sequence, Tuple, cast
from uuid import UUID
from httpx import AsyncClient
@@ -19,8 +33,6 @@ from httpx._types import (
HeaderTypes,
)
from loguru import logger
from fastmcp import Context
from mcp.server.fastmcp.exceptions import ToolError
import logfire
from basic_memory.config import BasicMemoryConfig, ConfigManager, ProjectMode, has_cloud_credentials
@@ -46,6 +58,9 @@ from basic_memory.workspace_context import (
workspace_permalink_context,
)
if TYPE_CHECKING:
from fastmcp import Context
# --- Workspace provider injection ---
# Mirrors the set_client_factory() pattern in async_client.py.
# The cloud MCP server sets a provider that queries its own database directly,
@@ -54,6 +69,14 @@ _workspace_provider: Optional[Callable[[], Awaitable[list[WorkspaceInfo]]]] = No
_WORKSPACE_PROJECT_INDEX_STATE_KEY = "workspace_project_index"
class WorkspaceProjectLookupMiss(ValueError):
"""A project was absent from the workspace index (as opposed to ambiguous).
Misses are retried once against a freshly rebuilt index, because the
session cache may simply predate an out-of-band project creation (#956).
"""
@dataclass(frozen=True)
class WorkspaceProjectEntry:
"""A cloud project resolved together with the workspace that owns it."""
@@ -460,6 +483,17 @@ def _canonical_memory_path_for_workspace(
# Outcome: lookups preserve the complete workspace/project canonical permalink.
if not normalized_remainder:
normalized_remainder = project_permalink
# Same index-form rule as _canonical_memory_path_for_active_route (#957):
# without an active workspace permalink context, stored permalinks are
# project-qualified and a workspace-prefixed pattern cannot match.
if "*" in normalized_remainder and current_workspace_permalink_context() is None:
return build_qualified_permalink_reference(
project_permalink,
normalized_remainder,
include_project=True,
)
return build_qualified_permalink_reference(
project_permalink,
normalized_remainder,
@@ -477,6 +511,24 @@ def _canonical_memory_path_for_active_route(
) -> str:
"""Return the canonical permalink path for the currently routed project/workspace."""
project_prefix = active_project.permalink
# Trigger: the path contains a glob wildcard (folder/*) and no server-side
# workspace permalink context is active.
# Why: patterns match raw against the search index, so they must mirror the
# stored permalink form. The contextvar is what qualified permalinks at
# write time — when it is absent, stored rows are project-qualified and a
# workspace prefix (from the client's cached_workspace display state)
# guarantees zero matches (#957). Direct lookups keep full qualification
# because the link resolver understands it; patterns have no fallback.
# Outcome: without the contextvar, qualify patterns with the project prefix
# only; with it, fall through to normal workspace canonicalization.
if "*" in path and current_workspace_permalink_context() is None:
if not include_project:
return path
if path == project_prefix or path.startswith(f"{project_prefix}/"):
return path
return f"{project_prefix}/{path}"
workspace_remainder = path
if include_project and (path == project_prefix or path.startswith(f"{project_prefix}/")):
# Trigger: the memory URL already names the active project root/prefix
@@ -722,9 +774,15 @@ async def _fetch_workspace_project_entries(
async def _ensure_workspace_project_index(
context: Optional[Context] = None,
*,
force_refresh: bool = False,
) -> WorkspaceProjectIndex:
"""Build or load the session-local workspace/project lookup index."""
if context:
"""Build or load the session-local workspace/project lookup index.
force_refresh bypasses the cached index and rebuilds from discovery
used by resolve_workspace_project_identifier when a lookup misses (#956).
"""
if context and not force_refresh:
cached_raw = await context.get_state(_WORKSPACE_PROJECT_INDEX_STATE_KEY)
cached_index = _workspace_project_index_from_state(cached_raw)
if cached_index is not None:
@@ -798,13 +856,99 @@ async def ensure_workspace_project_index(
return await _ensure_workspace_project_index(context=context)
def _match_workspace_identifier(
workspaces: tuple[WorkspaceInfo, ...],
workspace_identifier: str,
) -> WorkspaceInfo:
"""Resolve the first segment of a qualified route to a single workspace.
The edit_note/write_note contract advertises that the workspace segment may be a
slug, tenant_id, or display name. We honor those forms in a fixed priority order so
that adding tenant_id/name support never changes the meaning of an identifier that
already resolves today:
1. slug (casefold) existing behavior, checked first so working routes are stable.
2. tenant_id exact match against the opaque id (no casefolding, mirroring the
precedent in ``workspace_matches_exact_identifier``).
3. display name (casefold) names are not guaranteed unique, so a name that matches
multiple workspaces is rejected rather than silently picking one.
"""
# Trigger: identifier equals a workspace slug (casefold).
# Why: slug is the canonical routing key; resolving it first guarantees a workspace
# whose display name collides with another workspace's slug yields to the slug owner.
# Outcome: return the slug owner before tenant_id/name are considered.
slug_matches = [
workspace
for workspace in workspaces
if workspace.slug.casefold() == workspace_identifier.casefold()
]
if slug_matches:
return slug_matches[0]
# Trigger: identifier exactly equals a workspace tenant_id (an opaque id).
# Why: tenant_ids are unique, so an exact hit is unambiguous and needs no tie-break.
tenant_matches = [
workspace for workspace in workspaces if workspace.tenant_id == workspace_identifier
]
if tenant_matches:
return tenant_matches[0]
# Trigger: identifier matches one or more workspace display names (casefold).
# Why: names are not guaranteed unique; failing fast on collisions keeps routing
# deterministic and tells the caller exactly how to disambiguate.
name_matches = [
workspace
for workspace in workspaces
if workspace.name.casefold() == workspace_identifier.casefold()
]
if len(name_matches) > 1:
candidates = ", ".join(workspace.slug for workspace in name_matches)
raise ValueError(
f"Workspace name '{workspace_identifier}' matched multiple workspaces "
f"(slugs: {candidates}). Use the workspace slug or tenant_id to disambiguate."
)
if name_matches:
return name_matches[0]
available = ", ".join(workspace.slug for workspace in workspaces)
raise ValueError(
f"Workspace '{workspace_identifier}' was not found by slug, tenant_id, or name. "
f"Available workspace slugs: {available}"
)
async def resolve_workspace_project_identifier(
project: str,
context: Optional[Context] = None,
) -> WorkspaceProjectEntry:
"""Resolve a project by external_id (UUID), qualified name, or unqualified name."""
index = await _ensure_workspace_project_index(context=context)
try:
return await _resolve_workspace_project_from_index(index, project, context)
except WorkspaceProjectLookupMiss:
# Trigger: the lookup missed the session-cached index.
# Why: a miss is exactly the signal the cache may be stale — projects
# created out-of-band (CLI, a teammate in a shared workspace) post-date
# the index built at session start (#956).
# Outcome: rebuild the index once and retry; a second miss is authoritative
# and its error (with the refreshed project list) propagates.
logger.info(
f"Workspace project lookup missed for '{project}'; refreshing index and retrying"
)
refreshed = await _ensure_workspace_project_index(context=context, force_refresh=True)
return await _resolve_workspace_project_from_index(refreshed, project, context)
async def _resolve_workspace_project_from_index(
index: WorkspaceProjectIndex,
project: str,
context: Optional[Context] = None,
) -> WorkspaceProjectEntry:
"""Resolve a project against one concrete index snapshot.
Raises WorkspaceProjectLookupMiss for absent projects (retryable via index
refresh) and plain ValueError for ambiguity, which a refresh cannot fix.
"""
# Fast path: direct lookup by external_id when the identifier is a UUID
# Canonicalize via str(UUID(...)) so uppercase, brace-wrapped, or urn:uuid forms
# all hash to the same lowercase-hyphenated key as the stored external_ids.
@@ -816,23 +960,14 @@ async def resolve_workspace_project_identifier(
except ValueError:
pass
workspace_slug, project_identifier = _split_qualified_project_identifier(project)
workspace_identifier, project_identifier = _split_qualified_project_identifier(project)
project_permalink = generate_permalink(project_identifier)
if workspace_slug:
workspace_matches = [
workspace
for workspace in index.workspaces
if workspace.slug.casefold() == workspace_slug.casefold()
]
if not workspace_matches:
available = ", ".join(workspace.slug for workspace in index.workspaces)
raise ValueError(
f"Workspace '{workspace_slug}' was not found. "
f"Available workspace slugs: {available}"
)
workspace = workspace_matches[0]
if workspace_identifier:
# Honor the documented "slug, name, or tenant_id" contract for the workspace
# segment; _match_workspace_identifier raises a clear error on ambiguous names
# and unknown identifiers, listing what forms were tried.
workspace = _match_workspace_identifier(index.workspaces, workspace_identifier)
matches = [
entry
for entry in index.entries_by_permalink.get(project_permalink, ())
@@ -843,7 +978,7 @@ async def resolve_workspace_project_identifier(
failed_workspace.tenant_id == workspace.tenant_id
for failed_workspace in index.failed_workspaces
):
raise ValueError(
raise WorkspaceProjectLookupMiss(
f"Projects for workspace '{workspace.name}' ({workspace.slug}) "
"could not be loaded. Retry after workspace discovery recovers."
)
@@ -852,7 +987,7 @@ async def resolve_workspace_project_identifier(
for entry in index.entries
if entry.workspace.tenant_id == workspace.tenant_id
)
raise ValueError(
raise WorkspaceProjectLookupMiss(
f"Project '{project_identifier}' was not found in workspace "
f"'{workspace.name}' ({workspace.slug}). Available projects: {available}"
)
@@ -877,7 +1012,7 @@ async def resolve_workspace_project_identifier(
"retry or use a qualified project from an indexed workspace."
)
available = ", ".join(entry.qualified_name for entry in index.entries)
raise ValueError(
raise WorkspaceProjectLookupMiss(
f"Project '{project}' was not found in indexed cloud workspaces. "
f"Available projects: {available}.{failed_note}"
)
@@ -1212,6 +1347,9 @@ async def resolve_project_and_path(
# Why: allow project-scoped memory URLs without requiring a separate project parameter
# Outcome: attempt to resolve the prefix as a project and route to it
if project_prefix:
# Deferred: ToolError lives in the mcp SDK, which must not load at CLI startup (#886).
from mcp.server.fastmcp.exceptions import ToolError
if cached_project and _project_matches_identifier(cached_project, project_prefix):
resolved_project = await resolve_project_parameter(project_prefix, context=context)
if resolved_project and generate_permalink(resolved_project) != generate_permalink(
@@ -1438,6 +1576,9 @@ async def get_project_client(
is_factory_mode,
)
# Deferred: ToolError lives in the mcp SDK, which must not load at CLI startup (#886).
from mcp.server.fastmcp.exceptions import ToolError
# When project_id (UUID) is provided, prefer it as the resolution identifier.
# external_id is unambiguous across workspaces; project name can collide.
project_identifier = project_id if project_id else project
@@ -0,0 +1,17 @@
"""Shared loader for the bundled cloud-discovery markdown resources."""
from pathlib import Path
def load_discovery_resource(filename: str) -> str:
"""Read a bundled discovery markdown file with promo placeholders rendered.
The markdown carries a {{OSS_DISCOUNT_CODE}} placeholder so the promo code
has one source of truth (cli.promo); substitute before it reaches users.
"""
# Import here to avoid pulling CLI promo machinery (analytics, rich, config)
# into the MCP server import graph at module load.
from basic_memory.cli.promo import OSS_DISCOUNT_CODE
content = (Path(__file__).parent / filename).read_text(encoding="utf-8")
return content.replace("{{OSS_DISCOUNT_CODE}}", OSS_DISCOUNT_CODE)
+18 -1
View File
@@ -54,7 +54,10 @@ def _format_entity_block(result: ContextResult) -> str:
lines.append("")
lines.append("### Relations")
for rel in relation_items:
lines.append(f"- {rel.relation_type} [[{rel.to_entity}]]")
# Unresolved forward references have no resolved entity yet; fall back
# to the literal target text instead of rendering [[None]] (#955)
target = rel.to_entity or rel.to_name
lines.append(f"- {rel.relation_type} [[{target}]]")
# --- Related entities (non-relation related results) ---
related_entities: list[EntitySummary | ObservationSummary] = [
@@ -112,6 +115,7 @@ def _format_context_markdown(graph: GraphContext, project: str) -> str:
@mcp.tool(
title="Build Context",
description="""Build context from a memory:// URI to continue conversations naturally.
Use this to follow up on previous discussions or explore related topics.
@@ -131,6 +135,7 @@ def _format_context_markdown(graph: GraphContext, project: str) -> str:
- "json" (default): Structured JSON with internal fields excluded
- "text": Compact markdown text for LLM consumption
""",
tags={"navigation", "notes"},
annotations={"readOnlyHint": True, "openWorldHint": False},
)
async def build_context(
@@ -209,6 +214,18 @@ async def build_context(
Raises:
ToolError: If project doesn't exist or depth parameter is invalid
"""
# Validate pagination arguments before they reach the context service.
# Trigger: page < 1 or page_size < 1 (e.g. page_size=0 or negative).
# Why: a non-positive page_size flows into context_service as limit, where the
# primary slice does primary = primary[:limit] — so limit=0 truncates the
# requested entity to [] and the caller's valid memory:// lookup silently
# returns primary_count=0. Mirrors recent_activity's guard for consistency.
# Outcome: caller gets an explicit ValueError instead of a dropped primary result.
if page < 1:
raise ValueError(f"page must be >= 1, got {page}")
if page_size < 1:
raise ValueError(f"page_size must be >= 1, got {page_size}")
# Detect project from memory URL prefix before routing.
# project_id routes by external UUID, so it bypasses URL discovery entirely.
if project is None and project_id is None:
+2
View File
@@ -17,7 +17,9 @@ from basic_memory.mcp.tools.utils import call_put, call_post, resolve_entity_id
@mcp.tool(
title="Create Canvas",
description="Create an Obsidian canvas file to visualize concepts and connections.",
tags={"canvas", "notes"},
annotations={"destructiveHint": False, "idempotentHint": True, "openWorldHint": False},
)
async def canvas(
@@ -105,7 +105,9 @@ def _format_document_for_chatgpt(
@mcp.tool(
title="Search Knowledge Base",
description="Search for content across the knowledge base",
tags={"search"},
annotations={"readOnlyHint": True, "openWorldHint": False},
)
async def search(
@@ -167,7 +169,9 @@ async def search(
@mcp.tool(
title="Fetch Document",
description="Fetch the full contents of a search result document",
tags={"search", "notes"},
annotations={"readOnlyHint": True, "openWorldHint": False},
)
async def fetch(
+4 -4
View File
@@ -1,15 +1,15 @@
"""Cloud information MCP tool."""
from pathlib import Path
from basic_memory.mcp.resources.discovery import load_discovery_resource
from basic_memory.mcp.server import mcp
@mcp.tool(
"cloud_info",
title="Cloud Info",
tags={"cloud"},
annotations={"readOnlyHint": True, "openWorldHint": False},
)
def cloud_info() -> str:
"""Return optional Basic Memory Cloud information and setup guidance."""
content_path = Path(__file__).parent.parent / "resources" / "cloud_info.md"
return content_path.read_text(encoding="utf-8")
return load_discovery_resource("cloud_info.md")

Some files were not shown because too many files have changed in this diff Show More