Compare commits

..

4 Commits

Author SHA1 Message Date
Drew Cain 4d0566c983 fix(mcp): drop stale API version from diagnostics, suppress duplicated structured output
- Remove orphaned __api_version__ ('v0' since 2025; real API routes are /v2,
  and nothing else reads the constant)
- output_schema=None so FastMCP doesn't duplicate the markdown report into
  structuredContent (halves the payload)
- Add title/tags annotations so #963 and #968 are merge-order independent

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Drew Cain <groksrc@gmail.com>
2026-06-11 14:56:35 -05:00
Drew Cain 8f444f9fe9 test(mcp): use exact-equality assertions for redacted URLs
CodeQL flags dotted-host substring checks against URLs
(py/incomplete-url-substring-sanitization); exact matches are also
stronger assertions.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Drew Cain <groksrc@gmail.com>
2026-06-11 09:42:22 -05:00
Drew Cain cbc2d5feb1 fix(mcp): redact database_url credentials in diagnostics output
Extends the existing _SECRET_FIELDS redaction to also strip the
user:password userinfo component from any URL-bearing config fields
(database_url) before surfacing them in diagnostics output. Adds
_redact_url() helper and a _URL_FIELDS set so future credential-bearing
URL fields are easy to add. Postgres passwords no longer leak to MCP
clients or support transcripts; host/port/db-name remain visible.

Fixes the P1 security finding raised in review of PR #963.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Drew Cain <groksrc@gmail.com>
2026-06-11 09:37:00 -05:00
Drew Cain 29be0299f0 feat(mcp): add basic_memory_diagnostics tool for version and system info
Adds a new read-only MCP tool `basic_memory_diagnostics` that surfaces:
- Basic Memory package version and API version (from __init__.py)
- Python version and platform/architecture details
- Config file path and full config.json contents (cloud_api_key redacted)

Resolves #187.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Drew Cain <groksrc@gmail.com>
2026-06-11 01:10:21 -05:00
204 changed files with 5021 additions and 9759 deletions
+2 -2
View File
@@ -6,14 +6,14 @@
},
"metadata": {
"description": "Official Basic Memory plugins from the canonical basic-memory repository",
"version": "0.22.1"
"version": "0.22.0"
},
"plugins": [
{
"name": "basic-memory",
"source": "./plugins/claude-code",
"description": "The bridge between Claude's working memory and Basic Memory's durable knowledge graph \u2014 session briefings, pre-compaction checkpoints, and capture reflexes",
"version": "0.22.1",
"version": "0.22.0",
"author": {
"name": "Basic Machines"
},
+36 -50
View File
@@ -28,10 +28,7 @@ You are an expert release manager for the Basic Memory project. When the user ru
#### Documentation Validation
1. **Changelog Check**
- CHANGELOG.md contains entry for target version **already landed on `main`**
(main only accepts changes via PR, so the changelog entry must go through
its own PR before running the release; the recipe pre-flight-checks for a
`## vX.Y.Z` heading)
- CHANGELOG.md contains entry for target version
- Entry includes all major features and fixes
- Breaking changes are documented
@@ -44,15 +41,11 @@ just release <version>
The justfile target handles:
- ✅ Version format validation
- ✅ Git status and branch checks
-Changelog entry check (must already be on `main`)
- ✅ Quality checks (`just lint` + `just typecheck`)
-Quality checks (`just check` - lint, format, type-check, tests)
- ✅ Version update across all consolidated manifests via `just set-version` (Python
package + Claude Code plugin/marketplaces + Codex plugin + Hermes + OpenClaw)
-Release PR: commits the bump on a `release/vX.Y.Z` branch, opens a PR
(`chore(core): release vX.Y.Z`), and rebase-merges it — the `main` ruleset
rejects direct pushes and the repo disallows merge commits
- ✅ Tags the rebased bump commit on `main` (found by commit subject, since
the rebase rewrites the SHA) and pushes the tag
-Automatic commit with proper message
- ✅ Tag creation and pushing to GitHub
- ✅ Release workflow trigger (automatic on tag push)
The GitHub Actions workflow (`.github/workflows/release.yml`) then:
@@ -96,7 +89,7 @@ After PyPI release is published, update the MCP registry:
2. **Publish to MCP Registry**
```bash
# from the basic-memory repo root
cd /Users/drew/code/basic-memory
mcp-publisher publish
```
@@ -116,50 +109,43 @@ After PyPI release is published, update the MCP registry:
#### Website Updates
**1. basicmemory.com** (sibling `basicmemory.com` repo —
`basicmachines-co/basicmemory.com`, formerly `basicmachines.co`)
- **No version bump needed.** The marketing site is an Astro + React app and
carries **no hardcoded Basic Memory version number** anywhere in its UI
(`hero.tsx` and the rest of the site have no version string). The old
instruction to bump `src/components/sections/hero.tsx` is obsolete — that
file no longer holds a version. Release announcements are dated blog posts,
not an in-place edit.
- **Skip entirely for patch releases.**
- **Significant releases only — optional announcement post**:
**1. basicmachines.co** (`/Users/drew/code/basicmachines.co`)
- **Goal**: Update version number displayed on the homepage
- **Location**: Search for "Basic Memory v0." in the codebase to find version displays
- **What to update**:
- Hero section heading that shows "Basic Memory v{VERSION}"
- "What's New in v{VERSION}" section heading
- Feature highlights array (look for array of features with title/description)
- **Process**:
1. Pull latest from GitHub: `git pull origin main`
2. Create release branch: `git checkout -b release/v{VERSION}`
3. Add a dated post under `src/content/blog/` modeled on an existing
release post (e.g. `basic-memory-v0-19-0-release.md`), summarizing 35
headline features from `CHANGELOG.md`
4. Commit (`git commit -s -m "..."`), push, and open a PR against
`basicmachines-co/basicmemory.com`
- **Deploy**: follow that repo's deployment process.
3. Search codebase for current version number (e.g., "v0.16.1")
4. Update version numbers to new release version
5. Update feature highlights with 3-5 key features from this release (extract from CHANGELOG.md)
6. Commit changes: `git commit -m "chore: update to v{VERSION}"`
7. Push branch: `git push origin release/v{VERSION}`
- **Deploy**: Follow deployment process for basicmachines.co
**2. docs.basicmemory.com** (sibling `docs.basicmemory.com` repo)
- **Goal**: Add a What's New page for the release and bump the homepage badge
- **Site shape**: Nuxt/Docus content site. The changelog page
(`content/2.whats-new/*.changelog.md`) auto-fetches GitHub releases — no
manual changelog update needed. See that repo's CLAUDE.md "Version Bump
Checklist".
**2. docs.basicmemory.com** (`/Users/drew/code/docs.basicmemory.com`)
- **Goal**: Add new release notes section to the latest-releases page
- **File**: `src/pages/latest-releases.mdx`
- **What to do**:
1. Pull latest from GitHub: `git pull origin main`
2. Create release branch: `git checkout -b release/v{VERSION}`
3. Read `CHANGELOG.md` in the `basic-memory` repo to get release content
4. **New minor/major release**: add `content/2.whats-new/1.v{VERSION}.md`
modeled on the previous version page (frontmatter title/description,
headline feature first, then sections, then an Upgrading note) and
renumber the existing what's-new pages down one slot (URLs don't
change — Nuxt strips the numeric prefixes)
5. **Patch release**: append a short note to the current version's page
instead of creating a new one
6. Update the homepage version badge in `content/index.md` (the
`v0.XX →` button text and its `to: /whats-new/v{VERSION}` link)
7. If the release adds user-facing features, update the relevant guide
and reference pages (`content/3.cloud/`, `content/9.reference/`)
8. Commit: `git commit -s -m "docs: add v{VERSION} release notes"`
9. Push branch and open a PR; merge after the release is tagged
- **Deploy**: push to main auto-deploys to development; production requires
manual workflow dispatch via GitHub Actions
3. Read the existing file to understand the format and structure
4. Read `/Users/drew/code/basic-memory/CHANGELOG.md` to get release content
5. Add new release section **at the top** (after MDX imports, before other releases)
6. Follow the existing pattern:
- Heading: `## [v{VERSION}](github-link) — YYYY-MM-DD`
- Focus statement if applicable
- `<Info>` block with highlights (3-5 key items)
- Sections for Features, Bug Fixes, Breaking Changes, etc.
- Link to full changelog at the end
- Separator `---` between releases
7. Commit changes: `git commit -m "docs: add v{VERSION} release notes"`
8. Push branch: `git push origin release/v{VERSION}`
- **Source content**: Extract and format sections from CHANGELOG.md for this version
- **Deploy**: Follow deployment process for docs.basicmemory.com
**4. Announce Release**
- Post to Discord community if significant changes
+4 -11
View File
@@ -300,9 +300,7 @@ See SPEC-16 for full context manager refactor details.
### Release Process
Releases are driven by `just release` / `just beta` — never by a bare `git tag`. The recipes bump version metadata, run pre-flight checks, land the bump on `main` through a release PR, tag, and push the tag. GitHub Actions then publishes to PyPI and updates the Homebrew formula.
**Main requires PRs.** The `main` ruleset rejects direct pushes ("Changes must be made through a pull request") and the repo disallows merge commits, so the recipes push a `release/vX.Y.Z` branch, open a PR titled `chore(core): release vX.Y.Z`, rebase-merge it with `gh pr merge --rebase`, then tag the rebased bump commit on `main` (located by its commit subject, since rebasing rewrites the SHA) and push the tag. The CHANGELOG entry for the version must already be on `main` — land it via a normal PR before running the recipe (it pre-flight-checks for a `## vX.Y.Z` heading).
Releases are driven by `just release` / `just beta` — never by a bare `git tag`. The recipes bump version metadata, run pre-flight checks, commit, tag, and push. GitHub Actions then publishes to PyPI and updates the Homebrew formula.
**Stable release:**
@@ -310,7 +308,7 @@ Releases are driven by `just release` / `just beta` — never by a bare `git tag
just release v0.21.3
```
The recipe runs `just lint` + `just typecheck`, then updates every release manifest through `scripts/update_versions.py`: `src/basic_memory/__init__.py`, `server.json`, the root Claude marketplace, the Claude Code plugin manifest and local marketplace, the Hermes `plugin.yaml`, and the OpenClaw `package.json`. It commits as `chore: update version to X.Y.Z for vX.Y.Z release` on a `release/vX.Y.Z` branch, lands it on `main` via a rebase-merged PR, then tags the rebased commit and pushes the tag. After the tag lands, the `Release` workflow builds the Python package, publishes to PyPI, creates the GitHub release with auto-generated notes, publishes the OpenClaw npm package, and updates the Homebrew formula. The recipe finishes by printing the post-release tasks the workflow doesn't cover.
The recipe runs `just lint` + `just typecheck`, then updates every release manifest through `scripts/update_versions.py`: `src/basic_memory/__init__.py`, `server.json`, the root Claude marketplace, the Claude Code plugin manifest and local marketplace, the Hermes `plugin.yaml`, and the OpenClaw `package.json`. It commits as `chore: update version to X.Y.Z for vX.Y.Z release`, creates the `vX.Y.Z` tag, and pushes both the commit and the tag to `origin/main`. After the tag lands, the `Release` workflow builds the Python package, publishes to PyPI, creates the GitHub release with auto-generated notes, publishes the OpenClaw npm package, and updates the Homebrew formula. The recipe finishes by printing the post-release tasks the workflow doesn't cover.
**Beta release:** `just beta v0.21.3b1` — same flow with a beta-suffixed tag. PyPI consumers install with `pip install basic-memory --pre`.
@@ -321,13 +319,8 @@ The recipe runs `just lint` + `just typecheck`, then updates every release manif
**Do not tag releases by hand.** A bare `git tag vX.Y.Z` skips the in-code version bump. Package metadata is still correct (uv-dynamic-versioning derives it from the git tag) but `basic-memory --version` reports the previous release, which is what happened with v0.21.2 → v0.21.3.
**Post-release tasks** the recipe surfaces but doesn't run:
- `docs.basicmemory.com` — add a What's New page under `content/2.whats-new/` and bump the version badge in `content/index.md` (the changelog page auto-fetches GitHub releases; see that repo's CLAUDE.md version-bump checklist)
- `basicmemory.com`the marketing site (Astro + React, repo
`basicmachines-co/basicmemory.com`, formerly `basicmachines.co`) carries **no
hardcoded version number** in its UI, so there is nothing to bump. For a
significant release, optionally add a dated announcement post under
`src/content/blog/` (model it on an existing `basic-memory-vX-Y-Z-release.md`).
Skip entirely for routine patch releases.
- `docs.basicmemory.com` — add notes to `src/pages/latest-releases.mdx`
- `basicmachines.co`bump version in `src/components/sections/hero.tsx`
- MCP Registry — `mcp-publisher publish` from the repo root
See `.claude/commands/release/release.md` (and `beta.md`, `release-check.md`, `changelog.md` alongside it) for the full release + post-release runbook, including the slash commands.
-44
View File
@@ -1,49 +1,5 @@
# CHANGELOG
## v0.22.1 (2026-06-12)
Follow-up patch to v0.22.0. Fixes project and default-project resolution on
fresh installs, MCP workspace routing, sync project selection, and CLI startup
latency, plus a few MCP parity additions.
### Features
- Added a `workspace` parameter to `write_note` for parity with `edit_note`.
- **#826**: Added `title` and `tags` annotations to all MCP tool decorators
(phase 1).
- **#930**: `search_notes` now comma-splits `note_types`, `entity_types`, and
`categories`.
- **#971**: Added the manual-pages flow — manpage seed schema, flow docs, and
verification fixes.
### Bug Fixes
- Fresh installs no longer fail when the projects table is empty: resolve now
points them at project setup, the first project is promoted to default when
the config default is missing from the database, the promoted default state
is returned from the project-create API, and a default can be set when none
is currently set. An existing database default is preserved when repairing a
missing config default.
- **#949**: Sync skips projects without an absolute local path and excludes
orphan DB projects that are absent from config.
- **#952 / #981**: Resolved workspace display names and tenant ids in qualified
project routes, closing out the manual verification findings.
- `note_types`/`entity_types`/`categories` are normalized on the direct-call
path, with non-string list elements rejected.
- Vector-search hydration keys on `(type, id)` to prevent id collisions.
- `file_utils` requires line-anchored frontmatter fences.
- CLI startup is faster: FastAPI and app imports are deferred out of CLI
startup, and rich/typer modules are preloaded before an in-place upgrade.
- `config.json` is written atomically.
- In-memory SQLite sessions are serialized so concurrent rollbacks cannot
destroy writes.
### Maintenance
- Release recipes route through PRs and wait for the release PR merge to land
before tagging; the release runbook is refreshed and stripped of
user-specific absolute paths.
## v0.22.0 (2026-06-11)
Team-safe cloud sync. New additive `bm cloud push` and `bm cloud pull`
+1 -2
View File
@@ -6,7 +6,6 @@
[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)
![](https://badge.mcpx.dev?type=server 'MCP Server')
![](https://badge.mcpx.dev?type=dev 'MCP Dev')
[![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/basicmachines-co/basic-memory)
## Skip the install — try Basic Memory in the cloud
@@ -591,7 +590,7 @@ retention).
| `BASIC_MEMORY_IMPORT_UPLOAD_MAX_BYTES` | `104857600` | Max uploaded import size |
```bash
BASIC_MEMORY_LOG_LEVEL=DEBUG basic-memory reindex
BASIC_MEMORY_LOG_LEVEL=DEBUG basic-memory sync
tail -f ~/.basic-memory/basic-memory.log
```
+3 -3
View File
@@ -111,7 +111,7 @@ You can run Basic Memory CLI commands inside the container using `docker exec`:
docker exec basic-memory-server basic-memory status
# Sync files
docker exec basic-memory-server basic-memory reindex
docker exec basic-memory-server basic-memory sync
# Show help
docker exec basic-memory-server basic-memory --help
@@ -137,7 +137,7 @@ When using Docker volumes, you'll need to configure projects to point to your mo
3. **Sync the new project:**
```bash
docker exec basic-memory-server basic-memory reindex
docker exec basic-memory-server basic-memory sync
```
### Example: Setting up an Obsidian Vault
@@ -157,7 +157,7 @@ docker exec basic-memory-server basic-memory project create obsidian /app/data
docker exec basic-memory-server basic-memory project set-default obsidian
# Sync to index all files
docker exec basic-memory-server basic-memory reindex
docker exec basic-memory-server basic-memory sync
```
### Environment Variables
+2 -2
View File
@@ -184,8 +184,8 @@ finance/ (lowercase f)
Use Basic Memory's built-in conflict detection:
```bash
# Index local file changes (conflicts are handled during the scan)
basic-memory reindex
# Sync will report conflicts
basic-memory sync
# Check sync status for warnings
basic-memory status
-183
View File
@@ -1,183 +0,0 @@
# Manual Pages
Basic Memory's manual is written in the style of Unix man pages — and
implemented as Basic Memory notes ([#952](https://github.com/basicmachines-co/basic-memory/issues/952)).
Every page is a markdown note conforming to the `Manpage` schema, `SEE ALSO`
entries are real knowledge-graph relations, and every example on every page
was executed against a live project before the page shipped. The manual
documents the tools; the tools verify the manual.
## Where it lives
The canonical manual is the **`manual` project in the Basic Memory team
workspace** (cloud, shared). Anyone can build their own: the schema ships as
an opt-in seed at `plugins/claude-code/schemas/manpage.md` — copy it into any
project's folder and start writing pages against it.
Layout:
```
manual/
├── schemas/Manpage.md # the manpage schema (type: schema)
├── man1/ # CLI commands bm(1), bm-status(1), ...
├── man3/ # MCP tools write-note(3), search-notes(3), ...
├── man5/ # file formats bm-note(5), bm-observation(5), ...
├── man7/ # concepts basic-memory(7), semantic-memory(7), ...
├── playground/ # scratch notes for destructive examples
└── diagrams/ # canvas visualizations of the manual graph
```
### Why "man1", "man3", "man5"?
The folder names are Unix's, unchanged since 1971. The manual is divided
into numbered **sections**, pages physically live in directories named
after them (`/usr/share/man/man1`, `man5`, ...), and the number tells you
what *kind* of thing is documented — not importance, not reading order:
- **1** — user commands (`ls`, `grep`)
- **2** — system calls
- **3** — library functions / APIs (`printf(3)`)
- **4** — devices
- **5** — file formats and config files (`crontab(5)`, `passwd(5)`)
- **6** — games (really)
- **7** — miscellanea: concepts, conventions, overviews (`regex(7)`, `signal(7)`)
- **8** — system administration
That's also why man page names carry the parenthesized number —
`crontab(1)` is the command, `crontab(5)` is the file format, same name in
two sections. `man 5 crontab` picks the section explicitly.
This manual copies that layout with the sections that have a Basic Memory
analog:
- **man1/**`bm` CLI commands → `bm-status(1)`
- **man3/** — MCP tools, our equivalent of the "library API" section → `write-note(3)`
- **man5/** — file formats: note syntax, observations, relations, schemas → `bm-note(5)`
- **man7/** — concepts → `basic-memory(7)`, `semantic-memory(7)`
- **8** is reserved for admin/cloud operations but has no pages yet; 2, 4,
and 6 have no analog (no system calls, no devices, and no games — yet)
When a page says `see_also [[bm-note(5)]]`, the `(5)` reads "the
file-format page," exactly the way a Unix manual cross-references — except
here it's a traversable relation in the graph instead of a typographic
convention. The manual explains its own conventions in `man-pages(7)`
fittingly, the same page name Linux uses for this, and that almost nobody
ever reads.
## Page anatomy
Pages use the classic headers where applicable: `NAME`, `SYNOPSIS`,
`DESCRIPTION`, `PARAMETERS`, `MCP USAGE`, `CLI EQUIVALENT`, `EXAMPLES`,
`GOTCHAS`, `SEE ALSO`. Frontmatter (validated by the schema):
```yaml
type: manpage
section: 3 # 1 | 3 | 5 | 7 | 8
name: write-note # page name without section suffix
summary: create or overwrite a markdown note in the knowledge base
generated: hand # hand | registry | typer (regeneration ownership)
tool: write_note # section-3 pages: the MCP tool documented
command: basic-memory status # section-1 pages: the CLI command documented
verified: 0.21.6 mcp+cli # version + path(s) that proved the page
```
Field knowledge accumulates as observations — `[gotcha]`, `[bug]` (with issue
links), `[pattern]` — and `SEE ALSO` entries are `see_also` relations, so the
manual is a navigable graph, not a folder of files.
## How to use it
Man-style reads (any MCP client or the CLI):
```bash
# read a page
bm tool read-note "man3/write-note-3" --project manual
# apropos — find pages by section, tool, or text
bm tool search-notes --project manual # then filter, or via MCP:
# search_notes(project="manual", metadata_filters={"type": "manpage", "section": 3})
# search_notes(project="manual", metadata_filters={"type": "manpage", "tool": "write_note"})
# traverse SEE ALSO from any page
# build_context(url="man3/write-note-3", project="manual")
```
A future `bm man <topic>` command is thin sugar over exactly these calls.
And for the real thing — `man bm` in an actual terminal:
```bash
bm man install # copies bundled groff pages to ~/.local/share/man
man bm # the overview page, rendered by man(1)
man basic-memory # same page via its alias
```
`bm man install` warns with a one-line `MANPATH` fix if the install root
isn't searched by your `man`. Agents with shell access can use `man bm` as
an offline quick reference; the full per-tool detail stays in the manual
project's section-3 pages.
## The verification discipline
Two rules make the manual trustworthy:
1. **Examples must have run.** An `EXAMPLES` (or `MCP USAGE` / `CLI
EQUIVALENT`) block contains only commands that actually executed against
the manual project. Destructive operations (`delete_note`, `move_note`,
destructive `edit_note`) run only against `playground/` notes — never
against pages. The `verified:` field records the version and which path
proved the page: `mcp` (live service), `cli` (dev checkout), or both.
2. **The schema is the linter.** Validate the whole manual any time:
```bash
bm tool schema-validate manpage --project manual
# → {"total_notes": 38, "valid_count": 38, "warning_count": 0, ...}
```
`bm orphans --project manual` confirms every page is connected to the
graph, and `schema_diff`/`schema_infer` report drift between the schema
and how pages are actually written.
Because verification exercises real tool calls against the live service,
building the manual doubles as an end-to-end smoke test. The initial build
found six bugs in one pass (#954#959) — including the verification rule
catching a test that asserted a bug as expected output (#958).
## Adding or updating a page
1. Run the commands you intend to document; keep the actual output.
2. Write the page with `write_note`, passing frontmatter through the
`metadata` parameter (nested YAML in content frontmatter is unreliable on
some clients):
```
write_note(title="my-tool(3)", directory="man3", project="manual",
note_type="manpage",
metadata={"section": 3, "name": "my-tool",
"summary": "...", "generated": "hand",
"tool": "my_tool", "verified": "<version> mcp"})
```
3. Link related pages in `SEE ALSO` with `see_also [[other-page(3)]]`.
Forward references to pages that don't exist yet are fine — they resolve
automatically when the target is written.
4. Validate: `bm tool schema-validate manpage --project manual`.
For mechanical updates to generated sections, prefer `edit_note` with
`replace_section` / `insert_after_section` so curated content (EXAMPLES,
GOTCHAS, SEE ALSO, observations) survives — that ownership split is what the
`generated:` field declares.
## Roadmap
- **Registry generator** — section-3 SYNOPSIS/PARAMETERS generated from the
MCP tool registry (docstrings + pydantic schemas), section-1 from Typer
help; the hand-written corpus is the template spec. Regenerate-and-diff in
CI becomes the drift gate.
- **`bm man <topic>`** — CLI sugar over `read_note` + metadata search.
(`bm man install` + a hand-written `bm.1` already ship — the first slice
of [#610](https://github.com/basicmachines-co/basic-memory/issues/610);
the generator will produce per-command pages from the same extraction.)
- **Docs site** — the notes remain canonical for sections 5 and 7, code is
canonical for 1 and 3; both render to the hosted docs site.
+1 -1
View File
@@ -38,7 +38,7 @@ from typing import Any, Callable
from agent.memory_provider import MemoryProvider
from tools.registry import tool_error
__version__ = "0.22.1"
__version__ = "0.22.0"
logger = logging.getLogger("hermes.memory.basic-memory")
+1 -1
View File
@@ -1,5 +1,5 @@
name: basic-memory
version: 0.22.1
version: 0.22.0
description: "Basic Memory — persistent knowledge graph backed by the basic-memory MCP server"
pip_dependencies:
- mcp
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@basicmemory/openclaw-basic-memory",
"version": "0.22.1",
"version": "0.22.0",
"type": "module",
"main": "./dist/index.js",
"types": "./dist/index.d.ts",
+26 -118
View File
@@ -383,31 +383,17 @@ release version:
echo "❌ Tag {{version}} already exists"
exit 1
fi
# Changelog must already be on main (land it via a normal PR first)
if ! grep -q "^## {{version}} " CHANGELOG.md; then
echo "❌ CHANGELOG.md has no entry for {{version}}. Land one via PR first."
exit 1
fi
# Run quality checks
echo "🔍 Running lint checks..."
just lint
just typecheck
# Update all package manifests to the one Basic Memory product version.
echo "📝 Updating consolidated package versions..."
just set-version "{{version}}"
# Trigger: main's ruleset rejects direct pushes ("Changes must be made
# through a pull request").
# Why: the version bump must land on main before the tag is cut, so it
# rides a release PR that is rebase-merged (the repo disallows merge
# commits).
# Outcome: the bump commit gets a new SHA on main; the tag is created on
# that rebased commit, found by its commit subject.
COMMIT_SUBJECT="chore: update version to $VERSION_NUM for {{version}} release"
git checkout -b "release/{{version}}"
# Commit version update
git add \
src/basic_memory/__init__.py \
server.json \
@@ -418,58 +404,23 @@ release version:
integrations/hermes/plugin.yaml \
integrations/hermes/__init__.py \
integrations/openclaw/package.json
git commit -s -m "$COMMIT_SUBJECT"
echo "📤 Opening release PR..."
git push -u origin "release/{{version}}"
gh pr create --title "chore(core): release {{version}}" \
--body "Version bump for {{version}}. See CHANGELOG.md for release notes."
# Trigger: the PR may not be mergeable synchronously (merge gates,
# required checks added later, or GitHub still computing mergeability).
# Why: the tag must point at the bump commit on main, so the recipe
# cannot tag until the merge has actually landed.
# Outcome: try a direct rebase-merge, fall back to queueing auto-merge,
# then poll main for the rebased bump commit before tagging.
if ! gh pr merge "release/{{version}}" --rebase --delete-branch; then
echo "⚠️ Direct merge did not complete (merge gates pending?). Queueing auto-merge..."
gh pr merge "release/{{version}}" --rebase --delete-branch --auto
fi
echo "⏳ Waiting for the bump commit to land on main..."
TAG_COMMIT=""
for _ in $(seq 1 60); do
git fetch origin main --quiet
TAG_COMMIT=$(git log FETCH_HEAD --fixed-strings --grep "$COMMIT_SUBJECT" --format='%H' -1)
[[ -n "$TAG_COMMIT" ]] && break
sleep 5
done
if [[ -z "$TAG_COMMIT" ]]; then
echo "❌ Bump commit not on main after 5 minutes (merge still pending?)."
echo " Once the release PR merges, finish the release manually:"
echo " git fetch origin main"
echo " git tag {{version}} \$(git log FETCH_HEAD --fixed-strings --grep \"$COMMIT_SUBJECT\" --format='%H' -1)"
echo " git push origin {{version}}"
exit 1
fi
git checkout main
git pull --ff-only origin main
git branch -D "release/{{version}}" 2>/dev/null || true
echo "🏷️ Creating tag {{version}} at $TAG_COMMIT..."
git tag "{{version}}" "$TAG_COMMIT"
git commit -s -m "chore: update version to $VERSION_NUM for {{version}} release"
# Create and push tag
echo "🏷️ Creating tag {{version}}..."
git tag "{{version}}"
echo "📤 Pushing to GitHub..."
git push origin main
git push origin "{{version}}"
echo "✅ Release {{version}} created successfully!"
echo "📦 GitHub Actions will build and publish to PyPI"
echo "🔗 Monitor at: https://github.com/basicmachines-co/basic-memory/actions"
echo ""
echo "📝 REMINDER: Post-release tasks:"
echo " 1. docs.basicmemory.com - Add a What's New page under content/2.whats-new/"
echo " and bump the badge in content/index.md (see that repo's CLAUDE.md)"
echo " 2. basicmemory.com - No version number in the site UI; for a significant"
echo " release optionally add a post under src/content/blog/. Skip for patches."
echo " 1. docs.basicmemory.com - Add release notes to src/pages/latest-releases.mdx"
echo " 2. basicmachines.co - Update version in src/components/sections/hero.tsx"
echo " 3. MCP Registry - Run: mcp-publisher publish"
echo " See: .claude/commands/release/release.md for detailed instructions"
@@ -516,15 +467,7 @@ beta version:
echo "📝 Updating consolidated package versions..."
just set-version "{{version}}"
# Trigger: main's ruleset rejects direct pushes ("Changes must be made
# through a pull request").
# Why: the version bump must land on main before the tag is cut, so it
# rides a release PR that is rebase-merged (the repo disallows merge
# commits).
# Outcome: the bump commit gets a new SHA on main; the tag is created on
# that rebased commit, found by its commit subject.
COMMIT_SUBJECT="chore: update version to $VERSION_NUM for {{version}} beta release"
git checkout -b "release/{{version}}"
# Commit version update
git add \
src/basic_memory/__init__.py \
server.json \
@@ -535,59 +478,24 @@ beta version:
integrations/hermes/plugin.yaml \
integrations/hermes/__init__.py \
integrations/openclaw/package.json
git commit -s -m "$COMMIT_SUBJECT"
echo "📤 Opening release PR..."
git push -u origin "release/{{version}}"
gh pr create --title "chore(core): release {{version}}" \
--body "Version bump for {{version}} beta."
# Trigger: the PR may not be mergeable synchronously (merge gates,
# required checks added later, or GitHub still computing mergeability).
# Why: the tag must point at the bump commit on main, so the recipe
# cannot tag until the merge has actually landed.
# Outcome: try a direct rebase-merge, fall back to queueing auto-merge,
# then poll main for the rebased bump commit before tagging.
if ! gh pr merge "release/{{version}}" --rebase --delete-branch; then
echo "⚠️ Direct merge did not complete (merge gates pending?). Queueing auto-merge..."
gh pr merge "release/{{version}}" --rebase --delete-branch --auto
fi
echo "⏳ Waiting for the bump commit to land on main..."
TAG_COMMIT=""
for _ in $(seq 1 60); do
git fetch origin main --quiet
TAG_COMMIT=$(git log FETCH_HEAD --fixed-strings --grep "$COMMIT_SUBJECT" --format='%H' -1)
[[ -n "$TAG_COMMIT" ]] && break
sleep 5
done
if [[ -z "$TAG_COMMIT" ]]; then
echo "❌ Bump commit not on main after 5 minutes (merge still pending?)."
echo " Once the release PR merges, finish the release manually:"
echo " git fetch origin main"
echo " git tag {{version}} \$(git log FETCH_HEAD --fixed-strings --grep \"$COMMIT_SUBJECT\" --format='%H' -1)"
echo " git push origin {{version}}"
exit 1
fi
git checkout main
git pull --ff-only origin main
git branch -D "release/{{version}}" 2>/dev/null || true
echo "🏷️ Creating tag {{version}} at $TAG_COMMIT..."
git tag "{{version}}" "$TAG_COMMIT"
git commit -s -m "chore: update version to $VERSION_NUM for {{version}} beta release"
# Create and push tag
echo "🏷️ Creating tag {{version}}..."
git tag "{{version}}"
echo "📤 Pushing to GitHub..."
git push origin main
git push origin "{{version}}"
echo "✅ Beta release {{version}} created successfully!"
echo "📦 GitHub Actions will build and publish to PyPI as pre-release"
echo "🔗 Monitor at: https://github.com/basicmachines-co/basic-memory/actions"
echo "📥 Install with: uv tool install basic-memory --pre"
echo ""
echo "📝 REMINDER: For stable releases, update documentation sites:"
echo " 1. docs.basicmemory.com - Add a What's New page under content/2.whats-new/"
echo " and bump the badge in content/index.md (see that repo's CLAUDE.md)"
echo " 2. basicmemory.com - No version number in the site UI; for a significant"
echo " release optionally add a post under src/content/blog/. Skip for patches."
echo " 1. docs.basicmemory.com - Add release notes to src/pages/latest-releases.mdx"
echo " 2. basicmachines.co - Update version in src/components/sections/hero.tsx"
echo " See: .claude/commands/release/release.md for detailed instructions"
# List all available recipes
@@ -6,14 +6,14 @@
},
"metadata": {
"description": "Official plugins from Basic Machines for knowledge management and AI-assisted development",
"version": "0.22.1"
"version": "0.22.0"
},
"plugins": [
{
"name": "basic-memory",
"source": "./",
"description": "The bridge between Claude's working memory and Basic Memory's durable knowledge graph \u2014 session briefings, pre-compaction checkpoints, and capture reflexes",
"version": "0.22.1",
"version": "0.22.0",
"author": {
"name": "Basic Machines"
},
@@ -1,7 +1,7 @@
{
"name": "basic-memory",
"description": "The bridge between Claude's working memory and Basic Memory's durable knowledge graph \u2014 session briefings, pre-compaction checkpoints, and capture reflexes",
"version": "0.22.1",
"version": "0.22.0",
"author": {
"name": "Basic Machines"
},
-55
View File
@@ -1,55 +0,0 @@
---
title: Manpage
type: schema
entity: Manpage
version: 1
schema:
gotcha?(array): string, sharp edges and surprising behavior learned from live verification
example?(array): string, worked examples beyond the generated synopsis
pattern?(array): string, recommended idioms and usage patterns
bug?(array): string, known defects affecting this surface, with issue links
see_also?(array): Entity, related manual pages — the SEE ALSO graph
settings:
validation: warn
frontmatter:
section(enum, Unix manual section number): [1, 3, 5, 7, 8]
name: string, page name without section suffix (e.g. write-note)
summary: string, one-line NAME description
generated?(enum, who owns the mechanical sections): [registry, typer, hand]
tool?: string, MCP tool this page documents (section 3 pages)
command?: string, CLI command this page documents (section 1 pages)
verified?: string, version and path that verified this page (e.g. 0.21.6 mcp+cli)
since?: string, version this surface first appeared
---
# Manpage
A **ManpageNote** is one page of a Unix-style manual implemented as Basic
Memory notes (issue #952): commands in section 1, MCP tools in section 3,
file formats in section 5, concepts in section 7, admin in section 8. The
manual becomes a knowledge graph — `SEE ALSO` entries are typed relations,
and pages are found by structured recall:
`search_notes(metadata_filters={"type": "manpage", "section": 3})`.
This schema is an opt-in seed for documentation projects; the canonical
manual lives in the Basic Memory team workspace `manual` project.
## What makes a good ManpageNote
- **NAME / SYNOPSIS / DESCRIPTION** — classic man-page structure, with
PARAMETERS, MCP USAGE, CLI EQUIVALENT, EXAMPLES, GOTCHAS, SEE ALSO where
applicable.
- **Verified examples** — EXAMPLES contain only commands that actually ran;
the `verified` field records the version and path (mcp, cli, or both).
- **generated** — declares regeneration ownership: `registry` (from the MCP
tool registry) and `typer` (from CLI help) pages get mechanical sections
rewritten; curated sections (EXAMPLES, GOTCHAS, SEE ALSO, observations)
are never overwritten.
- **gotcha / bug observations** — field knowledge accumulates on pages
without being clobbered by regeneration; bugs link their tracking issues.
## Frontmatter
`type: manpage` plus `section` makes the manual queryable like `man -k`:
by section, by `tool`, by `command`, or by missing/stale `verified` stamps.
Validation is `warn`, never blocking.
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "codex",
"version": "0.22.1",
"version": "0.22.0",
"description": "A Codex-native bridge to Basic Memory for durable engineering context, decisions, and resumable checkpoints.",
"author": {
"name": "Basic Machines",
+2 -2
View File
@@ -6,12 +6,12 @@
"url": "https://github.com/basicmachines-co/basic-memory.git",
"source": "github"
},
"version": "0.22.1",
"version": "0.22.0",
"packages": [
{
"registryType": "pypi",
"identifier": "basic-memory",
"version": "0.22.1",
"version": "0.22.0",
"runtimeHint": "uvx",
"runtimeArguments": [
{
+1 -4
View File
@@ -1,7 +1,4 @@
"""basic-memory - Local-first knowledge management combining Zettelkasten with knowledge graphs"""
# Package version - updated by release automation
__version__ = "0.22.1"
# API version for FastAPI - independent of package version
__api_version__ = "v0"
__version__ = "0.22.0"
@@ -17,7 +17,6 @@ from fastapi import APIRouter, HTTPException, Response, Path
from loguru import logger
import logfire
from basic_memory import db
from basic_memory.ignore_utils import (
IGNORED_PATH_REJECTION_DETAIL,
load_gitignore_patterns,
@@ -35,8 +34,6 @@ from basic_memory.deps import (
ProjectExternalIdPathDep,
SyncServiceV2ExternalDep,
TaskSchedulerDep,
SessionDep,
SessionMakerDep,
)
from basic_memory.schemas import DeleteEntitiesResponse
from basic_memory.schemas.base import Entity
@@ -84,7 +81,6 @@ async def get_graph(
project_id: ProjectExternalIdPathDep,
entity_repository: EntityRepositoryV2ExternalDep,
relation_repository: RelationRepositoryV2ExternalDep,
session: SessionDep,
) -> GraphResponse:
"""Return all entities and resolved relations for knowledge graph visualization.
@@ -100,7 +96,7 @@ async def get_graph(
logger.info("API v2 request: get_graph")
# Fetch all entities for this project
entities = await entity_repository.find_all(session, use_load_options=False)
entities = await entity_repository.find_all(use_load_options=False)
nodes = [
GraphNode(
external_id=entity.external_id,
@@ -112,7 +108,7 @@ async def get_graph(
]
# Fetch all resolved relations (to_id is not null) with eager-loaded entities
relations = await relation_repository.find_all(session)
relations = await relation_repository.find_all()
edges = [
GraphEdge(
from_id=relation.from_entity.external_id,
@@ -134,7 +130,6 @@ async def get_graph(
async def get_orphan_entities(
project_id: ProjectExternalIdPathDep,
entity_repository: EntityRepositoryV2ExternalDep,
session: SessionDep,
) -> OrphanEntitiesResponse:
"""Return entities that have no incoming or outgoing relations."""
with logfire.span(
@@ -145,7 +140,7 @@ async def get_orphan_entities(
):
logger.info("API v2 request: get_orphan_entities")
entities = await entity_repository.find_without_relations(session)
entities = await entity_repository.find_without_relations()
nodes = [
GraphNode(
external_id=entity.external_id,
@@ -170,7 +165,6 @@ async def resolve_identifier(
link_resolver: LinkResolverV2ExternalDep,
entity_repository: EntityRepositoryV2ExternalDep,
project_repository: ProjectRepositoryDep,
session: SessionDep,
) -> EntityResolveResponse:
"""Resolve a string identifier (external_id, permalink, title, or path) to entity info.
@@ -209,15 +203,12 @@ async def resolve_identifier(
):
logger.info(f"API v2 request: resolve_identifier for '{data.identifier}'")
entity = await entity_repository.get_by_external_id(session, data.identifier)
entity = await entity_repository.get_by_external_id(data.identifier)
resolution_method = "external_id" if entity else "search"
if not entity:
entity = await link_resolver.resolve_link(
data.identifier,
source_path=data.source_path,
strict=data.strict,
session=session,
data.identifier, source_path=data.source_path, strict=data.strict
)
if entity:
if entity.permalink == data.identifier:
@@ -232,7 +223,7 @@ async def resolve_identifier(
if not entity:
raise HTTPException(status_code=404, detail=f"Entity not found: '{data.identifier}'")
owner_project = await project_repository.get_by_id(session, entity.project_id)
owner_project = await project_repository.get_by_id(entity.project_id)
if not owner_project: # pragma: no cover
raise HTTPException(
status_code=500,
@@ -313,7 +304,6 @@ async def sync_file(
project_config: ProjectConfigV2ExternalDep,
search_service: SearchServiceV2ExternalDep,
app_config: AppConfigDep,
session_maker: SessionMakerDep,
) -> EntityResponseV2:
"""Index a single markdown file that exists on disk but is not indexed yet.
@@ -416,8 +406,7 @@ async def sync_file(
# Trigger: the file may already have a DB row (e.g. modified on disk after indexing)
# Why: the indexer needs to know whether to insert or update the entity
# Outcome: new is computed from the database instead of assumed by the caller
async with db.scoped_session(session_maker) as session:
existing = await sync_service.entity_repository.get_by_file_path(session, file_path)
existing = await sync_service.entity_repository.get_by_file_path(file_path)
synced = await sync_service.sync_one_markdown_file(
file_path, new=existing is None, index_search=True
)
@@ -446,7 +435,6 @@ async def sync_file(
async def get_entity_by_id(
project_id: ProjectExternalIdPathDep,
entity_repository: EntityRepositoryV2ExternalDep,
session: SessionDep,
entity_id: str = Path(..., description="Entity external ID (UUID)"),
) -> EntityResponseV2:
"""Get an entity by its external ID (UUID).
@@ -471,7 +459,7 @@ async def get_entity_by_id(
):
logger.info(f"API v2 request: get_entity_by_id entity_id={entity_id}")
entity = await entity_repository.get_by_external_id(session, entity_id)
entity = await entity_repository.get_by_external_id(entity_id)
if not entity:
raise HTTPException(
status_code=404, detail=f"Entity with external_id '{entity_id}' not found"
@@ -548,7 +536,6 @@ async def update_entity_by_id(
entity_repository: EntityRepositoryV2ExternalDep,
task_scheduler: TaskSchedulerDep,
app_config: AppConfigDep,
session_maker: SessionMakerDep,
entity_id: str = Path(..., description="Entity external ID (UUID)"),
) -> EntityResponseV2:
"""Update an entity by external ID.
@@ -570,8 +557,7 @@ async def update_entity_by_id(
):
logger.info(f"API v2 request: update_entity_by_id entity_id={entity_id}")
async with db.scoped_session(session_maker) as session:
existing = await entity_repository.get_by_external_id(session, entity_id)
existing = await entity_repository.get_by_external_id(entity_id)
created = existing is None
if existing:
@@ -582,12 +568,10 @@ async def update_entity_by_id(
write_result = await entity_service.create_entity_with_content(data)
entity = write_result.entity
if entity.external_id != entity_id:
async with db.scoped_session(session_maker) as session:
entity = await entity_repository.update(
session,
entity.id,
{"external_id": entity_id},
)
entity = await entity_repository.update(
entity.id,
{"external_id": entity_id},
)
# external_id fixup only changes the DB row. The file content is unchanged,
# so the markdown captured during the write remains valid downstream.
if not entity:
@@ -623,7 +607,6 @@ async def edit_entity_by_id(
entity_repository: EntityRepositoryV2ExternalDep,
task_scheduler: TaskSchedulerDep,
app_config: AppConfigDep,
session_maker: SessionMakerDep,
entity_id: str = Path(..., description="Entity external ID (UUID)"),
) -> EntityResponseV2:
"""Edit an existing entity by external ID using operations like append, prepend, etc.
@@ -648,8 +631,7 @@ async def edit_entity_by_id(
f"API v2 request: edit_entity_by_id entity_id={entity_id}, operation='{data.operation}'"
)
async with db.scoped_session(session_maker) as session:
entity = await entity_repository.get_by_external_id(session, entity_id)
entity = await entity_repository.get_by_external_id(entity_id)
if not entity: # pragma: no cover
raise HTTPException(
status_code=404, detail=f"Entity with external_id '{entity_id}' not found"
@@ -696,7 +678,6 @@ async def delete_entity_by_id(
project_id: ProjectExternalIdPathDep,
entity_service: EntityServiceV2ExternalDep,
entity_repository: EntityRepositoryV2ExternalDep,
session_maker: SessionMakerDep,
entity_id: str = Path(..., description="Entity external ID (UUID)"),
) -> DeleteEntitiesResponse:
"""Delete an entity by external ID.
@@ -717,8 +698,7 @@ async def delete_entity_by_id(
):
logger.info(f"API v2 request: delete_entity_by_id entity_id={entity_id}")
async with db.scoped_session(session_maker) as session:
entity = await entity_repository.get_by_external_id(session, entity_id)
entity = await entity_repository.get_by_external_id(entity_id)
if entity is None:
logger.info(f"API v2 response: external_id={entity_id} not found, deleted=False")
return DeleteEntitiesResponse(deleted=False)
@@ -744,7 +724,6 @@ async def move_entity(
app_config: AppConfigDep,
search_service: SearchServiceV2ExternalDep,
task_scheduler: TaskSchedulerDep,
session_maker: SessionMakerDep,
entity_id: str = Path(..., description="Entity external ID (UUID)"),
) -> EntityResponseV2:
"""Move an entity to a new file location.
@@ -772,8 +751,7 @@ async def move_entity(
try:
# First, get the entity by external_id to verify it exists
async with db.scoped_session(session_maker) as session:
entity = await entity_repository.get_by_external_id(session, entity_id)
entity = await entity_repository.get_by_external_id(entity_id)
if not entity: # pragma: no cover
raise HTTPException(
status_code=404, detail=f"Entity with external_id '{entity_id}' not found"
@@ -788,11 +766,9 @@ async def move_entity(
)
# Reindex at new location
async with db.scoped_session(session_maker) as session:
reindexed_entity = await entity_service.link_resolver.resolve_link(
data.destination_path,
session=session,
)
reindexed_entity = await entity_service.link_resolver.resolve_link(
data.destination_path
)
if reindexed_entity:
await search_service.index_entity(reindexed_entity)
_schedule_vector_sync_if_enabled(
@@ -829,7 +805,6 @@ async def move_directory(
app_config: AppConfigDep,
search_service: SearchServiceV2ExternalDep,
task_scheduler: TaskSchedulerDep,
session_maker: SessionMakerDep,
) -> DirectoryMoveResult:
"""Move all entities in a directory to a new location.
@@ -865,10 +840,7 @@ async def move_directory(
# Reindex moved entities
for file_path in result.moved_files:
async with db.scoped_session(session_maker) as session:
entity = await entity_service.link_resolver.resolve_link(
file_path, session=session
)
entity = await entity_service.link_resolver.resolve_link(file_path)
if entity:
await search_service.index_entity(entity)
_schedule_vector_sync_if_enabled(
@@ -10,12 +10,7 @@ from fastapi import APIRouter, Query, Path
from loguru import logger
import logfire
from basic_memory import db
from basic_memory.deps import (
ContextServiceV2ExternalDep,
EntityRepositoryV2ExternalDep,
SessionMakerDep,
)
from basic_memory.deps import ContextServiceV2ExternalDep, EntityRepositoryV2ExternalDep
from basic_memory.schemas.base import TimeFrame, parse_timeframe
from basic_memory.schemas.memory import (
GraphContext,
@@ -32,7 +27,6 @@ router = APIRouter(tags=["memory"])
async def recent(
context_service: ContextServiceV2ExternalDep,
entity_repository: EntityRepositoryV2ExternalDep,
session_maker: SessionMakerDep,
project_id: str = Path(..., description="Project external UUID"),
type: Annotated[list[SearchItemType] | None, Query()] = None,
depth: int = 1,
@@ -101,14 +95,9 @@ async def recent(
phase="shape_response",
result_count=len(context.results),
):
async with db.scoped_session(session_maker) as session:
recent_context = await to_graph_context(
context,
entity_repository=entity_repository,
session=session,
page=page,
page_size=page_size,
)
recent_context = await to_graph_context(
context, entity_repository=entity_repository, page=page, page_size=page_size
)
logger.debug(f"V2 Recent context: {recent_context.model_dump_json()}")
return recent_context
@@ -120,7 +109,6 @@ async def recent(
async def get_memory_context(
context_service: ContextServiceV2ExternalDep,
entity_repository: EntityRepositoryV2ExternalDep,
session_maker: SessionMakerDep,
uri: str,
project_id: str = Path(..., description="Project external UUID"),
depth: int = 1,
@@ -189,11 +177,6 @@ async def get_memory_context(
phase="shape_response",
result_count=len(context.results),
):
async with db.scoped_session(session_maker) as session:
return await to_graph_context(
context,
entity_repository=entity_repository,
session=session,
page=page,
page_size=page_size,
)
return await to_graph_context(
context, entity_repository=entity_repository, page=page, page_size=page_size
)
@@ -16,7 +16,6 @@ from typing import Literal, Optional
from fastapi import APIRouter, HTTPException, Body, Query, Path
from loguru import logger
from basic_memory import db
from basic_memory.deps import (
ProjectServiceDep,
ProjectRepositoryDep,
@@ -24,8 +23,6 @@ from basic_memory.deps import (
SyncServiceV2ExternalDep,
TaskSchedulerDep,
ProjectExternalIdPathDep,
SessionDep,
SessionMakerDep,
)
from basic_memory.schemas import SyncReportResponse
from basic_memory.models import Project
@@ -57,22 +54,21 @@ def _split_qualified_project_identifier(identifier: str) -> tuple[str | None, st
async def _resolve_project_identifier_candidate(
session: SessionDep,
project_repository: ProjectRepository,
identifier: str,
) -> tuple[Project | None, ProjectResolveMethod]:
"""Resolve one project identifier candidate and report the matching method."""
identifier_permalink = generate_permalink(identifier)
project = await project_repository.get_by_external_id(session, identifier)
project = await project_repository.get_by_external_id(identifier)
if project:
return project, "external_id"
project = await project_repository.get_by_permalink(session, identifier_permalink)
project = await project_repository.get_by_permalink(identifier_permalink)
if project:
return project, "permalink"
project = await project_repository.get_by_name_case_insensitive(session, identifier)
project = await project_repository.get_by_name_case_insensitive(identifier)
if project:
return project, "name" # pragma: no cover
@@ -80,13 +76,11 @@ async def _resolve_project_identifier_candidate(
async def _resolve_project_identifier(
session: SessionDep,
project_repository: ProjectRepository,
identifier: str,
) -> tuple[Project | None, ProjectResolveMethod]:
"""Resolve exact identifiers first, then accepted workspace-qualified forms."""
project, resolution_method = await _resolve_project_identifier_candidate(
session,
project_repository,
identifier,
)
@@ -102,7 +96,6 @@ async def _resolve_project_identifier(
# only needs the project segment to validate the active project.
# Outcome: models can follow the hint verbatim instead of looping on a 404.
project, resolution_method = await _resolve_project_identifier_candidate(
session,
project_repository,
project_identifier,
)
@@ -200,7 +193,7 @@ async def add_project(
return ProjectStatusResponse( # pyright: ignore [reportCallIssue]
message=f"Project '{new_project.name}' added successfully",
status="success",
default=new_project.is_default or False,
default=project_data.set_default,
new_project=ProjectItem(
id=new_project.id,
external_id=new_project.external_id,
@@ -284,7 +277,6 @@ async def get_project_status(
@router.post("/resolve", response_model=ProjectResolveResponse)
async def resolve_project_identifier(
data: ProjectResolveRequest,
session: SessionDep,
project_repository: ProjectRepositoryDep,
) -> ProjectResolveResponse:
"""Resolve a project identifier (name, permalink, or external_id) to project info.
@@ -323,26 +315,12 @@ async def resolve_project_identifier(
logger.info(f"API v2 request: resolve_project_identifier for '{data.identifier}'")
project, resolution_method = await _resolve_project_identifier(
session,
project_repository,
data.identifier,
)
if not project:
detail = f"Project not found: '{data.identifier}'"
# Trigger: resolution missed and the projects table is empty.
# Why: a fresh install bootstraps config.json's default project before any
# reconciliation has created database rows (the one-shot CLI never runs
# the server lifespan), so the first read fails on the configured
# default and the bare not-found message reads as a broken install
# rather than a missing first-run step (#974 follow-up).
# Outcome: the error names the setup command instead.
if not await project_repository.find_all(session, limit=1, use_load_options=False):
detail = (
f"{detail}. No projects are set up yet — run "
"'basic-memory project add <name> <path>' to create one."
)
raise HTTPException(status_code=404, detail=detail)
raise HTTPException(status_code=404, detail=f"Project not found: '{data.identifier}'")
return ProjectResolveResponse(
external_id=project.external_id,
@@ -358,7 +336,6 @@ async def resolve_project_identifier(
@router.get("/{project_id}", response_model=ProjectItem)
async def get_project_by_id(
session: SessionDep,
project_repository: ProjectRepositoryDep,
project_id: str = Path(..., description="Project external ID (UUID)"),
) -> ProjectItem:
@@ -381,7 +358,7 @@ async def get_project_by_id(
"""
logger.info(f"API v2 request: get_project_by_id for project_id={project_id}")
project = await project_repository.get_by_external_id(session, project_id)
project = await project_repository.get_by_external_id(project_id)
if not project:
raise HTTPException(
status_code=404, detail=f"Project with external_id '{project_id}' not found"
@@ -399,14 +376,12 @@ async def get_project_by_id(
@router.get("/{project_id}/info", response_model=ProjectInfoResponse)
async def get_project_info_by_id(
project_service: ProjectServiceDep,
session_maker: SessionMakerDep,
project_repository: ProjectRepositoryDep,
project_id: str = Path(..., description="Project external ID (UUID)"),
) -> ProjectInfoResponse:
"""Get detailed project information by external ID."""
logger.info(f"API v2 request: get_project_info_by_id for project_id={project_id}")
async with db.scoped_session(session_maker) as session:
project = await project_repository.get_by_external_id(session, project_id)
project = await project_repository.get_by_external_id(project_id)
if not project:
raise HTTPException(
status_code=404, detail=f"Project with external_id '{project_id}' not found"
@@ -417,7 +392,6 @@ async def get_project_info_by_id(
@router.patch("/{project_id}", response_model=ProjectStatusResponse)
async def update_project_by_id(
project_service: ProjectServiceDep,
session_maker: SessionMakerDep,
project_repository: ProjectRepositoryDep,
project_id: str = Path(..., description="Project external ID (UUID)"),
path: Optional[str] = Body(None, description="New absolute path for the project"),
@@ -448,8 +422,7 @@ async def update_project_by_id(
raise HTTPException(status_code=400, detail="Path must be absolute")
# Get original project info for the response
async with db.scoped_session(session_maker) as session:
old_project = await project_repository.get_by_external_id(session, project_id)
old_project = await project_repository.get_by_external_id(project_id)
if not old_project:
raise HTTPException(
status_code=404, detail=f"Project with external_id '{project_id}' not found"
@@ -470,8 +443,7 @@ async def update_project_by_id(
await project_service.update_project(old_project.name, is_active=is_active)
# Get updated project info (use the same external_id)
async with db.scoped_session(session_maker) as session:
updated_project = await project_repository.get_by_external_id(session, project_id)
updated_project = await project_repository.get_by_external_id(project_id)
if not updated_project: # pragma: no cover
raise HTTPException(
status_code=404,
@@ -498,7 +470,6 @@ async def update_project_by_id(
@router.delete("/{project_id}", response_model=ProjectStatusResponse)
async def delete_project_by_id(
project_service: ProjectServiceDep,
session_maker: SessionMakerDep,
project_repository: ProjectRepositoryDep,
project_id: str = Path(..., description="Project external ID (UUID)"),
delete_notes: bool = Query(
@@ -525,8 +496,7 @@ async def delete_project_by_id(
)
try:
async with db.scoped_session(session_maker) as session:
old_project = await project_repository.get_by_external_id(session, project_id)
old_project = await project_repository.get_by_external_id(project_id)
if not old_project:
raise HTTPException(
status_code=404, detail=f"Project with external_id '{project_id}' not found"
@@ -569,7 +539,6 @@ async def delete_project_by_id(
@router.put("/{project_id}/default", response_model=ProjectStatusResponse)
async def set_default_project_by_id(
project_service: ProjectServiceDep,
session_maker: SessionMakerDep,
project_repository: ProjectRepositoryDep,
project_id: str = Path(..., description="Project external ID (UUID)"),
) -> ProjectStatusResponse:
@@ -590,14 +559,15 @@ async def set_default_project_by_id(
logger.info(f"API v2 request: set_default_project_by_id for project_id={project_id}")
try:
# Get the old default project from database. It may be absent during
# bootstrap/recovery (no default row yet); that is a valid state, not an
# error, so we only echo it back when one exists.
async with db.scoped_session(session_maker) as session:
default_project = await project_repository.get_default_project(session)
# Get the old default project from database
default_project = await project_repository.get_default_project()
if not default_project:
raise HTTPException( # pragma: no cover
status_code=404, detail="No default project is currently set"
)
# Get the new default project by external_id
new_default_project = await project_repository.get_by_external_id(session, project_id)
# Get the new default project by external_id
new_default_project = await project_repository.get_by_external_id(project_id)
if not new_default_project:
raise HTTPException(
status_code=404, detail=f"Project with external_id '{project_id}' not found"
@@ -606,27 +576,17 @@ async def set_default_project_by_id(
# Set as default using project name (service layer still uses names internally)
await project_service.set_default_project(new_default_project.name)
# Trigger: a previous default existed
# Why: ProjectStatusResponse.old_project is Optional; the no-default
# bootstrap case must succeed with old_project=None
# Outcome: response echoes the prior default only when there was one
old_project = (
ProjectItem(
return ProjectStatusResponse(
message=f"Project '{new_default_project.name}' set as default successfully",
status="success",
default=True,
old_project=ProjectItem(
id=default_project.id,
external_id=default_project.external_id,
name=default_project.name,
path=default_project.path,
is_default=False,
)
if default_project
else None
)
return ProjectStatusResponse(
message=f"Project '{new_default_project.name}' set as default successfully",
status="success",
default=True,
old_project=old_project,
),
new_project=ProjectItem(
id=new_default_project.id,
external_id=new_default_project.external_id,
@@ -10,7 +10,6 @@ from typing import Any
from fastapi import APIRouter, HTTPException, status, Path
from loguru import logger
from basic_memory import db
from basic_memory.api.v2.utils import to_graph_context, to_search_results
from basic_memory.api.template_loader import template_loader
from basic_memory.schemas.base import parse_timeframe
@@ -19,7 +18,6 @@ from basic_memory.deps import (
EntityRepositoryV2ExternalDep,
SearchServiceV2ExternalDep,
EntityServiceV2ExternalDep,
SessionMakerDep,
)
from basic_memory.schemas.prompt import (
ContinueConversationRequest,
@@ -38,7 +36,6 @@ async def continue_conversation(
entity_service: EntityServiceV2ExternalDep,
context_service: ContextServiceV2ExternalDep,
entity_repository: EntityRepositoryV2ExternalDep,
session_maker: SessionMakerDep,
request: ContinueConversationRequest,
project_id: str = Path(..., description="Project external UUID"),
) -> PromptResponse:
@@ -85,10 +82,9 @@ async def continue_conversation(
)
# Process results into the schema format
async with db.scoped_session(session_maker) as session:
graph_context = await to_graph_context(
context_result, entity_repository=entity_repository, session=session
)
graph_context = await to_graph_context(
context_result, entity_repository=entity_repository
)
# Add results to our collection (limit to top results for each permalink)
if graph_context.results:
@@ -113,10 +109,7 @@ async def continue_conversation(
max_related=request.related_items_limit,
include_observations=True,
)
async with db.scoped_session(session_maker) as session:
recent_context = await to_graph_context(
context_result, entity_repository=entity_repository, session=session
)
recent_context = await to_graph_context(context_result, entity_repository=entity_repository)
hierarchical_results = recent_context.results[:5] # Limit to top 5 recent items
@@ -16,14 +16,11 @@ from fastapi import APIRouter, HTTPException, Response, Path
from loguru import logger
import logfire
from basic_memory import db
from basic_memory.deps import (
ProjectConfigV2ExternalDep,
FileServiceV2ExternalDep,
EntityRepositoryV2ExternalDep,
SearchServiceV2ExternalDep,
SessionDep,
SessionMakerDep,
)
from basic_memory.models.knowledge import Entity as EntityModel
from basic_memory.schemas.v2.resource import (
@@ -41,7 +38,6 @@ async def get_resource_content(
config: ProjectConfigV2ExternalDep,
entity_repository: EntityRepositoryV2ExternalDep,
file_service: FileServiceV2ExternalDep,
session: SessionDep,
project_id: str = Path(..., description="Project external UUID"),
entity_id: str = Path(..., description="Entity external UUID"),
) -> Response:
@@ -74,7 +70,7 @@ async def get_resource_content(
action="get_content",
phase="load_entity",
):
entity = await entity_repository.get_by_external_id(session, entity_id)
entity = await entity_repository.get_by_external_id(entity_id)
if not entity:
raise HTTPException(status_code=404, detail=f"Entity {entity_id} not found")
@@ -125,7 +121,6 @@ async def create_resource(
file_service: FileServiceV2ExternalDep,
entity_repository: EntityRepositoryV2ExternalDep,
search_service: SearchServiceV2ExternalDep,
session_maker: SessionMakerDep,
project_id: str = Path(..., description="Project external UUID"),
) -> ResourceResponse:
"""Create a new resource file.
@@ -163,14 +158,13 @@ async def create_resource(
"Path must be relative and stay within project boundaries.",
)
async with db.scoped_session(session_maker) as session:
existing_entity = await entity_repository.get_by_file_path(session, data.file_path)
if existing_entity:
raise HTTPException(
status_code=409,
detail=f"Resource already exists at {data.file_path} with entity_id {existing_entity.external_id}. "
f"Use PUT /resource/{existing_entity.external_id} to update it.",
)
existing_entity = await entity_repository.get_by_file_path(data.file_path)
if existing_entity:
raise HTTPException(
status_code=409,
detail=f"Resource already exists at {data.file_path} with entity_id {existing_entity.external_id}. "
f"Use PUT /resource/{existing_entity.external_id} to update it.",
)
with logfire.span(
"api.resource.create.write_file",
@@ -209,8 +203,7 @@ async def create_resource(
action="create",
phase="upsert_entity",
):
async with db.scoped_session(session_maker) as session:
entity = await entity_repository.add(session, entity)
entity = await entity_repository.add(entity)
with logfire.span(
"api.resource.create.search_index",
@@ -243,7 +236,6 @@ async def update_resource(
file_service: FileServiceV2ExternalDep,
entity_repository: EntityRepositoryV2ExternalDep,
search_service: SearchServiceV2ExternalDep,
session_maker: SessionMakerDep,
project_id: str = Path(..., description="Project external UUID"),
entity_id: str = Path(..., description="Entity external UUID"),
) -> ResourceResponse:
@@ -273,8 +265,7 @@ async def update_resource(
action="update",
):
try:
async with db.scoped_session(session_maker) as session:
entity = await entity_repository.get_by_external_id(session, entity_id)
entity = await entity_repository.get_by_external_id(entity_id)
if not entity:
raise HTTPException(status_code=404, detail=f"Entity {entity_id} not found")
@@ -324,19 +315,17 @@ async def update_resource(
action="update",
phase="update_entity",
):
async with db.scoped_session(session_maker) as session:
updated_entity = await entity_repository.update(
session,
entity.id,
{
"title": file_name,
"note_type": note_type,
"content_type": content_type,
"file_path": target_file_path,
"checksum": checksum,
"updated_at": file_metadata.modified_at,
},
)
updated_entity = await entity_repository.update(
entity.id,
{
"title": file_name,
"note_type": note_type,
"content_type": content_type,
"file_path": target_file_path,
"checksum": checksum,
"updated_at": file_metadata.modified_at,
},
)
if updated_entity is None:
raise HTTPException(status_code=404, detail=f"Entity {entity_id} not found")
@@ -13,13 +13,11 @@ from pathlib import Path as FilePath
import frontmatter
from fastapi import APIRouter, Path, Query
from loguru import logger
from sqlalchemy.ext.asyncio import AsyncSession
from basic_memory.deps import (
EntityRepositoryV2ExternalDep,
FileServiceV2ExternalDep,
LinkResolverV2ExternalDep,
SessionDep,
)
from basic_memory.models.knowledge import Entity
from basic_memory.schemas.schema import (
@@ -133,7 +131,6 @@ async def validate_schema(
entity_repository: EntityRepositoryV2ExternalDep,
file_service: FileServiceV2ExternalDep,
link_resolver: LinkResolverV2ExternalDep,
session: SessionDep,
project_id: str = Path(..., description="Project external UUID"),
note_type: str | None = Query(None, description="Note type to validate"),
identifier: str | None = Query(None, description="Specific note identifier"),
@@ -153,7 +150,7 @@ async def validate_schema(
if identifier:
# Resolve identifier flexibly (permalink, title, path, fuzzy)
# to match how read_note and other tools resolve identifiers
entity = await link_resolver.resolve_link(identifier, session=session)
entity = await link_resolver.resolve_link(identifier)
if not entity:
return ValidationReport(note_type=note_type, total_notes=0, total_entities=0)
@@ -162,7 +159,6 @@ async def validate_schema(
async def search_fn(query: str) -> list[dict]:
entities = await _find_schema_entities(
session,
entity_repository,
query,
allow_reference_match=isinstance(schema_ref, str) and query == schema_ref,
@@ -191,7 +187,7 @@ async def validate_schema(
)
# --- Batch validation by note type ---
entities = await _find_by_note_type(session, entity_repository, note_type) if note_type else []
entities = await _find_by_note_type(entity_repository, note_type) if note_type else []
for entity in entities:
frontmatter = _entity_frontmatter(entity)
@@ -199,7 +195,6 @@ async def validate_schema(
async def search_fn(query: str) -> list[dict]:
entities = await _find_schema_entities(
session,
entity_repository,
query,
allow_reference_match=isinstance(schema_ref, str) and query == schema_ref,
@@ -235,7 +230,6 @@ async def validate_schema(
@router.post("/schema/infer", response_model=InferenceReport)
async def infer_schema_endpoint(
entity_repository: EntityRepositoryV2ExternalDep,
session: SessionDep,
project_id: str = Path(..., description="Project external UUID"),
note_type: str = Query(..., description="Note type to analyze"),
threshold: float = Query(0.25, description="Minimum frequency for optional fields"),
@@ -245,7 +239,7 @@ async def infer_schema_endpoint(
Examines observation categories and relation types across all notes
of the given type. Returns frequency analysis and suggested Picoschema.
"""
entities = await _find_by_note_type(session, entity_repository, note_type)
entities = await _find_by_note_type(entity_repository, note_type)
notes_data = [_entity_to_note_data(entity) for entity in entities]
result = infer_schema(note_type, notes_data, optional_threshold=threshold)
@@ -280,7 +274,6 @@ async def infer_schema_endpoint(
async def diff_schema_endpoint(
entity_repository: EntityRepositoryV2ExternalDep,
file_service: FileServiceV2ExternalDep,
session: SessionDep,
note_type: str = Path(..., description="Note type to check for drift"),
project_id: str = Path(..., description="Project external UUID"),
):
@@ -292,7 +285,7 @@ async def diff_schema_endpoint(
"""
async def search_fn(query: str) -> list[dict]:
entities = await _find_schema_entities(session, entity_repository, query)
entities = await _find_schema_entities(entity_repository, query)
return [await _schema_frontmatter_from_file(file_service, e) for e in entities]
# Resolve schema by note type
@@ -303,7 +296,7 @@ async def diff_schema_endpoint(
return DriftReport(note_type=note_type, schema_found=False)
# Collect all notes of this type
entities = await _find_by_note_type(session, entity_repository, note_type)
entities = await _find_by_note_type(entity_repository, note_type)
notes_data = [_entity_to_note_data(entity) for entity in entities]
result = diff_schema(schema_def, notes_data)
@@ -338,18 +331,16 @@ async def diff_schema_endpoint(
async def _find_by_note_type(
session: AsyncSession,
entity_repository: EntityRepositoryV2ExternalDep,
note_type: str,
) -> list[Entity]:
"""Find all entities of a given type using the repository's select pattern."""
query = entity_repository.select().where(Entity.note_type == note_type)
result = await entity_repository.execute_query(session, query)
result = await entity_repository.execute_query(query)
return list(result.scalars().all())
async def _find_schema_entities(
session: AsyncSession,
entity_repository: EntityRepositoryV2ExternalDep,
target_note_type: str,
*,
@@ -364,7 +355,7 @@ async def _find_schema_entities(
exact reference matching by title/permalink (explicit schema references)
"""
query = entity_repository.select().where(Entity.note_type == "schema")
result = await entity_repository.execute_query(session, query)
result = await entity_repository.execute_query(query)
entities = list(result.scalars().all())
normalized_target = generate_permalink(target_note_type)
+2 -9
View File
@@ -1,7 +1,6 @@
from typing import Any, Protocol, Optional, List, Sequence
import logfire
from sqlalchemy.ext.asyncio import AsyncSession
from basic_memory.repository.search_repository import SearchIndexRow
from basic_memory.schemas.memory import (
EntitySummary,
@@ -20,11 +19,7 @@ from basic_memory.services.context_service import (
class EntityBatchLookup(Protocol):
async def find_by_ids_for_hydration(
self,
session: AsyncSession,
ids: List[int],
*,
include_cross_project: bool = False,
self, ids: List[int], *, include_cross_project: bool = False
) -> Sequence[Any]: ...
@@ -47,7 +42,6 @@ def _search_item_type(value: str | SearchItemType) -> SearchItemType:
async def to_graph_context(
context_result: ServiceContextResult,
entity_repository: EntityBatchLookup,
session: AsyncSession,
page: Optional[int] = None,
page_size: Optional[int] = None,
) -> GraphContext:
@@ -96,7 +90,7 @@ async def to_graph_context(
result_count=len(entity_ids_needed),
):
entities = await entity_repository.find_by_ids_for_hydration(
session, list(entity_ids_needed), include_cross_project=True
list(entity_ids_needed), include_cross_project=True
)
for e in entities:
entity_title_lookup[e.id] = e.title
@@ -153,7 +147,6 @@ async def to_graph_context(
from_entity_id=item.from_id,
from_entity_external_id=from_ext_id,
to_entity=to_title,
to_name=item.to_name,
to_entity_id=item.to_id,
to_entity_external_id=to_ext_id,
created_at=item.created_at,
-3
View File
@@ -83,11 +83,8 @@ def app_callback(
# Skip for 'mcp' command - it has its own lifespan that handles initialization
# Skip for API-using commands (status, sync, etc.) - they handle initialization via deps.py
# Skip for 'reset' command - it manages its own database lifecycle
# Skip for 'man' - it only copies packaged files; a broken local database
# must not block installing the offline docs
skip_init_commands = {
"doctor",
"man",
"mcp",
"status",
"sync",
-15
View File
@@ -168,20 +168,6 @@ def _manual_update_hint(source: InstallSource) -> str:
)
def _preload_lazy_console_modules() -> None:
"""Import modules the post-upgrade output path defers until print time.
Trigger: an in-place upgrade is about to replace this installation on disk.
Why: rich and typer defer some imports until print/excepthook time; once
`brew upgrade` / `uv tool upgrade` removes the running version's files,
those imports raise ModuleNotFoundError and the final status message
crashes the exiting process.
Outcome: import the deferred modules now, while the files still exist.
"""
import rich._emoji_codes # noqa: F401
import typer.rich_utils # noqa: F401
def _save_last_checked_timestamp(config_manager: ConfigManager, checked_at: datetime) -> None:
"""Persist the timestamp for the most recent attempted update check."""
config = config_manager.load_config()
@@ -302,7 +288,6 @@ def run_auto_update(
else BREW_UPGRADE_TIMEOUT_SECONDS
)
_preload_lazy_console_modules()
install_result = _run_subprocess(
command,
timeout_seconds=timeout,
@@ -4,7 +4,6 @@ from . import ci, status, db, doctor, import_memory_json, mcp, import_claude_con
from . import (
import_claude_projects,
import_chatgpt,
man,
tool,
project,
format,
@@ -30,5 +29,4 @@ __all__ = [
"schema",
"update",
"workspace",
"man",
]
+2 -12
View File
@@ -34,10 +34,8 @@ from basic_memory.ci.project_updates import (
from basic_memory.cli.app import app
from basic_memory.cli.commands.command_utils import run_with_cleanup
from basic_memory.cli.commands.routing import force_routing, validate_routing_flags
# MCP tool functions are imported inside the async helpers below: importing
# basic_memory.mcp.tools loads the entire tool stack (fastmcp, mcp SDK,
# SQLAlchemy), which would slow every CLI invocation, including --help (#886).
from basic_memory.mcp.tools import search_notes as mcp_search_notes
from basic_memory.mcp.tools import write_note as mcp_write_note
console = Console()
@@ -254,10 +252,6 @@ async def seed_project_update_schemas(
refresh: bool = False,
) -> list[str]:
"""Seed Auto BM schema notes without overwriting customized schemas."""
# Deferred: loading the MCP tool stack at module import slows CLI startup (#886).
from basic_memory.mcp.tools import search_notes as mcp_search_notes
from basic_memory.mcp.tools import write_note as mcp_write_note
seeded: list[str] = []
routed_project = _routed_project(project=project, project_id=project_id, workspace=workspace)
for spec in schema_seed_specs():
@@ -301,10 +295,6 @@ async def publish_project_update_note(
note: ProjectUpdateNote,
) -> dict[str, Any]:
"""Search by idempotency key and then upsert the deterministic note path."""
# Deferred: loading the MCP tool stack at module import slows CLI startup (#886).
from basic_memory.mcp.tools import search_notes as mcp_search_notes
from basic_memory.mcp.tools import write_note as mcp_write_note
routed_project = _routed_project(
project=config.project,
project_id=config.project_id,
@@ -3,10 +3,12 @@
import asyncio
from typing import Optional, TypeVar, Coroutine, Any
from mcp.server.fastmcp.exceptions import ToolError
import typer
from rich.console import Console
from basic_memory import db
from basic_memory.config import ConfigManager
from basic_memory.mcp.async_client import get_client
from basic_memory.mcp.clients import ProjectClient
@@ -29,9 +31,6 @@ def run_with_cleanup(coro: Coroutine[Any, Any, T]) -> T:
Returns:
The result of the coroutine
"""
# Deferred: basic_memory.db pulls SQLAlchemy + Alembic, which must not load
# at CLI import time — only when a command actually runs (#886).
from basic_memory import db
async def _with_cleanup() -> T:
try:
@@ -54,8 +53,6 @@ async def run_sync(
force_full: If True, force a full scan bypassing watermark optimization
run_in_background: If True, return immediately; if False, wait for completion
"""
# Deferred: ToolError lives in the mcp SDK, which must not load at CLI startup (#886).
from mcp.server.fastmcp.exceptions import ToolError
# Resolve default project so get_client() can route per-project
project = project or ConfigManager().default_project
@@ -89,9 +86,6 @@ async def run_sync(
async def get_project_info(project: str):
"""Get project information via API endpoint."""
# Deferred: ToolError lives in the mcp SDK, which must not load at CLI startup (#886).
from mcp.server.fastmcp.exceptions import ToolError
try:
async with get_client(project_name=project) as client:
project_item = await get_active_project(client, project, None)
+13 -41
View File
@@ -1,27 +1,24 @@
"""Database management commands."""
# PEP 563 lazy annotations let signatures reference IndexProgress without importing
# the indexing stack at module load; reset/reindex import their heavy database and
# sync dependencies at call time so CLI startup stays fast (#886).
from __future__ import annotations
import os
from dataclasses import dataclass
from pathlib import Path, PurePosixPath, PureWindowsPath
from typing import TYPE_CHECKING
import psutil
import typer
from loguru import logger
from rich.console import Console
from rich.progress import Progress, SpinnerColumn, TextColumn, BarColumn, TaskProgressColumn
from sqlalchemy.exc import OperationalError
from basic_memory import db
from basic_memory.cli.app import app
from basic_memory.cli.commands.command_utils import run_with_cleanup
from basic_memory.config import ConfigManager, ProjectMode
if TYPE_CHECKING:
from basic_memory.indexing import IndexProgress
from basic_memory.indexing import IndexProgress
from basic_memory.repository import ProjectRepository
from basic_memory.services.initialization import reconcile_projects_with_config
from basic_memory.sync.sync_service import get_sync_service
console = Console()
@@ -162,13 +159,6 @@ async def _reindex_projects(app_config):
This ensures all database operations use the same event loop,
and proper cleanup happens when the function completes.
"""
# Deferred: SQLAlchemy, repositories, and the sync stack load only when a
# reindex actually runs, not on every CLI start (#886).
from basic_memory import db
from basic_memory.repository import ProjectRepository
from basic_memory.services.initialization import reconcile_projects_with_config
from basic_memory.sync.sync_service import get_sync_service
try:
await reconcile_projects_with_config(app_config)
@@ -177,9 +167,8 @@ async def _reindex_projects(app_config):
db_path=app_config.database_path,
db_type=db.DatabaseType.FILESYSTEM,
)
project_repository = ProjectRepository()
async with db.scoped_session(session_maker) as session:
projects = await project_repository.get_active_projects(session)
project_repository = ProjectRepository(session_maker)
projects = await project_repository.get_active_projects()
for project in projects:
console.print(f" Indexing [cyan]{project.name}[/cyan]...")
@@ -208,12 +197,6 @@ def reset(
),
): # pragma: no cover
"""Reset database (drop all tables and recreate)."""
# Deferred: SQLAlchemy and the db module load only when a reset actually
# runs, not on every CLI start (#886).
from sqlalchemy.exc import OperationalError
from basic_memory import db
console.print(
"[yellow]Note:[/yellow] This only deletes the index database. "
"Your markdown note files will not be affected.\n"
@@ -337,15 +320,10 @@ async def _reindex(
project: str | None,
):
"""Run reindex operations."""
# Deferred: SQLAlchemy, repositories, and the sync stack load only when a
# reindex actually runs, not on every CLI start (#886).
from basic_memory import db
from basic_memory.repository import EntityRepository, ProjectRepository
from basic_memory.repository import EntityRepository
from basic_memory.repository.search_repository import create_search_repository
from basic_memory.services.initialization import reconcile_projects_with_config
from basic_memory.services.search_service import SearchService
from basic_memory.services.file_service import FileService
from basic_memory.sync.sync_service import get_sync_service
from basic_memory.markdown.markdown_processor import MarkdownProcessor
from basic_memory.markdown.entity_parser import EntityParser
@@ -356,9 +334,8 @@ async def _reindex(
db_path=app_config.database_path,
db_type=db.DatabaseType.FILESYSTEM,
)
project_repository = ProjectRepository()
async with db.scoped_session(session_maker) as session:
projects = await project_repository.get_active_projects(session)
project_repository = ProjectRepository(session_maker)
projects = await project_repository.get_active_projects()
if project:
projects = [p for p in projects if p.name == project]
@@ -420,7 +397,7 @@ async def _reindex(
console.print(
f" Building vector embeddings ([cyan]{embedding_mode_label}[/cyan])..."
)
entity_repository = EntityRepository(project_id=proj.id)
entity_repository = EntityRepository(session_maker, project_id=proj.id)
search_repository = create_search_repository(
session_maker, project_id=proj.id, app_config=app_config
)
@@ -428,12 +405,7 @@ async def _reindex(
entity_parser = EntityParser(project_path)
markdown_processor = MarkdownProcessor(entity_parser, app_config=app_config)
file_service = FileService(project_path, markdown_processor, app_config=app_config)
search_service = SearchService(
search_repository,
entity_repository,
file_service,
session_maker=session_maker,
)
search_service = SearchService(search_repository, entity_repository, file_service)
with Progress(
SpinnerColumn(),
+5 -68
View File
@@ -7,12 +7,16 @@ import uuid
from pathlib import Path
from loguru import logger
from mcp.server.fastmcp.exceptions import ToolError
from rich.console import Console
import typer
from basic_memory.cli.app import app
from basic_memory.cli.commands.command_utils import run_with_cleanup
from basic_memory.cli.commands.routing import force_routing, validate_routing_flags
from basic_memory.markdown.entity_parser import EntityParser
from basic_memory.markdown.markdown_processor import MarkdownProcessor
from basic_memory.markdown.schemas import EntityFrontmatter, EntityMarkdown
from basic_memory.mcp.async_client import get_client
from basic_memory.mcp.clients import KnowledgeClient, ProjectClient, SearchClient
from basic_memory.schemas.base import Entity
@@ -23,72 +27,8 @@ from basic_memory.schemas import SyncReportResponse
console = Console()
def _is_default_project_delete_error(error: Exception) -> bool:
"""Return True only for the API guard that blocks deleting the default project."""
error_text = str(error)
return "Cannot delete default project" in error_text
async def _delete_doctor_project_locally(project_name: str, project_id: str) -> None:
"""Remove the generated doctor project when the public API guard blocks cleanup."""
from basic_memory import db
from basic_memory.config import ConfigManager
from basic_memory.repository import ProjectRepository
config_manager = ConfigManager()
repository = ProjectRepository()
_, session_maker = await db.get_or_create_db(
db_path=config_manager.config.database_path,
db_type=db.DatabaseType.FILESYSTEM,
)
async with db.scoped_session(session_maker) as session:
project = await repository.get_by_external_id(session, project_id)
if project is None:
raise ValueError(f"Doctor cleanup project '{project_id}' not found")
if project.name != project_name:
raise ValueError(
f"Doctor cleanup expected project '{project_name}', found '{project.name}'"
)
await repository.delete(session, project.id)
config = config_manager.load_config()
if project_name in config.projects:
del config.projects[project_name]
if config.default_project == project_name:
config.default_project = next(iter(config.projects), None)
config_manager.save_config(config)
async def _delete_doctor_project(
project_client: ProjectClient, project_name: str, project_id: str
) -> None:
"""Delete the generated doctor project without weakening the public API guard."""
# Deferred: ToolError lives in the mcp SDK, which must not load at CLI startup (#886).
from mcp.server.fastmcp.exceptions import ToolError
try:
await project_client.delete_project(project_id)
except ToolError as exc:
if not _is_default_project_delete_error(exc):
raise
# Trigger: fresh local configs can promote the generated doctor project
# to default because the placeholder default has no DB row.
# Why: the project is disposable doctor-owned state, while the public API
# must keep rejecting default-project deletion for normal callers.
# Outcome: cleanup removes only the exact doctor project it created.
await _delete_doctor_project_locally(project_name, project_id)
async def run_doctor() -> None:
"""Run local consistency checks for file <-> database flows."""
# Deferred: the markdown parsing stack is only needed while the checks run,
# and importing it at module level slows every CLI invocation (#886).
from basic_memory.markdown.entity_parser import EntityParser
from basic_memory.markdown.markdown_processor import MarkdownProcessor
from basic_memory.markdown.schemas import EntityFrontmatter, EntityMarkdown
console.print("[blue]Running Basic Memory doctor checks...[/blue]")
project_name = f"doctor-{uuid.uuid4().hex[:8]}"
@@ -187,7 +127,7 @@ async def run_doctor() -> None:
finally:
if project_id:
await _delete_doctor_project(project_client, project_name, project_id)
await project_client.delete_project(project_id)
console.print("[green]Doctor checks passed.[/green]")
@@ -200,9 +140,6 @@ def doctor(
cloud: bool = typer.Option(False, "--cloud", help="Force cloud API routing"),
) -> None:
"""Run local consistency checks to verify file/database sync."""
# Deferred: ToolError lives in the mcp SDK, which must not load at CLI startup (#886).
from mcp.server.fastmcp.exceptions import ToolError
try:
validate_routing_flags(local, cloud)
# Doctor runs local filesystem checks — always default to local routing
@@ -1,34 +1,25 @@
"""Import command for ChatGPT conversations."""
# PEP 563 lazy annotations keep heavy importer types out of module import (#886).
from __future__ import annotations
import json
from pathlib import Path
from typing import TYPE_CHECKING, Annotated, Tuple
from typing import Annotated, Tuple
import typer
from basic_memory.cli.app import import_app
from basic_memory.cli.commands.command_utils import run_with_cleanup
from basic_memory.config import ConfigManager, get_project_config
from basic_memory.importers import ChatGPTImporter
from basic_memory.markdown import EntityParser, MarkdownProcessor
from basic_memory.services.file_service import FileService
from loguru import logger
from rich.console import Console
from rich.panel import Panel
if TYPE_CHECKING:
from basic_memory.markdown import MarkdownProcessor
from basic_memory.services.file_service import FileService
console = Console()
async def get_importer_dependencies() -> Tuple[MarkdownProcessor, FileService]:
"""Get MarkdownProcessor and FileService instances for importers."""
# Deferred: the markdown/file-service stack pulls SQLAlchemy and must load
# only when an import actually runs, not on every CLI start (#886).
from basic_memory.markdown import EntityParser, MarkdownProcessor
from basic_memory.services.file_service import FileService
config = get_project_config()
app_config = ConfigManager().config
entity_parser = EntityParser(config.home)
@@ -69,9 +60,6 @@ def import_chatgpt(
console.print(f"\nImporting chats from {conversations_json}...writing to {base_path}")
# Create importer and run import
# Deferred: importer stack loads at import-command run time only (#886).
from basic_memory.importers import ChatGPTImporter
importer = ChatGPTImporter(
config.home, markdown_processor, file_service, project_name=config.name
)
@@ -1,34 +1,25 @@
"""Import command for basic-memory CLI to import chat data from conversations2.json format."""
# PEP 563 lazy annotations keep heavy importer types out of module import (#886).
from __future__ import annotations
import json
from pathlib import Path
from typing import TYPE_CHECKING, Annotated, Tuple
from typing import Annotated, Tuple
import typer
from basic_memory.cli.app import claude_app
from basic_memory.cli.commands.command_utils import run_with_cleanup
from basic_memory.config import ConfigManager, get_project_config
from basic_memory.importers.claude_conversations_importer import ClaudeConversationsImporter
from basic_memory.markdown import EntityParser, MarkdownProcessor
from basic_memory.services.file_service import FileService
from loguru import logger
from rich.console import Console
from rich.panel import Panel
if TYPE_CHECKING:
from basic_memory.markdown import MarkdownProcessor
from basic_memory.services.file_service import FileService
console = Console()
async def get_importer_dependencies() -> Tuple[MarkdownProcessor, FileService]:
"""Get MarkdownProcessor and FileService instances for importers."""
# Deferred: the markdown/file-service stack pulls SQLAlchemy and must load
# only when an import actually runs, not on every CLI start (#886).
from basic_memory.markdown import EntityParser, MarkdownProcessor
from basic_memory.services.file_service import FileService
config = get_project_config()
app_config = ConfigManager().config
entity_parser = EntityParser(config.home)
@@ -66,9 +57,6 @@ def import_claude(
markdown_processor, file_service = run_with_cleanup(get_importer_dependencies())
# Create the importer
# Deferred: importer stack loads at import-command run time only (#886).
from basic_memory.importers.claude_conversations_importer import ClaudeConversationsImporter
importer = ClaudeConversationsImporter(
config.home, markdown_processor, file_service, project_name=config.name
)
@@ -1,34 +1,25 @@
"""Import command for basic-memory CLI to import project data from Claude.ai."""
# PEP 563 lazy annotations keep heavy importer types out of module import (#886).
from __future__ import annotations
import json
from pathlib import Path
from typing import TYPE_CHECKING, Annotated, Tuple
from typing import Annotated, Tuple
import typer
from basic_memory.cli.app import claude_app
from basic_memory.cli.commands.command_utils import run_with_cleanup
from basic_memory.config import ConfigManager, get_project_config
from basic_memory.importers.claude_projects_importer import ClaudeProjectsImporter
from basic_memory.markdown import EntityParser, MarkdownProcessor
from basic_memory.services.file_service import FileService
from loguru import logger
from rich.console import Console
from rich.panel import Panel
if TYPE_CHECKING:
from basic_memory.markdown import MarkdownProcessor
from basic_memory.services.file_service import FileService
console = Console()
async def get_importer_dependencies() -> Tuple[MarkdownProcessor, FileService]:
"""Get MarkdownProcessor and FileService instances for importers."""
# Deferred: the markdown/file-service stack pulls SQLAlchemy and must load
# only when an import actually runs, not on every CLI start (#886).
from basic_memory.markdown import EntityParser, MarkdownProcessor
from basic_memory.services.file_service import FileService
config = get_project_config()
app_config = ConfigManager().config
entity_parser = EntityParser(config.home)
@@ -65,9 +56,6 @@ def import_projects(
markdown_processor, file_service = run_with_cleanup(get_importer_dependencies())
# Create the importer
# Deferred: importer stack loads at import-command run time only (#886).
from basic_memory.importers.claude_projects_importer import ClaudeProjectsImporter
importer = ClaudeProjectsImporter(
config.home, markdown_processor, file_service, project_name=config.name
)
@@ -1,34 +1,25 @@
"""Import command for basic-memory CLI to import from JSON memory format."""
# PEP 563 lazy annotations keep heavy importer types out of module import (#886).
from __future__ import annotations
import json
from pathlib import Path
from typing import TYPE_CHECKING, Annotated, Tuple
from typing import Annotated, Tuple
import typer
from basic_memory.cli.app import import_app
from basic_memory.cli.commands.command_utils import run_with_cleanup
from basic_memory.config import ConfigManager, get_project_config
from basic_memory.importers.memory_json_importer import MemoryJsonImporter
from basic_memory.markdown import EntityParser, MarkdownProcessor
from basic_memory.services.file_service import FileService
from loguru import logger
from rich.console import Console
from rich.panel import Panel
if TYPE_CHECKING:
from basic_memory.markdown import MarkdownProcessor
from basic_memory.services.file_service import FileService
console = Console()
async def get_importer_dependencies() -> Tuple[MarkdownProcessor, FileService]:
"""Get MarkdownProcessor and FileService instances for importers."""
# Deferred: the markdown/file-service stack pulls SQLAlchemy and must load
# only when an import actually runs, not on every CLI start (#886).
from basic_memory.markdown import EntityParser, MarkdownProcessor
from basic_memory.services.file_service import FileService
config = get_project_config()
app_config = ConfigManager().config
entity_parser = EntityParser(config.home)
@@ -64,9 +55,6 @@ def memory_json(
markdown_processor, file_service = run_with_cleanup(get_importer_dependencies())
# Create the importer
# Deferred: importer stack loads at import-command run time only (#886).
from basic_memory.importers.memory_json_importer import MemoryJsonImporter
importer = MemoryJsonImporter(
config.home, markdown_processor, file_service, project_name=config.name
)
-76
View File
@@ -1,76 +0,0 @@
"""Install the bundled man pages so `man bm` works."""
import shutil
import subprocess
from pathlib import Path
from typing import Annotated, Optional
import typer
from rich.console import Console
from basic_memory.cli.app import app
console = Console()
man_app = typer.Typer(help="Manage the bm man pages.")
app.add_typer(man_app, name="man")
# Bundled groff sources ship inside the package (src/basic_memory/man).
_MAN_SOURCE_DIR = Path(__file__).parent.parent.parent / "man"
def _default_man_root() -> Path:
# Why ~/.local/share/man: manpath(1) derives man directories from PATH
# entries on both man-db (Linux) and BSD man (macOS), so ~/.local/bin on
# PATH — the pipx/uv tool layout — makes this root searchable without any
# MANPATH configuration.
return Path.home() / ".local" / "share" / "man"
def _man_root_on_manpath(man_root: Path) -> Optional[bool]:
"""Best-effort check whether man(1) will search man_root; None if unknown."""
try:
result = subprocess.run(["manpath"], capture_output=True, text=True, timeout=5)
except (FileNotFoundError, subprocess.TimeoutExpired):
return None
if result.returncode != 0:
return None
paths = [entry.rstrip("/") for entry in result.stdout.strip().split(":") if entry]
return str(man_root).rstrip("/") in paths
@man_app.command()
def install(
directory: Annotated[
Optional[Path],
typer.Option(
"--dir",
help="Man root to install into (default: ~/.local/share/man)",
),
] = None,
) -> None:
"""Install the bm man pages, then try `man bm`."""
man_root = (directory or _default_man_root()).expanduser()
man1 = man_root / "man1"
man1.mkdir(parents=True, exist_ok=True)
pages = sorted(_MAN_SOURCE_DIR.glob("*.1"))
if not pages: # pragma: no cover - broken packaging, not a runtime state
console.print("[red]No bundled man pages found — broken installation[/red]")
raise typer.Exit(1)
for page in pages:
shutil.copyfile(page, man1 / page.name)
console.print(f"installed {man1 / page.name}")
# Trigger: the chosen root is provably absent from manpath output.
# Why: a silent install into an unsearched directory looks like success
# but `man bm` still fails; say so and hand over the one-line fix.
# Outcome: actionable hint; unknown (None) stays quiet to avoid false alarms.
if _man_root_on_manpath(man_root) is False:
console.print(
f"\n[yellow]{man_root} is not on your manpath.[/yellow] Add it with:\n"
f' export MANPATH="{man_root}:$MANPATH"'
)
console.print("\nTry: [bold]man bm[/bold]")
+1 -3
View File
@@ -5,6 +5,7 @@ from typing import Annotated, Optional
import typer
from loguru import logger
from mcp.server.fastmcp.exceptions import ToolError
from rich.console import Console
from rich.table import Table
@@ -49,9 +50,6 @@ def orphans(
"""
from basic_memory.cli.commands.command_utils import run_with_cleanup
# Deferred: ToolError lives in the mcp SDK, which must not load at CLI startup (#886).
from mcp.server.fastmcp.exceptions import ToolError
try:
validate_routing_flags(local, cloud)
with force_routing(local=local, cloud=cloud):
+20 -23
View File
@@ -1050,13 +1050,12 @@ async def _detach_local_project_row(app_config: BasicMemoryConfig, name: str) ->
db_type=db.DatabaseType.FILESYSTEM,
)
try:
repo = ProjectRepository()
async with db.scoped_session(session_maker) as session:
existing = await repo.get_by_name(session, name)
if existing is None:
return False
await repo.delete(session, existing.id)
return True
repo = ProjectRepository(session_maker)
existing = await repo.get_by_name(name)
if existing is None:
return False
await repo.delete(existing.id)
return True
finally:
# CLI-only: safe to tear down the global DB singleton here since
# set-cloud/set-local never run inside a long-lived MCP/API server.
@@ -1083,22 +1082,20 @@ async def _attach_local_project_row(app_config: BasicMemoryConfig, name: str, pa
db_type=db.DatabaseType.FILESYSTEM,
)
try:
repo = ProjectRepository()
async with db.scoped_session(session_maker) as session:
existing = await repo.get_by_name(session, name)
if existing is None:
await repo.create(
session,
{
"name": name,
"path": path,
"permalink": generate_permalink(name),
"is_active": True,
},
)
return
if existing.path != path:
await repo.update_path(session, existing.id, path)
repo = ProjectRepository(session_maker)
existing = await repo.get_by_name(name)
if existing is None:
await repo.create(
{
"name": name,
"path": path,
"permalink": generate_permalink(name),
"is_active": True,
}
)
return
if existing.path != path:
await repo.update_path(existing.id, path)
finally:
# CLI-only: safe to tear down the global DB singleton here since
# set-cloud/set-local never run inside a long-lived MCP/API server.
+3 -13
View File
@@ -20,10 +20,9 @@ from basic_memory.cli.app import app
from basic_memory.cli.commands.command_utils import run_with_cleanup
from basic_memory.cli.commands.routing import force_routing, validate_routing_flags
from basic_memory.config import ConfigManager
# MCP tool functions are imported inside each command: importing
# basic_memory.mcp.tools loads the entire tool stack (fastmcp, mcp SDK,
# SQLAlchemy), which would slow every CLI invocation, including --help (#886).
from basic_memory.mcp.tools import schema_diff as mcp_schema_diff
from basic_memory.mcp.tools import schema_infer as mcp_schema_infer
from basic_memory.mcp.tools import schema_validate as mcp_schema_validate
console = Console()
@@ -190,9 +189,6 @@ def validate(
Use --local to force local routing when cloud mode is enabled.
Use --cloud to force cloud routing when cloud mode is disabled.
"""
# Deferred: loading the MCP tool stack at module import slows CLI startup (#886).
from basic_memory.mcp.tools import schema_validate as mcp_schema_validate
try:
validate_routing_flags(local, cloud)
project_name = _resolve_project_name(project)
@@ -276,9 +272,6 @@ def infer(
Use --local to force local routing when cloud mode is enabled.
Use --cloud to force cloud routing when cloud mode is disabled.
"""
# Deferred: loading the MCP tool stack at module import slows CLI startup (#886).
from basic_memory.mcp.tools import schema_infer as mcp_schema_infer
try:
validate_routing_flags(local, cloud)
project_name = _resolve_project_name(project)
@@ -359,9 +352,6 @@ def diff(
Use --local to force local routing when cloud mode is enabled.
Use --cloud to force cloud routing when cloud mode is disabled.
"""
# Deferred: loading the MCP tool stack at module import slows CLI startup (#886).
from basic_memory.mcp.tools import schema_diff as mcp_schema_diff
try:
validate_routing_flags(local, cloud)
project_name = _resolve_project_name(project)
+2 -11
View File
@@ -5,6 +5,7 @@ import json
import time
from typing import Annotated, Dict, Optional, Set
from mcp.server.fastmcp.exceptions import ToolError
import typer
from loguru import logger
from rich.console import Console
@@ -189,16 +190,9 @@ async def run_status(
if sync_report.total == 0:
return project_item.name, sync_report
if time.monotonic() >= deadline:
# Why the hint: indexing is done by the sync coordinator, which
# only runs inside a live server (bm mcp / hosted API). In a
# CLI-only session nothing will ever drain the pending count,
# so this wait cannot succeed — point at the command that
# actually indexes (#959).
raise StatusTimeout(
f"Timed out after {timeout:g}s waiting for '{project_item.name}' "
f"to finish indexing ({sync_report.total} pending change(s) remaining). "
f"If no Basic Memory server is running, pending changes are never "
f"indexed — run 'bm reindex --project {project_item.name}' instead."
f"to finish indexing ({sync_report.total} pending change(s) remaining)."
)
await asyncio.sleep(poll_interval)
@@ -230,9 +224,6 @@ def status(
"""
from basic_memory.cli.commands.command_utils import run_with_cleanup
# Deferred: ToolError lives in the mcp SDK, which must not load at CLI startup (#886).
from mcp.server.fastmcp.exceptions import ToolError
# Trigger: --wait with a negative --timeout
# Why: a negative deadline times out on the very first poll, producing a confusing
# "Timed out after -5s" message instead of flagging the bad input. Raised
+12 -40
View File
@@ -14,10 +14,18 @@ from loguru import logger
from basic_memory.cli.app import app
from basic_memory.cli.commands.command_utils import run_with_cleanup
from basic_memory.cli.commands.routing import force_routing, validate_routing_flags
# MCP tool functions are imported inside each command: importing
# basic_memory.mcp.tools loads the entire tool stack (fastmcp, mcp SDK,
# SQLAlchemy), which would slow every CLI invocation, including --help (#886).
from basic_memory.mcp.tools import build_context as mcp_build_context
from basic_memory.mcp.tools import delete_note as mcp_delete_note
from basic_memory.mcp.tools import edit_note as mcp_edit_note
from basic_memory.mcp.tools import list_memory_projects as mcp_list_projects
from basic_memory.mcp.tools import list_workspaces as mcp_list_workspaces
from basic_memory.mcp.tools import read_note as mcp_read_note
from basic_memory.mcp.tools import recent_activity as mcp_recent_activity
from basic_memory.mcp.tools import schema_diff as mcp_schema_diff
from basic_memory.mcp.tools import schema_infer as mcp_schema_infer
from basic_memory.mcp.tools import schema_validate as mcp_schema_validate
from basic_memory.mcp.tools import search_notes as mcp_search
from basic_memory.mcp.tools import write_note as mcp_write_note
tool_app = typer.Typer()
app.add_typer(tool_app, name="tool", help="Access to MCP tools via CLI")
@@ -112,9 +120,6 @@ def write_note(
bm tool write-note --title "My Note" --folder "notes" --overwrite
bm tool write-note --title "My Note" --folder "notes" --local
"""
# Deferred: loading the MCP tool stack at module import slows CLI startup (#886).
from basic_memory.mcp.tools import write_note as mcp_write_note
try:
validate_routing_flags(local, cloud)
@@ -201,9 +206,6 @@ def read_note(
bm tool read-note my-note
bm tool read-note my-note --include-frontmatter
"""
# Deferred: loading the MCP tool stack at module import slows CLI startup (#886).
from basic_memory.mcp.tools import read_note as mcp_read_note
try:
validate_routing_flags(local, cloud)
@@ -270,9 +272,6 @@ def delete_note(
bm tool delete-note notes/old-draft
bm tool delete-note docs/archive --is-directory
"""
# Deferred: loading the MCP tool stack at module import slows CLI startup (#886).
from basic_memory.mcp.tools import delete_note as mcp_delete_note
try:
validate_routing_flags(local, cloud)
@@ -347,9 +346,6 @@ def edit_note(
bm tool edit-note my-note --operation find_replace --find-text "old" --content "new"
bm tool edit-note my-note --operation replace_section --section "## Notes" --content "updated"
"""
# Deferred: loading the MCP tool stack at module import slows CLI startup (#886).
from basic_memory.mcp.tools import edit_note as mcp_edit_note
try:
validate_routing_flags(local, cloud)
@@ -417,9 +413,6 @@ def build_context(
bm tool build-context memory://specs/search
bm tool build-context specs/search --depth 2 --timeframe 30d
"""
# Deferred: loading the MCP tool stack at module import slows CLI startup (#886).
from basic_memory.mcp.tools import build_context as mcp_build_context
try:
validate_routing_flags(local, cloud)
@@ -483,9 +476,6 @@ def recent_activity(
bm tool recent-activity --timeframe 30d --page-size 20
bm tool recent-activity --type entity --type observation
"""
# Deferred: loading the MCP tool stack at module import slows CLI startup (#886).
from basic_memory.mcp.tools import recent_activity as mcp_recent_activity
try:
validate_routing_flags(local, cloud)
@@ -592,9 +582,6 @@ def search_notes(
bm tool search-notes --meta status=draft
bm tool search-notes "auth" --entity-type observation --category requirement
"""
# Deferred: loading the MCP tool stack at module import slows CLI startup (#886).
from basic_memory.mcp.tools import search_notes as mcp_search
try:
validate_routing_flags(local, cloud)
@@ -700,9 +687,6 @@ def list_projects(
bm tool list-projects
bm tool list-projects --local
"""
# Deferred: loading the MCP tool stack at module import slows CLI startup (#886).
from basic_memory.mcp.tools import list_memory_projects as mcp_list_projects
try:
validate_routing_flags(local, cloud)
@@ -736,9 +720,6 @@ def list_workspaces(
bm tool list-workspaces
bm tool list-workspaces --cloud
"""
# Deferred: loading the MCP tool stack at module import slows CLI startup (#886).
from basic_memory.mcp.tools import list_workspaces as mcp_list_workspaces
try:
validate_routing_flags(local, cloud)
@@ -791,9 +772,6 @@ def schema_validate(
bm tool schema-validate people/ada-lovelace.md
bm tool schema-validate --project research
"""
# Deferred: loading the MCP tool stack at module import slows CLI startup (#886).
from basic_memory.mcp.tools import schema_validate as mcp_schema_validate
try:
validate_routing_flags(local, cloud)
@@ -862,9 +840,6 @@ def schema_infer(
bm tool schema-infer meeting --threshold 0.5
bm tool schema-infer person --project research
"""
# Deferred: loading the MCP tool stack at module import slows CLI startup (#886).
from basic_memory.mcp.tools import schema_infer as mcp_schema_infer
try:
validate_routing_flags(local, cloud)
@@ -921,9 +896,6 @@ def schema_diff(
bm tool schema-diff person
bm tool schema-diff person --project research
"""
# Deferred: loading the MCP tool stack at module import slows CLI startup (#886).
from basic_memory.mcp.tools import schema_diff as mcp_schema_diff
try:
validate_routing_flags(local, cloud)
-1
View File
@@ -24,7 +24,6 @@ if not _version_only_invocation(sys.argv[1:]):
import_claude_conversations,
import_claude_projects,
import_memory_json,
man,
mcp,
orphans,
project,
+2 -36
View File
@@ -4,7 +4,6 @@ import importlib.util
import json
import os
import shutil
import threading
from dataclasses import dataclass
from datetime import datetime
from enum import Enum
@@ -658,26 +657,6 @@ class BasicMemoryConfig(BaseSettings):
entry = self.projects.get(project_name)
return entry.mode if entry else ProjectMode.CLOUD
def is_locally_syncable(self, project_name: str, project_path: str) -> bool:
"""Whether a project should be synced/watched on the local filesystem.
Both conditions are required (issue #949):
* The project is present in config. Config is the source of truth, so a
stale database row that was removed from config but whose deletion
has not yet been reconciled, or whose reconciliation failed must
not be synced even though it still has a real directory on disk.
* Its path is absolute. An empty or relative path resolves against the
process cwd, so syncing it would adopt whatever directory the server
was launched from as the project root and mutate unrelated files.
Cloud-only projects (empty/slug path) and cloud projects with a real
local bisync copy (absolute path) are handled correctly by these two
conditions, so no separate mode check is needed.
"""
entry = self.projects.get(project_name)
return entry is not None and Path(project_path).is_absolute()
def set_project_mode(self, project_name: str, mode: ProjectMode) -> None:
"""Set the routing mode for a project.
@@ -1120,21 +1099,8 @@ def save_basic_memory_config(file_path: Path, config: BasicMemoryConfig) -> None
_secure_config_dir(file_path.parent)
# Use model_dump with mode='json' to serialize datetime objects properly
config_dict = config.model_dump(mode="json")
# Trigger: long-lived readers (MCP stdio server config reload, background
# auto-update threads) re-read config.json whenever its mtime changes,
# concurrently with CLI commands saving it.
# Why: writing the destination in place truncates it first, so a concurrent
# reader can observe empty/partial JSON and load_config() exits the process.
# Outcome: write a sibling temp file (unique per process/thread so parallel
# savers cannot interleave) and publish atomically via os.replace — readers
# always see either the old or the new complete document. (#940)
tmp_path = file_path.parent / f"{file_path.name}.{os.getpid()}.{threading.get_ident()}.tmp"
try:
tmp_path.write_text(json.dumps(config_dict, indent=2))
_secure_config_file(tmp_path)
os.replace(tmp_path, file_path)
finally:
tmp_path.unlink(missing_ok=True)
file_path.write_text(json.dumps(config_dict, indent=2))
_secure_config_file(file_path)
except Exception as e: # pragma: no cover
logger.error(f"Failed to save config: {e}")
+15 -26
View File
@@ -19,7 +19,7 @@ from sqlalchemy.ext.asyncio import (
AsyncEngine,
async_scoped_session,
)
from sqlalchemy.pool import AsyncAdaptedQueuePool, NullPool
from sqlalchemy.pool import NullPool
from basic_memory.repository.postgres_search_repository import PostgresSearchRepository
from basic_memory.repository.sqlite_search_repository import SQLiteSearchRepository
@@ -206,40 +206,29 @@ def _create_sqlite_engine(db_url: str, db_type: DatabaseType) -> AsyncEngine:
Configured async engine for SQLite
"""
# Configure connection args with Windows-specific settings
connect_args: dict[str, bool | float] = {"check_same_thread": False}
connect_args: dict[str, bool | float | None] = {"check_same_thread": False}
# Add Windows-specific parameters to improve reliability
if os.name == "nt": # Windows
connect_args.update(
{
"timeout": 30.0, # Increase timeout to 30 seconds for Windows
"isolation_level": None, # Use autocommit mode
}
)
if db_type == DatabaseType.MEMORY:
# Trigger: an in-memory SQLite URL would default to StaticPool, which hands the
# same DBAPI connection to every concurrently checked-out session.
# Why: concurrent asyncio tasks then share one transaction scope — a rollback
# issued by one session (scoped_session exception handling or the pool's
# reset-on-return) silently destroys another session's uncommitted writes (#940).
# Outcome: a single-connection blocking queue pool keeps the in-memory database
# alive for the engine's lifetime while serializing sessions at transaction
# granularity, restoring the isolation the repositories assume.
engine = create_async_engine(
db_url,
connect_args=connect_args,
poolclass=AsyncAdaptedQueuePool,
pool_size=1,
max_overflow=0,
)
elif os.name == "nt":
# Use NullPool for Windows filesystem databases to avoid connection pooling issues
engine = create_async_engine(
db_url,
connect_args=connect_args,
poolclass=NullPool, # Disable connection pooling on Windows
echo=False,
)
# Important: Do NOT use NullPool for in-memory databases as it will destroy the database
# between connections
if db_type == DatabaseType.FILESYSTEM:
engine = create_async_engine(
db_url,
connect_args=connect_args,
poolclass=NullPool, # Disable connection pooling on Windows
echo=False,
)
else:
# In-memory databases need connection pooling to maintain state
engine = create_async_engine(db_url, connect_args=connect_args)
else:
engine = create_async_engine(db_url, connect_args=connect_args)
-4
View File
@@ -25,8 +25,6 @@ from basic_memory.deps.db import (
EngineFactoryDep,
get_session_maker,
SessionMakerDep,
get_session,
SessionDep,
)
from basic_memory.deps.projects import (
@@ -171,8 +169,6 @@ __all__ = [
"EngineFactoryDep",
"get_session_maker",
"SessionMakerDep",
"get_session",
"SessionDep",
# Projects
"get_project_repository",
"ProjectRepositoryDep",
-10
View File
@@ -5,7 +5,6 @@ This module provides database-related dependencies:
- Session dependencies for request handling
"""
from collections.abc import AsyncGenerator
from typing import Annotated
from fastapi import Depends, Request
@@ -55,12 +54,3 @@ async def get_session_maker(engine_factory: EngineFactoryDep) -> async_sessionma
SessionMakerDep = Annotated[async_sessionmaker, Depends(get_session_maker)]
async def get_session(session_maker: SessionMakerDep) -> AsyncGenerator[AsyncSession, None]:
"""Yield a request-scoped SQLAlchemy session."""
async with db.scoped_session(session_maker) as session:
yield session
SessionDep = Annotated[AsyncSession, Depends(get_session)]
+35 -51
View File
@@ -12,7 +12,6 @@ from typing import Annotated
from fastapi import Depends, HTTPException, Path, status
from basic_memory import db
from basic_memory.config import ProjectConfig
from basic_memory.deps.db import SessionMakerDep
from basic_memory.repository.project_repository import ProjectRepository
@@ -22,9 +21,11 @@ from basic_memory.utils import generate_permalink
# --- Project Repository ---
async def get_project_repository() -> ProjectRepository:
async def get_project_repository(
session_maker: SessionMakerDep,
) -> ProjectRepository:
"""Get the project repository."""
return ProjectRepository()
return ProjectRepository(session_maker)
ProjectRepositoryDep = Annotated[ProjectRepository, Depends(get_project_repository)]
@@ -40,7 +41,6 @@ ProjectPathDep = Annotated[str, Path()]
async def get_project_id(
session_maker: SessionMakerDep,
project_repository: ProjectRepositoryDep,
project: ProjectPathDep,
) -> int:
@@ -61,17 +61,14 @@ async def get_project_id(
"""
# Convert project name to permalink for lookup
project_permalink = generate_permalink(str(project))
async with db.scoped_session(session_maker) as session:
project_obj = await project_repository.get_by_permalink(session, project_permalink)
if project_obj:
return project_obj.id
project_obj = await project_repository.get_by_permalink(project_permalink)
if project_obj:
return project_obj.id
# Try by name if permalink lookup fails
project_obj = await project_repository.get_by_name(
session, str(project)
) # pragma: no cover
if project_obj: # pragma: no cover
return project_obj.id
# Try by name if permalink lookup fails
project_obj = await project_repository.get_by_name(str(project)) # pragma: no cover
if project_obj: # pragma: no cover
return project_obj.id
# Not found
raise HTTPException( # pragma: no cover
@@ -86,9 +83,7 @@ ProjectIdDep = Annotated[int, Depends(get_project_id)]
async def get_project_config(
session_maker: SessionMakerDep,
project: ProjectPathDep,
project_repository: ProjectRepositoryDep,
project: ProjectPathDep, project_repository: ProjectRepositoryDep
) -> ProjectConfig: # pragma: no cover
"""Get the current project referenced from request state.
@@ -104,10 +99,9 @@ async def get_project_config(
"""
# Convert project name to permalink for lookup
project_permalink = generate_permalink(str(project))
async with db.scoped_session(session_maker) as session:
project_obj = await project_repository.get_by_permalink(session, project_permalink)
if project_obj:
return ProjectConfig(name=project_obj.name, home=pathlib.Path(project_obj.path))
project_obj = await project_repository.get_by_permalink(project_permalink)
if project_obj:
return ProjectConfig(name=project_obj.name, home=pathlib.Path(project_obj.path))
# Not found
raise HTTPException( # pragma: no cover
@@ -122,7 +116,6 @@ ProjectConfigDep = Annotated[ProjectConfig, Depends(get_project_config)]
async def validate_project_id(
session_maker: SessionMakerDep,
project_id: int,
project_repository: ProjectRepositoryDep,
) -> int:
@@ -141,13 +134,12 @@ async def validate_project_id(
Raises:
HTTPException: If project with that ID is not found
"""
async with db.scoped_session(session_maker) as session:
project_obj = await project_repository.get_by_id(session, project_id)
if not project_obj:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=f"Project with ID {project_id} not found.",
)
project_obj = await project_repository.get_by_id(project_id)
if not project_obj:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=f"Project with ID {project_id} not found.",
)
return project_id
@@ -155,9 +147,7 @@ ProjectIdPathDep = Annotated[int, Depends(validate_project_id)]
async def get_project_config_v2(
session_maker: SessionMakerDep,
project_id: ProjectIdPathDep,
project_repository: ProjectRepositoryDep,
project_id: ProjectIdPathDep, project_repository: ProjectRepositoryDep
) -> ProjectConfig: # pragma: no cover
"""Get the project config for v2 API (uses integer project_id from path).
@@ -171,10 +161,9 @@ async def get_project_config_v2(
Raises:
HTTPException: If project is not found
"""
async with db.scoped_session(session_maker) as session:
project_obj = await project_repository.get_by_id(session, project_id)
if project_obj:
return ProjectConfig(name=project_obj.name, home=pathlib.Path(project_obj.path))
project_obj = await project_repository.get_by_id(project_id)
if project_obj:
return ProjectConfig(name=project_obj.name, home=pathlib.Path(project_obj.path))
# Not found (this should not happen since ProjectIdPathDep already validates existence)
raise HTTPException( # pragma: no cover
@@ -189,7 +178,6 @@ ProjectConfigV2Dep = Annotated[ProjectConfig, Depends(get_project_config_v2)]
async def validate_project_external_id(
session_maker: SessionMakerDep,
project_id: str,
project_repository: ProjectRepositoryDep,
) -> int:
@@ -208,13 +196,12 @@ async def validate_project_external_id(
Raises:
HTTPException: If project with that external_id is not found
"""
async with db.scoped_session(session_maker) as session:
project_obj = await project_repository.get_by_external_id(session, project_id)
if not project_obj:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=f"Project with external_id '{project_id}' not found.",
)
project_obj = await project_repository.get_by_external_id(project_id)
if not project_obj:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=f"Project with external_id '{project_id}' not found.",
)
return project_obj.id
@@ -222,9 +209,7 @@ ProjectExternalIdPathDep = Annotated[int, Depends(validate_project_external_id)]
async def get_project_config_v2_external(
session_maker: SessionMakerDep,
project_id: ProjectExternalIdPathDep,
project_repository: ProjectRepositoryDep,
project_id: ProjectExternalIdPathDep, project_repository: ProjectRepositoryDep
) -> ProjectConfig: # pragma: no cover
"""Get the project config for v2 API (uses external_id UUID from path).
@@ -238,10 +223,9 @@ async def get_project_config_v2_external(
Raises:
HTTPException: If project is not found
"""
async with db.scoped_session(session_maker) as session:
project_obj = await project_repository.get_by_id(session, project_id)
if project_obj:
return ProjectConfig(name=project_obj.name, home=pathlib.Path(project_obj.path))
project_obj = await project_repository.get_by_id(project_id)
if project_obj:
return ProjectConfig(name=project_obj.name, home=pathlib.Path(project_obj.path))
# Not found (this should not happen since ProjectExternalIdPathDep already validates)
raise HTTPException( # pragma: no cover
+18 -9
View File
@@ -30,30 +30,33 @@ from basic_memory.repository.search_repository import SearchRepository, create_s
async def get_entity_repository(
session_maker: SessionMakerDep,
project_id: ProjectIdDep,
) -> EntityRepository:
"""Create an EntityRepository instance for the current project."""
return EntityRepository(project_id=project_id)
return EntityRepository(session_maker, project_id=project_id)
EntityRepositoryDep = Annotated[EntityRepository, Depends(get_entity_repository)]
async def get_entity_repository_v2( # pragma: no cover
session_maker: SessionMakerDep,
project_id: ProjectIdPathDep,
) -> EntityRepository:
"""Create an EntityRepository instance for v2 API (uses integer project_id from path)."""
return EntityRepository(project_id=project_id)
return EntityRepository(session_maker, project_id=project_id)
EntityRepositoryV2Dep = Annotated[EntityRepository, Depends(get_entity_repository_v2)]
async def get_entity_repository_v2_external(
session_maker: SessionMakerDep,
project_id: ProjectExternalIdPathDep,
) -> EntityRepository:
"""Create an EntityRepository instance for v2 API (uses external_id from path)."""
return EntityRepository(project_id=project_id)
return EntityRepository(session_maker, project_id=project_id)
EntityRepositoryV2ExternalDep = Annotated[
@@ -65,20 +68,22 @@ EntityRepositoryV2ExternalDep = Annotated[
async def get_observation_repository(
session_maker: SessionMakerDep,
project_id: ProjectIdDep,
) -> ObservationRepository:
"""Create an ObservationRepository instance for the current project."""
return ObservationRepository(project_id=project_id)
return ObservationRepository(session_maker, project_id=project_id)
ObservationRepositoryDep = Annotated[ObservationRepository, Depends(get_observation_repository)]
async def get_observation_repository_v2( # pragma: no cover
session_maker: SessionMakerDep,
project_id: ProjectIdPathDep,
) -> ObservationRepository:
"""Create an ObservationRepository instance for v2 API."""
return ObservationRepository(project_id=project_id)
return ObservationRepository(session_maker, project_id=project_id)
ObservationRepositoryV2Dep = Annotated[
@@ -87,10 +92,11 @@ ObservationRepositoryV2Dep = Annotated[
async def get_observation_repository_v2_external(
session_maker: SessionMakerDep,
project_id: ProjectExternalIdPathDep,
) -> ObservationRepository:
"""Create an ObservationRepository instance for v2 API (uses external_id)."""
return ObservationRepository(project_id=project_id)
return ObservationRepository(session_maker, project_id=project_id)
ObservationRepositoryV2ExternalDep = Annotated[
@@ -102,30 +108,33 @@ ObservationRepositoryV2ExternalDep = Annotated[
async def get_relation_repository(
session_maker: SessionMakerDep,
project_id: ProjectIdDep,
) -> RelationRepository:
"""Create a RelationRepository instance for the current project."""
return RelationRepository(project_id=project_id)
return RelationRepository(session_maker, project_id=project_id)
RelationRepositoryDep = Annotated[RelationRepository, Depends(get_relation_repository)]
async def get_relation_repository_v2( # pragma: no cover
session_maker: SessionMakerDep,
project_id: ProjectIdPathDep,
) -> RelationRepository:
"""Create a RelationRepository instance for v2 API."""
return RelationRepository(project_id=project_id)
return RelationRepository(session_maker, project_id=project_id)
RelationRepositoryV2Dep = Annotated[RelationRepository, Depends(get_relation_repository_v2)]
async def get_relation_repository_v2_external(
session_maker: SessionMakerDep,
project_id: ProjectExternalIdPathDep,
) -> RelationRepository:
"""Create a RelationRepository instance for v2 API (uses external_id)."""
return RelationRepository(project_id=project_id)
return RelationRepository(session_maker, project_id=project_id)
RelationRepositoryV2ExternalDep = Annotated[
+10 -59
View File
@@ -16,7 +16,6 @@ from fastapi import Depends
from loguru import logger
from basic_memory.deps.config import AppConfigDep
from basic_memory.deps.db import SessionMakerDep
from basic_memory.deps.projects import (
ProjectConfigDep,
ProjectConfigV2Dep,
@@ -160,10 +159,9 @@ async def get_search_service(
search_repository: SearchRepositoryDep,
entity_repository: EntityRepositoryDep,
file_service: FileServiceDep,
session_maker: SessionMakerDep,
) -> SearchService:
"""Create SearchService with dependencies."""
return SearchService(search_repository, entity_repository, file_service, session_maker)
return SearchService(search_repository, entity_repository, file_service)
SearchServiceDep = Annotated[SearchService, Depends(get_search_service)]
@@ -173,10 +171,9 @@ async def get_search_service_v2( # pragma: no cover
search_repository: SearchRepositoryV2Dep,
entity_repository: EntityRepositoryV2Dep,
file_service: FileServiceV2Dep,
session_maker: SessionMakerDep,
) -> SearchService:
"""Create SearchService for v2 API."""
return SearchService(search_repository, entity_repository, file_service, session_maker)
return SearchService(search_repository, entity_repository, file_service)
SearchServiceV2Dep = Annotated[SearchService, Depends(get_search_service_v2)]
@@ -186,10 +183,9 @@ async def get_search_service_v2_external(
search_repository: SearchRepositoryV2ExternalDep,
entity_repository: EntityRepositoryV2ExternalDep,
file_service: FileServiceV2ExternalDep,
session_maker: SessionMakerDep,
) -> SearchService:
"""Create SearchService for v2 API (uses external_id)."""
return SearchService(search_repository, entity_repository, file_service, session_maker)
return SearchService(search_repository, entity_repository, file_service)
SearchServiceV2ExternalDep = Annotated[SearchService, Depends(get_search_service_v2_external)]
@@ -199,45 +195,27 @@ SearchServiceV2ExternalDep = Annotated[SearchService, Depends(get_search_service
async def get_link_resolver(
entity_repository: EntityRepositoryDep,
search_service: SearchServiceDep,
session_maker: SessionMakerDep,
entity_repository: EntityRepositoryDep, search_service: SearchServiceDep
) -> LinkResolver:
return LinkResolver(
entity_repository=entity_repository,
search_service=search_service,
session_maker=session_maker,
)
return LinkResolver(entity_repository=entity_repository, search_service=search_service)
LinkResolverDep = Annotated[LinkResolver, Depends(get_link_resolver)]
async def get_link_resolver_v2( # pragma: no cover
entity_repository: EntityRepositoryV2Dep,
search_service: SearchServiceV2Dep,
session_maker: SessionMakerDep,
entity_repository: EntityRepositoryV2Dep, search_service: SearchServiceV2Dep
) -> LinkResolver:
return LinkResolver(
entity_repository=entity_repository,
search_service=search_service,
session_maker=session_maker,
)
return LinkResolver(entity_repository=entity_repository, search_service=search_service)
LinkResolverV2Dep = Annotated[LinkResolver, Depends(get_link_resolver_v2)]
async def get_link_resolver_v2_external(
entity_repository: EntityRepositoryV2ExternalDep,
search_service: SearchServiceV2ExternalDep,
session_maker: SessionMakerDep,
entity_repository: EntityRepositoryV2ExternalDep, search_service: SearchServiceV2ExternalDep
) -> LinkResolver:
return LinkResolver(
entity_repository=entity_repository,
search_service=search_service,
session_maker=session_maker,
)
return LinkResolver(entity_repository=entity_repository, search_service=search_service)
LinkResolverV2ExternalDep = Annotated[LinkResolver, Depends(get_link_resolver_v2_external)]
@@ -254,7 +232,6 @@ async def get_entity_service(
file_service: FileServiceDep,
link_resolver: LinkResolverDep,
search_service: SearchServiceDep,
session_maker: SessionMakerDep,
app_config: AppConfigDep,
) -> EntityService:
"""Create EntityService with repository."""
@@ -265,7 +242,6 @@ async def get_entity_service(
entity_parser=entity_parser,
file_service=file_service,
link_resolver=link_resolver,
session_maker=session_maker,
search_service=search_service,
app_config=app_config,
)
@@ -282,7 +258,6 @@ async def get_entity_service_v2( # pragma: no cover
file_service: FileServiceV2Dep,
link_resolver: LinkResolverV2Dep,
search_service: SearchServiceV2Dep,
session_maker: SessionMakerDep,
app_config: AppConfigDep,
) -> EntityService:
"""Create EntityService for v2 API."""
@@ -293,7 +268,6 @@ async def get_entity_service_v2( # pragma: no cover
entity_parser=entity_parser,
file_service=file_service,
link_resolver=link_resolver,
session_maker=session_maker,
search_service=search_service,
app_config=app_config,
)
@@ -310,7 +284,6 @@ async def get_entity_service_v2_external(
file_service: FileServiceV2ExternalDep,
link_resolver: LinkResolverV2ExternalDep,
search_service: SearchServiceV2ExternalDep,
session_maker: SessionMakerDep,
app_config: AppConfigDep,
) -> EntityService:
"""Create EntityService for v2 API (uses external_id)."""
@@ -321,7 +294,6 @@ async def get_entity_service_v2_external(
entity_parser=entity_parser,
file_service=file_service,
link_resolver=link_resolver,
session_maker=session_maker,
search_service=search_service,
app_config=app_config,
)
@@ -338,14 +310,12 @@ async def get_context_service(
entity_repository: EntityRepositoryDep,
observation_repository: ObservationRepositoryDep,
link_resolver: LinkResolverDep,
session_maker: SessionMakerDep,
) -> ContextService:
return ContextService(
search_repository=search_repository,
entity_repository=entity_repository,
observation_repository=observation_repository,
link_resolver=link_resolver,
session_maker=session_maker,
)
@@ -357,7 +327,6 @@ async def get_context_service_v2( # pragma: no cover
entity_repository: EntityRepositoryV2Dep,
observation_repository: ObservationRepositoryV2Dep,
link_resolver: LinkResolverV2Dep,
session_maker: SessionMakerDep,
) -> ContextService:
"""Create ContextService for v2 API."""
return ContextService(
@@ -365,7 +334,6 @@ async def get_context_service_v2( # pragma: no cover
entity_repository=entity_repository,
observation_repository=observation_repository,
link_resolver=link_resolver,
session_maker=session_maker,
)
@@ -377,7 +345,6 @@ async def get_context_service_v2_external(
entity_repository: EntityRepositoryV2ExternalDep,
observation_repository: ObservationRepositoryV2ExternalDep,
link_resolver: LinkResolverV2ExternalDep,
session_maker: SessionMakerDep,
) -> ContextService:
"""Create ContextService for v2 API (uses external_id)."""
return ContextService(
@@ -385,7 +352,6 @@ async def get_context_service_v2_external(
entity_repository=entity_repository,
observation_repository=observation_repository,
link_resolver=link_resolver,
session_maker=session_maker,
)
@@ -404,7 +370,6 @@ async def get_sync_service(
project_repository: ProjectRepositoryDep,
search_service: SearchServiceDep,
file_service: FileServiceDep,
session_maker: SessionMakerDep,
) -> SyncService: # pragma: no cover
return SyncService(
app_config=app_config,
@@ -415,7 +380,6 @@ async def get_sync_service(
project_repository=project_repository,
search_service=search_service,
file_service=file_service,
session_maker=session_maker,
)
@@ -431,7 +395,6 @@ async def get_sync_service_v2(
project_repository: ProjectRepositoryDep,
search_service: SearchServiceV2Dep,
file_service: FileServiceV2Dep,
session_maker: SessionMakerDep,
) -> SyncService: # pragma: no cover
"""Create SyncService for v2 API."""
return SyncService(
@@ -443,7 +406,6 @@ async def get_sync_service_v2(
project_repository=project_repository,
search_service=search_service,
file_service=file_service,
session_maker=session_maker,
)
@@ -459,7 +421,6 @@ async def get_sync_service_v2_external(
project_repository: ProjectRepositoryDep,
search_service: SearchServiceV2ExternalDep,
file_service: FileServiceV2ExternalDep,
session_maker: SessionMakerDep,
) -> SyncService: # pragma: no cover
"""Create SyncService for v2 API (uses external_id)."""
return SyncService(
@@ -471,7 +432,6 @@ async def get_sync_service_v2_external(
project_repository=project_repository,
search_service=search_service,
file_service=file_service,
session_maker=session_maker,
)
@@ -575,7 +535,6 @@ TaskSchedulerDep = Annotated[TaskScheduler, Depends(get_task_scheduler)]
async def get_project_service(
project_repository: ProjectRepositoryDep,
session_maker: SessionMakerDep,
app_config: AppConfigDep,
) -> ProjectService:
"""Create ProjectService with repository and a system-level FileService for directory operations."""
@@ -584,9 +543,7 @@ async def get_project_service(
entity_parser = EntityParser(Path.home())
markdown_processor = MarkdownProcessor(entity_parser, app_config=app_config)
file_service = FileService(Path.home(), markdown_processor, app_config=app_config)
return ProjectService(
repository=project_repository, session_maker=session_maker, file_service=file_service
)
return ProjectService(repository=project_repository, file_service=file_service)
ProjectServiceDep = Annotated[ProjectService, Depends(get_project_service)]
@@ -597,12 +554,10 @@ ProjectServiceDep = Annotated[ProjectService, Depends(get_project_service)]
async def get_directory_service(
entity_repository: EntityRepositoryDep,
session_maker: SessionMakerDep,
) -> DirectoryService:
"""Create DirectoryService with dependencies."""
return DirectoryService(
entity_repository=entity_repository,
session_maker=session_maker,
)
@@ -611,12 +566,10 @@ DirectoryServiceDep = Annotated[DirectoryService, Depends(get_directory_service)
async def get_directory_service_v2( # pragma: no cover
entity_repository: EntityRepositoryV2Dep,
session_maker: SessionMakerDep,
) -> DirectoryService:
"""Create DirectoryService for v2 API (uses integer project_id from path)."""
return DirectoryService(
entity_repository=entity_repository,
session_maker=session_maker,
)
@@ -625,12 +578,10 @@ DirectoryServiceV2Dep = Annotated[DirectoryService, Depends(get_directory_servic
async def get_directory_service_v2_external(
entity_repository: EntityRepositoryV2ExternalDep,
session_maker: SessionMakerDep,
) -> DirectoryService:
"""Create DirectoryService for v2 API (uses external_id from path)."""
return DirectoryService(
entity_repository=entity_repository,
session_maker=session_maker,
)
+22 -68
View File
@@ -302,76 +302,26 @@ async def format_file(
return None
# A frontmatter fence is a line containing exactly `---`, optionally followed by
# trailing horizontal whitespace. Anchoring to a full line (rather than a bare
# substring/`startswith`) prevents single-line content like
# `---\nstatus: active\n---\nBody` — where `\n` is a literal backslash-n, not a
# newline — from being misread as frontmatter. See issue #972.
_FENCE_RE = re.compile(r"^---[ \t]*$")
def _split_frontmatter(content: str) -> Optional[tuple[str, str]]:
"""Split content into (yaml_block, body) when it opens with a line-anchored fence.
The opening fence must be the very first line and the closing fence must be a
later line, each matching exactly `---` (with optional trailing whitespace).
Returns:
A `(yaml_block, body)` tuple when a complete fenced block is present, or
``None`` when the content does not open with a frontmatter fence.
Raises:
ParseError: If the content opens with a fence but has no closing fence.
"""
lines = content.splitlines(keepends=True)
# Skip leading blank lines: a document may begin with whitespace before the
# opening fence (e.g. a heredoc/dedented string starting with a newline). This
# does NOT relax line-anchoring — the opening fence must still be the first
# non-blank line, all on its own, so a single-line `---\\nstatus...` (literal
# backslash-n) is still rejected. See issue #972.
start = 0
while start < len(lines) and lines[start].strip() == "":
start += 1
if start >= len(lines) or not _FENCE_RE.match(lines[start].rstrip("\r\n")):
return None
# Find the closing fence on its own line somewhere after the opening fence.
for index in range(start + 1, len(lines)):
if _FENCE_RE.match(lines[index].rstrip("\r\n")):
yaml_block = "".join(lines[start + 1 : index])
body = "".join(lines[index + 1 :])
return yaml_block, body
raise ParseError("Invalid frontmatter format")
def has_frontmatter(content: str) -> bool:
"""
Check if content contains valid YAML frontmatter.
Frontmatter requires `---` fences on their own lines; an inline `---` (such as
a single-line string that merely starts with the characters `---`) is not
frontmatter.
Args:
content: Content to check
Returns:
True if content has line-anchored frontmatter fences, False otherwise
True if content has valid frontmatter markers (---), False otherwise
"""
if not content:
return False
# Strip BOM before checking for frontmatter markers
content = strip_bom(content)
try:
return _split_frontmatter(content) is not None
except ParseError:
# An opening fence with no closing fence is not usable frontmatter.
content = strip_bom(content).strip()
if not content.startswith("---"):
return False
return "---" in content[3:]
def parse_frontmatter(content: str) -> Dict[str, Any]:
"""
@@ -389,14 +339,17 @@ def parse_frontmatter(content: str) -> Dict[str, Any]:
try:
# Strip BOM before parsing frontmatter
content = strip_bom(content)
split = _split_frontmatter(content)
if split is None:
if not content.strip().startswith("---"):
raise ParseError("Content has no frontmatter")
yaml_block, _ = split
# Split on first two occurrences of ---
parts = content.split("---", 2)
if len(parts) < 3:
raise ParseError("Invalid frontmatter format")
# Parse YAML
try:
frontmatter = yaml.safe_load(yaml_block)
frontmatter = yaml.safe_load(parts[1])
# Handle empty frontmatter (None from yaml.safe_load)
if frontmatter is None:
return {}
@@ -428,17 +381,18 @@ def remove_frontmatter(content: str) -> str:
ParseError: If content starts with frontmatter marker but is malformed
"""
# Strip BOM before processing
content = strip_bom(content)
content = strip_bom(content).strip()
split = _split_frontmatter(content)
# Trigger: content does not open with a line-anchored fence
# Why: inline `---` is ordinary content, not frontmatter (issue #972)
# Outcome: return the content untouched (stripped to preserve prior behavior)
if split is None:
return content.strip()
# Return as-is if no frontmatter marker
if not content.startswith("---"):
return content
_, body = split
return body.strip()
# Split on first two occurrences of ---
parts = content.split("---", 2)
if len(parts) < 3:
raise ParseError("Invalid frontmatter format")
return parts[2].strip()
def dump_frontmatter(post: frontmatter.Post) -> str:
+70 -109
View File
@@ -3,18 +3,15 @@
from __future__ import annotations
import asyncio
from contextlib import asynccontextmanager
from dataclasses import dataclass
from datetime import datetime
from pathlib import Path
from typing import AsyncIterator, Awaitable, Callable, Mapping, TypeVar
from typing import Awaitable, Callable, Mapping, TypeVar
from loguru import logger
from sqlalchemy.exc import IntegrityError
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
import logfire
from basic_memory import db
from basic_memory.config import BasicMemoryConfig
from basic_memory.file_utils import compute_checksum, has_frontmatter, remove_frontmatter
from basic_memory.markdown.schemas import EntityMarkdown
@@ -72,7 +69,6 @@ class BatchIndexer:
relation_repository: RelationRepository,
search_service: SearchService,
file_writer: IndexFileWriter,
session_maker: async_sessionmaker[AsyncSession],
) -> None:
self.app_config = app_config
self.entity_service = entity_service
@@ -80,19 +76,6 @@ class BatchIndexer:
self.relation_repository = relation_repository
self.search_service = search_service
self.file_writer = file_writer
self.session_maker = session_maker
@asynccontextmanager
async def _session_scope(
self, session: AsyncSession | None = None
) -> AsyncIterator[AsyncSession]:
"""Use the caller's session or open a service-owned transaction."""
if session is not None:
yield session
return
async with db.scoped_session(self.session_maker) as owned_session:
yield owned_session
async def index_files(
self,
@@ -166,10 +149,9 @@ class BatchIndexer:
max_concurrent=max_concurrent,
)
async with self._session_scope() as session:
refreshed_entities = await self.entity_repository.find_by_ids(
session, [prepared.entity_id for prepared in prepared_entities.values()]
)
refreshed_entities = await self.entity_repository.find_by_ids(
[prepared.entity_id for prepared in prepared_entities.values()]
)
entities_by_id = {entity.id: entity for entity in refreshed_entities}
refreshed, refresh_errors = await self._run_bounded(
@@ -214,7 +196,12 @@ class BatchIndexer:
prepared = await self._prepare_markdown_file(file)
if existing_permalink_by_path is None:
with logfire.span("index.markdown_file.load_permalink_map", path=file.path):
existing_permalink_by_path = await self._get_file_path_to_permalink_map()
existing_permalink_by_path = {
path: permalink
for path, permalink in (
await self.entity_repository.get_file_path_to_permalink_map()
).items()
}
reserved_permalinks = {
permalink
@@ -239,8 +226,7 @@ class BatchIndexer:
path=file.path,
entity_id=persisted.entity.id,
):
async with self._session_scope() as session:
refreshed = await self.entity_repository.find_by_ids(session, [persisted.entity.id])
refreshed = await self.entity_repository.find_by_ids([persisted.entity.id])
if len(refreshed) != 1: # pragma: no cover
raise ValueError(f"Failed to reload indexed entity for {file.path}")
entity = refreshed[0]
@@ -263,17 +249,6 @@ class BatchIndexer:
markdown_content=prepared_entity.markdown_content,
)
async def _get_file_path_to_permalink_map(self) -> dict[str, str | None]:
"""Load current file-path to permalink mappings in a service-owned session."""
async with self._session_scope() as session:
permalink_by_path: dict[str, str | None] = {
path: permalink
for path, permalink in (
await self.entity_repository.get_file_path_to_permalink_map(session)
).items()
}
return permalink_by_path
# --- Preparation ---
async def _prepare_markdown_file(self, file: IndexInputFile) -> _PreparedMarkdownFile:
@@ -308,7 +283,12 @@ class BatchIndexer:
return {}, {}
if existing_permalink_by_path is None:
existing_permalink_by_path = await self._get_file_path_to_permalink_map()
existing_permalink_by_path = {
path: permalink
for path, permalink in (
await self.entity_repository.get_file_path_to_permalink_map()
).items()
}
batch_paths = set(prepared_markdown)
reserved_permalinks = {
@@ -426,10 +406,7 @@ class BatchIndexer:
async def _upsert_regular_file(self, file: IndexInputFile) -> _PreparedEntity:
checksum = await self._resolve_checksum(file)
async with self._session_scope() as session:
existing = await self.entity_repository.get_by_file_path(
session, file.path, load_relations=False
)
existing = await self.entity_repository.get_by_file_path(file.path, load_relations=False)
is_new_entity = existing is None
if existing is None:
@@ -447,8 +424,7 @@ class BatchIndexer:
)
try:
async with self._session_scope() as session:
created = await self.entity_repository.add(session, entity)
created = await self.entity_repository.add(entity)
entity_id = created.id
except IntegrityError as exc:
message = str(exc)
@@ -460,12 +436,10 @@ class BatchIndexer:
and "file_path" in message
)
):
async with self._session_scope() as session:
existing = await self.entity_repository.get_by_file_path(
session,
file.path,
load_relations=False,
)
existing = await self.entity_repository.get_by_file_path(
file.path,
load_relations=False,
)
if existing is None:
raise ValueError(
f"Entity not found after file_path conflict: {file.path}"
@@ -476,12 +450,10 @@ class BatchIndexer:
else:
entity_id = existing.id
async with self._session_scope() as session:
updated = await self.entity_repository.update(
session,
entity_id,
self._entity_metadata_updates(file, checksum, include_created_at=is_new_entity),
)
updated = await self.entity_repository.update(
entity_id,
self._entity_metadata_updates(file, checksum, include_created_at=is_new_entity),
)
if updated is None:
raise ValueError(f"Failed to update file entity metadata for {file.path}")
@@ -504,7 +476,10 @@ class BatchIndexer:
max_concurrent: int,
) -> tuple[int, int]:
unresolved_relation_lists = await asyncio.gather(
*(self._find_unresolved_relations_for_entity(entity_id) for entity_id in entity_ids)
*(
self.relation_repository.find_unresolved_relations_for_entity(entity_id)
for entity_id in entity_ids
)
)
unresolved_relations = [
relation for relation_list in unresolved_relation_lists for relation in relation_list
@@ -524,26 +499,22 @@ class BatchIndexer:
# link text, mismatching this with the sync_service forward-reference
# path and producing confidently-wrong graph edges. See
# sync_service.resolve_forward_references for the same change.
async with self._session_scope() as session:
resolved_entity = await self.entity_service.link_resolver.resolve_link(
relation.to_name, strict=True, session=session
)
resolved_entity = await self.entity_service.link_resolver.resolve_link(
relation.to_name, strict=True
)
if resolved_entity is None or resolved_entity.id == relation.from_id:
return 0
try:
async with self._session_scope() as session:
await self.relation_repository.update(
session,
relation.id,
{
"to_id": resolved_entity.id,
"to_name": resolved_entity.title,
},
)
await self.relation_repository.update(
relation.id,
{
"to_id": resolved_entity.id,
"to_name": resolved_entity.title,
},
)
except IntegrityError:
async with self._session_scope() as session:
await self.relation_repository.delete(session, relation.id)
await self.relation_repository.delete(relation.id)
return 1
except Exception as exc: # pragma: no cover - defensive logging
logger.warning(
@@ -560,19 +531,15 @@ class BatchIndexer:
)
remaining_relation_lists = await asyncio.gather(
*(self._find_unresolved_relations_for_entity(entity_id) for entity_id in entity_ids)
*(
self.relation_repository.find_unresolved_relations_for_entity(entity_id)
for entity_id in entity_ids
)
)
remaining_unresolved = sum(len(relations) for relations in remaining_relation_lists)
return sum(resolved_counts), remaining_unresolved
async def _find_unresolved_relations_for_entity(self, entity_id: int):
"""Load unresolved relations for one entity in a service-owned session."""
async with self._session_scope() as session:
return await self.relation_repository.find_unresolved_relations_for_entity(
session, entity_id
)
# --- Search refresh ---
async def _refresh_search_index(
@@ -598,36 +565,30 @@ class BatchIndexer:
resolve_relations: bool = True,
reload_entity: bool = True,
) -> _PersistedMarkdownFile:
async with self._session_scope() as session:
existing = await self.entity_repository.get_by_file_path(
session,
prepared.file.path,
load_relations=False,
)
if is_new is None:
is_new = existing is None
entity = await self.entity_service.upsert_entity_from_markdown(
Path(prepared.file.path),
prepared.markdown,
is_new=is_new,
existing_entity=existing,
resolve_relations=resolve_relations,
reload_entity=reload_entity,
session=session,
)
prepared = await self._reconcile_persisted_permalink(prepared, entity)
metadata_updates = self._entity_metadata_updates(prepared.file, prepared.final_checksum)
updated = await self.entity_repository.update_fields(
session,
entity.id,
metadata_updates,
)
if not updated:
raise ValueError(
f"Failed to update markdown entity metadata for {prepared.file.path}"
)
self._apply_entity_metadata_updates(entity, metadata_updates)
return _PersistedMarkdownFile(prepared=prepared, entity=entity)
existing = await self.entity_repository.get_by_file_path(
prepared.file.path,
load_relations=False,
)
if is_new is None:
is_new = existing is None
entity = await self.entity_service.upsert_entity_from_markdown(
Path(prepared.file.path),
prepared.markdown,
is_new=is_new,
existing_entity=existing,
resolve_relations=resolve_relations,
reload_entity=reload_entity,
)
prepared = await self._reconcile_persisted_permalink(prepared, entity)
metadata_updates = self._entity_metadata_updates(prepared.file, prepared.final_checksum)
updated = await self.entity_repository.update_fields(
entity.id,
metadata_updates,
)
if not updated:
raise ValueError(f"Failed to update markdown entity metadata for {prepared.file.path}")
self._apply_entity_metadata_updates(entity, metadata_updates)
return _PersistedMarkdownFile(prepared=prepared, entity=entity)
async def _reconcile_persisted_permalink(
self,
-1
View File
@@ -1 +0,0 @@
.so man1/bm.1
-134
View File
@@ -1,134 +0,0 @@
.TH BM 1 "2026-06-11" "basic-memory" "Basic Memory Manual"
.SH NAME
bm \- local-first knowledge base for humans and AI agents
.SH SYNOPSIS
.B bm
.I COMMAND
.RI [ ARGS ]...
.br
.B basic-memory
.I COMMAND
.RI [ ARGS ]...
.SH DESCRIPTION
.B bm
manages Basic Memory projects: plain markdown files that form a knowledge
graph. Files are the source of truth; SQLite provides indexing and
full-text search; the same operations are exposed to AI agents over the
Model Context Protocol (MCP) and to humans and scripts through this CLI.
.PP
Notes use semantic markdown: observations
.RB ( "\- [category] text #tag" )
and relations
.RB ( "\- relation_type [[Target]]" )
become queryable graph structure. Projects route independently to the
local API or to Basic Memory Cloud.
.SH COMMANDS
Knowledge operations:
.TP
.B bm tool
CLI access to the MCP tools (write-note, read-note, search-notes,
build-context, ...). These emit JSON and are the scriptable surface.
.TP
.B bm status
Show sync status between files and the database.
.TP
.B bm reindex
Index local file changes and rebuild search/embeddings. This is the manual
sync trigger when no MCP server is running.
.TP
.B bm doctor
Run end-to-end file/database consistency checks.
.TP
.B bm orphans
List entities with no relations in the knowledge graph.
.TP
.B bm format
Run configured formatters over note files.
.PP
Projects and schemas:
.TP
.B bm project
Add, remove, list projects; set the default; flip a project between local
and cloud routing.
.TP
.B bm schema
List, validate, infer, and drift-check Picoschema note-type contracts.
.PP
Data and cloud:
.TP
.B bm import
Import from ChatGPT, Claude, or memory.json exports. Imports write files;
run
.B bm reindex
afterwards.
.TP
.B bm cloud
Authenticate, sync (push/pull/sync/bisync), snapshots, and team workspace
administration.
.PP
Infrastructure:
.TP
.B bm mcp
Run the MCP server (hosts the live file watcher).
.TP
.B bm man
Manage these man pages
.RB ( "bm man install" ).
.TP
.B bm reset
Drop and recreate the database (destructive).
.PP
Most commands accept
.B \-\-project
.I NAME
to target a project and
.BR \-\-local / \-\-cloud
to override routing.
.SH EXAMPLES
Write and find a note from the shell:
.PP
.nf
.RS
echo "# Standup notes" | bm tool write-note \\
\-\-title "Standup" \-\-folder notes
bm tool search-notes "standup"
.RE
.fi
.PP
Pick up files created outside the tools:
.PP
.nf
.RS
bm status # shows pending changes
bm reindex # indexes them
.RE
.fi
.SH FILES
.TP
.I ~/.basic-memory/config.json
Projects, default project, per-project routing modes, cloud settings.
.TP
.I ~/.basic-memory/memory.db
SQLite index (derived; safe to rebuild with bm reindex).
.SH ENVIRONMENT
.TP
.B BASIC_MEMORY_FORCE_LOCAL
Force local routing regardless of cloud mode.
.TP
.B BASIC_MEMORY_LOG_LEVEL
Logging verbosity (e.g. DEBUG).
.SH SEE ALSO
Full manual (machine-readable, agent-traversable): the
.I manual
Basic Memory project \(em see docs/manual-pages.md in the repository.
Documentation: https://docs.basicmemory.com
.PP
For AI agents: the complete tool reference lives in the manual project as
section\-3 pages (write-note(3), search-notes(3), ...), queryable via
.B search_notes
with
.BR "metadata_filters={\(dqtype\(dq: \(dqmanpage\(dq}" .
.SH BUGS
https://github.com/basicmachines-co/basic-memory/issues
.SH AUTHORS
Basic Machines (https://basicmachines.co)
+13 -38
View File
@@ -5,16 +5,14 @@ from dataclasses import dataclass
from threading import RLock
from typing import TYPE_CHECKING, Annotated, Any, AsyncIterator, Callable, Optional
from fastapi import Depends, FastAPI, Request
from httpx import ASGITransport, AsyncClient, Timeout
from loguru import logger
import logfire
from basic_memory.config import ConfigManager, ProjectMode, has_cloud_credentials
from basic_memory.config import ConfigManager, ProjectMode
if TYPE_CHECKING:
# FastAPI is only needed when a request routes through the local ASGI
# transport; importing it at module level costs ~0.1s on every CLI start (#886).
from fastapi import FastAPI
from sqlalchemy.ext.asyncio import AsyncEngine, AsyncSession, async_sessionmaker
LocalDatabaseState = tuple["AsyncEngine", "async_sessionmaker[AsyncSession]"]
@@ -30,8 +28,8 @@ class _PreparedLocalAsgiDatabase:
_prepared_local_asgi_database_lock = RLock()
_prepared_local_asgi_database_prepare_locks: dict["FastAPI", Lock] = {}
_prepared_local_asgi_databases: dict["FastAPI", _PreparedLocalAsgiDatabase] = {}
_prepared_local_asgi_database_prepare_locks: dict[FastAPI, Lock] = {}
_prepared_local_asgi_databases: dict[FastAPI, _PreparedLocalAsgiDatabase] = {}
def _force_local_mode() -> bool:
@@ -59,7 +57,7 @@ def _build_timeout() -> Timeout:
)
def _build_asgi_client(app: "FastAPI", timeout: Timeout) -> AsyncClient:
def _build_asgi_client(app: FastAPI, timeout: Timeout) -> AsyncClient:
"""Create a local ASGI client for an already-prepared FastAPI app."""
from basic_memory.workspace_context import workspace_permalink_headers
@@ -73,7 +71,7 @@ def _build_asgi_client(app: "FastAPI", timeout: Timeout) -> AsyncClient:
)
def _get_prepared_local_asgi_database_prepare_lock(app: "FastAPI") -> Lock:
def _get_prepared_local_asgi_database_prepare_lock(app: FastAPI) -> Lock:
"""Get the async lock that serializes first-time DB preparation for an app."""
with _prepared_local_asgi_database_lock:
prepare_lock = _prepared_local_asgi_database_prepare_locks.get(app)
@@ -84,10 +82,8 @@ def _get_prepared_local_asgi_database_prepare_lock(app: "FastAPI") -> Lock:
@asynccontextmanager
async def _resolve_local_asgi_database(app: "FastAPI") -> AsyncIterator[LocalDatabaseState]:
async def _resolve_local_asgi_database(app: FastAPI) -> AsyncIterator[LocalDatabaseState]:
"""Resolve database state for a local ASGI request."""
# Imported on first local-ASGI use so CLI startup never pays for FastAPI (#886).
from fastapi import Depends, Request
from fastapi.dependencies.utils import get_dependant, solve_dependencies
from basic_memory.deps import get_engine_factory
@@ -131,7 +127,7 @@ async def _resolve_local_asgi_database(app: "FastAPI") -> AsyncIterator[LocalDat
yield await resolve_database_state(**solved.values)
def _retain_prepared_local_asgi_database(app: "FastAPI") -> bool:
def _retain_prepared_local_asgi_database(app: FastAPI) -> bool:
"""Retain an active local ASGI database preparation if one exists."""
with _prepared_local_asgi_database_lock:
active = _prepared_local_asgi_databases.get(app)
@@ -143,7 +139,7 @@ def _retain_prepared_local_asgi_database(app: "FastAPI") -> bool:
def _install_prepared_local_asgi_database(
app: "FastAPI",
app: FastAPI,
database_state: LocalDatabaseState,
dependency_context: AbstractAsyncContextManager[LocalDatabaseState],
) -> None:
@@ -167,7 +163,7 @@ def _install_prepared_local_asgi_database(
)
def _restore_local_asgi_state_attribute(app: "FastAPI", name: str, previous_value: object) -> None:
def _restore_local_asgi_state_attribute(app: FastAPI, name: str, previous_value: object) -> None:
"""Restore a FastAPI app.state attribute captured before local ASGI preparation."""
if previous_value is _MISSING_STATE_VALUE:
if hasattr(app.state, name):
@@ -177,7 +173,7 @@ def _restore_local_asgi_state_attribute(app: "FastAPI", name: str, previous_valu
def _release_prepared_local_asgi_database(
app: "FastAPI",
app: FastAPI,
) -> AbstractAsyncContextManager[LocalDatabaseState] | None:
"""Release local ASGI database state after a client context exits."""
with _prepared_local_asgi_database_lock:
@@ -200,7 +196,7 @@ def _release_prepared_local_asgi_database(
@asynccontextmanager
async def _prepared_local_asgi_database(app: "FastAPI") -> AsyncIterator[None]:
async def _prepared_local_asgi_database(app: FastAPI) -> AsyncIterator[None]:
"""Initialize local ASGI database state before the first request."""
prepare_lock = _get_prepared_local_asgi_database_prepare_lock(app)
async with prepare_lock:
@@ -372,8 +368,7 @@ async def get_client(
1. Factory injection.
2. Explicit routing flags (--local/--cloud).
3. Per-project mode routing when project_name is provided.
4. Cloud routing when a workspace selector is provided.
5. Local ASGI transport by default.
4. Local ASGI transport by default.
"""
if _client_factory:
async with _client_factory(workspace=workspace) as client:
@@ -433,26 +428,6 @@ async def get_client(
yield client
return
# --- Workspace-selector routing ---
# Trigger: caller passed a cloud workspace selector and nothing above routed.
# Why: a workspace names a cloud tenant — silently serving the request from
# the local ASGI app sent writes to the wrong destination (#954: a cloud
# project create either failed on the cloud-style path or landed locally).
# Outcome: route to the cloud proxy when credentials exist; without
# credentials fail fast instead of pretending the operation succeeded.
if workspace is not None:
if not has_cloud_credentials(config):
raise RuntimeError(
f"A cloud workspace was requested ('{workspace}') but no cloud "
"credentials were found. Run 'bm cloud login' or "
"'bm cloud set-key <key>' first, or omit the workspace selector "
"for a local operation."
)
logger.debug(f"Workspace selector '{workspace}' provided - using cloud proxy client")
async with _cloud_client(config, timeout, workspace=workspace) as client:
yield client
return
# --- Default fallback ---
logger.debug("Default routing - using ASGI client for local Basic Memory API")
async with _asgi_client(timeout) as client:
+1 -5
View File
@@ -7,9 +7,7 @@ from typing import Optional, Any
from httpx import AsyncClient
# call_* helpers live in basic_memory.mcp.tools.utils; importing that at module
# level executes the whole tools package (fastmcp + mcp SDK) during CLI startup,
# so each method defers the import to call time instead (#886).
from basic_memory.mcp.tools.utils import call_get
class DirectoryClient:
@@ -57,8 +55,6 @@ class DirectoryClient:
Raises:
ToolError: If the request fails
"""
from basic_memory.mcp.tools.utils import call_get
params: dict = {
"dir_name": dir_name,
"depth": depth,
+1 -26
View File
@@ -8,10 +8,7 @@ from typing import Any
from httpx import AsyncClient
import logfire
# call_* helpers live in basic_memory.mcp.tools.utils; importing that at module
# level executes the whole tools package (fastmcp + mcp SDK) during CLI startup,
# so each method defers the import to call time instead (#886).
from basic_memory.mcp.tools.utils import call_get, call_post, call_put, call_patch, call_delete
from basic_memory.schemas.response import (
EntityResponse,
DeleteEntitiesResponse,
@@ -60,8 +57,6 @@ class KnowledgeClient:
Raises:
ToolError: If the request fails
"""
from basic_memory.mcp.tools.utils import call_post
with logfire.span(
"mcp.client.knowledge.create_entity",
client_name="knowledge",
@@ -94,8 +89,6 @@ class KnowledgeClient:
Raises:
ToolError: If the request fails
"""
from basic_memory.mcp.tools.utils import call_put
with logfire.span(
"mcp.client.knowledge.update_entity",
client_name="knowledge",
@@ -123,8 +116,6 @@ class KnowledgeClient:
Raises:
ToolError: If the entity is not found or request fails
"""
from basic_memory.mcp.tools.utils import call_get
with logfire.span(
"mcp.client.knowledge.get_entity",
client_name="knowledge",
@@ -156,8 +147,6 @@ class KnowledgeClient:
Raises:
ToolError: If the request fails
"""
from basic_memory.mcp.tools.utils import call_patch
with logfire.span(
"mcp.client.knowledge.patch_entity",
client_name="knowledge",
@@ -185,8 +174,6 @@ class KnowledgeClient:
Raises:
ToolError: If the entity is not found or request fails
"""
from basic_memory.mcp.tools.utils import call_delete
with logfire.span(
"mcp.client.knowledge.delete_entity",
client_name="knowledge",
@@ -214,8 +201,6 @@ class KnowledgeClient:
Raises:
ToolError: If the request fails
"""
from basic_memory.mcp.tools.utils import call_put
with logfire.span(
"mcp.client.knowledge.move_entity",
client_name="knowledge",
@@ -246,8 +231,6 @@ class KnowledgeClient:
Raises:
ToolError: If the request fails
"""
from basic_memory.mcp.tools.utils import call_post
with logfire.span(
"mcp.client.knowledge.move_directory",
client_name="knowledge",
@@ -278,8 +261,6 @@ class KnowledgeClient:
Raises:
ToolError: If the request fails
"""
from basic_memory.mcp.tools.utils import call_post
with logfire.span(
"mcp.client.knowledge.delete_directory",
client_name="knowledge",
@@ -309,8 +290,6 @@ class KnowledgeClient:
Raises:
ToolError: If the file does not exist on disk or indexing fails
"""
from basic_memory.mcp.tools.utils import call_post
with logfire.span(
"mcp.client.knowledge.sync_file",
client_name="knowledge",
@@ -330,8 +309,6 @@ class KnowledgeClient:
async def get_orphans(self) -> list[GraphNode]:
"""Get entities that have no incoming or outgoing relations."""
from basic_memory.mcp.tools.utils import call_get
with logfire.span(
"mcp.client.knowledge.get_orphans",
client_name="knowledge",
@@ -361,8 +338,6 @@ class KnowledgeClient:
Raises:
ToolError: If the identifier cannot be resolved
"""
from basic_memory.mcp.tools.utils import call_post
with logfire.span(
"mcp.client.knowledge.resolve_entity",
client_name="knowledge",
+1 -8
View File
@@ -8,10 +8,7 @@ from typing import Optional
from httpx import AsyncClient
import logfire
# call_* helpers live in basic_memory.mcp.tools.utils; importing that at module
# level executes the whole tools package (fastmcp + mcp SDK) during CLI startup,
# so each method defers the import to call time instead (#886).
from basic_memory.mcp.tools.utils import call_get
from basic_memory.schemas.memory import GraphContext
@@ -66,8 +63,6 @@ class MemoryClient:
Raises:
ToolError: If the request fails
"""
from basic_memory.mcp.tools.utils import call_get
params: dict = {
"depth": depth,
"page": page,
@@ -118,8 +113,6 @@ class MemoryClient:
Raises:
ToolError: If the request fails
"""
from basic_memory.mcp.tools.utils import call_get
params: dict = {
"timeframe": timeframe,
"depth": depth,
+7 -21
View File
@@ -7,9 +7,13 @@ from typing import Any
from httpx import AsyncClient
# call_* helpers live in basic_memory.mcp.tools.utils; importing that at module
# level executes the whole tools package (fastmcp + mcp SDK) during CLI startup,
# so each method defers the import to call time instead (#886).
from basic_memory.mcp.tools.utils import (
call_delete,
call_get,
call_patch,
call_post,
call_put,
)
from basic_memory.schemas import ProjectInfoResponse, SyncReportResponse
from basic_memory.schemas.project_info import ProjectList, ProjectStatusResponse
from basic_memory.schemas.v2 import ProjectResolveResponse
@@ -49,8 +53,6 @@ class ProjectClient:
Raises:
ToolError: If the request fails
"""
from basic_memory.mcp.tools.utils import call_get
response = await call_get(
self.http_client,
"/v2/projects/",
@@ -69,8 +71,6 @@ class ProjectClient:
Raises:
ToolError: If the request fails
"""
from basic_memory.mcp.tools.utils import call_post
response = await call_post(
self.http_client,
"/v2/projects/",
@@ -93,8 +93,6 @@ class ProjectClient:
Raises:
ToolError: If the request fails
"""
from basic_memory.mcp.tools.utils import call_delete
url = f"/v2/projects/{project_external_id}"
if delete_notes:
url += "?delete_notes=true"
@@ -116,8 +114,6 @@ class ProjectClient:
Raises:
ToolError: If the request fails
"""
from basic_memory.mcp.tools.utils import call_post
response = await call_post(
self.http_client,
"/v2/projects/resolve",
@@ -137,8 +133,6 @@ class ProjectClient:
Raises:
ToolError: If the request fails
"""
from basic_memory.mcp.tools.utils import call_put
response = await call_put(
self.http_client,
f"/v2/projects/{project_external_id}/default",
@@ -160,8 +154,6 @@ class ProjectClient:
Raises:
ToolError: If the request fails
"""
from basic_memory.mcp.tools.utils import call_patch
response = await call_patch(
self.http_client,
f"/v2/projects/{project_external_id}",
@@ -189,8 +181,6 @@ class ProjectClient:
Raises:
ToolError: If the request fails
"""
from basic_memory.mcp.tools.utils import call_post
url = f"/v2/projects/{project_external_id}/sync"
params = []
if force_full:
@@ -214,8 +204,6 @@ class ProjectClient:
Raises:
ToolError: If the request fails
"""
from basic_memory.mcp.tools.utils import call_post
response = await call_post(
self.http_client,
f"/v2/projects/{project_external_id}/status",
@@ -234,8 +222,6 @@ class ProjectClient:
Raises:
ToolError: If the request fails
"""
from basic_memory.mcp.tools.utils import call_get
response = await call_get(
self.http_client,
f"/v2/projects/{project_external_id}/info",
+1 -5
View File
@@ -6,9 +6,7 @@ Encapsulates all /v2/projects/{project_id}/resource/* endpoints.
from httpx import AsyncClient, Response
import logfire
# call_* helpers live in basic_memory.mcp.tools.utils; importing that at module
# level executes the whole tools package (fastmcp + mcp SDK) during CLI startup,
# so each method defers the import to call time instead (#886).
from basic_memory.mcp.tools.utils import call_get
class ResourceClient:
@@ -51,8 +49,6 @@ class ResourceClient:
Raises:
ToolError: If the resource is not found or request fails
"""
from basic_memory.mcp.tools.utils import call_get
with logfire.span(
"mcp.client.resource.read",
client_name="resource",
+1 -9
View File
@@ -5,9 +5,7 @@ Encapsulates all /v2/projects/{project_id}/schema/* endpoints.
from httpx import AsyncClient
# call_* helpers live in basic_memory.mcp.tools.utils; importing that at module
# level executes the whole tools package (fastmcp + mcp SDK) during CLI startup,
# so each method defers the import to call time instead (#886).
from basic_memory.mcp.tools.utils import call_post, call_get
from basic_memory.schemas.schema import (
ValidationReport,
InferenceReport,
@@ -58,8 +56,6 @@ class SchemaClient:
Raises:
ToolError: If the request fails
"""
from basic_memory.mcp.tools.utils import call_post
params: dict[str, str] = {}
if note_type:
params["note_type"] = note_type
@@ -91,8 +87,6 @@ class SchemaClient:
Raises:
ToolError: If the request fails
"""
from basic_memory.mcp.tools.utils import call_post
response = await call_post(
self.http_client,
f"{self._base_path}/infer",
@@ -112,8 +106,6 @@ class SchemaClient:
Raises:
ToolError: If the request fails
"""
from basic_memory.mcp.tools.utils import call_get
response = await call_get(
self.http_client,
f"{self._base_path}/diff/{note_type}",
+1 -6
View File
@@ -8,10 +8,7 @@ from typing import Any
from httpx import AsyncClient
import logfire
# call_* helpers live in basic_memory.mcp.tools.utils; importing that at module
# level executes the whole tools package (fastmcp + mcp SDK) during CLI startup,
# so each method defers the import to call time instead (#886).
from basic_memory.mcp.tools.utils import call_post
from basic_memory.schemas.search import SearchResponse
@@ -60,8 +57,6 @@ class SearchClient:
Raises:
ToolError: If the request fails
"""
from basic_memory.mcp.tools.utils import call_post
with logfire.span(
"mcp.client.search.search",
client_name="search",
+23 -164
View File
@@ -8,24 +8,10 @@ The resolve_project_parameter function is a thin wrapper for backwards
compatibility with existing MCP tools.
"""
# PEP 563 lazy annotations keep `Context` usable in signatures without importing
# fastmcp at module load — the fastmcp/mcp stack costs ~0.5s of CLI startup (#886).
from __future__ import annotations
import asyncio
from contextlib import asynccontextmanager, nullcontext
from typing import (
TYPE_CHECKING,
AsyncIterator,
Awaitable,
Callable,
List,
Optional,
Sequence,
Tuple,
cast,
)
from dataclasses import dataclass, field
from typing import AsyncIterator, Awaitable, Callable, List, Optional, Sequence, Tuple, cast
from uuid import UUID
from httpx import AsyncClient
@@ -33,6 +19,8 @@ from httpx._types import (
HeaderTypes,
)
from loguru import logger
from fastmcp import Context
from mcp.server.fastmcp.exceptions import ToolError
import logfire
from basic_memory.config import BasicMemoryConfig, ConfigManager, ProjectMode, has_cloud_credentials
@@ -58,9 +46,6 @@ from basic_memory.workspace_context import (
workspace_permalink_context,
)
if TYPE_CHECKING:
from fastmcp import Context
# --- Workspace provider injection ---
# Mirrors the set_client_factory() pattern in async_client.py.
# The cloud MCP server sets a provider that queries its own database directly,
@@ -69,14 +54,6 @@ _workspace_provider: Optional[Callable[[], Awaitable[list[WorkspaceInfo]]]] = No
_WORKSPACE_PROJECT_INDEX_STATE_KEY = "workspace_project_index"
class WorkspaceProjectLookupMiss(ValueError):
"""A project was absent from the workspace index (as opposed to ambiguous).
Misses are retried once against a freshly rebuilt index, because the
session cache may simply predate an out-of-band project creation (#956).
"""
@dataclass(frozen=True)
class WorkspaceProjectEntry:
"""A cloud project resolved together with the workspace that owns it."""
@@ -483,17 +460,6 @@ def _canonical_memory_path_for_workspace(
# Outcome: lookups preserve the complete workspace/project canonical permalink.
if not normalized_remainder:
normalized_remainder = project_permalink
# Same index-form rule as _canonical_memory_path_for_active_route (#957):
# without an active workspace permalink context, stored permalinks are
# project-qualified and a workspace-prefixed pattern cannot match.
if "*" in normalized_remainder and current_workspace_permalink_context() is None:
return build_qualified_permalink_reference(
project_permalink,
normalized_remainder,
include_project=True,
)
return build_qualified_permalink_reference(
project_permalink,
normalized_remainder,
@@ -511,24 +477,6 @@ def _canonical_memory_path_for_active_route(
) -> str:
"""Return the canonical permalink path for the currently routed project/workspace."""
project_prefix = active_project.permalink
# Trigger: the path contains a glob wildcard (folder/*) and no server-side
# workspace permalink context is active.
# Why: patterns match raw against the search index, so they must mirror the
# stored permalink form. The contextvar is what qualified permalinks at
# write time — when it is absent, stored rows are project-qualified and a
# workspace prefix (from the client's cached_workspace display state)
# guarantees zero matches (#957). Direct lookups keep full qualification
# because the link resolver understands it; patterns have no fallback.
# Outcome: without the contextvar, qualify patterns with the project prefix
# only; with it, fall through to normal workspace canonicalization.
if "*" in path and current_workspace_permalink_context() is None:
if not include_project:
return path
if path == project_prefix or path.startswith(f"{project_prefix}/"):
return path
return f"{project_prefix}/{path}"
workspace_remainder = path
if include_project and (path == project_prefix or path.startswith(f"{project_prefix}/")):
# Trigger: the memory URL already names the active project root/prefix
@@ -774,15 +722,9 @@ async def _fetch_workspace_project_entries(
async def _ensure_workspace_project_index(
context: Optional[Context] = None,
*,
force_refresh: bool = False,
) -> WorkspaceProjectIndex:
"""Build or load the session-local workspace/project lookup index.
force_refresh bypasses the cached index and rebuilds from discovery
used by resolve_workspace_project_identifier when a lookup misses (#956).
"""
if context and not force_refresh:
"""Build or load the session-local workspace/project lookup index."""
if context:
cached_raw = await context.get_state(_WORKSPACE_PROJECT_INDEX_STATE_KEY)
cached_index = _workspace_project_index_from_state(cached_raw)
if cached_index is not None:
@@ -856,99 +798,13 @@ async def ensure_workspace_project_index(
return await _ensure_workspace_project_index(context=context)
def _match_workspace_identifier(
workspaces: tuple[WorkspaceInfo, ...],
workspace_identifier: str,
) -> WorkspaceInfo:
"""Resolve the first segment of a qualified route to a single workspace.
The edit_note/write_note contract advertises that the workspace segment may be a
slug, tenant_id, or display name. We honor those forms in a fixed priority order so
that adding tenant_id/name support never changes the meaning of an identifier that
already resolves today:
1. slug (casefold) existing behavior, checked first so working routes are stable.
2. tenant_id exact match against the opaque id (no casefolding, mirroring the
precedent in ``workspace_matches_exact_identifier``).
3. display name (casefold) names are not guaranteed unique, so a name that matches
multiple workspaces is rejected rather than silently picking one.
"""
# Trigger: identifier equals a workspace slug (casefold).
# Why: slug is the canonical routing key; resolving it first guarantees a workspace
# whose display name collides with another workspace's slug yields to the slug owner.
# Outcome: return the slug owner before tenant_id/name are considered.
slug_matches = [
workspace
for workspace in workspaces
if workspace.slug.casefold() == workspace_identifier.casefold()
]
if slug_matches:
return slug_matches[0]
# Trigger: identifier exactly equals a workspace tenant_id (an opaque id).
# Why: tenant_ids are unique, so an exact hit is unambiguous and needs no tie-break.
tenant_matches = [
workspace for workspace in workspaces if workspace.tenant_id == workspace_identifier
]
if tenant_matches:
return tenant_matches[0]
# Trigger: identifier matches one or more workspace display names (casefold).
# Why: names are not guaranteed unique; failing fast on collisions keeps routing
# deterministic and tells the caller exactly how to disambiguate.
name_matches = [
workspace
for workspace in workspaces
if workspace.name.casefold() == workspace_identifier.casefold()
]
if len(name_matches) > 1:
candidates = ", ".join(workspace.slug for workspace in name_matches)
raise ValueError(
f"Workspace name '{workspace_identifier}' matched multiple workspaces "
f"(slugs: {candidates}). Use the workspace slug or tenant_id to disambiguate."
)
if name_matches:
return name_matches[0]
available = ", ".join(workspace.slug for workspace in workspaces)
raise ValueError(
f"Workspace '{workspace_identifier}' was not found by slug, tenant_id, or name. "
f"Available workspace slugs: {available}"
)
async def resolve_workspace_project_identifier(
project: str,
context: Optional[Context] = None,
) -> WorkspaceProjectEntry:
"""Resolve a project by external_id (UUID), qualified name, or unqualified name."""
index = await _ensure_workspace_project_index(context=context)
try:
return await _resolve_workspace_project_from_index(index, project, context)
except WorkspaceProjectLookupMiss:
# Trigger: the lookup missed the session-cached index.
# Why: a miss is exactly the signal the cache may be stale — projects
# created out-of-band (CLI, a teammate in a shared workspace) post-date
# the index built at session start (#956).
# Outcome: rebuild the index once and retry; a second miss is authoritative
# and its error (with the refreshed project list) propagates.
logger.info(
f"Workspace project lookup missed for '{project}'; refreshing index and retrying"
)
refreshed = await _ensure_workspace_project_index(context=context, force_refresh=True)
return await _resolve_workspace_project_from_index(refreshed, project, context)
async def _resolve_workspace_project_from_index(
index: WorkspaceProjectIndex,
project: str,
context: Optional[Context] = None,
) -> WorkspaceProjectEntry:
"""Resolve a project against one concrete index snapshot.
Raises WorkspaceProjectLookupMiss for absent projects (retryable via index
refresh) and plain ValueError for ambiguity, which a refresh cannot fix.
"""
# Fast path: direct lookup by external_id when the identifier is a UUID
# Canonicalize via str(UUID(...)) so uppercase, brace-wrapped, or urn:uuid forms
# all hash to the same lowercase-hyphenated key as the stored external_ids.
@@ -960,14 +816,23 @@ async def _resolve_workspace_project_from_index(
except ValueError:
pass
workspace_identifier, project_identifier = _split_qualified_project_identifier(project)
workspace_slug, project_identifier = _split_qualified_project_identifier(project)
project_permalink = generate_permalink(project_identifier)
if workspace_identifier:
# Honor the documented "slug, name, or tenant_id" contract for the workspace
# segment; _match_workspace_identifier raises a clear error on ambiguous names
# and unknown identifiers, listing what forms were tried.
workspace = _match_workspace_identifier(index.workspaces, workspace_identifier)
if workspace_slug:
workspace_matches = [
workspace
for workspace in index.workspaces
if workspace.slug.casefold() == workspace_slug.casefold()
]
if not workspace_matches:
available = ", ".join(workspace.slug for workspace in index.workspaces)
raise ValueError(
f"Workspace '{workspace_slug}' was not found. "
f"Available workspace slugs: {available}"
)
workspace = workspace_matches[0]
matches = [
entry
for entry in index.entries_by_permalink.get(project_permalink, ())
@@ -978,7 +843,7 @@ async def _resolve_workspace_project_from_index(
failed_workspace.tenant_id == workspace.tenant_id
for failed_workspace in index.failed_workspaces
):
raise WorkspaceProjectLookupMiss(
raise ValueError(
f"Projects for workspace '{workspace.name}' ({workspace.slug}) "
"could not be loaded. Retry after workspace discovery recovers."
)
@@ -987,7 +852,7 @@ async def _resolve_workspace_project_from_index(
for entry in index.entries
if entry.workspace.tenant_id == workspace.tenant_id
)
raise WorkspaceProjectLookupMiss(
raise ValueError(
f"Project '{project_identifier}' was not found in workspace "
f"'{workspace.name}' ({workspace.slug}). Available projects: {available}"
)
@@ -1012,7 +877,7 @@ async def _resolve_workspace_project_from_index(
"retry or use a qualified project from an indexed workspace."
)
available = ", ".join(entry.qualified_name for entry in index.entries)
raise WorkspaceProjectLookupMiss(
raise ValueError(
f"Project '{project}' was not found in indexed cloud workspaces. "
f"Available projects: {available}.{failed_note}"
)
@@ -1347,9 +1212,6 @@ async def resolve_project_and_path(
# Why: allow project-scoped memory URLs without requiring a separate project parameter
# Outcome: attempt to resolve the prefix as a project and route to it
if project_prefix:
# Deferred: ToolError lives in the mcp SDK, which must not load at CLI startup (#886).
from mcp.server.fastmcp.exceptions import ToolError
if cached_project and _project_matches_identifier(cached_project, project_prefix):
resolved_project = await resolve_project_parameter(project_prefix, context=context)
if resolved_project and generate_permalink(resolved_project) != generate_permalink(
@@ -1576,9 +1438,6 @@ async def get_project_client(
is_factory_mode,
)
# Deferred: ToolError lives in the mcp SDK, which must not load at CLI startup (#886).
from mcp.server.fastmcp.exceptions import ToolError
# When project_id (UUID) is provided, prefer it as the resolution identifier.
# external_id is unambiguous across workspaces; project name can collide.
project_identifier = project_id if project_id else project
@@ -1,17 +0,0 @@
"""Shared loader for the bundled cloud-discovery markdown resources."""
from pathlib import Path
def load_discovery_resource(filename: str) -> str:
"""Read a bundled discovery markdown file with promo placeholders rendered.
The markdown carries a {{OSS_DISCOUNT_CODE}} placeholder so the promo code
has one source of truth (cli.promo); substitute before it reaches users.
"""
# Import here to avoid pulling CLI promo machinery (analytics, rich, config)
# into the MCP server import graph at module load.
from basic_memory.cli.promo import OSS_DISCOUNT_CODE
content = (Path(__file__).parent / filename).read_text(encoding="utf-8")
return content.replace("{{OSS_DISCOUNT_CODE}}", OSS_DISCOUNT_CODE)
+4
View File
@@ -36,7 +36,11 @@ from basic_memory.mcp.tools.chatgpt_tools import search, fetch
# Schema tools
from basic_memory.mcp.tools.schema import schema_validate, schema_infer, schema_diff
# Diagnostics tool
from basic_memory.mcp.tools.basic_memory_diagnostics import basic_memory_diagnostics
__all__ = [
"basic_memory_diagnostics",
"build_context",
"canvas",
"cloud_info",
@@ -0,0 +1,132 @@
"""Diagnostic tool for Basic Memory version and system information."""
import json
import platform
import sys
from urllib.parse import urlparse, urlunparse
import basic_memory
from basic_memory.config import CONFIG_FILE_NAME, ConfigManager
from basic_memory.mcp.server import mcp
# Fields in BasicMemoryConfig that contain secrets and must never be surfaced.
_SECRET_FIELDS = frozenset({"cloud_api_key"})
# Fields whose values are URLs that may embed user:password credentials.
# The userinfo component is stripped before surfacing.
_URL_FIELDS = frozenset({"database_url"})
def _redact_url(url: str) -> str:
"""Strip the userinfo (user:password) from a URL string.
Replaces any credentials with *** so the host/path remain visible for
diagnostics (e.g. ``postgresql://***@localhost/mydb``). If the value
cannot be parsed as a URL it is returned unchanged.
"""
try:
parsed = urlparse(url)
except Exception: # pragma: no cover — urlparse is very permissive
return url
if not parsed.hostname:
# Not a meaningful URL (e.g. a bare file path); leave it alone.
return url
if parsed.username or parsed.password:
# Rebuild with credentials replaced by a placeholder.
netloc = f"***@{parsed.hostname}"
if parsed.port:
netloc = f"{netloc}:{parsed.port}"
redacted = parsed._replace(netloc=netloc)
return urlunparse(redacted)
return url
def _redact_config(raw: dict) -> dict:
"""Return a copy of the raw config dict with secret fields removed.
- Keys in ``_SECRET_FIELDS`` are dropped entirely.
- Keys in ``_URL_FIELDS`` have their userinfo component stripped so that
host and database name remain visible for diagnostics.
Only top-level keys are processed. Nested keys within project entries are
not currently credential-bearing, but the two sets make the pattern easy
to extend.
"""
result: dict = {}
for k, v in raw.items():
if k in _SECRET_FIELDS:
# Drop entirely — value has no diagnostic value.
continue
if k in _URL_FIELDS and isinstance(v, str):
result[k] = _redact_url(v)
else:
result[k] = v
return result
@mcp.tool(
"basic_memory_diagnostics",
title="Basic Memory Diagnostics",
tags={"diagnostics"},
annotations={"readOnlyHint": True, "openWorldHint": False},
# The report is a markdown string; suppress FastMCP's wrap_result so the
# payload isn't duplicated into structuredContent.
output_schema=None,
)
def basic_memory_diagnostics() -> str:
"""Return version, system, and configuration diagnostics for Basic Memory.
Provides:
- Basic Memory package version
- Python version and platform details
- Config file path and its contents (secrets redacted)
Useful for troubleshooting installations and gathering information for
support requests. Read-only; never emits secrets or API keys.
"""
# --- Version information ---
bm_version = basic_memory.__version__
# --- System information ---
python_version = sys.version
platform_info = platform.platform()
machine = platform.machine()
# --- Configuration ---
manager = ConfigManager()
config_file = manager.config_dir / CONFIG_FILE_NAME
config_exists = config_file.exists()
if config_exists:
try:
raw_config = json.loads(config_file.read_text(encoding="utf-8"))
safe_config = _redact_config(raw_config)
config_dump = json.dumps(safe_config, indent=2, default=str)
except Exception as exc: # pragma: no cover
config_dump = f"<error reading config: {exc}>"
else:
config_dump = "<config file not found>"
lines = [
"# Basic Memory Diagnostics",
"",
"## Version",
f"- basic-memory: {bm_version}",
"",
"## System",
f"- Python: {python_version}",
f"- Platform: {platform_info}",
f"- Architecture: {machine}",
"",
"## Configuration",
f"- Config path: {config_file}",
f"- Config exists: {config_exists}",
"",
"```json",
config_dump,
"```",
]
return "\n".join(lines)
+1 -6
View File
@@ -54,10 +54,7 @@ def _format_entity_block(result: ContextResult) -> str:
lines.append("")
lines.append("### Relations")
for rel in relation_items:
# Unresolved forward references have no resolved entity yet; fall back
# to the literal target text instead of rendering [[None]] (#955)
target = rel.to_entity or rel.to_name
lines.append(f"- {rel.relation_type} [[{target}]]")
lines.append(f"- {rel.relation_type} [[{rel.to_entity}]]")
# --- Related entities (non-relation related results) ---
related_entities: list[EntitySummary | ObservationSummary] = [
@@ -115,7 +112,6 @@ def _format_context_markdown(graph: GraphContext, project: str) -> str:
@mcp.tool(
title="Build Context",
description="""Build context from a memory:// URI to continue conversations naturally.
Use this to follow up on previous discussions or explore related topics.
@@ -135,7 +131,6 @@ def _format_context_markdown(graph: GraphContext, project: str) -> str:
- "json" (default): Structured JSON with internal fields excluded
- "text": Compact markdown text for LLM consumption
""",
tags={"navigation", "notes"},
annotations={"readOnlyHint": True, "openWorldHint": False},
)
async def build_context(
-2
View File
@@ -17,9 +17,7 @@ from basic_memory.mcp.tools.utils import call_put, call_post, resolve_entity_id
@mcp.tool(
title="Create Canvas",
description="Create an Obsidian canvas file to visualize concepts and connections.",
tags={"canvas", "notes"},
annotations={"destructiveHint": False, "idempotentHint": True, "openWorldHint": False},
)
async def canvas(
@@ -105,9 +105,7 @@ def _format_document_for_chatgpt(
@mcp.tool(
title="Search Knowledge Base",
description="Search for content across the knowledge base",
tags={"search"},
annotations={"readOnlyHint": True, "openWorldHint": False},
)
async def search(
@@ -169,9 +167,7 @@ async def search(
@mcp.tool(
title="Fetch Document",
description="Fetch the full contents of a search result document",
tags={"search", "notes"},
annotations={"readOnlyHint": True, "openWorldHint": False},
)
async def fetch(
+4 -4
View File
@@ -1,15 +1,15 @@
"""Cloud information MCP tool."""
from basic_memory.mcp.resources.discovery import load_discovery_resource
from pathlib import Path
from basic_memory.mcp.server import mcp
@mcp.tool(
"cloud_info",
title="Cloud Info",
tags={"cloud"},
annotations={"readOnlyHint": True, "openWorldHint": False},
)
def cloud_info() -> str:
"""Return optional Basic Memory Cloud information and setup guidance."""
return load_discovery_resource("cloud_info.md")
content_path = Path(__file__).parent.parent / "resources" / "cloud_info.md"
return content_path.read_text(encoding="utf-8")
@@ -182,9 +182,7 @@ def _directory_path_for_delete(
@mcp.tool(
title="Delete Note",
description="Delete a note or directory by title, permalink, or path",
tags={"notes"},
annotations={"destructiveHint": True, "openWorldHint": False},
)
async def delete_note(
-2
View File
@@ -304,9 +304,7 @@ Error editing note '{identifier}': {error_message}
@mcp.tool(
title="Edit Note",
description="Edit an existing markdown note using various operations like append, prepend, find_replace, replace_section, insert_before_section, or insert_after_section.",
tags={"notes"},
annotations={"destructiveHint": False, "openWorldHint": False},
)
async def edit_note(
@@ -11,9 +11,7 @@ from basic_memory.mcp.server import mcp
@mcp.tool(
title="List Directory",
description="List directory contents with filtering and depth control.",
tags={"navigation", "notes"},
annotations={"readOnlyHint": True, "openWorldHint": False},
)
async def list_directory(
-2
View File
@@ -357,9 +357,7 @@ delete_note("{identifier}")
@mcp.tool(
title="Move Note",
description="Move a note or directory to a new location, updating database and maintaining links.",
tags={"notes"},
annotations={"destructiveHint": False, "openWorldHint": False},
)
async def move_note(
@@ -357,8 +357,6 @@ def _format_project_list_json(
@mcp.tool(
"list_memory_projects",
title="List Memory Projects",
tags={"projects"},
annotations={"readOnlyHint": True, "openWorldHint": False},
)
async def list_memory_projects(
@@ -477,14 +475,10 @@ async def _resolve_workspace_routing(
if workspace is None:
return None
forced_local = _explicit_routing() and _force_local_mode()
explicit_cloud_routing = _explicit_routing() and not _force_local_mode()
config = ConfigManager().config
# Resolve whenever credentials make workspace discovery possible — not only
# under explicit --cloud. A workspace selector implies cloud routing
# (get_client routes it to the cloud proxy, #954), and the transport needs
# the tenant id in X-Workspace-ID, not a slug or display name.
should_resolve_workspace = is_factory_mode() or (
has_cloud_credentials(config) and not forced_local
explicit_cloud_routing and has_cloud_credentials(config)
)
if not should_resolve_workspace:
return workspace
@@ -501,8 +495,6 @@ async def _resolve_workspace_routing(
@mcp.tool(
"create_memory_project",
title="Create Memory Project",
tags={"projects"},
annotations={"destructiveHint": False, "openWorldHint": False},
)
async def create_memory_project(
@@ -525,9 +517,8 @@ async def create_memory_project(
workspace: Optional cloud workspace selector to create the project in. Slug is
preferred for AI callers, but tenant_id and unique name are also accepted.
When omitted, the connection's default workspace is used. Discover values
via `list_workspaces`. A workspace selector implies cloud routing:
without cloud credentials the call fails fast instead of silently
creating a local project (#954).
via `list_workspaces`. In local mode the selector is passed through
without slug resolution.
output_format: "text" returns the existing human-readable result text.
"json" returns structured project creation metadata.
context: Optional FastMCP context for progress/status logging.
@@ -645,8 +636,6 @@ async def create_memory_project(
@mcp.tool(
title="Delete Project",
tags={"projects"},
annotations={"destructiveHint": True, "openWorldHint": False},
)
async def delete_project(
@@ -665,9 +654,8 @@ async def delete_project(
workspace: Optional cloud workspace selector to delete the project from.
Slug is preferred for AI callers, but tenant_id and unique name are
also accepted. When omitted, the connection's default workspace is
used. A workspace selector implies cloud routing: without cloud
credentials the call fails fast, matching create_memory_project
behavior (#954).
used. In local mode the selector is passed through without slug
resolution, matching create_memory_project behavior.
Returns:
Confirmation message about project deletion
@@ -155,9 +155,7 @@ def optimize_image(img, content_length, max_output_bytes=350000):
@mcp.tool(
title="Read Content",
description="Read a file's raw content by path or permalink",
tags={"notes"},
annotations={"readOnlyHint": True, "openWorldHint": False},
)
async def read_content(
-2
View File
@@ -74,9 +74,7 @@ def _parse_opening_frontmatter(content: str) -> tuple[str, dict | None]:
@mcp.tool(
title="Read Note",
description="Read a markdown note by title or permalink.",
tags={"notes"},
# TODO: re-enable once MCP client rendering is working
# meta={"ui/resourceUri": "ui://basic-memory/note-preview"},
annotations={"readOnlyHint": True, "openWorldHint": False},
@@ -26,7 +26,6 @@ from basic_memory.schemas.search import SearchItemType
@mcp.tool(
title="Recent Activity",
description="""Get recent activity for a project or across all projects.
Timeframe supports natural language formats like:
@@ -37,7 +36,6 @@ from basic_memory.schemas.search import SearchItemType
- "3 weeks ago"
Or standard formats like "7d"
""",
tags={"navigation", "notes"},
annotations={"readOnlyHint": True, "openWorldHint": False},
)
async def recent_activity(
+4 -4
View File
@@ -1,15 +1,15 @@
"""Release notes MCP tool."""
from basic_memory.mcp.resources.discovery import load_discovery_resource
from pathlib import Path
from basic_memory.mcp.server import mcp
@mcp.tool(
"release_notes",
title="Release Notes",
tags={"cloud"},
annotations={"readOnlyHint": True, "openWorldHint": False},
)
def release_notes() -> str:
"""Return the latest product release notes for optional user review."""
return load_discovery_resource("release_notes.md")
content_path = Path(__file__).parent.parent / "resources" / "release_notes.md"
return content_path.read_text(encoding="utf-8")
-6
View File
@@ -204,9 +204,7 @@ def _no_schema_guidance(note_type: str, tool_name: str) -> str:
@mcp.tool(
title="Validate Schema",
description="Validate notes against their Picoschema definitions.",
tags={"schema"},
annotations={"readOnlyHint": True, "openWorldHint": False},
)
async def schema_validate(
@@ -319,9 +317,7 @@ async def schema_validate(
@mcp.tool(
title="Infer Schema",
description="Analyze existing notes and suggest a Picoschema definition.",
tags={"schema"},
annotations={"readOnlyHint": True, "openWorldHint": False},
)
async def schema_infer(
@@ -442,9 +438,7 @@ async def schema_infer(
@mcp.tool(
title="Schema Diff",
description="Detect drift between a schema definition and actual note usage.",
tags={"schema"},
annotations={"readOnlyHint": True, "openWorldHint": False},
)
async def schema_diff(
+5 -29
View File
@@ -14,7 +14,7 @@ from basic_memory.config import ConfigManager, has_cloud_credentials
from basic_memory.utils import (
build_canonical_permalink,
coerce_dict,
parse_str_list,
coerce_list,
parse_tags,
strict_search_tags,
)
@@ -612,9 +612,7 @@ async def _search_all_projects(
@mcp.tool(
title="Search Notes",
description="Search across all content in the knowledge base with advanced syntax support.",
tags={"search"},
# TODO: re-enable once MCP client rendering is working
# meta={"ui/resourceUri": "ui://basic-memory/search-results"},
annotations={"readOnlyHint": True, "openWorldHint": False},
@@ -651,30 +649,24 @@ async def search_notes(
# Plural-vs-singular trips models constantly. Accept the singular too.
note_types: Annotated[
List[str] | None,
# parse_str_list, not coerce_list: "note,task" must split into ["note", "task"]
# consistent with how tags are handled (#910/#930). coerce_list wraps the whole
# comma string as the single literal type ["note,task"], which matches nothing.
BeforeValidator(parse_str_list),
BeforeValidator(coerce_list),
Field(default=None, validation_alias=AliasChoices("note_types", "note_type", "types")),
"Filter by the 'type' field in note frontmatter (e.g. 'note', 'chapter', 'person'). "
"Accepts a list, a comma-separated string (e.g. 'note,task'), or a JSON-array string. "
"Case-insensitive.",
] = None,
entity_types: Annotated[
List[str] | None,
BeforeValidator(parse_str_list),
BeforeValidator(coerce_list),
Field(default=None, validation_alias=AliasChoices("entity_types", "entity_type")),
"Filter by knowledge graph item type: 'entity' (whole notes), 'observation', or "
"'relation'. Defaults to 'entity'. Do NOT pass schema/frontmatter types like "
"'Chapter' here — use note_types instead. "
"Accepts a list, a comma-separated string (e.g. 'entity,observation'), or a JSON-array string.",
"'Chapter' here — use note_types instead.",
] = None,
categories: Annotated[
List[str] | None,
BeforeValidator(parse_str_list),
BeforeValidator(coerce_list),
Field(default=None, validation_alias=AliasChoices("categories", "category")),
"Filter observation results to these exact categories (e.g. ['requirement']). "
"Accepts a list, a comma-separated string (e.g. 'requirement,decision'), or a JSON-array string. "
"Pair with entity_types=['observation'] to return only observations whose "
"category matches exactly — not every row mentioning the word.",
] = None,
@@ -828,11 +820,6 @@ async def search_notes(
Formatted markdown text (output_format="text"), dict (output_format="json"),
or helpful error guidance string if search fails
Pagination note: `total` is exact only for text/title/permalink searches.
Vector and hybrid searches skip the count query (it would cost a second
semantic retrieval pass) and report `total: 0` even when results are
returned use `has_more` for pagination in those modes.
Examples:
# Basic text search
results = await search_notes("project planning")
@@ -906,17 +893,6 @@ async def search_notes(
if page_size < 1:
raise ValueError(f"page_size must be >= 1, got {page_size}")
# Trigger: list params arrived via a direct function call instead of the MCP layer.
# Why: the BeforeValidator annotations only run through MCP/Pydantic validation; direct
# callers (e.g. `bm tool search-notes --type note,task` in cli/commands/tool.py,
# which Typer collects as the one-element list ["note,task"]) would otherwise
# forward the comma string as one literal type that matches nothing (#930).
# Outcome: comma-split/list normalization applies on every path; parse_str_list is
# idempotent, so MCP-validated input passes through unchanged.
note_types = parse_str_list(note_types) if note_types is not None else []
entity_types = parse_str_list(entity_types) if entity_types is not None else []
categories = parse_str_list(categories) if categories is not None else []
# Avoid mutable-default-argument footguns. Treat None as "no filter".
# Lowercase note_types so "Chapter" matches the stored "chapter".
note_types = [t.lower() for t in note_types] if note_types else []
-4
View File
@@ -18,9 +18,7 @@ def _text_block(message: str) -> List[ContentBlock]:
@mcp.tool(
title="Search Notes (UI)",
description="Search notes and return an embedded MCP-UI resource (raw HTML).",
tags={"search", "ui"},
output_schema=None,
annotations={"readOnlyHint": True, "openWorldHint": False},
)
@@ -94,9 +92,7 @@ async def search_notes_ui(
@mcp.tool(
title="Read Note (UI)",
description="Read a note and return an embedded MCP-UI resource (raw HTML).",
tags={"notes", "ui"},
output_schema=None,
annotations={"readOnlyHint": True, "openWorldHint": False},
)
-2
View File
@@ -11,9 +11,7 @@ from basic_memory.mcp.tools.read_note import read_note
@mcp.tool(
title="View Note",
description="View a note as a formatted artifact for better readability.",
tags={"notes"},
annotations={"readOnlyHint": True, "openWorldHint": False},
)
async def view_note(
-2
View File
@@ -45,9 +45,7 @@ def _workspace_list_response(workspaces: list[WorkspaceInfo]) -> WorkspaceListRe
@mcp.tool(
title="List Workspaces",
description="List available cloud workspaces (tenant_id, type, role, and name).",
tags={"cloud", "projects"},
annotations={"readOnlyHint": True, "openWorldHint": False},
)
async def list_workspaces(
-39
View File
@@ -25,39 +25,8 @@ from basic_memory.workspace_context import current_workspace_permalink_context
TagType = Union[List[str], str, None]
def _compose_workspace_project_route(
*,
workspace: Optional[str],
project: Optional[str],
project_id: Optional[str],
) -> Optional[str]:
"""Return the explicit project route requested by workspace/project args."""
if workspace is None:
return project
cleaned_workspace = workspace.strip().strip("/")
if not cleaned_workspace:
raise ValueError("workspace must not be empty when provided")
if "/" in cleaned_workspace:
raise ValueError("workspace must be a single workspace slug, name, or tenant_id")
if project_id is not None:
raise ValueError("workspace cannot be combined with project_id; use project_id alone")
if project is None or not project.strip().strip("/"):
raise ValueError("workspace requires an explicit project argument")
cleaned_project = project.strip().strip("/")
if "/" in cleaned_project:
raise ValueError(
"Use either workspace='workspace' with project='project', "
"or project='workspace/project', not both"
)
return f"{cleaned_workspace}/{cleaned_project}"
@mcp.tool(
title="Write Note",
description="Create a markdown note. If the note already exists, returns an error by default — pass overwrite=True to replace.",
tags={"notes"},
annotations={"destructiveHint": True, "idempotentHint": False, "openWorldHint": False},
)
async def write_note(
@@ -69,7 +38,6 @@ async def write_note(
Field(validation_alias=AliasChoices("directory", "folder", "dir", "path")),
],
project: Optional[str] = None,
workspace: Optional[str] = None,
project_id: Optional[str] = None,
tags: list[str] | str | None = None,
note_type: str = "note",
@@ -125,8 +93,6 @@ async def write_note(
form (or project_id) to disambiguate. If unknown, use
list_memory_projects() to discover available projects and their
qualified names.
workspace: Workspace slug, name, or tenant_id. When provided with `project`,
routes as `workspace/project`. Cannot be combined with `project_id`.
project_id: Project external_id (UUID). Prefer this over `project` when known
it routes to the exact project regardless of name collisions across cloud
workspaces. Takes precedence over `project`. Get from list_memory_projects().
@@ -204,11 +170,6 @@ async def write_note(
effective_overwrite = (
overwrite if overwrite is not None else ConfigManager().config.write_note_overwrite_default
)
project = _compose_workspace_project_route(
workspace=workspace,
project=project,
project_id=project_id,
)
with logfire.span(
"mcp.tool.write_note",
+138 -138
View File
@@ -7,11 +7,12 @@ from typing import List, Optional, Sequence, Union, Any
from loguru import logger
from sqlalchemy import exists, func, select
from sqlalchemy.exc import IntegrityError
from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
from sqlalchemy.orm import load_only, selectinload
from sqlalchemy.orm.interfaces import LoaderOption
from sqlalchemy.engine import Row
from basic_memory import db
from basic_memory.models.knowledge import Entity, Observation, Relation
from basic_memory.repository.repository import Repository
@@ -23,17 +24,16 @@ class EntityRepository(Repository[Entity]):
to strings before passing to repository methods.
"""
def __init__(self, project_id: int):
"""Initialize with project_id filter.
def __init__(self, session_maker: async_sessionmaker[AsyncSession], project_id: int):
"""Initialize with session maker and project_id filter.
Args:
session_maker: SQLAlchemy session maker
project_id: Project ID to filter all operations by
"""
super().__init__(Entity, project_id=project_id)
super().__init__(session_maker, Entity, project_id=project_id)
async def get_by_id(
self, session: AsyncSession, entity_id: int, *, load_relations: bool = True
) -> Optional[Entity]:
async def get_by_id(self, entity_id: int, *, load_relations: bool = True) -> Optional[Entity]:
"""Get entity by numeric ID.
Args:
@@ -42,24 +42,23 @@ class EntityRepository(Repository[Entity]):
Returns:
Entity if found, None otherwise
"""
if not load_relations:
result = await session.execute(self.select().where(Entity.id == entity_id))
return result.scalars().one_or_none()
async with db.scoped_session(self.session_maker) as session:
if not load_relations:
result = await session.execute(self.select().where(Entity.id == entity_id))
return result.scalars().one_or_none()
return await self.select_by_id(session, entity_id)
return await self.select_by_id(session, entity_id)
async def _find_one_by_query(
self, session: AsyncSession, query, *, load_relations: bool
) -> Optional[Entity]:
async def _find_one_by_query(self, query, *, load_relations: bool) -> Optional[Entity]:
"""Return one entity row with optional eager loading."""
if load_relations:
return await self.find_one(session, query)
return await self.find_one(query)
result = await self.execute_query(session, query, use_query_options=False)
result = await self.execute_query(query, use_query_options=False)
return result.scalars().one_or_none()
async def get_by_external_id(
self, session: AsyncSession, external_id: str, *, load_relations: bool = True
self, external_id: str, *, load_relations: bool = True
) -> Optional[Entity]:
"""Get entity by external UUID.
@@ -70,10 +69,10 @@ class EntityRepository(Repository[Entity]):
Entity if found, None otherwise
"""
query = self.select().where(Entity.external_id == external_id)
return await self._find_one_by_query(session, query, load_relations=load_relations)
return await self._find_one_by_query(query, load_relations=load_relations)
async def get_by_permalink(
self, session: AsyncSession, permalink: str, *, load_relations: bool = True
self, permalink: str, *, load_relations: bool = True
) -> Optional[Entity]:
"""Get entity by permalink.
@@ -81,11 +80,9 @@ class EntityRepository(Repository[Entity]):
permalink: Unique identifier for the entity
"""
query = self.select().where(Entity.permalink == permalink)
return await self._find_one_by_query(session, query, load_relations=load_relations)
return await self._find_one_by_query(query, load_relations=load_relations)
async def get_by_title(
self, session: AsyncSession, title: str, *, load_relations: bool = True
) -> Sequence[Entity]:
async def get_by_title(self, title: str, *, load_relations: bool = True) -> Sequence[Entity]:
"""Get entities by title, ordered by shortest path first.
When multiple entities share the same title (in different folders),
@@ -100,11 +97,11 @@ class EntityRepository(Repository[Entity]):
.where(Entity.title == title)
.order_by(func.length(Entity.file_path), Entity.file_path)
)
result = await self.execute_query(session, query, use_query_options=load_relations)
result = await self.execute_query(query, use_query_options=load_relations)
return list(result.scalars().all())
async def get_by_file_path(
self, session: AsyncSession, file_path: Union[Path, str], *, load_relations: bool = True
self, file_path: Union[Path, str], *, load_relations: bool = True
) -> Optional[Entity]:
"""Get entity by file_path.
@@ -112,13 +109,13 @@ class EntityRepository(Repository[Entity]):
file_path: Path to the entity file (will be converted to string internally)
"""
query = self.select().where(Entity.file_path == Path(file_path).as_posix())
return await self._find_one_by_query(session, query, load_relations=load_relations)
return await self._find_one_by_query(query, load_relations=load_relations)
# -------------------------------------------------------------------------
# Lightweight methods for permalink resolution (no eager loading)
# -------------------------------------------------------------------------
async def permalink_exists(self, session: AsyncSession, permalink: str) -> bool:
async def permalink_exists(self, permalink: str) -> bool:
"""Check if a permalink exists without loading the full entity.
This is much faster than get_by_permalink() as it skips eager loading
@@ -132,12 +129,10 @@ class EntityRepository(Repository[Entity]):
"""
query = select(Entity.id).where(Entity.permalink == permalink).limit(1)
query = self._add_project_filter(query)
result = await self.execute_query(session, query, use_query_options=False)
result = await self.execute_query(query, use_query_options=False)
return result.scalar_one_or_none() is not None
async def get_file_path_for_permalink(
self, session: AsyncSession, permalink: str
) -> Optional[str]:
async def get_file_path_for_permalink(self, permalink: str) -> Optional[str]:
"""Get the file_path for a permalink without loading the full entity.
Use when you only need the file_path, not the full entity with relations.
@@ -150,12 +145,10 @@ class EntityRepository(Repository[Entity]):
"""
query = select(Entity.file_path).where(Entity.permalink == permalink)
query = self._add_project_filter(query)
result = await self.execute_query(session, query, use_query_options=False)
result = await self.execute_query(query, use_query_options=False)
return result.scalar_one_or_none()
async def get_permalink_for_file_path(
self, session: AsyncSession, file_path: Union[Path, str]
) -> Optional[str]:
async def get_permalink_for_file_path(self, file_path: Union[Path, str]) -> Optional[str]:
"""Get the permalink for a file_path without loading the full entity.
Use when you only need the permalink, not the full entity with relations.
@@ -168,10 +161,10 @@ class EntityRepository(Repository[Entity]):
"""
query = select(Entity.permalink).where(Entity.file_path == Path(file_path).as_posix())
query = self._add_project_filter(query)
result = await self.execute_query(session, query, use_query_options=False)
result = await self.execute_query(query, use_query_options=False)
return result.scalar_one_or_none()
async def get_all_permalinks(self, session: AsyncSession) -> List[str]:
async def get_all_permalinks(self) -> List[str]:
"""Get all permalinks for this project.
Optimized for bulk operations - returns only permalink strings
@@ -182,11 +175,11 @@ class EntityRepository(Repository[Entity]):
"""
query = select(Entity.permalink)
query = self._add_project_filter(query)
result = await self.execute_query(session, query, use_query_options=False)
result = await self.execute_query(query, use_query_options=False)
return list(result.scalars().all())
async def find_by_ids_for_hydration(
self, session: AsyncSession, ids: List[int], *, include_cross_project: bool = False
self, ids: List[int], *, include_cross_project: bool = False
) -> Sequence[Entity]:
"""Fetch minimal entity fields needed for context hydration.
@@ -210,10 +203,10 @@ class EntityRepository(Repository[Entity]):
if not include_cross_project:
query = self._add_project_filter(query)
result = await self.execute_query(session, query, use_query_options=False)
result = await self.execute_query(query, use_query_options=False)
return list(result.scalars().all())
async def get_permalink_to_file_path_map(self, session: AsyncSession) -> dict[str, str]:
async def get_permalink_to_file_path_map(self) -> dict[str, str]:
"""Get a mapping of permalink -> file_path for all entities.
Optimized for bulk permalink resolution - loads minimal data in one query.
@@ -223,10 +216,10 @@ class EntityRepository(Repository[Entity]):
"""
query = select(Entity.permalink, Entity.file_path)
query = self._add_project_filter(query)
result = await self.execute_query(session, query, use_query_options=False)
result = await self.execute_query(query, use_query_options=False)
return {row.permalink: row.file_path for row in result.all()}
async def get_file_path_to_permalink_map(self, session: AsyncSession) -> dict[str, str]:
async def get_file_path_to_permalink_map(self) -> dict[str, str]:
"""Get a mapping of file_path -> permalink for all entities.
Optimized for bulk permalink resolution - loads minimal data in one query.
@@ -236,7 +229,7 @@ class EntityRepository(Repository[Entity]):
"""
query = select(Entity.file_path, Entity.permalink)
query = self._add_project_filter(query)
result = await self.execute_query(session, query, use_query_options=False)
result = await self.execute_query(query, use_query_options=False)
return {row.file_path: row.permalink for row in result.all()}
async def get_by_file_paths(
@@ -269,7 +262,7 @@ class EntityRepository(Repository[Entity]):
result = await session.execute(query) # pragma: no cover
return list(result.all()) # pragma: no cover
async def find_by_checksum(self, session: AsyncSession, checksum: str) -> Sequence[Entity]:
async def find_by_checksum(self, checksum: str) -> Sequence[Entity]:
"""Find entities with the given checksum.
Used for move detection - finds entities that may have been moved to a new path.
@@ -283,12 +276,10 @@ class EntityRepository(Repository[Entity]):
"""
query = self.select().where(Entity.checksum == checksum)
# Don't load relationships for move detection - we only need file_path and checksum
result = await self.execute_query(session, query, use_query_options=False)
result = await self.execute_query(query, use_query_options=False)
return list(result.scalars().all())
async def find_by_checksums(
self, session: AsyncSession, checksums: Sequence[str]
) -> Sequence[Entity]:
async def find_by_checksums(self, checksums: Sequence[str]) -> Sequence[Entity]:
"""Find entities with any of the given checksums (batch query for move detection).
This is a batch-optimized version of find_by_checksum() that queries multiple checksums
@@ -313,18 +304,16 @@ class EntityRepository(Repository[Entity]):
# Query: SELECT * FROM entities WHERE checksum IN (checksum1, checksum2, ...)
query = self.select().where(Entity.checksum.in_(checksums)) # pragma: no cover
# Don't load relationships for move detection - we only need file_path and checksum
result = await self.execute_query(
session, query, use_query_options=False
) # pragma: no cover
result = await self.execute_query(query, use_query_options=False) # pragma: no cover
return list(result.scalars().all()) # pragma: no cover
async def delete_by_file_path(self, session: AsyncSession, file_path: Union[Path, str]) -> bool:
async def delete_by_file_path(self, file_path: Union[Path, str]) -> bool:
"""Delete entity with the provided file_path.
Args:
file_path: Path to the entity file (will be converted to string internally)
"""
return await self.delete_by_fields(session, file_path=Path(file_path).as_posix())
return await self.delete_by_fields(file_path=Path(file_path).as_posix())
def get_load_options(self) -> List[LoaderOption]:
"""Get SQLAlchemy loader options for eager loading relationships."""
@@ -338,9 +327,7 @@ class EntityRepository(Repository[Entity]):
selectinload(Entity.incoming_relations).selectinload(Relation.to_entity),
]
async def find_by_permalinks(
self, session: AsyncSession, permalinks: List[str]
) -> Sequence[Entity]:
async def find_by_permalinks(self, permalinks: List[str]) -> Sequence[Entity]:
"""Find multiple entities by their permalink.
Args:
@@ -355,10 +342,10 @@ class EntityRepository(Repository[Entity]):
self.select().options(*self.get_load_options()).where(Entity.permalink.in_(permalinks))
)
result = await self.execute_query(session, query)
result = await self.execute_query(query)
return list(result.scalars().all())
async def upsert_entity(self, session: AsyncSession, entity: Entity) -> Entity:
async def upsert_entity(self, entity: Entity) -> Entity:
"""Insert or update entity using simple try/catch with database-level conflict resolution.
Handles file_path race conditions by checking for existing entity on IntegrityError.
@@ -370,84 +357,99 @@ class EntityRepository(Repository[Entity]):
Returns:
The inserted or updated entity
"""
# Set project_id if applicable and not already set
self._set_project_id_if_needed(entity)
async with db.scoped_session(self.session_maker) as session:
# Set project_id if applicable and not already set
self._set_project_id_if_needed(entity)
# Try simple insert first.
try:
async with session.begin_nested():
# Try simple insert first
try:
session.add(entity)
await session.flush()
except IntegrityError as e:
# Check if this is a FOREIGN KEY constraint failure
# SQLite: "FOREIGN KEY constraint failed"
# Postgres: "violates foreign key constraint"
error_str = str(e)
if (
"FOREIGN KEY constraint failed" in error_str
or "violates foreign key constraint" in error_str
):
# Import locally to avoid circular dependency (repository -> services -> repository)
from basic_memory.services.exceptions import SyncFatalError
# Project doesn't exist in database - this is a fatal sync error
raise SyncFatalError(
f"Cannot sync file '{entity.file_path}': "
f"project_id={entity.project_id} does not exist in database. "
f"The project may have been deleted. This sync will be terminated."
) from e
# Re-query after the nested rollback to get a fresh, attached entity.
existing_result = await session.execute(
select(Entity)
.where(Entity.file_path == entity.file_path, Entity.project_id == entity.project_id)
.options(*self.get_load_options())
)
existing_entity = existing_result.scalar_one_or_none()
if existing_entity:
# File path conflict - update the existing entity
logger.debug(
f"Resolving file_path conflict for {entity.file_path}, "
f"entity_id={existing_entity.id}, observations={len(entity.observations)}"
)
# Use merge to avoid session state conflicts. Preserve stable
# external_id so external references survive re-indexing.
entity.id = existing_entity.id
entity.external_id = existing_entity.external_id
# Ensure observations reference the correct entity_id.
for obs in entity.observations:
obs.entity_id = existing_entity.id
obs.id = None
merged_entity = await session.merge(entity)
await session.flush()
# Re-query to get proper relationships loaded.
final_result = await session.execute(
select(Entity)
.where(Entity.id == merged_entity.id)
# Return with relationships loaded
query = (
self.select()
.where(Entity.file_path == entity.file_path)
.options(*self.get_load_options())
)
return final_result.scalar_one()
result = await session.execute(query)
found = result.scalar_one_or_none()
if not found: # pragma: no cover
raise RuntimeError(
f"Failed to retrieve entity after insert: {entity.file_path}"
)
return found
# No file_path conflict - must be permalink conflict.
return await self._handle_permalink_conflict(entity, session)
except IntegrityError as e:
# Check if this is a FOREIGN KEY constraint failure
# SQLite: "FOREIGN KEY constraint failed"
# Postgres: "violates foreign key constraint"
error_str = str(e)
if (
"FOREIGN KEY constraint failed" in error_str
or "violates foreign key constraint" in error_str
):
# Import locally to avoid circular dependency (repository -> services -> repository)
from basic_memory.services.exceptions import SyncFatalError
# Return with relationships loaded.
query = (
self.select()
.where(Entity.file_path == entity.file_path)
.options(*self.get_load_options())
)
result = await session.execute(query)
found = result.scalar_one_or_none()
if not found: # pragma: no cover
raise RuntimeError(f"Failed to retrieve entity after insert: {entity.file_path}")
return found
# Project doesn't exist in database - this is a fatal sync error
raise SyncFatalError(
f"Cannot sync file '{entity.file_path}': "
f"project_id={entity.project_id} does not exist in database. "
f"The project may have been deleted. This sync will be terminated."
) from e
async def get_all_file_paths(self, session: AsyncSession) -> List[str]:
await session.rollback()
# Re-query after rollback to get a fresh, attached entity
existing_result = await session.execute(
select(Entity)
.where(
Entity.file_path == entity.file_path, Entity.project_id == entity.project_id
)
.options(*self.get_load_options())
)
existing_entity = existing_result.scalar_one_or_none()
if existing_entity:
# File path conflict - update the existing entity
logger.debug(
f"Resolving file_path conflict for {entity.file_path}, "
f"entity_id={existing_entity.id}, observations={len(entity.observations)}"
)
# Use merge to avoid session state conflicts
# Set the ID to update existing entity
entity.id = existing_entity.id
# Preserve the stable external_id so that external references
# (e.g. public share links) survive re-indexing
entity.external_id = existing_entity.external_id
# Ensure observations reference the correct entity_id
for obs in entity.observations:
obs.entity_id = existing_entity.id
# Clear any existing ID to force INSERT as new observation
obs.id = None
# Merge the entity which will update the existing one
merged_entity = await session.merge(entity)
await session.commit()
# Re-query to get proper relationships loaded
final_result = await session.execute(
select(Entity)
.where(Entity.id == merged_entity.id)
.options(*self.get_load_options())
)
return final_result.scalar_one()
else:
# No file_path conflict - must be permalink conflict
# Generate unique permalink and retry
entity = await self._handle_permalink_conflict(entity, session)
return entity
async def get_all_file_paths(self) -> List[str]:
"""Get all file paths for this project - optimized for deletion detection.
Returns only file_path strings without loading entities or relationships.
@@ -459,10 +461,10 @@ class EntityRepository(Repository[Entity]):
query = select(Entity.file_path)
query = self._add_project_filter(query)
result = await self.execute_query(session, query, use_query_options=False)
result = await self.execute_query(query, use_query_options=False)
return list(result.scalars().all())
async def find_without_relations(self, session: AsyncSession) -> Sequence[Entity]:
async def find_without_relations(self) -> Sequence[Entity]:
"""Find entities that have no incoming or outgoing relations."""
# Trigger: entity appears as a source in any relation.
# Why: even unresolved outgoing links mean the entity references another node.
@@ -475,10 +477,10 @@ class EntityRepository(Repository[Entity]):
has_incoming = exists().where(Relation.to_id == Entity.id)
query = self.select().where(~has_outgoing).where(~has_incoming).order_by(Entity.file_path)
result = await self.execute_query(session, query, use_query_options=False)
result = await self.execute_query(query, use_query_options=False)
return list(result.scalars().all())
async def get_distinct_directories(self, session: AsyncSession) -> List[str]:
async def get_distinct_directories(self) -> List[str]:
"""Extract unique directory paths from file_path column.
Optimized method for getting directory structure without loading full entities
@@ -492,7 +494,7 @@ class EntityRepository(Repository[Entity]):
query = self._add_project_filter(query)
# Execute with use_query_options=False to skip eager loading
result = await self.execute_query(session, query, use_query_options=False)
result = await self.execute_query(query, use_query_options=False)
file_paths = [row for row in result.scalars().all()]
# Parse file paths to extract unique directories
@@ -506,9 +508,7 @@ class EntityRepository(Repository[Entity]):
return sorted(directories)
async def find_by_directory_prefix(
self, session: AsyncSession, directory_prefix: str
) -> Sequence[Entity]:
async def find_by_directory_prefix(self, directory_prefix: str) -> Sequence[Entity]:
"""Find entities whose file_path starts with the given directory prefix.
Optimized method for listing directory contents without loading all entities.
@@ -524,7 +524,7 @@ class EntityRepository(Repository[Entity]):
# Build SQL LIKE pattern
if directory_prefix == "" or directory_prefix == "/":
# Root directory - return all entities
return await self.find_all(session)
return await self.find_all()
# Remove leading/trailing slashes for consistency
directory_prefix = directory_prefix.strip("/")
@@ -537,7 +537,7 @@ class EntityRepository(Repository[Entity]):
query = self.select().where(Entity.file_path.like(pattern))
# Skip eager loading - we only need basic entity fields for directory trees
result = await self.execute_query(session, query, use_query_options=False)
result = await self.execute_query(query, use_query_options=False)
return list(result.scalars().all())
async def _handle_permalink_conflict(self, entity: Entity, session: AsyncSession) -> Entity:
@@ -4,8 +4,9 @@ from pathlib import Path
from typing import Any, Mapping, Optional
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
from basic_memory import db
from basic_memory.models import Entity, NoteContent
from basic_memory.repository.repository import Repository
@@ -29,9 +30,9 @@ NOTE_CONTENT_MUTABLE_FIELDS = frozenset(
class NoteContentRepository(Repository[NoteContent]):
"""Repository for project-scoped note materialization state."""
def __init__(self, project_id: int):
"""Initialize with project-scoped filtering."""
super().__init__(NoteContent, project_id=project_id)
def __init__(self, session_maker: async_sessionmaker[AsyncSession], project_id: int):
"""Initialize with session maker and project-scoped filtering."""
super().__init__(session_maker, NoteContent, project_id=project_id)
def _coerce_note_content(
self, data: Mapping[str, Any] | NoteContent
@@ -74,22 +75,16 @@ class NoteContentRepository(Repository[NoteContent]):
note_content.external_id = entity.external_id
note_content.file_path = Path(entity.file_path).as_posix()
async def get_by_entity_id(
self, session: AsyncSession, entity_id: int
) -> Optional[NoteContent]:
async def get_by_entity_id(self, entity_id: int) -> Optional[NoteContent]:
"""Get note content by the owning entity identifier."""
return await self.find_by_id(session, entity_id)
return await self.find_by_id(entity_id)
async def get_by_external_id(
self, session: AsyncSession, external_id: str
) -> Optional[NoteContent]:
async def get_by_external_id(self, external_id: str) -> Optional[NoteContent]:
"""Get note content by the mirrored entity external identifier."""
query = self.select().where(NoteContent.external_id == external_id)
return await self.find_one(session, query)
return await self.find_one(query)
async def get_by_file_path(
self, session: AsyncSession, file_path: Path | str
) -> Optional[NoteContent]:
async def get_by_file_path(self, file_path: Path | str) -> Optional[NoteContent]:
"""Get note content by file path, preferring rows whose entity still owns that path."""
normalized_path = Path(file_path).as_posix()
@@ -109,88 +104,88 @@ class NoteContentRepository(Repository[NoteContent]):
.options(*self.get_load_options())
)
result = await session.execute(query)
return result.scalars().first()
async with db.scoped_session(self.session_maker) as session:
result = await session.execute(query)
return result.scalars().first()
async def create(
self, session: AsyncSession, data: Mapping[str, Any] | NoteContent
) -> NoteContent:
async def create(self, data: Mapping[str, Any] | NoteContent) -> NoteContent:
"""Create a note_content row aligned to its owning entity."""
note_content, _ = self._coerce_note_content(data)
await self._align_identity_fields(session, note_content)
session.add(note_content)
await session.flush()
created = await self.select_by_id(session, note_content.entity_id)
if created is None: # pragma: no cover
raise ValueError(
f"Can't find NoteContent for entity {note_content.entity_id} after add"
)
return created
async def upsert(
self, session: AsyncSession, data: Mapping[str, Any] | NoteContent
) -> NoteContent:
"""Insert or update note_content while keeping mirrored identity fields in sync."""
note_content, provided_fields = self._coerce_note_content(data)
await self._align_identity_fields(session, note_content)
existing = await self.select_by_id(session, note_content.entity_id)
if existing is None:
async with db.scoped_session(self.session_maker) as session:
await self._align_identity_fields(session, note_content)
session.add(note_content)
await session.flush()
created = await self.select_by_id(session, note_content.entity_id)
if created is None: # pragma: no cover
raise ValueError(
f"Can't find NoteContent for entity {note_content.entity_id} after upsert"
f"Can't find NoteContent for entity {note_content.entity_id} after add"
)
return created
fields_to_update = (provided_fields - {"entity_id"}) | {
"project_id",
"external_id",
"file_path",
}
for column_name in fields_to_update:
setattr(existing, column_name, getattr(note_content, column_name))
async def upsert(self, data: Mapping[str, Any] | NoteContent) -> NoteContent:
"""Insert or update note_content while keeping mirrored identity fields in sync."""
note_content, provided_fields = self._coerce_note_content(data)
await session.flush()
updated = await self.select_by_id(session, existing.entity_id)
if updated is None: # pragma: no cover
raise ValueError(f"Can't find NoteContent for entity {existing.entity_id} after upsert")
return updated
async with db.scoped_session(self.session_maker) as session:
await self._align_identity_fields(session, note_content)
existing = await self.select_by_id(session, note_content.entity_id)
async def update_state_fields(
self, session: AsyncSession, entity_id: int, **updates: Any
) -> Optional[NoteContent]:
if existing is None:
session.add(note_content)
await session.flush()
created = await self.select_by_id(session, note_content.entity_id)
if created is None: # pragma: no cover
raise ValueError(
f"Can't find NoteContent for entity {note_content.entity_id} after upsert"
)
return created
fields_to_update = (provided_fields - {"entity_id"}) | {
"project_id",
"external_id",
"file_path",
}
for column_name in fields_to_update:
setattr(existing, column_name, getattr(note_content, column_name))
await session.flush()
updated = await self.select_by_id(session, existing.entity_id)
if updated is None: # pragma: no cover
raise ValueError(
f"Can't find NoteContent for entity {existing.entity_id} after upsert"
)
return updated
async def update_state_fields(self, entity_id: int, **updates: Any) -> Optional[NoteContent]:
"""Update sync fields and re-align project_id, external_id, and file_path from entity."""
invalid_fields = set(updates) - NOTE_CONTENT_MUTABLE_FIELDS
if invalid_fields:
invalid_list = ", ".join(sorted(invalid_fields))
raise ValueError(f"Unsupported note_content update fields: {invalid_list}")
note_content = await self.select_by_id(session, entity_id)
if note_content is None:
return None
async with db.scoped_session(self.session_maker) as session:
note_content = await self.select_by_id(session, entity_id)
if note_content is None:
return None
await self._align_identity_fields(session, note_content)
for field_name, value in updates.items():
setattr(note_content, field_name, value)
await self._align_identity_fields(session, note_content)
for field_name, value in updates.items():
setattr(note_content, field_name, value)
await session.flush()
updated = await self.select_by_id(session, entity_id)
if updated is None: # pragma: no cover
raise ValueError(f"Can't find NoteContent for entity {entity_id} after update")
return updated
await session.flush()
updated = await self.select_by_id(session, entity_id)
if updated is None: # pragma: no cover
raise ValueError(f"Can't find NoteContent for entity {entity_id} after update")
return updated
async def delete_by_entity_id(self, session: AsyncSession, entity_id: int) -> bool:
async def delete_by_entity_id(self, entity_id: int) -> bool:
"""Delete note_content by entity identifier."""
note_content = await self.select_by_id(session, entity_id)
if note_content is None:
return False
async with db.scoped_session(self.session_maker) as session:
note_content = await self.select_by_id(session, entity_id)
if note_content is None:
return False
await session.delete(note_content)
await session.flush()
return True
await session.delete(note_content)
return True
@@ -3,7 +3,7 @@
from typing import Dict, List, Sequence
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy.ext.asyncio import async_sessionmaker
from sqlalchemy.orm import selectinload
from sqlalchemy.orm.interfaces import LoaderOption
@@ -14,46 +14,44 @@ from basic_memory.repository.repository import Repository
class ObservationRepository(Repository[Observation]):
"""Repository for Observation model with memory-specific operations."""
def __init__(self, project_id: int):
"""Initialize with project_id filter.
def __init__(self, session_maker: async_sessionmaker, project_id: int):
"""Initialize with session maker and project_id filter.
Args:
session_maker: SQLAlchemy session maker
project_id: Project ID to filter all operations by
"""
super().__init__(Observation, project_id=project_id)
super().__init__(session_maker, Observation, project_id=project_id)
def get_load_options(self) -> List[LoaderOption]:
"""Eager-load parent entity to prevent N+1 if obs.entity is accessed."""
return [selectinload(Observation.entity)]
async def find_by_entity(self, session: AsyncSession, entity_id: int) -> Sequence[Observation]:
async def find_by_entity(self, entity_id: int) -> Sequence[Observation]:
"""Find all observations for a specific entity."""
query = self.select().filter(Observation.entity_id == entity_id)
result = await self.execute_query(session, query)
query = select(Observation).filter(Observation.entity_id == entity_id)
result = await self.execute_query(query)
return result.scalars().all()
async def find_by_context(self, session: AsyncSession, context: str) -> Sequence[Observation]:
async def find_by_context(self, context: str) -> Sequence[Observation]:
"""Find observations with a specific context."""
query = self.select().filter(Observation.context == context)
result = await self.execute_query(session, query)
query = select(Observation).filter(Observation.context == context)
result = await self.execute_query(query)
return result.scalars().all()
async def find_by_category(self, session: AsyncSession, category: str) -> Sequence[Observation]:
async def find_by_category(self, category: str) -> Sequence[Observation]:
"""Find observations with a specific context."""
query = self.select().filter(Observation.category == category)
result = await self.execute_query(session, query)
query = select(Observation).filter(Observation.category == category)
result = await self.execute_query(query)
return result.scalars().all()
async def observation_categories(self, session: AsyncSession) -> Sequence[str]:
async def observation_categories(self) -> Sequence[str]:
"""Return a list of all observation categories."""
query = select(Observation.category).distinct()
query = self._add_project_filter(query)
result = await self.execute_query(session, query, use_query_options=False)
result = await self.execute_query(query, use_query_options=False)
return result.scalars().all()
async def find_by_entities(
self, session: AsyncSession, entity_ids: List[int]
) -> Dict[int, List[Observation]]:
async def find_by_entities(self, entity_ids: List[int]) -> Dict[int, List[Observation]]:
"""Find all observations for multiple entities in a single query.
Args:
@@ -66,8 +64,8 @@ class ObservationRepository(Repository[Observation]):
return {}
# Query observations for all entities in the list
query = self.select().filter(Observation.entity_id.in_(entity_ids))
result = await self.execute_query(session, query)
query = select(Observation).filter(Observation.entity_id.in_(entity_ids))
result = await self.execute_query(query)
observations = result.scalars().all()
# Group observations by entity_id
@@ -18,7 +18,6 @@ from basic_memory.repository.search_index_row import SearchIndexRow
from basic_memory.repository.search_repository_base import (
SearchRepositoryBase,
VectorChunkState,
relaxed_query_words,
)
from basic_memory.repository.metadata_filters import parse_metadata_filters
from basic_memory.repository.semantic_errors import SemanticDependenciesMissingError
@@ -177,14 +176,6 @@ class PostgresSearchRepository(SearchRepositoryBase):
# For non-Boolean queries, prepare single term
return self._prepare_single_term(term, is_prefix)
@staticmethod
def _relaxed_tsquery_text(search_text: Optional[str]) -> Optional[str]:
"""OR-relaxed tsquery expression for a failed strict query, or None."""
words = relaxed_query_words(search_text)
if not words:
return None
return " | ".join(f"{word}:*" for word in words)
def _prepare_boolean_query(self, query: str) -> str:
"""Convert Boolean query to tsquery format.
@@ -245,12 +236,7 @@ class PostgresSearchRepository(SearchRepositoryBase):
# Handle multi-word queries
if " " in cleaned_term:
# Strip sentence punctuation from word edges so question-form
# queries produce clean lexemes (parity with SQLite FTS5 prep).
# The tsquery tokenizer ignores this punctuation anyway; leaving it
# in only risks tsquery syntax errors. Interior characters are kept.
words = [w.strip("?!.,;") for w in cleaned_term.split()]
words = [w for w in words if w]
words = [w for w in cleaned_term.split() if w.strip()]
if not words:
# All characters were special chars, search won't match anything
# Return a safe search term that won't cause syntax errors
@@ -263,11 +249,8 @@ class PostgresSearchRepository(SearchRepositoryBase):
# Join with AND operator
return " & ".join(prepared_words)
# Single word: strip edge punctuation; guard the now-empty case so a
# bare ":*"/"" never reaches tsquery.
cleaned_term = cleaned_term.strip().strip("?!.,;")
if not cleaned_term:
return "NOSPECIALCHARS:*"
# Single word
cleaned_term = cleaned_term.strip()
if is_prefix:
return f"{cleaned_term}:*"
else:
@@ -925,8 +908,6 @@ class PostgresSearchRepository(SearchRepositoryBase):
min_similarity: Optional[float] = None,
limit: int = 10,
offset: int = 0,
allow_relaxed: bool = False,
session: AsyncSession | None = None,
) -> List[SearchIndexRow]:
"""Search across all indexed content using PostgreSQL tsvector."""
# --- Dispatch vector / hybrid modes (shared logic) ---
@@ -997,32 +978,10 @@ class PostgresSearchRepository(SearchRepositoryBase):
"""
logger.trace(f"Search {sql} params: {params}")
async def run_search(active_session: AsyncSession):
result = await active_session.execute(text(sql), params)
rows = result.fetchall()
# Trigger: multi-word natural-language query matched nothing
# under the default all-terms-AND tsquery semantics.
# Why: questions rarely have every word in one document;
# without relaxation the FTS half of hybrid search contributes
# zero candidates (parity with the SQLite path).
# Outcome: one retry with OR-joined prefix lexemes; ts_rank
# still ranks multi-term matches first.
relaxed = (
self._relaxed_tsquery_text(search_text) if allow_relaxed and not rows else None
)
if relaxed and params.get("text"):
params["text"] = relaxed
result = await active_session.execute(text(sql), params)
rows = result.fetchall()
return rows
try:
if session is not None:
rows = await run_search(session)
else:
async with db.scoped_session(self.session_maker) as owned_session:
rows = await run_search(owned_session)
async with db.scoped_session(self.session_maker) as session:
result = await session.execute(text(sql), params)
rows = result.fetchall()
except Exception as e:
if self._is_tsquery_syntax_error(e):
logger.warning(f"tsquery syntax error for search term: {search_text}, error: {e}")
@@ -5,6 +5,6 @@ from basic_memory.models.project import Project
class ProjectInfoRepository(Repository):
"""Repository for statistics queries."""
def __init__(self):
def __init__(self, session_maker):
# Initialize with Project model as a reference
super().__init__(Project)
super().__init__(session_maker, Project)
+102 -101
View File
@@ -7,8 +7,9 @@ from typing import Optional, Sequence, Union
from loguru import logger
from sqlalchemy import Executable, inspect as sa_inspect, select, text
from sqlalchemy.exc import NoResultFound, OperationalError
from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
from basic_memory import db
from basic_memory.models.project import Project
from basic_memory.repository.repository import Repository
@@ -78,22 +79,20 @@ class ProjectRepository(Repository[Project]):
Each entity, observation, and relation belongs to a specific project.
"""
def __init__(self):
"""Initialize the repository."""
super().__init__(Project)
def __init__(self, session_maker: async_sessionmaker[AsyncSession]):
"""Initialize with session maker."""
super().__init__(session_maker, Project)
async def get_by_name(self, session: AsyncSession, name: str) -> Optional[Project]:
async def get_by_name(self, name: str) -> Optional[Project]:
"""Get project by name (exact match).
Args:
name: Unique name of the project
"""
query = self.select().where(Project.name == name)
return await self.find_one(session, query)
return await self.find_one(query)
async def get_by_name_case_insensitive(
self, session: AsyncSession, name: str
) -> Optional[Project]:
async def get_by_name_case_insensitive(self, name: str) -> Optional[Project]:
"""Get project by name (case-insensitive match).
Args:
@@ -103,27 +102,27 @@ class ProjectRepository(Repository[Project]):
Project if found, None otherwise
"""
query = self.select().where(Project.name.ilike(name))
return await self.find_one(session, query)
return await self.find_one(query)
async def get_by_permalink(self, session: AsyncSession, permalink: str) -> Optional[Project]:
async def get_by_permalink(self, permalink: str) -> Optional[Project]:
"""Get project by permalink.
Args:
permalink: URL-friendly identifier for the project
"""
query = self.select().where(Project.permalink == permalink)
return await self.find_one(session, query)
return await self.find_one(query)
async def get_by_path(self, session: AsyncSession, path: Union[Path, str]) -> Optional[Project]:
async def get_by_path(self, path: Union[Path, str]) -> Optional[Project]:
"""Get project by filesystem path.
Args:
path: Path to the project directory (will be converted to string internally)
"""
query = self.select().where(Project.path == Path(path).as_posix())
return await self.find_one(session, query)
return await self.find_one(query)
async def get_by_id(self, session: AsyncSession, project_id: int) -> Optional[Project]:
async def get_by_id(self, project_id: int) -> Optional[Project]:
"""Get project by numeric ID.
Args:
@@ -132,11 +131,10 @@ class ProjectRepository(Repository[Project]):
Returns:
Project if found, None otherwise
"""
return await self.select_by_id(session, project_id)
async with db.scoped_session(self.session_maker) as session:
return await self.select_by_id(session, project_id)
async def get_by_external_id(
self, session: AsyncSession, external_id: str
) -> Optional[Project]:
async def get_by_external_id(self, external_id: str) -> Optional[Project]:
"""Get project by external UUID.
Args:
@@ -146,20 +144,20 @@ class ProjectRepository(Repository[Project]):
Project if found, None otherwise
"""
query = self.select().where(Project.external_id == external_id)
return await self.find_one(session, query)
return await self.find_one(query)
async def get_default_project(self, session: AsyncSession) -> Optional[Project]:
async def get_default_project(self) -> Optional[Project]:
"""Get the default project (the one marked as is_default=True)."""
query = self.select().where(Project.is_default.is_(True))
return await self.find_one(session, query)
return await self.find_one(query)
async def get_active_projects(self, session: AsyncSession) -> Sequence[Project]:
async def get_active_projects(self) -> Sequence[Project]:
"""Get all active projects."""
query = self.select().where(Project.is_active == True) # noqa: E712
result = await self.execute_query(session, query)
result = await self.execute_query(query)
return list(result.scalars().all())
async def set_as_default(self, session: AsyncSession, project_id: int) -> Optional[Project]:
async def set_as_default(self, project_id: int) -> Optional[Project]:
"""Set a project as the default and unset previous default.
Args:
@@ -168,21 +166,22 @@ class ProjectRepository(Repository[Project]):
Returns:
The updated project if found, None otherwise
"""
# First, clear the default flag for all projects using direct SQL
await session.execute(
text("UPDATE project SET is_default = NULL WHERE is_default IS NOT NULL")
)
await session.flush()
# Set the new default project
target_project = await self.select_by_id(session, project_id)
if target_project:
target_project.is_default = True
async with db.scoped_session(self.session_maker) as session:
# First, clear the default flag for all projects using direct SQL
await session.execute(
text("UPDATE project SET is_default = NULL WHERE is_default IS NOT NULL")
)
await session.flush()
return target_project
return None # pragma: no cover
async def delete(self, session: AsyncSession, entity_id: int) -> bool:
# Set the new default project
target_project = await self.select_by_id(session, project_id)
if target_project:
target_project.is_default = True
await session.flush()
return target_project
return None # pragma: no cover
async def delete(self, entity_id: int) -> bool:
"""Delete a project and its derived search rows in one transaction.
The cascade picture differs by backend:
@@ -205,60 +204,62 @@ class ProjectRepository(Repository[Project]):
Inspect the connection once and skip whichever is missing.
"""
logger.debug(f"Deleting Project and search rows for project_id: {entity_id}")
try:
result = await session.execute(select(self.Model).filter(self.primary_key == entity_id))
project = result.scalars().one()
except NoResultFound:
logger.debug(f"No Project found to delete: {entity_id}")
return False
async with db.scoped_session(self.session_maker) as session:
try:
result = await session.execute(
select(self.Model).filter(self.primary_key == entity_id)
)
project = result.scalars().one()
except NoResultFound:
logger.debug(f"No Project found to delete: {entity_id}")
return False
dialect_name = session.bind.dialect.name if session.bind else "sqlite"
is_sqlite = dialect_name == "sqlite"
dialect_name = session.bind.dialect.name if session.bind else "sqlite"
is_sqlite = dialect_name == "sqlite"
existing_tables = await session.run_sync(
lambda sync_session: set(sa_inspect(sync_session.connection()).get_table_names())
)
# search_index: SQLite has no FK on the FTS5 virtual table; Postgres
# cascades from the project FK, so the explicit DELETE is redundant.
if is_sqlite and "search_index" in existing_tables:
await session.execute(
text("DELETE FROM search_index WHERE project_id = :project_id"),
{"project_id": entity_id},
existing_tables = await session.run_sync(
lambda sync_session: set(sa_inspect(sync_session.connection()).get_table_names())
)
# search_vector_chunks: no FK to project on either backend, so both
# backends need this. SQLite must purge vec0 embeddings first
# (rowid pseudocolumn — Postgres uses chunk_id and would 500 here);
# Postgres' chunk_id FK CASCADE handles its embeddings cleanup when
# we delete the chunk rows below.
if "search_vector_chunks" in existing_tables:
if is_sqlite and "search_vector_embeddings" in existing_tables:
# Extension loading is per-connection. We must load vec0 on
# *this* session before the DELETE; otherwise a different
# pooled connection might have written embeddings that we'd
# silently leave behind.
if await _load_sqlite_vec_on_session(session):
await session.execute(
text(
"DELETE FROM search_vector_embeddings WHERE rowid IN ("
"SELECT id FROM search_vector_chunks "
"WHERE project_id = :project_id)"
),
{"project_id": entity_id},
)
await session.execute(
text("DELETE FROM search_vector_chunks WHERE project_id = :project_id"),
{"project_id": entity_id},
)
# search_index: SQLite has no FK on the FTS5 virtual table; Postgres
# cascades from the project FK, so the explicit DELETE is redundant.
if is_sqlite and "search_index" in existing_tables:
await session.execute(
text("DELETE FROM search_index WHERE project_id = :project_id"),
{"project_id": entity_id},
)
await session.delete(project)
await session.flush()
logger.debug(f"Deleted Project and search rows for project_id: {entity_id}")
return True
# search_vector_chunks: no FK to project on either backend, so both
# backends need this. SQLite must purge vec0 embeddings first
# (rowid pseudocolumn — Postgres uses chunk_id and would 500 here);
# Postgres' chunk_id FK CASCADE handles its embeddings cleanup when
# we delete the chunk rows below.
if "search_vector_chunks" in existing_tables:
if is_sqlite and "search_vector_embeddings" in existing_tables:
# Extension loading is per-connection. We must load vec0 on
# *this* session before the DELETE; otherwise a different
# pooled connection might have written embeddings that we'd
# silently leave behind.
if await _load_sqlite_vec_on_session(session):
await session.execute(
text(
"DELETE FROM search_vector_embeddings WHERE rowid IN ("
"SELECT id FROM search_vector_chunks "
"WHERE project_id = :project_id)"
),
{"project_id": entity_id},
)
await session.execute(
text("DELETE FROM search_vector_chunks WHERE project_id = :project_id"),
{"project_id": entity_id},
)
await session.delete(project)
logger.debug(f"Deleted Project and search rows for project_id: {entity_id}")
return True
async def scalar_vec_query(
self, session: AsyncSession, query: Executable, params: Optional[dict] = None
self, query: Executable, params: Optional[dict] = None
) -> Optional[int]:
"""Run a scalar COUNT query that reads the sqlite-vec vec0 table.
@@ -270,17 +271,16 @@ class ProjectRepository(Repository[Project]):
Returns None when sqlite-vec cannot be loaded on this Python build, so
callers can fall back to the genuinely-missing-dependency path.
"""
# Trigger: query reads the vec0-backed search_vector_embeddings table.
# Why: vec0 modules are only visible on a connection that loaded sqlite-vec.
# Outcome: load the extension here, or signal absence so the caller degrades.
if not await _load_sqlite_vec_on_session(session):
return None
result = await session.execute(query, params)
return result.scalar()
async with db.scoped_session(self.session_maker) as session:
# Trigger: query reads the vec0-backed search_vector_embeddings table.
# Why: vec0 modules are only visible on a connection that loaded sqlite-vec.
# Outcome: load the extension here, or signal absence so the caller degrades.
if not await _load_sqlite_vec_on_session(session):
return None
result = await session.execute(query, params)
return result.scalar()
async def update_path(
self, session: AsyncSession, project_id: int, new_path: str
) -> Optional[Project]:
async def update_path(self, project_id: int, new_path: str) -> Optional[Project]:
"""Update project path.
Args:
@@ -290,9 +290,10 @@ class ProjectRepository(Repository[Project]):
Returns:
The updated project if found, None otherwise
"""
project = await self.select_by_id(session, project_id)
if project:
project.path = new_path
await session.flush()
return project
return None
async with db.scoped_session(self.session_maker) as session:
project = await self.select_by_id(session, project_id)
if project:
project.path = new_path
await session.flush()
return project
return None
@@ -6,10 +6,11 @@ from sqlalchemy import and_, delete, select
from sqlalchemy.engine import CursorResult
from sqlalchemy.dialects.postgresql import insert as pg_insert
from sqlalchemy.dialects.sqlite import insert as sqlite_insert
from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy.ext.asyncio import async_sessionmaker
from sqlalchemy.orm import selectinload, aliased
from sqlalchemy.orm.interfaces import LoaderOption
from basic_memory import db
from basic_memory.models import Relation, Entity
from basic_memory.repository.repository import Repository
@@ -17,20 +18,17 @@ from basic_memory.repository.repository import Repository
class RelationRepository(Repository[Relation]):
"""Repository for Relation model with memory-specific operations."""
def __init__(self, project_id: int):
"""Initialize with project_id filter.
def __init__(self, session_maker: async_sessionmaker, project_id: int):
"""Initialize with session maker and project_id filter.
Args:
session_maker: SQLAlchemy session maker
project_id: Project ID to filter all operations by
"""
super().__init__(Relation, project_id=project_id)
super().__init__(session_maker, Relation, project_id=project_id)
async def find_relation(
self,
session: AsyncSession,
from_permalink: str,
to_permalink: str,
relation_type: str,
self, from_permalink: str, to_permalink: str, relation_type: str
) -> Optional[Relation]:
"""Find a relation by its from and to path IDs."""
from_entity = aliased(Entity)
@@ -48,44 +46,36 @@ class RelationRepository(Repository[Relation]):
)
)
)
query = self._add_project_filter(query)
return await self.find_one(session, query)
return await self.find_one(query)
async def find_by_entities(
self, session: AsyncSession, from_id: int, to_id: int
) -> Sequence[Relation]:
async def find_by_entities(self, from_id: int, to_id: int) -> Sequence[Relation]:
"""Find all relations between two entities."""
query = self.select().where((Relation.from_id == from_id) & (Relation.to_id == to_id))
result = await self.execute_query(session, query)
query = select(Relation).where((Relation.from_id == from_id) & (Relation.to_id == to_id))
result = await self.execute_query(query)
return result.scalars().all()
async def find_by_type(self, session: AsyncSession, relation_type: str) -> Sequence[Relation]:
async def find_by_type(self, relation_type: str) -> Sequence[Relation]:
"""Find all relations of a specific type."""
query = self.select().filter(Relation.relation_type == relation_type)
result = await self.execute_query(session, query)
query = select(Relation).filter(Relation.relation_type == relation_type)
result = await self.execute_query(query)
return result.scalars().all()
async def delete_outgoing_relations_from_entity(
self, session: AsyncSession, entity_id: int
) -> None:
async def delete_outgoing_relations_from_entity(self, entity_id: int) -> None:
"""Delete outgoing relations for an entity.
Only deletes relations where this entity is the source (from_id),
as these are the ones owned by this entity's markdown file.
"""
query = delete(Relation).where(Relation.from_id == entity_id)
query = query.where(Relation.project_id == self.project_id)
await session.execute(query)
async with db.scoped_session(self.session_maker) as session:
await session.execute(delete(Relation).where(Relation.from_id == entity_id))
async def find_unresolved_relations(self, session: AsyncSession) -> Sequence[Relation]:
async def find_unresolved_relations(self) -> Sequence[Relation]:
"""Find all unresolved relations, where to_id is null."""
query = self.select().filter(Relation.to_id.is_(None))
result = await self.execute_query(session, query)
query = select(Relation).filter(Relation.to_id.is_(None))
result = await self.execute_query(query)
return result.scalars().all()
async def find_unresolved_relations_for_entity(
self, session: AsyncSession, entity_id: int
) -> Sequence[Relation]:
async def find_unresolved_relations_for_entity(self, entity_id: int) -> Sequence[Relation]:
"""Find unresolved relations for a specific entity.
Args:
@@ -94,13 +84,11 @@ class RelationRepository(Repository[Relation]):
Returns:
List of unresolved relations where this entity is the source.
"""
query = self.select().filter(Relation.from_id == entity_id, Relation.to_id.is_(None))
result = await self.execute_query(session, query)
query = select(Relation).filter(Relation.from_id == entity_id, Relation.to_id.is_(None))
result = await self.execute_query(query)
return result.scalars().all()
async def add_all_ignore_duplicates(
self, session: AsyncSession, relations: List[Relation]
) -> int:
async def add_all_ignore_duplicates(self, relations: List[Relation]) -> int:
"""Bulk insert relations, ignoring duplicates.
Uses ON CONFLICT DO NOTHING to skip relations that would violate the
@@ -132,23 +120,27 @@ class RelationRepository(Repository[Relation]):
for r in relations
]
# Check dialect to use appropriate insert
dialect_name = session.bind.dialect.name if session.bind else "sqlite"
async with db.scoped_session(self.session_maker) as session:
# Check dialect to use appropriate insert
dialect_name = session.bind.dialect.name if session.bind else "sqlite"
if dialect_name == "postgresql": # pragma: no cover
# PostgreSQL: use RETURNING to count inserted rows
# (rowcount is 0 for ON CONFLICT DO NOTHING)
stmt = ( # pragma: no cover
pg_insert(Relation).values(values).on_conflict_do_nothing().returning(Relation.id)
)
result = await session.execute(stmt) # pragma: no cover
return len(result.fetchall()) # pragma: no cover
else:
# SQLite: rowcount works correctly
stmt = sqlite_insert(Relation).values(values)
stmt = stmt.on_conflict_do_nothing()
result = cast(CursorResult[Any], await session.execute(stmt))
return result.rowcount if result.rowcount > 0 else 0
if dialect_name == "postgresql": # pragma: no cover
# PostgreSQL: use RETURNING to count inserted rows
# (rowcount is 0 for ON CONFLICT DO NOTHING)
stmt = ( # pragma: no cover
pg_insert(Relation)
.values(values)
.on_conflict_do_nothing()
.returning(Relation.id)
)
result = await session.execute(stmt) # pragma: no cover
return len(result.fetchall()) # pragma: no cover
else:
# SQLite: rowcount works correctly
stmt = sqlite_insert(Relation).values(values)
stmt = stmt.on_conflict_do_nothing()
result = cast(CursorResult[Any], await session.execute(stmt))
return result.rowcount if result.rowcount > 0 else 0
def get_load_options(self) -> List[LoaderOption]:
return [selectinload(Relation.from_entity), selectinload(Relation.to_entity)]
+201 -181
View File
@@ -1,6 +1,7 @@
"""Base repository implementation."""
from typing import Optional, Any, Sequence, TypeVar, List, Dict, cast
from typing import Type, Optional, Any, Sequence, TypeVar, List, Dict, cast
from loguru import logger
from sqlalchemy import (
@@ -15,23 +16,27 @@ from sqlalchemy import (
update as sqlalchemy_update,
)
from sqlalchemy.engine import CursorResult
from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy.exc import NoResultFound
from sqlalchemy.ext.asyncio import async_sessionmaker, AsyncSession
from sqlalchemy.orm.interfaces import LoaderOption
from sqlalchemy.sql.elements import ColumnElement
from basic_memory import db
from basic_memory.models import Base
T = TypeVar("T", bound=Base)
class Repository[T: Base]:
"""Base repository implementation with explicit caller-owned sessions."""
"""Base repository implementation with generic CRUD operations."""
def __init__(
self,
Model: type[T],
session_maker: async_sessionmaker[AsyncSession],
Model: Type[T],
project_id: Optional[int] = None,
):
self.session_maker = session_maker
self.project_id = project_id
if Model:
self.Model = Model
@@ -93,61 +98,62 @@ class Repository[T: Base]:
result = await session.execute(query)
return result.scalars().all()
async def add(self, session: AsyncSession, model: T) -> T:
async def add(self, model: T) -> T:
"""
Add a model to the repository. This will also add related objects
:param session: the caller-owned session to use
:param model: the model to add
:return: the added model instance
"""
# Set project_id if applicable and not already set
self._set_project_id_if_needed(model)
async with db.scoped_session(self.session_maker) as session:
# Set project_id if applicable and not already set
self._set_project_id_if_needed(model)
session.add(model)
await session.flush()
session.add(model)
await session.flush()
# Query within same session
found = await self.select_by_id(session, model.id) # pyright: ignore [reportAttributeAccessIssue]
if found is None: # pragma: no cover
logger.error(
"Failed to retrieve model after add",
model_type=self.Model.__name__,
model_id=model.id, # pyright: ignore
)
raise ValueError(
f"Can't find {self.Model.__name__} with ID {model.id} after session.add" # pyright: ignore
)
return found
# Query within same session
found = await self.select_by_id(session, model.id) # pyright: ignore [reportAttributeAccessIssue]
if found is None: # pragma: no cover
logger.error(
"Failed to retrieve model after add",
model_type=self.Model.__name__,
model_id=model.id, # pyright: ignore
)
raise ValueError(
f"Can't find {self.Model.__name__} with ID {model.id} after session.add" # pyright: ignore
)
return found
async def add_all(self, session: AsyncSession, models: List[T]) -> Sequence[T]:
async def add_all(self, models: List[T]) -> Sequence[T]:
"""
Add a list of models to the repository. This will also add related objects
:param session: the caller-owned session to use
:param models: the models to add
:return: the added models instances
"""
# set the project id if not present in models
for model in models:
self._set_project_id_if_needed(model)
async with db.scoped_session(self.session_maker) as session:
# set the project id if not present in models
for model in models:
self._set_project_id_if_needed(model)
session.add_all(models)
await session.flush()
session.add_all(models)
await session.flush()
# Query within same session
return await self.select_by_ids(session, [m.id for m in models]) # pyright: ignore [reportAttributeAccessIssue]
# Query within same session
return await self.select_by_ids(session, [m.id for m in models]) # pyright: ignore [reportAttributeAccessIssue]
async def add_all_no_return(self, session: AsyncSession, models: List[T]) -> int:
async def add_all_no_return(self, models: List[T]) -> int:
"""Insert models without reloading them afterward."""
if not models:
return 0
for model in models:
self._set_project_id_if_needed(model)
async with db.scoped_session(self.session_maker) as session:
for model in models:
self._set_project_id_if_needed(model)
session.add_all(models)
await session.flush()
logger.debug(f"Added {len(models)} {self.Model.__name__} records")
return len(models)
session.add_all(models)
await session.flush()
logger.debug(f"Added {len(models)} {self.Model.__name__} records")
return len(models)
def select(self, *entities: Any) -> Select:
"""Create a new SELECT statement.
@@ -164,55 +170,55 @@ class Repository[T: Base]:
return self._add_project_filter(query)
async def find_all(
self,
session: AsyncSession,
skip: int = 0,
limit: Optional[int] = None,
use_load_options: bool = True,
self, skip: int = 0, limit: Optional[int] = None, use_load_options: bool = True
) -> Sequence[T]:
"""Fetch records from the database with pagination.
Args:
session: The caller-owned session to use
skip: Number of records to skip
limit: Maximum number of records to return
use_load_options: Whether to apply eager loading options (default: True)
"""
logger.debug(f"Finding all {self.Model.__name__} (skip={skip}, limit={limit})")
query = select(self.Model).offset(skip)
async with db.scoped_session(self.session_maker) as session:
query = select(self.Model).offset(skip)
# Only apply load options if requested
if use_load_options:
query = query.options(*self.get_load_options())
# Only apply load options if requested
if use_load_options:
query = query.options(*self.get_load_options())
# Add project filter if applicable
query = self._add_project_filter(query)
# Add project filter if applicable
query = self._add_project_filter(query)
if limit:
query = query.limit(limit)
if limit:
query = query.limit(limit)
result = await session.execute(query)
result = await session.execute(query)
items = result.scalars().all()
logger.debug(f"Found {len(items)} {self.Model.__name__} records")
return items
items = result.scalars().all()
logger.debug(f"Found {len(items)} {self.Model.__name__} records")
return items
async def find_by_id(self, session: AsyncSession, entity_id: Any) -> Optional[T]:
async def find_by_id(self, entity_id: int) -> Optional[T]:
"""Fetch an entity by its unique identifier."""
logger.debug(f"Finding {self.Model.__name__} by ID: {entity_id}")
return await self.select_by_id(session, entity_id)
async def find_by_ids(self, session: AsyncSession, ids: List[Any]) -> Sequence[T]:
async with db.scoped_session(self.session_maker) as session:
return await self.select_by_id(session, entity_id)
async def find_by_ids(self, ids: List[int]) -> Sequence[T]:
"""Fetch multiple entities by their identifiers in a single query."""
logger.debug(f"Finding {self.Model.__name__} by IDs: {ids}")
return await self.select_by_ids(session, ids)
async def find_one(self, session: AsyncSession, query: Select[tuple[T]]) -> Optional[T]:
async with db.scoped_session(self.session_maker) as session:
return await self.select_by_ids(session, ids)
async def find_one(self, query: Select[tuple[T]]) -> Optional[T]:
"""Execute a query and retrieve a single record."""
# add in load options
query = query.options(*self.get_load_options())
result = await self.execute_query(session, query)
result = await self.execute_query(query)
entity = result.scalars().one_or_none()
if entity:
@@ -221,42 +227,12 @@ class Repository[T: Base]:
logger.trace(f"No {self.Model.__name__} found")
return entity
async def create(self, session: AsyncSession, data: dict[str, Any]) -> T:
async def create(self, data: dict) -> T:
"""Create a new record from a model instance."""
logger.debug(f"Creating {self.Model.__name__} from entity_data: {data}")
# Only include valid columns that are provided in entity_data
model_data = self.get_model_data(data)
# Add project_id if applicable and not already provided
if self.has_project_id and self.project_id is not None and "project_id" not in model_data:
model_data["project_id"] = self.project_id
model = self.Model(**model_data)
session.add(model)
await session.flush()
return_instance = await self.select_by_id(session, model.id) # pyright: ignore [reportAttributeAccessIssue]
if return_instance is None: # pragma: no cover
logger.error(
"Failed to retrieve model after create",
model_type=self.Model.__name__,
model_id=model.id, # pyright: ignore
)
raise ValueError(
f"Can't find {self.Model.__name__} with ID {model.id} after session.add" # pyright: ignore
)
return return_instance
async def create_all(
self, session: AsyncSession, data_list: List[dict[str, Any]]
) -> Sequence[T]:
"""Create multiple records in a single transaction."""
logger.debug(f"Bulk creating {len(data_list)} {self.Model.__name__} instances")
# Only include valid columns that are provided in entity_data
model_list = []
for d in data_list:
model_data = self.get_model_data(d)
async with db.scoped_session(self.session_maker) as session:
# Only include valid columns that are provided in entity_data
model_data = self.get_model_data(data)
# Add project_id if applicable and not already provided
if (
@@ -264,126 +240,169 @@ class Repository[T: Base]:
and self.project_id is not None
and "project_id" not in model_data
):
model_data["project_id"] = self.project_id # pragma: no cover
model_data["project_id"] = self.project_id
model_list.append(self.Model(**model_data))
model = self.Model(**model_data)
session.add(model)
await session.flush()
session.add_all(model_list)
await session.flush()
return_instance = await self.select_by_id(session, model.id) # pyright: ignore [reportAttributeAccessIssue]
if return_instance is None: # pragma: no cover
logger.error(
"Failed to retrieve model after create",
model_type=self.Model.__name__,
model_id=model.id, # pyright: ignore
)
raise ValueError(
f"Can't find {self.Model.__name__} with ID {model.id} after session.add" # pyright: ignore
)
return return_instance
return await self.select_by_ids(session, [model.id for model in model_list]) # pyright: ignore [reportAttributeAccessIssue]
async def create_all(self, data_list: List[dict]) -> Sequence[T]:
"""Create multiple records in a single transaction."""
logger.debug(f"Bulk creating {len(data_list)} {self.Model.__name__} instances")
async def update(
self,
session: AsyncSession,
entity_id: Any,
entity_data: dict[str, Any] | T,
) -> Optional[T]:
async with db.scoped_session(self.session_maker) as session:
# Only include valid columns that are provided in entity_data
model_list = []
for d in data_list:
model_data = self.get_model_data(d)
# Add project_id if applicable and not already provided
if (
self.has_project_id
and self.project_id is not None
and "project_id" not in model_data
):
model_data["project_id"] = self.project_id # pragma: no cover
model_list.append(self.Model(**model_data))
session.add_all(model_list)
await session.flush()
return await self.select_by_ids(session, [model.id for model in model_list]) # pyright: ignore [reportAttributeAccessIssue]
async def update(self, entity_id: int, entity_data: dict[str, Any] | T) -> Optional[T]:
"""Update an entity with the given data."""
logger.debug(f"Updating {self.Model.__name__} {entity_id} with data: {entity_data}")
entity = await self.select_by_id(session, entity_id)
if entity is None:
logger.debug(f"No {self.Model.__name__} found to update: {entity_id}")
return None
async with db.scoped_session(self.session_maker) as session:
try:
result = await session.execute(
select(self.Model).filter(self.primary_key == entity_id)
)
entity = result.scalars().one()
if isinstance(entity_data, dict):
update_data = cast(dict[str, Any], entity_data)
for key in self.valid_columns:
if key in update_data:
setattr(entity, key, update_data[key])
if isinstance(entity_data, dict):
update_data = cast(dict[str, Any], entity_data)
for key in self.valid_columns:
if key in update_data:
setattr(entity, key, update_data[key])
elif isinstance(entity_data, self.Model):
for column in self.valid_columns:
setattr(entity, column, getattr(entity_data, column))
elif isinstance(entity_data, self.Model):
for column in self.valid_columns:
setattr(entity, column, getattr(entity_data, column))
await session.flush()
await session.refresh(entity)
await session.flush() # Make sure changes are flushed
await session.refresh(entity) # Refresh
logger.debug(f"Updated {self.Model.__name__}: {entity_id}")
return await self.select_by_id(session, entity.id) # pyright: ignore [reportAttributeAccessIssue]
logger.debug(f"Updated {self.Model.__name__}: {entity_id}")
return await self.select_by_id(session, entity.id) # pyright: ignore [reportAttributeAccessIssue]
async def update_fields(
self, session: AsyncSession, entity_id: Any, entity_data: dict[str, Any]
) -> bool:
except NoResultFound:
logger.debug(f"No {self.Model.__name__} found to update: {entity_id}")
return None
async def update_fields(self, entity_id: Any, entity_data: dict[str, Any]) -> bool:
"""Update columns without reloading the model graph afterward."""
update_data = {k: v for k, v in entity_data.items() if k in self.valid_columns}
if not update_data:
return True
conditions = [self.primary_key == entity_id]
if self.has_project_id and self.project_id is not None:
conditions.append(getattr(self.Model, "project_id") == self.project_id)
async with db.scoped_session(self.session_maker) as session:
conditions = [self.primary_key == entity_id]
if self.has_project_id and self.project_id is not None:
conditions.append(getattr(self.Model, "project_id") == self.project_id)
result = cast(
CursorResult[Any],
await session.execute(
sqlalchemy_update(self.Model).where(and_(*conditions)).values(**update_data)
),
)
return result.rowcount > 0
result = cast(
CursorResult[Any],
await session.execute(
sqlalchemy_update(self.Model).where(and_(*conditions)).values(**update_data)
),
)
return result.rowcount > 0
async def delete(self, session: AsyncSession, entity_id: int) -> bool:
async def delete(self, entity_id: int) -> bool:
"""Delete an entity from the database."""
logger.debug(f"Deleting {self.Model.__name__}: {entity_id}")
entity = await self.select_by_id(session, entity_id)
if entity is None:
logger.debug(f"No {self.Model.__name__} found to delete: {entity_id}")
return False
async with db.scoped_session(self.session_maker) as session:
try:
result = await session.execute(
select(self.Model).filter(self.primary_key == entity_id)
)
entity = result.scalars().one()
await session.delete(entity)
await session.delete(entity)
await session.flush()
logger.debug(f"Deleted {self.Model.__name__}: {entity_id}")
return True
except NoResultFound:
logger.debug(f"No {self.Model.__name__} found to delete: {entity_id}")
return False
logger.debug(f"Deleted {self.Model.__name__}: {entity_id}")
return True
async def delete_by_ids(self, session: AsyncSession, ids: List[int]) -> int:
async def delete_by_ids(self, ids: List[int]) -> int:
"""Delete records matching given IDs."""
logger.debug(f"Deleting {self.Model.__name__} by ids: {ids}")
conditions = [self.primary_key.in_(ids)]
async with db.scoped_session(self.session_maker) as session:
conditions = [self.primary_key.in_(ids)]
# Add project_id filter if applicable
if self.has_project_id and self.project_id is not None: # pragma: no cover
conditions.append(getattr(self.Model, "project_id") == self.project_id)
# Add project_id filter if applicable
if self.has_project_id and self.project_id is not None: # pragma: no cover
conditions.append(getattr(self.Model, "project_id") == self.project_id)
query = delete(self.Model).where(and_(*conditions))
result = cast(CursorResult[Any], await session.execute(query))
logger.debug(f"Deleted {result.rowcount} records")
return result.rowcount
query = delete(self.Model).where(and_(*conditions))
result = cast(CursorResult[Any], await session.execute(query))
logger.debug(f"Deleted {result.rowcount} records")
return result.rowcount
async def delete_by_fields(self, session: AsyncSession, **filters: Any) -> bool:
async def delete_by_fields(self, **filters: Any) -> bool:
"""Delete records matching given field values."""
logger.debug(f"Deleting {self.Model.__name__} by fields: {filters}")
conditions = [getattr(self.Model, field) == value for field, value in filters.items()]
async with db.scoped_session(self.session_maker) as session:
conditions = [getattr(self.Model, field) == value for field, value in filters.items()]
# Add project_id filter if applicable
if self.has_project_id and self.project_id is not None:
conditions.append(getattr(self.Model, "project_id") == self.project_id)
# Add project_id filter if applicable
if self.has_project_id and self.project_id is not None:
conditions.append(getattr(self.Model, "project_id") == self.project_id)
query = delete(self.Model).where(and_(*conditions))
result = cast(CursorResult[Any], await session.execute(query))
deleted = result.rowcount > 0
logger.debug(f"Deleted {result.rowcount} records")
return deleted
query = delete(self.Model).where(and_(*conditions))
result = cast(CursorResult[Any], await session.execute(query))
deleted = result.rowcount > 0
logger.debug(f"Deleted {result.rowcount} records")
return deleted
async def count(self, session: AsyncSession, query: Executable | None = None) -> int:
async def count(self, query: Executable | None = None) -> int:
"""Count entities in the database table."""
if query is None:
query = select(func.count()).select_from(self.Model)
# Add project filter if applicable
if isinstance(query, Select) and self.has_project_id and self.project_id is not None:
query = query.where(
getattr(self.Model, "project_id") == self.project_id
) # pragma: no cover
async with db.scoped_session(self.session_maker) as session:
if query is None:
query = select(func.count()).select_from(self.Model)
# Add project filter if applicable
if (
isinstance(query, Select)
and self.has_project_id
and self.project_id is not None
):
query = query.where(
getattr(self.Model, "project_id") == self.project_id
) # pragma: no cover
result = await session.execute(query)
scalar = result.scalar()
count = scalar if scalar is not None else 0
logger.debug(f"Counted {count} {self.Model.__name__} records")
return count
result = await session.execute(query)
scalar = result.scalar()
count = scalar if scalar is not None else 0
logger.debug(f"Counted {count} {self.Model.__name__} records")
return count
async def execute_query(
self,
session: AsyncSession,
query: Executable,
params: Optional[Dict[str, Any]] = None,
use_query_options: bool = True,
@@ -392,8 +411,9 @@ class Repository[T: Base]:
query = query.options(*self.get_load_options()) if use_query_options else query
logger.trace(f"Executing query: {query}, params: {params}")
result = await session.execute(query, params)
return result
async with db.scoped_session(self.session_maker) as session:
result = await session.execute(query, params)
return result
def get_load_options(self) -> List[LoaderOption]:
"""Get list of loader options for eager loading relationships.
@@ -36,7 +36,6 @@ class SearchIndexRow:
category: Optional[str] = None # observations
from_id: Optional[int] = None # relations
to_id: Optional[int] = None # relations
to_name: Optional[str] = None # relations: literal target text, set even when unresolved
relation_type: Optional[str] = None # relations
# Matched chunk text from vector search (the actual content that matched the query)
@@ -95,7 +94,6 @@ class SearchIndexRow:
else self.metadata,
"from_id": self.from_id,
"to_id": self.to_id,
"to_name": self.to_name,
"relation_type": self.relation_type,
"entity_id": self.entity_id,
"category": self.category,
@@ -48,8 +48,6 @@ class SearchRepository(Protocol):
min_similarity: Optional[float] = None,
limit: int = 10,
offset: int = 0,
allow_relaxed: bool = False,
session: AsyncSession | None = None,
) -> List[SearchIndexRow]:
"""Search across indexed content."""
...
@@ -40,55 +40,6 @@ BULLET_PATTERN = re.compile(r"^[\-\*]\s+")
OVERSIZED_ENTITY_VECTOR_SHARD_SIZE = 256
_SQLITE_MAX_PREPARE_WINDOW = 8
# Interrogative/function words contribute lexical noise when a strict
# full-text query is relaxed: "when OR did OR a" matches loud wrong documents
# that displace genuine results from the ranking window.
RELAXATION_STOPWORDS = frozenset(
"a an and are as at be but by did do does for from had has have how i in is it of on "
"or that the their they this to was we were what when where which who whom whose why "
"will with you your".split()
)
def relaxed_query_words(search_text: Optional[str]) -> Optional[list[str]]:
"""Content-bearing words for OR-relaxing a strict full-text query.
Returns None when relaxation must not apply. These eligibility rules match
SearchService._is_relaxed_fts_fallback_eligible so the hybrid FTS branch
relaxes exactly the same query shapes as the service-level FTS path:
- empty / quoted / explicit-boolean queries (user intent is not
second-guessed);
- fewer than three alphanumeric tokens (short queries like "New Feature"
over-broaden under OR and in hybrid the relaxed FTS-only rows normalize
to 1.0 and can outrank the vector result the user wanted);
- any pure-digit token ("root note 1", "SPEC 16") identifier-like queries
over-broaden and create false positives under OR.
"""
if not search_text:
return None
stripped = search_text.strip()
if '"' in stripped or any(op in f" {stripped} " for op in (" AND ", " OR ", " NOT ")):
return None
# Eligibility checks run on raw alphanumeric tokens (parity with the
# service), before stopword filtering.
tokens = re.findall(r"[A-Za-z0-9]+", stripped.lower())
if len(tokens) < 3 or any(token.isdigit() for token in tokens):
return None
words = [word.strip("?!.,;:") for word in stripped.split()]
words = [
word
for word in words
if word and word.isalnum() and word.lower() not in RELAXATION_STOPWORDS
]
return words or None
# Entity, observation, and relation rows in search_index carry ids from independent
# auto-increment sequences, so a bare id is ambiguous across row types. Every map in
# the vector/hybrid retrieval path must key rows by (type, id) to avoid collisions.
type SearchIndexKey = tuple[str, int]
@dataclass
class VectorSyncBatchResult:
@@ -273,7 +224,6 @@ class SearchRepositoryBase(ABC):
min_similarity: Optional[float] = None,
limit: int = 10,
offset: int = 0,
allow_relaxed: bool = False,
) -> List[SearchIndexRow]:
"""Search across all indexed content.
@@ -1907,7 +1857,7 @@ class SearchRepositoryBase(ABC):
# ------------------------------------------------------------------
@staticmethod
def _parse_chunk_key(chunk_key: str) -> SearchIndexKey:
def _parse_chunk_key(chunk_key: str) -> tuple[str, int]:
"""Parse a chunk_key like 'observation:5:0' into (type, search_index_id)."""
parts = chunk_key.split(":")
return parts[0], int(parts[1])
@@ -1982,27 +1932,26 @@ class SearchRepositoryBase(ABC):
hydrate_start = time.perf_counter()
# Build per-search_index_row similarity scores from chunk-level results.
# Each chunk_key encodes the search_index row type and id; keep both as the
# key because different row types can share the same numeric id (#982).
# Each chunk_key encodes the search_index row type and id.
# Track the best similarity per row (for ranking) and all chunks (for context).
similarity_by_si_key: dict[SearchIndexKey, float] = {}
chunks_by_si_key: dict[SearchIndexKey, list[tuple[float, str]]] = {}
similarity_by_si_id: dict[int, float] = {}
chunks_by_si_id: dict[int, list[tuple[float, str]]] = {}
for row in vector_rows:
chunk_key = row.get("chunk_key", "")
distance = float(row["best_distance"])
similarity = self._distance_to_similarity(distance)
chunk_text = row.get("chunk_text", "")
try:
si_key = self._parse_chunk_key(chunk_key)
_, si_id = self._parse_chunk_key(chunk_key)
except (ValueError, IndexError):
# Fallback: group by entity_id for chunks without parseable keys
continue
current = similarity_by_si_key.get(si_key)
current = similarity_by_si_id.get(si_id)
if current is None or similarity > current:
similarity_by_si_key[si_key] = similarity
chunks_by_si_key.setdefault(si_key, []).append((similarity, chunk_text))
similarity_by_si_id[si_id] = similarity
chunks_by_si_id.setdefault(si_id, []).append((similarity, chunk_text))
if not similarity_by_si_key:
if not similarity_by_si_id:
hydrate_ms = (time.perf_counter() - hydrate_start) * 1000
_log_vector_summary()
return []
@@ -2013,17 +1962,16 @@ class SearchRepositoryBase(ABC):
min_similarity if min_similarity is not None else self._semantic_min_similarity
)
if effective_min_similarity > 0.0:
similarity_by_si_key = {
k: v for k, v in similarity_by_si_key.items() if v >= effective_min_similarity
similarity_by_si_id = {
k: v for k, v in similarity_by_si_id.items() if v >= effective_min_similarity
}
if not similarity_by_si_key:
if not similarity_by_si_id:
hydrate_ms = (time.perf_counter() - hydrate_start) * 1000
_log_vector_summary()
return []
# Fetch the actual search_index rows. Colliding (type, id) keys share one
# bare id, so deduplicate while preserving first-seen order.
si_ids = list(dict.fromkeys(si_id for _, si_id in similarity_by_si_key))
# Fetch the actual search_index rows
si_ids = list(similarity_by_si_id.keys())
search_index_rows = await self._fetch_search_index_rows_by_ids(si_ids)
# Apply optional filters if requested
@@ -2055,14 +2003,16 @@ class SearchRepositoryBase(ABC):
limit=VECTOR_FILTER_SCAN_LIMIT,
offset=0,
)
# Use (type, id) tuples to avoid collisions between different
# Use (id, type) tuples to avoid collisions between different
# search_index row types that share the same auto-increment id.
allowed_keys = {(row.type, row.id) for row in filtered_rows if row.id is not None}
search_index_rows = {k: v for k, v in search_index_rows.items() if k in allowed_keys}
allowed_keys = {(row.id, row.type) for row in filtered_rows if row.id is not None}
search_index_rows = {
k: v for k, v in search_index_rows.items() if (v.id, v.type) in allowed_keys
}
ranked_rows: list[SearchIndexRow] = []
for si_key, similarity in similarity_by_si_key.items():
row = search_index_rows.get(si_key)
for si_id, similarity in similarity_by_si_id.items():
row = search_index_rows.get(si_id)
if row is None:
continue
@@ -2072,7 +2022,7 @@ class SearchRepositoryBase(ABC):
if content_snippet and len(content_snippet) <= SMALL_NOTE_CONTENT_LIMIT:
matched_chunk_text = content_snippet
else:
si_chunks = chunks_by_si_key.get(si_key, [])
si_chunks = chunks_by_si_id.get(si_id, [])
si_chunks.sort(key=lambda c: c[0], reverse=True)
top_texts = [text for _, text in si_chunks[:TOP_CHUNKS_PER_RESULT]]
matched_chunk_text = "\n---\n".join(top_texts) if top_texts else None
@@ -2138,12 +2088,8 @@ class SearchRepositoryBase(ABC):
async def _fetch_search_index_rows_by_ids(
self, row_ids: list[int]
) -> dict[SearchIndexKey, SearchIndexRow]:
"""Fetch search_index rows by id, keyed by (type, id) to disambiguate types.
A bare id can match one row per type (independent id sequences), so the
result must carry every matching row rather than letting one clobber another.
"""
) -> dict[int, SearchIndexRow]:
"""Fetch search_index rows by their primary key (id), any type."""
if not row_ids:
return {}
placeholders = ",".join(f":id_{idx}" for idx in range(len(row_ids)))
@@ -2160,11 +2106,11 @@ class SearchRepositoryBase(ABC):
WHERE project_id = :project_id
AND id IN ({placeholders})
"""
result: dict[SearchIndexKey, SearchIndexRow] = {}
result: dict[int, SearchIndexRow] = {}
async with db.scoped_session(self.session_maker) as session:
row_result = await session.execute(text(sql), params)
for row in row_result.fetchall():
result[(row.type, row.id)] = SearchIndexRow(
result[row.id] = SearchIndexRow(
project_id=self.project_id,
id=row.id,
title=row.title,
@@ -2210,7 +2156,7 @@ class SearchRepositoryBase(ABC):
) -> List[SearchIndexRow]:
"""Fuse FTS and vector results using score-based fusion.
Uses the search_index (type, id) pair as the fusion key. The formula
Uses search_index row id as the fusion key. The formula
``max(vec, fts) + FUSION_BONUS * min(vec, fts)`` preserves
the dominant signal and rewards dual-source agreement.
"""
@@ -2219,9 +2165,6 @@ class SearchRepositoryBase(ABC):
query_start = time.perf_counter()
candidate_limit = max(self._semantic_vector_k, (limit + offset) * 10)
fts_start = time.perf_counter()
# allow_relaxed: question-form queries rarely AND-match, and a dead FTS
# branch silently degrades hybrid to vector-only ranking. Fusion plus
# bm25 keep relaxed lexical candidates from dominating precision.
fts_results = await self.search(
search_text=search_text,
permalink=permalink,
@@ -2235,7 +2178,6 @@ class SearchRepositoryBase(ABC):
retrieval_mode=SearchRetrievalMode.FTS,
limit=candidate_limit,
offset=0,
allow_relaxed=True,
)
fts_ms = (time.perf_counter() - fts_start) * 1000
vector_start = time.perf_counter()
@@ -2257,19 +2199,17 @@ class SearchRepositoryBase(ABC):
vector_ms = (time.perf_counter() - vector_start) * 1000
fusion_start = time.perf_counter()
# --- Score-based fusion keyed on (type, id) ---
# A bare row id collides across row types (independent id sequences), so
# fusion must key on (type, id) or distinct rows would merge (#982).
# --- Score-based fusion keyed on search_index row id ---
# FTS scores are normalized to [0, 1] (BM25 is unbounded).
# Vector scores are used raw — already calibrated [0, 1] by _distance_to_similarity().
rows_by_key: dict[SearchIndexKey, SearchIndexRow] = {}
rows_by_id: dict[int, SearchIndexRow] = {}
# Normalize FTS scores to [0, 1] — handles both SQLite (negative bm25)
# and Postgres (positive ts_rank) by using absolute values
fts_abs = [abs(row.score or 0.0) for row in fts_results]
fts_max = max(fts_abs) if fts_abs else 1.0
fts_scores: dict[SearchIndexKey, float] = {}
fts_scores: dict[int, float] = {}
for row in fts_results:
if row.id is None:
continue
@@ -2277,32 +2217,32 @@ class SearchRepositoryBase(ABC):
# Gate: FTS scores below threshold contribute zero
if norm < FTS_GATE_THRESHOLD:
norm = 0.0
fts_scores[(row.type, row.id)] = norm
rows_by_key[(row.type, row.id)] = row
fts_scores[row.id] = norm
rows_by_id[row.id] = row
vec_scores: dict[SearchIndexKey, float] = {}
vec_scores: dict[int, float] = {}
for row in vector_results:
if row.id is None:
continue
# Trigger: no re-normalization by vec_max
# Why: vector similarity is already calibrated [0, 1]; re-normalizing
# inflates weak matches when the entire result set is mediocre
vec_scores[(row.type, row.id)] = row.score or 0.0
rows_by_key[(row.type, row.id)] = row
vec_scores[row.id] = row.score or 0.0
rows_by_id[row.id] = row
# Fuse: max(v, f) + FUSION_BONUS * min(v, f)
# Preserves the dominant signal; bonus rewards dual-source agreement.
# Output range: [0, 1.3] for dual-source, [0, 1.0] for single-source.
fused_scores: dict[SearchIndexKey, float] = {}
for row_key in fts_scores.keys() | vec_scores.keys():
v = vec_scores.get(row_key, 0.0)
f = fts_scores.get(row_key, 0.0)
fused_scores[row_key] = max(v, f) + FUSION_BONUS * min(v, f)
fused_scores: dict[int, float] = {}
for row_id in fts_scores.keys() | vec_scores.keys():
v = vec_scores.get(row_id, 0.0)
f = fts_scores.get(row_id, 0.0)
fused_scores[row_id] = max(v, f) + FUSION_BONUS * min(v, f)
ranked = sorted(fused_scores.items(), key=lambda item: item[1], reverse=True)
output: list[SearchIndexRow] = []
for row_key, fused_score in ranked[offset : offset + limit]:
row = rows_by_key[row_key]
for row_id, fused_score in ranked[offset : offset + limit]:
row = rows_by_id[row_id]
# Trigger: FTS-only results have no matched_chunk_text from vector search.
# Why: without chunk text, API falls back to truncated content, losing answer text.
# Outcome: FTS-only results get full content_snippet as matched_chunk.
@@ -23,10 +23,7 @@ from basic_memory.models.search import (
from basic_memory.repository.embedding_provider import EmbeddingProvider
from basic_memory.repository.embedding_provider_factory import create_embedding_provider
from basic_memory.repository.search_index_row import SearchIndexRow
from basic_memory.repository.search_repository_base import (
SearchRepositoryBase,
relaxed_query_words,
)
from basic_memory.repository.search_repository_base import SearchRepositoryBase
from basic_memory.repository.metadata_filters import parse_metadata_filters, build_sqlite_json_path
from basic_memory.repository.semantic_errors import SemanticDependenciesMissingError
from basic_memory.schemas.search import SearchItemType, SearchRetrievalMode
@@ -258,19 +255,6 @@ class SQLiteSearchRepository(SearchRepositoryBase):
if "*" in term and all(c.isalnum() or c in "*_-" for c in term):
return term
# Natural-language queries arrive with sentence punctuation that FTS5
# treats as syntax ("When did Melanie paint a sunrise?"). The tokenizer
# ignores this punctuation in the INDEX, so stripping it from word
# edges loses nothing — but leaving it forces the whole question into
# an exact-phrase match that returns zero rows, silently disabling the
# FTS half of hybrid search. Interior characters (hyphens, slashes —
# permalinks and paths) are untouched.
if " " in term:
words = [word.strip("?!.,;:") for word in term.split()]
term = " ".join(word for word in words if word)
if not term:
return ""
# Characters that can cause FTS5 syntax errors when used as operators
# We're more conservative here - only quote when we detect problematic patterns
problematic_chars = [
@@ -367,14 +351,6 @@ class SQLiteSearchRepository(SearchRepositoryBase):
# For non-Boolean queries, use the single term preparation logic
return self._prepare_single_term(term, is_prefix)
@staticmethod
def _relaxed_fts_text(search_text: Optional[str]) -> Optional[str]:
"""OR-relaxed FTS5 expression for a failed strict query, or None."""
words = relaxed_query_words(search_text)
if not words:
return None
return " OR ".join(f"{word}*" for word in words)
# ------------------------------------------------------------------
# sqlite-vec extension loading (SQLite-specific)
# ------------------------------------------------------------------
@@ -977,16 +953,8 @@ class SQLiteSearchRepository(SearchRepositoryBase):
min_similarity: Optional[float] = None,
limit: int = 10,
offset: int = 0,
allow_relaxed: bool = False,
session: AsyncSession | None = None,
) -> List[SearchIndexRow]:
"""Search across all indexed content using SQLite FTS5.
``allow_relaxed=True`` retries a zero-result strict multi-word query
with OR-joined content terms. Only the hybrid path opts in: its FTS
branch otherwise contributes nothing for question-form queries.
Service-level FTS searches keep their own conservative fallback.
"""
"""Search across all indexed content using SQLite FTS5."""
# --- Dispatch vector / hybrid modes (shared logic) ---
dispatched = await self._dispatch_retrieval_mode(
search_text=search_text,
@@ -1049,31 +1017,10 @@ class SQLiteSearchRepository(SearchRepositoryBase):
"""
logger.trace(f"Search {sql} params: {params}")
async def run_search(active_session: AsyncSession):
result = await active_session.execute(text(sql), params)
rows = result.fetchall()
# Trigger: multi-word natural-language query matched nothing
# under the default all-terms-AND semantics.
# Why: questions ("when did X do Y") rarely have every word in
# one document; without relaxation the FTS half of hybrid
# search contributes zero candidates and ranking degrades to
# vector-only.
# Outcome: one retry with OR-joined prefix terms; bm25 still
# ranks multi-term matches first.
relaxed = self._relaxed_fts_text(search_text) if allow_relaxed and not rows else None
if relaxed and params.get("text"):
params["text"] = relaxed
result = await active_session.execute(text(sql), params)
rows = result.fetchall()
return rows
try:
if session is not None:
rows = await run_search(session)
else:
async with db.scoped_session(self.session_maker) as owned_session:
rows = await run_search(owned_session)
async with db.scoped_session(self.session_maker) as session:
result = await session.execute(text(sql), params)
rows = result.fetchall()
except Exception as e:
# Handle FTS5 syntax errors and provide user-friendly feedback
if self._is_fts5_syntax_error(e): # pragma: no cover
+1 -4
View File
@@ -19,6 +19,7 @@ from pathlib import Path
from typing import List, Optional, Annotated, Dict
from annotated_types import MinLen, MaxLen
from dateparser import parse
from pydantic import BaseModel, BeforeValidator, Field, model_validator, computed_field
@@ -91,10 +92,6 @@ def parse_timeframe(timeframe: str) -> datetime:
parse_timeframe('1d') -> 2025-06-04 14:50:00-07:00 (24 hours ago with local timezone)
parse_timeframe('1 week ago') -> 2025-05-29 14:50:00-07:00 (1 week ago with local timezone)
"""
# Deferred: dateparser costs ~0.13s to import; schemas load on every CLI
# start, but timeframe parsing only happens per request (#886).
from dateparser import parse
if timeframe.lower() == "today":
# For "today", return 1 day ago to ensure we capture recent activity across timezones
# This handles the case where client and server are in different timezones
-3
View File
@@ -155,9 +155,6 @@ class RelationSummary(BaseModel):
from_entity_id: Optional[int] = None
from_entity_external_id: Optional[str] = None
to_entity: Optional[str] = None
# Literal target text from the markdown; present even when the relation is
# an unresolved forward reference (to_entity is None until the target exists)
to_name: Optional[str] = None
to_entity_id: Optional[int] = None
to_entity_external_id: Optional[str] = None
created_at: Annotated[
+9 -63
View File
@@ -2,18 +2,15 @@
from __future__ import annotations
from contextlib import asynccontextmanager
from dataclasses import dataclass, field
from datetime import datetime, timezone
from typing import Any, AsyncIterator, List, Optional, Tuple, TYPE_CHECKING
from typing import Any, List, Optional, Tuple, TYPE_CHECKING
from loguru import logger
from sqlalchemy import text
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
import logfire
from basic_memory import db
from basic_memory.repository.entity_repository import EntityRepository
from basic_memory.repository.observation_repository import ObservationRepository
from basic_memory.repository.postgres_search_repository import PostgresSearchRepository
@@ -21,7 +18,6 @@ from basic_memory.repository.search_repository import SearchRepository, SearchIn
from basic_memory.schemas.memory import MemoryUrl, memory_url_path
from basic_memory.schemas.search import SearchItemType
from basic_memory.utils import generate_permalink
from basic_memory.workspace_context import workspace_slug_for_canonical_permalinks
if TYPE_CHECKING:
from basic_memory.services.link_resolver import LinkResolver
@@ -40,7 +36,6 @@ class ContextResultRow:
from_id: Optional[int] = None
to_id: Optional[int] = None
relation_type: Optional[str] = None
to_name: Optional[str] = None
content: Optional[str] = None
category: Optional[str] = None
entity_id: Optional[int] = None
@@ -94,21 +89,11 @@ class ContextService:
entity_repository: EntityRepository,
observation_repository: ObservationRepository,
link_resolver: Optional[LinkResolver] = None,
session_maker: async_sessionmaker[AsyncSession] | None = None,
):
self.search_repository = search_repository
self.entity_repository = entity_repository
self.observation_repository = observation_repository
self.link_resolver = link_resolver
self.session_maker = session_maker
@asynccontextmanager
async def _session_scope(self) -> AsyncIterator[AsyncSession]:
"""Open a service-owned transaction for core repository reads."""
if self.session_maker is None: # pragma: no cover
raise ValueError("session_maker is required for ContextService")
async with db.scoped_session(self.session_maker) as session:
yield session
async def build_context(
self,
@@ -158,29 +143,6 @@ class ContextService:
primary = await self.search_repository.search(
permalink_match=normalized_path, limit=fetch_limit, offset=offset
)
# Trigger: a workspace-qualified pattern matched nothing while a
# workspace permalink context is active.
# Why: rows written before workspace canonicalization (or via
# clients that didn't forward workspace headers) store
# project-qualified permalinks; a workspace-prefixed pattern
# can never match those legacy rows (#957).
# Outcome: retry once with the workspace prefix stripped so the
# pattern matches the index form the rows actually carry.
if not primary:
workspace_slug = workspace_slug_for_canonical_permalinks()
ws_prefix = f"{workspace_slug}/" if workspace_slug else None
if ws_prefix and normalized_path.startswith(ws_prefix):
fallback_path = normalized_path.removeprefix(ws_prefix)
logger.debug(
f"Pattern search fallback without workspace prefix: "
f"'{fallback_path}'"
)
primary = await self.search_repository.search(
permalink_match=fallback_path,
limit=fetch_limit,
offset=offset,
)
else:
normalized_path = generate_permalink(path, split_extension=False)
logger.debug(f"Direct lookup for '{normalized_path}'")
@@ -189,13 +151,9 @@ class ContextService:
)
if not primary and self.link_resolver:
async with self._session_scope() as session:
entity = await self.link_resolver.resolve_link(
path,
use_search=True,
strict=False,
session=session,
)
entity = await self.link_resolver.resolve_link(
path, use_search=True, strict=False
)
if entity:
logger.debug(
f"LinkResolver resolved '{path}' to permalink '{entity.permalink}'"
@@ -251,10 +209,9 @@ class ContextService:
phase="load_observations",
result_count=len(entity_ids),
):
async with self._session_scope() as session:
observations_by_entity = await self.observation_repository.find_by_entities(
session, entity_ids
)
observations_by_entity = await self.observation_repository.find_by_entities(
entity_ids
)
logger.debug(f"Found observations for {len(observations_by_entity)} entities")
metadata = ContextMetadata(
@@ -435,7 +392,6 @@ class ContextService:
from_id=row.from_id,
to_id=row.to_id,
relation_type=row.relation_type,
to_name=row.to_name,
content=row.content,
category=row.category,
entity_id=row.entity_id,
@@ -470,7 +426,6 @@ class ContextService:
CAST(NULL AS INTEGER) as from_id,
CAST(NULL AS INTEGER) as to_id,
CAST(NULL AS TEXT) as relation_type,
CAST(NULL AS TEXT) as to_name,
CAST(NULL AS TEXT) as content,
CAST(NULL AS TEXT) as category,
CAST(NULL AS INTEGER) as entity_id,
@@ -523,10 +478,6 @@ class ContextService:
WHEN step_type = 1 THEN r.relation_type
ELSE NULL
END as relation_type,
CASE
WHEN step_type = 1 THEN r.to_name
ELSE NULL
END as to_name,
CAST(NULL AS TEXT) as content,
CAST(NULL AS TEXT) as category,
CAST(NULL AS INTEGER) as entity_id,
@@ -590,7 +541,6 @@ class ContextService:
from_id,
to_id,
relation_type,
to_name,
content,
category,
entity_id,
@@ -600,7 +550,7 @@ class ContextService:
FROM entity_graph
WHERE depth > 0
GROUP BY type, id, title, permalink, file_path, from_id, to_id,
relation_type, to_name, content, category, entity_id, root_id, created_at
relation_type, content, category, entity_id, root_id, created_at
ORDER BY depth, type, id
LIMIT :max_results
""")
@@ -629,7 +579,6 @@ class ContextService:
NULL as from_id,
NULL as to_id,
NULL as relation_type,
NULL as to_name,
NULL as content,
NULL as category,
NULL as entity_id,
@@ -657,7 +606,6 @@ class ContextService:
r.from_id,
r.to_id,
r.relation_type,
r.to_name,
NULL as content,
NULL as category,
NULL as entity_id,
@@ -696,7 +644,6 @@ class ContextService:
NULL as from_id,
NULL as to_id,
NULL as relation_type,
NULL as to_name,
NULL as content,
NULL as category,
NULL as entity_id,
@@ -730,7 +677,6 @@ class ContextService:
from_id,
to_id,
relation_type,
to_name,
content,
category,
entity_id,
@@ -740,7 +686,7 @@ class ContextService:
FROM entity_graph
WHERE depth > 0
GROUP BY type, id, title, permalink, file_path, from_id, to_id,
relation_type, to_name, content, category, entity_id, root_id, created_at
relation_type, content, category, entity_id, root_id, created_at
ORDER BY depth, type, id
LIMIT :max_results
""")

Some files were not shown because too many files have changed in this diff Show More