mirror of
https://github.com/basicmachines-co/basic-memory
synced 2026-06-21 13:47:35 +00:00
Compare commits
39 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 2d42ba16e3 | |||
| 82d42268c8 | |||
| fef6f61ffa | |||
| a6ad20a0b7 | |||
| bb65be817b | |||
| dcb4fe927f | |||
| b5f13d6903 | |||
| 6d06c4a4e7 | |||
| 16869867da | |||
| 18da6194f9 | |||
| a6d0784335 | |||
| 232f469065 | |||
| 5a08cfd9ac | |||
| b997d858cd | |||
| 598965c389 | |||
| 8dd6451dfe | |||
| d46c68806e | |||
| 1d9ff3c94f | |||
| 515b2c8365 | |||
| 32a1c208b3 | |||
| 25732b2fe2 | |||
| b92b0340d5 | |||
| 0247ef0ead | |||
| 253e240d68 | |||
| b3bdd5914f | |||
| 85c701b8c2 | |||
| 1ad3a350ad | |||
| a148e72f56 | |||
| 33e741fd29 | |||
| c9770375e9 | |||
| 0811c48252 | |||
| 747e64e5e9 | |||
| be00df27c7 | |||
| 3ce42de57e | |||
| a143072d35 | |||
| 497a4e0a43 | |||
| ec5fac8d76 | |||
| 05bfd0f04e | |||
| 338f357f55 |
@@ -6,14 +6,14 @@
|
||||
},
|
||||
"metadata": {
|
||||
"description": "Official Basic Memory plugins from the canonical basic-memory repository",
|
||||
"version": "0.22.0"
|
||||
"version": "0.22.1"
|
||||
},
|
||||
"plugins": [
|
||||
{
|
||||
"name": "basic-memory",
|
||||
"source": "./plugins/claude-code",
|
||||
"description": "The bridge between Claude's working memory and Basic Memory's durable knowledge graph \u2014 session briefings, pre-compaction checkpoints, and capture reflexes",
|
||||
"version": "0.22.0",
|
||||
"version": "0.22.1",
|
||||
"author": {
|
||||
"name": "Basic Machines"
|
||||
},
|
||||
|
||||
@@ -28,7 +28,10 @@ You are an expert release manager for the Basic Memory project. When the user ru
|
||||
|
||||
#### Documentation Validation
|
||||
1. **Changelog Check**
|
||||
- CHANGELOG.md contains entry for target version
|
||||
- CHANGELOG.md contains entry for target version **already landed on `main`**
|
||||
(main only accepts changes via PR, so the changelog entry must go through
|
||||
its own PR before running the release; the recipe pre-flight-checks for a
|
||||
`## vX.Y.Z` heading)
|
||||
- Entry includes all major features and fixes
|
||||
- Breaking changes are documented
|
||||
|
||||
@@ -41,11 +44,15 @@ just release <version>
|
||||
The justfile target handles:
|
||||
- ✅ Version format validation
|
||||
- ✅ Git status and branch checks
|
||||
- ✅ Quality checks (`just check` - lint, format, type-check, tests)
|
||||
- ✅ Changelog entry check (must already be on `main`)
|
||||
- ✅ Quality checks (`just lint` + `just typecheck`)
|
||||
- ✅ Version update across all consolidated manifests via `just set-version` (Python
|
||||
package + Claude Code plugin/marketplaces + Codex plugin + Hermes + OpenClaw)
|
||||
- ✅ Automatic commit with proper message
|
||||
- ✅ Tag creation and pushing to GitHub
|
||||
- ✅ Release PR: commits the bump on a `release/vX.Y.Z` branch, opens a PR
|
||||
(`chore(core): release vX.Y.Z`), and rebase-merges it — the `main` ruleset
|
||||
rejects direct pushes and the repo disallows merge commits
|
||||
- ✅ Tags the rebased bump commit on `main` (found by commit subject, since
|
||||
the rebase rewrites the SHA) and pushes the tag
|
||||
- ✅ Release workflow trigger (automatic on tag push)
|
||||
|
||||
The GitHub Actions workflow (`.github/workflows/release.yml`) then:
|
||||
@@ -89,7 +96,7 @@ After PyPI release is published, update the MCP registry:
|
||||
|
||||
2. **Publish to MCP Registry**
|
||||
```bash
|
||||
cd /Users/drew/code/basic-memory
|
||||
# from the basic-memory repo root
|
||||
mcp-publisher publish
|
||||
```
|
||||
|
||||
@@ -109,43 +116,50 @@ After PyPI release is published, update the MCP registry:
|
||||
|
||||
#### Website Updates
|
||||
|
||||
**1. basicmachines.co** (`/Users/drew/code/basicmachines.co`)
|
||||
- **Goal**: Update version number displayed on the homepage
|
||||
- **Location**: Search for "Basic Memory v0." in the codebase to find version displays
|
||||
- **What to update**:
|
||||
- Hero section heading that shows "Basic Memory v{VERSION}"
|
||||
- "What's New in v{VERSION}" section heading
|
||||
- Feature highlights array (look for array of features with title/description)
|
||||
- **Process**:
|
||||
**1. basicmemory.com** (sibling `basicmemory.com` repo —
|
||||
`basicmachines-co/basicmemory.com`, formerly `basicmachines.co`)
|
||||
- **No version bump needed.** The marketing site is an Astro + React app and
|
||||
carries **no hardcoded Basic Memory version number** anywhere in its UI
|
||||
(`hero.tsx` and the rest of the site have no version string). The old
|
||||
instruction to bump `src/components/sections/hero.tsx` is obsolete — that
|
||||
file no longer holds a version. Release announcements are dated blog posts,
|
||||
not an in-place edit.
|
||||
- **Skip entirely for patch releases.**
|
||||
- **Significant releases only — optional announcement post**:
|
||||
1. Pull latest from GitHub: `git pull origin main`
|
||||
2. Create release branch: `git checkout -b release/v{VERSION}`
|
||||
3. Search codebase for current version number (e.g., "v0.16.1")
|
||||
4. Update version numbers to new release version
|
||||
5. Update feature highlights with 3-5 key features from this release (extract from CHANGELOG.md)
|
||||
6. Commit changes: `git commit -m "chore: update to v{VERSION}"`
|
||||
7. Push branch: `git push origin release/v{VERSION}`
|
||||
- **Deploy**: Follow deployment process for basicmachines.co
|
||||
3. Add a dated post under `src/content/blog/` modeled on an existing
|
||||
release post (e.g. `basic-memory-v0-19-0-release.md`), summarizing 3–5
|
||||
headline features from `CHANGELOG.md`
|
||||
4. Commit (`git commit -s -m "..."`), push, and open a PR against
|
||||
`basicmachines-co/basicmemory.com`
|
||||
- **Deploy**: follow that repo's deployment process.
|
||||
|
||||
**2. docs.basicmemory.com** (`/Users/drew/code/docs.basicmemory.com`)
|
||||
- **Goal**: Add new release notes section to the latest-releases page
|
||||
- **File**: `src/pages/latest-releases.mdx`
|
||||
**2. docs.basicmemory.com** (sibling `docs.basicmemory.com` repo)
|
||||
- **Goal**: Add a What's New page for the release and bump the homepage badge
|
||||
- **Site shape**: Nuxt/Docus content site. The changelog page
|
||||
(`content/2.whats-new/*.changelog.md`) auto-fetches GitHub releases — no
|
||||
manual changelog update needed. See that repo's CLAUDE.md "Version Bump
|
||||
Checklist".
|
||||
- **What to do**:
|
||||
1. Pull latest from GitHub: `git pull origin main`
|
||||
2. Create release branch: `git checkout -b release/v{VERSION}`
|
||||
3. Read the existing file to understand the format and structure
|
||||
4. Read `/Users/drew/code/basic-memory/CHANGELOG.md` to get release content
|
||||
5. Add new release section **at the top** (after MDX imports, before other releases)
|
||||
6. Follow the existing pattern:
|
||||
- Heading: `## [v{VERSION}](github-link) — YYYY-MM-DD`
|
||||
- Focus statement if applicable
|
||||
- `<Info>` block with highlights (3-5 key items)
|
||||
- Sections for Features, Bug Fixes, Breaking Changes, etc.
|
||||
- Link to full changelog at the end
|
||||
- Separator `---` between releases
|
||||
7. Commit changes: `git commit -m "docs: add v{VERSION} release notes"`
|
||||
8. Push branch: `git push origin release/v{VERSION}`
|
||||
- **Source content**: Extract and format sections from CHANGELOG.md for this version
|
||||
- **Deploy**: Follow deployment process for docs.basicmemory.com
|
||||
3. Read `CHANGELOG.md` in the `basic-memory` repo to get release content
|
||||
4. **New minor/major release**: add `content/2.whats-new/1.v{VERSION}.md`
|
||||
modeled on the previous version page (frontmatter title/description,
|
||||
headline feature first, then sections, then an Upgrading note) and
|
||||
renumber the existing what's-new pages down one slot (URLs don't
|
||||
change — Nuxt strips the numeric prefixes)
|
||||
5. **Patch release**: append a short note to the current version's page
|
||||
instead of creating a new one
|
||||
6. Update the homepage version badge in `content/index.md` (the
|
||||
`v0.XX →` button text and its `to: /whats-new/v{VERSION}` link)
|
||||
7. If the release adds user-facing features, update the relevant guide
|
||||
and reference pages (`content/3.cloud/`, `content/9.reference/`)
|
||||
8. Commit: `git commit -s -m "docs: add v{VERSION} release notes"`
|
||||
9. Push branch and open a PR; merge after the release is tagged
|
||||
- **Deploy**: push to main auto-deploys to development; production requires
|
||||
manual workflow dispatch via GitHub Actions
|
||||
|
||||
**4. Announce Release**
|
||||
- Post to Discord community if significant changes
|
||||
|
||||
@@ -300,7 +300,9 @@ See SPEC-16 for full context manager refactor details.
|
||||
|
||||
### Release Process
|
||||
|
||||
Releases are driven by `just release` / `just beta` — never by a bare `git tag`. The recipes bump version metadata, run pre-flight checks, commit, tag, and push. GitHub Actions then publishes to PyPI and updates the Homebrew formula.
|
||||
Releases are driven by `just release` / `just beta` — never by a bare `git tag`. The recipes bump version metadata, run pre-flight checks, land the bump on `main` through a release PR, tag, and push the tag. GitHub Actions then publishes to PyPI and updates the Homebrew formula.
|
||||
|
||||
**Main requires PRs.** The `main` ruleset rejects direct pushes ("Changes must be made through a pull request") and the repo disallows merge commits, so the recipes push a `release/vX.Y.Z` branch, open a PR titled `chore(core): release vX.Y.Z`, rebase-merge it with `gh pr merge --rebase`, then tag the rebased bump commit on `main` (located by its commit subject, since rebasing rewrites the SHA) and push the tag. The CHANGELOG entry for the version must already be on `main` — land it via a normal PR before running the recipe (it pre-flight-checks for a `## vX.Y.Z` heading).
|
||||
|
||||
**Stable release:**
|
||||
|
||||
@@ -308,7 +310,7 @@ Releases are driven by `just release` / `just beta` — never by a bare `git tag
|
||||
just release v0.21.3
|
||||
```
|
||||
|
||||
The recipe runs `just lint` + `just typecheck`, then updates every release manifest through `scripts/update_versions.py`: `src/basic_memory/__init__.py`, `server.json`, the root Claude marketplace, the Claude Code plugin manifest and local marketplace, the Hermes `plugin.yaml`, and the OpenClaw `package.json`. It commits as `chore: update version to X.Y.Z for vX.Y.Z release`, creates the `vX.Y.Z` tag, and pushes both the commit and the tag to `origin/main`. After the tag lands, the `Release` workflow builds the Python package, publishes to PyPI, creates the GitHub release with auto-generated notes, publishes the OpenClaw npm package, and updates the Homebrew formula. The recipe finishes by printing the post-release tasks the workflow doesn't cover.
|
||||
The recipe runs `just lint` + `just typecheck`, then updates every release manifest through `scripts/update_versions.py`: `src/basic_memory/__init__.py`, `server.json`, the root Claude marketplace, the Claude Code plugin manifest and local marketplace, the Hermes `plugin.yaml`, and the OpenClaw `package.json`. It commits as `chore: update version to X.Y.Z for vX.Y.Z release` on a `release/vX.Y.Z` branch, lands it on `main` via a rebase-merged PR, then tags the rebased commit and pushes the tag. After the tag lands, the `Release` workflow builds the Python package, publishes to PyPI, creates the GitHub release with auto-generated notes, publishes the OpenClaw npm package, and updates the Homebrew formula. The recipe finishes by printing the post-release tasks the workflow doesn't cover.
|
||||
|
||||
**Beta release:** `just beta v0.21.3b1` — same flow with a beta-suffixed tag. PyPI consumers install with `pip install basic-memory --pre`.
|
||||
|
||||
@@ -319,8 +321,13 @@ The recipe runs `just lint` + `just typecheck`, then updates every release manif
|
||||
**Do not tag releases by hand.** A bare `git tag vX.Y.Z` skips the in-code version bump. Package metadata is still correct (uv-dynamic-versioning derives it from the git tag) but `basic-memory --version` reports the previous release, which is what happened with v0.21.2 → v0.21.3.
|
||||
|
||||
**Post-release tasks** the recipe surfaces but doesn't run:
|
||||
- `docs.basicmemory.com` — add notes to `src/pages/latest-releases.mdx`
|
||||
- `basicmachines.co` — bump version in `src/components/sections/hero.tsx`
|
||||
- `docs.basicmemory.com` — add a What's New page under `content/2.whats-new/` and bump the version badge in `content/index.md` (the changelog page auto-fetches GitHub releases; see that repo's CLAUDE.md version-bump checklist)
|
||||
- `basicmemory.com` — the marketing site (Astro + React, repo
|
||||
`basicmachines-co/basicmemory.com`, formerly `basicmachines.co`) carries **no
|
||||
hardcoded version number** in its UI, so there is nothing to bump. For a
|
||||
significant release, optionally add a dated announcement post under
|
||||
`src/content/blog/` (model it on an existing `basic-memory-vX-Y-Z-release.md`).
|
||||
Skip entirely for routine patch releases.
|
||||
- MCP Registry — `mcp-publisher publish` from the repo root
|
||||
|
||||
See `.claude/commands/release/release.md` (and `beta.md`, `release-check.md`, `changelog.md` alongside it) for the full release + post-release runbook, including the slash commands.
|
||||
|
||||
@@ -1,5 +1,49 @@
|
||||
# CHANGELOG
|
||||
|
||||
## v0.22.1 (2026-06-12)
|
||||
|
||||
Follow-up patch to v0.22.0. Fixes project and default-project resolution on
|
||||
fresh installs, MCP workspace routing, sync project selection, and CLI startup
|
||||
latency, plus a few MCP parity additions.
|
||||
|
||||
### Features
|
||||
|
||||
- Added a `workspace` parameter to `write_note` for parity with `edit_note`.
|
||||
- **#826**: Added `title` and `tags` annotations to all MCP tool decorators
|
||||
(phase 1).
|
||||
- **#930**: `search_notes` now comma-splits `note_types`, `entity_types`, and
|
||||
`categories`.
|
||||
- **#971**: Added the manual-pages flow — manpage seed schema, flow docs, and
|
||||
verification fixes.
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
- Fresh installs no longer fail when the projects table is empty: resolve now
|
||||
points them at project setup, the first project is promoted to default when
|
||||
the config default is missing from the database, the promoted default state
|
||||
is returned from the project-create API, and a default can be set when none
|
||||
is currently set. An existing database default is preserved when repairing a
|
||||
missing config default.
|
||||
- **#949**: Sync skips projects without an absolute local path and excludes
|
||||
orphan DB projects that are absent from config.
|
||||
- **#952 / #981**: Resolved workspace display names and tenant ids in qualified
|
||||
project routes, closing out the manual verification findings.
|
||||
- `note_types`/`entity_types`/`categories` are normalized on the direct-call
|
||||
path, with non-string list elements rejected.
|
||||
- Vector-search hydration keys on `(type, id)` to prevent id collisions.
|
||||
- `file_utils` requires line-anchored frontmatter fences.
|
||||
- CLI startup is faster: FastAPI and app imports are deferred out of CLI
|
||||
startup, and rich/typer modules are preloaded before an in-place upgrade.
|
||||
- `config.json` is written atomically.
|
||||
- In-memory SQLite sessions are serialized so concurrent rollbacks cannot
|
||||
destroy writes.
|
||||
|
||||
### Maintenance
|
||||
|
||||
- Release recipes route through PRs and wait for the release PR merge to land
|
||||
before tagging; the release runbook is refreshed and stripped of
|
||||
user-specific absolute paths.
|
||||
|
||||
## v0.22.0 (2026-06-11)
|
||||
|
||||
Team-safe cloud sync. New additive `bm cloud push` and `bm cloud pull`
|
||||
|
||||
@@ -6,6 +6,7 @@
|
||||
[](https://github.com/astral-sh/ruff)
|
||||

|
||||

|
||||
[](https://deepwiki.com/basicmachines-co/basic-memory)
|
||||
|
||||
## Skip the install — try Basic Memory in the cloud
|
||||
|
||||
@@ -590,7 +591,7 @@ retention).
|
||||
| `BASIC_MEMORY_IMPORT_UPLOAD_MAX_BYTES` | `104857600` | Max uploaded import size |
|
||||
|
||||
```bash
|
||||
BASIC_MEMORY_LOG_LEVEL=DEBUG basic-memory sync
|
||||
BASIC_MEMORY_LOG_LEVEL=DEBUG basic-memory reindex
|
||||
tail -f ~/.basic-memory/basic-memory.log
|
||||
```
|
||||
|
||||
|
||||
+3
-3
@@ -111,7 +111,7 @@ You can run Basic Memory CLI commands inside the container using `docker exec`:
|
||||
docker exec basic-memory-server basic-memory status
|
||||
|
||||
# Sync files
|
||||
docker exec basic-memory-server basic-memory sync
|
||||
docker exec basic-memory-server basic-memory reindex
|
||||
|
||||
# Show help
|
||||
docker exec basic-memory-server basic-memory --help
|
||||
@@ -137,7 +137,7 @@ When using Docker volumes, you'll need to configure projects to point to your mo
|
||||
|
||||
3. **Sync the new project:**
|
||||
```bash
|
||||
docker exec basic-memory-server basic-memory sync
|
||||
docker exec basic-memory-server basic-memory reindex
|
||||
```
|
||||
|
||||
### Example: Setting up an Obsidian Vault
|
||||
@@ -157,7 +157,7 @@ docker exec basic-memory-server basic-memory project create obsidian /app/data
|
||||
docker exec basic-memory-server basic-memory project set-default obsidian
|
||||
|
||||
# Sync to index all files
|
||||
docker exec basic-memory-server basic-memory sync
|
||||
docker exec basic-memory-server basic-memory reindex
|
||||
```
|
||||
|
||||
### Environment Variables
|
||||
|
||||
@@ -184,8 +184,8 @@ finance/ (lowercase f)
|
||||
Use Basic Memory's built-in conflict detection:
|
||||
|
||||
```bash
|
||||
# Sync will report conflicts
|
||||
basic-memory sync
|
||||
# Index local file changes (conflicts are handled during the scan)
|
||||
basic-memory reindex
|
||||
|
||||
# Check sync status for warnings
|
||||
basic-memory status
|
||||
|
||||
@@ -0,0 +1,183 @@
|
||||
# Manual Pages
|
||||
|
||||
Basic Memory's manual is written in the style of Unix man pages — and
|
||||
implemented as Basic Memory notes ([#952](https://github.com/basicmachines-co/basic-memory/issues/952)).
|
||||
Every page is a markdown note conforming to the `Manpage` schema, `SEE ALSO`
|
||||
entries are real knowledge-graph relations, and every example on every page
|
||||
was executed against a live project before the page shipped. The manual
|
||||
documents the tools; the tools verify the manual.
|
||||
|
||||
## Where it lives
|
||||
|
||||
The canonical manual is the **`manual` project in the Basic Memory team
|
||||
workspace** (cloud, shared). Anyone can build their own: the schema ships as
|
||||
an opt-in seed at `plugins/claude-code/schemas/manpage.md` — copy it into any
|
||||
project's folder and start writing pages against it.
|
||||
|
||||
Layout:
|
||||
|
||||
```
|
||||
manual/
|
||||
├── schemas/Manpage.md # the manpage schema (type: schema)
|
||||
├── man1/ # CLI commands bm(1), bm-status(1), ...
|
||||
├── man3/ # MCP tools write-note(3), search-notes(3), ...
|
||||
├── man5/ # file formats bm-note(5), bm-observation(5), ...
|
||||
├── man7/ # concepts basic-memory(7), semantic-memory(7), ...
|
||||
├── playground/ # scratch notes for destructive examples
|
||||
└── diagrams/ # canvas visualizations of the manual graph
|
||||
```
|
||||
|
||||
### Why "man1", "man3", "man5"?
|
||||
|
||||
The folder names are Unix's, unchanged since 1971. The manual is divided
|
||||
into numbered **sections**, pages physically live in directories named
|
||||
after them (`/usr/share/man/man1`, `man5`, ...), and the number tells you
|
||||
what *kind* of thing is documented — not importance, not reading order:
|
||||
|
||||
- **1** — user commands (`ls`, `grep`)
|
||||
- **2** — system calls
|
||||
- **3** — library functions / APIs (`printf(3)`)
|
||||
- **4** — devices
|
||||
- **5** — file formats and config files (`crontab(5)`, `passwd(5)`)
|
||||
- **6** — games (really)
|
||||
- **7** — miscellanea: concepts, conventions, overviews (`regex(7)`, `signal(7)`)
|
||||
- **8** — system administration
|
||||
|
||||
That's also why man page names carry the parenthesized number —
|
||||
`crontab(1)` is the command, `crontab(5)` is the file format, same name in
|
||||
two sections. `man 5 crontab` picks the section explicitly.
|
||||
|
||||
This manual copies that layout with the sections that have a Basic Memory
|
||||
analog:
|
||||
|
||||
- **man1/** — `bm` CLI commands → `bm-status(1)`
|
||||
- **man3/** — MCP tools, our equivalent of the "library API" section → `write-note(3)`
|
||||
- **man5/** — file formats: note syntax, observations, relations, schemas → `bm-note(5)`
|
||||
- **man7/** — concepts → `basic-memory(7)`, `semantic-memory(7)`
|
||||
- **8** is reserved for admin/cloud operations but has no pages yet; 2, 4,
|
||||
and 6 have no analog (no system calls, no devices, and no games — yet)
|
||||
|
||||
When a page says `see_also [[bm-note(5)]]`, the `(5)` reads "the
|
||||
file-format page," exactly the way a Unix manual cross-references — except
|
||||
here it's a traversable relation in the graph instead of a typographic
|
||||
convention. The manual explains its own conventions in `man-pages(7)` —
|
||||
fittingly, the same page name Linux uses for this, and that almost nobody
|
||||
ever reads.
|
||||
|
||||
## Page anatomy
|
||||
|
||||
Pages use the classic headers where applicable: `NAME`, `SYNOPSIS`,
|
||||
`DESCRIPTION`, `PARAMETERS`, `MCP USAGE`, `CLI EQUIVALENT`, `EXAMPLES`,
|
||||
`GOTCHAS`, `SEE ALSO`. Frontmatter (validated by the schema):
|
||||
|
||||
```yaml
|
||||
type: manpage
|
||||
section: 3 # 1 | 3 | 5 | 7 | 8
|
||||
name: write-note # page name without section suffix
|
||||
summary: create or overwrite a markdown note in the knowledge base
|
||||
generated: hand # hand | registry | typer (regeneration ownership)
|
||||
tool: write_note # section-3 pages: the MCP tool documented
|
||||
command: basic-memory status # section-1 pages: the CLI command documented
|
||||
verified: 0.21.6 mcp+cli # version + path(s) that proved the page
|
||||
```
|
||||
|
||||
Field knowledge accumulates as observations — `[gotcha]`, `[bug]` (with issue
|
||||
links), `[pattern]` — and `SEE ALSO` entries are `see_also` relations, so the
|
||||
manual is a navigable graph, not a folder of files.
|
||||
|
||||
## How to use it
|
||||
|
||||
Man-style reads (any MCP client or the CLI):
|
||||
|
||||
```bash
|
||||
# read a page
|
||||
bm tool read-note "man3/write-note-3" --project manual
|
||||
|
||||
# apropos — find pages by section, tool, or text
|
||||
bm tool search-notes --project manual # then filter, or via MCP:
|
||||
# search_notes(project="manual", metadata_filters={"type": "manpage", "section": 3})
|
||||
# search_notes(project="manual", metadata_filters={"type": "manpage", "tool": "write_note"})
|
||||
|
||||
# traverse SEE ALSO from any page
|
||||
# build_context(url="man3/write-note-3", project="manual")
|
||||
```
|
||||
|
||||
A future `bm man <topic>` command is thin sugar over exactly these calls.
|
||||
|
||||
And for the real thing — `man bm` in an actual terminal:
|
||||
|
||||
```bash
|
||||
bm man install # copies bundled groff pages to ~/.local/share/man
|
||||
man bm # the overview page, rendered by man(1)
|
||||
man basic-memory # same page via its alias
|
||||
```
|
||||
|
||||
`bm man install` warns with a one-line `MANPATH` fix if the install root
|
||||
isn't searched by your `man`. Agents with shell access can use `man bm` as
|
||||
an offline quick reference; the full per-tool detail stays in the manual
|
||||
project's section-3 pages.
|
||||
|
||||
## The verification discipline
|
||||
|
||||
Two rules make the manual trustworthy:
|
||||
|
||||
1. **Examples must have run.** An `EXAMPLES` (or `MCP USAGE` / `CLI
|
||||
EQUIVALENT`) block contains only commands that actually executed against
|
||||
the manual project. Destructive operations (`delete_note`, `move_note`,
|
||||
destructive `edit_note`) run only against `playground/` notes — never
|
||||
against pages. The `verified:` field records the version and which path
|
||||
proved the page: `mcp` (live service), `cli` (dev checkout), or both.
|
||||
|
||||
2. **The schema is the linter.** Validate the whole manual any time:
|
||||
|
||||
```bash
|
||||
bm tool schema-validate manpage --project manual
|
||||
# → {"total_notes": 38, "valid_count": 38, "warning_count": 0, ...}
|
||||
```
|
||||
|
||||
`bm orphans --project manual` confirms every page is connected to the
|
||||
graph, and `schema_diff`/`schema_infer` report drift between the schema
|
||||
and how pages are actually written.
|
||||
|
||||
Because verification exercises real tool calls against the live service,
|
||||
building the manual doubles as an end-to-end smoke test. The initial build
|
||||
found six bugs in one pass (#954–#959) — including the verification rule
|
||||
catching a test that asserted a bug as expected output (#958).
|
||||
|
||||
## Adding or updating a page
|
||||
|
||||
1. Run the commands you intend to document; keep the actual output.
|
||||
2. Write the page with `write_note`, passing frontmatter through the
|
||||
`metadata` parameter (nested YAML in content frontmatter is unreliable on
|
||||
some clients):
|
||||
|
||||
```
|
||||
write_note(title="my-tool(3)", directory="man3", project="manual",
|
||||
note_type="manpage",
|
||||
metadata={"section": 3, "name": "my-tool",
|
||||
"summary": "...", "generated": "hand",
|
||||
"tool": "my_tool", "verified": "<version> mcp"})
|
||||
```
|
||||
|
||||
3. Link related pages in `SEE ALSO` with `see_also [[other-page(3)]]`.
|
||||
Forward references to pages that don't exist yet are fine — they resolve
|
||||
automatically when the target is written.
|
||||
4. Validate: `bm tool schema-validate manpage --project manual`.
|
||||
|
||||
For mechanical updates to generated sections, prefer `edit_note` with
|
||||
`replace_section` / `insert_after_section` so curated content (EXAMPLES,
|
||||
GOTCHAS, SEE ALSO, observations) survives — that ownership split is what the
|
||||
`generated:` field declares.
|
||||
|
||||
## Roadmap
|
||||
|
||||
- **Registry generator** — section-3 SYNOPSIS/PARAMETERS generated from the
|
||||
MCP tool registry (docstrings + pydantic schemas), section-1 from Typer
|
||||
help; the hand-written corpus is the template spec. Regenerate-and-diff in
|
||||
CI becomes the drift gate.
|
||||
- **`bm man <topic>`** — CLI sugar over `read_note` + metadata search.
|
||||
(`bm man install` + a hand-written `bm.1` already ship — the first slice
|
||||
of [#610](https://github.com/basicmachines-co/basic-memory/issues/610);
|
||||
the generator will produce per-command pages from the same extraction.)
|
||||
- **Docs site** — the notes remain canonical for sections 5 and 7, code is
|
||||
canonical for 1 and 3; both render to the hosted docs site.
|
||||
@@ -38,7 +38,7 @@ from typing import Any, Callable
|
||||
from agent.memory_provider import MemoryProvider
|
||||
from tools.registry import tool_error
|
||||
|
||||
__version__ = "0.22.0"
|
||||
__version__ = "0.22.1"
|
||||
|
||||
logger = logging.getLogger("hermes.memory.basic-memory")
|
||||
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
name: basic-memory
|
||||
version: 0.22.0
|
||||
version: 0.22.1
|
||||
description: "Basic Memory — persistent knowledge graph backed by the basic-memory MCP server"
|
||||
pip_dependencies:
|
||||
- mcp
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "@basicmemory/openclaw-basic-memory",
|
||||
"version": "0.22.0",
|
||||
"version": "0.22.1",
|
||||
"type": "module",
|
||||
"main": "./dist/index.js",
|
||||
"types": "./dist/index.d.ts",
|
||||
|
||||
@@ -383,17 +383,31 @@ release version:
|
||||
echo "❌ Tag {{version}} already exists"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
|
||||
# Changelog must already be on main (land it via a normal PR first)
|
||||
if ! grep -q "^## {{version}} " CHANGELOG.md; then
|
||||
echo "❌ CHANGELOG.md has no entry for {{version}}. Land one via PR first."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Run quality checks
|
||||
echo "🔍 Running lint checks..."
|
||||
just lint
|
||||
just typecheck
|
||||
|
||||
|
||||
# Update all package manifests to the one Basic Memory product version.
|
||||
echo "📝 Updating consolidated package versions..."
|
||||
just set-version "{{version}}"
|
||||
|
||||
# Commit version update
|
||||
# Trigger: main's ruleset rejects direct pushes ("Changes must be made
|
||||
# through a pull request").
|
||||
# Why: the version bump must land on main before the tag is cut, so it
|
||||
# rides a release PR that is rebase-merged (the repo disallows merge
|
||||
# commits).
|
||||
# Outcome: the bump commit gets a new SHA on main; the tag is created on
|
||||
# that rebased commit, found by its commit subject.
|
||||
COMMIT_SUBJECT="chore: update version to $VERSION_NUM for {{version}} release"
|
||||
git checkout -b "release/{{version}}"
|
||||
git add \
|
||||
src/basic_memory/__init__.py \
|
||||
server.json \
|
||||
@@ -404,23 +418,58 @@ release version:
|
||||
integrations/hermes/plugin.yaml \
|
||||
integrations/hermes/__init__.py \
|
||||
integrations/openclaw/package.json
|
||||
git commit -s -m "chore: update version to $VERSION_NUM for {{version}} release"
|
||||
|
||||
# Create and push tag
|
||||
echo "🏷️ Creating tag {{version}}..."
|
||||
git tag "{{version}}"
|
||||
|
||||
echo "📤 Pushing to GitHub..."
|
||||
git push origin main
|
||||
git commit -s -m "$COMMIT_SUBJECT"
|
||||
|
||||
echo "📤 Opening release PR..."
|
||||
git push -u origin "release/{{version}}"
|
||||
gh pr create --title "chore(core): release {{version}}" \
|
||||
--body "Version bump for {{version}}. See CHANGELOG.md for release notes."
|
||||
|
||||
# Trigger: the PR may not be mergeable synchronously (merge gates,
|
||||
# required checks added later, or GitHub still computing mergeability).
|
||||
# Why: the tag must point at the bump commit on main, so the recipe
|
||||
# cannot tag until the merge has actually landed.
|
||||
# Outcome: try a direct rebase-merge, fall back to queueing auto-merge,
|
||||
# then poll main for the rebased bump commit before tagging.
|
||||
if ! gh pr merge "release/{{version}}" --rebase --delete-branch; then
|
||||
echo "⚠️ Direct merge did not complete (merge gates pending?). Queueing auto-merge..."
|
||||
gh pr merge "release/{{version}}" --rebase --delete-branch --auto
|
||||
fi
|
||||
|
||||
echo "⏳ Waiting for the bump commit to land on main..."
|
||||
TAG_COMMIT=""
|
||||
for _ in $(seq 1 60); do
|
||||
git fetch origin main --quiet
|
||||
TAG_COMMIT=$(git log FETCH_HEAD --fixed-strings --grep "$COMMIT_SUBJECT" --format='%H' -1)
|
||||
[[ -n "$TAG_COMMIT" ]] && break
|
||||
sleep 5
|
||||
done
|
||||
if [[ -z "$TAG_COMMIT" ]]; then
|
||||
echo "❌ Bump commit not on main after 5 minutes (merge still pending?)."
|
||||
echo " Once the release PR merges, finish the release manually:"
|
||||
echo " git fetch origin main"
|
||||
echo " git tag {{version}} \$(git log FETCH_HEAD --fixed-strings --grep \"$COMMIT_SUBJECT\" --format='%H' -1)"
|
||||
echo " git push origin {{version}}"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
git checkout main
|
||||
git pull --ff-only origin main
|
||||
git branch -D "release/{{version}}" 2>/dev/null || true
|
||||
|
||||
echo "🏷️ Creating tag {{version}} at $TAG_COMMIT..."
|
||||
git tag "{{version}}" "$TAG_COMMIT"
|
||||
git push origin "{{version}}"
|
||||
|
||||
|
||||
echo "✅ Release {{version}} created successfully!"
|
||||
echo "📦 GitHub Actions will build and publish to PyPI"
|
||||
echo "🔗 Monitor at: https://github.com/basicmachines-co/basic-memory/actions"
|
||||
echo ""
|
||||
echo "📝 REMINDER: Post-release tasks:"
|
||||
echo " 1. docs.basicmemory.com - Add release notes to src/pages/latest-releases.mdx"
|
||||
echo " 2. basicmachines.co - Update version in src/components/sections/hero.tsx"
|
||||
echo " 1. docs.basicmemory.com - Add a What's New page under content/2.whats-new/"
|
||||
echo " and bump the badge in content/index.md (see that repo's CLAUDE.md)"
|
||||
echo " 2. basicmemory.com - No version number in the site UI; for a significant"
|
||||
echo " release optionally add a post under src/content/blog/. Skip for patches."
|
||||
echo " 3. MCP Registry - Run: mcp-publisher publish"
|
||||
echo " See: .claude/commands/release/release.md for detailed instructions"
|
||||
|
||||
@@ -467,7 +516,15 @@ beta version:
|
||||
echo "📝 Updating consolidated package versions..."
|
||||
just set-version "{{version}}"
|
||||
|
||||
# Commit version update
|
||||
# Trigger: main's ruleset rejects direct pushes ("Changes must be made
|
||||
# through a pull request").
|
||||
# Why: the version bump must land on main before the tag is cut, so it
|
||||
# rides a release PR that is rebase-merged (the repo disallows merge
|
||||
# commits).
|
||||
# Outcome: the bump commit gets a new SHA on main; the tag is created on
|
||||
# that rebased commit, found by its commit subject.
|
||||
COMMIT_SUBJECT="chore: update version to $VERSION_NUM for {{version}} beta release"
|
||||
git checkout -b "release/{{version}}"
|
||||
git add \
|
||||
src/basic_memory/__init__.py \
|
||||
server.json \
|
||||
@@ -478,24 +535,59 @@ beta version:
|
||||
integrations/hermes/plugin.yaml \
|
||||
integrations/hermes/__init__.py \
|
||||
integrations/openclaw/package.json
|
||||
git commit -s -m "chore: update version to $VERSION_NUM for {{version}} beta release"
|
||||
|
||||
# Create and push tag
|
||||
echo "🏷️ Creating tag {{version}}..."
|
||||
git tag "{{version}}"
|
||||
|
||||
echo "📤 Pushing to GitHub..."
|
||||
git push origin main
|
||||
git commit -s -m "$COMMIT_SUBJECT"
|
||||
|
||||
echo "📤 Opening release PR..."
|
||||
git push -u origin "release/{{version}}"
|
||||
gh pr create --title "chore(core): release {{version}}" \
|
||||
--body "Version bump for {{version}} beta."
|
||||
|
||||
# Trigger: the PR may not be mergeable synchronously (merge gates,
|
||||
# required checks added later, or GitHub still computing mergeability).
|
||||
# Why: the tag must point at the bump commit on main, so the recipe
|
||||
# cannot tag until the merge has actually landed.
|
||||
# Outcome: try a direct rebase-merge, fall back to queueing auto-merge,
|
||||
# then poll main for the rebased bump commit before tagging.
|
||||
if ! gh pr merge "release/{{version}}" --rebase --delete-branch; then
|
||||
echo "⚠️ Direct merge did not complete (merge gates pending?). Queueing auto-merge..."
|
||||
gh pr merge "release/{{version}}" --rebase --delete-branch --auto
|
||||
fi
|
||||
|
||||
echo "⏳ Waiting for the bump commit to land on main..."
|
||||
TAG_COMMIT=""
|
||||
for _ in $(seq 1 60); do
|
||||
git fetch origin main --quiet
|
||||
TAG_COMMIT=$(git log FETCH_HEAD --fixed-strings --grep "$COMMIT_SUBJECT" --format='%H' -1)
|
||||
[[ -n "$TAG_COMMIT" ]] && break
|
||||
sleep 5
|
||||
done
|
||||
if [[ -z "$TAG_COMMIT" ]]; then
|
||||
echo "❌ Bump commit not on main after 5 minutes (merge still pending?)."
|
||||
echo " Once the release PR merges, finish the release manually:"
|
||||
echo " git fetch origin main"
|
||||
echo " git tag {{version}} \$(git log FETCH_HEAD --fixed-strings --grep \"$COMMIT_SUBJECT\" --format='%H' -1)"
|
||||
echo " git push origin {{version}}"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
git checkout main
|
||||
git pull --ff-only origin main
|
||||
git branch -D "release/{{version}}" 2>/dev/null || true
|
||||
|
||||
echo "🏷️ Creating tag {{version}} at $TAG_COMMIT..."
|
||||
git tag "{{version}}" "$TAG_COMMIT"
|
||||
git push origin "{{version}}"
|
||||
|
||||
|
||||
echo "✅ Beta release {{version}} created successfully!"
|
||||
echo "📦 GitHub Actions will build and publish to PyPI as pre-release"
|
||||
echo "🔗 Monitor at: https://github.com/basicmachines-co/basic-memory/actions"
|
||||
echo "📥 Install with: uv tool install basic-memory --pre"
|
||||
echo ""
|
||||
echo "📝 REMINDER: For stable releases, update documentation sites:"
|
||||
echo " 1. docs.basicmemory.com - Add release notes to src/pages/latest-releases.mdx"
|
||||
echo " 2. basicmachines.co - Update version in src/components/sections/hero.tsx"
|
||||
echo " 1. docs.basicmemory.com - Add a What's New page under content/2.whats-new/"
|
||||
echo " and bump the badge in content/index.md (see that repo's CLAUDE.md)"
|
||||
echo " 2. basicmemory.com - No version number in the site UI; for a significant"
|
||||
echo " release optionally add a post under src/content/blog/. Skip for patches."
|
||||
echo " See: .claude/commands/release/release.md for detailed instructions"
|
||||
|
||||
# List all available recipes
|
||||
|
||||
@@ -6,14 +6,14 @@
|
||||
},
|
||||
"metadata": {
|
||||
"description": "Official plugins from Basic Machines for knowledge management and AI-assisted development",
|
||||
"version": "0.22.0"
|
||||
"version": "0.22.1"
|
||||
},
|
||||
"plugins": [
|
||||
{
|
||||
"name": "basic-memory",
|
||||
"source": "./",
|
||||
"description": "The bridge between Claude's working memory and Basic Memory's durable knowledge graph \u2014 session briefings, pre-compaction checkpoints, and capture reflexes",
|
||||
"version": "0.22.0",
|
||||
"version": "0.22.1",
|
||||
"author": {
|
||||
"name": "Basic Machines"
|
||||
},
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"name": "basic-memory",
|
||||
"description": "The bridge between Claude's working memory and Basic Memory's durable knowledge graph \u2014 session briefings, pre-compaction checkpoints, and capture reflexes",
|
||||
"version": "0.22.0",
|
||||
"version": "0.22.1",
|
||||
"author": {
|
||||
"name": "Basic Machines"
|
||||
},
|
||||
|
||||
@@ -0,0 +1,55 @@
|
||||
---
|
||||
title: Manpage
|
||||
type: schema
|
||||
entity: Manpage
|
||||
version: 1
|
||||
schema:
|
||||
gotcha?(array): string, sharp edges and surprising behavior learned from live verification
|
||||
example?(array): string, worked examples beyond the generated synopsis
|
||||
pattern?(array): string, recommended idioms and usage patterns
|
||||
bug?(array): string, known defects affecting this surface, with issue links
|
||||
see_also?(array): Entity, related manual pages — the SEE ALSO graph
|
||||
settings:
|
||||
validation: warn
|
||||
frontmatter:
|
||||
section(enum, Unix manual section number): [1, 3, 5, 7, 8]
|
||||
name: string, page name without section suffix (e.g. write-note)
|
||||
summary: string, one-line NAME description
|
||||
generated?(enum, who owns the mechanical sections): [registry, typer, hand]
|
||||
tool?: string, MCP tool this page documents (section 3 pages)
|
||||
command?: string, CLI command this page documents (section 1 pages)
|
||||
verified?: string, version and path that verified this page (e.g. 0.21.6 mcp+cli)
|
||||
since?: string, version this surface first appeared
|
||||
---
|
||||
|
||||
# Manpage
|
||||
|
||||
A **ManpageNote** is one page of a Unix-style manual implemented as Basic
|
||||
Memory notes (issue #952): commands in section 1, MCP tools in section 3,
|
||||
file formats in section 5, concepts in section 7, admin in section 8. The
|
||||
manual becomes a knowledge graph — `SEE ALSO` entries are typed relations,
|
||||
and pages are found by structured recall:
|
||||
`search_notes(metadata_filters={"type": "manpage", "section": 3})`.
|
||||
|
||||
This schema is an opt-in seed for documentation projects; the canonical
|
||||
manual lives in the Basic Memory team workspace `manual` project.
|
||||
|
||||
## What makes a good ManpageNote
|
||||
|
||||
- **NAME / SYNOPSIS / DESCRIPTION** — classic man-page structure, with
|
||||
PARAMETERS, MCP USAGE, CLI EQUIVALENT, EXAMPLES, GOTCHAS, SEE ALSO where
|
||||
applicable.
|
||||
- **Verified examples** — EXAMPLES contain only commands that actually ran;
|
||||
the `verified` field records the version and path (mcp, cli, or both).
|
||||
- **generated** — declares regeneration ownership: `registry` (from the MCP
|
||||
tool registry) and `typer` (from CLI help) pages get mechanical sections
|
||||
rewritten; curated sections (EXAMPLES, GOTCHAS, SEE ALSO, observations)
|
||||
are never overwritten.
|
||||
- **gotcha / bug observations** — field knowledge accumulates on pages
|
||||
without being clobbered by regeneration; bugs link their tracking issues.
|
||||
|
||||
## Frontmatter
|
||||
|
||||
`type: manpage` plus `section` makes the manual queryable like `man -k`:
|
||||
by section, by `tool`, by `command`, or by missing/stale `verified` stamps.
|
||||
Validation is `warn`, never blocking.
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "codex",
|
||||
"version": "0.22.0",
|
||||
"version": "0.22.1",
|
||||
"description": "A Codex-native bridge to Basic Memory for durable engineering context, decisions, and resumable checkpoints.",
|
||||
"author": {
|
||||
"name": "Basic Machines",
|
||||
|
||||
+2
-2
@@ -6,12 +6,12 @@
|
||||
"url": "https://github.com/basicmachines-co/basic-memory.git",
|
||||
"source": "github"
|
||||
},
|
||||
"version": "0.22.0",
|
||||
"version": "0.22.1",
|
||||
"packages": [
|
||||
{
|
||||
"registryType": "pypi",
|
||||
"identifier": "basic-memory",
|
||||
"version": "0.22.0",
|
||||
"version": "0.22.1",
|
||||
"runtimeHint": "uvx",
|
||||
"runtimeArguments": [
|
||||
{
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
"""basic-memory - Local-first knowledge management combining Zettelkasten with knowledge graphs"""
|
||||
|
||||
# Package version - updated by release automation
|
||||
__version__ = "0.22.0"
|
||||
__version__ = "0.22.1"
|
||||
|
||||
# API version for FastAPI - independent of package version
|
||||
__api_version__ = "v0"
|
||||
|
||||
@@ -17,6 +17,7 @@ from fastapi import APIRouter, HTTPException, Response, Path
|
||||
from loguru import logger
|
||||
|
||||
import logfire
|
||||
from basic_memory import db
|
||||
from basic_memory.ignore_utils import (
|
||||
IGNORED_PATH_REJECTION_DETAIL,
|
||||
load_gitignore_patterns,
|
||||
@@ -34,6 +35,8 @@ from basic_memory.deps import (
|
||||
ProjectExternalIdPathDep,
|
||||
SyncServiceV2ExternalDep,
|
||||
TaskSchedulerDep,
|
||||
SessionDep,
|
||||
SessionMakerDep,
|
||||
)
|
||||
from basic_memory.schemas import DeleteEntitiesResponse
|
||||
from basic_memory.schemas.base import Entity
|
||||
@@ -81,6 +84,7 @@ async def get_graph(
|
||||
project_id: ProjectExternalIdPathDep,
|
||||
entity_repository: EntityRepositoryV2ExternalDep,
|
||||
relation_repository: RelationRepositoryV2ExternalDep,
|
||||
session: SessionDep,
|
||||
) -> GraphResponse:
|
||||
"""Return all entities and resolved relations for knowledge graph visualization.
|
||||
|
||||
@@ -96,7 +100,7 @@ async def get_graph(
|
||||
logger.info("API v2 request: get_graph")
|
||||
|
||||
# Fetch all entities for this project
|
||||
entities = await entity_repository.find_all(use_load_options=False)
|
||||
entities = await entity_repository.find_all(session, use_load_options=False)
|
||||
nodes = [
|
||||
GraphNode(
|
||||
external_id=entity.external_id,
|
||||
@@ -108,7 +112,7 @@ async def get_graph(
|
||||
]
|
||||
|
||||
# Fetch all resolved relations (to_id is not null) with eager-loaded entities
|
||||
relations = await relation_repository.find_all()
|
||||
relations = await relation_repository.find_all(session)
|
||||
edges = [
|
||||
GraphEdge(
|
||||
from_id=relation.from_entity.external_id,
|
||||
@@ -130,6 +134,7 @@ async def get_graph(
|
||||
async def get_orphan_entities(
|
||||
project_id: ProjectExternalIdPathDep,
|
||||
entity_repository: EntityRepositoryV2ExternalDep,
|
||||
session: SessionDep,
|
||||
) -> OrphanEntitiesResponse:
|
||||
"""Return entities that have no incoming or outgoing relations."""
|
||||
with logfire.span(
|
||||
@@ -140,7 +145,7 @@ async def get_orphan_entities(
|
||||
):
|
||||
logger.info("API v2 request: get_orphan_entities")
|
||||
|
||||
entities = await entity_repository.find_without_relations()
|
||||
entities = await entity_repository.find_without_relations(session)
|
||||
nodes = [
|
||||
GraphNode(
|
||||
external_id=entity.external_id,
|
||||
@@ -165,6 +170,7 @@ async def resolve_identifier(
|
||||
link_resolver: LinkResolverV2ExternalDep,
|
||||
entity_repository: EntityRepositoryV2ExternalDep,
|
||||
project_repository: ProjectRepositoryDep,
|
||||
session: SessionDep,
|
||||
) -> EntityResolveResponse:
|
||||
"""Resolve a string identifier (external_id, permalink, title, or path) to entity info.
|
||||
|
||||
@@ -203,12 +209,15 @@ async def resolve_identifier(
|
||||
):
|
||||
logger.info(f"API v2 request: resolve_identifier for '{data.identifier}'")
|
||||
|
||||
entity = await entity_repository.get_by_external_id(data.identifier)
|
||||
entity = await entity_repository.get_by_external_id(session, data.identifier)
|
||||
resolution_method = "external_id" if entity else "search"
|
||||
|
||||
if not entity:
|
||||
entity = await link_resolver.resolve_link(
|
||||
data.identifier, source_path=data.source_path, strict=data.strict
|
||||
data.identifier,
|
||||
source_path=data.source_path,
|
||||
strict=data.strict,
|
||||
session=session,
|
||||
)
|
||||
if entity:
|
||||
if entity.permalink == data.identifier:
|
||||
@@ -223,7 +232,7 @@ async def resolve_identifier(
|
||||
if not entity:
|
||||
raise HTTPException(status_code=404, detail=f"Entity not found: '{data.identifier}'")
|
||||
|
||||
owner_project = await project_repository.get_by_id(entity.project_id)
|
||||
owner_project = await project_repository.get_by_id(session, entity.project_id)
|
||||
if not owner_project: # pragma: no cover
|
||||
raise HTTPException(
|
||||
status_code=500,
|
||||
@@ -304,6 +313,7 @@ async def sync_file(
|
||||
project_config: ProjectConfigV2ExternalDep,
|
||||
search_service: SearchServiceV2ExternalDep,
|
||||
app_config: AppConfigDep,
|
||||
session_maker: SessionMakerDep,
|
||||
) -> EntityResponseV2:
|
||||
"""Index a single markdown file that exists on disk but is not indexed yet.
|
||||
|
||||
@@ -406,7 +416,8 @@ async def sync_file(
|
||||
# Trigger: the file may already have a DB row (e.g. modified on disk after indexing)
|
||||
# Why: the indexer needs to know whether to insert or update the entity
|
||||
# Outcome: new is computed from the database instead of assumed by the caller
|
||||
existing = await sync_service.entity_repository.get_by_file_path(file_path)
|
||||
async with db.scoped_session(session_maker) as session:
|
||||
existing = await sync_service.entity_repository.get_by_file_path(session, file_path)
|
||||
synced = await sync_service.sync_one_markdown_file(
|
||||
file_path, new=existing is None, index_search=True
|
||||
)
|
||||
@@ -435,6 +446,7 @@ async def sync_file(
|
||||
async def get_entity_by_id(
|
||||
project_id: ProjectExternalIdPathDep,
|
||||
entity_repository: EntityRepositoryV2ExternalDep,
|
||||
session: SessionDep,
|
||||
entity_id: str = Path(..., description="Entity external ID (UUID)"),
|
||||
) -> EntityResponseV2:
|
||||
"""Get an entity by its external ID (UUID).
|
||||
@@ -459,7 +471,7 @@ async def get_entity_by_id(
|
||||
):
|
||||
logger.info(f"API v2 request: get_entity_by_id entity_id={entity_id}")
|
||||
|
||||
entity = await entity_repository.get_by_external_id(entity_id)
|
||||
entity = await entity_repository.get_by_external_id(session, entity_id)
|
||||
if not entity:
|
||||
raise HTTPException(
|
||||
status_code=404, detail=f"Entity with external_id '{entity_id}' not found"
|
||||
@@ -536,6 +548,7 @@ async def update_entity_by_id(
|
||||
entity_repository: EntityRepositoryV2ExternalDep,
|
||||
task_scheduler: TaskSchedulerDep,
|
||||
app_config: AppConfigDep,
|
||||
session_maker: SessionMakerDep,
|
||||
entity_id: str = Path(..., description="Entity external ID (UUID)"),
|
||||
) -> EntityResponseV2:
|
||||
"""Update an entity by external ID.
|
||||
@@ -557,7 +570,8 @@ async def update_entity_by_id(
|
||||
):
|
||||
logger.info(f"API v2 request: update_entity_by_id entity_id={entity_id}")
|
||||
|
||||
existing = await entity_repository.get_by_external_id(entity_id)
|
||||
async with db.scoped_session(session_maker) as session:
|
||||
existing = await entity_repository.get_by_external_id(session, entity_id)
|
||||
created = existing is None
|
||||
|
||||
if existing:
|
||||
@@ -568,10 +582,12 @@ async def update_entity_by_id(
|
||||
write_result = await entity_service.create_entity_with_content(data)
|
||||
entity = write_result.entity
|
||||
if entity.external_id != entity_id:
|
||||
entity = await entity_repository.update(
|
||||
entity.id,
|
||||
{"external_id": entity_id},
|
||||
)
|
||||
async with db.scoped_session(session_maker) as session:
|
||||
entity = await entity_repository.update(
|
||||
session,
|
||||
entity.id,
|
||||
{"external_id": entity_id},
|
||||
)
|
||||
# external_id fixup only changes the DB row. The file content is unchanged,
|
||||
# so the markdown captured during the write remains valid downstream.
|
||||
if not entity:
|
||||
@@ -607,6 +623,7 @@ async def edit_entity_by_id(
|
||||
entity_repository: EntityRepositoryV2ExternalDep,
|
||||
task_scheduler: TaskSchedulerDep,
|
||||
app_config: AppConfigDep,
|
||||
session_maker: SessionMakerDep,
|
||||
entity_id: str = Path(..., description="Entity external ID (UUID)"),
|
||||
) -> EntityResponseV2:
|
||||
"""Edit an existing entity by external ID using operations like append, prepend, etc.
|
||||
@@ -631,7 +648,8 @@ async def edit_entity_by_id(
|
||||
f"API v2 request: edit_entity_by_id entity_id={entity_id}, operation='{data.operation}'"
|
||||
)
|
||||
|
||||
entity = await entity_repository.get_by_external_id(entity_id)
|
||||
async with db.scoped_session(session_maker) as session:
|
||||
entity = await entity_repository.get_by_external_id(session, entity_id)
|
||||
if not entity: # pragma: no cover
|
||||
raise HTTPException(
|
||||
status_code=404, detail=f"Entity with external_id '{entity_id}' not found"
|
||||
@@ -678,6 +696,7 @@ async def delete_entity_by_id(
|
||||
project_id: ProjectExternalIdPathDep,
|
||||
entity_service: EntityServiceV2ExternalDep,
|
||||
entity_repository: EntityRepositoryV2ExternalDep,
|
||||
session_maker: SessionMakerDep,
|
||||
entity_id: str = Path(..., description="Entity external ID (UUID)"),
|
||||
) -> DeleteEntitiesResponse:
|
||||
"""Delete an entity by external ID.
|
||||
@@ -698,7 +717,8 @@ async def delete_entity_by_id(
|
||||
):
|
||||
logger.info(f"API v2 request: delete_entity_by_id entity_id={entity_id}")
|
||||
|
||||
entity = await entity_repository.get_by_external_id(entity_id)
|
||||
async with db.scoped_session(session_maker) as session:
|
||||
entity = await entity_repository.get_by_external_id(session, entity_id)
|
||||
if entity is None:
|
||||
logger.info(f"API v2 response: external_id={entity_id} not found, deleted=False")
|
||||
return DeleteEntitiesResponse(deleted=False)
|
||||
@@ -724,6 +744,7 @@ async def move_entity(
|
||||
app_config: AppConfigDep,
|
||||
search_service: SearchServiceV2ExternalDep,
|
||||
task_scheduler: TaskSchedulerDep,
|
||||
session_maker: SessionMakerDep,
|
||||
entity_id: str = Path(..., description="Entity external ID (UUID)"),
|
||||
) -> EntityResponseV2:
|
||||
"""Move an entity to a new file location.
|
||||
@@ -751,7 +772,8 @@ async def move_entity(
|
||||
|
||||
try:
|
||||
# First, get the entity by external_id to verify it exists
|
||||
entity = await entity_repository.get_by_external_id(entity_id)
|
||||
async with db.scoped_session(session_maker) as session:
|
||||
entity = await entity_repository.get_by_external_id(session, entity_id)
|
||||
if not entity: # pragma: no cover
|
||||
raise HTTPException(
|
||||
status_code=404, detail=f"Entity with external_id '{entity_id}' not found"
|
||||
@@ -766,9 +788,11 @@ async def move_entity(
|
||||
)
|
||||
|
||||
# Reindex at new location
|
||||
reindexed_entity = await entity_service.link_resolver.resolve_link(
|
||||
data.destination_path
|
||||
)
|
||||
async with db.scoped_session(session_maker) as session:
|
||||
reindexed_entity = await entity_service.link_resolver.resolve_link(
|
||||
data.destination_path,
|
||||
session=session,
|
||||
)
|
||||
if reindexed_entity:
|
||||
await search_service.index_entity(reindexed_entity)
|
||||
_schedule_vector_sync_if_enabled(
|
||||
@@ -805,6 +829,7 @@ async def move_directory(
|
||||
app_config: AppConfigDep,
|
||||
search_service: SearchServiceV2ExternalDep,
|
||||
task_scheduler: TaskSchedulerDep,
|
||||
session_maker: SessionMakerDep,
|
||||
) -> DirectoryMoveResult:
|
||||
"""Move all entities in a directory to a new location.
|
||||
|
||||
@@ -840,7 +865,10 @@ async def move_directory(
|
||||
|
||||
# Reindex moved entities
|
||||
for file_path in result.moved_files:
|
||||
entity = await entity_service.link_resolver.resolve_link(file_path)
|
||||
async with db.scoped_session(session_maker) as session:
|
||||
entity = await entity_service.link_resolver.resolve_link(
|
||||
file_path, session=session
|
||||
)
|
||||
if entity:
|
||||
await search_service.index_entity(entity)
|
||||
_schedule_vector_sync_if_enabled(
|
||||
|
||||
@@ -10,7 +10,12 @@ from fastapi import APIRouter, Query, Path
|
||||
from loguru import logger
|
||||
|
||||
import logfire
|
||||
from basic_memory.deps import ContextServiceV2ExternalDep, EntityRepositoryV2ExternalDep
|
||||
from basic_memory import db
|
||||
from basic_memory.deps import (
|
||||
ContextServiceV2ExternalDep,
|
||||
EntityRepositoryV2ExternalDep,
|
||||
SessionMakerDep,
|
||||
)
|
||||
from basic_memory.schemas.base import TimeFrame, parse_timeframe
|
||||
from basic_memory.schemas.memory import (
|
||||
GraphContext,
|
||||
@@ -27,6 +32,7 @@ router = APIRouter(tags=["memory"])
|
||||
async def recent(
|
||||
context_service: ContextServiceV2ExternalDep,
|
||||
entity_repository: EntityRepositoryV2ExternalDep,
|
||||
session_maker: SessionMakerDep,
|
||||
project_id: str = Path(..., description="Project external UUID"),
|
||||
type: Annotated[list[SearchItemType] | None, Query()] = None,
|
||||
depth: int = 1,
|
||||
@@ -95,9 +101,14 @@ async def recent(
|
||||
phase="shape_response",
|
||||
result_count=len(context.results),
|
||||
):
|
||||
recent_context = await to_graph_context(
|
||||
context, entity_repository=entity_repository, page=page, page_size=page_size
|
||||
)
|
||||
async with db.scoped_session(session_maker) as session:
|
||||
recent_context = await to_graph_context(
|
||||
context,
|
||||
entity_repository=entity_repository,
|
||||
session=session,
|
||||
page=page,
|
||||
page_size=page_size,
|
||||
)
|
||||
logger.debug(f"V2 Recent context: {recent_context.model_dump_json()}")
|
||||
return recent_context
|
||||
|
||||
@@ -109,6 +120,7 @@ async def recent(
|
||||
async def get_memory_context(
|
||||
context_service: ContextServiceV2ExternalDep,
|
||||
entity_repository: EntityRepositoryV2ExternalDep,
|
||||
session_maker: SessionMakerDep,
|
||||
uri: str,
|
||||
project_id: str = Path(..., description="Project external UUID"),
|
||||
depth: int = 1,
|
||||
@@ -177,6 +189,11 @@ async def get_memory_context(
|
||||
phase="shape_response",
|
||||
result_count=len(context.results),
|
||||
):
|
||||
return await to_graph_context(
|
||||
context, entity_repository=entity_repository, page=page, page_size=page_size
|
||||
)
|
||||
async with db.scoped_session(session_maker) as session:
|
||||
return await to_graph_context(
|
||||
context,
|
||||
entity_repository=entity_repository,
|
||||
session=session,
|
||||
page=page,
|
||||
page_size=page_size,
|
||||
)
|
||||
|
||||
@@ -16,6 +16,7 @@ from typing import Literal, Optional
|
||||
from fastapi import APIRouter, HTTPException, Body, Query, Path
|
||||
from loguru import logger
|
||||
|
||||
from basic_memory import db
|
||||
from basic_memory.deps import (
|
||||
ProjectServiceDep,
|
||||
ProjectRepositoryDep,
|
||||
@@ -23,6 +24,8 @@ from basic_memory.deps import (
|
||||
SyncServiceV2ExternalDep,
|
||||
TaskSchedulerDep,
|
||||
ProjectExternalIdPathDep,
|
||||
SessionDep,
|
||||
SessionMakerDep,
|
||||
)
|
||||
from basic_memory.schemas import SyncReportResponse
|
||||
from basic_memory.models import Project
|
||||
@@ -54,21 +57,22 @@ def _split_qualified_project_identifier(identifier: str) -> tuple[str | None, st
|
||||
|
||||
|
||||
async def _resolve_project_identifier_candidate(
|
||||
session: SessionDep,
|
||||
project_repository: ProjectRepository,
|
||||
identifier: str,
|
||||
) -> tuple[Project | None, ProjectResolveMethod]:
|
||||
"""Resolve one project identifier candidate and report the matching method."""
|
||||
identifier_permalink = generate_permalink(identifier)
|
||||
|
||||
project = await project_repository.get_by_external_id(identifier)
|
||||
project = await project_repository.get_by_external_id(session, identifier)
|
||||
if project:
|
||||
return project, "external_id"
|
||||
|
||||
project = await project_repository.get_by_permalink(identifier_permalink)
|
||||
project = await project_repository.get_by_permalink(session, identifier_permalink)
|
||||
if project:
|
||||
return project, "permalink"
|
||||
|
||||
project = await project_repository.get_by_name_case_insensitive(identifier)
|
||||
project = await project_repository.get_by_name_case_insensitive(session, identifier)
|
||||
if project:
|
||||
return project, "name" # pragma: no cover
|
||||
|
||||
@@ -76,11 +80,13 @@ async def _resolve_project_identifier_candidate(
|
||||
|
||||
|
||||
async def _resolve_project_identifier(
|
||||
session: SessionDep,
|
||||
project_repository: ProjectRepository,
|
||||
identifier: str,
|
||||
) -> tuple[Project | None, ProjectResolveMethod]:
|
||||
"""Resolve exact identifiers first, then accepted workspace-qualified forms."""
|
||||
project, resolution_method = await _resolve_project_identifier_candidate(
|
||||
session,
|
||||
project_repository,
|
||||
identifier,
|
||||
)
|
||||
@@ -96,6 +102,7 @@ async def _resolve_project_identifier(
|
||||
# only needs the project segment to validate the active project.
|
||||
# Outcome: models can follow the hint verbatim instead of looping on a 404.
|
||||
project, resolution_method = await _resolve_project_identifier_candidate(
|
||||
session,
|
||||
project_repository,
|
||||
project_identifier,
|
||||
)
|
||||
@@ -193,7 +200,7 @@ async def add_project(
|
||||
return ProjectStatusResponse( # pyright: ignore [reportCallIssue]
|
||||
message=f"Project '{new_project.name}' added successfully",
|
||||
status="success",
|
||||
default=project_data.set_default,
|
||||
default=new_project.is_default or False,
|
||||
new_project=ProjectItem(
|
||||
id=new_project.id,
|
||||
external_id=new_project.external_id,
|
||||
@@ -277,6 +284,7 @@ async def get_project_status(
|
||||
@router.post("/resolve", response_model=ProjectResolveResponse)
|
||||
async def resolve_project_identifier(
|
||||
data: ProjectResolveRequest,
|
||||
session: SessionDep,
|
||||
project_repository: ProjectRepositoryDep,
|
||||
) -> ProjectResolveResponse:
|
||||
"""Resolve a project identifier (name, permalink, or external_id) to project info.
|
||||
@@ -315,12 +323,26 @@ async def resolve_project_identifier(
|
||||
logger.info(f"API v2 request: resolve_project_identifier for '{data.identifier}'")
|
||||
|
||||
project, resolution_method = await _resolve_project_identifier(
|
||||
session,
|
||||
project_repository,
|
||||
data.identifier,
|
||||
)
|
||||
|
||||
if not project:
|
||||
raise HTTPException(status_code=404, detail=f"Project not found: '{data.identifier}'")
|
||||
detail = f"Project not found: '{data.identifier}'"
|
||||
# Trigger: resolution missed and the projects table is empty.
|
||||
# Why: a fresh install bootstraps config.json's default project before any
|
||||
# reconciliation has created database rows (the one-shot CLI never runs
|
||||
# the server lifespan), so the first read fails on the configured
|
||||
# default and the bare not-found message reads as a broken install
|
||||
# rather than a missing first-run step (#974 follow-up).
|
||||
# Outcome: the error names the setup command instead.
|
||||
if not await project_repository.find_all(session, limit=1, use_load_options=False):
|
||||
detail = (
|
||||
f"{detail}. No projects are set up yet — run "
|
||||
"'basic-memory project add <name> <path>' to create one."
|
||||
)
|
||||
raise HTTPException(status_code=404, detail=detail)
|
||||
|
||||
return ProjectResolveResponse(
|
||||
external_id=project.external_id,
|
||||
@@ -336,6 +358,7 @@ async def resolve_project_identifier(
|
||||
|
||||
@router.get("/{project_id}", response_model=ProjectItem)
|
||||
async def get_project_by_id(
|
||||
session: SessionDep,
|
||||
project_repository: ProjectRepositoryDep,
|
||||
project_id: str = Path(..., description="Project external ID (UUID)"),
|
||||
) -> ProjectItem:
|
||||
@@ -358,7 +381,7 @@ async def get_project_by_id(
|
||||
"""
|
||||
logger.info(f"API v2 request: get_project_by_id for project_id={project_id}")
|
||||
|
||||
project = await project_repository.get_by_external_id(project_id)
|
||||
project = await project_repository.get_by_external_id(session, project_id)
|
||||
if not project:
|
||||
raise HTTPException(
|
||||
status_code=404, detail=f"Project with external_id '{project_id}' not found"
|
||||
@@ -376,12 +399,14 @@ async def get_project_by_id(
|
||||
@router.get("/{project_id}/info", response_model=ProjectInfoResponse)
|
||||
async def get_project_info_by_id(
|
||||
project_service: ProjectServiceDep,
|
||||
session_maker: SessionMakerDep,
|
||||
project_repository: ProjectRepositoryDep,
|
||||
project_id: str = Path(..., description="Project external ID (UUID)"),
|
||||
) -> ProjectInfoResponse:
|
||||
"""Get detailed project information by external ID."""
|
||||
logger.info(f"API v2 request: get_project_info_by_id for project_id={project_id}")
|
||||
project = await project_repository.get_by_external_id(project_id)
|
||||
async with db.scoped_session(session_maker) as session:
|
||||
project = await project_repository.get_by_external_id(session, project_id)
|
||||
if not project:
|
||||
raise HTTPException(
|
||||
status_code=404, detail=f"Project with external_id '{project_id}' not found"
|
||||
@@ -392,6 +417,7 @@ async def get_project_info_by_id(
|
||||
@router.patch("/{project_id}", response_model=ProjectStatusResponse)
|
||||
async def update_project_by_id(
|
||||
project_service: ProjectServiceDep,
|
||||
session_maker: SessionMakerDep,
|
||||
project_repository: ProjectRepositoryDep,
|
||||
project_id: str = Path(..., description="Project external ID (UUID)"),
|
||||
path: Optional[str] = Body(None, description="New absolute path for the project"),
|
||||
@@ -422,7 +448,8 @@ async def update_project_by_id(
|
||||
raise HTTPException(status_code=400, detail="Path must be absolute")
|
||||
|
||||
# Get original project info for the response
|
||||
old_project = await project_repository.get_by_external_id(project_id)
|
||||
async with db.scoped_session(session_maker) as session:
|
||||
old_project = await project_repository.get_by_external_id(session, project_id)
|
||||
if not old_project:
|
||||
raise HTTPException(
|
||||
status_code=404, detail=f"Project with external_id '{project_id}' not found"
|
||||
@@ -443,7 +470,8 @@ async def update_project_by_id(
|
||||
await project_service.update_project(old_project.name, is_active=is_active)
|
||||
|
||||
# Get updated project info (use the same external_id)
|
||||
updated_project = await project_repository.get_by_external_id(project_id)
|
||||
async with db.scoped_session(session_maker) as session:
|
||||
updated_project = await project_repository.get_by_external_id(session, project_id)
|
||||
if not updated_project: # pragma: no cover
|
||||
raise HTTPException(
|
||||
status_code=404,
|
||||
@@ -470,6 +498,7 @@ async def update_project_by_id(
|
||||
@router.delete("/{project_id}", response_model=ProjectStatusResponse)
|
||||
async def delete_project_by_id(
|
||||
project_service: ProjectServiceDep,
|
||||
session_maker: SessionMakerDep,
|
||||
project_repository: ProjectRepositoryDep,
|
||||
project_id: str = Path(..., description="Project external ID (UUID)"),
|
||||
delete_notes: bool = Query(
|
||||
@@ -496,7 +525,8 @@ async def delete_project_by_id(
|
||||
)
|
||||
|
||||
try:
|
||||
old_project = await project_repository.get_by_external_id(project_id)
|
||||
async with db.scoped_session(session_maker) as session:
|
||||
old_project = await project_repository.get_by_external_id(session, project_id)
|
||||
if not old_project:
|
||||
raise HTTPException(
|
||||
status_code=404, detail=f"Project with external_id '{project_id}' not found"
|
||||
@@ -539,6 +569,7 @@ async def delete_project_by_id(
|
||||
@router.put("/{project_id}/default", response_model=ProjectStatusResponse)
|
||||
async def set_default_project_by_id(
|
||||
project_service: ProjectServiceDep,
|
||||
session_maker: SessionMakerDep,
|
||||
project_repository: ProjectRepositoryDep,
|
||||
project_id: str = Path(..., description="Project external ID (UUID)"),
|
||||
) -> ProjectStatusResponse:
|
||||
@@ -559,15 +590,14 @@ async def set_default_project_by_id(
|
||||
logger.info(f"API v2 request: set_default_project_by_id for project_id={project_id}")
|
||||
|
||||
try:
|
||||
# Get the old default project from database
|
||||
default_project = await project_repository.get_default_project()
|
||||
if not default_project:
|
||||
raise HTTPException( # pragma: no cover
|
||||
status_code=404, detail="No default project is currently set"
|
||||
)
|
||||
# Get the old default project from database. It may be absent during
|
||||
# bootstrap/recovery (no default row yet); that is a valid state, not an
|
||||
# error, so we only echo it back when one exists.
|
||||
async with db.scoped_session(session_maker) as session:
|
||||
default_project = await project_repository.get_default_project(session)
|
||||
|
||||
# Get the new default project by external_id
|
||||
new_default_project = await project_repository.get_by_external_id(project_id)
|
||||
# Get the new default project by external_id
|
||||
new_default_project = await project_repository.get_by_external_id(session, project_id)
|
||||
if not new_default_project:
|
||||
raise HTTPException(
|
||||
status_code=404, detail=f"Project with external_id '{project_id}' not found"
|
||||
@@ -576,17 +606,27 @@ async def set_default_project_by_id(
|
||||
# Set as default using project name (service layer still uses names internally)
|
||||
await project_service.set_default_project(new_default_project.name)
|
||||
|
||||
return ProjectStatusResponse(
|
||||
message=f"Project '{new_default_project.name}' set as default successfully",
|
||||
status="success",
|
||||
default=True,
|
||||
old_project=ProjectItem(
|
||||
# Trigger: a previous default existed
|
||||
# Why: ProjectStatusResponse.old_project is Optional; the no-default
|
||||
# bootstrap case must succeed with old_project=None
|
||||
# Outcome: response echoes the prior default only when there was one
|
||||
old_project = (
|
||||
ProjectItem(
|
||||
id=default_project.id,
|
||||
external_id=default_project.external_id,
|
||||
name=default_project.name,
|
||||
path=default_project.path,
|
||||
is_default=False,
|
||||
),
|
||||
)
|
||||
if default_project
|
||||
else None
|
||||
)
|
||||
|
||||
return ProjectStatusResponse(
|
||||
message=f"Project '{new_default_project.name}' set as default successfully",
|
||||
status="success",
|
||||
default=True,
|
||||
old_project=old_project,
|
||||
new_project=ProjectItem(
|
||||
id=new_default_project.id,
|
||||
external_id=new_default_project.external_id,
|
||||
|
||||
@@ -10,6 +10,7 @@ from typing import Any
|
||||
from fastapi import APIRouter, HTTPException, status, Path
|
||||
from loguru import logger
|
||||
|
||||
from basic_memory import db
|
||||
from basic_memory.api.v2.utils import to_graph_context, to_search_results
|
||||
from basic_memory.api.template_loader import template_loader
|
||||
from basic_memory.schemas.base import parse_timeframe
|
||||
@@ -18,6 +19,7 @@ from basic_memory.deps import (
|
||||
EntityRepositoryV2ExternalDep,
|
||||
SearchServiceV2ExternalDep,
|
||||
EntityServiceV2ExternalDep,
|
||||
SessionMakerDep,
|
||||
)
|
||||
from basic_memory.schemas.prompt import (
|
||||
ContinueConversationRequest,
|
||||
@@ -36,6 +38,7 @@ async def continue_conversation(
|
||||
entity_service: EntityServiceV2ExternalDep,
|
||||
context_service: ContextServiceV2ExternalDep,
|
||||
entity_repository: EntityRepositoryV2ExternalDep,
|
||||
session_maker: SessionMakerDep,
|
||||
request: ContinueConversationRequest,
|
||||
project_id: str = Path(..., description="Project external UUID"),
|
||||
) -> PromptResponse:
|
||||
@@ -82,9 +85,10 @@ async def continue_conversation(
|
||||
)
|
||||
|
||||
# Process results into the schema format
|
||||
graph_context = await to_graph_context(
|
||||
context_result, entity_repository=entity_repository
|
||||
)
|
||||
async with db.scoped_session(session_maker) as session:
|
||||
graph_context = await to_graph_context(
|
||||
context_result, entity_repository=entity_repository, session=session
|
||||
)
|
||||
|
||||
# Add results to our collection (limit to top results for each permalink)
|
||||
if graph_context.results:
|
||||
@@ -109,7 +113,10 @@ async def continue_conversation(
|
||||
max_related=request.related_items_limit,
|
||||
include_observations=True,
|
||||
)
|
||||
recent_context = await to_graph_context(context_result, entity_repository=entity_repository)
|
||||
async with db.scoped_session(session_maker) as session:
|
||||
recent_context = await to_graph_context(
|
||||
context_result, entity_repository=entity_repository, session=session
|
||||
)
|
||||
|
||||
hierarchical_results = recent_context.results[:5] # Limit to top 5 recent items
|
||||
|
||||
|
||||
@@ -16,11 +16,14 @@ from fastapi import APIRouter, HTTPException, Response, Path
|
||||
from loguru import logger
|
||||
|
||||
import logfire
|
||||
from basic_memory import db
|
||||
from basic_memory.deps import (
|
||||
ProjectConfigV2ExternalDep,
|
||||
FileServiceV2ExternalDep,
|
||||
EntityRepositoryV2ExternalDep,
|
||||
SearchServiceV2ExternalDep,
|
||||
SessionDep,
|
||||
SessionMakerDep,
|
||||
)
|
||||
from basic_memory.models.knowledge import Entity as EntityModel
|
||||
from basic_memory.schemas.v2.resource import (
|
||||
@@ -38,6 +41,7 @@ async def get_resource_content(
|
||||
config: ProjectConfigV2ExternalDep,
|
||||
entity_repository: EntityRepositoryV2ExternalDep,
|
||||
file_service: FileServiceV2ExternalDep,
|
||||
session: SessionDep,
|
||||
project_id: str = Path(..., description="Project external UUID"),
|
||||
entity_id: str = Path(..., description="Entity external UUID"),
|
||||
) -> Response:
|
||||
@@ -70,7 +74,7 @@ async def get_resource_content(
|
||||
action="get_content",
|
||||
phase="load_entity",
|
||||
):
|
||||
entity = await entity_repository.get_by_external_id(entity_id)
|
||||
entity = await entity_repository.get_by_external_id(session, entity_id)
|
||||
if not entity:
|
||||
raise HTTPException(status_code=404, detail=f"Entity {entity_id} not found")
|
||||
|
||||
@@ -121,6 +125,7 @@ async def create_resource(
|
||||
file_service: FileServiceV2ExternalDep,
|
||||
entity_repository: EntityRepositoryV2ExternalDep,
|
||||
search_service: SearchServiceV2ExternalDep,
|
||||
session_maker: SessionMakerDep,
|
||||
project_id: str = Path(..., description="Project external UUID"),
|
||||
) -> ResourceResponse:
|
||||
"""Create a new resource file.
|
||||
@@ -158,13 +163,14 @@ async def create_resource(
|
||||
"Path must be relative and stay within project boundaries.",
|
||||
)
|
||||
|
||||
existing_entity = await entity_repository.get_by_file_path(data.file_path)
|
||||
if existing_entity:
|
||||
raise HTTPException(
|
||||
status_code=409,
|
||||
detail=f"Resource already exists at {data.file_path} with entity_id {existing_entity.external_id}. "
|
||||
f"Use PUT /resource/{existing_entity.external_id} to update it.",
|
||||
)
|
||||
async with db.scoped_session(session_maker) as session:
|
||||
existing_entity = await entity_repository.get_by_file_path(session, data.file_path)
|
||||
if existing_entity:
|
||||
raise HTTPException(
|
||||
status_code=409,
|
||||
detail=f"Resource already exists at {data.file_path} with entity_id {existing_entity.external_id}. "
|
||||
f"Use PUT /resource/{existing_entity.external_id} to update it.",
|
||||
)
|
||||
|
||||
with logfire.span(
|
||||
"api.resource.create.write_file",
|
||||
@@ -203,7 +209,8 @@ async def create_resource(
|
||||
action="create",
|
||||
phase="upsert_entity",
|
||||
):
|
||||
entity = await entity_repository.add(entity)
|
||||
async with db.scoped_session(session_maker) as session:
|
||||
entity = await entity_repository.add(session, entity)
|
||||
|
||||
with logfire.span(
|
||||
"api.resource.create.search_index",
|
||||
@@ -236,6 +243,7 @@ async def update_resource(
|
||||
file_service: FileServiceV2ExternalDep,
|
||||
entity_repository: EntityRepositoryV2ExternalDep,
|
||||
search_service: SearchServiceV2ExternalDep,
|
||||
session_maker: SessionMakerDep,
|
||||
project_id: str = Path(..., description="Project external UUID"),
|
||||
entity_id: str = Path(..., description="Entity external UUID"),
|
||||
) -> ResourceResponse:
|
||||
@@ -265,7 +273,8 @@ async def update_resource(
|
||||
action="update",
|
||||
):
|
||||
try:
|
||||
entity = await entity_repository.get_by_external_id(entity_id)
|
||||
async with db.scoped_session(session_maker) as session:
|
||||
entity = await entity_repository.get_by_external_id(session, entity_id)
|
||||
if not entity:
|
||||
raise HTTPException(status_code=404, detail=f"Entity {entity_id} not found")
|
||||
|
||||
@@ -315,17 +324,19 @@ async def update_resource(
|
||||
action="update",
|
||||
phase="update_entity",
|
||||
):
|
||||
updated_entity = await entity_repository.update(
|
||||
entity.id,
|
||||
{
|
||||
"title": file_name,
|
||||
"note_type": note_type,
|
||||
"content_type": content_type,
|
||||
"file_path": target_file_path,
|
||||
"checksum": checksum,
|
||||
"updated_at": file_metadata.modified_at,
|
||||
},
|
||||
)
|
||||
async with db.scoped_session(session_maker) as session:
|
||||
updated_entity = await entity_repository.update(
|
||||
session,
|
||||
entity.id,
|
||||
{
|
||||
"title": file_name,
|
||||
"note_type": note_type,
|
||||
"content_type": content_type,
|
||||
"file_path": target_file_path,
|
||||
"checksum": checksum,
|
||||
"updated_at": file_metadata.modified_at,
|
||||
},
|
||||
)
|
||||
if updated_entity is None:
|
||||
raise HTTPException(status_code=404, detail=f"Entity {entity_id} not found")
|
||||
|
||||
|
||||
@@ -13,11 +13,13 @@ from pathlib import Path as FilePath
|
||||
import frontmatter
|
||||
from fastapi import APIRouter, Path, Query
|
||||
from loguru import logger
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
from basic_memory.deps import (
|
||||
EntityRepositoryV2ExternalDep,
|
||||
FileServiceV2ExternalDep,
|
||||
LinkResolverV2ExternalDep,
|
||||
SessionDep,
|
||||
)
|
||||
from basic_memory.models.knowledge import Entity
|
||||
from basic_memory.schemas.schema import (
|
||||
@@ -131,6 +133,7 @@ async def validate_schema(
|
||||
entity_repository: EntityRepositoryV2ExternalDep,
|
||||
file_service: FileServiceV2ExternalDep,
|
||||
link_resolver: LinkResolverV2ExternalDep,
|
||||
session: SessionDep,
|
||||
project_id: str = Path(..., description="Project external UUID"),
|
||||
note_type: str | None = Query(None, description="Note type to validate"),
|
||||
identifier: str | None = Query(None, description="Specific note identifier"),
|
||||
@@ -150,7 +153,7 @@ async def validate_schema(
|
||||
if identifier:
|
||||
# Resolve identifier flexibly (permalink, title, path, fuzzy)
|
||||
# to match how read_note and other tools resolve identifiers
|
||||
entity = await link_resolver.resolve_link(identifier)
|
||||
entity = await link_resolver.resolve_link(identifier, session=session)
|
||||
if not entity:
|
||||
return ValidationReport(note_type=note_type, total_notes=0, total_entities=0)
|
||||
|
||||
@@ -159,6 +162,7 @@ async def validate_schema(
|
||||
|
||||
async def search_fn(query: str) -> list[dict]:
|
||||
entities = await _find_schema_entities(
|
||||
session,
|
||||
entity_repository,
|
||||
query,
|
||||
allow_reference_match=isinstance(schema_ref, str) and query == schema_ref,
|
||||
@@ -187,7 +191,7 @@ async def validate_schema(
|
||||
)
|
||||
|
||||
# --- Batch validation by note type ---
|
||||
entities = await _find_by_note_type(entity_repository, note_type) if note_type else []
|
||||
entities = await _find_by_note_type(session, entity_repository, note_type) if note_type else []
|
||||
|
||||
for entity in entities:
|
||||
frontmatter = _entity_frontmatter(entity)
|
||||
@@ -195,6 +199,7 @@ async def validate_schema(
|
||||
|
||||
async def search_fn(query: str) -> list[dict]:
|
||||
entities = await _find_schema_entities(
|
||||
session,
|
||||
entity_repository,
|
||||
query,
|
||||
allow_reference_match=isinstance(schema_ref, str) and query == schema_ref,
|
||||
@@ -230,6 +235,7 @@ async def validate_schema(
|
||||
@router.post("/schema/infer", response_model=InferenceReport)
|
||||
async def infer_schema_endpoint(
|
||||
entity_repository: EntityRepositoryV2ExternalDep,
|
||||
session: SessionDep,
|
||||
project_id: str = Path(..., description="Project external UUID"),
|
||||
note_type: str = Query(..., description="Note type to analyze"),
|
||||
threshold: float = Query(0.25, description="Minimum frequency for optional fields"),
|
||||
@@ -239,7 +245,7 @@ async def infer_schema_endpoint(
|
||||
Examines observation categories and relation types across all notes
|
||||
of the given type. Returns frequency analysis and suggested Picoschema.
|
||||
"""
|
||||
entities = await _find_by_note_type(entity_repository, note_type)
|
||||
entities = await _find_by_note_type(session, entity_repository, note_type)
|
||||
notes_data = [_entity_to_note_data(entity) for entity in entities]
|
||||
|
||||
result = infer_schema(note_type, notes_data, optional_threshold=threshold)
|
||||
@@ -274,6 +280,7 @@ async def infer_schema_endpoint(
|
||||
async def diff_schema_endpoint(
|
||||
entity_repository: EntityRepositoryV2ExternalDep,
|
||||
file_service: FileServiceV2ExternalDep,
|
||||
session: SessionDep,
|
||||
note_type: str = Path(..., description="Note type to check for drift"),
|
||||
project_id: str = Path(..., description="Project external UUID"),
|
||||
):
|
||||
@@ -285,7 +292,7 @@ async def diff_schema_endpoint(
|
||||
"""
|
||||
|
||||
async def search_fn(query: str) -> list[dict]:
|
||||
entities = await _find_schema_entities(entity_repository, query)
|
||||
entities = await _find_schema_entities(session, entity_repository, query)
|
||||
return [await _schema_frontmatter_from_file(file_service, e) for e in entities]
|
||||
|
||||
# Resolve schema by note type
|
||||
@@ -296,7 +303,7 @@ async def diff_schema_endpoint(
|
||||
return DriftReport(note_type=note_type, schema_found=False)
|
||||
|
||||
# Collect all notes of this type
|
||||
entities = await _find_by_note_type(entity_repository, note_type)
|
||||
entities = await _find_by_note_type(session, entity_repository, note_type)
|
||||
notes_data = [_entity_to_note_data(entity) for entity in entities]
|
||||
|
||||
result = diff_schema(schema_def, notes_data)
|
||||
@@ -331,16 +338,18 @@ async def diff_schema_endpoint(
|
||||
|
||||
|
||||
async def _find_by_note_type(
|
||||
session: AsyncSession,
|
||||
entity_repository: EntityRepositoryV2ExternalDep,
|
||||
note_type: str,
|
||||
) -> list[Entity]:
|
||||
"""Find all entities of a given type using the repository's select pattern."""
|
||||
query = entity_repository.select().where(Entity.note_type == note_type)
|
||||
result = await entity_repository.execute_query(query)
|
||||
result = await entity_repository.execute_query(session, query)
|
||||
return list(result.scalars().all())
|
||||
|
||||
|
||||
async def _find_schema_entities(
|
||||
session: AsyncSession,
|
||||
entity_repository: EntityRepositoryV2ExternalDep,
|
||||
target_note_type: str,
|
||||
*,
|
||||
@@ -355,7 +364,7 @@ async def _find_schema_entities(
|
||||
exact reference matching by title/permalink (explicit schema references)
|
||||
"""
|
||||
query = entity_repository.select().where(Entity.note_type == "schema")
|
||||
result = await entity_repository.execute_query(query)
|
||||
result = await entity_repository.execute_query(session, query)
|
||||
entities = list(result.scalars().all())
|
||||
|
||||
normalized_target = generate_permalink(target_note_type)
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
from typing import Any, Protocol, Optional, List, Sequence
|
||||
|
||||
import logfire
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
from basic_memory.repository.search_repository import SearchIndexRow
|
||||
from basic_memory.schemas.memory import (
|
||||
EntitySummary,
|
||||
@@ -19,7 +20,11 @@ from basic_memory.services.context_service import (
|
||||
|
||||
class EntityBatchLookup(Protocol):
|
||||
async def find_by_ids_for_hydration(
|
||||
self, ids: List[int], *, include_cross_project: bool = False
|
||||
self,
|
||||
session: AsyncSession,
|
||||
ids: List[int],
|
||||
*,
|
||||
include_cross_project: bool = False,
|
||||
) -> Sequence[Any]: ...
|
||||
|
||||
|
||||
@@ -42,6 +47,7 @@ def _search_item_type(value: str | SearchItemType) -> SearchItemType:
|
||||
async def to_graph_context(
|
||||
context_result: ServiceContextResult,
|
||||
entity_repository: EntityBatchLookup,
|
||||
session: AsyncSession,
|
||||
page: Optional[int] = None,
|
||||
page_size: Optional[int] = None,
|
||||
) -> GraphContext:
|
||||
@@ -90,7 +96,7 @@ async def to_graph_context(
|
||||
result_count=len(entity_ids_needed),
|
||||
):
|
||||
entities = await entity_repository.find_by_ids_for_hydration(
|
||||
list(entity_ids_needed), include_cross_project=True
|
||||
session, list(entity_ids_needed), include_cross_project=True
|
||||
)
|
||||
for e in entities:
|
||||
entity_title_lookup[e.id] = e.title
|
||||
@@ -147,6 +153,7 @@ async def to_graph_context(
|
||||
from_entity_id=item.from_id,
|
||||
from_entity_external_id=from_ext_id,
|
||||
to_entity=to_title,
|
||||
to_name=item.to_name,
|
||||
to_entity_id=item.to_id,
|
||||
to_entity_external_id=to_ext_id,
|
||||
created_at=item.created_at,
|
||||
|
||||
@@ -83,8 +83,11 @@ def app_callback(
|
||||
# Skip for 'mcp' command - it has its own lifespan that handles initialization
|
||||
# Skip for API-using commands (status, sync, etc.) - they handle initialization via deps.py
|
||||
# Skip for 'reset' command - it manages its own database lifecycle
|
||||
# Skip for 'man' - it only copies packaged files; a broken local database
|
||||
# must not block installing the offline docs
|
||||
skip_init_commands = {
|
||||
"doctor",
|
||||
"man",
|
||||
"mcp",
|
||||
"status",
|
||||
"sync",
|
||||
|
||||
@@ -168,6 +168,20 @@ def _manual_update_hint(source: InstallSource) -> str:
|
||||
)
|
||||
|
||||
|
||||
def _preload_lazy_console_modules() -> None:
|
||||
"""Import modules the post-upgrade output path defers until print time.
|
||||
|
||||
Trigger: an in-place upgrade is about to replace this installation on disk.
|
||||
Why: rich and typer defer some imports until print/excepthook time; once
|
||||
`brew upgrade` / `uv tool upgrade` removes the running version's files,
|
||||
those imports raise ModuleNotFoundError and the final status message
|
||||
crashes the exiting process.
|
||||
Outcome: import the deferred modules now, while the files still exist.
|
||||
"""
|
||||
import rich._emoji_codes # noqa: F401
|
||||
import typer.rich_utils # noqa: F401
|
||||
|
||||
|
||||
def _save_last_checked_timestamp(config_manager: ConfigManager, checked_at: datetime) -> None:
|
||||
"""Persist the timestamp for the most recent attempted update check."""
|
||||
config = config_manager.load_config()
|
||||
@@ -288,6 +302,7 @@ def run_auto_update(
|
||||
else BREW_UPGRADE_TIMEOUT_SECONDS
|
||||
)
|
||||
|
||||
_preload_lazy_console_modules()
|
||||
install_result = _run_subprocess(
|
||||
command,
|
||||
timeout_seconds=timeout,
|
||||
|
||||
@@ -4,6 +4,7 @@ from . import ci, status, db, doctor, import_memory_json, mcp, import_claude_con
|
||||
from . import (
|
||||
import_claude_projects,
|
||||
import_chatgpt,
|
||||
man,
|
||||
tool,
|
||||
project,
|
||||
format,
|
||||
@@ -29,4 +30,5 @@ __all__ = [
|
||||
"schema",
|
||||
"update",
|
||||
"workspace",
|
||||
"man",
|
||||
]
|
||||
|
||||
@@ -34,8 +34,10 @@ from basic_memory.ci.project_updates import (
|
||||
from basic_memory.cli.app import app
|
||||
from basic_memory.cli.commands.command_utils import run_with_cleanup
|
||||
from basic_memory.cli.commands.routing import force_routing, validate_routing_flags
|
||||
from basic_memory.mcp.tools import search_notes as mcp_search_notes
|
||||
from basic_memory.mcp.tools import write_note as mcp_write_note
|
||||
|
||||
# MCP tool functions are imported inside the async helpers below: importing
|
||||
# basic_memory.mcp.tools loads the entire tool stack (fastmcp, mcp SDK,
|
||||
# SQLAlchemy), which would slow every CLI invocation, including --help (#886).
|
||||
|
||||
|
||||
console = Console()
|
||||
@@ -252,6 +254,10 @@ async def seed_project_update_schemas(
|
||||
refresh: bool = False,
|
||||
) -> list[str]:
|
||||
"""Seed Auto BM schema notes without overwriting customized schemas."""
|
||||
# Deferred: loading the MCP tool stack at module import slows CLI startup (#886).
|
||||
from basic_memory.mcp.tools import search_notes as mcp_search_notes
|
||||
from basic_memory.mcp.tools import write_note as mcp_write_note
|
||||
|
||||
seeded: list[str] = []
|
||||
routed_project = _routed_project(project=project, project_id=project_id, workspace=workspace)
|
||||
for spec in schema_seed_specs():
|
||||
@@ -295,6 +301,10 @@ async def publish_project_update_note(
|
||||
note: ProjectUpdateNote,
|
||||
) -> dict[str, Any]:
|
||||
"""Search by idempotency key and then upsert the deterministic note path."""
|
||||
# Deferred: loading the MCP tool stack at module import slows CLI startup (#886).
|
||||
from basic_memory.mcp.tools import search_notes as mcp_search_notes
|
||||
from basic_memory.mcp.tools import write_note as mcp_write_note
|
||||
|
||||
routed_project = _routed_project(
|
||||
project=config.project,
|
||||
project_id=config.project_id,
|
||||
|
||||
@@ -3,12 +3,10 @@
|
||||
import asyncio
|
||||
from typing import Optional, TypeVar, Coroutine, Any
|
||||
|
||||
from mcp.server.fastmcp.exceptions import ToolError
|
||||
import typer
|
||||
|
||||
from rich.console import Console
|
||||
|
||||
from basic_memory import db
|
||||
from basic_memory.config import ConfigManager
|
||||
from basic_memory.mcp.async_client import get_client
|
||||
from basic_memory.mcp.clients import ProjectClient
|
||||
@@ -31,6 +29,9 @@ def run_with_cleanup(coro: Coroutine[Any, Any, T]) -> T:
|
||||
Returns:
|
||||
The result of the coroutine
|
||||
"""
|
||||
# Deferred: basic_memory.db pulls SQLAlchemy + Alembic, which must not load
|
||||
# at CLI import time — only when a command actually runs (#886).
|
||||
from basic_memory import db
|
||||
|
||||
async def _with_cleanup() -> T:
|
||||
try:
|
||||
@@ -53,6 +54,8 @@ async def run_sync(
|
||||
force_full: If True, force a full scan bypassing watermark optimization
|
||||
run_in_background: If True, return immediately; if False, wait for completion
|
||||
"""
|
||||
# Deferred: ToolError lives in the mcp SDK, which must not load at CLI startup (#886).
|
||||
from mcp.server.fastmcp.exceptions import ToolError
|
||||
|
||||
# Resolve default project so get_client() can route per-project
|
||||
project = project or ConfigManager().default_project
|
||||
@@ -86,6 +89,9 @@ async def run_sync(
|
||||
|
||||
async def get_project_info(project: str):
|
||||
"""Get project information via API endpoint."""
|
||||
# Deferred: ToolError lives in the mcp SDK, which must not load at CLI startup (#886).
|
||||
from mcp.server.fastmcp.exceptions import ToolError
|
||||
|
||||
try:
|
||||
async with get_client(project_name=project) as client:
|
||||
project_item = await get_active_project(client, project, None)
|
||||
|
||||
@@ -1,24 +1,27 @@
|
||||
"""Database management commands."""
|
||||
|
||||
# PEP 563 lazy annotations let signatures reference IndexProgress without importing
|
||||
# the indexing stack at module load; reset/reindex import their heavy database and
|
||||
# sync dependencies at call time so CLI startup stays fast (#886).
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path, PurePosixPath, PureWindowsPath
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
import psutil
|
||||
import typer
|
||||
from loguru import logger
|
||||
from rich.console import Console
|
||||
from rich.progress import Progress, SpinnerColumn, TextColumn, BarColumn, TaskProgressColumn
|
||||
from sqlalchemy.exc import OperationalError
|
||||
|
||||
from basic_memory import db
|
||||
from basic_memory.cli.app import app
|
||||
from basic_memory.cli.commands.command_utils import run_with_cleanup
|
||||
from basic_memory.config import ConfigManager, ProjectMode
|
||||
from basic_memory.indexing import IndexProgress
|
||||
from basic_memory.repository import ProjectRepository
|
||||
from basic_memory.services.initialization import reconcile_projects_with_config
|
||||
from basic_memory.sync.sync_service import get_sync_service
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from basic_memory.indexing import IndexProgress
|
||||
|
||||
console = Console()
|
||||
|
||||
@@ -159,6 +162,13 @@ async def _reindex_projects(app_config):
|
||||
This ensures all database operations use the same event loop,
|
||||
and proper cleanup happens when the function completes.
|
||||
"""
|
||||
# Deferred: SQLAlchemy, repositories, and the sync stack load only when a
|
||||
# reindex actually runs, not on every CLI start (#886).
|
||||
from basic_memory import db
|
||||
from basic_memory.repository import ProjectRepository
|
||||
from basic_memory.services.initialization import reconcile_projects_with_config
|
||||
from basic_memory.sync.sync_service import get_sync_service
|
||||
|
||||
try:
|
||||
await reconcile_projects_with_config(app_config)
|
||||
|
||||
@@ -167,8 +177,9 @@ async def _reindex_projects(app_config):
|
||||
db_path=app_config.database_path,
|
||||
db_type=db.DatabaseType.FILESYSTEM,
|
||||
)
|
||||
project_repository = ProjectRepository(session_maker)
|
||||
projects = await project_repository.get_active_projects()
|
||||
project_repository = ProjectRepository()
|
||||
async with db.scoped_session(session_maker) as session:
|
||||
projects = await project_repository.get_active_projects(session)
|
||||
|
||||
for project in projects:
|
||||
console.print(f" Indexing [cyan]{project.name}[/cyan]...")
|
||||
@@ -197,6 +208,12 @@ def reset(
|
||||
),
|
||||
): # pragma: no cover
|
||||
"""Reset database (drop all tables and recreate)."""
|
||||
# Deferred: SQLAlchemy and the db module load only when a reset actually
|
||||
# runs, not on every CLI start (#886).
|
||||
from sqlalchemy.exc import OperationalError
|
||||
|
||||
from basic_memory import db
|
||||
|
||||
console.print(
|
||||
"[yellow]Note:[/yellow] This only deletes the index database. "
|
||||
"Your markdown note files will not be affected.\n"
|
||||
@@ -320,10 +337,15 @@ async def _reindex(
|
||||
project: str | None,
|
||||
):
|
||||
"""Run reindex operations."""
|
||||
from basic_memory.repository import EntityRepository
|
||||
# Deferred: SQLAlchemy, repositories, and the sync stack load only when a
|
||||
# reindex actually runs, not on every CLI start (#886).
|
||||
from basic_memory import db
|
||||
from basic_memory.repository import EntityRepository, ProjectRepository
|
||||
from basic_memory.repository.search_repository import create_search_repository
|
||||
from basic_memory.services.initialization import reconcile_projects_with_config
|
||||
from basic_memory.services.search_service import SearchService
|
||||
from basic_memory.services.file_service import FileService
|
||||
from basic_memory.sync.sync_service import get_sync_service
|
||||
from basic_memory.markdown.markdown_processor import MarkdownProcessor
|
||||
from basic_memory.markdown.entity_parser import EntityParser
|
||||
|
||||
@@ -334,8 +356,9 @@ async def _reindex(
|
||||
db_path=app_config.database_path,
|
||||
db_type=db.DatabaseType.FILESYSTEM,
|
||||
)
|
||||
project_repository = ProjectRepository(session_maker)
|
||||
projects = await project_repository.get_active_projects()
|
||||
project_repository = ProjectRepository()
|
||||
async with db.scoped_session(session_maker) as session:
|
||||
projects = await project_repository.get_active_projects(session)
|
||||
|
||||
if project:
|
||||
projects = [p for p in projects if p.name == project]
|
||||
@@ -397,7 +420,7 @@ async def _reindex(
|
||||
console.print(
|
||||
f" Building vector embeddings ([cyan]{embedding_mode_label}[/cyan])..."
|
||||
)
|
||||
entity_repository = EntityRepository(session_maker, project_id=proj.id)
|
||||
entity_repository = EntityRepository(project_id=proj.id)
|
||||
search_repository = create_search_repository(
|
||||
session_maker, project_id=proj.id, app_config=app_config
|
||||
)
|
||||
@@ -405,7 +428,12 @@ async def _reindex(
|
||||
entity_parser = EntityParser(project_path)
|
||||
markdown_processor = MarkdownProcessor(entity_parser, app_config=app_config)
|
||||
file_service = FileService(project_path, markdown_processor, app_config=app_config)
|
||||
search_service = SearchService(search_repository, entity_repository, file_service)
|
||||
search_service = SearchService(
|
||||
search_repository,
|
||||
entity_repository,
|
||||
file_service,
|
||||
session_maker=session_maker,
|
||||
)
|
||||
|
||||
with Progress(
|
||||
SpinnerColumn(),
|
||||
|
||||
@@ -7,16 +7,12 @@ import uuid
|
||||
from pathlib import Path
|
||||
|
||||
from loguru import logger
|
||||
from mcp.server.fastmcp.exceptions import ToolError
|
||||
from rich.console import Console
|
||||
import typer
|
||||
|
||||
from basic_memory.cli.app import app
|
||||
from basic_memory.cli.commands.command_utils import run_with_cleanup
|
||||
from basic_memory.cli.commands.routing import force_routing, validate_routing_flags
|
||||
from basic_memory.markdown.entity_parser import EntityParser
|
||||
from basic_memory.markdown.markdown_processor import MarkdownProcessor
|
||||
from basic_memory.markdown.schemas import EntityFrontmatter, EntityMarkdown
|
||||
from basic_memory.mcp.async_client import get_client
|
||||
from basic_memory.mcp.clients import KnowledgeClient, ProjectClient, SearchClient
|
||||
from basic_memory.schemas.base import Entity
|
||||
@@ -27,8 +23,72 @@ from basic_memory.schemas import SyncReportResponse
|
||||
console = Console()
|
||||
|
||||
|
||||
def _is_default_project_delete_error(error: Exception) -> bool:
|
||||
"""Return True only for the API guard that blocks deleting the default project."""
|
||||
error_text = str(error)
|
||||
return "Cannot delete default project" in error_text
|
||||
|
||||
|
||||
async def _delete_doctor_project_locally(project_name: str, project_id: str) -> None:
|
||||
"""Remove the generated doctor project when the public API guard blocks cleanup."""
|
||||
from basic_memory import db
|
||||
from basic_memory.config import ConfigManager
|
||||
from basic_memory.repository import ProjectRepository
|
||||
|
||||
config_manager = ConfigManager()
|
||||
repository = ProjectRepository()
|
||||
_, session_maker = await db.get_or_create_db(
|
||||
db_path=config_manager.config.database_path,
|
||||
db_type=db.DatabaseType.FILESYSTEM,
|
||||
)
|
||||
|
||||
async with db.scoped_session(session_maker) as session:
|
||||
project = await repository.get_by_external_id(session, project_id)
|
||||
if project is None:
|
||||
raise ValueError(f"Doctor cleanup project '{project_id}' not found")
|
||||
if project.name != project_name:
|
||||
raise ValueError(
|
||||
f"Doctor cleanup expected project '{project_name}', found '{project.name}'"
|
||||
)
|
||||
await repository.delete(session, project.id)
|
||||
|
||||
config = config_manager.load_config()
|
||||
if project_name in config.projects:
|
||||
del config.projects[project_name]
|
||||
if config.default_project == project_name:
|
||||
config.default_project = next(iter(config.projects), None)
|
||||
config_manager.save_config(config)
|
||||
|
||||
|
||||
async def _delete_doctor_project(
|
||||
project_client: ProjectClient, project_name: str, project_id: str
|
||||
) -> None:
|
||||
"""Delete the generated doctor project without weakening the public API guard."""
|
||||
# Deferred: ToolError lives in the mcp SDK, which must not load at CLI startup (#886).
|
||||
from mcp.server.fastmcp.exceptions import ToolError
|
||||
|
||||
try:
|
||||
await project_client.delete_project(project_id)
|
||||
except ToolError as exc:
|
||||
if not _is_default_project_delete_error(exc):
|
||||
raise
|
||||
|
||||
# Trigger: fresh local configs can promote the generated doctor project
|
||||
# to default because the placeholder default has no DB row.
|
||||
# Why: the project is disposable doctor-owned state, while the public API
|
||||
# must keep rejecting default-project deletion for normal callers.
|
||||
# Outcome: cleanup removes only the exact doctor project it created.
|
||||
await _delete_doctor_project_locally(project_name, project_id)
|
||||
|
||||
|
||||
async def run_doctor() -> None:
|
||||
"""Run local consistency checks for file <-> database flows."""
|
||||
# Deferred: the markdown parsing stack is only needed while the checks run,
|
||||
# and importing it at module level slows every CLI invocation (#886).
|
||||
from basic_memory.markdown.entity_parser import EntityParser
|
||||
from basic_memory.markdown.markdown_processor import MarkdownProcessor
|
||||
from basic_memory.markdown.schemas import EntityFrontmatter, EntityMarkdown
|
||||
|
||||
console.print("[blue]Running Basic Memory doctor checks...[/blue]")
|
||||
|
||||
project_name = f"doctor-{uuid.uuid4().hex[:8]}"
|
||||
@@ -127,7 +187,7 @@ async def run_doctor() -> None:
|
||||
|
||||
finally:
|
||||
if project_id:
|
||||
await project_client.delete_project(project_id)
|
||||
await _delete_doctor_project(project_client, project_name, project_id)
|
||||
|
||||
console.print("[green]Doctor checks passed.[/green]")
|
||||
|
||||
@@ -140,6 +200,9 @@ def doctor(
|
||||
cloud: bool = typer.Option(False, "--cloud", help="Force cloud API routing"),
|
||||
) -> None:
|
||||
"""Run local consistency checks to verify file/database sync."""
|
||||
# Deferred: ToolError lives in the mcp SDK, which must not load at CLI startup (#886).
|
||||
from mcp.server.fastmcp.exceptions import ToolError
|
||||
|
||||
try:
|
||||
validate_routing_flags(local, cloud)
|
||||
# Doctor runs local filesystem checks — always default to local routing
|
||||
|
||||
@@ -1,25 +1,34 @@
|
||||
"""Import command for ChatGPT conversations."""
|
||||
|
||||
# PEP 563 lazy annotations keep heavy importer types out of module import (#886).
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from pathlib import Path
|
||||
from typing import Annotated, Tuple
|
||||
from typing import TYPE_CHECKING, Annotated, Tuple
|
||||
|
||||
import typer
|
||||
from basic_memory.cli.app import import_app
|
||||
from basic_memory.cli.commands.command_utils import run_with_cleanup
|
||||
from basic_memory.config import ConfigManager, get_project_config
|
||||
from basic_memory.importers import ChatGPTImporter
|
||||
from basic_memory.markdown import EntityParser, MarkdownProcessor
|
||||
from basic_memory.services.file_service import FileService
|
||||
from loguru import logger
|
||||
from rich.console import Console
|
||||
from rich.panel import Panel
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from basic_memory.markdown import MarkdownProcessor
|
||||
from basic_memory.services.file_service import FileService
|
||||
|
||||
console = Console()
|
||||
|
||||
|
||||
async def get_importer_dependencies() -> Tuple[MarkdownProcessor, FileService]:
|
||||
"""Get MarkdownProcessor and FileService instances for importers."""
|
||||
# Deferred: the markdown/file-service stack pulls SQLAlchemy and must load
|
||||
# only when an import actually runs, not on every CLI start (#886).
|
||||
from basic_memory.markdown import EntityParser, MarkdownProcessor
|
||||
from basic_memory.services.file_service import FileService
|
||||
|
||||
config = get_project_config()
|
||||
app_config = ConfigManager().config
|
||||
entity_parser = EntityParser(config.home)
|
||||
@@ -60,6 +69,9 @@ def import_chatgpt(
|
||||
console.print(f"\nImporting chats from {conversations_json}...writing to {base_path}")
|
||||
|
||||
# Create importer and run import
|
||||
# Deferred: importer stack loads at import-command run time only (#886).
|
||||
from basic_memory.importers import ChatGPTImporter
|
||||
|
||||
importer = ChatGPTImporter(
|
||||
config.home, markdown_processor, file_service, project_name=config.name
|
||||
)
|
||||
|
||||
@@ -1,25 +1,34 @@
|
||||
"""Import command for basic-memory CLI to import chat data from conversations2.json format."""
|
||||
|
||||
# PEP 563 lazy annotations keep heavy importer types out of module import (#886).
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from pathlib import Path
|
||||
from typing import Annotated, Tuple
|
||||
from typing import TYPE_CHECKING, Annotated, Tuple
|
||||
|
||||
import typer
|
||||
from basic_memory.cli.app import claude_app
|
||||
from basic_memory.cli.commands.command_utils import run_with_cleanup
|
||||
from basic_memory.config import ConfigManager, get_project_config
|
||||
from basic_memory.importers.claude_conversations_importer import ClaudeConversationsImporter
|
||||
from basic_memory.markdown import EntityParser, MarkdownProcessor
|
||||
from basic_memory.services.file_service import FileService
|
||||
from loguru import logger
|
||||
from rich.console import Console
|
||||
from rich.panel import Panel
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from basic_memory.markdown import MarkdownProcessor
|
||||
from basic_memory.services.file_service import FileService
|
||||
|
||||
console = Console()
|
||||
|
||||
|
||||
async def get_importer_dependencies() -> Tuple[MarkdownProcessor, FileService]:
|
||||
"""Get MarkdownProcessor and FileService instances for importers."""
|
||||
# Deferred: the markdown/file-service stack pulls SQLAlchemy and must load
|
||||
# only when an import actually runs, not on every CLI start (#886).
|
||||
from basic_memory.markdown import EntityParser, MarkdownProcessor
|
||||
from basic_memory.services.file_service import FileService
|
||||
|
||||
config = get_project_config()
|
||||
app_config = ConfigManager().config
|
||||
entity_parser = EntityParser(config.home)
|
||||
@@ -57,6 +66,9 @@ def import_claude(
|
||||
markdown_processor, file_service = run_with_cleanup(get_importer_dependencies())
|
||||
|
||||
# Create the importer
|
||||
# Deferred: importer stack loads at import-command run time only (#886).
|
||||
from basic_memory.importers.claude_conversations_importer import ClaudeConversationsImporter
|
||||
|
||||
importer = ClaudeConversationsImporter(
|
||||
config.home, markdown_processor, file_service, project_name=config.name
|
||||
)
|
||||
|
||||
@@ -1,25 +1,34 @@
|
||||
"""Import command for basic-memory CLI to import project data from Claude.ai."""
|
||||
|
||||
# PEP 563 lazy annotations keep heavy importer types out of module import (#886).
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from pathlib import Path
|
||||
from typing import Annotated, Tuple
|
||||
from typing import TYPE_CHECKING, Annotated, Tuple
|
||||
|
||||
import typer
|
||||
from basic_memory.cli.app import claude_app
|
||||
from basic_memory.cli.commands.command_utils import run_with_cleanup
|
||||
from basic_memory.config import ConfigManager, get_project_config
|
||||
from basic_memory.importers.claude_projects_importer import ClaudeProjectsImporter
|
||||
from basic_memory.markdown import EntityParser, MarkdownProcessor
|
||||
from basic_memory.services.file_service import FileService
|
||||
from loguru import logger
|
||||
from rich.console import Console
|
||||
from rich.panel import Panel
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from basic_memory.markdown import MarkdownProcessor
|
||||
from basic_memory.services.file_service import FileService
|
||||
|
||||
console = Console()
|
||||
|
||||
|
||||
async def get_importer_dependencies() -> Tuple[MarkdownProcessor, FileService]:
|
||||
"""Get MarkdownProcessor and FileService instances for importers."""
|
||||
# Deferred: the markdown/file-service stack pulls SQLAlchemy and must load
|
||||
# only when an import actually runs, not on every CLI start (#886).
|
||||
from basic_memory.markdown import EntityParser, MarkdownProcessor
|
||||
from basic_memory.services.file_service import FileService
|
||||
|
||||
config = get_project_config()
|
||||
app_config = ConfigManager().config
|
||||
entity_parser = EntityParser(config.home)
|
||||
@@ -56,6 +65,9 @@ def import_projects(
|
||||
markdown_processor, file_service = run_with_cleanup(get_importer_dependencies())
|
||||
|
||||
# Create the importer
|
||||
# Deferred: importer stack loads at import-command run time only (#886).
|
||||
from basic_memory.importers.claude_projects_importer import ClaudeProjectsImporter
|
||||
|
||||
importer = ClaudeProjectsImporter(
|
||||
config.home, markdown_processor, file_service, project_name=config.name
|
||||
)
|
||||
|
||||
@@ -1,25 +1,34 @@
|
||||
"""Import command for basic-memory CLI to import from JSON memory format."""
|
||||
|
||||
# PEP 563 lazy annotations keep heavy importer types out of module import (#886).
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from pathlib import Path
|
||||
from typing import Annotated, Tuple
|
||||
from typing import TYPE_CHECKING, Annotated, Tuple
|
||||
|
||||
import typer
|
||||
from basic_memory.cli.app import import_app
|
||||
from basic_memory.cli.commands.command_utils import run_with_cleanup
|
||||
from basic_memory.config import ConfigManager, get_project_config
|
||||
from basic_memory.importers.memory_json_importer import MemoryJsonImporter
|
||||
from basic_memory.markdown import EntityParser, MarkdownProcessor
|
||||
from basic_memory.services.file_service import FileService
|
||||
from loguru import logger
|
||||
from rich.console import Console
|
||||
from rich.panel import Panel
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from basic_memory.markdown import MarkdownProcessor
|
||||
from basic_memory.services.file_service import FileService
|
||||
|
||||
console = Console()
|
||||
|
||||
|
||||
async def get_importer_dependencies() -> Tuple[MarkdownProcessor, FileService]:
|
||||
"""Get MarkdownProcessor and FileService instances for importers."""
|
||||
# Deferred: the markdown/file-service stack pulls SQLAlchemy and must load
|
||||
# only when an import actually runs, not on every CLI start (#886).
|
||||
from basic_memory.markdown import EntityParser, MarkdownProcessor
|
||||
from basic_memory.services.file_service import FileService
|
||||
|
||||
config = get_project_config()
|
||||
app_config = ConfigManager().config
|
||||
entity_parser = EntityParser(config.home)
|
||||
@@ -55,6 +64,9 @@ def memory_json(
|
||||
markdown_processor, file_service = run_with_cleanup(get_importer_dependencies())
|
||||
|
||||
# Create the importer
|
||||
# Deferred: importer stack loads at import-command run time only (#886).
|
||||
from basic_memory.importers.memory_json_importer import MemoryJsonImporter
|
||||
|
||||
importer = MemoryJsonImporter(
|
||||
config.home, markdown_processor, file_service, project_name=config.name
|
||||
)
|
||||
|
||||
@@ -0,0 +1,76 @@
|
||||
"""Install the bundled man pages so `man bm` works."""
|
||||
|
||||
import shutil
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
from typing import Annotated, Optional
|
||||
|
||||
import typer
|
||||
from rich.console import Console
|
||||
|
||||
from basic_memory.cli.app import app
|
||||
|
||||
console = Console()
|
||||
|
||||
man_app = typer.Typer(help="Manage the bm man pages.")
|
||||
app.add_typer(man_app, name="man")
|
||||
|
||||
# Bundled groff sources ship inside the package (src/basic_memory/man).
|
||||
_MAN_SOURCE_DIR = Path(__file__).parent.parent.parent / "man"
|
||||
|
||||
|
||||
def _default_man_root() -> Path:
|
||||
# Why ~/.local/share/man: manpath(1) derives man directories from PATH
|
||||
# entries on both man-db (Linux) and BSD man (macOS), so ~/.local/bin on
|
||||
# PATH — the pipx/uv tool layout — makes this root searchable without any
|
||||
# MANPATH configuration.
|
||||
return Path.home() / ".local" / "share" / "man"
|
||||
|
||||
|
||||
def _man_root_on_manpath(man_root: Path) -> Optional[bool]:
|
||||
"""Best-effort check whether man(1) will search man_root; None if unknown."""
|
||||
try:
|
||||
result = subprocess.run(["manpath"], capture_output=True, text=True, timeout=5)
|
||||
except (FileNotFoundError, subprocess.TimeoutExpired):
|
||||
return None
|
||||
if result.returncode != 0:
|
||||
return None
|
||||
paths = [entry.rstrip("/") for entry in result.stdout.strip().split(":") if entry]
|
||||
return str(man_root).rstrip("/") in paths
|
||||
|
||||
|
||||
@man_app.command()
|
||||
def install(
|
||||
directory: Annotated[
|
||||
Optional[Path],
|
||||
typer.Option(
|
||||
"--dir",
|
||||
help="Man root to install into (default: ~/.local/share/man)",
|
||||
),
|
||||
] = None,
|
||||
) -> None:
|
||||
"""Install the bm man pages, then try `man bm`."""
|
||||
man_root = (directory or _default_man_root()).expanduser()
|
||||
man1 = man_root / "man1"
|
||||
man1.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
pages = sorted(_MAN_SOURCE_DIR.glob("*.1"))
|
||||
if not pages: # pragma: no cover - broken packaging, not a runtime state
|
||||
console.print("[red]No bundled man pages found — broken installation[/red]")
|
||||
raise typer.Exit(1)
|
||||
|
||||
for page in pages:
|
||||
shutil.copyfile(page, man1 / page.name)
|
||||
console.print(f"installed {man1 / page.name}")
|
||||
|
||||
# Trigger: the chosen root is provably absent from manpath output.
|
||||
# Why: a silent install into an unsearched directory looks like success
|
||||
# but `man bm` still fails; say so and hand over the one-line fix.
|
||||
# Outcome: actionable hint; unknown (None) stays quiet to avoid false alarms.
|
||||
if _man_root_on_manpath(man_root) is False:
|
||||
console.print(
|
||||
f"\n[yellow]{man_root} is not on your manpath.[/yellow] Add it with:\n"
|
||||
f' export MANPATH="{man_root}:$MANPATH"'
|
||||
)
|
||||
|
||||
console.print("\nTry: [bold]man bm[/bold]")
|
||||
@@ -5,7 +5,6 @@ from typing import Annotated, Optional
|
||||
|
||||
import typer
|
||||
from loguru import logger
|
||||
from mcp.server.fastmcp.exceptions import ToolError
|
||||
from rich.console import Console
|
||||
from rich.table import Table
|
||||
|
||||
@@ -50,6 +49,9 @@ def orphans(
|
||||
"""
|
||||
from basic_memory.cli.commands.command_utils import run_with_cleanup
|
||||
|
||||
# Deferred: ToolError lives in the mcp SDK, which must not load at CLI startup (#886).
|
||||
from mcp.server.fastmcp.exceptions import ToolError
|
||||
|
||||
try:
|
||||
validate_routing_flags(local, cloud)
|
||||
with force_routing(local=local, cloud=cloud):
|
||||
|
||||
@@ -1050,12 +1050,13 @@ async def _detach_local_project_row(app_config: BasicMemoryConfig, name: str) ->
|
||||
db_type=db.DatabaseType.FILESYSTEM,
|
||||
)
|
||||
try:
|
||||
repo = ProjectRepository(session_maker)
|
||||
existing = await repo.get_by_name(name)
|
||||
if existing is None:
|
||||
return False
|
||||
await repo.delete(existing.id)
|
||||
return True
|
||||
repo = ProjectRepository()
|
||||
async with db.scoped_session(session_maker) as session:
|
||||
existing = await repo.get_by_name(session, name)
|
||||
if existing is None:
|
||||
return False
|
||||
await repo.delete(session, existing.id)
|
||||
return True
|
||||
finally:
|
||||
# CLI-only: safe to tear down the global DB singleton here since
|
||||
# set-cloud/set-local never run inside a long-lived MCP/API server.
|
||||
@@ -1082,20 +1083,22 @@ async def _attach_local_project_row(app_config: BasicMemoryConfig, name: str, pa
|
||||
db_type=db.DatabaseType.FILESYSTEM,
|
||||
)
|
||||
try:
|
||||
repo = ProjectRepository(session_maker)
|
||||
existing = await repo.get_by_name(name)
|
||||
if existing is None:
|
||||
await repo.create(
|
||||
{
|
||||
"name": name,
|
||||
"path": path,
|
||||
"permalink": generate_permalink(name),
|
||||
"is_active": True,
|
||||
}
|
||||
)
|
||||
return
|
||||
if existing.path != path:
|
||||
await repo.update_path(existing.id, path)
|
||||
repo = ProjectRepository()
|
||||
async with db.scoped_session(session_maker) as session:
|
||||
existing = await repo.get_by_name(session, name)
|
||||
if existing is None:
|
||||
await repo.create(
|
||||
session,
|
||||
{
|
||||
"name": name,
|
||||
"path": path,
|
||||
"permalink": generate_permalink(name),
|
||||
"is_active": True,
|
||||
},
|
||||
)
|
||||
return
|
||||
if existing.path != path:
|
||||
await repo.update_path(session, existing.id, path)
|
||||
finally:
|
||||
# CLI-only: safe to tear down the global DB singleton here since
|
||||
# set-cloud/set-local never run inside a long-lived MCP/API server.
|
||||
|
||||
@@ -20,9 +20,10 @@ from basic_memory.cli.app import app
|
||||
from basic_memory.cli.commands.command_utils import run_with_cleanup
|
||||
from basic_memory.cli.commands.routing import force_routing, validate_routing_flags
|
||||
from basic_memory.config import ConfigManager
|
||||
from basic_memory.mcp.tools import schema_diff as mcp_schema_diff
|
||||
from basic_memory.mcp.tools import schema_infer as mcp_schema_infer
|
||||
from basic_memory.mcp.tools import schema_validate as mcp_schema_validate
|
||||
|
||||
# MCP tool functions are imported inside each command: importing
|
||||
# basic_memory.mcp.tools loads the entire tool stack (fastmcp, mcp SDK,
|
||||
# SQLAlchemy), which would slow every CLI invocation, including --help (#886).
|
||||
|
||||
console = Console()
|
||||
|
||||
@@ -189,6 +190,9 @@ def validate(
|
||||
Use --local to force local routing when cloud mode is enabled.
|
||||
Use --cloud to force cloud routing when cloud mode is disabled.
|
||||
"""
|
||||
# Deferred: loading the MCP tool stack at module import slows CLI startup (#886).
|
||||
from basic_memory.mcp.tools import schema_validate as mcp_schema_validate
|
||||
|
||||
try:
|
||||
validate_routing_flags(local, cloud)
|
||||
project_name = _resolve_project_name(project)
|
||||
@@ -272,6 +276,9 @@ def infer(
|
||||
Use --local to force local routing when cloud mode is enabled.
|
||||
Use --cloud to force cloud routing when cloud mode is disabled.
|
||||
"""
|
||||
# Deferred: loading the MCP tool stack at module import slows CLI startup (#886).
|
||||
from basic_memory.mcp.tools import schema_infer as mcp_schema_infer
|
||||
|
||||
try:
|
||||
validate_routing_flags(local, cloud)
|
||||
project_name = _resolve_project_name(project)
|
||||
@@ -352,6 +359,9 @@ def diff(
|
||||
Use --local to force local routing when cloud mode is enabled.
|
||||
Use --cloud to force cloud routing when cloud mode is disabled.
|
||||
"""
|
||||
# Deferred: loading the MCP tool stack at module import slows CLI startup (#886).
|
||||
from basic_memory.mcp.tools import schema_diff as mcp_schema_diff
|
||||
|
||||
try:
|
||||
validate_routing_flags(local, cloud)
|
||||
project_name = _resolve_project_name(project)
|
||||
|
||||
@@ -5,7 +5,6 @@ import json
|
||||
import time
|
||||
from typing import Annotated, Dict, Optional, Set
|
||||
|
||||
from mcp.server.fastmcp.exceptions import ToolError
|
||||
import typer
|
||||
from loguru import logger
|
||||
from rich.console import Console
|
||||
@@ -190,9 +189,16 @@ async def run_status(
|
||||
if sync_report.total == 0:
|
||||
return project_item.name, sync_report
|
||||
if time.monotonic() >= deadline:
|
||||
# Why the hint: indexing is done by the sync coordinator, which
|
||||
# only runs inside a live server (bm mcp / hosted API). In a
|
||||
# CLI-only session nothing will ever drain the pending count,
|
||||
# so this wait cannot succeed — point at the command that
|
||||
# actually indexes (#959).
|
||||
raise StatusTimeout(
|
||||
f"Timed out after {timeout:g}s waiting for '{project_item.name}' "
|
||||
f"to finish indexing ({sync_report.total} pending change(s) remaining)."
|
||||
f"to finish indexing ({sync_report.total} pending change(s) remaining). "
|
||||
f"If no Basic Memory server is running, pending changes are never "
|
||||
f"indexed — run 'bm reindex --project {project_item.name}' instead."
|
||||
)
|
||||
await asyncio.sleep(poll_interval)
|
||||
|
||||
@@ -224,6 +230,9 @@ def status(
|
||||
"""
|
||||
from basic_memory.cli.commands.command_utils import run_with_cleanup
|
||||
|
||||
# Deferred: ToolError lives in the mcp SDK, which must not load at CLI startup (#886).
|
||||
from mcp.server.fastmcp.exceptions import ToolError
|
||||
|
||||
# Trigger: --wait with a negative --timeout
|
||||
# Why: a negative deadline times out on the very first poll, producing a confusing
|
||||
# "Timed out after -5s" message instead of flagging the bad input. Raised
|
||||
|
||||
@@ -14,18 +14,10 @@ from loguru import logger
|
||||
from basic_memory.cli.app import app
|
||||
from basic_memory.cli.commands.command_utils import run_with_cleanup
|
||||
from basic_memory.cli.commands.routing import force_routing, validate_routing_flags
|
||||
from basic_memory.mcp.tools import build_context as mcp_build_context
|
||||
from basic_memory.mcp.tools import delete_note as mcp_delete_note
|
||||
from basic_memory.mcp.tools import edit_note as mcp_edit_note
|
||||
from basic_memory.mcp.tools import list_memory_projects as mcp_list_projects
|
||||
from basic_memory.mcp.tools import list_workspaces as mcp_list_workspaces
|
||||
from basic_memory.mcp.tools import read_note as mcp_read_note
|
||||
from basic_memory.mcp.tools import recent_activity as mcp_recent_activity
|
||||
from basic_memory.mcp.tools import schema_diff as mcp_schema_diff
|
||||
from basic_memory.mcp.tools import schema_infer as mcp_schema_infer
|
||||
from basic_memory.mcp.tools import schema_validate as mcp_schema_validate
|
||||
from basic_memory.mcp.tools import search_notes as mcp_search
|
||||
from basic_memory.mcp.tools import write_note as mcp_write_note
|
||||
|
||||
# MCP tool functions are imported inside each command: importing
|
||||
# basic_memory.mcp.tools loads the entire tool stack (fastmcp, mcp SDK,
|
||||
# SQLAlchemy), which would slow every CLI invocation, including --help (#886).
|
||||
|
||||
tool_app = typer.Typer()
|
||||
app.add_typer(tool_app, name="tool", help="Access to MCP tools via CLI")
|
||||
@@ -120,6 +112,9 @@ def write_note(
|
||||
bm tool write-note --title "My Note" --folder "notes" --overwrite
|
||||
bm tool write-note --title "My Note" --folder "notes" --local
|
||||
"""
|
||||
# Deferred: loading the MCP tool stack at module import slows CLI startup (#886).
|
||||
from basic_memory.mcp.tools import write_note as mcp_write_note
|
||||
|
||||
try:
|
||||
validate_routing_flags(local, cloud)
|
||||
|
||||
@@ -206,6 +201,9 @@ def read_note(
|
||||
bm tool read-note my-note
|
||||
bm tool read-note my-note --include-frontmatter
|
||||
"""
|
||||
# Deferred: loading the MCP tool stack at module import slows CLI startup (#886).
|
||||
from basic_memory.mcp.tools import read_note as mcp_read_note
|
||||
|
||||
try:
|
||||
validate_routing_flags(local, cloud)
|
||||
|
||||
@@ -272,6 +270,9 @@ def delete_note(
|
||||
bm tool delete-note notes/old-draft
|
||||
bm tool delete-note docs/archive --is-directory
|
||||
"""
|
||||
# Deferred: loading the MCP tool stack at module import slows CLI startup (#886).
|
||||
from basic_memory.mcp.tools import delete_note as mcp_delete_note
|
||||
|
||||
try:
|
||||
validate_routing_flags(local, cloud)
|
||||
|
||||
@@ -346,6 +347,9 @@ def edit_note(
|
||||
bm tool edit-note my-note --operation find_replace --find-text "old" --content "new"
|
||||
bm tool edit-note my-note --operation replace_section --section "## Notes" --content "updated"
|
||||
"""
|
||||
# Deferred: loading the MCP tool stack at module import slows CLI startup (#886).
|
||||
from basic_memory.mcp.tools import edit_note as mcp_edit_note
|
||||
|
||||
try:
|
||||
validate_routing_flags(local, cloud)
|
||||
|
||||
@@ -413,6 +417,9 @@ def build_context(
|
||||
bm tool build-context memory://specs/search
|
||||
bm tool build-context specs/search --depth 2 --timeframe 30d
|
||||
"""
|
||||
# Deferred: loading the MCP tool stack at module import slows CLI startup (#886).
|
||||
from basic_memory.mcp.tools import build_context as mcp_build_context
|
||||
|
||||
try:
|
||||
validate_routing_flags(local, cloud)
|
||||
|
||||
@@ -476,6 +483,9 @@ def recent_activity(
|
||||
bm tool recent-activity --timeframe 30d --page-size 20
|
||||
bm tool recent-activity --type entity --type observation
|
||||
"""
|
||||
# Deferred: loading the MCP tool stack at module import slows CLI startup (#886).
|
||||
from basic_memory.mcp.tools import recent_activity as mcp_recent_activity
|
||||
|
||||
try:
|
||||
validate_routing_flags(local, cloud)
|
||||
|
||||
@@ -582,6 +592,9 @@ def search_notes(
|
||||
bm tool search-notes --meta status=draft
|
||||
bm tool search-notes "auth" --entity-type observation --category requirement
|
||||
"""
|
||||
# Deferred: loading the MCP tool stack at module import slows CLI startup (#886).
|
||||
from basic_memory.mcp.tools import search_notes as mcp_search
|
||||
|
||||
try:
|
||||
validate_routing_flags(local, cloud)
|
||||
|
||||
@@ -687,6 +700,9 @@ def list_projects(
|
||||
bm tool list-projects
|
||||
bm tool list-projects --local
|
||||
"""
|
||||
# Deferred: loading the MCP tool stack at module import slows CLI startup (#886).
|
||||
from basic_memory.mcp.tools import list_memory_projects as mcp_list_projects
|
||||
|
||||
try:
|
||||
validate_routing_flags(local, cloud)
|
||||
|
||||
@@ -720,6 +736,9 @@ def list_workspaces(
|
||||
bm tool list-workspaces
|
||||
bm tool list-workspaces --cloud
|
||||
"""
|
||||
# Deferred: loading the MCP tool stack at module import slows CLI startup (#886).
|
||||
from basic_memory.mcp.tools import list_workspaces as mcp_list_workspaces
|
||||
|
||||
try:
|
||||
validate_routing_flags(local, cloud)
|
||||
|
||||
@@ -772,6 +791,9 @@ def schema_validate(
|
||||
bm tool schema-validate people/ada-lovelace.md
|
||||
bm tool schema-validate --project research
|
||||
"""
|
||||
# Deferred: loading the MCP tool stack at module import slows CLI startup (#886).
|
||||
from basic_memory.mcp.tools import schema_validate as mcp_schema_validate
|
||||
|
||||
try:
|
||||
validate_routing_flags(local, cloud)
|
||||
|
||||
@@ -840,6 +862,9 @@ def schema_infer(
|
||||
bm tool schema-infer meeting --threshold 0.5
|
||||
bm tool schema-infer person --project research
|
||||
"""
|
||||
# Deferred: loading the MCP tool stack at module import slows CLI startup (#886).
|
||||
from basic_memory.mcp.tools import schema_infer as mcp_schema_infer
|
||||
|
||||
try:
|
||||
validate_routing_flags(local, cloud)
|
||||
|
||||
@@ -896,6 +921,9 @@ def schema_diff(
|
||||
bm tool schema-diff person
|
||||
bm tool schema-diff person --project research
|
||||
"""
|
||||
# Deferred: loading the MCP tool stack at module import slows CLI startup (#886).
|
||||
from basic_memory.mcp.tools import schema_diff as mcp_schema_diff
|
||||
|
||||
try:
|
||||
validate_routing_flags(local, cloud)
|
||||
|
||||
|
||||
@@ -24,6 +24,7 @@ if not _version_only_invocation(sys.argv[1:]):
|
||||
import_claude_conversations,
|
||||
import_claude_projects,
|
||||
import_memory_json,
|
||||
man,
|
||||
mcp,
|
||||
orphans,
|
||||
project,
|
||||
|
||||
@@ -4,6 +4,7 @@ import importlib.util
|
||||
import json
|
||||
import os
|
||||
import shutil
|
||||
import threading
|
||||
from dataclasses import dataclass
|
||||
from datetime import datetime
|
||||
from enum import Enum
|
||||
@@ -657,6 +658,26 @@ class BasicMemoryConfig(BaseSettings):
|
||||
entry = self.projects.get(project_name)
|
||||
return entry.mode if entry else ProjectMode.CLOUD
|
||||
|
||||
def is_locally_syncable(self, project_name: str, project_path: str) -> bool:
|
||||
"""Whether a project should be synced/watched on the local filesystem.
|
||||
|
||||
Both conditions are required (issue #949):
|
||||
|
||||
* The project is present in config. Config is the source of truth, so a
|
||||
stale database row that was removed from config — but whose deletion
|
||||
has not yet been reconciled, or whose reconciliation failed — must
|
||||
not be synced even though it still has a real directory on disk.
|
||||
* Its path is absolute. An empty or relative path resolves against the
|
||||
process cwd, so syncing it would adopt whatever directory the server
|
||||
was launched from as the project root and mutate unrelated files.
|
||||
|
||||
Cloud-only projects (empty/slug path) and cloud projects with a real
|
||||
local bisync copy (absolute path) are handled correctly by these two
|
||||
conditions, so no separate mode check is needed.
|
||||
"""
|
||||
entry = self.projects.get(project_name)
|
||||
return entry is not None and Path(project_path).is_absolute()
|
||||
|
||||
def set_project_mode(self, project_name: str, mode: ProjectMode) -> None:
|
||||
"""Set the routing mode for a project.
|
||||
|
||||
@@ -1099,8 +1120,21 @@ def save_basic_memory_config(file_path: Path, config: BasicMemoryConfig) -> None
|
||||
_secure_config_dir(file_path.parent)
|
||||
# Use model_dump with mode='json' to serialize datetime objects properly
|
||||
config_dict = config.model_dump(mode="json")
|
||||
file_path.write_text(json.dumps(config_dict, indent=2))
|
||||
_secure_config_file(file_path)
|
||||
# Trigger: long-lived readers (MCP stdio server config reload, background
|
||||
# auto-update threads) re-read config.json whenever its mtime changes,
|
||||
# concurrently with CLI commands saving it.
|
||||
# Why: writing the destination in place truncates it first, so a concurrent
|
||||
# reader can observe empty/partial JSON and load_config() exits the process.
|
||||
# Outcome: write a sibling temp file (unique per process/thread so parallel
|
||||
# savers cannot interleave) and publish atomically via os.replace — readers
|
||||
# always see either the old or the new complete document. (#940)
|
||||
tmp_path = file_path.parent / f"{file_path.name}.{os.getpid()}.{threading.get_ident()}.tmp"
|
||||
try:
|
||||
tmp_path.write_text(json.dumps(config_dict, indent=2))
|
||||
_secure_config_file(tmp_path)
|
||||
os.replace(tmp_path, file_path)
|
||||
finally:
|
||||
tmp_path.unlink(missing_ok=True)
|
||||
except Exception as e: # pragma: no cover
|
||||
logger.error(f"Failed to save config: {e}")
|
||||
|
||||
|
||||
+26
-15
@@ -19,7 +19,7 @@ from sqlalchemy.ext.asyncio import (
|
||||
AsyncEngine,
|
||||
async_scoped_session,
|
||||
)
|
||||
from sqlalchemy.pool import NullPool
|
||||
from sqlalchemy.pool import AsyncAdaptedQueuePool, NullPool
|
||||
|
||||
from basic_memory.repository.postgres_search_repository import PostgresSearchRepository
|
||||
from basic_memory.repository.sqlite_search_repository import SQLiteSearchRepository
|
||||
@@ -206,29 +206,40 @@ def _create_sqlite_engine(db_url: str, db_type: DatabaseType) -> AsyncEngine:
|
||||
Configured async engine for SQLite
|
||||
"""
|
||||
# Configure connection args with Windows-specific settings
|
||||
connect_args: dict[str, bool | float | None] = {"check_same_thread": False}
|
||||
connect_args: dict[str, bool | float] = {"check_same_thread": False}
|
||||
|
||||
# Add Windows-specific parameters to improve reliability
|
||||
if os.name == "nt": # Windows
|
||||
connect_args.update(
|
||||
{
|
||||
"timeout": 30.0, # Increase timeout to 30 seconds for Windows
|
||||
"isolation_level": None, # Use autocommit mode
|
||||
}
|
||||
)
|
||||
|
||||
if db_type == DatabaseType.MEMORY:
|
||||
# Trigger: an in-memory SQLite URL would default to StaticPool, which hands the
|
||||
# same DBAPI connection to every concurrently checked-out session.
|
||||
# Why: concurrent asyncio tasks then share one transaction scope — a rollback
|
||||
# issued by one session (scoped_session exception handling or the pool's
|
||||
# reset-on-return) silently destroys another session's uncommitted writes (#940).
|
||||
# Outcome: a single-connection blocking queue pool keeps the in-memory database
|
||||
# alive for the engine's lifetime while serializing sessions at transaction
|
||||
# granularity, restoring the isolation the repositories assume.
|
||||
engine = create_async_engine(
|
||||
db_url,
|
||||
connect_args=connect_args,
|
||||
poolclass=AsyncAdaptedQueuePool,
|
||||
pool_size=1,
|
||||
max_overflow=0,
|
||||
)
|
||||
elif os.name == "nt":
|
||||
# Use NullPool for Windows filesystem databases to avoid connection pooling issues
|
||||
# Important: Do NOT use NullPool for in-memory databases as it will destroy the database
|
||||
# between connections
|
||||
if db_type == DatabaseType.FILESYSTEM:
|
||||
engine = create_async_engine(
|
||||
db_url,
|
||||
connect_args=connect_args,
|
||||
poolclass=NullPool, # Disable connection pooling on Windows
|
||||
echo=False,
|
||||
)
|
||||
else:
|
||||
# In-memory databases need connection pooling to maintain state
|
||||
engine = create_async_engine(db_url, connect_args=connect_args)
|
||||
engine = create_async_engine(
|
||||
db_url,
|
||||
connect_args=connect_args,
|
||||
poolclass=NullPool, # Disable connection pooling on Windows
|
||||
echo=False,
|
||||
)
|
||||
else:
|
||||
engine = create_async_engine(db_url, connect_args=connect_args)
|
||||
|
||||
|
||||
@@ -25,6 +25,8 @@ from basic_memory.deps.db import (
|
||||
EngineFactoryDep,
|
||||
get_session_maker,
|
||||
SessionMakerDep,
|
||||
get_session,
|
||||
SessionDep,
|
||||
)
|
||||
|
||||
from basic_memory.deps.projects import (
|
||||
@@ -169,6 +171,8 @@ __all__ = [
|
||||
"EngineFactoryDep",
|
||||
"get_session_maker",
|
||||
"SessionMakerDep",
|
||||
"get_session",
|
||||
"SessionDep",
|
||||
# Projects
|
||||
"get_project_repository",
|
||||
"ProjectRepositoryDep",
|
||||
|
||||
@@ -5,6 +5,7 @@ This module provides database-related dependencies:
|
||||
- Session dependencies for request handling
|
||||
"""
|
||||
|
||||
from collections.abc import AsyncGenerator
|
||||
from typing import Annotated
|
||||
|
||||
from fastapi import Depends, Request
|
||||
@@ -54,3 +55,12 @@ async def get_session_maker(engine_factory: EngineFactoryDep) -> async_sessionma
|
||||
|
||||
|
||||
SessionMakerDep = Annotated[async_sessionmaker, Depends(get_session_maker)]
|
||||
|
||||
|
||||
async def get_session(session_maker: SessionMakerDep) -> AsyncGenerator[AsyncSession, None]:
|
||||
"""Yield a request-scoped SQLAlchemy session."""
|
||||
async with db.scoped_session(session_maker) as session:
|
||||
yield session
|
||||
|
||||
|
||||
SessionDep = Annotated[AsyncSession, Depends(get_session)]
|
||||
|
||||
@@ -12,6 +12,7 @@ from typing import Annotated
|
||||
|
||||
from fastapi import Depends, HTTPException, Path, status
|
||||
|
||||
from basic_memory import db
|
||||
from basic_memory.config import ProjectConfig
|
||||
from basic_memory.deps.db import SessionMakerDep
|
||||
from basic_memory.repository.project_repository import ProjectRepository
|
||||
@@ -21,11 +22,9 @@ from basic_memory.utils import generate_permalink
|
||||
# --- Project Repository ---
|
||||
|
||||
|
||||
async def get_project_repository(
|
||||
session_maker: SessionMakerDep,
|
||||
) -> ProjectRepository:
|
||||
async def get_project_repository() -> ProjectRepository:
|
||||
"""Get the project repository."""
|
||||
return ProjectRepository(session_maker)
|
||||
return ProjectRepository()
|
||||
|
||||
|
||||
ProjectRepositoryDep = Annotated[ProjectRepository, Depends(get_project_repository)]
|
||||
@@ -41,6 +40,7 @@ ProjectPathDep = Annotated[str, Path()]
|
||||
|
||||
|
||||
async def get_project_id(
|
||||
session_maker: SessionMakerDep,
|
||||
project_repository: ProjectRepositoryDep,
|
||||
project: ProjectPathDep,
|
||||
) -> int:
|
||||
@@ -61,14 +61,17 @@ async def get_project_id(
|
||||
"""
|
||||
# Convert project name to permalink for lookup
|
||||
project_permalink = generate_permalink(str(project))
|
||||
project_obj = await project_repository.get_by_permalink(project_permalink)
|
||||
if project_obj:
|
||||
return project_obj.id
|
||||
async with db.scoped_session(session_maker) as session:
|
||||
project_obj = await project_repository.get_by_permalink(session, project_permalink)
|
||||
if project_obj:
|
||||
return project_obj.id
|
||||
|
||||
# Try by name if permalink lookup fails
|
||||
project_obj = await project_repository.get_by_name(str(project)) # pragma: no cover
|
||||
if project_obj: # pragma: no cover
|
||||
return project_obj.id
|
||||
# Try by name if permalink lookup fails
|
||||
project_obj = await project_repository.get_by_name(
|
||||
session, str(project)
|
||||
) # pragma: no cover
|
||||
if project_obj: # pragma: no cover
|
||||
return project_obj.id
|
||||
|
||||
# Not found
|
||||
raise HTTPException( # pragma: no cover
|
||||
@@ -83,7 +86,9 @@ ProjectIdDep = Annotated[int, Depends(get_project_id)]
|
||||
|
||||
|
||||
async def get_project_config(
|
||||
project: ProjectPathDep, project_repository: ProjectRepositoryDep
|
||||
session_maker: SessionMakerDep,
|
||||
project: ProjectPathDep,
|
||||
project_repository: ProjectRepositoryDep,
|
||||
) -> ProjectConfig: # pragma: no cover
|
||||
"""Get the current project referenced from request state.
|
||||
|
||||
@@ -99,9 +104,10 @@ async def get_project_config(
|
||||
"""
|
||||
# Convert project name to permalink for lookup
|
||||
project_permalink = generate_permalink(str(project))
|
||||
project_obj = await project_repository.get_by_permalink(project_permalink)
|
||||
if project_obj:
|
||||
return ProjectConfig(name=project_obj.name, home=pathlib.Path(project_obj.path))
|
||||
async with db.scoped_session(session_maker) as session:
|
||||
project_obj = await project_repository.get_by_permalink(session, project_permalink)
|
||||
if project_obj:
|
||||
return ProjectConfig(name=project_obj.name, home=pathlib.Path(project_obj.path))
|
||||
|
||||
# Not found
|
||||
raise HTTPException( # pragma: no cover
|
||||
@@ -116,6 +122,7 @@ ProjectConfigDep = Annotated[ProjectConfig, Depends(get_project_config)]
|
||||
|
||||
|
||||
async def validate_project_id(
|
||||
session_maker: SessionMakerDep,
|
||||
project_id: int,
|
||||
project_repository: ProjectRepositoryDep,
|
||||
) -> int:
|
||||
@@ -134,12 +141,13 @@ async def validate_project_id(
|
||||
Raises:
|
||||
HTTPException: If project with that ID is not found
|
||||
"""
|
||||
project_obj = await project_repository.get_by_id(project_id)
|
||||
if not project_obj:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_404_NOT_FOUND,
|
||||
detail=f"Project with ID {project_id} not found.",
|
||||
)
|
||||
async with db.scoped_session(session_maker) as session:
|
||||
project_obj = await project_repository.get_by_id(session, project_id)
|
||||
if not project_obj:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_404_NOT_FOUND,
|
||||
detail=f"Project with ID {project_id} not found.",
|
||||
)
|
||||
return project_id
|
||||
|
||||
|
||||
@@ -147,7 +155,9 @@ ProjectIdPathDep = Annotated[int, Depends(validate_project_id)]
|
||||
|
||||
|
||||
async def get_project_config_v2(
|
||||
project_id: ProjectIdPathDep, project_repository: ProjectRepositoryDep
|
||||
session_maker: SessionMakerDep,
|
||||
project_id: ProjectIdPathDep,
|
||||
project_repository: ProjectRepositoryDep,
|
||||
) -> ProjectConfig: # pragma: no cover
|
||||
"""Get the project config for v2 API (uses integer project_id from path).
|
||||
|
||||
@@ -161,9 +171,10 @@ async def get_project_config_v2(
|
||||
Raises:
|
||||
HTTPException: If project is not found
|
||||
"""
|
||||
project_obj = await project_repository.get_by_id(project_id)
|
||||
if project_obj:
|
||||
return ProjectConfig(name=project_obj.name, home=pathlib.Path(project_obj.path))
|
||||
async with db.scoped_session(session_maker) as session:
|
||||
project_obj = await project_repository.get_by_id(session, project_id)
|
||||
if project_obj:
|
||||
return ProjectConfig(name=project_obj.name, home=pathlib.Path(project_obj.path))
|
||||
|
||||
# Not found (this should not happen since ProjectIdPathDep already validates existence)
|
||||
raise HTTPException( # pragma: no cover
|
||||
@@ -178,6 +189,7 @@ ProjectConfigV2Dep = Annotated[ProjectConfig, Depends(get_project_config_v2)]
|
||||
|
||||
|
||||
async def validate_project_external_id(
|
||||
session_maker: SessionMakerDep,
|
||||
project_id: str,
|
||||
project_repository: ProjectRepositoryDep,
|
||||
) -> int:
|
||||
@@ -196,12 +208,13 @@ async def validate_project_external_id(
|
||||
Raises:
|
||||
HTTPException: If project with that external_id is not found
|
||||
"""
|
||||
project_obj = await project_repository.get_by_external_id(project_id)
|
||||
if not project_obj:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_404_NOT_FOUND,
|
||||
detail=f"Project with external_id '{project_id}' not found.",
|
||||
)
|
||||
async with db.scoped_session(session_maker) as session:
|
||||
project_obj = await project_repository.get_by_external_id(session, project_id)
|
||||
if not project_obj:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_404_NOT_FOUND,
|
||||
detail=f"Project with external_id '{project_id}' not found.",
|
||||
)
|
||||
return project_obj.id
|
||||
|
||||
|
||||
@@ -209,7 +222,9 @@ ProjectExternalIdPathDep = Annotated[int, Depends(validate_project_external_id)]
|
||||
|
||||
|
||||
async def get_project_config_v2_external(
|
||||
project_id: ProjectExternalIdPathDep, project_repository: ProjectRepositoryDep
|
||||
session_maker: SessionMakerDep,
|
||||
project_id: ProjectExternalIdPathDep,
|
||||
project_repository: ProjectRepositoryDep,
|
||||
) -> ProjectConfig: # pragma: no cover
|
||||
"""Get the project config for v2 API (uses external_id UUID from path).
|
||||
|
||||
@@ -223,9 +238,10 @@ async def get_project_config_v2_external(
|
||||
Raises:
|
||||
HTTPException: If project is not found
|
||||
"""
|
||||
project_obj = await project_repository.get_by_id(project_id)
|
||||
if project_obj:
|
||||
return ProjectConfig(name=project_obj.name, home=pathlib.Path(project_obj.path))
|
||||
async with db.scoped_session(session_maker) as session:
|
||||
project_obj = await project_repository.get_by_id(session, project_id)
|
||||
if project_obj:
|
||||
return ProjectConfig(name=project_obj.name, home=pathlib.Path(project_obj.path))
|
||||
|
||||
# Not found (this should not happen since ProjectExternalIdPathDep already validates)
|
||||
raise HTTPException( # pragma: no cover
|
||||
|
||||
@@ -30,33 +30,30 @@ from basic_memory.repository.search_repository import SearchRepository, create_s
|
||||
|
||||
|
||||
async def get_entity_repository(
|
||||
session_maker: SessionMakerDep,
|
||||
project_id: ProjectIdDep,
|
||||
) -> EntityRepository:
|
||||
"""Create an EntityRepository instance for the current project."""
|
||||
return EntityRepository(session_maker, project_id=project_id)
|
||||
return EntityRepository(project_id=project_id)
|
||||
|
||||
|
||||
EntityRepositoryDep = Annotated[EntityRepository, Depends(get_entity_repository)]
|
||||
|
||||
|
||||
async def get_entity_repository_v2( # pragma: no cover
|
||||
session_maker: SessionMakerDep,
|
||||
project_id: ProjectIdPathDep,
|
||||
) -> EntityRepository:
|
||||
"""Create an EntityRepository instance for v2 API (uses integer project_id from path)."""
|
||||
return EntityRepository(session_maker, project_id=project_id)
|
||||
return EntityRepository(project_id=project_id)
|
||||
|
||||
|
||||
EntityRepositoryV2Dep = Annotated[EntityRepository, Depends(get_entity_repository_v2)]
|
||||
|
||||
|
||||
async def get_entity_repository_v2_external(
|
||||
session_maker: SessionMakerDep,
|
||||
project_id: ProjectExternalIdPathDep,
|
||||
) -> EntityRepository:
|
||||
"""Create an EntityRepository instance for v2 API (uses external_id from path)."""
|
||||
return EntityRepository(session_maker, project_id=project_id)
|
||||
return EntityRepository(project_id=project_id)
|
||||
|
||||
|
||||
EntityRepositoryV2ExternalDep = Annotated[
|
||||
@@ -68,22 +65,20 @@ EntityRepositoryV2ExternalDep = Annotated[
|
||||
|
||||
|
||||
async def get_observation_repository(
|
||||
session_maker: SessionMakerDep,
|
||||
project_id: ProjectIdDep,
|
||||
) -> ObservationRepository:
|
||||
"""Create an ObservationRepository instance for the current project."""
|
||||
return ObservationRepository(session_maker, project_id=project_id)
|
||||
return ObservationRepository(project_id=project_id)
|
||||
|
||||
|
||||
ObservationRepositoryDep = Annotated[ObservationRepository, Depends(get_observation_repository)]
|
||||
|
||||
|
||||
async def get_observation_repository_v2( # pragma: no cover
|
||||
session_maker: SessionMakerDep,
|
||||
project_id: ProjectIdPathDep,
|
||||
) -> ObservationRepository:
|
||||
"""Create an ObservationRepository instance for v2 API."""
|
||||
return ObservationRepository(session_maker, project_id=project_id)
|
||||
return ObservationRepository(project_id=project_id)
|
||||
|
||||
|
||||
ObservationRepositoryV2Dep = Annotated[
|
||||
@@ -92,11 +87,10 @@ ObservationRepositoryV2Dep = Annotated[
|
||||
|
||||
|
||||
async def get_observation_repository_v2_external(
|
||||
session_maker: SessionMakerDep,
|
||||
project_id: ProjectExternalIdPathDep,
|
||||
) -> ObservationRepository:
|
||||
"""Create an ObservationRepository instance for v2 API (uses external_id)."""
|
||||
return ObservationRepository(session_maker, project_id=project_id)
|
||||
return ObservationRepository(project_id=project_id)
|
||||
|
||||
|
||||
ObservationRepositoryV2ExternalDep = Annotated[
|
||||
@@ -108,33 +102,30 @@ ObservationRepositoryV2ExternalDep = Annotated[
|
||||
|
||||
|
||||
async def get_relation_repository(
|
||||
session_maker: SessionMakerDep,
|
||||
project_id: ProjectIdDep,
|
||||
) -> RelationRepository:
|
||||
"""Create a RelationRepository instance for the current project."""
|
||||
return RelationRepository(session_maker, project_id=project_id)
|
||||
return RelationRepository(project_id=project_id)
|
||||
|
||||
|
||||
RelationRepositoryDep = Annotated[RelationRepository, Depends(get_relation_repository)]
|
||||
|
||||
|
||||
async def get_relation_repository_v2( # pragma: no cover
|
||||
session_maker: SessionMakerDep,
|
||||
project_id: ProjectIdPathDep,
|
||||
) -> RelationRepository:
|
||||
"""Create a RelationRepository instance for v2 API."""
|
||||
return RelationRepository(session_maker, project_id=project_id)
|
||||
return RelationRepository(project_id=project_id)
|
||||
|
||||
|
||||
RelationRepositoryV2Dep = Annotated[RelationRepository, Depends(get_relation_repository_v2)]
|
||||
|
||||
|
||||
async def get_relation_repository_v2_external(
|
||||
session_maker: SessionMakerDep,
|
||||
project_id: ProjectExternalIdPathDep,
|
||||
) -> RelationRepository:
|
||||
"""Create a RelationRepository instance for v2 API (uses external_id)."""
|
||||
return RelationRepository(session_maker, project_id=project_id)
|
||||
return RelationRepository(project_id=project_id)
|
||||
|
||||
|
||||
RelationRepositoryV2ExternalDep = Annotated[
|
||||
|
||||
@@ -16,6 +16,7 @@ from fastapi import Depends
|
||||
from loguru import logger
|
||||
|
||||
from basic_memory.deps.config import AppConfigDep
|
||||
from basic_memory.deps.db import SessionMakerDep
|
||||
from basic_memory.deps.projects import (
|
||||
ProjectConfigDep,
|
||||
ProjectConfigV2Dep,
|
||||
@@ -159,9 +160,10 @@ async def get_search_service(
|
||||
search_repository: SearchRepositoryDep,
|
||||
entity_repository: EntityRepositoryDep,
|
||||
file_service: FileServiceDep,
|
||||
session_maker: SessionMakerDep,
|
||||
) -> SearchService:
|
||||
"""Create SearchService with dependencies."""
|
||||
return SearchService(search_repository, entity_repository, file_service)
|
||||
return SearchService(search_repository, entity_repository, file_service, session_maker)
|
||||
|
||||
|
||||
SearchServiceDep = Annotated[SearchService, Depends(get_search_service)]
|
||||
@@ -171,9 +173,10 @@ async def get_search_service_v2( # pragma: no cover
|
||||
search_repository: SearchRepositoryV2Dep,
|
||||
entity_repository: EntityRepositoryV2Dep,
|
||||
file_service: FileServiceV2Dep,
|
||||
session_maker: SessionMakerDep,
|
||||
) -> SearchService:
|
||||
"""Create SearchService for v2 API."""
|
||||
return SearchService(search_repository, entity_repository, file_service)
|
||||
return SearchService(search_repository, entity_repository, file_service, session_maker)
|
||||
|
||||
|
||||
SearchServiceV2Dep = Annotated[SearchService, Depends(get_search_service_v2)]
|
||||
@@ -183,9 +186,10 @@ async def get_search_service_v2_external(
|
||||
search_repository: SearchRepositoryV2ExternalDep,
|
||||
entity_repository: EntityRepositoryV2ExternalDep,
|
||||
file_service: FileServiceV2ExternalDep,
|
||||
session_maker: SessionMakerDep,
|
||||
) -> SearchService:
|
||||
"""Create SearchService for v2 API (uses external_id)."""
|
||||
return SearchService(search_repository, entity_repository, file_service)
|
||||
return SearchService(search_repository, entity_repository, file_service, session_maker)
|
||||
|
||||
|
||||
SearchServiceV2ExternalDep = Annotated[SearchService, Depends(get_search_service_v2_external)]
|
||||
@@ -195,27 +199,45 @@ SearchServiceV2ExternalDep = Annotated[SearchService, Depends(get_search_service
|
||||
|
||||
|
||||
async def get_link_resolver(
|
||||
entity_repository: EntityRepositoryDep, search_service: SearchServiceDep
|
||||
entity_repository: EntityRepositoryDep,
|
||||
search_service: SearchServiceDep,
|
||||
session_maker: SessionMakerDep,
|
||||
) -> LinkResolver:
|
||||
return LinkResolver(entity_repository=entity_repository, search_service=search_service)
|
||||
return LinkResolver(
|
||||
entity_repository=entity_repository,
|
||||
search_service=search_service,
|
||||
session_maker=session_maker,
|
||||
)
|
||||
|
||||
|
||||
LinkResolverDep = Annotated[LinkResolver, Depends(get_link_resolver)]
|
||||
|
||||
|
||||
async def get_link_resolver_v2( # pragma: no cover
|
||||
entity_repository: EntityRepositoryV2Dep, search_service: SearchServiceV2Dep
|
||||
entity_repository: EntityRepositoryV2Dep,
|
||||
search_service: SearchServiceV2Dep,
|
||||
session_maker: SessionMakerDep,
|
||||
) -> LinkResolver:
|
||||
return LinkResolver(entity_repository=entity_repository, search_service=search_service)
|
||||
return LinkResolver(
|
||||
entity_repository=entity_repository,
|
||||
search_service=search_service,
|
||||
session_maker=session_maker,
|
||||
)
|
||||
|
||||
|
||||
LinkResolverV2Dep = Annotated[LinkResolver, Depends(get_link_resolver_v2)]
|
||||
|
||||
|
||||
async def get_link_resolver_v2_external(
|
||||
entity_repository: EntityRepositoryV2ExternalDep, search_service: SearchServiceV2ExternalDep
|
||||
entity_repository: EntityRepositoryV2ExternalDep,
|
||||
search_service: SearchServiceV2ExternalDep,
|
||||
session_maker: SessionMakerDep,
|
||||
) -> LinkResolver:
|
||||
return LinkResolver(entity_repository=entity_repository, search_service=search_service)
|
||||
return LinkResolver(
|
||||
entity_repository=entity_repository,
|
||||
search_service=search_service,
|
||||
session_maker=session_maker,
|
||||
)
|
||||
|
||||
|
||||
LinkResolverV2ExternalDep = Annotated[LinkResolver, Depends(get_link_resolver_v2_external)]
|
||||
@@ -232,6 +254,7 @@ async def get_entity_service(
|
||||
file_service: FileServiceDep,
|
||||
link_resolver: LinkResolverDep,
|
||||
search_service: SearchServiceDep,
|
||||
session_maker: SessionMakerDep,
|
||||
app_config: AppConfigDep,
|
||||
) -> EntityService:
|
||||
"""Create EntityService with repository."""
|
||||
@@ -242,6 +265,7 @@ async def get_entity_service(
|
||||
entity_parser=entity_parser,
|
||||
file_service=file_service,
|
||||
link_resolver=link_resolver,
|
||||
session_maker=session_maker,
|
||||
search_service=search_service,
|
||||
app_config=app_config,
|
||||
)
|
||||
@@ -258,6 +282,7 @@ async def get_entity_service_v2( # pragma: no cover
|
||||
file_service: FileServiceV2Dep,
|
||||
link_resolver: LinkResolverV2Dep,
|
||||
search_service: SearchServiceV2Dep,
|
||||
session_maker: SessionMakerDep,
|
||||
app_config: AppConfigDep,
|
||||
) -> EntityService:
|
||||
"""Create EntityService for v2 API."""
|
||||
@@ -268,6 +293,7 @@ async def get_entity_service_v2( # pragma: no cover
|
||||
entity_parser=entity_parser,
|
||||
file_service=file_service,
|
||||
link_resolver=link_resolver,
|
||||
session_maker=session_maker,
|
||||
search_service=search_service,
|
||||
app_config=app_config,
|
||||
)
|
||||
@@ -284,6 +310,7 @@ async def get_entity_service_v2_external(
|
||||
file_service: FileServiceV2ExternalDep,
|
||||
link_resolver: LinkResolverV2ExternalDep,
|
||||
search_service: SearchServiceV2ExternalDep,
|
||||
session_maker: SessionMakerDep,
|
||||
app_config: AppConfigDep,
|
||||
) -> EntityService:
|
||||
"""Create EntityService for v2 API (uses external_id)."""
|
||||
@@ -294,6 +321,7 @@ async def get_entity_service_v2_external(
|
||||
entity_parser=entity_parser,
|
||||
file_service=file_service,
|
||||
link_resolver=link_resolver,
|
||||
session_maker=session_maker,
|
||||
search_service=search_service,
|
||||
app_config=app_config,
|
||||
)
|
||||
@@ -310,12 +338,14 @@ async def get_context_service(
|
||||
entity_repository: EntityRepositoryDep,
|
||||
observation_repository: ObservationRepositoryDep,
|
||||
link_resolver: LinkResolverDep,
|
||||
session_maker: SessionMakerDep,
|
||||
) -> ContextService:
|
||||
return ContextService(
|
||||
search_repository=search_repository,
|
||||
entity_repository=entity_repository,
|
||||
observation_repository=observation_repository,
|
||||
link_resolver=link_resolver,
|
||||
session_maker=session_maker,
|
||||
)
|
||||
|
||||
|
||||
@@ -327,6 +357,7 @@ async def get_context_service_v2( # pragma: no cover
|
||||
entity_repository: EntityRepositoryV2Dep,
|
||||
observation_repository: ObservationRepositoryV2Dep,
|
||||
link_resolver: LinkResolverV2Dep,
|
||||
session_maker: SessionMakerDep,
|
||||
) -> ContextService:
|
||||
"""Create ContextService for v2 API."""
|
||||
return ContextService(
|
||||
@@ -334,6 +365,7 @@ async def get_context_service_v2( # pragma: no cover
|
||||
entity_repository=entity_repository,
|
||||
observation_repository=observation_repository,
|
||||
link_resolver=link_resolver,
|
||||
session_maker=session_maker,
|
||||
)
|
||||
|
||||
|
||||
@@ -345,6 +377,7 @@ async def get_context_service_v2_external(
|
||||
entity_repository: EntityRepositoryV2ExternalDep,
|
||||
observation_repository: ObservationRepositoryV2ExternalDep,
|
||||
link_resolver: LinkResolverV2ExternalDep,
|
||||
session_maker: SessionMakerDep,
|
||||
) -> ContextService:
|
||||
"""Create ContextService for v2 API (uses external_id)."""
|
||||
return ContextService(
|
||||
@@ -352,6 +385,7 @@ async def get_context_service_v2_external(
|
||||
entity_repository=entity_repository,
|
||||
observation_repository=observation_repository,
|
||||
link_resolver=link_resolver,
|
||||
session_maker=session_maker,
|
||||
)
|
||||
|
||||
|
||||
@@ -370,6 +404,7 @@ async def get_sync_service(
|
||||
project_repository: ProjectRepositoryDep,
|
||||
search_service: SearchServiceDep,
|
||||
file_service: FileServiceDep,
|
||||
session_maker: SessionMakerDep,
|
||||
) -> SyncService: # pragma: no cover
|
||||
return SyncService(
|
||||
app_config=app_config,
|
||||
@@ -380,6 +415,7 @@ async def get_sync_service(
|
||||
project_repository=project_repository,
|
||||
search_service=search_service,
|
||||
file_service=file_service,
|
||||
session_maker=session_maker,
|
||||
)
|
||||
|
||||
|
||||
@@ -395,6 +431,7 @@ async def get_sync_service_v2(
|
||||
project_repository: ProjectRepositoryDep,
|
||||
search_service: SearchServiceV2Dep,
|
||||
file_service: FileServiceV2Dep,
|
||||
session_maker: SessionMakerDep,
|
||||
) -> SyncService: # pragma: no cover
|
||||
"""Create SyncService for v2 API."""
|
||||
return SyncService(
|
||||
@@ -406,6 +443,7 @@ async def get_sync_service_v2(
|
||||
project_repository=project_repository,
|
||||
search_service=search_service,
|
||||
file_service=file_service,
|
||||
session_maker=session_maker,
|
||||
)
|
||||
|
||||
|
||||
@@ -421,6 +459,7 @@ async def get_sync_service_v2_external(
|
||||
project_repository: ProjectRepositoryDep,
|
||||
search_service: SearchServiceV2ExternalDep,
|
||||
file_service: FileServiceV2ExternalDep,
|
||||
session_maker: SessionMakerDep,
|
||||
) -> SyncService: # pragma: no cover
|
||||
"""Create SyncService for v2 API (uses external_id)."""
|
||||
return SyncService(
|
||||
@@ -432,6 +471,7 @@ async def get_sync_service_v2_external(
|
||||
project_repository=project_repository,
|
||||
search_service=search_service,
|
||||
file_service=file_service,
|
||||
session_maker=session_maker,
|
||||
)
|
||||
|
||||
|
||||
@@ -535,6 +575,7 @@ TaskSchedulerDep = Annotated[TaskScheduler, Depends(get_task_scheduler)]
|
||||
|
||||
async def get_project_service(
|
||||
project_repository: ProjectRepositoryDep,
|
||||
session_maker: SessionMakerDep,
|
||||
app_config: AppConfigDep,
|
||||
) -> ProjectService:
|
||||
"""Create ProjectService with repository and a system-level FileService for directory operations."""
|
||||
@@ -543,7 +584,9 @@ async def get_project_service(
|
||||
entity_parser = EntityParser(Path.home())
|
||||
markdown_processor = MarkdownProcessor(entity_parser, app_config=app_config)
|
||||
file_service = FileService(Path.home(), markdown_processor, app_config=app_config)
|
||||
return ProjectService(repository=project_repository, file_service=file_service)
|
||||
return ProjectService(
|
||||
repository=project_repository, session_maker=session_maker, file_service=file_service
|
||||
)
|
||||
|
||||
|
||||
ProjectServiceDep = Annotated[ProjectService, Depends(get_project_service)]
|
||||
@@ -554,10 +597,12 @@ ProjectServiceDep = Annotated[ProjectService, Depends(get_project_service)]
|
||||
|
||||
async def get_directory_service(
|
||||
entity_repository: EntityRepositoryDep,
|
||||
session_maker: SessionMakerDep,
|
||||
) -> DirectoryService:
|
||||
"""Create DirectoryService with dependencies."""
|
||||
return DirectoryService(
|
||||
entity_repository=entity_repository,
|
||||
session_maker=session_maker,
|
||||
)
|
||||
|
||||
|
||||
@@ -566,10 +611,12 @@ DirectoryServiceDep = Annotated[DirectoryService, Depends(get_directory_service)
|
||||
|
||||
async def get_directory_service_v2( # pragma: no cover
|
||||
entity_repository: EntityRepositoryV2Dep,
|
||||
session_maker: SessionMakerDep,
|
||||
) -> DirectoryService:
|
||||
"""Create DirectoryService for v2 API (uses integer project_id from path)."""
|
||||
return DirectoryService(
|
||||
entity_repository=entity_repository,
|
||||
session_maker=session_maker,
|
||||
)
|
||||
|
||||
|
||||
@@ -578,10 +625,12 @@ DirectoryServiceV2Dep = Annotated[DirectoryService, Depends(get_directory_servic
|
||||
|
||||
async def get_directory_service_v2_external(
|
||||
entity_repository: EntityRepositoryV2ExternalDep,
|
||||
session_maker: SessionMakerDep,
|
||||
) -> DirectoryService:
|
||||
"""Create DirectoryService for v2 API (uses external_id from path)."""
|
||||
return DirectoryService(
|
||||
entity_repository=entity_repository,
|
||||
session_maker=session_maker,
|
||||
)
|
||||
|
||||
|
||||
|
||||
@@ -302,26 +302,76 @@ async def format_file(
|
||||
return None
|
||||
|
||||
|
||||
# A frontmatter fence is a line containing exactly `---`, optionally followed by
|
||||
# trailing horizontal whitespace. Anchoring to a full line (rather than a bare
|
||||
# substring/`startswith`) prevents single-line content like
|
||||
# `---\nstatus: active\n---\nBody` — where `\n` is a literal backslash-n, not a
|
||||
# newline — from being misread as frontmatter. See issue #972.
|
||||
_FENCE_RE = re.compile(r"^---[ \t]*$")
|
||||
|
||||
|
||||
def _split_frontmatter(content: str) -> Optional[tuple[str, str]]:
|
||||
"""Split content into (yaml_block, body) when it opens with a line-anchored fence.
|
||||
|
||||
The opening fence must be the very first line and the closing fence must be a
|
||||
later line, each matching exactly `---` (with optional trailing whitespace).
|
||||
|
||||
Returns:
|
||||
A `(yaml_block, body)` tuple when a complete fenced block is present, or
|
||||
``None`` when the content does not open with a frontmatter fence.
|
||||
|
||||
Raises:
|
||||
ParseError: If the content opens with a fence but has no closing fence.
|
||||
"""
|
||||
lines = content.splitlines(keepends=True)
|
||||
|
||||
# Skip leading blank lines: a document may begin with whitespace before the
|
||||
# opening fence (e.g. a heredoc/dedented string starting with a newline). This
|
||||
# does NOT relax line-anchoring — the opening fence must still be the first
|
||||
# non-blank line, all on its own, so a single-line `---\\nstatus...` (literal
|
||||
# backslash-n) is still rejected. See issue #972.
|
||||
start = 0
|
||||
while start < len(lines) and lines[start].strip() == "":
|
||||
start += 1
|
||||
|
||||
if start >= len(lines) or not _FENCE_RE.match(lines[start].rstrip("\r\n")):
|
||||
return None
|
||||
|
||||
# Find the closing fence on its own line somewhere after the opening fence.
|
||||
for index in range(start + 1, len(lines)):
|
||||
if _FENCE_RE.match(lines[index].rstrip("\r\n")):
|
||||
yaml_block = "".join(lines[start + 1 : index])
|
||||
body = "".join(lines[index + 1 :])
|
||||
return yaml_block, body
|
||||
|
||||
raise ParseError("Invalid frontmatter format")
|
||||
|
||||
|
||||
def has_frontmatter(content: str) -> bool:
|
||||
"""
|
||||
Check if content contains valid YAML frontmatter.
|
||||
|
||||
Frontmatter requires `---` fences on their own lines; an inline `---` (such as
|
||||
a single-line string that merely starts with the characters `---`) is not
|
||||
frontmatter.
|
||||
|
||||
Args:
|
||||
content: Content to check
|
||||
|
||||
Returns:
|
||||
True if content has valid frontmatter markers (---), False otherwise
|
||||
True if content has line-anchored frontmatter fences, False otherwise
|
||||
"""
|
||||
if not content:
|
||||
return False
|
||||
|
||||
# Strip BOM before checking for frontmatter markers
|
||||
content = strip_bom(content).strip()
|
||||
if not content.startswith("---"):
|
||||
content = strip_bom(content)
|
||||
try:
|
||||
return _split_frontmatter(content) is not None
|
||||
except ParseError:
|
||||
# An opening fence with no closing fence is not usable frontmatter.
|
||||
return False
|
||||
|
||||
return "---" in content[3:]
|
||||
|
||||
|
||||
def parse_frontmatter(content: str) -> Dict[str, Any]:
|
||||
"""
|
||||
@@ -339,17 +389,14 @@ def parse_frontmatter(content: str) -> Dict[str, Any]:
|
||||
try:
|
||||
# Strip BOM before parsing frontmatter
|
||||
content = strip_bom(content)
|
||||
if not content.strip().startswith("---"):
|
||||
split = _split_frontmatter(content)
|
||||
if split is None:
|
||||
raise ParseError("Content has no frontmatter")
|
||||
|
||||
# Split on first two occurrences of ---
|
||||
parts = content.split("---", 2)
|
||||
if len(parts) < 3:
|
||||
raise ParseError("Invalid frontmatter format")
|
||||
yaml_block, _ = split
|
||||
|
||||
# Parse YAML
|
||||
try:
|
||||
frontmatter = yaml.safe_load(parts[1])
|
||||
frontmatter = yaml.safe_load(yaml_block)
|
||||
# Handle empty frontmatter (None from yaml.safe_load)
|
||||
if frontmatter is None:
|
||||
return {}
|
||||
@@ -381,18 +428,17 @@ def remove_frontmatter(content: str) -> str:
|
||||
ParseError: If content starts with frontmatter marker but is malformed
|
||||
"""
|
||||
# Strip BOM before processing
|
||||
content = strip_bom(content).strip()
|
||||
content = strip_bom(content)
|
||||
|
||||
# Return as-is if no frontmatter marker
|
||||
if not content.startswith("---"):
|
||||
return content
|
||||
split = _split_frontmatter(content)
|
||||
# Trigger: content does not open with a line-anchored fence
|
||||
# Why: inline `---` is ordinary content, not frontmatter (issue #972)
|
||||
# Outcome: return the content untouched (stripped to preserve prior behavior)
|
||||
if split is None:
|
||||
return content.strip()
|
||||
|
||||
# Split on first two occurrences of ---
|
||||
parts = content.split("---", 2)
|
||||
if len(parts) < 3:
|
||||
raise ParseError("Invalid frontmatter format")
|
||||
|
||||
return parts[2].strip()
|
||||
_, body = split
|
||||
return body.strip()
|
||||
|
||||
|
||||
def dump_frontmatter(post: frontmatter.Post) -> str:
|
||||
|
||||
@@ -3,15 +3,18 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
from contextlib import asynccontextmanager
|
||||
from dataclasses import dataclass
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
from typing import Awaitable, Callable, Mapping, TypeVar
|
||||
from typing import AsyncIterator, Awaitable, Callable, Mapping, TypeVar
|
||||
|
||||
from loguru import logger
|
||||
from sqlalchemy.exc import IntegrityError
|
||||
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
|
||||
|
||||
import logfire
|
||||
from basic_memory import db
|
||||
from basic_memory.config import BasicMemoryConfig
|
||||
from basic_memory.file_utils import compute_checksum, has_frontmatter, remove_frontmatter
|
||||
from basic_memory.markdown.schemas import EntityMarkdown
|
||||
@@ -69,6 +72,7 @@ class BatchIndexer:
|
||||
relation_repository: RelationRepository,
|
||||
search_service: SearchService,
|
||||
file_writer: IndexFileWriter,
|
||||
session_maker: async_sessionmaker[AsyncSession],
|
||||
) -> None:
|
||||
self.app_config = app_config
|
||||
self.entity_service = entity_service
|
||||
@@ -76,6 +80,19 @@ class BatchIndexer:
|
||||
self.relation_repository = relation_repository
|
||||
self.search_service = search_service
|
||||
self.file_writer = file_writer
|
||||
self.session_maker = session_maker
|
||||
|
||||
@asynccontextmanager
|
||||
async def _session_scope(
|
||||
self, session: AsyncSession | None = None
|
||||
) -> AsyncIterator[AsyncSession]:
|
||||
"""Use the caller's session or open a service-owned transaction."""
|
||||
if session is not None:
|
||||
yield session
|
||||
return
|
||||
|
||||
async with db.scoped_session(self.session_maker) as owned_session:
|
||||
yield owned_session
|
||||
|
||||
async def index_files(
|
||||
self,
|
||||
@@ -149,9 +166,10 @@ class BatchIndexer:
|
||||
max_concurrent=max_concurrent,
|
||||
)
|
||||
|
||||
refreshed_entities = await self.entity_repository.find_by_ids(
|
||||
[prepared.entity_id for prepared in prepared_entities.values()]
|
||||
)
|
||||
async with self._session_scope() as session:
|
||||
refreshed_entities = await self.entity_repository.find_by_ids(
|
||||
session, [prepared.entity_id for prepared in prepared_entities.values()]
|
||||
)
|
||||
entities_by_id = {entity.id: entity for entity in refreshed_entities}
|
||||
|
||||
refreshed, refresh_errors = await self._run_bounded(
|
||||
@@ -196,12 +214,7 @@ class BatchIndexer:
|
||||
prepared = await self._prepare_markdown_file(file)
|
||||
if existing_permalink_by_path is None:
|
||||
with logfire.span("index.markdown_file.load_permalink_map", path=file.path):
|
||||
existing_permalink_by_path = {
|
||||
path: permalink
|
||||
for path, permalink in (
|
||||
await self.entity_repository.get_file_path_to_permalink_map()
|
||||
).items()
|
||||
}
|
||||
existing_permalink_by_path = await self._get_file_path_to_permalink_map()
|
||||
|
||||
reserved_permalinks = {
|
||||
permalink
|
||||
@@ -226,7 +239,8 @@ class BatchIndexer:
|
||||
path=file.path,
|
||||
entity_id=persisted.entity.id,
|
||||
):
|
||||
refreshed = await self.entity_repository.find_by_ids([persisted.entity.id])
|
||||
async with self._session_scope() as session:
|
||||
refreshed = await self.entity_repository.find_by_ids(session, [persisted.entity.id])
|
||||
if len(refreshed) != 1: # pragma: no cover
|
||||
raise ValueError(f"Failed to reload indexed entity for {file.path}")
|
||||
entity = refreshed[0]
|
||||
@@ -249,6 +263,17 @@ class BatchIndexer:
|
||||
markdown_content=prepared_entity.markdown_content,
|
||||
)
|
||||
|
||||
async def _get_file_path_to_permalink_map(self) -> dict[str, str | None]:
|
||||
"""Load current file-path to permalink mappings in a service-owned session."""
|
||||
async with self._session_scope() as session:
|
||||
permalink_by_path: dict[str, str | None] = {
|
||||
path: permalink
|
||||
for path, permalink in (
|
||||
await self.entity_repository.get_file_path_to_permalink_map(session)
|
||||
).items()
|
||||
}
|
||||
return permalink_by_path
|
||||
|
||||
# --- Preparation ---
|
||||
|
||||
async def _prepare_markdown_file(self, file: IndexInputFile) -> _PreparedMarkdownFile:
|
||||
@@ -283,12 +308,7 @@ class BatchIndexer:
|
||||
return {}, {}
|
||||
|
||||
if existing_permalink_by_path is None:
|
||||
existing_permalink_by_path = {
|
||||
path: permalink
|
||||
for path, permalink in (
|
||||
await self.entity_repository.get_file_path_to_permalink_map()
|
||||
).items()
|
||||
}
|
||||
existing_permalink_by_path = await self._get_file_path_to_permalink_map()
|
||||
|
||||
batch_paths = set(prepared_markdown)
|
||||
reserved_permalinks = {
|
||||
@@ -406,7 +426,10 @@ class BatchIndexer:
|
||||
|
||||
async def _upsert_regular_file(self, file: IndexInputFile) -> _PreparedEntity:
|
||||
checksum = await self._resolve_checksum(file)
|
||||
existing = await self.entity_repository.get_by_file_path(file.path, load_relations=False)
|
||||
async with self._session_scope() as session:
|
||||
existing = await self.entity_repository.get_by_file_path(
|
||||
session, file.path, load_relations=False
|
||||
)
|
||||
is_new_entity = existing is None
|
||||
|
||||
if existing is None:
|
||||
@@ -424,7 +447,8 @@ class BatchIndexer:
|
||||
)
|
||||
|
||||
try:
|
||||
created = await self.entity_repository.add(entity)
|
||||
async with self._session_scope() as session:
|
||||
created = await self.entity_repository.add(session, entity)
|
||||
entity_id = created.id
|
||||
except IntegrityError as exc:
|
||||
message = str(exc)
|
||||
@@ -436,10 +460,12 @@ class BatchIndexer:
|
||||
and "file_path" in message
|
||||
)
|
||||
):
|
||||
existing = await self.entity_repository.get_by_file_path(
|
||||
file.path,
|
||||
load_relations=False,
|
||||
)
|
||||
async with self._session_scope() as session:
|
||||
existing = await self.entity_repository.get_by_file_path(
|
||||
session,
|
||||
file.path,
|
||||
load_relations=False,
|
||||
)
|
||||
if existing is None:
|
||||
raise ValueError(
|
||||
f"Entity not found after file_path conflict: {file.path}"
|
||||
@@ -450,10 +476,12 @@ class BatchIndexer:
|
||||
else:
|
||||
entity_id = existing.id
|
||||
|
||||
updated = await self.entity_repository.update(
|
||||
entity_id,
|
||||
self._entity_metadata_updates(file, checksum, include_created_at=is_new_entity),
|
||||
)
|
||||
async with self._session_scope() as session:
|
||||
updated = await self.entity_repository.update(
|
||||
session,
|
||||
entity_id,
|
||||
self._entity_metadata_updates(file, checksum, include_created_at=is_new_entity),
|
||||
)
|
||||
if updated is None:
|
||||
raise ValueError(f"Failed to update file entity metadata for {file.path}")
|
||||
|
||||
@@ -476,10 +504,7 @@ class BatchIndexer:
|
||||
max_concurrent: int,
|
||||
) -> tuple[int, int]:
|
||||
unresolved_relation_lists = await asyncio.gather(
|
||||
*(
|
||||
self.relation_repository.find_unresolved_relations_for_entity(entity_id)
|
||||
for entity_id in entity_ids
|
||||
)
|
||||
*(self._find_unresolved_relations_for_entity(entity_id) for entity_id in entity_ids)
|
||||
)
|
||||
unresolved_relations = [
|
||||
relation for relation_list in unresolved_relation_lists for relation in relation_list
|
||||
@@ -499,22 +524,26 @@ class BatchIndexer:
|
||||
# link text, mismatching this with the sync_service forward-reference
|
||||
# path and producing confidently-wrong graph edges. See
|
||||
# sync_service.resolve_forward_references for the same change.
|
||||
resolved_entity = await self.entity_service.link_resolver.resolve_link(
|
||||
relation.to_name, strict=True
|
||||
)
|
||||
async with self._session_scope() as session:
|
||||
resolved_entity = await self.entity_service.link_resolver.resolve_link(
|
||||
relation.to_name, strict=True, session=session
|
||||
)
|
||||
if resolved_entity is None or resolved_entity.id == relation.from_id:
|
||||
return 0
|
||||
|
||||
try:
|
||||
await self.relation_repository.update(
|
||||
relation.id,
|
||||
{
|
||||
"to_id": resolved_entity.id,
|
||||
"to_name": resolved_entity.title,
|
||||
},
|
||||
)
|
||||
async with self._session_scope() as session:
|
||||
await self.relation_repository.update(
|
||||
session,
|
||||
relation.id,
|
||||
{
|
||||
"to_id": resolved_entity.id,
|
||||
"to_name": resolved_entity.title,
|
||||
},
|
||||
)
|
||||
except IntegrityError:
|
||||
await self.relation_repository.delete(relation.id)
|
||||
async with self._session_scope() as session:
|
||||
await self.relation_repository.delete(session, relation.id)
|
||||
return 1
|
||||
except Exception as exc: # pragma: no cover - defensive logging
|
||||
logger.warning(
|
||||
@@ -531,15 +560,19 @@ class BatchIndexer:
|
||||
)
|
||||
|
||||
remaining_relation_lists = await asyncio.gather(
|
||||
*(
|
||||
self.relation_repository.find_unresolved_relations_for_entity(entity_id)
|
||||
for entity_id in entity_ids
|
||||
)
|
||||
*(self._find_unresolved_relations_for_entity(entity_id) for entity_id in entity_ids)
|
||||
)
|
||||
remaining_unresolved = sum(len(relations) for relations in remaining_relation_lists)
|
||||
|
||||
return sum(resolved_counts), remaining_unresolved
|
||||
|
||||
async def _find_unresolved_relations_for_entity(self, entity_id: int):
|
||||
"""Load unresolved relations for one entity in a service-owned session."""
|
||||
async with self._session_scope() as session:
|
||||
return await self.relation_repository.find_unresolved_relations_for_entity(
|
||||
session, entity_id
|
||||
)
|
||||
|
||||
# --- Search refresh ---
|
||||
|
||||
async def _refresh_search_index(
|
||||
@@ -565,30 +598,36 @@ class BatchIndexer:
|
||||
resolve_relations: bool = True,
|
||||
reload_entity: bool = True,
|
||||
) -> _PersistedMarkdownFile:
|
||||
existing = await self.entity_repository.get_by_file_path(
|
||||
prepared.file.path,
|
||||
load_relations=False,
|
||||
)
|
||||
if is_new is None:
|
||||
is_new = existing is None
|
||||
entity = await self.entity_service.upsert_entity_from_markdown(
|
||||
Path(prepared.file.path),
|
||||
prepared.markdown,
|
||||
is_new=is_new,
|
||||
existing_entity=existing,
|
||||
resolve_relations=resolve_relations,
|
||||
reload_entity=reload_entity,
|
||||
)
|
||||
prepared = await self._reconcile_persisted_permalink(prepared, entity)
|
||||
metadata_updates = self._entity_metadata_updates(prepared.file, prepared.final_checksum)
|
||||
updated = await self.entity_repository.update_fields(
|
||||
entity.id,
|
||||
metadata_updates,
|
||||
)
|
||||
if not updated:
|
||||
raise ValueError(f"Failed to update markdown entity metadata for {prepared.file.path}")
|
||||
self._apply_entity_metadata_updates(entity, metadata_updates)
|
||||
return _PersistedMarkdownFile(prepared=prepared, entity=entity)
|
||||
async with self._session_scope() as session:
|
||||
existing = await self.entity_repository.get_by_file_path(
|
||||
session,
|
||||
prepared.file.path,
|
||||
load_relations=False,
|
||||
)
|
||||
if is_new is None:
|
||||
is_new = existing is None
|
||||
entity = await self.entity_service.upsert_entity_from_markdown(
|
||||
Path(prepared.file.path),
|
||||
prepared.markdown,
|
||||
is_new=is_new,
|
||||
existing_entity=existing,
|
||||
resolve_relations=resolve_relations,
|
||||
reload_entity=reload_entity,
|
||||
session=session,
|
||||
)
|
||||
prepared = await self._reconcile_persisted_permalink(prepared, entity)
|
||||
metadata_updates = self._entity_metadata_updates(prepared.file, prepared.final_checksum)
|
||||
updated = await self.entity_repository.update_fields(
|
||||
session,
|
||||
entity.id,
|
||||
metadata_updates,
|
||||
)
|
||||
if not updated:
|
||||
raise ValueError(
|
||||
f"Failed to update markdown entity metadata for {prepared.file.path}"
|
||||
)
|
||||
self._apply_entity_metadata_updates(entity, metadata_updates)
|
||||
return _PersistedMarkdownFile(prepared=prepared, entity=entity)
|
||||
|
||||
async def _reconcile_persisted_permalink(
|
||||
self,
|
||||
|
||||
@@ -0,0 +1 @@
|
||||
.so man1/bm.1
|
||||
@@ -0,0 +1,134 @@
|
||||
.TH BM 1 "2026-06-11" "basic-memory" "Basic Memory Manual"
|
||||
.SH NAME
|
||||
bm \- local-first knowledge base for humans and AI agents
|
||||
.SH SYNOPSIS
|
||||
.B bm
|
||||
.I COMMAND
|
||||
.RI [ ARGS ]...
|
||||
.br
|
||||
.B basic-memory
|
||||
.I COMMAND
|
||||
.RI [ ARGS ]...
|
||||
.SH DESCRIPTION
|
||||
.B bm
|
||||
manages Basic Memory projects: plain markdown files that form a knowledge
|
||||
graph. Files are the source of truth; SQLite provides indexing and
|
||||
full-text search; the same operations are exposed to AI agents over the
|
||||
Model Context Protocol (MCP) and to humans and scripts through this CLI.
|
||||
.PP
|
||||
Notes use semantic markdown: observations
|
||||
.RB ( "\- [category] text #tag" )
|
||||
and relations
|
||||
.RB ( "\- relation_type [[Target]]" )
|
||||
become queryable graph structure. Projects route independently to the
|
||||
local API or to Basic Memory Cloud.
|
||||
.SH COMMANDS
|
||||
Knowledge operations:
|
||||
.TP
|
||||
.B bm tool
|
||||
CLI access to the MCP tools (write-note, read-note, search-notes,
|
||||
build-context, ...). These emit JSON and are the scriptable surface.
|
||||
.TP
|
||||
.B bm status
|
||||
Show sync status between files and the database.
|
||||
.TP
|
||||
.B bm reindex
|
||||
Index local file changes and rebuild search/embeddings. This is the manual
|
||||
sync trigger when no MCP server is running.
|
||||
.TP
|
||||
.B bm doctor
|
||||
Run end-to-end file/database consistency checks.
|
||||
.TP
|
||||
.B bm orphans
|
||||
List entities with no relations in the knowledge graph.
|
||||
.TP
|
||||
.B bm format
|
||||
Run configured formatters over note files.
|
||||
.PP
|
||||
Projects and schemas:
|
||||
.TP
|
||||
.B bm project
|
||||
Add, remove, list projects; set the default; flip a project between local
|
||||
and cloud routing.
|
||||
.TP
|
||||
.B bm schema
|
||||
List, validate, infer, and drift-check Picoschema note-type contracts.
|
||||
.PP
|
||||
Data and cloud:
|
||||
.TP
|
||||
.B bm import
|
||||
Import from ChatGPT, Claude, or memory.json exports. Imports write files;
|
||||
run
|
||||
.B bm reindex
|
||||
afterwards.
|
||||
.TP
|
||||
.B bm cloud
|
||||
Authenticate, sync (push/pull/sync/bisync), snapshots, and team workspace
|
||||
administration.
|
||||
.PP
|
||||
Infrastructure:
|
||||
.TP
|
||||
.B bm mcp
|
||||
Run the MCP server (hosts the live file watcher).
|
||||
.TP
|
||||
.B bm man
|
||||
Manage these man pages
|
||||
.RB ( "bm man install" ).
|
||||
.TP
|
||||
.B bm reset
|
||||
Drop and recreate the database (destructive).
|
||||
.PP
|
||||
Most commands accept
|
||||
.B \-\-project
|
||||
.I NAME
|
||||
to target a project and
|
||||
.BR \-\-local / \-\-cloud
|
||||
to override routing.
|
||||
.SH EXAMPLES
|
||||
Write and find a note from the shell:
|
||||
.PP
|
||||
.nf
|
||||
.RS
|
||||
echo "# Standup notes" | bm tool write-note \\
|
||||
\-\-title "Standup" \-\-folder notes
|
||||
bm tool search-notes "standup"
|
||||
.RE
|
||||
.fi
|
||||
.PP
|
||||
Pick up files created outside the tools:
|
||||
.PP
|
||||
.nf
|
||||
.RS
|
||||
bm status # shows pending changes
|
||||
bm reindex # indexes them
|
||||
.RE
|
||||
.fi
|
||||
.SH FILES
|
||||
.TP
|
||||
.I ~/.basic-memory/config.json
|
||||
Projects, default project, per-project routing modes, cloud settings.
|
||||
.TP
|
||||
.I ~/.basic-memory/memory.db
|
||||
SQLite index (derived; safe to rebuild with bm reindex).
|
||||
.SH ENVIRONMENT
|
||||
.TP
|
||||
.B BASIC_MEMORY_FORCE_LOCAL
|
||||
Force local routing regardless of cloud mode.
|
||||
.TP
|
||||
.B BASIC_MEMORY_LOG_LEVEL
|
||||
Logging verbosity (e.g. DEBUG).
|
||||
.SH SEE ALSO
|
||||
Full manual (machine-readable, agent-traversable): the
|
||||
.I manual
|
||||
Basic Memory project \(em see docs/manual-pages.md in the repository.
|
||||
Documentation: https://docs.basicmemory.com
|
||||
.PP
|
||||
For AI agents: the complete tool reference lives in the manual project as
|
||||
section\-3 pages (write-note(3), search-notes(3), ...), queryable via
|
||||
.B search_notes
|
||||
with
|
||||
.BR "metadata_filters={\(dqtype\(dq: \(dqmanpage\(dq}" .
|
||||
.SH BUGS
|
||||
https://github.com/basicmachines-co/basic-memory/issues
|
||||
.SH AUTHORS
|
||||
Basic Machines (https://basicmachines.co)
|
||||
@@ -5,14 +5,16 @@ from dataclasses import dataclass
|
||||
from threading import RLock
|
||||
from typing import TYPE_CHECKING, Annotated, Any, AsyncIterator, Callable, Optional
|
||||
|
||||
from fastapi import Depends, FastAPI, Request
|
||||
from httpx import ASGITransport, AsyncClient, Timeout
|
||||
from loguru import logger
|
||||
|
||||
import logfire
|
||||
from basic_memory.config import ConfigManager, ProjectMode
|
||||
from basic_memory.config import ConfigManager, ProjectMode, has_cloud_credentials
|
||||
|
||||
if TYPE_CHECKING:
|
||||
# FastAPI is only needed when a request routes through the local ASGI
|
||||
# transport; importing it at module level costs ~0.1s on every CLI start (#886).
|
||||
from fastapi import FastAPI
|
||||
from sqlalchemy.ext.asyncio import AsyncEngine, AsyncSession, async_sessionmaker
|
||||
|
||||
LocalDatabaseState = tuple["AsyncEngine", "async_sessionmaker[AsyncSession]"]
|
||||
@@ -28,8 +30,8 @@ class _PreparedLocalAsgiDatabase:
|
||||
|
||||
|
||||
_prepared_local_asgi_database_lock = RLock()
|
||||
_prepared_local_asgi_database_prepare_locks: dict[FastAPI, Lock] = {}
|
||||
_prepared_local_asgi_databases: dict[FastAPI, _PreparedLocalAsgiDatabase] = {}
|
||||
_prepared_local_asgi_database_prepare_locks: dict["FastAPI", Lock] = {}
|
||||
_prepared_local_asgi_databases: dict["FastAPI", _PreparedLocalAsgiDatabase] = {}
|
||||
|
||||
|
||||
def _force_local_mode() -> bool:
|
||||
@@ -57,7 +59,7 @@ def _build_timeout() -> Timeout:
|
||||
)
|
||||
|
||||
|
||||
def _build_asgi_client(app: FastAPI, timeout: Timeout) -> AsyncClient:
|
||||
def _build_asgi_client(app: "FastAPI", timeout: Timeout) -> AsyncClient:
|
||||
"""Create a local ASGI client for an already-prepared FastAPI app."""
|
||||
from basic_memory.workspace_context import workspace_permalink_headers
|
||||
|
||||
@@ -71,7 +73,7 @@ def _build_asgi_client(app: FastAPI, timeout: Timeout) -> AsyncClient:
|
||||
)
|
||||
|
||||
|
||||
def _get_prepared_local_asgi_database_prepare_lock(app: FastAPI) -> Lock:
|
||||
def _get_prepared_local_asgi_database_prepare_lock(app: "FastAPI") -> Lock:
|
||||
"""Get the async lock that serializes first-time DB preparation for an app."""
|
||||
with _prepared_local_asgi_database_lock:
|
||||
prepare_lock = _prepared_local_asgi_database_prepare_locks.get(app)
|
||||
@@ -82,8 +84,10 @@ def _get_prepared_local_asgi_database_prepare_lock(app: FastAPI) -> Lock:
|
||||
|
||||
|
||||
@asynccontextmanager
|
||||
async def _resolve_local_asgi_database(app: FastAPI) -> AsyncIterator[LocalDatabaseState]:
|
||||
async def _resolve_local_asgi_database(app: "FastAPI") -> AsyncIterator[LocalDatabaseState]:
|
||||
"""Resolve database state for a local ASGI request."""
|
||||
# Imported on first local-ASGI use so CLI startup never pays for FastAPI (#886).
|
||||
from fastapi import Depends, Request
|
||||
from fastapi.dependencies.utils import get_dependant, solve_dependencies
|
||||
|
||||
from basic_memory.deps import get_engine_factory
|
||||
@@ -127,7 +131,7 @@ async def _resolve_local_asgi_database(app: FastAPI) -> AsyncIterator[LocalDatab
|
||||
yield await resolve_database_state(**solved.values)
|
||||
|
||||
|
||||
def _retain_prepared_local_asgi_database(app: FastAPI) -> bool:
|
||||
def _retain_prepared_local_asgi_database(app: "FastAPI") -> bool:
|
||||
"""Retain an active local ASGI database preparation if one exists."""
|
||||
with _prepared_local_asgi_database_lock:
|
||||
active = _prepared_local_asgi_databases.get(app)
|
||||
@@ -139,7 +143,7 @@ def _retain_prepared_local_asgi_database(app: FastAPI) -> bool:
|
||||
|
||||
|
||||
def _install_prepared_local_asgi_database(
|
||||
app: FastAPI,
|
||||
app: "FastAPI",
|
||||
database_state: LocalDatabaseState,
|
||||
dependency_context: AbstractAsyncContextManager[LocalDatabaseState],
|
||||
) -> None:
|
||||
@@ -163,7 +167,7 @@ def _install_prepared_local_asgi_database(
|
||||
)
|
||||
|
||||
|
||||
def _restore_local_asgi_state_attribute(app: FastAPI, name: str, previous_value: object) -> None:
|
||||
def _restore_local_asgi_state_attribute(app: "FastAPI", name: str, previous_value: object) -> None:
|
||||
"""Restore a FastAPI app.state attribute captured before local ASGI preparation."""
|
||||
if previous_value is _MISSING_STATE_VALUE:
|
||||
if hasattr(app.state, name):
|
||||
@@ -173,7 +177,7 @@ def _restore_local_asgi_state_attribute(app: FastAPI, name: str, previous_value:
|
||||
|
||||
|
||||
def _release_prepared_local_asgi_database(
|
||||
app: FastAPI,
|
||||
app: "FastAPI",
|
||||
) -> AbstractAsyncContextManager[LocalDatabaseState] | None:
|
||||
"""Release local ASGI database state after a client context exits."""
|
||||
with _prepared_local_asgi_database_lock:
|
||||
@@ -196,7 +200,7 @@ def _release_prepared_local_asgi_database(
|
||||
|
||||
|
||||
@asynccontextmanager
|
||||
async def _prepared_local_asgi_database(app: FastAPI) -> AsyncIterator[None]:
|
||||
async def _prepared_local_asgi_database(app: "FastAPI") -> AsyncIterator[None]:
|
||||
"""Initialize local ASGI database state before the first request."""
|
||||
prepare_lock = _get_prepared_local_asgi_database_prepare_lock(app)
|
||||
async with prepare_lock:
|
||||
@@ -368,7 +372,8 @@ async def get_client(
|
||||
1. Factory injection.
|
||||
2. Explicit routing flags (--local/--cloud).
|
||||
3. Per-project mode routing when project_name is provided.
|
||||
4. Local ASGI transport by default.
|
||||
4. Cloud routing when a workspace selector is provided.
|
||||
5. Local ASGI transport by default.
|
||||
"""
|
||||
if _client_factory:
|
||||
async with _client_factory(workspace=workspace) as client:
|
||||
@@ -428,6 +433,26 @@ async def get_client(
|
||||
yield client
|
||||
return
|
||||
|
||||
# --- Workspace-selector routing ---
|
||||
# Trigger: caller passed a cloud workspace selector and nothing above routed.
|
||||
# Why: a workspace names a cloud tenant — silently serving the request from
|
||||
# the local ASGI app sent writes to the wrong destination (#954: a cloud
|
||||
# project create either failed on the cloud-style path or landed locally).
|
||||
# Outcome: route to the cloud proxy when credentials exist; without
|
||||
# credentials fail fast instead of pretending the operation succeeded.
|
||||
if workspace is not None:
|
||||
if not has_cloud_credentials(config):
|
||||
raise RuntimeError(
|
||||
f"A cloud workspace was requested ('{workspace}') but no cloud "
|
||||
"credentials were found. Run 'bm cloud login' or "
|
||||
"'bm cloud set-key <key>' first, or omit the workspace selector "
|
||||
"for a local operation."
|
||||
)
|
||||
logger.debug(f"Workspace selector '{workspace}' provided - using cloud proxy client")
|
||||
async with _cloud_client(config, timeout, workspace=workspace) as client:
|
||||
yield client
|
||||
return
|
||||
|
||||
# --- Default fallback ---
|
||||
logger.debug("Default routing - using ASGI client for local Basic Memory API")
|
||||
async with _asgi_client(timeout) as client:
|
||||
|
||||
@@ -7,7 +7,9 @@ from typing import Optional, Any
|
||||
|
||||
from httpx import AsyncClient
|
||||
|
||||
from basic_memory.mcp.tools.utils import call_get
|
||||
# call_* helpers live in basic_memory.mcp.tools.utils; importing that at module
|
||||
# level executes the whole tools package (fastmcp + mcp SDK) during CLI startup,
|
||||
# so each method defers the import to call time instead (#886).
|
||||
|
||||
|
||||
class DirectoryClient:
|
||||
@@ -55,6 +57,8 @@ class DirectoryClient:
|
||||
Raises:
|
||||
ToolError: If the request fails
|
||||
"""
|
||||
from basic_memory.mcp.tools.utils import call_get
|
||||
|
||||
params: dict = {
|
||||
"dir_name": dir_name,
|
||||
"depth": depth,
|
||||
|
||||
@@ -8,7 +8,10 @@ from typing import Any
|
||||
from httpx import AsyncClient
|
||||
|
||||
import logfire
|
||||
from basic_memory.mcp.tools.utils import call_get, call_post, call_put, call_patch, call_delete
|
||||
|
||||
# call_* helpers live in basic_memory.mcp.tools.utils; importing that at module
|
||||
# level executes the whole tools package (fastmcp + mcp SDK) during CLI startup,
|
||||
# so each method defers the import to call time instead (#886).
|
||||
from basic_memory.schemas.response import (
|
||||
EntityResponse,
|
||||
DeleteEntitiesResponse,
|
||||
@@ -57,6 +60,8 @@ class KnowledgeClient:
|
||||
Raises:
|
||||
ToolError: If the request fails
|
||||
"""
|
||||
from basic_memory.mcp.tools.utils import call_post
|
||||
|
||||
with logfire.span(
|
||||
"mcp.client.knowledge.create_entity",
|
||||
client_name="knowledge",
|
||||
@@ -89,6 +94,8 @@ class KnowledgeClient:
|
||||
Raises:
|
||||
ToolError: If the request fails
|
||||
"""
|
||||
from basic_memory.mcp.tools.utils import call_put
|
||||
|
||||
with logfire.span(
|
||||
"mcp.client.knowledge.update_entity",
|
||||
client_name="knowledge",
|
||||
@@ -116,6 +123,8 @@ class KnowledgeClient:
|
||||
Raises:
|
||||
ToolError: If the entity is not found or request fails
|
||||
"""
|
||||
from basic_memory.mcp.tools.utils import call_get
|
||||
|
||||
with logfire.span(
|
||||
"mcp.client.knowledge.get_entity",
|
||||
client_name="knowledge",
|
||||
@@ -147,6 +156,8 @@ class KnowledgeClient:
|
||||
Raises:
|
||||
ToolError: If the request fails
|
||||
"""
|
||||
from basic_memory.mcp.tools.utils import call_patch
|
||||
|
||||
with logfire.span(
|
||||
"mcp.client.knowledge.patch_entity",
|
||||
client_name="knowledge",
|
||||
@@ -174,6 +185,8 @@ class KnowledgeClient:
|
||||
Raises:
|
||||
ToolError: If the entity is not found or request fails
|
||||
"""
|
||||
from basic_memory.mcp.tools.utils import call_delete
|
||||
|
||||
with logfire.span(
|
||||
"mcp.client.knowledge.delete_entity",
|
||||
client_name="knowledge",
|
||||
@@ -201,6 +214,8 @@ class KnowledgeClient:
|
||||
Raises:
|
||||
ToolError: If the request fails
|
||||
"""
|
||||
from basic_memory.mcp.tools.utils import call_put
|
||||
|
||||
with logfire.span(
|
||||
"mcp.client.knowledge.move_entity",
|
||||
client_name="knowledge",
|
||||
@@ -231,6 +246,8 @@ class KnowledgeClient:
|
||||
Raises:
|
||||
ToolError: If the request fails
|
||||
"""
|
||||
from basic_memory.mcp.tools.utils import call_post
|
||||
|
||||
with logfire.span(
|
||||
"mcp.client.knowledge.move_directory",
|
||||
client_name="knowledge",
|
||||
@@ -261,6 +278,8 @@ class KnowledgeClient:
|
||||
Raises:
|
||||
ToolError: If the request fails
|
||||
"""
|
||||
from basic_memory.mcp.tools.utils import call_post
|
||||
|
||||
with logfire.span(
|
||||
"mcp.client.knowledge.delete_directory",
|
||||
client_name="knowledge",
|
||||
@@ -290,6 +309,8 @@ class KnowledgeClient:
|
||||
Raises:
|
||||
ToolError: If the file does not exist on disk or indexing fails
|
||||
"""
|
||||
from basic_memory.mcp.tools.utils import call_post
|
||||
|
||||
with logfire.span(
|
||||
"mcp.client.knowledge.sync_file",
|
||||
client_name="knowledge",
|
||||
@@ -309,6 +330,8 @@ class KnowledgeClient:
|
||||
|
||||
async def get_orphans(self) -> list[GraphNode]:
|
||||
"""Get entities that have no incoming or outgoing relations."""
|
||||
from basic_memory.mcp.tools.utils import call_get
|
||||
|
||||
with logfire.span(
|
||||
"mcp.client.knowledge.get_orphans",
|
||||
client_name="knowledge",
|
||||
@@ -338,6 +361,8 @@ class KnowledgeClient:
|
||||
Raises:
|
||||
ToolError: If the identifier cannot be resolved
|
||||
"""
|
||||
from basic_memory.mcp.tools.utils import call_post
|
||||
|
||||
with logfire.span(
|
||||
"mcp.client.knowledge.resolve_entity",
|
||||
client_name="knowledge",
|
||||
|
||||
@@ -8,7 +8,10 @@ from typing import Optional
|
||||
from httpx import AsyncClient
|
||||
|
||||
import logfire
|
||||
from basic_memory.mcp.tools.utils import call_get
|
||||
|
||||
# call_* helpers live in basic_memory.mcp.tools.utils; importing that at module
|
||||
# level executes the whole tools package (fastmcp + mcp SDK) during CLI startup,
|
||||
# so each method defers the import to call time instead (#886).
|
||||
from basic_memory.schemas.memory import GraphContext
|
||||
|
||||
|
||||
@@ -63,6 +66,8 @@ class MemoryClient:
|
||||
Raises:
|
||||
ToolError: If the request fails
|
||||
"""
|
||||
from basic_memory.mcp.tools.utils import call_get
|
||||
|
||||
params: dict = {
|
||||
"depth": depth,
|
||||
"page": page,
|
||||
@@ -113,6 +118,8 @@ class MemoryClient:
|
||||
Raises:
|
||||
ToolError: If the request fails
|
||||
"""
|
||||
from basic_memory.mcp.tools.utils import call_get
|
||||
|
||||
params: dict = {
|
||||
"timeframe": timeframe,
|
||||
"depth": depth,
|
||||
|
||||
@@ -7,13 +7,9 @@ from typing import Any
|
||||
|
||||
from httpx import AsyncClient
|
||||
|
||||
from basic_memory.mcp.tools.utils import (
|
||||
call_delete,
|
||||
call_get,
|
||||
call_patch,
|
||||
call_post,
|
||||
call_put,
|
||||
)
|
||||
# call_* helpers live in basic_memory.mcp.tools.utils; importing that at module
|
||||
# level executes the whole tools package (fastmcp + mcp SDK) during CLI startup,
|
||||
# so each method defers the import to call time instead (#886).
|
||||
from basic_memory.schemas import ProjectInfoResponse, SyncReportResponse
|
||||
from basic_memory.schemas.project_info import ProjectList, ProjectStatusResponse
|
||||
from basic_memory.schemas.v2 import ProjectResolveResponse
|
||||
@@ -53,6 +49,8 @@ class ProjectClient:
|
||||
Raises:
|
||||
ToolError: If the request fails
|
||||
"""
|
||||
from basic_memory.mcp.tools.utils import call_get
|
||||
|
||||
response = await call_get(
|
||||
self.http_client,
|
||||
"/v2/projects/",
|
||||
@@ -71,6 +69,8 @@ class ProjectClient:
|
||||
Raises:
|
||||
ToolError: If the request fails
|
||||
"""
|
||||
from basic_memory.mcp.tools.utils import call_post
|
||||
|
||||
response = await call_post(
|
||||
self.http_client,
|
||||
"/v2/projects/",
|
||||
@@ -93,6 +93,8 @@ class ProjectClient:
|
||||
Raises:
|
||||
ToolError: If the request fails
|
||||
"""
|
||||
from basic_memory.mcp.tools.utils import call_delete
|
||||
|
||||
url = f"/v2/projects/{project_external_id}"
|
||||
if delete_notes:
|
||||
url += "?delete_notes=true"
|
||||
@@ -114,6 +116,8 @@ class ProjectClient:
|
||||
Raises:
|
||||
ToolError: If the request fails
|
||||
"""
|
||||
from basic_memory.mcp.tools.utils import call_post
|
||||
|
||||
response = await call_post(
|
||||
self.http_client,
|
||||
"/v2/projects/resolve",
|
||||
@@ -133,6 +137,8 @@ class ProjectClient:
|
||||
Raises:
|
||||
ToolError: If the request fails
|
||||
"""
|
||||
from basic_memory.mcp.tools.utils import call_put
|
||||
|
||||
response = await call_put(
|
||||
self.http_client,
|
||||
f"/v2/projects/{project_external_id}/default",
|
||||
@@ -154,6 +160,8 @@ class ProjectClient:
|
||||
Raises:
|
||||
ToolError: If the request fails
|
||||
"""
|
||||
from basic_memory.mcp.tools.utils import call_patch
|
||||
|
||||
response = await call_patch(
|
||||
self.http_client,
|
||||
f"/v2/projects/{project_external_id}",
|
||||
@@ -181,6 +189,8 @@ class ProjectClient:
|
||||
Raises:
|
||||
ToolError: If the request fails
|
||||
"""
|
||||
from basic_memory.mcp.tools.utils import call_post
|
||||
|
||||
url = f"/v2/projects/{project_external_id}/sync"
|
||||
params = []
|
||||
if force_full:
|
||||
@@ -204,6 +214,8 @@ class ProjectClient:
|
||||
Raises:
|
||||
ToolError: If the request fails
|
||||
"""
|
||||
from basic_memory.mcp.tools.utils import call_post
|
||||
|
||||
response = await call_post(
|
||||
self.http_client,
|
||||
f"/v2/projects/{project_external_id}/status",
|
||||
@@ -222,6 +234,8 @@ class ProjectClient:
|
||||
Raises:
|
||||
ToolError: If the request fails
|
||||
"""
|
||||
from basic_memory.mcp.tools.utils import call_get
|
||||
|
||||
response = await call_get(
|
||||
self.http_client,
|
||||
f"/v2/projects/{project_external_id}/info",
|
||||
|
||||
@@ -6,7 +6,9 @@ Encapsulates all /v2/projects/{project_id}/resource/* endpoints.
|
||||
from httpx import AsyncClient, Response
|
||||
|
||||
import logfire
|
||||
from basic_memory.mcp.tools.utils import call_get
|
||||
# call_* helpers live in basic_memory.mcp.tools.utils; importing that at module
|
||||
# level executes the whole tools package (fastmcp + mcp SDK) during CLI startup,
|
||||
# so each method defers the import to call time instead (#886).
|
||||
|
||||
|
||||
class ResourceClient:
|
||||
@@ -49,6 +51,8 @@ class ResourceClient:
|
||||
Raises:
|
||||
ToolError: If the resource is not found or request fails
|
||||
"""
|
||||
from basic_memory.mcp.tools.utils import call_get
|
||||
|
||||
with logfire.span(
|
||||
"mcp.client.resource.read",
|
||||
client_name="resource",
|
||||
|
||||
@@ -5,7 +5,9 @@ Encapsulates all /v2/projects/{project_id}/schema/* endpoints.
|
||||
|
||||
from httpx import AsyncClient
|
||||
|
||||
from basic_memory.mcp.tools.utils import call_post, call_get
|
||||
# call_* helpers live in basic_memory.mcp.tools.utils; importing that at module
|
||||
# level executes the whole tools package (fastmcp + mcp SDK) during CLI startup,
|
||||
# so each method defers the import to call time instead (#886).
|
||||
from basic_memory.schemas.schema import (
|
||||
ValidationReport,
|
||||
InferenceReport,
|
||||
@@ -56,6 +58,8 @@ class SchemaClient:
|
||||
Raises:
|
||||
ToolError: If the request fails
|
||||
"""
|
||||
from basic_memory.mcp.tools.utils import call_post
|
||||
|
||||
params: dict[str, str] = {}
|
||||
if note_type:
|
||||
params["note_type"] = note_type
|
||||
@@ -87,6 +91,8 @@ class SchemaClient:
|
||||
Raises:
|
||||
ToolError: If the request fails
|
||||
"""
|
||||
from basic_memory.mcp.tools.utils import call_post
|
||||
|
||||
response = await call_post(
|
||||
self.http_client,
|
||||
f"{self._base_path}/infer",
|
||||
@@ -106,6 +112,8 @@ class SchemaClient:
|
||||
Raises:
|
||||
ToolError: If the request fails
|
||||
"""
|
||||
from basic_memory.mcp.tools.utils import call_get
|
||||
|
||||
response = await call_get(
|
||||
self.http_client,
|
||||
f"{self._base_path}/diff/{note_type}",
|
||||
|
||||
@@ -8,7 +8,10 @@ from typing import Any
|
||||
from httpx import AsyncClient
|
||||
|
||||
import logfire
|
||||
from basic_memory.mcp.tools.utils import call_post
|
||||
|
||||
# call_* helpers live in basic_memory.mcp.tools.utils; importing that at module
|
||||
# level executes the whole tools package (fastmcp + mcp SDK) during CLI startup,
|
||||
# so each method defers the import to call time instead (#886).
|
||||
from basic_memory.schemas.search import SearchResponse
|
||||
|
||||
|
||||
@@ -57,6 +60,8 @@ class SearchClient:
|
||||
Raises:
|
||||
ToolError: If the request fails
|
||||
"""
|
||||
from basic_memory.mcp.tools.utils import call_post
|
||||
|
||||
with logfire.span(
|
||||
"mcp.client.search.search",
|
||||
client_name="search",
|
||||
|
||||
@@ -8,10 +8,24 @@ The resolve_project_parameter function is a thin wrapper for backwards
|
||||
compatibility with existing MCP tools.
|
||||
"""
|
||||
|
||||
# PEP 563 lazy annotations keep `Context` usable in signatures without importing
|
||||
# fastmcp at module load — the fastmcp/mcp stack costs ~0.5s of CLI startup (#886).
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
from contextlib import asynccontextmanager, nullcontext
|
||||
from typing import (
|
||||
TYPE_CHECKING,
|
||||
AsyncIterator,
|
||||
Awaitable,
|
||||
Callable,
|
||||
List,
|
||||
Optional,
|
||||
Sequence,
|
||||
Tuple,
|
||||
cast,
|
||||
)
|
||||
from dataclasses import dataclass, field
|
||||
from typing import AsyncIterator, Awaitable, Callable, List, Optional, Sequence, Tuple, cast
|
||||
from uuid import UUID
|
||||
|
||||
from httpx import AsyncClient
|
||||
@@ -19,8 +33,6 @@ from httpx._types import (
|
||||
HeaderTypes,
|
||||
)
|
||||
from loguru import logger
|
||||
from fastmcp import Context
|
||||
from mcp.server.fastmcp.exceptions import ToolError
|
||||
|
||||
import logfire
|
||||
from basic_memory.config import BasicMemoryConfig, ConfigManager, ProjectMode, has_cloud_credentials
|
||||
@@ -46,6 +58,9 @@ from basic_memory.workspace_context import (
|
||||
workspace_permalink_context,
|
||||
)
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from fastmcp import Context
|
||||
|
||||
# --- Workspace provider injection ---
|
||||
# Mirrors the set_client_factory() pattern in async_client.py.
|
||||
# The cloud MCP server sets a provider that queries its own database directly,
|
||||
@@ -54,6 +69,14 @@ _workspace_provider: Optional[Callable[[], Awaitable[list[WorkspaceInfo]]]] = No
|
||||
_WORKSPACE_PROJECT_INDEX_STATE_KEY = "workspace_project_index"
|
||||
|
||||
|
||||
class WorkspaceProjectLookupMiss(ValueError):
|
||||
"""A project was absent from the workspace index (as opposed to ambiguous).
|
||||
|
||||
Misses are retried once against a freshly rebuilt index, because the
|
||||
session cache may simply predate an out-of-band project creation (#956).
|
||||
"""
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class WorkspaceProjectEntry:
|
||||
"""A cloud project resolved together with the workspace that owns it."""
|
||||
@@ -460,6 +483,17 @@ def _canonical_memory_path_for_workspace(
|
||||
# Outcome: lookups preserve the complete workspace/project canonical permalink.
|
||||
if not normalized_remainder:
|
||||
normalized_remainder = project_permalink
|
||||
|
||||
# Same index-form rule as _canonical_memory_path_for_active_route (#957):
|
||||
# without an active workspace permalink context, stored permalinks are
|
||||
# project-qualified and a workspace-prefixed pattern cannot match.
|
||||
if "*" in normalized_remainder and current_workspace_permalink_context() is None:
|
||||
return build_qualified_permalink_reference(
|
||||
project_permalink,
|
||||
normalized_remainder,
|
||||
include_project=True,
|
||||
)
|
||||
|
||||
return build_qualified_permalink_reference(
|
||||
project_permalink,
|
||||
normalized_remainder,
|
||||
@@ -477,6 +511,24 @@ def _canonical_memory_path_for_active_route(
|
||||
) -> str:
|
||||
"""Return the canonical permalink path for the currently routed project/workspace."""
|
||||
project_prefix = active_project.permalink
|
||||
|
||||
# Trigger: the path contains a glob wildcard (folder/*) and no server-side
|
||||
# workspace permalink context is active.
|
||||
# Why: patterns match raw against the search index, so they must mirror the
|
||||
# stored permalink form. The contextvar is what qualified permalinks at
|
||||
# write time — when it is absent, stored rows are project-qualified and a
|
||||
# workspace prefix (from the client's cached_workspace display state)
|
||||
# guarantees zero matches (#957). Direct lookups keep full qualification
|
||||
# because the link resolver understands it; patterns have no fallback.
|
||||
# Outcome: without the contextvar, qualify patterns with the project prefix
|
||||
# only; with it, fall through to normal workspace canonicalization.
|
||||
if "*" in path and current_workspace_permalink_context() is None:
|
||||
if not include_project:
|
||||
return path
|
||||
if path == project_prefix or path.startswith(f"{project_prefix}/"):
|
||||
return path
|
||||
return f"{project_prefix}/{path}"
|
||||
|
||||
workspace_remainder = path
|
||||
if include_project and (path == project_prefix or path.startswith(f"{project_prefix}/")):
|
||||
# Trigger: the memory URL already names the active project root/prefix
|
||||
@@ -722,9 +774,15 @@ async def _fetch_workspace_project_entries(
|
||||
|
||||
async def _ensure_workspace_project_index(
|
||||
context: Optional[Context] = None,
|
||||
*,
|
||||
force_refresh: bool = False,
|
||||
) -> WorkspaceProjectIndex:
|
||||
"""Build or load the session-local workspace/project lookup index."""
|
||||
if context:
|
||||
"""Build or load the session-local workspace/project lookup index.
|
||||
|
||||
force_refresh bypasses the cached index and rebuilds from discovery —
|
||||
used by resolve_workspace_project_identifier when a lookup misses (#956).
|
||||
"""
|
||||
if context and not force_refresh:
|
||||
cached_raw = await context.get_state(_WORKSPACE_PROJECT_INDEX_STATE_KEY)
|
||||
cached_index = _workspace_project_index_from_state(cached_raw)
|
||||
if cached_index is not None:
|
||||
@@ -798,13 +856,99 @@ async def ensure_workspace_project_index(
|
||||
return await _ensure_workspace_project_index(context=context)
|
||||
|
||||
|
||||
def _match_workspace_identifier(
|
||||
workspaces: tuple[WorkspaceInfo, ...],
|
||||
workspace_identifier: str,
|
||||
) -> WorkspaceInfo:
|
||||
"""Resolve the first segment of a qualified route to a single workspace.
|
||||
|
||||
The edit_note/write_note contract advertises that the workspace segment may be a
|
||||
slug, tenant_id, or display name. We honor those forms in a fixed priority order so
|
||||
that adding tenant_id/name support never changes the meaning of an identifier that
|
||||
already resolves today:
|
||||
|
||||
1. slug (casefold) — existing behavior, checked first so working routes are stable.
|
||||
2. tenant_id — exact match against the opaque id (no casefolding, mirroring the
|
||||
precedent in ``workspace_matches_exact_identifier``).
|
||||
3. display name (casefold) — names are not guaranteed unique, so a name that matches
|
||||
multiple workspaces is rejected rather than silently picking one.
|
||||
"""
|
||||
# Trigger: identifier equals a workspace slug (casefold).
|
||||
# Why: slug is the canonical routing key; resolving it first guarantees a workspace
|
||||
# whose display name collides with another workspace's slug yields to the slug owner.
|
||||
# Outcome: return the slug owner before tenant_id/name are considered.
|
||||
slug_matches = [
|
||||
workspace
|
||||
for workspace in workspaces
|
||||
if workspace.slug.casefold() == workspace_identifier.casefold()
|
||||
]
|
||||
if slug_matches:
|
||||
return slug_matches[0]
|
||||
|
||||
# Trigger: identifier exactly equals a workspace tenant_id (an opaque id).
|
||||
# Why: tenant_ids are unique, so an exact hit is unambiguous and needs no tie-break.
|
||||
tenant_matches = [
|
||||
workspace for workspace in workspaces if workspace.tenant_id == workspace_identifier
|
||||
]
|
||||
if tenant_matches:
|
||||
return tenant_matches[0]
|
||||
|
||||
# Trigger: identifier matches one or more workspace display names (casefold).
|
||||
# Why: names are not guaranteed unique; failing fast on collisions keeps routing
|
||||
# deterministic and tells the caller exactly how to disambiguate.
|
||||
name_matches = [
|
||||
workspace
|
||||
for workspace in workspaces
|
||||
if workspace.name.casefold() == workspace_identifier.casefold()
|
||||
]
|
||||
if len(name_matches) > 1:
|
||||
candidates = ", ".join(workspace.slug for workspace in name_matches)
|
||||
raise ValueError(
|
||||
f"Workspace name '{workspace_identifier}' matched multiple workspaces "
|
||||
f"(slugs: {candidates}). Use the workspace slug or tenant_id to disambiguate."
|
||||
)
|
||||
if name_matches:
|
||||
return name_matches[0]
|
||||
|
||||
available = ", ".join(workspace.slug for workspace in workspaces)
|
||||
raise ValueError(
|
||||
f"Workspace '{workspace_identifier}' was not found by slug, tenant_id, or name. "
|
||||
f"Available workspace slugs: {available}"
|
||||
)
|
||||
|
||||
|
||||
async def resolve_workspace_project_identifier(
|
||||
project: str,
|
||||
context: Optional[Context] = None,
|
||||
) -> WorkspaceProjectEntry:
|
||||
"""Resolve a project by external_id (UUID), qualified name, or unqualified name."""
|
||||
index = await _ensure_workspace_project_index(context=context)
|
||||
try:
|
||||
return await _resolve_workspace_project_from_index(index, project, context)
|
||||
except WorkspaceProjectLookupMiss:
|
||||
# Trigger: the lookup missed the session-cached index.
|
||||
# Why: a miss is exactly the signal the cache may be stale — projects
|
||||
# created out-of-band (CLI, a teammate in a shared workspace) post-date
|
||||
# the index built at session start (#956).
|
||||
# Outcome: rebuild the index once and retry; a second miss is authoritative
|
||||
# and its error (with the refreshed project list) propagates.
|
||||
logger.info(
|
||||
f"Workspace project lookup missed for '{project}'; refreshing index and retrying"
|
||||
)
|
||||
refreshed = await _ensure_workspace_project_index(context=context, force_refresh=True)
|
||||
return await _resolve_workspace_project_from_index(refreshed, project, context)
|
||||
|
||||
|
||||
async def _resolve_workspace_project_from_index(
|
||||
index: WorkspaceProjectIndex,
|
||||
project: str,
|
||||
context: Optional[Context] = None,
|
||||
) -> WorkspaceProjectEntry:
|
||||
"""Resolve a project against one concrete index snapshot.
|
||||
|
||||
Raises WorkspaceProjectLookupMiss for absent projects (retryable via index
|
||||
refresh) and plain ValueError for ambiguity, which a refresh cannot fix.
|
||||
"""
|
||||
# Fast path: direct lookup by external_id when the identifier is a UUID
|
||||
# Canonicalize via str(UUID(...)) so uppercase, brace-wrapped, or urn:uuid forms
|
||||
# all hash to the same lowercase-hyphenated key as the stored external_ids.
|
||||
@@ -816,23 +960,14 @@ async def resolve_workspace_project_identifier(
|
||||
except ValueError:
|
||||
pass
|
||||
|
||||
workspace_slug, project_identifier = _split_qualified_project_identifier(project)
|
||||
workspace_identifier, project_identifier = _split_qualified_project_identifier(project)
|
||||
project_permalink = generate_permalink(project_identifier)
|
||||
|
||||
if workspace_slug:
|
||||
workspace_matches = [
|
||||
workspace
|
||||
for workspace in index.workspaces
|
||||
if workspace.slug.casefold() == workspace_slug.casefold()
|
||||
]
|
||||
if not workspace_matches:
|
||||
available = ", ".join(workspace.slug for workspace in index.workspaces)
|
||||
raise ValueError(
|
||||
f"Workspace '{workspace_slug}' was not found. "
|
||||
f"Available workspace slugs: {available}"
|
||||
)
|
||||
|
||||
workspace = workspace_matches[0]
|
||||
if workspace_identifier:
|
||||
# Honor the documented "slug, name, or tenant_id" contract for the workspace
|
||||
# segment; _match_workspace_identifier raises a clear error on ambiguous names
|
||||
# and unknown identifiers, listing what forms were tried.
|
||||
workspace = _match_workspace_identifier(index.workspaces, workspace_identifier)
|
||||
matches = [
|
||||
entry
|
||||
for entry in index.entries_by_permalink.get(project_permalink, ())
|
||||
@@ -843,7 +978,7 @@ async def resolve_workspace_project_identifier(
|
||||
failed_workspace.tenant_id == workspace.tenant_id
|
||||
for failed_workspace in index.failed_workspaces
|
||||
):
|
||||
raise ValueError(
|
||||
raise WorkspaceProjectLookupMiss(
|
||||
f"Projects for workspace '{workspace.name}' ({workspace.slug}) "
|
||||
"could not be loaded. Retry after workspace discovery recovers."
|
||||
)
|
||||
@@ -852,7 +987,7 @@ async def resolve_workspace_project_identifier(
|
||||
for entry in index.entries
|
||||
if entry.workspace.tenant_id == workspace.tenant_id
|
||||
)
|
||||
raise ValueError(
|
||||
raise WorkspaceProjectLookupMiss(
|
||||
f"Project '{project_identifier}' was not found in workspace "
|
||||
f"'{workspace.name}' ({workspace.slug}). Available projects: {available}"
|
||||
)
|
||||
@@ -877,7 +1012,7 @@ async def resolve_workspace_project_identifier(
|
||||
"retry or use a qualified project from an indexed workspace."
|
||||
)
|
||||
available = ", ".join(entry.qualified_name for entry in index.entries)
|
||||
raise ValueError(
|
||||
raise WorkspaceProjectLookupMiss(
|
||||
f"Project '{project}' was not found in indexed cloud workspaces. "
|
||||
f"Available projects: {available}.{failed_note}"
|
||||
)
|
||||
@@ -1212,6 +1347,9 @@ async def resolve_project_and_path(
|
||||
# Why: allow project-scoped memory URLs without requiring a separate project parameter
|
||||
# Outcome: attempt to resolve the prefix as a project and route to it
|
||||
if project_prefix:
|
||||
# Deferred: ToolError lives in the mcp SDK, which must not load at CLI startup (#886).
|
||||
from mcp.server.fastmcp.exceptions import ToolError
|
||||
|
||||
if cached_project and _project_matches_identifier(cached_project, project_prefix):
|
||||
resolved_project = await resolve_project_parameter(project_prefix, context=context)
|
||||
if resolved_project and generate_permalink(resolved_project) != generate_permalink(
|
||||
@@ -1438,6 +1576,9 @@ async def get_project_client(
|
||||
is_factory_mode,
|
||||
)
|
||||
|
||||
# Deferred: ToolError lives in the mcp SDK, which must not load at CLI startup (#886).
|
||||
from mcp.server.fastmcp.exceptions import ToolError
|
||||
|
||||
# When project_id (UUID) is provided, prefer it as the resolution identifier.
|
||||
# external_id is unambiguous across workspaces; project name can collide.
|
||||
project_identifier = project_id if project_id else project
|
||||
|
||||
@@ -0,0 +1,17 @@
|
||||
"""Shared loader for the bundled cloud-discovery markdown resources."""
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
def load_discovery_resource(filename: str) -> str:
|
||||
"""Read a bundled discovery markdown file with promo placeholders rendered.
|
||||
|
||||
The markdown carries a {{OSS_DISCOUNT_CODE}} placeholder so the promo code
|
||||
has one source of truth (cli.promo); substitute before it reaches users.
|
||||
"""
|
||||
# Import here to avoid pulling CLI promo machinery (analytics, rich, config)
|
||||
# into the MCP server import graph at module load.
|
||||
from basic_memory.cli.promo import OSS_DISCOUNT_CODE
|
||||
|
||||
content = (Path(__file__).parent / filename).read_text(encoding="utf-8")
|
||||
return content.replace("{{OSS_DISCOUNT_CODE}}", OSS_DISCOUNT_CODE)
|
||||
@@ -54,7 +54,10 @@ def _format_entity_block(result: ContextResult) -> str:
|
||||
lines.append("")
|
||||
lines.append("### Relations")
|
||||
for rel in relation_items:
|
||||
lines.append(f"- {rel.relation_type} [[{rel.to_entity}]]")
|
||||
# Unresolved forward references have no resolved entity yet; fall back
|
||||
# to the literal target text instead of rendering [[None]] (#955)
|
||||
target = rel.to_entity or rel.to_name
|
||||
lines.append(f"- {rel.relation_type} [[{target}]]")
|
||||
|
||||
# --- Related entities (non-relation related results) ---
|
||||
related_entities: list[EntitySummary | ObservationSummary] = [
|
||||
@@ -112,6 +115,7 @@ def _format_context_markdown(graph: GraphContext, project: str) -> str:
|
||||
|
||||
|
||||
@mcp.tool(
|
||||
title="Build Context",
|
||||
description="""Build context from a memory:// URI to continue conversations naturally.
|
||||
|
||||
Use this to follow up on previous discussions or explore related topics.
|
||||
@@ -131,6 +135,7 @@ def _format_context_markdown(graph: GraphContext, project: str) -> str:
|
||||
- "json" (default): Structured JSON with internal fields excluded
|
||||
- "text": Compact markdown text for LLM consumption
|
||||
""",
|
||||
tags={"navigation", "notes"},
|
||||
annotations={"readOnlyHint": True, "openWorldHint": False},
|
||||
)
|
||||
async def build_context(
|
||||
|
||||
@@ -17,7 +17,9 @@ from basic_memory.mcp.tools.utils import call_put, call_post, resolve_entity_id
|
||||
|
||||
|
||||
@mcp.tool(
|
||||
title="Create Canvas",
|
||||
description="Create an Obsidian canvas file to visualize concepts and connections.",
|
||||
tags={"canvas", "notes"},
|
||||
annotations={"destructiveHint": False, "idempotentHint": True, "openWorldHint": False},
|
||||
)
|
||||
async def canvas(
|
||||
|
||||
@@ -105,7 +105,9 @@ def _format_document_for_chatgpt(
|
||||
|
||||
|
||||
@mcp.tool(
|
||||
title="Search Knowledge Base",
|
||||
description="Search for content across the knowledge base",
|
||||
tags={"search"},
|
||||
annotations={"readOnlyHint": True, "openWorldHint": False},
|
||||
)
|
||||
async def search(
|
||||
@@ -167,7 +169,9 @@ async def search(
|
||||
|
||||
|
||||
@mcp.tool(
|
||||
title="Fetch Document",
|
||||
description="Fetch the full contents of a search result document",
|
||||
tags={"search", "notes"},
|
||||
annotations={"readOnlyHint": True, "openWorldHint": False},
|
||||
)
|
||||
async def fetch(
|
||||
|
||||
@@ -1,15 +1,15 @@
|
||||
"""Cloud information MCP tool."""
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
from basic_memory.mcp.resources.discovery import load_discovery_resource
|
||||
from basic_memory.mcp.server import mcp
|
||||
|
||||
|
||||
@mcp.tool(
|
||||
"cloud_info",
|
||||
title="Cloud Info",
|
||||
tags={"cloud"},
|
||||
annotations={"readOnlyHint": True, "openWorldHint": False},
|
||||
)
|
||||
def cloud_info() -> str:
|
||||
"""Return optional Basic Memory Cloud information and setup guidance."""
|
||||
content_path = Path(__file__).parent.parent / "resources" / "cloud_info.md"
|
||||
return content_path.read_text(encoding="utf-8")
|
||||
return load_discovery_resource("cloud_info.md")
|
||||
|
||||
@@ -182,7 +182,9 @@ def _directory_path_for_delete(
|
||||
|
||||
|
||||
@mcp.tool(
|
||||
title="Delete Note",
|
||||
description="Delete a note or directory by title, permalink, or path",
|
||||
tags={"notes"},
|
||||
annotations={"destructiveHint": True, "openWorldHint": False},
|
||||
)
|
||||
async def delete_note(
|
||||
|
||||
@@ -304,7 +304,9 @@ Error editing note '{identifier}': {error_message}
|
||||
|
||||
|
||||
@mcp.tool(
|
||||
title="Edit Note",
|
||||
description="Edit an existing markdown note using various operations like append, prepend, find_replace, replace_section, insert_before_section, or insert_after_section.",
|
||||
tags={"notes"},
|
||||
annotations={"destructiveHint": False, "openWorldHint": False},
|
||||
)
|
||||
async def edit_note(
|
||||
|
||||
@@ -11,7 +11,9 @@ from basic_memory.mcp.server import mcp
|
||||
|
||||
|
||||
@mcp.tool(
|
||||
title="List Directory",
|
||||
description="List directory contents with filtering and depth control.",
|
||||
tags={"navigation", "notes"},
|
||||
annotations={"readOnlyHint": True, "openWorldHint": False},
|
||||
)
|
||||
async def list_directory(
|
||||
|
||||
@@ -357,7 +357,9 @@ delete_note("{identifier}")
|
||||
|
||||
|
||||
@mcp.tool(
|
||||
title="Move Note",
|
||||
description="Move a note or directory to a new location, updating database and maintaining links.",
|
||||
tags={"notes"},
|
||||
annotations={"destructiveHint": False, "openWorldHint": False},
|
||||
)
|
||||
async def move_note(
|
||||
|
||||
@@ -357,6 +357,8 @@ def _format_project_list_json(
|
||||
|
||||
@mcp.tool(
|
||||
"list_memory_projects",
|
||||
title="List Memory Projects",
|
||||
tags={"projects"},
|
||||
annotations={"readOnlyHint": True, "openWorldHint": False},
|
||||
)
|
||||
async def list_memory_projects(
|
||||
@@ -475,10 +477,14 @@ async def _resolve_workspace_routing(
|
||||
if workspace is None:
|
||||
return None
|
||||
|
||||
explicit_cloud_routing = _explicit_routing() and not _force_local_mode()
|
||||
forced_local = _explicit_routing() and _force_local_mode()
|
||||
config = ConfigManager().config
|
||||
# Resolve whenever credentials make workspace discovery possible — not only
|
||||
# under explicit --cloud. A workspace selector implies cloud routing
|
||||
# (get_client routes it to the cloud proxy, #954), and the transport needs
|
||||
# the tenant id in X-Workspace-ID, not a slug or display name.
|
||||
should_resolve_workspace = is_factory_mode() or (
|
||||
explicit_cloud_routing and has_cloud_credentials(config)
|
||||
has_cloud_credentials(config) and not forced_local
|
||||
)
|
||||
if not should_resolve_workspace:
|
||||
return workspace
|
||||
@@ -495,6 +501,8 @@ async def _resolve_workspace_routing(
|
||||
|
||||
@mcp.tool(
|
||||
"create_memory_project",
|
||||
title="Create Memory Project",
|
||||
tags={"projects"},
|
||||
annotations={"destructiveHint": False, "openWorldHint": False},
|
||||
)
|
||||
async def create_memory_project(
|
||||
@@ -517,8 +525,9 @@ async def create_memory_project(
|
||||
workspace: Optional cloud workspace selector to create the project in. Slug is
|
||||
preferred for AI callers, but tenant_id and unique name are also accepted.
|
||||
When omitted, the connection's default workspace is used. Discover values
|
||||
via `list_workspaces`. In local mode the selector is passed through
|
||||
without slug resolution.
|
||||
via `list_workspaces`. A workspace selector implies cloud routing:
|
||||
without cloud credentials the call fails fast instead of silently
|
||||
creating a local project (#954).
|
||||
output_format: "text" returns the existing human-readable result text.
|
||||
"json" returns structured project creation metadata.
|
||||
context: Optional FastMCP context for progress/status logging.
|
||||
@@ -636,6 +645,8 @@ async def create_memory_project(
|
||||
|
||||
|
||||
@mcp.tool(
|
||||
title="Delete Project",
|
||||
tags={"projects"},
|
||||
annotations={"destructiveHint": True, "openWorldHint": False},
|
||||
)
|
||||
async def delete_project(
|
||||
@@ -654,8 +665,9 @@ async def delete_project(
|
||||
workspace: Optional cloud workspace selector to delete the project from.
|
||||
Slug is preferred for AI callers, but tenant_id and unique name are
|
||||
also accepted. When omitted, the connection's default workspace is
|
||||
used. In local mode the selector is passed through without slug
|
||||
resolution, matching create_memory_project behavior.
|
||||
used. A workspace selector implies cloud routing: without cloud
|
||||
credentials the call fails fast, matching create_memory_project
|
||||
behavior (#954).
|
||||
|
||||
Returns:
|
||||
Confirmation message about project deletion
|
||||
|
||||
@@ -155,7 +155,9 @@ def optimize_image(img, content_length, max_output_bytes=350000):
|
||||
|
||||
|
||||
@mcp.tool(
|
||||
title="Read Content",
|
||||
description="Read a file's raw content by path or permalink",
|
||||
tags={"notes"},
|
||||
annotations={"readOnlyHint": True, "openWorldHint": False},
|
||||
)
|
||||
async def read_content(
|
||||
|
||||
@@ -74,7 +74,9 @@ def _parse_opening_frontmatter(content: str) -> tuple[str, dict | None]:
|
||||
|
||||
|
||||
@mcp.tool(
|
||||
title="Read Note",
|
||||
description="Read a markdown note by title or permalink.",
|
||||
tags={"notes"},
|
||||
# TODO: re-enable once MCP client rendering is working
|
||||
# meta={"ui/resourceUri": "ui://basic-memory/note-preview"},
|
||||
annotations={"readOnlyHint": True, "openWorldHint": False},
|
||||
|
||||
@@ -26,6 +26,7 @@ from basic_memory.schemas.search import SearchItemType
|
||||
|
||||
|
||||
@mcp.tool(
|
||||
title="Recent Activity",
|
||||
description="""Get recent activity for a project or across all projects.
|
||||
|
||||
Timeframe supports natural language formats like:
|
||||
@@ -36,6 +37,7 @@ from basic_memory.schemas.search import SearchItemType
|
||||
- "3 weeks ago"
|
||||
Or standard formats like "7d"
|
||||
""",
|
||||
tags={"navigation", "notes"},
|
||||
annotations={"readOnlyHint": True, "openWorldHint": False},
|
||||
)
|
||||
async def recent_activity(
|
||||
|
||||
@@ -1,15 +1,15 @@
|
||||
"""Release notes MCP tool."""
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
from basic_memory.mcp.resources.discovery import load_discovery_resource
|
||||
from basic_memory.mcp.server import mcp
|
||||
|
||||
|
||||
@mcp.tool(
|
||||
"release_notes",
|
||||
title="Release Notes",
|
||||
tags={"cloud"},
|
||||
annotations={"readOnlyHint": True, "openWorldHint": False},
|
||||
)
|
||||
def release_notes() -> str:
|
||||
"""Return the latest product release notes for optional user review."""
|
||||
content_path = Path(__file__).parent.parent / "resources" / "release_notes.md"
|
||||
return content_path.read_text(encoding="utf-8")
|
||||
return load_discovery_resource("release_notes.md")
|
||||
|
||||
@@ -204,7 +204,9 @@ def _no_schema_guidance(note_type: str, tool_name: str) -> str:
|
||||
|
||||
|
||||
@mcp.tool(
|
||||
title="Validate Schema",
|
||||
description="Validate notes against their Picoschema definitions.",
|
||||
tags={"schema"},
|
||||
annotations={"readOnlyHint": True, "openWorldHint": False},
|
||||
)
|
||||
async def schema_validate(
|
||||
@@ -317,7 +319,9 @@ async def schema_validate(
|
||||
|
||||
|
||||
@mcp.tool(
|
||||
title="Infer Schema",
|
||||
description="Analyze existing notes and suggest a Picoschema definition.",
|
||||
tags={"schema"},
|
||||
annotations={"readOnlyHint": True, "openWorldHint": False},
|
||||
)
|
||||
async def schema_infer(
|
||||
@@ -438,7 +442,9 @@ async def schema_infer(
|
||||
|
||||
|
||||
@mcp.tool(
|
||||
title="Schema Diff",
|
||||
description="Detect drift between a schema definition and actual note usage.",
|
||||
tags={"schema"},
|
||||
annotations={"readOnlyHint": True, "openWorldHint": False},
|
||||
)
|
||||
async def schema_diff(
|
||||
|
||||
@@ -14,7 +14,7 @@ from basic_memory.config import ConfigManager, has_cloud_credentials
|
||||
from basic_memory.utils import (
|
||||
build_canonical_permalink,
|
||||
coerce_dict,
|
||||
coerce_list,
|
||||
parse_str_list,
|
||||
parse_tags,
|
||||
strict_search_tags,
|
||||
)
|
||||
@@ -612,7 +612,9 @@ async def _search_all_projects(
|
||||
|
||||
|
||||
@mcp.tool(
|
||||
title="Search Notes",
|
||||
description="Search across all content in the knowledge base with advanced syntax support.",
|
||||
tags={"search"},
|
||||
# TODO: re-enable once MCP client rendering is working
|
||||
# meta={"ui/resourceUri": "ui://basic-memory/search-results"},
|
||||
annotations={"readOnlyHint": True, "openWorldHint": False},
|
||||
@@ -649,24 +651,30 @@ async def search_notes(
|
||||
# Plural-vs-singular trips models constantly. Accept the singular too.
|
||||
note_types: Annotated[
|
||||
List[str] | None,
|
||||
BeforeValidator(coerce_list),
|
||||
# parse_str_list, not coerce_list: "note,task" must split into ["note", "task"]
|
||||
# consistent with how tags are handled (#910/#930). coerce_list wraps the whole
|
||||
# comma string as the single literal type ["note,task"], which matches nothing.
|
||||
BeforeValidator(parse_str_list),
|
||||
Field(default=None, validation_alias=AliasChoices("note_types", "note_type", "types")),
|
||||
"Filter by the 'type' field in note frontmatter (e.g. 'note', 'chapter', 'person'). "
|
||||
"Accepts a list, a comma-separated string (e.g. 'note,task'), or a JSON-array string. "
|
||||
"Case-insensitive.",
|
||||
] = None,
|
||||
entity_types: Annotated[
|
||||
List[str] | None,
|
||||
BeforeValidator(coerce_list),
|
||||
BeforeValidator(parse_str_list),
|
||||
Field(default=None, validation_alias=AliasChoices("entity_types", "entity_type")),
|
||||
"Filter by knowledge graph item type: 'entity' (whole notes), 'observation', or "
|
||||
"'relation'. Defaults to 'entity'. Do NOT pass schema/frontmatter types like "
|
||||
"'Chapter' here — use note_types instead.",
|
||||
"'Chapter' here — use note_types instead. "
|
||||
"Accepts a list, a comma-separated string (e.g. 'entity,observation'), or a JSON-array string.",
|
||||
] = None,
|
||||
categories: Annotated[
|
||||
List[str] | None,
|
||||
BeforeValidator(coerce_list),
|
||||
BeforeValidator(parse_str_list),
|
||||
Field(default=None, validation_alias=AliasChoices("categories", "category")),
|
||||
"Filter observation results to these exact categories (e.g. ['requirement']). "
|
||||
"Accepts a list, a comma-separated string (e.g. 'requirement,decision'), or a JSON-array string. "
|
||||
"Pair with entity_types=['observation'] to return only observations whose "
|
||||
"category matches exactly — not every row mentioning the word.",
|
||||
] = None,
|
||||
@@ -820,6 +828,11 @@ async def search_notes(
|
||||
Formatted markdown text (output_format="text"), dict (output_format="json"),
|
||||
or helpful error guidance string if search fails
|
||||
|
||||
Pagination note: `total` is exact only for text/title/permalink searches.
|
||||
Vector and hybrid searches skip the count query (it would cost a second
|
||||
semantic retrieval pass) and report `total: 0` even when results are
|
||||
returned — use `has_more` for pagination in those modes.
|
||||
|
||||
Examples:
|
||||
# Basic text search
|
||||
results = await search_notes("project planning")
|
||||
@@ -893,6 +906,17 @@ async def search_notes(
|
||||
if page_size < 1:
|
||||
raise ValueError(f"page_size must be >= 1, got {page_size}")
|
||||
|
||||
# Trigger: list params arrived via a direct function call instead of the MCP layer.
|
||||
# Why: the BeforeValidator annotations only run through MCP/Pydantic validation; direct
|
||||
# callers (e.g. `bm tool search-notes --type note,task` in cli/commands/tool.py,
|
||||
# which Typer collects as the one-element list ["note,task"]) would otherwise
|
||||
# forward the comma string as one literal type that matches nothing (#930).
|
||||
# Outcome: comma-split/list normalization applies on every path; parse_str_list is
|
||||
# idempotent, so MCP-validated input passes through unchanged.
|
||||
note_types = parse_str_list(note_types) if note_types is not None else []
|
||||
entity_types = parse_str_list(entity_types) if entity_types is not None else []
|
||||
categories = parse_str_list(categories) if categories is not None else []
|
||||
|
||||
# Avoid mutable-default-argument footguns. Treat None as "no filter".
|
||||
# Lowercase note_types so "Chapter" matches the stored "chapter".
|
||||
note_types = [t.lower() for t in note_types] if note_types else []
|
||||
|
||||
@@ -18,7 +18,9 @@ def _text_block(message: str) -> List[ContentBlock]:
|
||||
|
||||
|
||||
@mcp.tool(
|
||||
title="Search Notes (UI)",
|
||||
description="Search notes and return an embedded MCP-UI resource (raw HTML).",
|
||||
tags={"search", "ui"},
|
||||
output_schema=None,
|
||||
annotations={"readOnlyHint": True, "openWorldHint": False},
|
||||
)
|
||||
@@ -92,7 +94,9 @@ async def search_notes_ui(
|
||||
|
||||
|
||||
@mcp.tool(
|
||||
title="Read Note (UI)",
|
||||
description="Read a note and return an embedded MCP-UI resource (raw HTML).",
|
||||
tags={"notes", "ui"},
|
||||
output_schema=None,
|
||||
annotations={"readOnlyHint": True, "openWorldHint": False},
|
||||
)
|
||||
|
||||
@@ -11,7 +11,9 @@ from basic_memory.mcp.tools.read_note import read_note
|
||||
|
||||
|
||||
@mcp.tool(
|
||||
title="View Note",
|
||||
description="View a note as a formatted artifact for better readability.",
|
||||
tags={"notes"},
|
||||
annotations={"readOnlyHint": True, "openWorldHint": False},
|
||||
)
|
||||
async def view_note(
|
||||
|
||||
@@ -45,7 +45,9 @@ def _workspace_list_response(workspaces: list[WorkspaceInfo]) -> WorkspaceListRe
|
||||
|
||||
|
||||
@mcp.tool(
|
||||
title="List Workspaces",
|
||||
description="List available cloud workspaces (tenant_id, type, role, and name).",
|
||||
tags={"cloud", "projects"},
|
||||
annotations={"readOnlyHint": True, "openWorldHint": False},
|
||||
)
|
||||
async def list_workspaces(
|
||||
|
||||
@@ -25,8 +25,39 @@ from basic_memory.workspace_context import current_workspace_permalink_context
|
||||
TagType = Union[List[str], str, None]
|
||||
|
||||
|
||||
def _compose_workspace_project_route(
|
||||
*,
|
||||
workspace: Optional[str],
|
||||
project: Optional[str],
|
||||
project_id: Optional[str],
|
||||
) -> Optional[str]:
|
||||
"""Return the explicit project route requested by workspace/project args."""
|
||||
if workspace is None:
|
||||
return project
|
||||
|
||||
cleaned_workspace = workspace.strip().strip("/")
|
||||
if not cleaned_workspace:
|
||||
raise ValueError("workspace must not be empty when provided")
|
||||
if "/" in cleaned_workspace:
|
||||
raise ValueError("workspace must be a single workspace slug, name, or tenant_id")
|
||||
if project_id is not None:
|
||||
raise ValueError("workspace cannot be combined with project_id; use project_id alone")
|
||||
if project is None or not project.strip().strip("/"):
|
||||
raise ValueError("workspace requires an explicit project argument")
|
||||
|
||||
cleaned_project = project.strip().strip("/")
|
||||
if "/" in cleaned_project:
|
||||
raise ValueError(
|
||||
"Use either workspace='workspace' with project='project', "
|
||||
"or project='workspace/project', not both"
|
||||
)
|
||||
return f"{cleaned_workspace}/{cleaned_project}"
|
||||
|
||||
|
||||
@mcp.tool(
|
||||
title="Write Note",
|
||||
description="Create a markdown note. If the note already exists, returns an error by default — pass overwrite=True to replace.",
|
||||
tags={"notes"},
|
||||
annotations={"destructiveHint": True, "idempotentHint": False, "openWorldHint": False},
|
||||
)
|
||||
async def write_note(
|
||||
@@ -38,6 +69,7 @@ async def write_note(
|
||||
Field(validation_alias=AliasChoices("directory", "folder", "dir", "path")),
|
||||
],
|
||||
project: Optional[str] = None,
|
||||
workspace: Optional[str] = None,
|
||||
project_id: Optional[str] = None,
|
||||
tags: list[str] | str | None = None,
|
||||
note_type: str = "note",
|
||||
@@ -93,6 +125,8 @@ async def write_note(
|
||||
form (or project_id) to disambiguate. If unknown, use
|
||||
list_memory_projects() to discover available projects and their
|
||||
qualified names.
|
||||
workspace: Workspace slug, name, or tenant_id. When provided with `project`,
|
||||
routes as `workspace/project`. Cannot be combined with `project_id`.
|
||||
project_id: Project external_id (UUID). Prefer this over `project` when known —
|
||||
it routes to the exact project regardless of name collisions across cloud
|
||||
workspaces. Takes precedence over `project`. Get from list_memory_projects().
|
||||
@@ -170,6 +204,11 @@ async def write_note(
|
||||
effective_overwrite = (
|
||||
overwrite if overwrite is not None else ConfigManager().config.write_note_overwrite_default
|
||||
)
|
||||
project = _compose_workspace_project_route(
|
||||
workspace=workspace,
|
||||
project=project,
|
||||
project_id=project_id,
|
||||
)
|
||||
|
||||
with logfire.span(
|
||||
"mcp.tool.write_note",
|
||||
|
||||
@@ -7,12 +7,11 @@ from typing import List, Optional, Sequence, Union, Any
|
||||
from loguru import logger
|
||||
from sqlalchemy import exists, func, select
|
||||
from sqlalchemy.exc import IntegrityError
|
||||
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
from sqlalchemy.orm import load_only, selectinload
|
||||
from sqlalchemy.orm.interfaces import LoaderOption
|
||||
from sqlalchemy.engine import Row
|
||||
|
||||
from basic_memory import db
|
||||
from basic_memory.models.knowledge import Entity, Observation, Relation
|
||||
from basic_memory.repository.repository import Repository
|
||||
|
||||
@@ -24,16 +23,17 @@ class EntityRepository(Repository[Entity]):
|
||||
to strings before passing to repository methods.
|
||||
"""
|
||||
|
||||
def __init__(self, session_maker: async_sessionmaker[AsyncSession], project_id: int):
|
||||
"""Initialize with session maker and project_id filter.
|
||||
def __init__(self, project_id: int):
|
||||
"""Initialize with project_id filter.
|
||||
|
||||
Args:
|
||||
session_maker: SQLAlchemy session maker
|
||||
project_id: Project ID to filter all operations by
|
||||
"""
|
||||
super().__init__(session_maker, Entity, project_id=project_id)
|
||||
super().__init__(Entity, project_id=project_id)
|
||||
|
||||
async def get_by_id(self, entity_id: int, *, load_relations: bool = True) -> Optional[Entity]:
|
||||
async def get_by_id(
|
||||
self, session: AsyncSession, entity_id: int, *, load_relations: bool = True
|
||||
) -> Optional[Entity]:
|
||||
"""Get entity by numeric ID.
|
||||
|
||||
Args:
|
||||
@@ -42,23 +42,24 @@ class EntityRepository(Repository[Entity]):
|
||||
Returns:
|
||||
Entity if found, None otherwise
|
||||
"""
|
||||
async with db.scoped_session(self.session_maker) as session:
|
||||
if not load_relations:
|
||||
result = await session.execute(self.select().where(Entity.id == entity_id))
|
||||
return result.scalars().one_or_none()
|
||||
if not load_relations:
|
||||
result = await session.execute(self.select().where(Entity.id == entity_id))
|
||||
return result.scalars().one_or_none()
|
||||
|
||||
return await self.select_by_id(session, entity_id)
|
||||
return await self.select_by_id(session, entity_id)
|
||||
|
||||
async def _find_one_by_query(self, query, *, load_relations: bool) -> Optional[Entity]:
|
||||
async def _find_one_by_query(
|
||||
self, session: AsyncSession, query, *, load_relations: bool
|
||||
) -> Optional[Entity]:
|
||||
"""Return one entity row with optional eager loading."""
|
||||
if load_relations:
|
||||
return await self.find_one(query)
|
||||
return await self.find_one(session, query)
|
||||
|
||||
result = await self.execute_query(query, use_query_options=False)
|
||||
result = await self.execute_query(session, query, use_query_options=False)
|
||||
return result.scalars().one_or_none()
|
||||
|
||||
async def get_by_external_id(
|
||||
self, external_id: str, *, load_relations: bool = True
|
||||
self, session: AsyncSession, external_id: str, *, load_relations: bool = True
|
||||
) -> Optional[Entity]:
|
||||
"""Get entity by external UUID.
|
||||
|
||||
@@ -69,10 +70,10 @@ class EntityRepository(Repository[Entity]):
|
||||
Entity if found, None otherwise
|
||||
"""
|
||||
query = self.select().where(Entity.external_id == external_id)
|
||||
return await self._find_one_by_query(query, load_relations=load_relations)
|
||||
return await self._find_one_by_query(session, query, load_relations=load_relations)
|
||||
|
||||
async def get_by_permalink(
|
||||
self, permalink: str, *, load_relations: bool = True
|
||||
self, session: AsyncSession, permalink: str, *, load_relations: bool = True
|
||||
) -> Optional[Entity]:
|
||||
"""Get entity by permalink.
|
||||
|
||||
@@ -80,9 +81,11 @@ class EntityRepository(Repository[Entity]):
|
||||
permalink: Unique identifier for the entity
|
||||
"""
|
||||
query = self.select().where(Entity.permalink == permalink)
|
||||
return await self._find_one_by_query(query, load_relations=load_relations)
|
||||
return await self._find_one_by_query(session, query, load_relations=load_relations)
|
||||
|
||||
async def get_by_title(self, title: str, *, load_relations: bool = True) -> Sequence[Entity]:
|
||||
async def get_by_title(
|
||||
self, session: AsyncSession, title: str, *, load_relations: bool = True
|
||||
) -> Sequence[Entity]:
|
||||
"""Get entities by title, ordered by shortest path first.
|
||||
|
||||
When multiple entities share the same title (in different folders),
|
||||
@@ -97,11 +100,11 @@ class EntityRepository(Repository[Entity]):
|
||||
.where(Entity.title == title)
|
||||
.order_by(func.length(Entity.file_path), Entity.file_path)
|
||||
)
|
||||
result = await self.execute_query(query, use_query_options=load_relations)
|
||||
result = await self.execute_query(session, query, use_query_options=load_relations)
|
||||
return list(result.scalars().all())
|
||||
|
||||
async def get_by_file_path(
|
||||
self, file_path: Union[Path, str], *, load_relations: bool = True
|
||||
self, session: AsyncSession, file_path: Union[Path, str], *, load_relations: bool = True
|
||||
) -> Optional[Entity]:
|
||||
"""Get entity by file_path.
|
||||
|
||||
@@ -109,13 +112,13 @@ class EntityRepository(Repository[Entity]):
|
||||
file_path: Path to the entity file (will be converted to string internally)
|
||||
"""
|
||||
query = self.select().where(Entity.file_path == Path(file_path).as_posix())
|
||||
return await self._find_one_by_query(query, load_relations=load_relations)
|
||||
return await self._find_one_by_query(session, query, load_relations=load_relations)
|
||||
|
||||
# -------------------------------------------------------------------------
|
||||
# Lightweight methods for permalink resolution (no eager loading)
|
||||
# -------------------------------------------------------------------------
|
||||
|
||||
async def permalink_exists(self, permalink: str) -> bool:
|
||||
async def permalink_exists(self, session: AsyncSession, permalink: str) -> bool:
|
||||
"""Check if a permalink exists without loading the full entity.
|
||||
|
||||
This is much faster than get_by_permalink() as it skips eager loading
|
||||
@@ -129,10 +132,12 @@ class EntityRepository(Repository[Entity]):
|
||||
"""
|
||||
query = select(Entity.id).where(Entity.permalink == permalink).limit(1)
|
||||
query = self._add_project_filter(query)
|
||||
result = await self.execute_query(query, use_query_options=False)
|
||||
result = await self.execute_query(session, query, use_query_options=False)
|
||||
return result.scalar_one_or_none() is not None
|
||||
|
||||
async def get_file_path_for_permalink(self, permalink: str) -> Optional[str]:
|
||||
async def get_file_path_for_permalink(
|
||||
self, session: AsyncSession, permalink: str
|
||||
) -> Optional[str]:
|
||||
"""Get the file_path for a permalink without loading the full entity.
|
||||
|
||||
Use when you only need the file_path, not the full entity with relations.
|
||||
@@ -145,10 +150,12 @@ class EntityRepository(Repository[Entity]):
|
||||
"""
|
||||
query = select(Entity.file_path).where(Entity.permalink == permalink)
|
||||
query = self._add_project_filter(query)
|
||||
result = await self.execute_query(query, use_query_options=False)
|
||||
result = await self.execute_query(session, query, use_query_options=False)
|
||||
return result.scalar_one_or_none()
|
||||
|
||||
async def get_permalink_for_file_path(self, file_path: Union[Path, str]) -> Optional[str]:
|
||||
async def get_permalink_for_file_path(
|
||||
self, session: AsyncSession, file_path: Union[Path, str]
|
||||
) -> Optional[str]:
|
||||
"""Get the permalink for a file_path without loading the full entity.
|
||||
|
||||
Use when you only need the permalink, not the full entity with relations.
|
||||
@@ -161,10 +168,10 @@ class EntityRepository(Repository[Entity]):
|
||||
"""
|
||||
query = select(Entity.permalink).where(Entity.file_path == Path(file_path).as_posix())
|
||||
query = self._add_project_filter(query)
|
||||
result = await self.execute_query(query, use_query_options=False)
|
||||
result = await self.execute_query(session, query, use_query_options=False)
|
||||
return result.scalar_one_or_none()
|
||||
|
||||
async def get_all_permalinks(self) -> List[str]:
|
||||
async def get_all_permalinks(self, session: AsyncSession) -> List[str]:
|
||||
"""Get all permalinks for this project.
|
||||
|
||||
Optimized for bulk operations - returns only permalink strings
|
||||
@@ -175,11 +182,11 @@ class EntityRepository(Repository[Entity]):
|
||||
"""
|
||||
query = select(Entity.permalink)
|
||||
query = self._add_project_filter(query)
|
||||
result = await self.execute_query(query, use_query_options=False)
|
||||
result = await self.execute_query(session, query, use_query_options=False)
|
||||
return list(result.scalars().all())
|
||||
|
||||
async def find_by_ids_for_hydration(
|
||||
self, ids: List[int], *, include_cross_project: bool = False
|
||||
self, session: AsyncSession, ids: List[int], *, include_cross_project: bool = False
|
||||
) -> Sequence[Entity]:
|
||||
"""Fetch minimal entity fields needed for context hydration.
|
||||
|
||||
@@ -203,10 +210,10 @@ class EntityRepository(Repository[Entity]):
|
||||
if not include_cross_project:
|
||||
query = self._add_project_filter(query)
|
||||
|
||||
result = await self.execute_query(query, use_query_options=False)
|
||||
result = await self.execute_query(session, query, use_query_options=False)
|
||||
return list(result.scalars().all())
|
||||
|
||||
async def get_permalink_to_file_path_map(self) -> dict[str, str]:
|
||||
async def get_permalink_to_file_path_map(self, session: AsyncSession) -> dict[str, str]:
|
||||
"""Get a mapping of permalink -> file_path for all entities.
|
||||
|
||||
Optimized for bulk permalink resolution - loads minimal data in one query.
|
||||
@@ -216,10 +223,10 @@ class EntityRepository(Repository[Entity]):
|
||||
"""
|
||||
query = select(Entity.permalink, Entity.file_path)
|
||||
query = self._add_project_filter(query)
|
||||
result = await self.execute_query(query, use_query_options=False)
|
||||
result = await self.execute_query(session, query, use_query_options=False)
|
||||
return {row.permalink: row.file_path for row in result.all()}
|
||||
|
||||
async def get_file_path_to_permalink_map(self) -> dict[str, str]:
|
||||
async def get_file_path_to_permalink_map(self, session: AsyncSession) -> dict[str, str]:
|
||||
"""Get a mapping of file_path -> permalink for all entities.
|
||||
|
||||
Optimized for bulk permalink resolution - loads minimal data in one query.
|
||||
@@ -229,7 +236,7 @@ class EntityRepository(Repository[Entity]):
|
||||
"""
|
||||
query = select(Entity.file_path, Entity.permalink)
|
||||
query = self._add_project_filter(query)
|
||||
result = await self.execute_query(query, use_query_options=False)
|
||||
result = await self.execute_query(session, query, use_query_options=False)
|
||||
return {row.file_path: row.permalink for row in result.all()}
|
||||
|
||||
async def get_by_file_paths(
|
||||
@@ -262,7 +269,7 @@ class EntityRepository(Repository[Entity]):
|
||||
result = await session.execute(query) # pragma: no cover
|
||||
return list(result.all()) # pragma: no cover
|
||||
|
||||
async def find_by_checksum(self, checksum: str) -> Sequence[Entity]:
|
||||
async def find_by_checksum(self, session: AsyncSession, checksum: str) -> Sequence[Entity]:
|
||||
"""Find entities with the given checksum.
|
||||
|
||||
Used for move detection - finds entities that may have been moved to a new path.
|
||||
@@ -276,10 +283,12 @@ class EntityRepository(Repository[Entity]):
|
||||
"""
|
||||
query = self.select().where(Entity.checksum == checksum)
|
||||
# Don't load relationships for move detection - we only need file_path and checksum
|
||||
result = await self.execute_query(query, use_query_options=False)
|
||||
result = await self.execute_query(session, query, use_query_options=False)
|
||||
return list(result.scalars().all())
|
||||
|
||||
async def find_by_checksums(self, checksums: Sequence[str]) -> Sequence[Entity]:
|
||||
async def find_by_checksums(
|
||||
self, session: AsyncSession, checksums: Sequence[str]
|
||||
) -> Sequence[Entity]:
|
||||
"""Find entities with any of the given checksums (batch query for move detection).
|
||||
|
||||
This is a batch-optimized version of find_by_checksum() that queries multiple checksums
|
||||
@@ -304,16 +313,18 @@ class EntityRepository(Repository[Entity]):
|
||||
# Query: SELECT * FROM entities WHERE checksum IN (checksum1, checksum2, ...)
|
||||
query = self.select().where(Entity.checksum.in_(checksums)) # pragma: no cover
|
||||
# Don't load relationships for move detection - we only need file_path and checksum
|
||||
result = await self.execute_query(query, use_query_options=False) # pragma: no cover
|
||||
result = await self.execute_query(
|
||||
session, query, use_query_options=False
|
||||
) # pragma: no cover
|
||||
return list(result.scalars().all()) # pragma: no cover
|
||||
|
||||
async def delete_by_file_path(self, file_path: Union[Path, str]) -> bool:
|
||||
async def delete_by_file_path(self, session: AsyncSession, file_path: Union[Path, str]) -> bool:
|
||||
"""Delete entity with the provided file_path.
|
||||
|
||||
Args:
|
||||
file_path: Path to the entity file (will be converted to string internally)
|
||||
"""
|
||||
return await self.delete_by_fields(file_path=Path(file_path).as_posix())
|
||||
return await self.delete_by_fields(session, file_path=Path(file_path).as_posix())
|
||||
|
||||
def get_load_options(self) -> List[LoaderOption]:
|
||||
"""Get SQLAlchemy loader options for eager loading relationships."""
|
||||
@@ -327,7 +338,9 @@ class EntityRepository(Repository[Entity]):
|
||||
selectinload(Entity.incoming_relations).selectinload(Relation.to_entity),
|
||||
]
|
||||
|
||||
async def find_by_permalinks(self, permalinks: List[str]) -> Sequence[Entity]:
|
||||
async def find_by_permalinks(
|
||||
self, session: AsyncSession, permalinks: List[str]
|
||||
) -> Sequence[Entity]:
|
||||
"""Find multiple entities by their permalink.
|
||||
|
||||
Args:
|
||||
@@ -342,10 +355,10 @@ class EntityRepository(Repository[Entity]):
|
||||
self.select().options(*self.get_load_options()).where(Entity.permalink.in_(permalinks))
|
||||
)
|
||||
|
||||
result = await self.execute_query(query)
|
||||
result = await self.execute_query(session, query)
|
||||
return list(result.scalars().all())
|
||||
|
||||
async def upsert_entity(self, entity: Entity) -> Entity:
|
||||
async def upsert_entity(self, session: AsyncSession, entity: Entity) -> Entity:
|
||||
"""Insert or update entity using simple try/catch with database-level conflict resolution.
|
||||
|
||||
Handles file_path race conditions by checking for existing entity on IntegrityError.
|
||||
@@ -357,99 +370,84 @@ class EntityRepository(Repository[Entity]):
|
||||
Returns:
|
||||
The inserted or updated entity
|
||||
"""
|
||||
async with db.scoped_session(self.session_maker) as session:
|
||||
# Set project_id if applicable and not already set
|
||||
self._set_project_id_if_needed(entity)
|
||||
# Set project_id if applicable and not already set
|
||||
self._set_project_id_if_needed(entity)
|
||||
|
||||
# Try simple insert first
|
||||
try:
|
||||
# Try simple insert first.
|
||||
try:
|
||||
async with session.begin_nested():
|
||||
session.add(entity)
|
||||
await session.flush()
|
||||
except IntegrityError as e:
|
||||
# Check if this is a FOREIGN KEY constraint failure
|
||||
# SQLite: "FOREIGN KEY constraint failed"
|
||||
# Postgres: "violates foreign key constraint"
|
||||
error_str = str(e)
|
||||
if (
|
||||
"FOREIGN KEY constraint failed" in error_str
|
||||
or "violates foreign key constraint" in error_str
|
||||
):
|
||||
# Import locally to avoid circular dependency (repository -> services -> repository)
|
||||
from basic_memory.services.exceptions import SyncFatalError
|
||||
|
||||
# Return with relationships loaded
|
||||
query = (
|
||||
self.select()
|
||||
.where(Entity.file_path == entity.file_path)
|
||||
.options(*self.get_load_options())
|
||||
# Project doesn't exist in database - this is a fatal sync error
|
||||
raise SyncFatalError(
|
||||
f"Cannot sync file '{entity.file_path}': "
|
||||
f"project_id={entity.project_id} does not exist in database. "
|
||||
f"The project may have been deleted. This sync will be terminated."
|
||||
) from e
|
||||
|
||||
# Re-query after the nested rollback to get a fresh, attached entity.
|
||||
existing_result = await session.execute(
|
||||
select(Entity)
|
||||
.where(Entity.file_path == entity.file_path, Entity.project_id == entity.project_id)
|
||||
.options(*self.get_load_options())
|
||||
)
|
||||
existing_entity = existing_result.scalar_one_or_none()
|
||||
|
||||
if existing_entity:
|
||||
# File path conflict - update the existing entity
|
||||
logger.debug(
|
||||
f"Resolving file_path conflict for {entity.file_path}, "
|
||||
f"entity_id={existing_entity.id}, observations={len(entity.observations)}"
|
||||
)
|
||||
result = await session.execute(query)
|
||||
found = result.scalar_one_or_none()
|
||||
if not found: # pragma: no cover
|
||||
raise RuntimeError(
|
||||
f"Failed to retrieve entity after insert: {entity.file_path}"
|
||||
)
|
||||
return found
|
||||
# Use merge to avoid session state conflicts. Preserve stable
|
||||
# external_id so external references survive re-indexing.
|
||||
entity.id = existing_entity.id
|
||||
entity.external_id = existing_entity.external_id
|
||||
|
||||
except IntegrityError as e:
|
||||
# Check if this is a FOREIGN KEY constraint failure
|
||||
# SQLite: "FOREIGN KEY constraint failed"
|
||||
# Postgres: "violates foreign key constraint"
|
||||
error_str = str(e)
|
||||
if (
|
||||
"FOREIGN KEY constraint failed" in error_str
|
||||
or "violates foreign key constraint" in error_str
|
||||
):
|
||||
# Import locally to avoid circular dependency (repository -> services -> repository)
|
||||
from basic_memory.services.exceptions import SyncFatalError
|
||||
# Ensure observations reference the correct entity_id.
|
||||
for obs in entity.observations:
|
||||
obs.entity_id = existing_entity.id
|
||||
obs.id = None
|
||||
|
||||
# Project doesn't exist in database - this is a fatal sync error
|
||||
raise SyncFatalError(
|
||||
f"Cannot sync file '{entity.file_path}': "
|
||||
f"project_id={entity.project_id} does not exist in database. "
|
||||
f"The project may have been deleted. This sync will be terminated."
|
||||
) from e
|
||||
merged_entity = await session.merge(entity)
|
||||
await session.flush()
|
||||
|
||||
await session.rollback()
|
||||
|
||||
# Re-query after rollback to get a fresh, attached entity
|
||||
existing_result = await session.execute(
|
||||
# Re-query to get proper relationships loaded.
|
||||
final_result = await session.execute(
|
||||
select(Entity)
|
||||
.where(
|
||||
Entity.file_path == entity.file_path, Entity.project_id == entity.project_id
|
||||
)
|
||||
.where(Entity.id == merged_entity.id)
|
||||
.options(*self.get_load_options())
|
||||
)
|
||||
existing_entity = existing_result.scalar_one_or_none()
|
||||
return final_result.scalar_one()
|
||||
|
||||
if existing_entity:
|
||||
# File path conflict - update the existing entity
|
||||
logger.debug(
|
||||
f"Resolving file_path conflict for {entity.file_path}, "
|
||||
f"entity_id={existing_entity.id}, observations={len(entity.observations)}"
|
||||
)
|
||||
# Use merge to avoid session state conflicts
|
||||
# Set the ID to update existing entity
|
||||
entity.id = existing_entity.id
|
||||
# Preserve the stable external_id so that external references
|
||||
# (e.g. public share links) survive re-indexing
|
||||
entity.external_id = existing_entity.external_id
|
||||
# No file_path conflict - must be permalink conflict.
|
||||
return await self._handle_permalink_conflict(entity, session)
|
||||
|
||||
# Ensure observations reference the correct entity_id
|
||||
for obs in entity.observations:
|
||||
obs.entity_id = existing_entity.id
|
||||
# Clear any existing ID to force INSERT as new observation
|
||||
obs.id = None
|
||||
# Return with relationships loaded.
|
||||
query = (
|
||||
self.select()
|
||||
.where(Entity.file_path == entity.file_path)
|
||||
.options(*self.get_load_options())
|
||||
)
|
||||
result = await session.execute(query)
|
||||
found = result.scalar_one_or_none()
|
||||
if not found: # pragma: no cover
|
||||
raise RuntimeError(f"Failed to retrieve entity after insert: {entity.file_path}")
|
||||
return found
|
||||
|
||||
# Merge the entity which will update the existing one
|
||||
merged_entity = await session.merge(entity)
|
||||
|
||||
await session.commit()
|
||||
|
||||
# Re-query to get proper relationships loaded
|
||||
final_result = await session.execute(
|
||||
select(Entity)
|
||||
.where(Entity.id == merged_entity.id)
|
||||
.options(*self.get_load_options())
|
||||
)
|
||||
return final_result.scalar_one()
|
||||
|
||||
else:
|
||||
# No file_path conflict - must be permalink conflict
|
||||
# Generate unique permalink and retry
|
||||
entity = await self._handle_permalink_conflict(entity, session)
|
||||
return entity
|
||||
|
||||
async def get_all_file_paths(self) -> List[str]:
|
||||
async def get_all_file_paths(self, session: AsyncSession) -> List[str]:
|
||||
"""Get all file paths for this project - optimized for deletion detection.
|
||||
|
||||
Returns only file_path strings without loading entities or relationships.
|
||||
@@ -461,10 +459,10 @@ class EntityRepository(Repository[Entity]):
|
||||
query = select(Entity.file_path)
|
||||
query = self._add_project_filter(query)
|
||||
|
||||
result = await self.execute_query(query, use_query_options=False)
|
||||
result = await self.execute_query(session, query, use_query_options=False)
|
||||
return list(result.scalars().all())
|
||||
|
||||
async def find_without_relations(self) -> Sequence[Entity]:
|
||||
async def find_without_relations(self, session: AsyncSession) -> Sequence[Entity]:
|
||||
"""Find entities that have no incoming or outgoing relations."""
|
||||
# Trigger: entity appears as a source in any relation.
|
||||
# Why: even unresolved outgoing links mean the entity references another node.
|
||||
@@ -477,10 +475,10 @@ class EntityRepository(Repository[Entity]):
|
||||
has_incoming = exists().where(Relation.to_id == Entity.id)
|
||||
|
||||
query = self.select().where(~has_outgoing).where(~has_incoming).order_by(Entity.file_path)
|
||||
result = await self.execute_query(query, use_query_options=False)
|
||||
result = await self.execute_query(session, query, use_query_options=False)
|
||||
return list(result.scalars().all())
|
||||
|
||||
async def get_distinct_directories(self) -> List[str]:
|
||||
async def get_distinct_directories(self, session: AsyncSession) -> List[str]:
|
||||
"""Extract unique directory paths from file_path column.
|
||||
|
||||
Optimized method for getting directory structure without loading full entities
|
||||
@@ -494,7 +492,7 @@ class EntityRepository(Repository[Entity]):
|
||||
query = self._add_project_filter(query)
|
||||
|
||||
# Execute with use_query_options=False to skip eager loading
|
||||
result = await self.execute_query(query, use_query_options=False)
|
||||
result = await self.execute_query(session, query, use_query_options=False)
|
||||
file_paths = [row for row in result.scalars().all()]
|
||||
|
||||
# Parse file paths to extract unique directories
|
||||
@@ -508,7 +506,9 @@ class EntityRepository(Repository[Entity]):
|
||||
|
||||
return sorted(directories)
|
||||
|
||||
async def find_by_directory_prefix(self, directory_prefix: str) -> Sequence[Entity]:
|
||||
async def find_by_directory_prefix(
|
||||
self, session: AsyncSession, directory_prefix: str
|
||||
) -> Sequence[Entity]:
|
||||
"""Find entities whose file_path starts with the given directory prefix.
|
||||
|
||||
Optimized method for listing directory contents without loading all entities.
|
||||
@@ -524,7 +524,7 @@ class EntityRepository(Repository[Entity]):
|
||||
# Build SQL LIKE pattern
|
||||
if directory_prefix == "" or directory_prefix == "/":
|
||||
# Root directory - return all entities
|
||||
return await self.find_all()
|
||||
return await self.find_all(session)
|
||||
|
||||
# Remove leading/trailing slashes for consistency
|
||||
directory_prefix = directory_prefix.strip("/")
|
||||
@@ -537,7 +537,7 @@ class EntityRepository(Repository[Entity]):
|
||||
query = self.select().where(Entity.file_path.like(pattern))
|
||||
|
||||
# Skip eager loading - we only need basic entity fields for directory trees
|
||||
result = await self.execute_query(query, use_query_options=False)
|
||||
result = await self.execute_query(session, query, use_query_options=False)
|
||||
return list(result.scalars().all())
|
||||
|
||||
async def _handle_permalink_conflict(self, entity: Entity, session: AsyncSession) -> Entity:
|
||||
|
||||
@@ -4,9 +4,8 @@ from pathlib import Path
|
||||
from typing import Any, Mapping, Optional
|
||||
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
from basic_memory import db
|
||||
from basic_memory.models import Entity, NoteContent
|
||||
from basic_memory.repository.repository import Repository
|
||||
|
||||
@@ -30,9 +29,9 @@ NOTE_CONTENT_MUTABLE_FIELDS = frozenset(
|
||||
class NoteContentRepository(Repository[NoteContent]):
|
||||
"""Repository for project-scoped note materialization state."""
|
||||
|
||||
def __init__(self, session_maker: async_sessionmaker[AsyncSession], project_id: int):
|
||||
"""Initialize with session maker and project-scoped filtering."""
|
||||
super().__init__(session_maker, NoteContent, project_id=project_id)
|
||||
def __init__(self, project_id: int):
|
||||
"""Initialize with project-scoped filtering."""
|
||||
super().__init__(NoteContent, project_id=project_id)
|
||||
|
||||
def _coerce_note_content(
|
||||
self, data: Mapping[str, Any] | NoteContent
|
||||
@@ -75,16 +74,22 @@ class NoteContentRepository(Repository[NoteContent]):
|
||||
note_content.external_id = entity.external_id
|
||||
note_content.file_path = Path(entity.file_path).as_posix()
|
||||
|
||||
async def get_by_entity_id(self, entity_id: int) -> Optional[NoteContent]:
|
||||
async def get_by_entity_id(
|
||||
self, session: AsyncSession, entity_id: int
|
||||
) -> Optional[NoteContent]:
|
||||
"""Get note content by the owning entity identifier."""
|
||||
return await self.find_by_id(entity_id)
|
||||
return await self.find_by_id(session, entity_id)
|
||||
|
||||
async def get_by_external_id(self, external_id: str) -> Optional[NoteContent]:
|
||||
async def get_by_external_id(
|
||||
self, session: AsyncSession, external_id: str
|
||||
) -> Optional[NoteContent]:
|
||||
"""Get note content by the mirrored entity external identifier."""
|
||||
query = self.select().where(NoteContent.external_id == external_id)
|
||||
return await self.find_one(query)
|
||||
return await self.find_one(session, query)
|
||||
|
||||
async def get_by_file_path(self, file_path: Path | str) -> Optional[NoteContent]:
|
||||
async def get_by_file_path(
|
||||
self, session: AsyncSession, file_path: Path | str
|
||||
) -> Optional[NoteContent]:
|
||||
"""Get note content by file path, preferring rows whose entity still owns that path."""
|
||||
normalized_path = Path(file_path).as_posix()
|
||||
|
||||
@@ -104,88 +109,88 @@ class NoteContentRepository(Repository[NoteContent]):
|
||||
.options(*self.get_load_options())
|
||||
)
|
||||
|
||||
async with db.scoped_session(self.session_maker) as session:
|
||||
result = await session.execute(query)
|
||||
return result.scalars().first()
|
||||
result = await session.execute(query)
|
||||
return result.scalars().first()
|
||||
|
||||
async def create(self, data: Mapping[str, Any] | NoteContent) -> NoteContent:
|
||||
async def create(
|
||||
self, session: AsyncSession, data: Mapping[str, Any] | NoteContent
|
||||
) -> NoteContent:
|
||||
"""Create a note_content row aligned to its owning entity."""
|
||||
note_content, _ = self._coerce_note_content(data)
|
||||
|
||||
async with db.scoped_session(self.session_maker) as session:
|
||||
await self._align_identity_fields(session, note_content)
|
||||
session.add(note_content)
|
||||
await session.flush()
|
||||
await self._align_identity_fields(session, note_content)
|
||||
session.add(note_content)
|
||||
await session.flush()
|
||||
|
||||
created = await self.select_by_id(session, note_content.entity_id)
|
||||
if created is None: # pragma: no cover
|
||||
raise ValueError(
|
||||
f"Can't find NoteContent for entity {note_content.entity_id} after add"
|
||||
)
|
||||
return created
|
||||
created = await self.select_by_id(session, note_content.entity_id)
|
||||
if created is None: # pragma: no cover
|
||||
raise ValueError(
|
||||
f"Can't find NoteContent for entity {note_content.entity_id} after add"
|
||||
)
|
||||
return created
|
||||
|
||||
async def upsert(self, data: Mapping[str, Any] | NoteContent) -> NoteContent:
|
||||
async def upsert(
|
||||
self, session: AsyncSession, data: Mapping[str, Any] | NoteContent
|
||||
) -> NoteContent:
|
||||
"""Insert or update note_content while keeping mirrored identity fields in sync."""
|
||||
note_content, provided_fields = self._coerce_note_content(data)
|
||||
|
||||
async with db.scoped_session(self.session_maker) as session:
|
||||
await self._align_identity_fields(session, note_content)
|
||||
existing = await self.select_by_id(session, note_content.entity_id)
|
||||
|
||||
if existing is None:
|
||||
session.add(note_content)
|
||||
await session.flush()
|
||||
created = await self.select_by_id(session, note_content.entity_id)
|
||||
if created is None: # pragma: no cover
|
||||
raise ValueError(
|
||||
f"Can't find NoteContent for entity {note_content.entity_id} after upsert"
|
||||
)
|
||||
return created
|
||||
|
||||
fields_to_update = (provided_fields - {"entity_id"}) | {
|
||||
"project_id",
|
||||
"external_id",
|
||||
"file_path",
|
||||
}
|
||||
for column_name in fields_to_update:
|
||||
setattr(existing, column_name, getattr(note_content, column_name))
|
||||
await self._align_identity_fields(session, note_content)
|
||||
existing = await self.select_by_id(session, note_content.entity_id)
|
||||
|
||||
if existing is None:
|
||||
session.add(note_content)
|
||||
await session.flush()
|
||||
updated = await self.select_by_id(session, existing.entity_id)
|
||||
if updated is None: # pragma: no cover
|
||||
created = await self.select_by_id(session, note_content.entity_id)
|
||||
if created is None: # pragma: no cover
|
||||
raise ValueError(
|
||||
f"Can't find NoteContent for entity {existing.entity_id} after upsert"
|
||||
f"Can't find NoteContent for entity {note_content.entity_id} after upsert"
|
||||
)
|
||||
return updated
|
||||
return created
|
||||
|
||||
async def update_state_fields(self, entity_id: int, **updates: Any) -> Optional[NoteContent]:
|
||||
fields_to_update = (provided_fields - {"entity_id"}) | {
|
||||
"project_id",
|
||||
"external_id",
|
||||
"file_path",
|
||||
}
|
||||
for column_name in fields_to_update:
|
||||
setattr(existing, column_name, getattr(note_content, column_name))
|
||||
|
||||
await session.flush()
|
||||
updated = await self.select_by_id(session, existing.entity_id)
|
||||
if updated is None: # pragma: no cover
|
||||
raise ValueError(f"Can't find NoteContent for entity {existing.entity_id} after upsert")
|
||||
return updated
|
||||
|
||||
async def update_state_fields(
|
||||
self, session: AsyncSession, entity_id: int, **updates: Any
|
||||
) -> Optional[NoteContent]:
|
||||
"""Update sync fields and re-align project_id, external_id, and file_path from entity."""
|
||||
invalid_fields = set(updates) - NOTE_CONTENT_MUTABLE_FIELDS
|
||||
if invalid_fields:
|
||||
invalid_list = ", ".join(sorted(invalid_fields))
|
||||
raise ValueError(f"Unsupported note_content update fields: {invalid_list}")
|
||||
|
||||
async with db.scoped_session(self.session_maker) as session:
|
||||
note_content = await self.select_by_id(session, entity_id)
|
||||
if note_content is None:
|
||||
return None
|
||||
note_content = await self.select_by_id(session, entity_id)
|
||||
if note_content is None:
|
||||
return None
|
||||
|
||||
await self._align_identity_fields(session, note_content)
|
||||
for field_name, value in updates.items():
|
||||
setattr(note_content, field_name, value)
|
||||
await self._align_identity_fields(session, note_content)
|
||||
for field_name, value in updates.items():
|
||||
setattr(note_content, field_name, value)
|
||||
|
||||
await session.flush()
|
||||
updated = await self.select_by_id(session, entity_id)
|
||||
if updated is None: # pragma: no cover
|
||||
raise ValueError(f"Can't find NoteContent for entity {entity_id} after update")
|
||||
return updated
|
||||
await session.flush()
|
||||
updated = await self.select_by_id(session, entity_id)
|
||||
if updated is None: # pragma: no cover
|
||||
raise ValueError(f"Can't find NoteContent for entity {entity_id} after update")
|
||||
return updated
|
||||
|
||||
async def delete_by_entity_id(self, entity_id: int) -> bool:
|
||||
async def delete_by_entity_id(self, session: AsyncSession, entity_id: int) -> bool:
|
||||
"""Delete note_content by entity identifier."""
|
||||
async with db.scoped_session(self.session_maker) as session:
|
||||
note_content = await self.select_by_id(session, entity_id)
|
||||
if note_content is None:
|
||||
return False
|
||||
note_content = await self.select_by_id(session, entity_id)
|
||||
if note_content is None:
|
||||
return False
|
||||
|
||||
await session.delete(note_content)
|
||||
return True
|
||||
await session.delete(note_content)
|
||||
await session.flush()
|
||||
return True
|
||||
|
||||
@@ -3,7 +3,7 @@
|
||||
from typing import Dict, List, Sequence
|
||||
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.ext.asyncio import async_sessionmaker
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
from sqlalchemy.orm import selectinload
|
||||
from sqlalchemy.orm.interfaces import LoaderOption
|
||||
|
||||
@@ -14,44 +14,46 @@ from basic_memory.repository.repository import Repository
|
||||
class ObservationRepository(Repository[Observation]):
|
||||
"""Repository for Observation model with memory-specific operations."""
|
||||
|
||||
def __init__(self, session_maker: async_sessionmaker, project_id: int):
|
||||
"""Initialize with session maker and project_id filter.
|
||||
def __init__(self, project_id: int):
|
||||
"""Initialize with project_id filter.
|
||||
|
||||
Args:
|
||||
session_maker: SQLAlchemy session maker
|
||||
project_id: Project ID to filter all operations by
|
||||
"""
|
||||
super().__init__(session_maker, Observation, project_id=project_id)
|
||||
super().__init__(Observation, project_id=project_id)
|
||||
|
||||
def get_load_options(self) -> List[LoaderOption]:
|
||||
"""Eager-load parent entity to prevent N+1 if obs.entity is accessed."""
|
||||
return [selectinload(Observation.entity)]
|
||||
|
||||
async def find_by_entity(self, entity_id: int) -> Sequence[Observation]:
|
||||
async def find_by_entity(self, session: AsyncSession, entity_id: int) -> Sequence[Observation]:
|
||||
"""Find all observations for a specific entity."""
|
||||
query = select(Observation).filter(Observation.entity_id == entity_id)
|
||||
result = await self.execute_query(query)
|
||||
query = self.select().filter(Observation.entity_id == entity_id)
|
||||
result = await self.execute_query(session, query)
|
||||
return result.scalars().all()
|
||||
|
||||
async def find_by_context(self, context: str) -> Sequence[Observation]:
|
||||
async def find_by_context(self, session: AsyncSession, context: str) -> Sequence[Observation]:
|
||||
"""Find observations with a specific context."""
|
||||
query = select(Observation).filter(Observation.context == context)
|
||||
result = await self.execute_query(query)
|
||||
query = self.select().filter(Observation.context == context)
|
||||
result = await self.execute_query(session, query)
|
||||
return result.scalars().all()
|
||||
|
||||
async def find_by_category(self, category: str) -> Sequence[Observation]:
|
||||
async def find_by_category(self, session: AsyncSession, category: str) -> Sequence[Observation]:
|
||||
"""Find observations with a specific context."""
|
||||
query = select(Observation).filter(Observation.category == category)
|
||||
result = await self.execute_query(query)
|
||||
query = self.select().filter(Observation.category == category)
|
||||
result = await self.execute_query(session, query)
|
||||
return result.scalars().all()
|
||||
|
||||
async def observation_categories(self) -> Sequence[str]:
|
||||
async def observation_categories(self, session: AsyncSession) -> Sequence[str]:
|
||||
"""Return a list of all observation categories."""
|
||||
query = select(Observation.category).distinct()
|
||||
result = await self.execute_query(query, use_query_options=False)
|
||||
query = self._add_project_filter(query)
|
||||
result = await self.execute_query(session, query, use_query_options=False)
|
||||
return result.scalars().all()
|
||||
|
||||
async def find_by_entities(self, entity_ids: List[int]) -> Dict[int, List[Observation]]:
|
||||
async def find_by_entities(
|
||||
self, session: AsyncSession, entity_ids: List[int]
|
||||
) -> Dict[int, List[Observation]]:
|
||||
"""Find all observations for multiple entities in a single query.
|
||||
|
||||
Args:
|
||||
@@ -64,8 +66,8 @@ class ObservationRepository(Repository[Observation]):
|
||||
return {}
|
||||
|
||||
# Query observations for all entities in the list
|
||||
query = select(Observation).filter(Observation.entity_id.in_(entity_ids))
|
||||
result = await self.execute_query(query)
|
||||
query = self.select().filter(Observation.entity_id.in_(entity_ids))
|
||||
result = await self.execute_query(session, query)
|
||||
observations = result.scalars().all()
|
||||
|
||||
# Group observations by entity_id
|
||||
|
||||
@@ -18,6 +18,7 @@ from basic_memory.repository.search_index_row import SearchIndexRow
|
||||
from basic_memory.repository.search_repository_base import (
|
||||
SearchRepositoryBase,
|
||||
VectorChunkState,
|
||||
relaxed_query_words,
|
||||
)
|
||||
from basic_memory.repository.metadata_filters import parse_metadata_filters
|
||||
from basic_memory.repository.semantic_errors import SemanticDependenciesMissingError
|
||||
@@ -176,6 +177,14 @@ class PostgresSearchRepository(SearchRepositoryBase):
|
||||
# For non-Boolean queries, prepare single term
|
||||
return self._prepare_single_term(term, is_prefix)
|
||||
|
||||
@staticmethod
|
||||
def _relaxed_tsquery_text(search_text: Optional[str]) -> Optional[str]:
|
||||
"""OR-relaxed tsquery expression for a failed strict query, or None."""
|
||||
words = relaxed_query_words(search_text)
|
||||
if not words:
|
||||
return None
|
||||
return " | ".join(f"{word}:*" for word in words)
|
||||
|
||||
def _prepare_boolean_query(self, query: str) -> str:
|
||||
"""Convert Boolean query to tsquery format.
|
||||
|
||||
@@ -236,7 +245,12 @@ class PostgresSearchRepository(SearchRepositoryBase):
|
||||
|
||||
# Handle multi-word queries
|
||||
if " " in cleaned_term:
|
||||
words = [w for w in cleaned_term.split() if w.strip()]
|
||||
# Strip sentence punctuation from word edges so question-form
|
||||
# queries produce clean lexemes (parity with SQLite FTS5 prep).
|
||||
# The tsquery tokenizer ignores this punctuation anyway; leaving it
|
||||
# in only risks tsquery syntax errors. Interior characters are kept.
|
||||
words = [w.strip("?!.,;") for w in cleaned_term.split()]
|
||||
words = [w for w in words if w]
|
||||
if not words:
|
||||
# All characters were special chars, search won't match anything
|
||||
# Return a safe search term that won't cause syntax errors
|
||||
@@ -249,8 +263,11 @@ class PostgresSearchRepository(SearchRepositoryBase):
|
||||
# Join with AND operator
|
||||
return " & ".join(prepared_words)
|
||||
|
||||
# Single word
|
||||
cleaned_term = cleaned_term.strip()
|
||||
# Single word: strip edge punctuation; guard the now-empty case so a
|
||||
# bare ":*"/"" never reaches tsquery.
|
||||
cleaned_term = cleaned_term.strip().strip("?!.,;")
|
||||
if not cleaned_term:
|
||||
return "NOSPECIALCHARS:*"
|
||||
if is_prefix:
|
||||
return f"{cleaned_term}:*"
|
||||
else:
|
||||
@@ -908,6 +925,8 @@ class PostgresSearchRepository(SearchRepositoryBase):
|
||||
min_similarity: Optional[float] = None,
|
||||
limit: int = 10,
|
||||
offset: int = 0,
|
||||
allow_relaxed: bool = False,
|
||||
session: AsyncSession | None = None,
|
||||
) -> List[SearchIndexRow]:
|
||||
"""Search across all indexed content using PostgreSQL tsvector."""
|
||||
# --- Dispatch vector / hybrid modes (shared logic) ---
|
||||
@@ -978,10 +997,32 @@ class PostgresSearchRepository(SearchRepositoryBase):
|
||||
"""
|
||||
|
||||
logger.trace(f"Search {sql} params: {params}")
|
||||
try:
|
||||
async with db.scoped_session(self.session_maker) as session:
|
||||
result = await session.execute(text(sql), params)
|
||||
|
||||
async def run_search(active_session: AsyncSession):
|
||||
result = await active_session.execute(text(sql), params)
|
||||
rows = result.fetchall()
|
||||
# Trigger: multi-word natural-language query matched nothing
|
||||
# under the default all-terms-AND tsquery semantics.
|
||||
# Why: questions rarely have every word in one document;
|
||||
# without relaxation the FTS half of hybrid search contributes
|
||||
# zero candidates (parity with the SQLite path).
|
||||
# Outcome: one retry with OR-joined prefix lexemes; ts_rank
|
||||
# still ranks multi-term matches first.
|
||||
relaxed = (
|
||||
self._relaxed_tsquery_text(search_text) if allow_relaxed and not rows else None
|
||||
)
|
||||
if relaxed and params.get("text"):
|
||||
params["text"] = relaxed
|
||||
result = await active_session.execute(text(sql), params)
|
||||
rows = result.fetchall()
|
||||
return rows
|
||||
|
||||
try:
|
||||
if session is not None:
|
||||
rows = await run_search(session)
|
||||
else:
|
||||
async with db.scoped_session(self.session_maker) as owned_session:
|
||||
rows = await run_search(owned_session)
|
||||
except Exception as e:
|
||||
if self._is_tsquery_syntax_error(e):
|
||||
logger.warning(f"tsquery syntax error for search term: {search_text}, error: {e}")
|
||||
|
||||
@@ -5,6 +5,6 @@ from basic_memory.models.project import Project
|
||||
class ProjectInfoRepository(Repository):
|
||||
"""Repository for statistics queries."""
|
||||
|
||||
def __init__(self, session_maker):
|
||||
def __init__(self):
|
||||
# Initialize with Project model as a reference
|
||||
super().__init__(session_maker, Project)
|
||||
super().__init__(Project)
|
||||
|
||||
@@ -7,9 +7,8 @@ from typing import Optional, Sequence, Union
|
||||
from loguru import logger
|
||||
from sqlalchemy import Executable, inspect as sa_inspect, select, text
|
||||
from sqlalchemy.exc import NoResultFound, OperationalError
|
||||
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
from basic_memory import db
|
||||
from basic_memory.models.project import Project
|
||||
from basic_memory.repository.repository import Repository
|
||||
|
||||
@@ -79,20 +78,22 @@ class ProjectRepository(Repository[Project]):
|
||||
Each entity, observation, and relation belongs to a specific project.
|
||||
"""
|
||||
|
||||
def __init__(self, session_maker: async_sessionmaker[AsyncSession]):
|
||||
"""Initialize with session maker."""
|
||||
super().__init__(session_maker, Project)
|
||||
def __init__(self):
|
||||
"""Initialize the repository."""
|
||||
super().__init__(Project)
|
||||
|
||||
async def get_by_name(self, name: str) -> Optional[Project]:
|
||||
async def get_by_name(self, session: AsyncSession, name: str) -> Optional[Project]:
|
||||
"""Get project by name (exact match).
|
||||
|
||||
Args:
|
||||
name: Unique name of the project
|
||||
"""
|
||||
query = self.select().where(Project.name == name)
|
||||
return await self.find_one(query)
|
||||
return await self.find_one(session, query)
|
||||
|
||||
async def get_by_name_case_insensitive(self, name: str) -> Optional[Project]:
|
||||
async def get_by_name_case_insensitive(
|
||||
self, session: AsyncSession, name: str
|
||||
) -> Optional[Project]:
|
||||
"""Get project by name (case-insensitive match).
|
||||
|
||||
Args:
|
||||
@@ -102,27 +103,27 @@ class ProjectRepository(Repository[Project]):
|
||||
Project if found, None otherwise
|
||||
"""
|
||||
query = self.select().where(Project.name.ilike(name))
|
||||
return await self.find_one(query)
|
||||
return await self.find_one(session, query)
|
||||
|
||||
async def get_by_permalink(self, permalink: str) -> Optional[Project]:
|
||||
async def get_by_permalink(self, session: AsyncSession, permalink: str) -> Optional[Project]:
|
||||
"""Get project by permalink.
|
||||
|
||||
Args:
|
||||
permalink: URL-friendly identifier for the project
|
||||
"""
|
||||
query = self.select().where(Project.permalink == permalink)
|
||||
return await self.find_one(query)
|
||||
return await self.find_one(session, query)
|
||||
|
||||
async def get_by_path(self, path: Union[Path, str]) -> Optional[Project]:
|
||||
async def get_by_path(self, session: AsyncSession, path: Union[Path, str]) -> Optional[Project]:
|
||||
"""Get project by filesystem path.
|
||||
|
||||
Args:
|
||||
path: Path to the project directory (will be converted to string internally)
|
||||
"""
|
||||
query = self.select().where(Project.path == Path(path).as_posix())
|
||||
return await self.find_one(query)
|
||||
return await self.find_one(session, query)
|
||||
|
||||
async def get_by_id(self, project_id: int) -> Optional[Project]:
|
||||
async def get_by_id(self, session: AsyncSession, project_id: int) -> Optional[Project]:
|
||||
"""Get project by numeric ID.
|
||||
|
||||
Args:
|
||||
@@ -131,10 +132,11 @@ class ProjectRepository(Repository[Project]):
|
||||
Returns:
|
||||
Project if found, None otherwise
|
||||
"""
|
||||
async with db.scoped_session(self.session_maker) as session:
|
||||
return await self.select_by_id(session, project_id)
|
||||
return await self.select_by_id(session, project_id)
|
||||
|
||||
async def get_by_external_id(self, external_id: str) -> Optional[Project]:
|
||||
async def get_by_external_id(
|
||||
self, session: AsyncSession, external_id: str
|
||||
) -> Optional[Project]:
|
||||
"""Get project by external UUID.
|
||||
|
||||
Args:
|
||||
@@ -144,20 +146,20 @@ class ProjectRepository(Repository[Project]):
|
||||
Project if found, None otherwise
|
||||
"""
|
||||
query = self.select().where(Project.external_id == external_id)
|
||||
return await self.find_one(query)
|
||||
return await self.find_one(session, query)
|
||||
|
||||
async def get_default_project(self) -> Optional[Project]:
|
||||
async def get_default_project(self, session: AsyncSession) -> Optional[Project]:
|
||||
"""Get the default project (the one marked as is_default=True)."""
|
||||
query = self.select().where(Project.is_default.is_(True))
|
||||
return await self.find_one(query)
|
||||
return await self.find_one(session, query)
|
||||
|
||||
async def get_active_projects(self) -> Sequence[Project]:
|
||||
async def get_active_projects(self, session: AsyncSession) -> Sequence[Project]:
|
||||
"""Get all active projects."""
|
||||
query = self.select().where(Project.is_active == True) # noqa: E712
|
||||
result = await self.execute_query(query)
|
||||
result = await self.execute_query(session, query)
|
||||
return list(result.scalars().all())
|
||||
|
||||
async def set_as_default(self, project_id: int) -> Optional[Project]:
|
||||
async def set_as_default(self, session: AsyncSession, project_id: int) -> Optional[Project]:
|
||||
"""Set a project as the default and unset previous default.
|
||||
|
||||
Args:
|
||||
@@ -166,22 +168,21 @@ class ProjectRepository(Repository[Project]):
|
||||
Returns:
|
||||
The updated project if found, None otherwise
|
||||
"""
|
||||
async with db.scoped_session(self.session_maker) as session:
|
||||
# First, clear the default flag for all projects using direct SQL
|
||||
await session.execute(
|
||||
text("UPDATE project SET is_default = NULL WHERE is_default IS NOT NULL")
|
||||
)
|
||||
# First, clear the default flag for all projects using direct SQL
|
||||
await session.execute(
|
||||
text("UPDATE project SET is_default = NULL WHERE is_default IS NOT NULL")
|
||||
)
|
||||
await session.flush()
|
||||
|
||||
# Set the new default project
|
||||
target_project = await self.select_by_id(session, project_id)
|
||||
if target_project:
|
||||
target_project.is_default = True
|
||||
await session.flush()
|
||||
return target_project
|
||||
return None # pragma: no cover
|
||||
|
||||
# Set the new default project
|
||||
target_project = await self.select_by_id(session, project_id)
|
||||
if target_project:
|
||||
target_project.is_default = True
|
||||
await session.flush()
|
||||
return target_project
|
||||
return None # pragma: no cover
|
||||
|
||||
async def delete(self, entity_id: int) -> bool:
|
||||
async def delete(self, session: AsyncSession, entity_id: int) -> bool:
|
||||
"""Delete a project and its derived search rows in one transaction.
|
||||
|
||||
The cascade picture differs by backend:
|
||||
@@ -204,62 +205,60 @@ class ProjectRepository(Repository[Project]):
|
||||
Inspect the connection once and skip whichever is missing.
|
||||
"""
|
||||
logger.debug(f"Deleting Project and search rows for project_id: {entity_id}")
|
||||
async with db.scoped_session(self.session_maker) as session:
|
||||
try:
|
||||
result = await session.execute(
|
||||
select(self.Model).filter(self.primary_key == entity_id)
|
||||
)
|
||||
project = result.scalars().one()
|
||||
except NoResultFound:
|
||||
logger.debug(f"No Project found to delete: {entity_id}")
|
||||
return False
|
||||
try:
|
||||
result = await session.execute(select(self.Model).filter(self.primary_key == entity_id))
|
||||
project = result.scalars().one()
|
||||
except NoResultFound:
|
||||
logger.debug(f"No Project found to delete: {entity_id}")
|
||||
return False
|
||||
|
||||
dialect_name = session.bind.dialect.name if session.bind else "sqlite"
|
||||
is_sqlite = dialect_name == "sqlite"
|
||||
dialect_name = session.bind.dialect.name if session.bind else "sqlite"
|
||||
is_sqlite = dialect_name == "sqlite"
|
||||
|
||||
existing_tables = await session.run_sync(
|
||||
lambda sync_session: set(sa_inspect(sync_session.connection()).get_table_names())
|
||||
existing_tables = await session.run_sync(
|
||||
lambda sync_session: set(sa_inspect(sync_session.connection()).get_table_names())
|
||||
)
|
||||
|
||||
# search_index: SQLite has no FK on the FTS5 virtual table; Postgres
|
||||
# cascades from the project FK, so the explicit DELETE is redundant.
|
||||
if is_sqlite and "search_index" in existing_tables:
|
||||
await session.execute(
|
||||
text("DELETE FROM search_index WHERE project_id = :project_id"),
|
||||
{"project_id": entity_id},
|
||||
)
|
||||
|
||||
# search_index: SQLite has no FK on the FTS5 virtual table; Postgres
|
||||
# cascades from the project FK, so the explicit DELETE is redundant.
|
||||
if is_sqlite and "search_index" in existing_tables:
|
||||
await session.execute(
|
||||
text("DELETE FROM search_index WHERE project_id = :project_id"),
|
||||
{"project_id": entity_id},
|
||||
)
|
||||
# search_vector_chunks: no FK to project on either backend, so both
|
||||
# backends need this. SQLite must purge vec0 embeddings first
|
||||
# (rowid pseudocolumn — Postgres uses chunk_id and would 500 here);
|
||||
# Postgres' chunk_id FK CASCADE handles its embeddings cleanup when
|
||||
# we delete the chunk rows below.
|
||||
if "search_vector_chunks" in existing_tables:
|
||||
if is_sqlite and "search_vector_embeddings" in existing_tables:
|
||||
# Extension loading is per-connection. We must load vec0 on
|
||||
# *this* session before the DELETE; otherwise a different
|
||||
# pooled connection might have written embeddings that we'd
|
||||
# silently leave behind.
|
||||
if await _load_sqlite_vec_on_session(session):
|
||||
await session.execute(
|
||||
text(
|
||||
"DELETE FROM search_vector_embeddings WHERE rowid IN ("
|
||||
"SELECT id FROM search_vector_chunks "
|
||||
"WHERE project_id = :project_id)"
|
||||
),
|
||||
{"project_id": entity_id},
|
||||
)
|
||||
await session.execute(
|
||||
text("DELETE FROM search_vector_chunks WHERE project_id = :project_id"),
|
||||
{"project_id": entity_id},
|
||||
)
|
||||
|
||||
# search_vector_chunks: no FK to project on either backend, so both
|
||||
# backends need this. SQLite must purge vec0 embeddings first
|
||||
# (rowid pseudocolumn — Postgres uses chunk_id and would 500 here);
|
||||
# Postgres' chunk_id FK CASCADE handles its embeddings cleanup when
|
||||
# we delete the chunk rows below.
|
||||
if "search_vector_chunks" in existing_tables:
|
||||
if is_sqlite and "search_vector_embeddings" in existing_tables:
|
||||
# Extension loading is per-connection. We must load vec0 on
|
||||
# *this* session before the DELETE; otherwise a different
|
||||
# pooled connection might have written embeddings that we'd
|
||||
# silently leave behind.
|
||||
if await _load_sqlite_vec_on_session(session):
|
||||
await session.execute(
|
||||
text(
|
||||
"DELETE FROM search_vector_embeddings WHERE rowid IN ("
|
||||
"SELECT id FROM search_vector_chunks "
|
||||
"WHERE project_id = :project_id)"
|
||||
),
|
||||
{"project_id": entity_id},
|
||||
)
|
||||
await session.execute(
|
||||
text("DELETE FROM search_vector_chunks WHERE project_id = :project_id"),
|
||||
{"project_id": entity_id},
|
||||
)
|
||||
|
||||
await session.delete(project)
|
||||
logger.debug(f"Deleted Project and search rows for project_id: {entity_id}")
|
||||
return True
|
||||
await session.delete(project)
|
||||
await session.flush()
|
||||
logger.debug(f"Deleted Project and search rows for project_id: {entity_id}")
|
||||
return True
|
||||
|
||||
async def scalar_vec_query(
|
||||
self, query: Executable, params: Optional[dict] = None
|
||||
self, session: AsyncSession, query: Executable, params: Optional[dict] = None
|
||||
) -> Optional[int]:
|
||||
"""Run a scalar COUNT query that reads the sqlite-vec vec0 table.
|
||||
|
||||
@@ -271,16 +270,17 @@ class ProjectRepository(Repository[Project]):
|
||||
Returns None when sqlite-vec cannot be loaded on this Python build, so
|
||||
callers can fall back to the genuinely-missing-dependency path.
|
||||
"""
|
||||
async with db.scoped_session(self.session_maker) as session:
|
||||
# Trigger: query reads the vec0-backed search_vector_embeddings table.
|
||||
# Why: vec0 modules are only visible on a connection that loaded sqlite-vec.
|
||||
# Outcome: load the extension here, or signal absence so the caller degrades.
|
||||
if not await _load_sqlite_vec_on_session(session):
|
||||
return None
|
||||
result = await session.execute(query, params)
|
||||
return result.scalar()
|
||||
# Trigger: query reads the vec0-backed search_vector_embeddings table.
|
||||
# Why: vec0 modules are only visible on a connection that loaded sqlite-vec.
|
||||
# Outcome: load the extension here, or signal absence so the caller degrades.
|
||||
if not await _load_sqlite_vec_on_session(session):
|
||||
return None
|
||||
result = await session.execute(query, params)
|
||||
return result.scalar()
|
||||
|
||||
async def update_path(self, project_id: int, new_path: str) -> Optional[Project]:
|
||||
async def update_path(
|
||||
self, session: AsyncSession, project_id: int, new_path: str
|
||||
) -> Optional[Project]:
|
||||
"""Update project path.
|
||||
|
||||
Args:
|
||||
@@ -290,10 +290,9 @@ class ProjectRepository(Repository[Project]):
|
||||
Returns:
|
||||
The updated project if found, None otherwise
|
||||
"""
|
||||
async with db.scoped_session(self.session_maker) as session:
|
||||
project = await self.select_by_id(session, project_id)
|
||||
if project:
|
||||
project.path = new_path
|
||||
await session.flush()
|
||||
return project
|
||||
return None
|
||||
project = await self.select_by_id(session, project_id)
|
||||
if project:
|
||||
project.path = new_path
|
||||
await session.flush()
|
||||
return project
|
||||
return None
|
||||
|
||||
@@ -6,11 +6,10 @@ from sqlalchemy import and_, delete, select
|
||||
from sqlalchemy.engine import CursorResult
|
||||
from sqlalchemy.dialects.postgresql import insert as pg_insert
|
||||
from sqlalchemy.dialects.sqlite import insert as sqlite_insert
|
||||
from sqlalchemy.ext.asyncio import async_sessionmaker
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
from sqlalchemy.orm import selectinload, aliased
|
||||
from sqlalchemy.orm.interfaces import LoaderOption
|
||||
|
||||
from basic_memory import db
|
||||
from basic_memory.models import Relation, Entity
|
||||
from basic_memory.repository.repository import Repository
|
||||
|
||||
@@ -18,17 +17,20 @@ from basic_memory.repository.repository import Repository
|
||||
class RelationRepository(Repository[Relation]):
|
||||
"""Repository for Relation model with memory-specific operations."""
|
||||
|
||||
def __init__(self, session_maker: async_sessionmaker, project_id: int):
|
||||
"""Initialize with session maker and project_id filter.
|
||||
def __init__(self, project_id: int):
|
||||
"""Initialize with project_id filter.
|
||||
|
||||
Args:
|
||||
session_maker: SQLAlchemy session maker
|
||||
project_id: Project ID to filter all operations by
|
||||
"""
|
||||
super().__init__(session_maker, Relation, project_id=project_id)
|
||||
super().__init__(Relation, project_id=project_id)
|
||||
|
||||
async def find_relation(
|
||||
self, from_permalink: str, to_permalink: str, relation_type: str
|
||||
self,
|
||||
session: AsyncSession,
|
||||
from_permalink: str,
|
||||
to_permalink: str,
|
||||
relation_type: str,
|
||||
) -> Optional[Relation]:
|
||||
"""Find a relation by its from and to path IDs."""
|
||||
from_entity = aliased(Entity)
|
||||
@@ -46,36 +48,44 @@ class RelationRepository(Repository[Relation]):
|
||||
)
|
||||
)
|
||||
)
|
||||
return await self.find_one(query)
|
||||
query = self._add_project_filter(query)
|
||||
return await self.find_one(session, query)
|
||||
|
||||
async def find_by_entities(self, from_id: int, to_id: int) -> Sequence[Relation]:
|
||||
async def find_by_entities(
|
||||
self, session: AsyncSession, from_id: int, to_id: int
|
||||
) -> Sequence[Relation]:
|
||||
"""Find all relations between two entities."""
|
||||
query = select(Relation).where((Relation.from_id == from_id) & (Relation.to_id == to_id))
|
||||
result = await self.execute_query(query)
|
||||
query = self.select().where((Relation.from_id == from_id) & (Relation.to_id == to_id))
|
||||
result = await self.execute_query(session, query)
|
||||
return result.scalars().all()
|
||||
|
||||
async def find_by_type(self, relation_type: str) -> Sequence[Relation]:
|
||||
async def find_by_type(self, session: AsyncSession, relation_type: str) -> Sequence[Relation]:
|
||||
"""Find all relations of a specific type."""
|
||||
query = select(Relation).filter(Relation.relation_type == relation_type)
|
||||
result = await self.execute_query(query)
|
||||
query = self.select().filter(Relation.relation_type == relation_type)
|
||||
result = await self.execute_query(session, query)
|
||||
return result.scalars().all()
|
||||
|
||||
async def delete_outgoing_relations_from_entity(self, entity_id: int) -> None:
|
||||
async def delete_outgoing_relations_from_entity(
|
||||
self, session: AsyncSession, entity_id: int
|
||||
) -> None:
|
||||
"""Delete outgoing relations for an entity.
|
||||
|
||||
Only deletes relations where this entity is the source (from_id),
|
||||
as these are the ones owned by this entity's markdown file.
|
||||
"""
|
||||
async with db.scoped_session(self.session_maker) as session:
|
||||
await session.execute(delete(Relation).where(Relation.from_id == entity_id))
|
||||
query = delete(Relation).where(Relation.from_id == entity_id)
|
||||
query = query.where(Relation.project_id == self.project_id)
|
||||
await session.execute(query)
|
||||
|
||||
async def find_unresolved_relations(self) -> Sequence[Relation]:
|
||||
async def find_unresolved_relations(self, session: AsyncSession) -> Sequence[Relation]:
|
||||
"""Find all unresolved relations, where to_id is null."""
|
||||
query = select(Relation).filter(Relation.to_id.is_(None))
|
||||
result = await self.execute_query(query)
|
||||
query = self.select().filter(Relation.to_id.is_(None))
|
||||
result = await self.execute_query(session, query)
|
||||
return result.scalars().all()
|
||||
|
||||
async def find_unresolved_relations_for_entity(self, entity_id: int) -> Sequence[Relation]:
|
||||
async def find_unresolved_relations_for_entity(
|
||||
self, session: AsyncSession, entity_id: int
|
||||
) -> Sequence[Relation]:
|
||||
"""Find unresolved relations for a specific entity.
|
||||
|
||||
Args:
|
||||
@@ -84,11 +94,13 @@ class RelationRepository(Repository[Relation]):
|
||||
Returns:
|
||||
List of unresolved relations where this entity is the source.
|
||||
"""
|
||||
query = select(Relation).filter(Relation.from_id == entity_id, Relation.to_id.is_(None))
|
||||
result = await self.execute_query(query)
|
||||
query = self.select().filter(Relation.from_id == entity_id, Relation.to_id.is_(None))
|
||||
result = await self.execute_query(session, query)
|
||||
return result.scalars().all()
|
||||
|
||||
async def add_all_ignore_duplicates(self, relations: List[Relation]) -> int:
|
||||
async def add_all_ignore_duplicates(
|
||||
self, session: AsyncSession, relations: List[Relation]
|
||||
) -> int:
|
||||
"""Bulk insert relations, ignoring duplicates.
|
||||
|
||||
Uses ON CONFLICT DO NOTHING to skip relations that would violate the
|
||||
@@ -120,27 +132,23 @@ class RelationRepository(Repository[Relation]):
|
||||
for r in relations
|
||||
]
|
||||
|
||||
async with db.scoped_session(self.session_maker) as session:
|
||||
# Check dialect to use appropriate insert
|
||||
dialect_name = session.bind.dialect.name if session.bind else "sqlite"
|
||||
# Check dialect to use appropriate insert
|
||||
dialect_name = session.bind.dialect.name if session.bind else "sqlite"
|
||||
|
||||
if dialect_name == "postgresql": # pragma: no cover
|
||||
# PostgreSQL: use RETURNING to count inserted rows
|
||||
# (rowcount is 0 for ON CONFLICT DO NOTHING)
|
||||
stmt = ( # pragma: no cover
|
||||
pg_insert(Relation)
|
||||
.values(values)
|
||||
.on_conflict_do_nothing()
|
||||
.returning(Relation.id)
|
||||
)
|
||||
result = await session.execute(stmt) # pragma: no cover
|
||||
return len(result.fetchall()) # pragma: no cover
|
||||
else:
|
||||
# SQLite: rowcount works correctly
|
||||
stmt = sqlite_insert(Relation).values(values)
|
||||
stmt = stmt.on_conflict_do_nothing()
|
||||
result = cast(CursorResult[Any], await session.execute(stmt))
|
||||
return result.rowcount if result.rowcount > 0 else 0
|
||||
if dialect_name == "postgresql": # pragma: no cover
|
||||
# PostgreSQL: use RETURNING to count inserted rows
|
||||
# (rowcount is 0 for ON CONFLICT DO NOTHING)
|
||||
stmt = ( # pragma: no cover
|
||||
pg_insert(Relation).values(values).on_conflict_do_nothing().returning(Relation.id)
|
||||
)
|
||||
result = await session.execute(stmt) # pragma: no cover
|
||||
return len(result.fetchall()) # pragma: no cover
|
||||
else:
|
||||
# SQLite: rowcount works correctly
|
||||
stmt = sqlite_insert(Relation).values(values)
|
||||
stmt = stmt.on_conflict_do_nothing()
|
||||
result = cast(CursorResult[Any], await session.execute(stmt))
|
||||
return result.rowcount if result.rowcount > 0 else 0
|
||||
|
||||
def get_load_options(self) -> List[LoaderOption]:
|
||||
return [selectinload(Relation.from_entity), selectinload(Relation.to_entity)]
|
||||
|
||||
@@ -1,7 +1,6 @@
|
||||
"""Base repository implementation."""
|
||||
|
||||
from typing import Type, Optional, Any, Sequence, TypeVar, List, Dict, cast
|
||||
|
||||
from typing import Optional, Any, Sequence, TypeVar, List, Dict, cast
|
||||
|
||||
from loguru import logger
|
||||
from sqlalchemy import (
|
||||
@@ -16,27 +15,23 @@ from sqlalchemy import (
|
||||
update as sqlalchemy_update,
|
||||
)
|
||||
from sqlalchemy.engine import CursorResult
|
||||
from sqlalchemy.exc import NoResultFound
|
||||
from sqlalchemy.ext.asyncio import async_sessionmaker, AsyncSession
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
from sqlalchemy.orm.interfaces import LoaderOption
|
||||
from sqlalchemy.sql.elements import ColumnElement
|
||||
|
||||
from basic_memory import db
|
||||
from basic_memory.models import Base
|
||||
|
||||
T = TypeVar("T", bound=Base)
|
||||
|
||||
|
||||
class Repository[T: Base]:
|
||||
"""Base repository implementation with generic CRUD operations."""
|
||||
"""Base repository implementation with explicit caller-owned sessions."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
session_maker: async_sessionmaker[AsyncSession],
|
||||
Model: Type[T],
|
||||
Model: type[T],
|
||||
project_id: Optional[int] = None,
|
||||
):
|
||||
self.session_maker = session_maker
|
||||
self.project_id = project_id
|
||||
if Model:
|
||||
self.Model = Model
|
||||
@@ -98,62 +93,61 @@ class Repository[T: Base]:
|
||||
result = await session.execute(query)
|
||||
return result.scalars().all()
|
||||
|
||||
async def add(self, model: T) -> T:
|
||||
async def add(self, session: AsyncSession, model: T) -> T:
|
||||
"""
|
||||
Add a model to the repository. This will also add related objects
|
||||
:param session: the caller-owned session to use
|
||||
:param model: the model to add
|
||||
:return: the added model instance
|
||||
"""
|
||||
async with db.scoped_session(self.session_maker) as session:
|
||||
# Set project_id if applicable and not already set
|
||||
self._set_project_id_if_needed(model)
|
||||
# Set project_id if applicable and not already set
|
||||
self._set_project_id_if_needed(model)
|
||||
|
||||
session.add(model)
|
||||
await session.flush()
|
||||
session.add(model)
|
||||
await session.flush()
|
||||
|
||||
# Query within same session
|
||||
found = await self.select_by_id(session, model.id) # pyright: ignore [reportAttributeAccessIssue]
|
||||
if found is None: # pragma: no cover
|
||||
logger.error(
|
||||
"Failed to retrieve model after add",
|
||||
model_type=self.Model.__name__,
|
||||
model_id=model.id, # pyright: ignore
|
||||
)
|
||||
raise ValueError(
|
||||
f"Can't find {self.Model.__name__} with ID {model.id} after session.add" # pyright: ignore
|
||||
)
|
||||
return found
|
||||
# Query within same session
|
||||
found = await self.select_by_id(session, model.id) # pyright: ignore [reportAttributeAccessIssue]
|
||||
if found is None: # pragma: no cover
|
||||
logger.error(
|
||||
"Failed to retrieve model after add",
|
||||
model_type=self.Model.__name__,
|
||||
model_id=model.id, # pyright: ignore
|
||||
)
|
||||
raise ValueError(
|
||||
f"Can't find {self.Model.__name__} with ID {model.id} after session.add" # pyright: ignore
|
||||
)
|
||||
return found
|
||||
|
||||
async def add_all(self, models: List[T]) -> Sequence[T]:
|
||||
async def add_all(self, session: AsyncSession, models: List[T]) -> Sequence[T]:
|
||||
"""
|
||||
Add a list of models to the repository. This will also add related objects
|
||||
:param session: the caller-owned session to use
|
||||
:param models: the models to add
|
||||
:return: the added models instances
|
||||
"""
|
||||
async with db.scoped_session(self.session_maker) as session:
|
||||
# set the project id if not present in models
|
||||
for model in models:
|
||||
self._set_project_id_if_needed(model)
|
||||
# set the project id if not present in models
|
||||
for model in models:
|
||||
self._set_project_id_if_needed(model)
|
||||
|
||||
session.add_all(models)
|
||||
await session.flush()
|
||||
session.add_all(models)
|
||||
await session.flush()
|
||||
|
||||
# Query within same session
|
||||
return await self.select_by_ids(session, [m.id for m in models]) # pyright: ignore [reportAttributeAccessIssue]
|
||||
# Query within same session
|
||||
return await self.select_by_ids(session, [m.id for m in models]) # pyright: ignore [reportAttributeAccessIssue]
|
||||
|
||||
async def add_all_no_return(self, models: List[T]) -> int:
|
||||
async def add_all_no_return(self, session: AsyncSession, models: List[T]) -> int:
|
||||
"""Insert models without reloading them afterward."""
|
||||
if not models:
|
||||
return 0
|
||||
|
||||
async with db.scoped_session(self.session_maker) as session:
|
||||
for model in models:
|
||||
self._set_project_id_if_needed(model)
|
||||
for model in models:
|
||||
self._set_project_id_if_needed(model)
|
||||
|
||||
session.add_all(models)
|
||||
await session.flush()
|
||||
logger.debug(f"Added {len(models)} {self.Model.__name__} records")
|
||||
return len(models)
|
||||
session.add_all(models)
|
||||
await session.flush()
|
||||
logger.debug(f"Added {len(models)} {self.Model.__name__} records")
|
||||
return len(models)
|
||||
|
||||
def select(self, *entities: Any) -> Select:
|
||||
"""Create a new SELECT statement.
|
||||
@@ -170,55 +164,55 @@ class Repository[T: Base]:
|
||||
return self._add_project_filter(query)
|
||||
|
||||
async def find_all(
|
||||
self, skip: int = 0, limit: Optional[int] = None, use_load_options: bool = True
|
||||
self,
|
||||
session: AsyncSession,
|
||||
skip: int = 0,
|
||||
limit: Optional[int] = None,
|
||||
use_load_options: bool = True,
|
||||
) -> Sequence[T]:
|
||||
"""Fetch records from the database with pagination.
|
||||
|
||||
Args:
|
||||
session: The caller-owned session to use
|
||||
skip: Number of records to skip
|
||||
limit: Maximum number of records to return
|
||||
use_load_options: Whether to apply eager loading options (default: True)
|
||||
"""
|
||||
logger.debug(f"Finding all {self.Model.__name__} (skip={skip}, limit={limit})")
|
||||
|
||||
async with db.scoped_session(self.session_maker) as session:
|
||||
query = select(self.Model).offset(skip)
|
||||
query = select(self.Model).offset(skip)
|
||||
|
||||
# Only apply load options if requested
|
||||
if use_load_options:
|
||||
query = query.options(*self.get_load_options())
|
||||
# Only apply load options if requested
|
||||
if use_load_options:
|
||||
query = query.options(*self.get_load_options())
|
||||
|
||||
# Add project filter if applicable
|
||||
query = self._add_project_filter(query)
|
||||
# Add project filter if applicable
|
||||
query = self._add_project_filter(query)
|
||||
|
||||
if limit:
|
||||
query = query.limit(limit)
|
||||
if limit:
|
||||
query = query.limit(limit)
|
||||
|
||||
result = await session.execute(query)
|
||||
result = await session.execute(query)
|
||||
|
||||
items = result.scalars().all()
|
||||
logger.debug(f"Found {len(items)} {self.Model.__name__} records")
|
||||
return items
|
||||
items = result.scalars().all()
|
||||
logger.debug(f"Found {len(items)} {self.Model.__name__} records")
|
||||
return items
|
||||
|
||||
async def find_by_id(self, entity_id: int) -> Optional[T]:
|
||||
async def find_by_id(self, session: AsyncSession, entity_id: Any) -> Optional[T]:
|
||||
"""Fetch an entity by its unique identifier."""
|
||||
logger.debug(f"Finding {self.Model.__name__} by ID: {entity_id}")
|
||||
return await self.select_by_id(session, entity_id)
|
||||
|
||||
async with db.scoped_session(self.session_maker) as session:
|
||||
return await self.select_by_id(session, entity_id)
|
||||
|
||||
async def find_by_ids(self, ids: List[int]) -> Sequence[T]:
|
||||
async def find_by_ids(self, session: AsyncSession, ids: List[Any]) -> Sequence[T]:
|
||||
"""Fetch multiple entities by their identifiers in a single query."""
|
||||
logger.debug(f"Finding {self.Model.__name__} by IDs: {ids}")
|
||||
return await self.select_by_ids(session, ids)
|
||||
|
||||
async with db.scoped_session(self.session_maker) as session:
|
||||
return await self.select_by_ids(session, ids)
|
||||
|
||||
async def find_one(self, query: Select[tuple[T]]) -> Optional[T]:
|
||||
async def find_one(self, session: AsyncSession, query: Select[tuple[T]]) -> Optional[T]:
|
||||
"""Execute a query and retrieve a single record."""
|
||||
# add in load options
|
||||
query = query.options(*self.get_load_options())
|
||||
result = await self.execute_query(query)
|
||||
result = await self.execute_query(session, query)
|
||||
entity = result.scalars().one_or_none()
|
||||
|
||||
if entity:
|
||||
@@ -227,12 +221,42 @@ class Repository[T: Base]:
|
||||
logger.trace(f"No {self.Model.__name__} found")
|
||||
return entity
|
||||
|
||||
async def create(self, data: dict) -> T:
|
||||
async def create(self, session: AsyncSession, data: dict[str, Any]) -> T:
|
||||
"""Create a new record from a model instance."""
|
||||
logger.debug(f"Creating {self.Model.__name__} from entity_data: {data}")
|
||||
async with db.scoped_session(self.session_maker) as session:
|
||||
# Only include valid columns that are provided in entity_data
|
||||
model_data = self.get_model_data(data)
|
||||
# Only include valid columns that are provided in entity_data
|
||||
model_data = self.get_model_data(data)
|
||||
|
||||
# Add project_id if applicable and not already provided
|
||||
if self.has_project_id and self.project_id is not None and "project_id" not in model_data:
|
||||
model_data["project_id"] = self.project_id
|
||||
|
||||
model = self.Model(**model_data)
|
||||
session.add(model)
|
||||
await session.flush()
|
||||
|
||||
return_instance = await self.select_by_id(session, model.id) # pyright: ignore [reportAttributeAccessIssue]
|
||||
if return_instance is None: # pragma: no cover
|
||||
logger.error(
|
||||
"Failed to retrieve model after create",
|
||||
model_type=self.Model.__name__,
|
||||
model_id=model.id, # pyright: ignore
|
||||
)
|
||||
raise ValueError(
|
||||
f"Can't find {self.Model.__name__} with ID {model.id} after session.add" # pyright: ignore
|
||||
)
|
||||
return return_instance
|
||||
|
||||
async def create_all(
|
||||
self, session: AsyncSession, data_list: List[dict[str, Any]]
|
||||
) -> Sequence[T]:
|
||||
"""Create multiple records in a single transaction."""
|
||||
logger.debug(f"Bulk creating {len(data_list)} {self.Model.__name__} instances")
|
||||
|
||||
# Only include valid columns that are provided in entity_data
|
||||
model_list = []
|
||||
for d in data_list:
|
||||
model_data = self.get_model_data(d)
|
||||
|
||||
# Add project_id if applicable and not already provided
|
||||
if (
|
||||
@@ -240,169 +264,126 @@ class Repository[T: Base]:
|
||||
and self.project_id is not None
|
||||
and "project_id" not in model_data
|
||||
):
|
||||
model_data["project_id"] = self.project_id
|
||||
model_data["project_id"] = self.project_id # pragma: no cover
|
||||
|
||||
model = self.Model(**model_data)
|
||||
session.add(model)
|
||||
await session.flush()
|
||||
model_list.append(self.Model(**model_data))
|
||||
|
||||
return_instance = await self.select_by_id(session, model.id) # pyright: ignore [reportAttributeAccessIssue]
|
||||
if return_instance is None: # pragma: no cover
|
||||
logger.error(
|
||||
"Failed to retrieve model after create",
|
||||
model_type=self.Model.__name__,
|
||||
model_id=model.id, # pyright: ignore
|
||||
)
|
||||
raise ValueError(
|
||||
f"Can't find {self.Model.__name__} with ID {model.id} after session.add" # pyright: ignore
|
||||
)
|
||||
return return_instance
|
||||
session.add_all(model_list)
|
||||
await session.flush()
|
||||
|
||||
async def create_all(self, data_list: List[dict]) -> Sequence[T]:
|
||||
"""Create multiple records in a single transaction."""
|
||||
logger.debug(f"Bulk creating {len(data_list)} {self.Model.__name__} instances")
|
||||
return await self.select_by_ids(session, [model.id for model in model_list]) # pyright: ignore [reportAttributeAccessIssue]
|
||||
|
||||
async with db.scoped_session(self.session_maker) as session:
|
||||
# Only include valid columns that are provided in entity_data
|
||||
model_list = []
|
||||
for d in data_list:
|
||||
model_data = self.get_model_data(d)
|
||||
|
||||
# Add project_id if applicable and not already provided
|
||||
if (
|
||||
self.has_project_id
|
||||
and self.project_id is not None
|
||||
and "project_id" not in model_data
|
||||
):
|
||||
model_data["project_id"] = self.project_id # pragma: no cover
|
||||
|
||||
model_list.append(self.Model(**model_data))
|
||||
|
||||
session.add_all(model_list)
|
||||
await session.flush()
|
||||
|
||||
return await self.select_by_ids(session, [model.id for model in model_list]) # pyright: ignore [reportAttributeAccessIssue]
|
||||
|
||||
async def update(self, entity_id: int, entity_data: dict[str, Any] | T) -> Optional[T]:
|
||||
async def update(
|
||||
self,
|
||||
session: AsyncSession,
|
||||
entity_id: Any,
|
||||
entity_data: dict[str, Any] | T,
|
||||
) -> Optional[T]:
|
||||
"""Update an entity with the given data."""
|
||||
logger.debug(f"Updating {self.Model.__name__} {entity_id} with data: {entity_data}")
|
||||
async with db.scoped_session(self.session_maker) as session:
|
||||
try:
|
||||
result = await session.execute(
|
||||
select(self.Model).filter(self.primary_key == entity_id)
|
||||
)
|
||||
entity = result.scalars().one()
|
||||
entity = await self.select_by_id(session, entity_id)
|
||||
if entity is None:
|
||||
logger.debug(f"No {self.Model.__name__} found to update: {entity_id}")
|
||||
return None
|
||||
|
||||
if isinstance(entity_data, dict):
|
||||
update_data = cast(dict[str, Any], entity_data)
|
||||
for key in self.valid_columns:
|
||||
if key in update_data:
|
||||
setattr(entity, key, update_data[key])
|
||||
if isinstance(entity_data, dict):
|
||||
update_data = cast(dict[str, Any], entity_data)
|
||||
for key in self.valid_columns:
|
||||
if key in update_data:
|
||||
setattr(entity, key, update_data[key])
|
||||
|
||||
elif isinstance(entity_data, self.Model):
|
||||
for column in self.valid_columns:
|
||||
setattr(entity, column, getattr(entity_data, column))
|
||||
elif isinstance(entity_data, self.Model):
|
||||
for column in self.valid_columns:
|
||||
setattr(entity, column, getattr(entity_data, column))
|
||||
|
||||
await session.flush() # Make sure changes are flushed
|
||||
await session.refresh(entity) # Refresh
|
||||
await session.flush()
|
||||
await session.refresh(entity)
|
||||
|
||||
logger.debug(f"Updated {self.Model.__name__}: {entity_id}")
|
||||
return await self.select_by_id(session, entity.id) # pyright: ignore [reportAttributeAccessIssue]
|
||||
logger.debug(f"Updated {self.Model.__name__}: {entity_id}")
|
||||
return await self.select_by_id(session, entity.id) # pyright: ignore [reportAttributeAccessIssue]
|
||||
|
||||
except NoResultFound:
|
||||
logger.debug(f"No {self.Model.__name__} found to update: {entity_id}")
|
||||
return None
|
||||
|
||||
async def update_fields(self, entity_id: Any, entity_data: dict[str, Any]) -> bool:
|
||||
async def update_fields(
|
||||
self, session: AsyncSession, entity_id: Any, entity_data: dict[str, Any]
|
||||
) -> bool:
|
||||
"""Update columns without reloading the model graph afterward."""
|
||||
update_data = {k: v for k, v in entity_data.items() if k in self.valid_columns}
|
||||
if not update_data:
|
||||
return True
|
||||
|
||||
async with db.scoped_session(self.session_maker) as session:
|
||||
conditions = [self.primary_key == entity_id]
|
||||
if self.has_project_id and self.project_id is not None:
|
||||
conditions.append(getattr(self.Model, "project_id") == self.project_id)
|
||||
conditions = [self.primary_key == entity_id]
|
||||
if self.has_project_id and self.project_id is not None:
|
||||
conditions.append(getattr(self.Model, "project_id") == self.project_id)
|
||||
|
||||
result = cast(
|
||||
CursorResult[Any],
|
||||
await session.execute(
|
||||
sqlalchemy_update(self.Model).where(and_(*conditions)).values(**update_data)
|
||||
),
|
||||
)
|
||||
return result.rowcount > 0
|
||||
result = cast(
|
||||
CursorResult[Any],
|
||||
await session.execute(
|
||||
sqlalchemy_update(self.Model).where(and_(*conditions)).values(**update_data)
|
||||
),
|
||||
)
|
||||
return result.rowcount > 0
|
||||
|
||||
async def delete(self, entity_id: int) -> bool:
|
||||
async def delete(self, session: AsyncSession, entity_id: int) -> bool:
|
||||
"""Delete an entity from the database."""
|
||||
logger.debug(f"Deleting {self.Model.__name__}: {entity_id}")
|
||||
async with db.scoped_session(self.session_maker) as session:
|
||||
try:
|
||||
result = await session.execute(
|
||||
select(self.Model).filter(self.primary_key == entity_id)
|
||||
)
|
||||
entity = result.scalars().one()
|
||||
await session.delete(entity)
|
||||
entity = await self.select_by_id(session, entity_id)
|
||||
if entity is None:
|
||||
logger.debug(f"No {self.Model.__name__} found to delete: {entity_id}")
|
||||
return False
|
||||
|
||||
logger.debug(f"Deleted {self.Model.__name__}: {entity_id}")
|
||||
return True
|
||||
except NoResultFound:
|
||||
logger.debug(f"No {self.Model.__name__} found to delete: {entity_id}")
|
||||
return False
|
||||
await session.delete(entity)
|
||||
await session.flush()
|
||||
|
||||
async def delete_by_ids(self, ids: List[int]) -> int:
|
||||
logger.debug(f"Deleted {self.Model.__name__}: {entity_id}")
|
||||
return True
|
||||
|
||||
async def delete_by_ids(self, session: AsyncSession, ids: List[int]) -> int:
|
||||
"""Delete records matching given IDs."""
|
||||
logger.debug(f"Deleting {self.Model.__name__} by ids: {ids}")
|
||||
async with db.scoped_session(self.session_maker) as session:
|
||||
conditions = [self.primary_key.in_(ids)]
|
||||
conditions = [self.primary_key.in_(ids)]
|
||||
|
||||
# Add project_id filter if applicable
|
||||
if self.has_project_id and self.project_id is not None: # pragma: no cover
|
||||
conditions.append(getattr(self.Model, "project_id") == self.project_id)
|
||||
# Add project_id filter if applicable
|
||||
if self.has_project_id and self.project_id is not None: # pragma: no cover
|
||||
conditions.append(getattr(self.Model, "project_id") == self.project_id)
|
||||
|
||||
query = delete(self.Model).where(and_(*conditions))
|
||||
result = cast(CursorResult[Any], await session.execute(query))
|
||||
logger.debug(f"Deleted {result.rowcount} records")
|
||||
return result.rowcount
|
||||
query = delete(self.Model).where(and_(*conditions))
|
||||
result = cast(CursorResult[Any], await session.execute(query))
|
||||
logger.debug(f"Deleted {result.rowcount} records")
|
||||
return result.rowcount
|
||||
|
||||
async def delete_by_fields(self, **filters: Any) -> bool:
|
||||
async def delete_by_fields(self, session: AsyncSession, **filters: Any) -> bool:
|
||||
"""Delete records matching given field values."""
|
||||
logger.debug(f"Deleting {self.Model.__name__} by fields: {filters}")
|
||||
async with db.scoped_session(self.session_maker) as session:
|
||||
conditions = [getattr(self.Model, field) == value for field, value in filters.items()]
|
||||
conditions = [getattr(self.Model, field) == value for field, value in filters.items()]
|
||||
|
||||
# Add project_id filter if applicable
|
||||
if self.has_project_id and self.project_id is not None:
|
||||
conditions.append(getattr(self.Model, "project_id") == self.project_id)
|
||||
# Add project_id filter if applicable
|
||||
if self.has_project_id and self.project_id is not None:
|
||||
conditions.append(getattr(self.Model, "project_id") == self.project_id)
|
||||
|
||||
query = delete(self.Model).where(and_(*conditions))
|
||||
result = cast(CursorResult[Any], await session.execute(query))
|
||||
deleted = result.rowcount > 0
|
||||
logger.debug(f"Deleted {result.rowcount} records")
|
||||
return deleted
|
||||
query = delete(self.Model).where(and_(*conditions))
|
||||
result = cast(CursorResult[Any], await session.execute(query))
|
||||
deleted = result.rowcount > 0
|
||||
logger.debug(f"Deleted {result.rowcount} records")
|
||||
return deleted
|
||||
|
||||
async def count(self, query: Executable | None = None) -> int:
|
||||
async def count(self, session: AsyncSession, query: Executable | None = None) -> int:
|
||||
"""Count entities in the database table."""
|
||||
async with db.scoped_session(self.session_maker) as session:
|
||||
if query is None:
|
||||
query = select(func.count()).select_from(self.Model)
|
||||
# Add project filter if applicable
|
||||
if (
|
||||
isinstance(query, Select)
|
||||
and self.has_project_id
|
||||
and self.project_id is not None
|
||||
):
|
||||
query = query.where(
|
||||
getattr(self.Model, "project_id") == self.project_id
|
||||
) # pragma: no cover
|
||||
if query is None:
|
||||
query = select(func.count()).select_from(self.Model)
|
||||
# Add project filter if applicable
|
||||
if isinstance(query, Select) and self.has_project_id and self.project_id is not None:
|
||||
query = query.where(
|
||||
getattr(self.Model, "project_id") == self.project_id
|
||||
) # pragma: no cover
|
||||
|
||||
result = await session.execute(query)
|
||||
scalar = result.scalar()
|
||||
count = scalar if scalar is not None else 0
|
||||
logger.debug(f"Counted {count} {self.Model.__name__} records")
|
||||
return count
|
||||
result = await session.execute(query)
|
||||
scalar = result.scalar()
|
||||
count = scalar if scalar is not None else 0
|
||||
logger.debug(f"Counted {count} {self.Model.__name__} records")
|
||||
return count
|
||||
|
||||
async def execute_query(
|
||||
self,
|
||||
session: AsyncSession,
|
||||
query: Executable,
|
||||
params: Optional[Dict[str, Any]] = None,
|
||||
use_query_options: bool = True,
|
||||
@@ -411,9 +392,8 @@ class Repository[T: Base]:
|
||||
|
||||
query = query.options(*self.get_load_options()) if use_query_options else query
|
||||
logger.trace(f"Executing query: {query}, params: {params}")
|
||||
async with db.scoped_session(self.session_maker) as session:
|
||||
result = await session.execute(query, params)
|
||||
return result
|
||||
result = await session.execute(query, params)
|
||||
return result
|
||||
|
||||
def get_load_options(self) -> List[LoaderOption]:
|
||||
"""Get list of loader options for eager loading relationships.
|
||||
|
||||
@@ -36,6 +36,7 @@ class SearchIndexRow:
|
||||
category: Optional[str] = None # observations
|
||||
from_id: Optional[int] = None # relations
|
||||
to_id: Optional[int] = None # relations
|
||||
to_name: Optional[str] = None # relations: literal target text, set even when unresolved
|
||||
relation_type: Optional[str] = None # relations
|
||||
|
||||
# Matched chunk text from vector search (the actual content that matched the query)
|
||||
@@ -94,6 +95,7 @@ class SearchIndexRow:
|
||||
else self.metadata,
|
||||
"from_id": self.from_id,
|
||||
"to_id": self.to_id,
|
||||
"to_name": self.to_name,
|
||||
"relation_type": self.relation_type,
|
||||
"entity_id": self.entity_id,
|
||||
"category": self.category,
|
||||
|
||||
@@ -48,6 +48,8 @@ class SearchRepository(Protocol):
|
||||
min_similarity: Optional[float] = None,
|
||||
limit: int = 10,
|
||||
offset: int = 0,
|
||||
allow_relaxed: bool = False,
|
||||
session: AsyncSession | None = None,
|
||||
) -> List[SearchIndexRow]:
|
||||
"""Search across indexed content."""
|
||||
...
|
||||
|
||||
@@ -40,6 +40,55 @@ BULLET_PATTERN = re.compile(r"^[\-\*]\s+")
|
||||
OVERSIZED_ENTITY_VECTOR_SHARD_SIZE = 256
|
||||
_SQLITE_MAX_PREPARE_WINDOW = 8
|
||||
|
||||
# Interrogative/function words contribute lexical noise when a strict
|
||||
# full-text query is relaxed: "when OR did OR a" matches loud wrong documents
|
||||
# that displace genuine results from the ranking window.
|
||||
RELAXATION_STOPWORDS = frozenset(
|
||||
"a an and are as at be but by did do does for from had has have how i in is it of on "
|
||||
"or that the their they this to was we were what when where which who whom whose why "
|
||||
"will with you your".split()
|
||||
)
|
||||
|
||||
|
||||
def relaxed_query_words(search_text: Optional[str]) -> Optional[list[str]]:
|
||||
"""Content-bearing words for OR-relaxing a strict full-text query.
|
||||
|
||||
Returns None when relaxation must not apply. These eligibility rules match
|
||||
SearchService._is_relaxed_fts_fallback_eligible so the hybrid FTS branch
|
||||
relaxes exactly the same query shapes as the service-level FTS path:
|
||||
|
||||
- empty / quoted / explicit-boolean queries (user intent is not
|
||||
second-guessed);
|
||||
- fewer than three alphanumeric tokens (short queries like "New Feature"
|
||||
over-broaden under OR — and in hybrid the relaxed FTS-only rows normalize
|
||||
to 1.0 and can outrank the vector result the user wanted);
|
||||
- any pure-digit token ("root note 1", "SPEC 16") — identifier-like queries
|
||||
over-broaden and create false positives under OR.
|
||||
"""
|
||||
if not search_text:
|
||||
return None
|
||||
stripped = search_text.strip()
|
||||
if '"' in stripped or any(op in f" {stripped} " for op in (" AND ", " OR ", " NOT ")):
|
||||
return None
|
||||
# Eligibility checks run on raw alphanumeric tokens (parity with the
|
||||
# service), before stopword filtering.
|
||||
tokens = re.findall(r"[A-Za-z0-9]+", stripped.lower())
|
||||
if len(tokens) < 3 or any(token.isdigit() for token in tokens):
|
||||
return None
|
||||
words = [word.strip("?!.,;:") for word in stripped.split()]
|
||||
words = [
|
||||
word
|
||||
for word in words
|
||||
if word and word.isalnum() and word.lower() not in RELAXATION_STOPWORDS
|
||||
]
|
||||
return words or None
|
||||
|
||||
|
||||
# Entity, observation, and relation rows in search_index carry ids from independent
|
||||
# auto-increment sequences, so a bare id is ambiguous across row types. Every map in
|
||||
# the vector/hybrid retrieval path must key rows by (type, id) to avoid collisions.
|
||||
type SearchIndexKey = tuple[str, int]
|
||||
|
||||
|
||||
@dataclass
|
||||
class VectorSyncBatchResult:
|
||||
@@ -224,6 +273,7 @@ class SearchRepositoryBase(ABC):
|
||||
min_similarity: Optional[float] = None,
|
||||
limit: int = 10,
|
||||
offset: int = 0,
|
||||
allow_relaxed: bool = False,
|
||||
) -> List[SearchIndexRow]:
|
||||
"""Search across all indexed content.
|
||||
|
||||
@@ -1857,7 +1907,7 @@ class SearchRepositoryBase(ABC):
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
@staticmethod
|
||||
def _parse_chunk_key(chunk_key: str) -> tuple[str, int]:
|
||||
def _parse_chunk_key(chunk_key: str) -> SearchIndexKey:
|
||||
"""Parse a chunk_key like 'observation:5:0' into (type, search_index_id)."""
|
||||
parts = chunk_key.split(":")
|
||||
return parts[0], int(parts[1])
|
||||
@@ -1932,26 +1982,27 @@ class SearchRepositoryBase(ABC):
|
||||
|
||||
hydrate_start = time.perf_counter()
|
||||
# Build per-search_index_row similarity scores from chunk-level results.
|
||||
# Each chunk_key encodes the search_index row type and id.
|
||||
# Each chunk_key encodes the search_index row type and id; keep both as the
|
||||
# key because different row types can share the same numeric id (#982).
|
||||
# Track the best similarity per row (for ranking) and all chunks (for context).
|
||||
similarity_by_si_id: dict[int, float] = {}
|
||||
chunks_by_si_id: dict[int, list[tuple[float, str]]] = {}
|
||||
similarity_by_si_key: dict[SearchIndexKey, float] = {}
|
||||
chunks_by_si_key: dict[SearchIndexKey, list[tuple[float, str]]] = {}
|
||||
for row in vector_rows:
|
||||
chunk_key = row.get("chunk_key", "")
|
||||
distance = float(row["best_distance"])
|
||||
similarity = self._distance_to_similarity(distance)
|
||||
chunk_text = row.get("chunk_text", "")
|
||||
try:
|
||||
_, si_id = self._parse_chunk_key(chunk_key)
|
||||
si_key = self._parse_chunk_key(chunk_key)
|
||||
except (ValueError, IndexError):
|
||||
# Fallback: group by entity_id for chunks without parseable keys
|
||||
continue
|
||||
current = similarity_by_si_id.get(si_id)
|
||||
current = similarity_by_si_key.get(si_key)
|
||||
if current is None or similarity > current:
|
||||
similarity_by_si_id[si_id] = similarity
|
||||
chunks_by_si_id.setdefault(si_id, []).append((similarity, chunk_text))
|
||||
similarity_by_si_key[si_key] = similarity
|
||||
chunks_by_si_key.setdefault(si_key, []).append((similarity, chunk_text))
|
||||
|
||||
if not similarity_by_si_id:
|
||||
if not similarity_by_si_key:
|
||||
hydrate_ms = (time.perf_counter() - hydrate_start) * 1000
|
||||
_log_vector_summary()
|
||||
return []
|
||||
@@ -1962,16 +2013,17 @@ class SearchRepositoryBase(ABC):
|
||||
min_similarity if min_similarity is not None else self._semantic_min_similarity
|
||||
)
|
||||
if effective_min_similarity > 0.0:
|
||||
similarity_by_si_id = {
|
||||
k: v for k, v in similarity_by_si_id.items() if v >= effective_min_similarity
|
||||
similarity_by_si_key = {
|
||||
k: v for k, v in similarity_by_si_key.items() if v >= effective_min_similarity
|
||||
}
|
||||
if not similarity_by_si_id:
|
||||
if not similarity_by_si_key:
|
||||
hydrate_ms = (time.perf_counter() - hydrate_start) * 1000
|
||||
_log_vector_summary()
|
||||
return []
|
||||
|
||||
# Fetch the actual search_index rows
|
||||
si_ids = list(similarity_by_si_id.keys())
|
||||
# Fetch the actual search_index rows. Colliding (type, id) keys share one
|
||||
# bare id, so deduplicate while preserving first-seen order.
|
||||
si_ids = list(dict.fromkeys(si_id for _, si_id in similarity_by_si_key))
|
||||
search_index_rows = await self._fetch_search_index_rows_by_ids(si_ids)
|
||||
|
||||
# Apply optional filters if requested
|
||||
@@ -2003,16 +2055,14 @@ class SearchRepositoryBase(ABC):
|
||||
limit=VECTOR_FILTER_SCAN_LIMIT,
|
||||
offset=0,
|
||||
)
|
||||
# Use (id, type) tuples to avoid collisions between different
|
||||
# Use (type, id) tuples to avoid collisions between different
|
||||
# search_index row types that share the same auto-increment id.
|
||||
allowed_keys = {(row.id, row.type) for row in filtered_rows if row.id is not None}
|
||||
search_index_rows = {
|
||||
k: v for k, v in search_index_rows.items() if (v.id, v.type) in allowed_keys
|
||||
}
|
||||
allowed_keys = {(row.type, row.id) for row in filtered_rows if row.id is not None}
|
||||
search_index_rows = {k: v for k, v in search_index_rows.items() if k in allowed_keys}
|
||||
|
||||
ranked_rows: list[SearchIndexRow] = []
|
||||
for si_id, similarity in similarity_by_si_id.items():
|
||||
row = search_index_rows.get(si_id)
|
||||
for si_key, similarity in similarity_by_si_key.items():
|
||||
row = search_index_rows.get(si_key)
|
||||
if row is None:
|
||||
continue
|
||||
|
||||
@@ -2022,7 +2072,7 @@ class SearchRepositoryBase(ABC):
|
||||
if content_snippet and len(content_snippet) <= SMALL_NOTE_CONTENT_LIMIT:
|
||||
matched_chunk_text = content_snippet
|
||||
else:
|
||||
si_chunks = chunks_by_si_id.get(si_id, [])
|
||||
si_chunks = chunks_by_si_key.get(si_key, [])
|
||||
si_chunks.sort(key=lambda c: c[0], reverse=True)
|
||||
top_texts = [text for _, text in si_chunks[:TOP_CHUNKS_PER_RESULT]]
|
||||
matched_chunk_text = "\n---\n".join(top_texts) if top_texts else None
|
||||
@@ -2088,8 +2138,12 @@ class SearchRepositoryBase(ABC):
|
||||
|
||||
async def _fetch_search_index_rows_by_ids(
|
||||
self, row_ids: list[int]
|
||||
) -> dict[int, SearchIndexRow]:
|
||||
"""Fetch search_index rows by their primary key (id), any type."""
|
||||
) -> dict[SearchIndexKey, SearchIndexRow]:
|
||||
"""Fetch search_index rows by id, keyed by (type, id) to disambiguate types.
|
||||
|
||||
A bare id can match one row per type (independent id sequences), so the
|
||||
result must carry every matching row rather than letting one clobber another.
|
||||
"""
|
||||
if not row_ids:
|
||||
return {}
|
||||
placeholders = ",".join(f":id_{idx}" for idx in range(len(row_ids)))
|
||||
@@ -2106,11 +2160,11 @@ class SearchRepositoryBase(ABC):
|
||||
WHERE project_id = :project_id
|
||||
AND id IN ({placeholders})
|
||||
"""
|
||||
result: dict[int, SearchIndexRow] = {}
|
||||
result: dict[SearchIndexKey, SearchIndexRow] = {}
|
||||
async with db.scoped_session(self.session_maker) as session:
|
||||
row_result = await session.execute(text(sql), params)
|
||||
for row in row_result.fetchall():
|
||||
result[row.id] = SearchIndexRow(
|
||||
result[(row.type, row.id)] = SearchIndexRow(
|
||||
project_id=self.project_id,
|
||||
id=row.id,
|
||||
title=row.title,
|
||||
@@ -2156,7 +2210,7 @@ class SearchRepositoryBase(ABC):
|
||||
) -> List[SearchIndexRow]:
|
||||
"""Fuse FTS and vector results using score-based fusion.
|
||||
|
||||
Uses search_index row id as the fusion key. The formula
|
||||
Uses the search_index (type, id) pair as the fusion key. The formula
|
||||
``max(vec, fts) + FUSION_BONUS * min(vec, fts)`` preserves
|
||||
the dominant signal and rewards dual-source agreement.
|
||||
"""
|
||||
@@ -2165,6 +2219,9 @@ class SearchRepositoryBase(ABC):
|
||||
query_start = time.perf_counter()
|
||||
candidate_limit = max(self._semantic_vector_k, (limit + offset) * 10)
|
||||
fts_start = time.perf_counter()
|
||||
# allow_relaxed: question-form queries rarely AND-match, and a dead FTS
|
||||
# branch silently degrades hybrid to vector-only ranking. Fusion plus
|
||||
# bm25 keep relaxed lexical candidates from dominating precision.
|
||||
fts_results = await self.search(
|
||||
search_text=search_text,
|
||||
permalink=permalink,
|
||||
@@ -2178,6 +2235,7 @@ class SearchRepositoryBase(ABC):
|
||||
retrieval_mode=SearchRetrievalMode.FTS,
|
||||
limit=candidate_limit,
|
||||
offset=0,
|
||||
allow_relaxed=True,
|
||||
)
|
||||
fts_ms = (time.perf_counter() - fts_start) * 1000
|
||||
vector_start = time.perf_counter()
|
||||
@@ -2199,17 +2257,19 @@ class SearchRepositoryBase(ABC):
|
||||
vector_ms = (time.perf_counter() - vector_start) * 1000
|
||||
fusion_start = time.perf_counter()
|
||||
|
||||
# --- Score-based fusion keyed on search_index row id ---
|
||||
# --- Score-based fusion keyed on (type, id) ---
|
||||
# A bare row id collides across row types (independent id sequences), so
|
||||
# fusion must key on (type, id) or distinct rows would merge (#982).
|
||||
# FTS scores are normalized to [0, 1] (BM25 is unbounded).
|
||||
# Vector scores are used raw — already calibrated [0, 1] by _distance_to_similarity().
|
||||
rows_by_id: dict[int, SearchIndexRow] = {}
|
||||
rows_by_key: dict[SearchIndexKey, SearchIndexRow] = {}
|
||||
|
||||
# Normalize FTS scores to [0, 1] — handles both SQLite (negative bm25)
|
||||
# and Postgres (positive ts_rank) by using absolute values
|
||||
fts_abs = [abs(row.score or 0.0) for row in fts_results]
|
||||
fts_max = max(fts_abs) if fts_abs else 1.0
|
||||
|
||||
fts_scores: dict[int, float] = {}
|
||||
fts_scores: dict[SearchIndexKey, float] = {}
|
||||
for row in fts_results:
|
||||
if row.id is None:
|
||||
continue
|
||||
@@ -2217,32 +2277,32 @@ class SearchRepositoryBase(ABC):
|
||||
# Gate: FTS scores below threshold contribute zero
|
||||
if norm < FTS_GATE_THRESHOLD:
|
||||
norm = 0.0
|
||||
fts_scores[row.id] = norm
|
||||
rows_by_id[row.id] = row
|
||||
fts_scores[(row.type, row.id)] = norm
|
||||
rows_by_key[(row.type, row.id)] = row
|
||||
|
||||
vec_scores: dict[int, float] = {}
|
||||
vec_scores: dict[SearchIndexKey, float] = {}
|
||||
for row in vector_results:
|
||||
if row.id is None:
|
||||
continue
|
||||
# Trigger: no re-normalization by vec_max
|
||||
# Why: vector similarity is already calibrated [0, 1]; re-normalizing
|
||||
# inflates weak matches when the entire result set is mediocre
|
||||
vec_scores[row.id] = row.score or 0.0
|
||||
rows_by_id[row.id] = row
|
||||
vec_scores[(row.type, row.id)] = row.score or 0.0
|
||||
rows_by_key[(row.type, row.id)] = row
|
||||
|
||||
# Fuse: max(v, f) + FUSION_BONUS * min(v, f)
|
||||
# Preserves the dominant signal; bonus rewards dual-source agreement.
|
||||
# Output range: [0, 1.3] for dual-source, [0, 1.0] for single-source.
|
||||
fused_scores: dict[int, float] = {}
|
||||
for row_id in fts_scores.keys() | vec_scores.keys():
|
||||
v = vec_scores.get(row_id, 0.0)
|
||||
f = fts_scores.get(row_id, 0.0)
|
||||
fused_scores[row_id] = max(v, f) + FUSION_BONUS * min(v, f)
|
||||
fused_scores: dict[SearchIndexKey, float] = {}
|
||||
for row_key in fts_scores.keys() | vec_scores.keys():
|
||||
v = vec_scores.get(row_key, 0.0)
|
||||
f = fts_scores.get(row_key, 0.0)
|
||||
fused_scores[row_key] = max(v, f) + FUSION_BONUS * min(v, f)
|
||||
|
||||
ranked = sorted(fused_scores.items(), key=lambda item: item[1], reverse=True)
|
||||
output: list[SearchIndexRow] = []
|
||||
for row_id, fused_score in ranked[offset : offset + limit]:
|
||||
row = rows_by_id[row_id]
|
||||
for row_key, fused_score in ranked[offset : offset + limit]:
|
||||
row = rows_by_key[row_key]
|
||||
# Trigger: FTS-only results have no matched_chunk_text from vector search.
|
||||
# Why: without chunk text, API falls back to truncated content, losing answer text.
|
||||
# Outcome: FTS-only results get full content_snippet as matched_chunk.
|
||||
|
||||
@@ -23,7 +23,10 @@ from basic_memory.models.search import (
|
||||
from basic_memory.repository.embedding_provider import EmbeddingProvider
|
||||
from basic_memory.repository.embedding_provider_factory import create_embedding_provider
|
||||
from basic_memory.repository.search_index_row import SearchIndexRow
|
||||
from basic_memory.repository.search_repository_base import SearchRepositoryBase
|
||||
from basic_memory.repository.search_repository_base import (
|
||||
SearchRepositoryBase,
|
||||
relaxed_query_words,
|
||||
)
|
||||
from basic_memory.repository.metadata_filters import parse_metadata_filters, build_sqlite_json_path
|
||||
from basic_memory.repository.semantic_errors import SemanticDependenciesMissingError
|
||||
from basic_memory.schemas.search import SearchItemType, SearchRetrievalMode
|
||||
@@ -255,6 +258,19 @@ class SQLiteSearchRepository(SearchRepositoryBase):
|
||||
if "*" in term and all(c.isalnum() or c in "*_-" for c in term):
|
||||
return term
|
||||
|
||||
# Natural-language queries arrive with sentence punctuation that FTS5
|
||||
# treats as syntax ("When did Melanie paint a sunrise?"). The tokenizer
|
||||
# ignores this punctuation in the INDEX, so stripping it from word
|
||||
# edges loses nothing — but leaving it forces the whole question into
|
||||
# an exact-phrase match that returns zero rows, silently disabling the
|
||||
# FTS half of hybrid search. Interior characters (hyphens, slashes —
|
||||
# permalinks and paths) are untouched.
|
||||
if " " in term:
|
||||
words = [word.strip("?!.,;:") for word in term.split()]
|
||||
term = " ".join(word for word in words if word)
|
||||
if not term:
|
||||
return ""
|
||||
|
||||
# Characters that can cause FTS5 syntax errors when used as operators
|
||||
# We're more conservative here - only quote when we detect problematic patterns
|
||||
problematic_chars = [
|
||||
@@ -351,6 +367,14 @@ class SQLiteSearchRepository(SearchRepositoryBase):
|
||||
# For non-Boolean queries, use the single term preparation logic
|
||||
return self._prepare_single_term(term, is_prefix)
|
||||
|
||||
@staticmethod
|
||||
def _relaxed_fts_text(search_text: Optional[str]) -> Optional[str]:
|
||||
"""OR-relaxed FTS5 expression for a failed strict query, or None."""
|
||||
words = relaxed_query_words(search_text)
|
||||
if not words:
|
||||
return None
|
||||
return " OR ".join(f"{word}*" for word in words)
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# sqlite-vec extension loading (SQLite-specific)
|
||||
# ------------------------------------------------------------------
|
||||
@@ -953,8 +977,16 @@ class SQLiteSearchRepository(SearchRepositoryBase):
|
||||
min_similarity: Optional[float] = None,
|
||||
limit: int = 10,
|
||||
offset: int = 0,
|
||||
allow_relaxed: bool = False,
|
||||
session: AsyncSession | None = None,
|
||||
) -> List[SearchIndexRow]:
|
||||
"""Search across all indexed content using SQLite FTS5."""
|
||||
"""Search across all indexed content using SQLite FTS5.
|
||||
|
||||
``allow_relaxed=True`` retries a zero-result strict multi-word query
|
||||
with OR-joined content terms. Only the hybrid path opts in: its FTS
|
||||
branch otherwise contributes nothing for question-form queries.
|
||||
Service-level FTS searches keep their own conservative fallback.
|
||||
"""
|
||||
# --- Dispatch vector / hybrid modes (shared logic) ---
|
||||
dispatched = await self._dispatch_retrieval_mode(
|
||||
search_text=search_text,
|
||||
@@ -1017,10 +1049,31 @@ class SQLiteSearchRepository(SearchRepositoryBase):
|
||||
"""
|
||||
|
||||
logger.trace(f"Search {sql} params: {params}")
|
||||
try:
|
||||
async with db.scoped_session(self.session_maker) as session:
|
||||
result = await session.execute(text(sql), params)
|
||||
|
||||
async def run_search(active_session: AsyncSession):
|
||||
result = await active_session.execute(text(sql), params)
|
||||
rows = result.fetchall()
|
||||
# Trigger: multi-word natural-language query matched nothing
|
||||
# under the default all-terms-AND semantics.
|
||||
# Why: questions ("when did X do Y") rarely have every word in
|
||||
# one document; without relaxation the FTS half of hybrid
|
||||
# search contributes zero candidates and ranking degrades to
|
||||
# vector-only.
|
||||
# Outcome: one retry with OR-joined prefix terms; bm25 still
|
||||
# ranks multi-term matches first.
|
||||
relaxed = self._relaxed_fts_text(search_text) if allow_relaxed and not rows else None
|
||||
if relaxed and params.get("text"):
|
||||
params["text"] = relaxed
|
||||
result = await active_session.execute(text(sql), params)
|
||||
rows = result.fetchall()
|
||||
return rows
|
||||
|
||||
try:
|
||||
if session is not None:
|
||||
rows = await run_search(session)
|
||||
else:
|
||||
async with db.scoped_session(self.session_maker) as owned_session:
|
||||
rows = await run_search(owned_session)
|
||||
except Exception as e:
|
||||
# Handle FTS5 syntax errors and provide user-friendly feedback
|
||||
if self._is_fts5_syntax_error(e): # pragma: no cover
|
||||
|
||||
@@ -19,7 +19,6 @@ from pathlib import Path
|
||||
from typing import List, Optional, Annotated, Dict
|
||||
|
||||
from annotated_types import MinLen, MaxLen
|
||||
from dateparser import parse
|
||||
|
||||
from pydantic import BaseModel, BeforeValidator, Field, model_validator, computed_field
|
||||
|
||||
@@ -92,6 +91,10 @@ def parse_timeframe(timeframe: str) -> datetime:
|
||||
parse_timeframe('1d') -> 2025-06-04 14:50:00-07:00 (24 hours ago with local timezone)
|
||||
parse_timeframe('1 week ago') -> 2025-05-29 14:50:00-07:00 (1 week ago with local timezone)
|
||||
"""
|
||||
# Deferred: dateparser costs ~0.13s to import; schemas load on every CLI
|
||||
# start, but timeframe parsing only happens per request (#886).
|
||||
from dateparser import parse
|
||||
|
||||
if timeframe.lower() == "today":
|
||||
# For "today", return 1 day ago to ensure we capture recent activity across timezones
|
||||
# This handles the case where client and server are in different timezones
|
||||
|
||||
@@ -155,6 +155,9 @@ class RelationSummary(BaseModel):
|
||||
from_entity_id: Optional[int] = None
|
||||
from_entity_external_id: Optional[str] = None
|
||||
to_entity: Optional[str] = None
|
||||
# Literal target text from the markdown; present even when the relation is
|
||||
# an unresolved forward reference (to_entity is None until the target exists)
|
||||
to_name: Optional[str] = None
|
||||
to_entity_id: Optional[int] = None
|
||||
to_entity_external_id: Optional[str] = None
|
||||
created_at: Annotated[
|
||||
|
||||
@@ -2,15 +2,18 @@
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from contextlib import asynccontextmanager
|
||||
from dataclasses import dataclass, field
|
||||
from datetime import datetime, timezone
|
||||
from typing import Any, List, Optional, Tuple, TYPE_CHECKING
|
||||
from typing import Any, AsyncIterator, List, Optional, Tuple, TYPE_CHECKING
|
||||
|
||||
|
||||
from loguru import logger
|
||||
from sqlalchemy import text
|
||||
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
|
||||
|
||||
import logfire
|
||||
from basic_memory import db
|
||||
from basic_memory.repository.entity_repository import EntityRepository
|
||||
from basic_memory.repository.observation_repository import ObservationRepository
|
||||
from basic_memory.repository.postgres_search_repository import PostgresSearchRepository
|
||||
@@ -18,6 +21,7 @@ from basic_memory.repository.search_repository import SearchRepository, SearchIn
|
||||
from basic_memory.schemas.memory import MemoryUrl, memory_url_path
|
||||
from basic_memory.schemas.search import SearchItemType
|
||||
from basic_memory.utils import generate_permalink
|
||||
from basic_memory.workspace_context import workspace_slug_for_canonical_permalinks
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from basic_memory.services.link_resolver import LinkResolver
|
||||
@@ -36,6 +40,7 @@ class ContextResultRow:
|
||||
from_id: Optional[int] = None
|
||||
to_id: Optional[int] = None
|
||||
relation_type: Optional[str] = None
|
||||
to_name: Optional[str] = None
|
||||
content: Optional[str] = None
|
||||
category: Optional[str] = None
|
||||
entity_id: Optional[int] = None
|
||||
@@ -89,11 +94,21 @@ class ContextService:
|
||||
entity_repository: EntityRepository,
|
||||
observation_repository: ObservationRepository,
|
||||
link_resolver: Optional[LinkResolver] = None,
|
||||
session_maker: async_sessionmaker[AsyncSession] | None = None,
|
||||
):
|
||||
self.search_repository = search_repository
|
||||
self.entity_repository = entity_repository
|
||||
self.observation_repository = observation_repository
|
||||
self.link_resolver = link_resolver
|
||||
self.session_maker = session_maker
|
||||
|
||||
@asynccontextmanager
|
||||
async def _session_scope(self) -> AsyncIterator[AsyncSession]:
|
||||
"""Open a service-owned transaction for core repository reads."""
|
||||
if self.session_maker is None: # pragma: no cover
|
||||
raise ValueError("session_maker is required for ContextService")
|
||||
async with db.scoped_session(self.session_maker) as session:
|
||||
yield session
|
||||
|
||||
async def build_context(
|
||||
self,
|
||||
@@ -143,6 +158,29 @@ class ContextService:
|
||||
primary = await self.search_repository.search(
|
||||
permalink_match=normalized_path, limit=fetch_limit, offset=offset
|
||||
)
|
||||
|
||||
# Trigger: a workspace-qualified pattern matched nothing while a
|
||||
# workspace permalink context is active.
|
||||
# Why: rows written before workspace canonicalization (or via
|
||||
# clients that didn't forward workspace headers) store
|
||||
# project-qualified permalinks; a workspace-prefixed pattern
|
||||
# can never match those legacy rows (#957).
|
||||
# Outcome: retry once with the workspace prefix stripped so the
|
||||
# pattern matches the index form the rows actually carry.
|
||||
if not primary:
|
||||
workspace_slug = workspace_slug_for_canonical_permalinks()
|
||||
ws_prefix = f"{workspace_slug}/" if workspace_slug else None
|
||||
if ws_prefix and normalized_path.startswith(ws_prefix):
|
||||
fallback_path = normalized_path.removeprefix(ws_prefix)
|
||||
logger.debug(
|
||||
f"Pattern search fallback without workspace prefix: "
|
||||
f"'{fallback_path}'"
|
||||
)
|
||||
primary = await self.search_repository.search(
|
||||
permalink_match=fallback_path,
|
||||
limit=fetch_limit,
|
||||
offset=offset,
|
||||
)
|
||||
else:
|
||||
normalized_path = generate_permalink(path, split_extension=False)
|
||||
logger.debug(f"Direct lookup for '{normalized_path}'")
|
||||
@@ -151,9 +189,13 @@ class ContextService:
|
||||
)
|
||||
|
||||
if not primary and self.link_resolver:
|
||||
entity = await self.link_resolver.resolve_link(
|
||||
path, use_search=True, strict=False
|
||||
)
|
||||
async with self._session_scope() as session:
|
||||
entity = await self.link_resolver.resolve_link(
|
||||
path,
|
||||
use_search=True,
|
||||
strict=False,
|
||||
session=session,
|
||||
)
|
||||
if entity:
|
||||
logger.debug(
|
||||
f"LinkResolver resolved '{path}' to permalink '{entity.permalink}'"
|
||||
@@ -209,9 +251,10 @@ class ContextService:
|
||||
phase="load_observations",
|
||||
result_count=len(entity_ids),
|
||||
):
|
||||
observations_by_entity = await self.observation_repository.find_by_entities(
|
||||
entity_ids
|
||||
)
|
||||
async with self._session_scope() as session:
|
||||
observations_by_entity = await self.observation_repository.find_by_entities(
|
||||
session, entity_ids
|
||||
)
|
||||
logger.debug(f"Found observations for {len(observations_by_entity)} entities")
|
||||
|
||||
metadata = ContextMetadata(
|
||||
@@ -392,6 +435,7 @@ class ContextService:
|
||||
from_id=row.from_id,
|
||||
to_id=row.to_id,
|
||||
relation_type=row.relation_type,
|
||||
to_name=row.to_name,
|
||||
content=row.content,
|
||||
category=row.category,
|
||||
entity_id=row.entity_id,
|
||||
@@ -426,6 +470,7 @@ class ContextService:
|
||||
CAST(NULL AS INTEGER) as from_id,
|
||||
CAST(NULL AS INTEGER) as to_id,
|
||||
CAST(NULL AS TEXT) as relation_type,
|
||||
CAST(NULL AS TEXT) as to_name,
|
||||
CAST(NULL AS TEXT) as content,
|
||||
CAST(NULL AS TEXT) as category,
|
||||
CAST(NULL AS INTEGER) as entity_id,
|
||||
@@ -478,6 +523,10 @@ class ContextService:
|
||||
WHEN step_type = 1 THEN r.relation_type
|
||||
ELSE NULL
|
||||
END as relation_type,
|
||||
CASE
|
||||
WHEN step_type = 1 THEN r.to_name
|
||||
ELSE NULL
|
||||
END as to_name,
|
||||
CAST(NULL AS TEXT) as content,
|
||||
CAST(NULL AS TEXT) as category,
|
||||
CAST(NULL AS INTEGER) as entity_id,
|
||||
@@ -541,6 +590,7 @@ class ContextService:
|
||||
from_id,
|
||||
to_id,
|
||||
relation_type,
|
||||
to_name,
|
||||
content,
|
||||
category,
|
||||
entity_id,
|
||||
@@ -550,7 +600,7 @@ class ContextService:
|
||||
FROM entity_graph
|
||||
WHERE depth > 0
|
||||
GROUP BY type, id, title, permalink, file_path, from_id, to_id,
|
||||
relation_type, content, category, entity_id, root_id, created_at
|
||||
relation_type, to_name, content, category, entity_id, root_id, created_at
|
||||
ORDER BY depth, type, id
|
||||
LIMIT :max_results
|
||||
""")
|
||||
@@ -579,6 +629,7 @@ class ContextService:
|
||||
NULL as from_id,
|
||||
NULL as to_id,
|
||||
NULL as relation_type,
|
||||
NULL as to_name,
|
||||
NULL as content,
|
||||
NULL as category,
|
||||
NULL as entity_id,
|
||||
@@ -606,6 +657,7 @@ class ContextService:
|
||||
r.from_id,
|
||||
r.to_id,
|
||||
r.relation_type,
|
||||
r.to_name,
|
||||
NULL as content,
|
||||
NULL as category,
|
||||
NULL as entity_id,
|
||||
@@ -644,6 +696,7 @@ class ContextService:
|
||||
NULL as from_id,
|
||||
NULL as to_id,
|
||||
NULL as relation_type,
|
||||
NULL as to_name,
|
||||
NULL as content,
|
||||
NULL as category,
|
||||
NULL as entity_id,
|
||||
@@ -677,6 +730,7 @@ class ContextService:
|
||||
from_id,
|
||||
to_id,
|
||||
relation_type,
|
||||
to_name,
|
||||
content,
|
||||
category,
|
||||
entity_id,
|
||||
@@ -686,7 +740,7 @@ class ContextService:
|
||||
FROM entity_graph
|
||||
WHERE depth > 0
|
||||
GROUP BY type, id, title, permalink, file_path, from_id, to_id,
|
||||
relation_type, content, category, entity_id, root_id, created_at
|
||||
relation_type, to_name, content, category, entity_id, root_id, created_at
|
||||
ORDER BY depth, type, id
|
||||
LIMIT :max_results
|
||||
""")
|
||||
|
||||
@@ -3,10 +3,14 @@
|
||||
import fnmatch
|
||||
import logging
|
||||
import os
|
||||
from contextlib import asynccontextmanager
|
||||
from datetime import datetime
|
||||
from typing import Dict, List, Optional, Sequence
|
||||
from typing import AsyncIterator, Dict, List, Optional, Sequence
|
||||
|
||||
|
||||
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
|
||||
|
||||
from basic_memory import db
|
||||
from basic_memory.models import Entity
|
||||
from basic_memory.repository import EntityRepository
|
||||
from basic_memory.schemas.directory import DirectoryNode
|
||||
@@ -28,19 +32,31 @@ def _mtime_to_datetime(entity: Entity) -> datetime:
|
||||
class DirectoryService:
|
||||
"""Service for working with directory trees."""
|
||||
|
||||
def __init__(self, entity_repository: EntityRepository):
|
||||
def __init__(
|
||||
self,
|
||||
entity_repository: EntityRepository,
|
||||
session_maker: async_sessionmaker[AsyncSession],
|
||||
):
|
||||
"""Initialize the directory service.
|
||||
|
||||
Args:
|
||||
entity_repository: Directory repository for data access.
|
||||
"""
|
||||
self.entity_repository = entity_repository
|
||||
self.session_maker = session_maker
|
||||
|
||||
@asynccontextmanager
|
||||
async def _session_scope(self) -> AsyncIterator[AsyncSession]:
|
||||
"""Open a service-owned transaction."""
|
||||
async with db.scoped_session(self.session_maker) as session:
|
||||
yield session
|
||||
|
||||
async def get_directory_tree(self) -> DirectoryNode:
|
||||
"""Build a hierarchical directory tree from indexed files."""
|
||||
|
||||
# Get all files from DB (flat list)
|
||||
entity_rows = await self.entity_repository.find_all()
|
||||
async with self._session_scope() as session:
|
||||
entity_rows = await self.entity_repository.find_all(session)
|
||||
|
||||
# Create a root directory node
|
||||
root_node = DirectoryNode(name="Root", directory_path="/", type="directory")
|
||||
@@ -114,7 +130,8 @@ class DirectoryService:
|
||||
DirectoryNode tree containing only folders (type="directory")
|
||||
"""
|
||||
# Get unique directories without loading entities
|
||||
directories = await self.entity_repository.get_distinct_directories()
|
||||
async with self._session_scope() as session:
|
||||
directories = await self.entity_repository.get_distinct_directories(session)
|
||||
|
||||
# Create a root directory node
|
||||
root_node = DirectoryNode(name="Root", directory_path="/", type="directory")
|
||||
@@ -179,7 +196,8 @@ class DirectoryService:
|
||||
# Optimize: Query only entities in the target directory
|
||||
# instead of loading the entire tree
|
||||
dir_prefix = dir_name.lstrip("/")
|
||||
entity_rows = await self.entity_repository.find_by_directory_prefix(dir_prefix)
|
||||
async with self._session_scope() as session:
|
||||
entity_rows = await self.entity_repository.find_by_directory_prefix(session, dir_prefix)
|
||||
|
||||
# Build a partial tree from only the relevant entities
|
||||
root_tree = self._build_directory_tree_from_entities(entity_rows, dir_name)
|
||||
|
||||
@@ -1,16 +1,18 @@
|
||||
"""Service for managing entities in the database."""
|
||||
|
||||
import asyncio
|
||||
from collections.abc import Callable
|
||||
from contextlib import asynccontextmanager
|
||||
from copy import deepcopy
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import Any, List, Optional, Sequence, Tuple, Union
|
||||
from typing import Any, AsyncIterator, List, Optional, Sequence, Tuple, Union
|
||||
|
||||
import frontmatter
|
||||
import yaml
|
||||
from loguru import logger
|
||||
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
|
||||
|
||||
from basic_memory import db
|
||||
from basic_memory.config import ProjectConfig, BasicMemoryConfig
|
||||
from basic_memory.file_utils import (
|
||||
has_frontmatter,
|
||||
@@ -101,6 +103,7 @@ class EntityService(BaseService[EntityModel]):
|
||||
relation_repository: RelationRepository,
|
||||
file_service: FileService,
|
||||
link_resolver: LinkResolver,
|
||||
session_maker: async_sessionmaker[AsyncSession],
|
||||
search_service: Optional[SearchService] = None,
|
||||
app_config: Optional[BasicMemoryConfig] = None,
|
||||
):
|
||||
@@ -110,6 +113,7 @@ class EntityService(BaseService[EntityModel]):
|
||||
self.entity_parser = entity_parser
|
||||
self.file_service = file_service
|
||||
self.link_resolver = link_resolver
|
||||
self.session_maker = session_maker
|
||||
self.search_service = search_service
|
||||
self.app_config = app_config
|
||||
self._project_permalink: Optional[str] = None
|
||||
@@ -117,8 +121,23 @@ class EntityService(BaseService[EntityModel]):
|
||||
# Default returns None for local/CLI usage. Cloud overrides this to read from UserContext.
|
||||
self.get_user_id: Callable[[], Optional[str]] = lambda: None
|
||||
|
||||
@asynccontextmanager
|
||||
async def _session_scope(
|
||||
self, session: AsyncSession | None = None
|
||||
) -> AsyncIterator[AsyncSession]:
|
||||
"""Use the caller's session or open a service-owned transaction."""
|
||||
if session is not None:
|
||||
yield session
|
||||
return
|
||||
|
||||
async with db.scoped_session(self.session_maker) as owned_session:
|
||||
yield owned_session
|
||||
|
||||
async def detect_file_path_conflicts(
|
||||
self, file_path: str, skip_check: bool = False
|
||||
self,
|
||||
file_path: str,
|
||||
skip_check: bool = False,
|
||||
session: AsyncSession | None = None,
|
||||
) -> List[str]:
|
||||
"""Detect potential file path conflicts for a given file path.
|
||||
|
||||
@@ -142,7 +161,8 @@ class EntityService(BaseService[EntityModel]):
|
||||
|
||||
# Load only file paths. Conflict detection is on the hot write path and
|
||||
# does not need observations or relations.
|
||||
existing_paths = await self.repository.get_all_file_paths()
|
||||
async with self._session_scope(session) as active_session:
|
||||
existing_paths = await self.repository.get_all_file_paths(active_session)
|
||||
|
||||
# Use the enhanced conflict detection utility
|
||||
return detect_potential_file_conflicts(file_path, existing_paths)
|
||||
@@ -152,6 +172,7 @@ class EntityService(BaseService[EntityModel]):
|
||||
file_path: Permalink | Path,
|
||||
markdown: Optional[EntityMarkdown] = None,
|
||||
skip_conflict_check: bool = False,
|
||||
session: AsyncSession | None = None,
|
||||
) -> str:
|
||||
"""Get or generate unique permalink for an entity.
|
||||
|
||||
@@ -169,71 +190,74 @@ class EntityService(BaseService[EntityModel]):
|
||||
file_path_str = Path(file_path).as_posix()
|
||||
|
||||
# Check for potential file path conflicts before resolving permalink
|
||||
conflicts = await self.detect_file_path_conflicts(
|
||||
file_path_str, skip_check=skip_conflict_check
|
||||
)
|
||||
if conflicts:
|
||||
logger.warning(
|
||||
f"Detected potential file path conflicts for '{file_path_str}': {conflicts}"
|
||||
async with self._session_scope(session) as active_session:
|
||||
conflicts = await self.detect_file_path_conflicts(
|
||||
file_path_str, skip_check=skip_conflict_check, session=active_session
|
||||
)
|
||||
if conflicts:
|
||||
logger.warning(
|
||||
f"Detected potential file path conflicts for '{file_path_str}': {conflicts}"
|
||||
)
|
||||
|
||||
# If markdown has explicit permalink, try to validate it
|
||||
if markdown and markdown.frontmatter.permalink:
|
||||
desired_permalink = markdown.frontmatter.permalink
|
||||
# Use lightweight method - we only need to check file_path
|
||||
existing_file_path = await self.repository.get_file_path_for_permalink(
|
||||
desired_permalink
|
||||
# If markdown has explicit permalink, try to validate it
|
||||
if markdown and markdown.frontmatter.permalink:
|
||||
desired_permalink = markdown.frontmatter.permalink
|
||||
# Use lightweight method - we only need to check file_path
|
||||
existing_file_path = await self.repository.get_file_path_for_permalink(
|
||||
active_session, desired_permalink
|
||||
)
|
||||
|
||||
# If no conflict or it's our own file, use as is
|
||||
if not existing_file_path or existing_file_path == file_path_str:
|
||||
return desired_permalink
|
||||
|
||||
# For existing files, try to find current permalink
|
||||
# Use lightweight method - we only need the permalink
|
||||
existing_permalink = await self.repository.get_permalink_for_file_path(
|
||||
active_session, file_path_str
|
||||
)
|
||||
if existing_permalink:
|
||||
return existing_permalink
|
||||
|
||||
# If no conflict or it's our own file, use as is
|
||||
if not existing_file_path or existing_file_path == file_path_str:
|
||||
return desired_permalink
|
||||
# New file - generate permalink
|
||||
if markdown and markdown.frontmatter.permalink:
|
||||
desired_permalink = markdown.frontmatter.permalink
|
||||
else:
|
||||
# Trigger: generating a permalink for a new file
|
||||
# Why: canonical permalinks may require project prefix for global addressing
|
||||
# Outcome: include project slug when enabled in config
|
||||
include_project = True
|
||||
if self.app_config:
|
||||
include_project = self.app_config.permalinks_include_project
|
||||
|
||||
# For existing files, try to find current permalink
|
||||
# Use lightweight method - we only need the permalink
|
||||
existing_permalink = await self.repository.get_permalink_for_file_path(file_path_str)
|
||||
if existing_permalink:
|
||||
return existing_permalink
|
||||
workspace_permalink = workspace_slug_for_canonical_permalinks()
|
||||
project_permalink = None
|
||||
# Trigger: project-prefixed permalinks are enabled, or organization workspace
|
||||
# context requires a complete workspace/project canonical permalink.
|
||||
# Why: project slug is the stable middle segment for globally addressable links.
|
||||
# Outcome: fetch and cache the project's permalink before building the canonical URL.
|
||||
if include_project or workspace_permalink:
|
||||
project_permalink = await self._get_project_permalink(active_session)
|
||||
|
||||
# New file - generate permalink
|
||||
if markdown and markdown.frontmatter.permalink:
|
||||
desired_permalink = markdown.frontmatter.permalink
|
||||
else:
|
||||
# Trigger: generating a permalink for a new file
|
||||
# Why: canonical permalinks may require project prefix for global addressing
|
||||
# Outcome: include project slug when enabled in config
|
||||
include_project = True
|
||||
if self.app_config:
|
||||
include_project = self.app_config.permalinks_include_project
|
||||
desired_permalink = build_canonical_permalink(
|
||||
project_permalink,
|
||||
file_path_str,
|
||||
include_project=include_project,
|
||||
workspace_permalink=workspace_permalink,
|
||||
)
|
||||
|
||||
workspace_permalink = workspace_slug_for_canonical_permalinks()
|
||||
project_permalink = None
|
||||
# Trigger: project-prefixed permalinks are enabled, or organization workspace
|
||||
# context requires a complete workspace/project canonical permalink.
|
||||
# Why: project slug is the stable middle segment for globally addressable links.
|
||||
# Outcome: fetch and cache the project's permalink before building the canonical URL.
|
||||
if include_project or workspace_permalink:
|
||||
project_permalink = await self._get_project_permalink()
|
||||
|
||||
desired_permalink = build_canonical_permalink(
|
||||
project_permalink,
|
||||
file_path_str,
|
||||
include_project=include_project,
|
||||
workspace_permalink=workspace_permalink,
|
||||
)
|
||||
|
||||
# Make unique if needed - enhanced to handle character conflicts
|
||||
# Use lightweight existence check instead of loading full entity
|
||||
permalink = desired_permalink
|
||||
suffix = 1
|
||||
while await self.repository.permalink_exists(permalink):
|
||||
permalink = f"{desired_permalink}-{suffix}"
|
||||
suffix += 1
|
||||
logger.debug(f"creating unique permalink: {permalink}")
|
||||
# Make unique if needed - enhanced to handle character conflicts
|
||||
# Use lightweight existence check instead of loading full entity
|
||||
permalink = desired_permalink
|
||||
suffix = 1
|
||||
while await self.repository.permalink_exists(active_session, permalink):
|
||||
permalink = f"{desired_permalink}-{suffix}"
|
||||
suffix += 1
|
||||
logger.debug(f"creating unique permalink: {permalink}")
|
||||
|
||||
return permalink
|
||||
|
||||
async def _get_project_permalink(self) -> Optional[str]:
|
||||
async def _get_project_permalink(self, session: AsyncSession) -> Optional[str]:
|
||||
"""Get and cache the current project's permalink."""
|
||||
if self._project_permalink is not None:
|
||||
return self._project_permalink
|
||||
@@ -242,8 +266,8 @@ class EntityService(BaseService[EntityModel]):
|
||||
if project_id is None: # pragma: no cover
|
||||
return None # pragma: no cover
|
||||
|
||||
project_repository = ProjectRepository(self.repository.session_maker)
|
||||
project = await project_repository.get_by_id(project_id)
|
||||
project_repository = ProjectRepository()
|
||||
project = await project_repository.get_by_id(session, project_id)
|
||||
if project:
|
||||
self._project_permalink = project.permalink
|
||||
return self._project_permalink
|
||||
@@ -343,6 +367,7 @@ class EntityService(BaseService[EntityModel]):
|
||||
current_permalink: str | None = None,
|
||||
content_markdown: EntityMarkdown | None = None,
|
||||
skip_conflict_check: bool = False,
|
||||
session: AsyncSession | None = None,
|
||||
) -> str | None:
|
||||
"""Resolve the canonical permalink for a create/update write."""
|
||||
if self.app_config and self.app_config.disable_permalinks:
|
||||
@@ -360,6 +385,7 @@ class EntityService(BaseService[EntityModel]):
|
||||
file_path,
|
||||
content_markdown,
|
||||
skip_conflict_check=skip_conflict_check,
|
||||
session=session,
|
||||
)
|
||||
schema._permalink = resolved_permalink
|
||||
return resolved_permalink
|
||||
@@ -437,6 +463,7 @@ class EntityService(BaseService[EntityModel]):
|
||||
*,
|
||||
check_storage_exists: bool = True,
|
||||
skip_conflict_check: bool = False,
|
||||
session: AsyncSession | None = None,
|
||||
) -> PreparedEntityWrite:
|
||||
"""Derive accepted markdown and entity fields for a new note.
|
||||
|
||||
@@ -469,6 +496,7 @@ class EntityService(BaseService[EntityModel]):
|
||||
file_path=file_path,
|
||||
content_markdown=content_markdown,
|
||||
skip_conflict_check=skip_conflict_check,
|
||||
session=session,
|
||||
)
|
||||
|
||||
# Build the final markdown once here. Local mode will write it immediately; cloud mode can
|
||||
@@ -496,6 +524,7 @@ class EntityService(BaseService[EntityModel]):
|
||||
existing_content: str,
|
||||
*,
|
||||
skip_conflict_check: bool = False,
|
||||
session: AsyncSession | None = None,
|
||||
) -> PreparedEntityWrite:
|
||||
"""Derive accepted markdown and entity fields for a full note replacement.
|
||||
|
||||
@@ -533,6 +562,7 @@ class EntityService(BaseService[EntityModel]):
|
||||
current_permalink=current_permalink,
|
||||
content_markdown=content_markdown,
|
||||
skip_conflict_check=skip_conflict_check,
|
||||
session=session,
|
||||
)
|
||||
|
||||
post = await schema_to_markdown(schema)
|
||||
@@ -573,6 +603,7 @@ class EntityService(BaseService[EntityModel]):
|
||||
find_text: Optional[str] = None,
|
||||
expected_replacements: int = 1,
|
||||
skip_conflict_check: bool = False,
|
||||
session: AsyncSession | None = None,
|
||||
) -> PreparedEntityWrite:
|
||||
"""Derive accepted markdown and entity fields for an edit request.
|
||||
|
||||
@@ -621,6 +652,7 @@ class EntityService(BaseService[EntityModel]):
|
||||
file_path,
|
||||
content_markdown,
|
||||
skip_conflict_check=skip_conflict_check,
|
||||
session=session,
|
||||
)
|
||||
|
||||
normalized_metadata = normalize_frontmatter_metadata(content_frontmatter or {})
|
||||
@@ -676,30 +708,34 @@ class EntityService(BaseService[EntityModel]):
|
||||
async def create_entity_with_content(self, schema: EntitySchema) -> EntityWriteResult:
|
||||
"""Create a new entity and return both the entity row and written markdown."""
|
||||
logger.debug(f"Creating entity: {schema.title}")
|
||||
# --- Prepare Accepted State ---
|
||||
# Derive the canonical markdown/entity fields before touching the filesystem.
|
||||
prepared = await self.prepare_create_entity_content(schema)
|
||||
self._sync_prepared_schema_state(schema, prepared)
|
||||
# --- Persist File, Then Indexable DB State ---
|
||||
# Local mode still writes the file immediately; the prepare object keeps semantics separate
|
||||
# from that persistence step.
|
||||
checksum = await self.file_service.write_file(prepared.file_path, prepared.markdown_content)
|
||||
entity = await self.upsert_entity_from_markdown(
|
||||
prepared.file_path,
|
||||
prepared.entity_markdown,
|
||||
is_new=True,
|
||||
)
|
||||
updated = await self.repository.update(entity.id, {"checksum": checksum})
|
||||
if not updated: # pragma: no cover
|
||||
raise ValueError(f"Failed to update entity checksum after create: {entity.id}")
|
||||
persisted_content, search_content = await self._read_persisted_write_content(
|
||||
prepared.file_path
|
||||
)
|
||||
return EntityWriteResult(
|
||||
entity=updated,
|
||||
content=persisted_content,
|
||||
search_content=search_content,
|
||||
)
|
||||
async with self._session_scope() as session:
|
||||
# --- Prepare Accepted State ---
|
||||
# Derive the canonical markdown/entity fields before touching the filesystem.
|
||||
prepared = await self.prepare_create_entity_content(schema, session=session)
|
||||
self._sync_prepared_schema_state(schema, prepared)
|
||||
# --- Persist File, Then Indexable DB State ---
|
||||
# Local mode still writes the file immediately; the prepare object keeps semantics separate
|
||||
# from that persistence step.
|
||||
checksum = await self.file_service.write_file(
|
||||
prepared.file_path, prepared.markdown_content
|
||||
)
|
||||
entity = await self.upsert_entity_from_markdown(
|
||||
prepared.file_path,
|
||||
prepared.entity_markdown,
|
||||
is_new=True,
|
||||
session=session,
|
||||
)
|
||||
updated = await self.repository.update(session, entity.id, {"checksum": checksum})
|
||||
if not updated: # pragma: no cover
|
||||
raise ValueError(f"Failed to update entity checksum after create: {entity.id}")
|
||||
persisted_content, search_content = await self._read_persisted_write_content(
|
||||
prepared.file_path
|
||||
)
|
||||
return EntityWriteResult(
|
||||
entity=updated,
|
||||
content=persisted_content,
|
||||
search_content=search_content,
|
||||
)
|
||||
|
||||
async def update_entity(self, entity: EntityModel, schema: EntitySchema) -> EntityModel:
|
||||
"""Update an entity's content and metadata."""
|
||||
@@ -716,58 +752,63 @@ class EntityService(BaseService[EntityModel]):
|
||||
f"Updating entity with permalink: {entity.permalink} content-type: {schema.content_type}"
|
||||
)
|
||||
|
||||
# --- Read Current File State ---
|
||||
# Full replacements merge with existing frontmatter, so local mode still needs the current
|
||||
# file contents as input to the prepare step.
|
||||
existing_content = await self.file_service.read_file_content(entity.file_path)
|
||||
prepared = await self.prepare_update_entity_content(
|
||||
entity,
|
||||
schema,
|
||||
existing_content,
|
||||
)
|
||||
self._sync_prepared_schema_state(schema, prepared)
|
||||
previous_file_path = Path(entity.file_path)
|
||||
# Trigger: a full replacement also renames the note to a different canonical path.
|
||||
# Why: Path.replace() overwrites existing files, so the destination must be conflict-free
|
||||
# before we write or we can clobber another note and only fail later at the DB layer.
|
||||
# Outcome: conflicting rename attempts fail before touching either file on disk.
|
||||
if (
|
||||
prepared.file_path.as_posix() != previous_file_path.as_posix()
|
||||
and await self.file_service.exists(prepared.file_path)
|
||||
and not self._paths_share_storage_target(previous_file_path, prepared.file_path)
|
||||
):
|
||||
raise EntityAlreadyExistsError(
|
||||
f"file already exists at destination path: {prepared.file_path.as_posix()}"
|
||||
async with self._session_scope() as session:
|
||||
# --- Read Current File State ---
|
||||
# Full replacements merge with existing frontmatter, so local mode still needs the current
|
||||
# file contents as input to the prepare step.
|
||||
existing_content = await self.file_service.read_file_content(entity.file_path)
|
||||
prepared = await self.prepare_update_entity_content(
|
||||
entity,
|
||||
schema,
|
||||
existing_content,
|
||||
session=session,
|
||||
)
|
||||
self._sync_prepared_schema_state(schema, prepared)
|
||||
previous_file_path = Path(entity.file_path)
|
||||
# Trigger: a full replacement also renames the note to a different canonical path.
|
||||
# Why: Path.replace() overwrites existing files, so the destination must be conflict-free
|
||||
# before we write or we can clobber another note and only fail later at the DB layer.
|
||||
# Outcome: conflicting rename attempts fail before touching either file on disk.
|
||||
if (
|
||||
prepared.file_path.as_posix() != previous_file_path.as_posix()
|
||||
and await self.file_service.exists(prepared.file_path)
|
||||
and not self._paths_share_storage_target(previous_file_path, prepared.file_path)
|
||||
):
|
||||
raise EntityAlreadyExistsError(
|
||||
f"file already exists at destination path: {prepared.file_path.as_posix()}"
|
||||
)
|
||||
# --- Persist Prepared State ---
|
||||
checksum = await self.file_service.write_file(
|
||||
prepared.file_path,
|
||||
prepared.markdown_content,
|
||||
)
|
||||
entity = await self.upsert_entity_from_markdown(
|
||||
prepared.file_path,
|
||||
prepared.entity_markdown,
|
||||
is_new=False,
|
||||
existing_entity=entity,
|
||||
session=session,
|
||||
)
|
||||
if prepared.file_path.as_posix() != previous_file_path.as_posix():
|
||||
# Trigger: a full replacement changed the canonical note path.
|
||||
# Why: the new file has already been written and the entity now points at it.
|
||||
# Outcome: remove the stale old file so local Basic Memory mirrors cloud's PGQ cleanup.
|
||||
if not self._paths_share_storage_target(previous_file_path, prepared.file_path):
|
||||
await self.file_service.delete_file(previous_file_path)
|
||||
entity = await self.repository.update(session, entity.id, {"checksum": checksum})
|
||||
if not entity: # pragma: no cover
|
||||
raise ValueError(
|
||||
f"Failed to update entity checksum after update: {prepared.file_path}"
|
||||
)
|
||||
persisted_content, search_content = await self._read_persisted_write_content(
|
||||
prepared.file_path
|
||||
)
|
||||
# --- Persist Prepared State ---
|
||||
checksum = await self.file_service.write_file(
|
||||
prepared.file_path,
|
||||
prepared.markdown_content,
|
||||
)
|
||||
entity = await self.upsert_entity_from_markdown(
|
||||
prepared.file_path,
|
||||
prepared.entity_markdown,
|
||||
is_new=False,
|
||||
existing_entity=entity,
|
||||
)
|
||||
if prepared.file_path.as_posix() != previous_file_path.as_posix():
|
||||
# Trigger: a full replacement changed the canonical note path.
|
||||
# Why: the new file has already been written and the entity now points at it.
|
||||
# Outcome: remove the stale old file so local Basic Memory mirrors cloud's PGQ cleanup.
|
||||
if not self._paths_share_storage_target(previous_file_path, prepared.file_path):
|
||||
await self.file_service.delete_file(previous_file_path)
|
||||
entity = await self.repository.update(entity.id, {"checksum": checksum})
|
||||
if not entity: # pragma: no cover
|
||||
raise ValueError(f"Failed to update entity checksum after update: {prepared.file_path}")
|
||||
persisted_content, search_content = await self._read_persisted_write_content(
|
||||
prepared.file_path
|
||||
)
|
||||
|
||||
return EntityWriteResult(
|
||||
entity=entity,
|
||||
content=persisted_content,
|
||||
search_content=search_content,
|
||||
)
|
||||
return EntityWriteResult(
|
||||
entity=entity,
|
||||
content=persisted_content,
|
||||
search_content=search_content,
|
||||
)
|
||||
|
||||
async def delete_entity(self, permalink_or_id: str | int) -> bool:
|
||||
"""Delete entity and its file."""
|
||||
@@ -812,7 +853,8 @@ class EntityService(BaseService[EntityModel]):
|
||||
# Trigger: repository.delete returns False when entity is already gone (NoResultFound)
|
||||
# Why: concurrent delete_directory requests can race to delete the same entity
|
||||
# Outcome: treat as success since the entity is deleted either way
|
||||
deleted = await self.repository.delete(entity.id)
|
||||
async with self._session_scope() as session:
|
||||
deleted = await self.repository.delete(session, entity.id)
|
||||
if not deleted:
|
||||
logger.info("Entity already removed from DB", entity_id=permalink_or_id)
|
||||
return True
|
||||
@@ -824,7 +866,8 @@ class EntityService(BaseService[EntityModel]):
|
||||
async def get_by_permalink(self, permalink: str) -> EntityModel:
|
||||
"""Get entity by type and name combination."""
|
||||
logger.debug(f"Getting entity by permalink: {permalink}")
|
||||
db_entity = await self.repository.get_by_permalink(permalink)
|
||||
async with self._session_scope() as session:
|
||||
db_entity = await self.repository.get_by_permalink(session, permalink)
|
||||
if not db_entity:
|
||||
raise EntityNotFoundError(f"Entity not found: {permalink}")
|
||||
return db_entity
|
||||
@@ -832,19 +875,25 @@ class EntityService(BaseService[EntityModel]):
|
||||
async def get_entities_by_id(self, ids: List[int]) -> Sequence[EntityModel]:
|
||||
"""Get specific entities and their relationships."""
|
||||
logger.debug(f"Getting entities: {ids}")
|
||||
return await self.repository.find_by_ids(ids)
|
||||
async with self._session_scope() as session:
|
||||
return await self.repository.find_by_ids(session, ids)
|
||||
|
||||
async def get_entities_by_permalinks(self, permalinks: List[str]) -> Sequence[EntityModel]:
|
||||
"""Get specific nodes and their relationships."""
|
||||
logger.debug(f"Getting entities permalinks: {permalinks}")
|
||||
return await self.repository.find_by_permalinks(permalinks)
|
||||
async with self._session_scope() as session:
|
||||
return await self.repository.find_by_permalinks(session, permalinks)
|
||||
|
||||
async def delete_entity_by_file_path(self, file_path: Union[str, Path]) -> None:
|
||||
"""Delete entity by file path."""
|
||||
await self.repository.delete_by_file_path(str(file_path))
|
||||
async with self._session_scope() as session:
|
||||
await self.repository.delete_by_file_path(session, str(file_path))
|
||||
|
||||
async def create_entity_from_markdown(
|
||||
self, file_path: Path, markdown: EntityMarkdown
|
||||
self,
|
||||
file_path: Path,
|
||||
markdown: EntityMarkdown,
|
||||
session: AsyncSession | None = None,
|
||||
) -> EntityModel:
|
||||
"""Create entity and observations only.
|
||||
|
||||
@@ -867,12 +916,13 @@ class EntityService(BaseService[EntityModel]):
|
||||
model.created_by = user_id
|
||||
model.last_updated_by = user_id
|
||||
|
||||
# Use UPSERT to handle conflicts cleanly
|
||||
try:
|
||||
return await self.repository.upsert_entity(model)
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to upsert entity for {file_path}: {e}")
|
||||
raise EntityCreationError(f"Failed to create entity: {str(e)}") from e
|
||||
async with self._session_scope(session) as active_session:
|
||||
# Use UPSERT to handle conflicts cleanly
|
||||
try:
|
||||
return await self.repository.upsert_entity(active_session, model)
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to upsert entity for {file_path}: {e}")
|
||||
raise EntityCreationError(f"Failed to create entity: {str(e)}") from e
|
||||
|
||||
async def update_entity_and_observations(
|
||||
self,
|
||||
@@ -880,6 +930,7 @@ class EntityService(BaseService[EntityModel]):
|
||||
markdown: EntityMarkdown,
|
||||
*,
|
||||
existing_entity: EntityModel | None = None,
|
||||
session: AsyncSession | None = None,
|
||||
) -> EntityModel:
|
||||
"""Update entity fields and observations.
|
||||
|
||||
@@ -888,65 +939,53 @@ class EntityService(BaseService[EntityModel]):
|
||||
"""
|
||||
logger.debug(f"Updating entity and observations: {file_path}")
|
||||
|
||||
if existing_entity is not None:
|
||||
db_entity = await self.repository.get_by_id(
|
||||
existing_entity.id,
|
||||
load_relations=False,
|
||||
async with self._session_scope(session) as active_session:
|
||||
if existing_entity is not None:
|
||||
db_entity = await self.repository.get_by_id(
|
||||
active_session,
|
||||
existing_entity.id,
|
||||
load_relations=False,
|
||||
)
|
||||
else:
|
||||
db_entity = await self.repository.get_by_file_path(
|
||||
active_session,
|
||||
file_path.as_posix(),
|
||||
load_relations=False,
|
||||
)
|
||||
if db_entity is None: # pragma: no cover
|
||||
raise EntityNotFoundError(f"Entity not found for file path: {file_path}")
|
||||
|
||||
# Observations are owned by the markdown file, so re-indexing replaces the old set.
|
||||
# We only need the entity id here; loading the old relationship collection is wasted work.
|
||||
await self.observation_repository.delete_by_fields(
|
||||
active_session, entity_id=db_entity.id
|
||||
)
|
||||
else:
|
||||
db_entity = await self.repository.get_by_file_path(
|
||||
file_path.as_posix(),
|
||||
load_relations=False,
|
||||
)
|
||||
if db_entity is None: # pragma: no cover
|
||||
raise EntityNotFoundError(f"Entity not found for file path: {file_path}")
|
||||
|
||||
# Observations are owned by the markdown file, so re-indexing replaces the old set.
|
||||
# We only need the entity id here; loading the old relationship collection is wasted work.
|
||||
await self.observation_repository.delete_by_fields(entity_id=db_entity.id)
|
||||
observations = [
|
||||
Observation(
|
||||
project_id=self.observation_repository.project_id,
|
||||
entity_id=db_entity.id,
|
||||
content=obs.content,
|
||||
category=obs.category,
|
||||
context=obs.context,
|
||||
tags=obs.tags,
|
||||
)
|
||||
for obs in markdown.observations
|
||||
]
|
||||
await self.observation_repository.add_all_no_return(active_session, observations)
|
||||
|
||||
observations = [
|
||||
Observation(
|
||||
project_id=self.observation_repository.project_id,
|
||||
entity_id=db_entity.id,
|
||||
content=obs.content,
|
||||
category=obs.category,
|
||||
context=obs.context,
|
||||
tags=obs.tags,
|
||||
)
|
||||
for obs in markdown.observations
|
||||
]
|
||||
await self.observation_repository.add_all_no_return(observations)
|
||||
self._apply_markdown_entity_fields(db_entity, file_path, markdown)
|
||||
|
||||
self._apply_markdown_entity_fields(db_entity, file_path, markdown)
|
||||
# checksum value is None == not finished with sync
|
||||
db_entity.checksum = None
|
||||
|
||||
# checksum value is None == not finished with sync
|
||||
db_entity.checksum = None
|
||||
# Set last_updated_by for cloud usage (preserve existing created_by)
|
||||
user_id = self.get_user_id()
|
||||
if user_id is not None:
|
||||
db_entity.last_updated_by = user_id
|
||||
|
||||
# Set last_updated_by for cloud usage (preserve existing created_by)
|
||||
user_id = self.get_user_id()
|
||||
if user_id is not None:
|
||||
db_entity.last_updated_by = user_id
|
||||
|
||||
entity_updates = {
|
||||
"title": db_entity.title,
|
||||
"note_type": db_entity.note_type,
|
||||
"permalink": db_entity.permalink,
|
||||
"file_path": db_entity.file_path,
|
||||
"content_type": db_entity.content_type,
|
||||
"created_at": db_entity.created_at,
|
||||
"updated_at": db_entity.updated_at,
|
||||
"entity_metadata": db_entity.entity_metadata,
|
||||
"checksum": db_entity.checksum,
|
||||
"last_updated_by": db_entity.last_updated_by,
|
||||
}
|
||||
updated = await self.repository.update_fields(
|
||||
db_entity.id,
|
||||
entity_updates,
|
||||
)
|
||||
if not updated: # pragma: no cover
|
||||
raise EntityNotFoundError(f"Entity not found for file path: {file_path}")
|
||||
return db_entity
|
||||
await active_session.flush()
|
||||
return db_entity
|
||||
|
||||
def _apply_markdown_entity_fields(
|
||||
self,
|
||||
@@ -981,23 +1020,29 @@ class EntityService(BaseService[EntityModel]):
|
||||
existing_entity: EntityModel | None = None,
|
||||
resolve_relations: bool = True,
|
||||
reload_entity: bool = True,
|
||||
session: AsyncSession | None = None,
|
||||
) -> EntityModel:
|
||||
"""Create/update entity and relations from parsed markdown."""
|
||||
if is_new:
|
||||
created = await self.create_entity_from_markdown(file_path, markdown)
|
||||
else:
|
||||
created = await self.update_entity_and_observations(
|
||||
file_path,
|
||||
async with self._session_scope(session) as active_session:
|
||||
if is_new:
|
||||
created = await self.create_entity_from_markdown(
|
||||
file_path, markdown, session=active_session
|
||||
)
|
||||
else:
|
||||
created = await self.update_entity_and_observations(
|
||||
file_path,
|
||||
markdown,
|
||||
existing_entity=existing_entity,
|
||||
session=active_session,
|
||||
)
|
||||
# Pass the entity through so relation work does not have to rediscover the source row.
|
||||
return await self.update_entity_relations(
|
||||
created,
|
||||
markdown,
|
||||
existing_entity=existing_entity,
|
||||
resolve_targets=resolve_relations,
|
||||
reload_entity=reload_entity,
|
||||
session=active_session,
|
||||
)
|
||||
# Pass the entity through so relation work does not have to rediscover the source row.
|
||||
return await self.update_entity_relations(
|
||||
created,
|
||||
markdown,
|
||||
resolve_targets=resolve_relations,
|
||||
reload_entity=reload_entity,
|
||||
)
|
||||
|
||||
async def update_entity_relations(
|
||||
self,
|
||||
@@ -1006,6 +1051,7 @@ class EntityService(BaseService[EntityModel]):
|
||||
*,
|
||||
resolve_targets: bool = True,
|
||||
reload_entity: bool = True,
|
||||
session: AsyncSession | None = None,
|
||||
) -> EntityModel:
|
||||
"""Update relations for entity.
|
||||
|
||||
@@ -1015,69 +1061,79 @@ class EntityService(BaseService[EntityModel]):
|
||||
entity_id = entity.id
|
||||
logger.debug(f"Updating relations for entity: {entity.file_path}")
|
||||
|
||||
# Clear existing relations first
|
||||
await self.relation_repository.delete_outgoing_relations_from_entity(entity_id)
|
||||
async with self._session_scope(session) as active_session:
|
||||
# Clear existing relations first
|
||||
await self.relation_repository.delete_outgoing_relations_from_entity(
|
||||
active_session, entity_id
|
||||
)
|
||||
|
||||
if markdown.relations:
|
||||
if resolve_targets:
|
||||
# Exact target resolution is useful for local sync, but expensive for cloud
|
||||
# one-file jobs. Cloud can write unresolved rows and let a relation repair pass
|
||||
# fill in to_id later.
|
||||
resolved_entities = await asyncio.gather(
|
||||
*(
|
||||
self.link_resolver.resolve_link(
|
||||
rel.target,
|
||||
strict=True,
|
||||
load_relations=False,
|
||||
if markdown.relations:
|
||||
if resolve_targets:
|
||||
# Exact target resolution is useful for local sync, but expensive for cloud
|
||||
# one-file jobs. Cloud can write unresolved rows and let a relation repair pass
|
||||
# fill in to_id later.
|
||||
resolved_entities = []
|
||||
for rel in markdown.relations:
|
||||
try:
|
||||
resolved_entities.append(
|
||||
await self.link_resolver.resolve_link(
|
||||
rel.target,
|
||||
strict=True,
|
||||
load_relations=False,
|
||||
session=active_session,
|
||||
)
|
||||
)
|
||||
except Exception as exc:
|
||||
resolved_entities.append(exc)
|
||||
else:
|
||||
resolved_entities = [None] * len(markdown.relations)
|
||||
|
||||
# Process results and create relation records
|
||||
relations_to_add = []
|
||||
for rel, resolved in zip(markdown.relations, resolved_entities):
|
||||
# Handle exceptions from gather and None results
|
||||
target_entity: Optional[Entity] = None
|
||||
if not isinstance(resolved, Exception):
|
||||
# Relation target resolution keeps exceptions as values so a failed lookup
|
||||
# becomes an unresolved forward reference instead of aborting the write.
|
||||
target_entity = resolved
|
||||
|
||||
if target_entity is None and not resolve_targets:
|
||||
target_entity = await self._resolve_deferred_self_relation(
|
||||
rel.target, entity, session=active_session
|
||||
)
|
||||
for rel in markdown.relations
|
||||
),
|
||||
return_exceptions=True,
|
||||
)
|
||||
else:
|
||||
resolved_entities = [None] * len(markdown.relations)
|
||||
|
||||
# Process results and create relation records
|
||||
relations_to_add = []
|
||||
for rel, resolved in zip(markdown.relations, resolved_entities):
|
||||
# Handle exceptions from gather and None results
|
||||
target_entity: Optional[Entity] = None
|
||||
if not isinstance(resolved, BaseException):
|
||||
# asyncio.gather(..., return_exceptions=True) can return any BaseException.
|
||||
target_entity = resolved
|
||||
# if the target is found, store the id
|
||||
target_id = target_entity.id if target_entity else None
|
||||
# if the target is found, store the title, otherwise add the target for a "forward link"
|
||||
target_name = target_entity.title if target_entity else rel.target
|
||||
|
||||
if target_entity is None and not resolve_targets:
|
||||
target_entity = await self._resolve_deferred_self_relation(rel.target, entity)
|
||||
# Create the relation
|
||||
relation = Relation(
|
||||
project_id=self.relation_repository.project_id,
|
||||
from_id=entity_id,
|
||||
to_id=target_id,
|
||||
to_name=target_name,
|
||||
relation_type=rel.type,
|
||||
context=rel.context,
|
||||
)
|
||||
relations_to_add.append(relation)
|
||||
|
||||
# if the target is found, store the id
|
||||
target_id = target_entity.id if target_entity else None
|
||||
# if the target is found, store the title, otherwise add the target for a "forward link"
|
||||
target_name = target_entity.title if target_entity else rel.target
|
||||
# Batch insert all relations
|
||||
if relations_to_add:
|
||||
await self.relation_repository.add_all_ignore_duplicates(
|
||||
active_session, relations_to_add
|
||||
)
|
||||
|
||||
# Create the relation
|
||||
relation = Relation(
|
||||
project_id=self.relation_repository.project_id,
|
||||
from_id=entity_id,
|
||||
to_id=target_id,
|
||||
to_name=target_name,
|
||||
relation_type=rel.type,
|
||||
context=rel.context,
|
||||
)
|
||||
relations_to_add.append(relation)
|
||||
if not reload_entity:
|
||||
return entity
|
||||
|
||||
# Batch insert all relations
|
||||
if relations_to_add:
|
||||
await self.relation_repository.add_all_ignore_duplicates(relations_to_add)
|
||||
|
||||
if not reload_entity:
|
||||
return entity
|
||||
|
||||
# Reload entity with relations via PK lookup (faster than get_by_file_path string match).
|
||||
reloaded = await self.repository.find_by_ids([entity_id])
|
||||
return reloaded[0]
|
||||
# Reload entity with relations via PK lookup (faster than get_by_file_path string match).
|
||||
reloaded = await self.repository.find_by_ids(active_session, [entity_id])
|
||||
return reloaded[0]
|
||||
|
||||
async def _resolve_deferred_self_relation(
|
||||
self, target: str, entity: EntityModel
|
||||
self, target: str, entity: EntityModel, session: AsyncSession | None = None
|
||||
) -> EntityModel | None:
|
||||
"""Resolve only self-relations that are safe to identify in deferred mode."""
|
||||
clean_target = target.strip()
|
||||
@@ -1101,9 +1157,12 @@ class EntityService(BaseService[EntityModel]):
|
||||
# Title-only links are ambiguous because Basic Memory allows duplicate titles.
|
||||
# Collapse them to self only when the title lookup proves this source is the sole candidate;
|
||||
# otherwise leave the relation unresolved so we do not create a wrong permanent edge.
|
||||
title_matches = await self.repository.get_by_title(clean_target, load_relations=False)
|
||||
if len(title_matches) == 1 and title_matches[0].id == entity.id:
|
||||
return entity
|
||||
async with self._session_scope(session) as active_session:
|
||||
title_matches = await self.repository.get_by_title(
|
||||
active_session, clean_target, load_relations=False
|
||||
)
|
||||
if len(title_matches) == 1 and title_matches[0].id == entity.id:
|
||||
return entity
|
||||
|
||||
return None
|
||||
|
||||
@@ -1166,43 +1225,46 @@ class EntityService(BaseService[EntityModel]):
|
||||
|
||||
file_path = Path(entity.file_path)
|
||||
current_content, _ = await self.file_service.read_file(file_path)
|
||||
# --- Prepare Against Explicit Base Content ---
|
||||
# The edit operation is the semantic step; file/DB writes below are just persistence of that
|
||||
# accepted result.
|
||||
prepared = await self.prepare_edit_entity_content(
|
||||
entity,
|
||||
current_content,
|
||||
operation=operation,
|
||||
content=content,
|
||||
section=section,
|
||||
find_text=find_text,
|
||||
expected_replacements=expected_replacements,
|
||||
)
|
||||
async with self._session_scope() as session:
|
||||
# --- Prepare Against Explicit Base Content ---
|
||||
# The edit operation is the semantic step; file/DB writes below are just persistence of that
|
||||
# accepted result.
|
||||
prepared = await self.prepare_edit_entity_content(
|
||||
entity,
|
||||
current_content,
|
||||
operation=operation,
|
||||
content=content,
|
||||
section=section,
|
||||
find_text=find_text,
|
||||
expected_replacements=expected_replacements,
|
||||
session=session,
|
||||
)
|
||||
|
||||
checksum = await self.file_service.write_file(
|
||||
file_path,
|
||||
prepared.markdown_content,
|
||||
)
|
||||
checksum = await self.file_service.write_file(
|
||||
file_path,
|
||||
prepared.markdown_content,
|
||||
)
|
||||
|
||||
# --- Rebuild Structured Knowledge State ---
|
||||
# Non-fast edits remain fully synchronous locally: once the file write succeeds, we refresh
|
||||
# observations, relations, and checksum in the same request.
|
||||
entity = await self.upsert_entity_from_markdown(
|
||||
file_path,
|
||||
prepared.entity_markdown,
|
||||
is_new=False,
|
||||
)
|
||||
# --- Rebuild Structured Knowledge State ---
|
||||
# Non-fast edits remain fully synchronous locally: once the file write succeeds, we refresh
|
||||
# observations, relations, and checksum in the same request.
|
||||
entity = await self.upsert_entity_from_markdown(
|
||||
file_path,
|
||||
prepared.entity_markdown,
|
||||
is_new=False,
|
||||
session=session,
|
||||
)
|
||||
|
||||
entity = await self.repository.update(entity.id, {"checksum": checksum})
|
||||
if not entity: # pragma: no cover
|
||||
raise ValueError(f"Failed to update entity checksum after edit: {file_path}")
|
||||
persisted_content, search_content = await self._read_persisted_write_content(file_path)
|
||||
entity = await self.repository.update(session, entity.id, {"checksum": checksum})
|
||||
if not entity: # pragma: no cover
|
||||
raise ValueError(f"Failed to update entity checksum after edit: {file_path}")
|
||||
persisted_content, search_content = await self._read_persisted_write_content(file_path)
|
||||
|
||||
return EntityWriteResult(
|
||||
entity=entity,
|
||||
content=persisted_content,
|
||||
search_content=search_content,
|
||||
)
|
||||
return EntityWriteResult(
|
||||
entity=entity,
|
||||
content=persisted_content,
|
||||
search_content=search_content,
|
||||
)
|
||||
|
||||
def apply_edit_operation(
|
||||
self,
|
||||
@@ -1528,11 +1590,12 @@ class EntityService(BaseService[EntityModel]):
|
||||
updates["checksum"] = new_checksum
|
||||
|
||||
# 9. Update database
|
||||
updated_entity = await self.repository.update(entity.id, updates)
|
||||
if not updated_entity:
|
||||
raise ValueError(f"Failed to update entity in database: {entity.id}")
|
||||
async with self._session_scope() as session:
|
||||
updated_entity = await self.repository.update(session, entity.id, updates)
|
||||
if not updated_entity:
|
||||
raise ValueError(f"Failed to update entity in database: {entity.id}")
|
||||
|
||||
return updated_entity
|
||||
return updated_entity
|
||||
|
||||
except Exception as e:
|
||||
# Rollback: try to restore original file location if move succeeded
|
||||
@@ -1581,7 +1644,8 @@ class EntityService(BaseService[EntityModel]):
|
||||
destination_directory = destination_directory.strip("/")
|
||||
|
||||
# Find all entities in the source directory
|
||||
entities = await self.repository.find_by_directory_prefix(source_directory)
|
||||
async with self._session_scope() as session:
|
||||
entities = await self.repository.find_by_directory_prefix(session, source_directory)
|
||||
|
||||
if not entities:
|
||||
logger.warning(f"No entities found in directory: {source_directory}")
|
||||
@@ -1665,7 +1729,8 @@ class EntityService(BaseService[EntityModel]):
|
||||
directory = directory.strip("/")
|
||||
|
||||
# Find all entities in the directory
|
||||
entities = await self.repository.find_by_directory_prefix(directory)
|
||||
async with self._session_scope() as session:
|
||||
entities = await self.repository.find_by_directory_prefix(session, directory)
|
||||
|
||||
if not entities:
|
||||
logger.warning(f"No entities found in directory: {directory}")
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user