mirror of
https://github.com/basicmachines-co/basic-memory
synced 2026-06-21 13:47:35 +00:00
Compare commits
32 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 5cbe1e5806 | |||
| 4791e19685 | |||
| 36848410a1 | |||
| a77b51a28e | |||
| 1a6a65571e | |||
| c8b00449d2 | |||
| 013864ebf0 | |||
| dd91b49054 | |||
| 7c96a0777d | |||
| 148e07c580 | |||
| 21334cc29b | |||
| db60942267 | |||
| 7bfac158df | |||
| 87616924ff | |||
| 5cb0502ed2 | |||
| a94a717b1b | |||
| 6e4bb72f10 | |||
| 11b0e31e24 | |||
| a5c9e77f16 | |||
| 30a89357cb | |||
| 222ec5d3b6 | |||
| d42aec7ea9 | |||
| 9809b469c6 | |||
| 76ac880f2d | |||
| ad3f2650d9 | |||
| d6508d985c | |||
| 7b95b9f37b | |||
| 0bce4be1a6 | |||
| a316424edf | |||
| af71cf4896 | |||
| e846ae85d8 | |||
| 63e4bcdf1d |
+192
-4
@@ -2,13 +2,201 @@
|
||||
|
||||
## Unreleased
|
||||
|
||||
## v0.20.2 (2026-03-10)
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
- Fix auto-update Homebrew detection: `brew outdated` exits 1 when a formula is outdated, not on error
|
||||
- Previously treated exit code 1 as a failure, causing "Automatic update check failed" instead of detecting the available update
|
||||
|
||||
## v0.20.1 (2026-03-10)
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
- **#661**: Fix `bm project list` MCP column to show transport type (stdio/https) instead of DB presence
|
||||
- Renamed "MCP (stdio)" column to "MCP"
|
||||
- Shows actual routing mode: `stdio` for local, `https` for cloud projects
|
||||
- Clears local path display for cloud-mode projects
|
||||
- **#662**: Invalidate config cache when file is modified by another process
|
||||
- Adds mtime-based cache validation to `ConfigManager.load_config()`
|
||||
- Long-lived processes (MCP stdio server) now detect external config changes
|
||||
- Fixes `bm project set-cloud` having no effect on running MCP server
|
||||
|
||||
## v0.20.0 (2026-03-10)
|
||||
|
||||
### Features
|
||||
|
||||
- **#643**: Default-on auto-update system and `bm update` command
|
||||
- Automatic background update checks for CLI installs (uv tool, Homebrew)
|
||||
- Install-source detection (homebrew, uv_tool, uvx, unknown) with uvx skip behavior
|
||||
- Periodic check gating via `auto_update_last_checked_at` + `update_check_interval` config
|
||||
- Manager-specific update flows: Homebrew (`brew upgrade`) and uv tool (`uv tool upgrade`)
|
||||
- Silent, non-blocking MCP behavior via daemon thread before server run
|
||||
- Manual commands: `bm update` (force check + apply) and `bm update --check` (check only)
|
||||
- New config fields: `auto_update`, `update_check_interval`, `auto_update_last_checked_at`
|
||||
|
||||
## v0.19.2 (2026-03-09)
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
- **#657**: Coerce string params to list/dict in MCP tools
|
||||
- MCP clients that serialize `list`/`dict` arguments as JSON strings no longer fail Pydantic validation
|
||||
- Adds `BeforeValidator` coercion to `search_notes` (`entity_types`, `note_types`, `tags`, `metadata_filters`), `write_note` (`metadata`), and `canvas` (`nodes`, `edges`)
|
||||
- **#655**: Handle SQLite and Windows semantic search regressions
|
||||
- Fix embedding status query for non-semantic SQLite databases
|
||||
- Windows-safe log file rotation with per-process log filenames
|
||||
- Robust `setup_logging` that handles all environments cleanly
|
||||
|
||||
## v0.19.1 (2026-03-08)
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
- **#649**: Enforce strict entity resolution in destructive MCP tools (`edit_note`, `move_note`, `delete_note`)
|
||||
- Prevents fuzzy-match fallback from silently editing/moving/deleting the wrong note
|
||||
- DST-related timeframe validation fix (round instead of truncate days)
|
||||
|
||||
### Features
|
||||
|
||||
- **#648**: Add `insert_before_section` and `insert_after_section` edit operations
|
||||
- Add `GET /knowledge/graph` endpoint for full graph visualization
|
||||
|
||||
### Dependencies
|
||||
|
||||
- Bump authlib from 1.6.6 to 1.6.7
|
||||
|
||||
## v0.19.0 (2026-03-07)
|
||||
|
||||
### Highlights
|
||||
|
||||
- **Semantic vector search** for SQLite and Postgres with FastEmbed embeddings
|
||||
- **Schema system** for validating and inferring knowledge base structure
|
||||
- **Per-project cloud routing** with API key authentication
|
||||
- **Upgraded to FastMCP 3.0** with tool annotations
|
||||
- **CLI overhaul** with JSON output, workspace awareness, and project dashboard
|
||||
|
||||
### Features
|
||||
|
||||
- **#550**: Add semantic vector search for SQLite and Postgres
|
||||
- FastEmbed-based embeddings with automatic backfill
|
||||
- Hybrid search combining full-text and vector similarity
|
||||
- Score-based fusion replacing RRF for better ranking
|
||||
- `min_similarity` override for tuning search precision
|
||||
- Semantic dependencies are now default, with optional extras fallback
|
||||
|
||||
- **#549**: Schema system for Basic Memory
|
||||
- `schema_infer` — infer schema from existing notes
|
||||
- `schema_validate` — validate notes against a schema definition
|
||||
- `schema_diff` — compare schemas across projects
|
||||
- Frontmatter validation support (#597)
|
||||
- Read schema definitions from file instead of stale DB metadata (#635)
|
||||
|
||||
- **#555**: Per-project local/cloud routing with API key auth
|
||||
- Individual projects route through cloud while others stay local
|
||||
- `basic-memory cloud set-key` and `basic-memory project set-cloud/set-local`
|
||||
- Stdio MCP honors per-project cloud routing (#590)
|
||||
|
||||
- **#598**: Upgrade FastMCP 2.12.3 to 3.0.1 with tool annotations
|
||||
|
||||
- **#585**: Add JSON output mode for MCP tools (default text)
|
||||
- `--json` output for CLI commands for scripting and CI
|
||||
|
||||
- **#576**: Add workspace selection flow for MCP and CLI
|
||||
- Workspace-aware cloud project listing
|
||||
- CLI refactoring for workspace support
|
||||
|
||||
- **#544**: Project-prefixed permalinks and memory URL routing
|
||||
|
||||
- **#632**: Add overwrite guard to `write_note` tool
|
||||
|
||||
- **#614**: `edit_note` append/prepend auto-creates note if not found
|
||||
|
||||
- **#609**: Richer content context in search results
|
||||
- Return matched chunk text in search results (#601)
|
||||
- Improved content hit rate
|
||||
|
||||
- **#602**: Add `created_by` and `last_updated_by` user tracking to Entity
|
||||
|
||||
- **#600**: Rename `entity_type` to `note_type` across codebase
|
||||
|
||||
- **#574**: Add `display_name` and `is_private` to ProjectItem
|
||||
|
||||
- **#569**: Expose `external_id` in EntityResponse and link resolver
|
||||
|
||||
- **#567**: Isolate default SQLite DB by config dir
|
||||
|
||||
- **#560**: Enable `default_project_mode` by default
|
||||
|
||||
- **#559**: Add `basic-memory watch` CLI command
|
||||
|
||||
- **#546**: Add cloud discovery touchpoints to CLI and MCP
|
||||
|
||||
- **#572**: CLI analytics via Umami event collector
|
||||
|
||||
- Replace project info with htop-inspired dashboard
|
||||
|
||||
- Merge `search_by_metadata` into `search_notes` with optional query
|
||||
|
||||
- Add `--strip-frontmatter` to `basic-memory tool read-note`
|
||||
- Default behavior is unchanged: `content` still includes raw markdown with frontmatter.
|
||||
- With `--strip-frontmatter`, both text and JSON modes return body-only markdown content.
|
||||
- JSON output now includes an additive `frontmatter` field with parsed YAML metadata (or `null`
|
||||
when no valid opening frontmatter block exists).
|
||||
|
||||
- Add `destination_folder` parameter to `move_note` tool
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
- **#644**: Fix default project resolution in cloud mode
|
||||
- ChatGPT search/fetch tools broken in cloud mode
|
||||
- `resolve_project_parameter` falls back to projects API
|
||||
|
||||
- **#638**: Restore API backward compatibility for v0.18.x clients
|
||||
|
||||
- **#637**: Create backup before config migration overwrites old format
|
||||
|
||||
- **#636**: `list_workspaces` bypasses factory pattern on cloud MCP server
|
||||
|
||||
- **#631**: `build_context` related_results schema validation failure
|
||||
|
||||
- **#613**: Reduce excessive log volume by demoting per-request noise to DEBUG
|
||||
|
||||
- **#612**: Handle quoted picoschema enum strings in YAML frontmatter
|
||||
|
||||
- **#607**: Guard against closed streams in promo and missing vector tables
|
||||
|
||||
- **#606**: Accept null for `expected_replacements` in `edit_note`
|
||||
|
||||
- **#595**: `recent_activity` dedup and pagination across MCP tools
|
||||
|
||||
- **#593**: Backend-specific distance-to-similarity conversion
|
||||
|
||||
- **#582**: Use LinkResolver fallback in `build_context` for flexible identifier matching
|
||||
|
||||
- **#577**: Replace RRF with score-based fusion in hybrid search
|
||||
|
||||
- **#575**: Remove hardcoded "main" default from `default_project`
|
||||
|
||||
- **#534**: Speed up `bm --version` startup
|
||||
|
||||
- Fix semantic embeddings not generated on fresh DB or upgrade
|
||||
|
||||
- Clarify `search_notes` parameter naming and fix `note_types` case sensitivity
|
||||
|
||||
- Parse `tag:` prefix at MCP tool level to avoid hybrid search failure
|
||||
|
||||
- Cap sqlite-vec knn k parameter at 4096 limit
|
||||
|
||||
- Parameterize SQL queries in search repository type filters
|
||||
|
||||
- Coerce list frontmatter values to strings for title and type fields
|
||||
|
||||
- Avoid `Post(**metadata)` crash when frontmatter contains 'content' or 'handler' keys
|
||||
|
||||
- Upgrade cryptography and python-multipart for security advisories
|
||||
|
||||
### Internal
|
||||
|
||||
- **#594**: Add `ty` as supplemental type checker
|
||||
- Batched vector sync orchestration across repositories
|
||||
- FastEmbed parallel guardrails and provider caching
|
||||
- Improved cloud CLI status and error messages
|
||||
- CI coverage and Postgres test fixes
|
||||
|
||||
## v0.18.5 (2026-02-13)
|
||||
|
||||
|
||||
@@ -23,6 +23,18 @@ Basic Memory lets you build persistent knowledge through natural conversations w
|
||||
Claude, while keeping everything in simple Markdown files on your computer. It uses the Model Context Protocol (MCP) to
|
||||
enable any compatible LLM to read and write to your local knowledge base.
|
||||
|
||||
## What's New in v0.19.0
|
||||
|
||||
- **Semantic Vector Search** — find notes by meaning, not just keywords. Combines full-text and vector similarity for hybrid search with FastEmbed embeddings.
|
||||
- **Schema System** — infer, validate, and diff the structure of your knowledge base with `schema_infer`, `schema_validate`, and `schema_diff` tools.
|
||||
- **Per-Project Cloud Routing** — route individual projects through the cloud while others stay local, using API key authentication (`basic-memory project set-cloud`).
|
||||
- **FastMCP 3.0** — upgraded to FastMCP 3.0 with tool annotations for better client integration.
|
||||
- **CLI Overhaul** — JSON output mode (`--json`) for scripting, workspace-aware commands, and an htop-inspired project dashboard.
|
||||
- **Smarter Editing** — `edit_note` append/prepend auto-creates notes if they don't exist; `write_note` has an overwrite guard to prevent accidental data loss.
|
||||
- **Richer Search Results** — matched chunk text returned in search results for better context.
|
||||
|
||||
See the full [CHANGELOG](CHANGELOG.md) for details.
|
||||
|
||||
- Website: [basicmemory.com](https://basicmemory.com?utm_source=github&utm_medium=referral&utm_campaign=readme)
|
||||
- Documentation: [docs.basicmemory.com](https://docs.basicmemory.com?utm_source=github&utm_medium=referral&utm_campaign=readme)
|
||||
- Community: [Discord](https://discord.gg/tyvKNccgqN?utm_source=github&utm_medium=referral&utm_campaign=readme)
|
||||
@@ -63,6 +75,36 @@ uv tool install basic-memory
|
||||
|
||||
You can view shared context via files in `~/basic-memory` (default directory location).
|
||||
|
||||
## Automatic Updates
|
||||
|
||||
Basic Memory includes a default-on auto-update flow for CLI installs.
|
||||
|
||||
- **Auto-install supported:** `uv tool` and Homebrew installs
|
||||
- **Default check interval:** every 24 hours (`86400` seconds)
|
||||
- **MCP-safe behavior:** update checks run silently in `basic-memory mcp` mode
|
||||
- **`uvx` behavior:** skipped (runtime is ephemeral and managed by `uvx`)
|
||||
|
||||
Manual update commands:
|
||||
|
||||
```bash
|
||||
# Check now and install if supported
|
||||
bm update
|
||||
|
||||
# Check only, do not install
|
||||
bm update --check
|
||||
```
|
||||
|
||||
Config options in `~/.basic-memory/config.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"auto_update": true,
|
||||
"update_check_interval": 86400
|
||||
}
|
||||
```
|
||||
|
||||
To disable automatic updates, set `"auto_update": false`.
|
||||
|
||||
## Why Basic Memory?
|
||||
|
||||
Most LLM interactions are ephemeral - you ask a question, get an answer, and everything is forgotten. Each conversation
|
||||
|
||||
@@ -0,0 +1,499 @@
|
||||
# Logfire Instrumentation Strategy
|
||||
|
||||
## Why
|
||||
|
||||
We want Logfire in Basic Memory for two specific use cases:
|
||||
|
||||
1. Local development and performance investigation
|
||||
2. Cloud deployments where Basic Memory runs inside Basic Memory Cloud
|
||||
|
||||
This instrumentation must be:
|
||||
|
||||
- Disabled by default
|
||||
- Useful when enabled
|
||||
- Safe for local-first users
|
||||
- Searchable in Logfire over time
|
||||
|
||||
The previous integration added telemetry, but it leaned too much on generic framework instrumentation. That created noisy spans with weak names and made the trace view harder to navigate. This strategy favors manual instrumentation around Basic Memory's real units of work.
|
||||
|
||||
## Core Principles
|
||||
|
||||
### 1. Default-off
|
||||
|
||||
Basic Memory should ship with Logfire disabled unless the operator explicitly enables it.
|
||||
|
||||
That means:
|
||||
|
||||
- no required token for normal local usage
|
||||
- no surprise outbound telemetry
|
||||
- no behavior change for existing users
|
||||
|
||||
### 2. Manual spans over automatic framework spans
|
||||
|
||||
We should not rely on broad auto-instrumentation for FastAPI, MCP, SQLAlchemy, or HTTP as the primary experience.
|
||||
|
||||
Why:
|
||||
|
||||
- auto-generated span names are often generic
|
||||
- routes and middleware produce too many low-signal spans
|
||||
- it becomes harder to answer product questions like "why was `write_note` slow?" or "where did sync time go?"
|
||||
|
||||
The preferred model is:
|
||||
|
||||
- one meaningful root span per high-level operation
|
||||
- a small number of child spans for important phases
|
||||
- optional targeted instrumentation only where it adds clear value
|
||||
|
||||
### 3. Logs must live inside traces
|
||||
|
||||
Basic Memory already uses `loguru` pervasively. The Logfire integration should preserve that and make those logs visible inside the active trace/span context.
|
||||
|
||||
If traces exist but the logs are detached from them, the integration is not doing its job.
|
||||
|
||||
### 4. Stable names, selective attributes
|
||||
|
||||
Span names should describe the operation class, not the specific input.
|
||||
|
||||
Good:
|
||||
|
||||
- `mcp.tool.write_note`
|
||||
- `sync.project.scan`
|
||||
- `search.execute`
|
||||
- `routing.resolve_project`
|
||||
|
||||
Bad:
|
||||
|
||||
- `Searching for "foo bar baz"`
|
||||
- `POST /v2/projects/123/search/`
|
||||
- `write note to /specs/api.md`
|
||||
|
||||
Dynamic values belong in attributes, not in the span name.
|
||||
|
||||
## What We Should Not Do
|
||||
|
||||
### Avoid broad FastAPI auto-instrumentation
|
||||
|
||||
We should not turn on `instrument_fastapi()` and treat that as the main telemetry story.
|
||||
|
||||
It may still be useful in narrowly scoped debugging, but it should not define the production trace shape. The meaningful root spans should come from Basic Memory's own entrypoints and service boundaries.
|
||||
|
||||
### Avoid per-file spans by default
|
||||
|
||||
`sync` can process many files. A span per file will explode trace cardinality and make performance views noisy.
|
||||
|
||||
Default behavior should be:
|
||||
|
||||
- one span for the project sync
|
||||
- child spans for scan, move handling, delete handling, markdown sync batch, relation resolution, embedding sync, watermark update
|
||||
- per-file spans only for failures or very slow outliers
|
||||
|
||||
### Avoid high-cardinality attributes on every span
|
||||
|
||||
Do not attach large or highly variable values everywhere:
|
||||
|
||||
- raw note content
|
||||
- file bodies
|
||||
- long search text
|
||||
- arbitrary metadata blobs
|
||||
- unique IDs that make every span shape distinct
|
||||
|
||||
Prefer compact, queryable attributes:
|
||||
|
||||
- `project_name`
|
||||
- `workspace_id`
|
||||
- `route_mode`
|
||||
- `scan_type`
|
||||
- `file_count`
|
||||
- `result_count`
|
||||
- `search_type`
|
||||
- `retrieval_mode`
|
||||
- `duration_ms`
|
||||
|
||||
## Proposed Architecture
|
||||
|
||||
Add a dedicated telemetry module in core Basic Memory, separate from logging setup.
|
||||
|
||||
Suggested shape:
|
||||
|
||||
```python
|
||||
# basic_memory/telemetry.py
|
||||
|
||||
def configure_telemetry(service_name: str, *, enable_logfire: bool) -> None: ...
|
||||
def telemetry_enabled() -> bool: ...
|
||||
def span(name: str, **attrs): ...
|
||||
def bind_telemetry_context(**attrs): ...
|
||||
```
|
||||
|
||||
This module should:
|
||||
|
||||
- configure Logfire only when explicitly enabled
|
||||
- set up the Logfire `loguru` handler
|
||||
- expose lightweight helpers so application code does not import `logfire` directly everywhere
|
||||
- degrade cleanly to no-op behavior when disabled
|
||||
|
||||
This keeps the rest of the codebase readable and makes it easy to reason about what telemetry is doing.
|
||||
|
||||
## Logging Integration Strategy
|
||||
|
||||
### Goal
|
||||
|
||||
When a span is active, logs emitted through `loguru` during that operation should show up in the same trace.
|
||||
|
||||
### Preferred design
|
||||
|
||||
1. Configure Logfire once in the telemetry bootstrap
|
||||
2. Add the Logfire `loguru` handler to the existing `loguru` configuration
|
||||
3. At operation boundaries, bind stable contextual fields with `loguru`
|
||||
4. Let logs emitted inside the span inherit the active trace context
|
||||
|
||||
### Context to bind
|
||||
|
||||
Bind only the fields that help correlate work across the system:
|
||||
|
||||
- `service_name`
|
||||
- `entrypoint`
|
||||
- `project_name`
|
||||
- `workspace_id`
|
||||
- `route_mode`
|
||||
- `tool_name`
|
||||
- `command_name`
|
||||
|
||||
This binding should happen at the root of an operation, not deep in leaf functions.
|
||||
|
||||
### Important nuance
|
||||
|
||||
We should not try to encode the entire trace model into logger extras. The logger context should be a human-meaningful slice of the active operation. Trace linkage comes from the active Logfire/OpenTelemetry context; logger extras are there to improve searchability and readability.
|
||||
|
||||
## Span Model
|
||||
|
||||
### Root spans
|
||||
|
||||
Each user-visible or system-visible operation should get one root span.
|
||||
|
||||
Examples:
|
||||
|
||||
- `cli.command.status`
|
||||
- `cli.command.project_sync`
|
||||
- `api.request.search`
|
||||
- `mcp.tool.write_note`
|
||||
- `mcp.tool.read_note`
|
||||
- `mcp.tool.search_notes`
|
||||
- `sync.project.run`
|
||||
- `db.semantic_backfill`
|
||||
|
||||
### Child spans
|
||||
|
||||
Child spans should represent real phases whose duration we care about.
|
||||
|
||||
Examples:
|
||||
|
||||
- `routing.client_session`
|
||||
- `routing.resolve_project`
|
||||
- `routing.resolve_workspace`
|
||||
- `api.search.execute`
|
||||
- `sync.project.scan`
|
||||
- `sync.project.detect_moves`
|
||||
- `sync.project.apply_changes`
|
||||
- `sync.project.resolve_relations`
|
||||
- `sync.project.sync_embeddings`
|
||||
- `sync.file.markdown`
|
||||
- `sync.file.regular`
|
||||
- `search.execute`
|
||||
- `search.relaxed_fts_retry`
|
||||
- `db.init`
|
||||
- `db.migrate`
|
||||
|
||||
### Span naming rules
|
||||
|
||||
- Use dot-separated names
|
||||
- Start with subsystem
|
||||
- Keep the verb at the end
|
||||
- Keep names stable across runs
|
||||
- Never include request-specific text in the span name
|
||||
|
||||
## Attribute Taxonomy
|
||||
|
||||
### Required attributes on root spans
|
||||
|
||||
Every root span should have a small common set:
|
||||
|
||||
- `service_name`
|
||||
- `entrypoint`
|
||||
- `project_name` when applicable
|
||||
- `workspace_id` when applicable
|
||||
- `route_mode` with values like `local_asgi`, `cloud_proxy`, `factory`
|
||||
|
||||
### Operation-specific attributes
|
||||
|
||||
Examples:
|
||||
|
||||
For search:
|
||||
|
||||
- `search_type`
|
||||
- `retrieval_mode`
|
||||
- `page`
|
||||
- `page_size`
|
||||
- `result_count`
|
||||
- `fallback_used`
|
||||
|
||||
For sync:
|
||||
|
||||
- `scan_type`
|
||||
- `force_full`
|
||||
- `new_count`
|
||||
- `modified_count`
|
||||
- `deleted_count`
|
||||
- `move_count`
|
||||
- `skipped_count`
|
||||
- `embeddings_enabled`
|
||||
|
||||
For note operations:
|
||||
|
||||
- `tool_name`
|
||||
- `note_type`
|
||||
- `directory`
|
||||
- `overwrite`
|
||||
- `output_format`
|
||||
|
||||
### Attributes to avoid by default
|
||||
|
||||
- full `query.text`
|
||||
- full note titles if they create privacy or cardinality issues
|
||||
- file content
|
||||
- raw frontmatter
|
||||
- raw HTTP bodies
|
||||
|
||||
If we need richer payloads for a local debugging session, that should be an explicit temporary mode, not the default telemetry shape.
|
||||
|
||||
## Instrumentation Plan By Layer
|
||||
|
||||
### 1. Entrypoints
|
||||
|
||||
Instrument these first:
|
||||
|
||||
- `cli.app` callback and major commands
|
||||
- API lifespan and selected routers
|
||||
- MCP server lifespan
|
||||
- MCP tool entrypoints
|
||||
|
||||
Why:
|
||||
|
||||
- this establishes clean root spans
|
||||
- it gives us trace boundaries that match how users think about the product
|
||||
|
||||
### 2. Routing and context resolution
|
||||
|
||||
Instrument:
|
||||
|
||||
- client routing decisions
|
||||
- workspace resolution
|
||||
- project resolution
|
||||
- default-project fallback
|
||||
|
||||
Why:
|
||||
|
||||
- Basic Memory has local/cloud/per-project routing logic
|
||||
- when something is slow or surprising, we need to know which path was taken
|
||||
|
||||
### 3. Sync and indexing
|
||||
|
||||
This is the highest-value area to instrument deeply.
|
||||
|
||||
Instrument:
|
||||
|
||||
- sync root
|
||||
- scan strategy decision
|
||||
- filesystem scan
|
||||
- move detection
|
||||
- delete handling
|
||||
- markdown sync phase
|
||||
- relation resolution
|
||||
- vector embedding sync
|
||||
- scan watermark update
|
||||
|
||||
Why:
|
||||
|
||||
- this is where performance work will happen
|
||||
- cloud and local both benefit from this visibility
|
||||
|
||||
### 4. Search
|
||||
|
||||
Instrument:
|
||||
|
||||
- search execution
|
||||
- retrieval mode
|
||||
- relaxed FTS fallback
|
||||
- result shaping
|
||||
|
||||
Why:
|
||||
|
||||
- search is user-facing and latency-sensitive
|
||||
- hybrid/vector/FTS paths need to be distinguishable
|
||||
|
||||
### 5. Database and initialization
|
||||
|
||||
Instrument selectively:
|
||||
|
||||
- DB init
|
||||
- migrations
|
||||
- semantic backfill
|
||||
- connection mode selection
|
||||
|
||||
Avoid full automatic SQL span firehose by default.
|
||||
|
||||
## Recommended Rollout Phases
|
||||
|
||||
## Task List
|
||||
|
||||
- [x] Phase 1: Bootstrap and config gating
|
||||
- [x] Phase 2: Root spans for entrypoints and primary operations
|
||||
- [x] Phase 3: Child spans for sync, search, and routing
|
||||
- [x] Phase 4: Failure-focused detail and final verification
|
||||
- [x] Phase 5: Loguru context binding and scoped context inheritance
|
||||
|
||||
## Recommended Rollout Phases
|
||||
|
||||
### Phase 1: Bootstrap and config gating
|
||||
|
||||
Add:
|
||||
|
||||
- telemetry bootstrap module
|
||||
- config/env gating
|
||||
- `loguru` + Logfire handler integration
|
||||
|
||||
This gives immediate value with low noise.
|
||||
|
||||
### Phase 2: Root spans for entrypoints and primary operations
|
||||
|
||||
Add:
|
||||
|
||||
- root spans for CLI, API, MCP, and main MCP tools
|
||||
- stable root attributes for project, workspace, route mode, and operation type
|
||||
|
||||
This gives us clean top-level traces that match how users think about the product.
|
||||
|
||||
### Phase 3: Child spans for sync, search, and routing
|
||||
|
||||
Add child spans to:
|
||||
|
||||
- sync
|
||||
- search
|
||||
- routing
|
||||
|
||||
This is the main performance-investigation layer.
|
||||
|
||||
### Phase 4: Failure-focused detail
|
||||
|
||||
Add selective deeper spans/log enrichment for:
|
||||
|
||||
- sync failures
|
||||
- relation resolution failures
|
||||
- slow file operations
|
||||
- cloud routing/auth failures
|
||||
|
||||
This keeps normal traces clean while improving debuggability.
|
||||
|
||||
### Phase 5: Loguru context binding and scoped context inheritance
|
||||
|
||||
Add:
|
||||
|
||||
- context-local telemetry state in `basic_memory.telemetry`
|
||||
- a shared `scope(...)` helper that opens a span and binds stable logger context together
|
||||
- context inheritance for routing, sync, and search so downstream `loguru` logs carry the active operation fields
|
||||
|
||||
This makes the trace view and the log stream tell the same story without forcing logger rewrites across the codebase.
|
||||
|
||||
## Local Dev Playbook
|
||||
|
||||
The fastest way to sanity-check the current trace shape is:
|
||||
|
||||
```bash
|
||||
LOGFIRE_TOKEN=lf_... just telemetry-smoke
|
||||
```
|
||||
|
||||
What this does:
|
||||
|
||||
- creates an isolated temp home, config dir, and project path
|
||||
- enables Logfire for the run
|
||||
- automatically exports to Logfire when `LOGFIRE_TOKEN` is present
|
||||
- defaults `BASIC_MEMORY_SEMANTIC_SEARCH_ENABLED=false` so the smoke run stays fast and trace-friendly
|
||||
- disables promo telemetry so the trace is about Basic Memory work, not analytics noise
|
||||
- runs a small CLI workflow:
|
||||
- `project add`
|
||||
- `tool write-note`
|
||||
- `tool read-note`
|
||||
- `tool edit-note`
|
||||
- `tool build-context`
|
||||
- `tool search-notes`
|
||||
- `doctor`
|
||||
|
||||
If you want to exercise the instrumentation without exporting anything upstream:
|
||||
|
||||
```bash
|
||||
BASIC_MEMORY_LOGFIRE_SEND_TO_LOGFIRE=false just telemetry-smoke
|
||||
```
|
||||
|
||||
If you want the smoke run to include vector or hybrid retrieval spans too:
|
||||
|
||||
```bash
|
||||
LOGFIRE_TOKEN=lf_... BASIC_MEMORY_SEMANTIC_SEARCH_ENABLED=true just telemetry-smoke
|
||||
```
|
||||
|
||||
The recipe sets `BASIC_MEMORY_LOGFIRE_ENVIRONMENT=telemetry-smoke` by default so these traces are easy to isolate in Logfire. Override it if you want the smoke traces grouped under a different environment name.
|
||||
|
||||
### What to look for
|
||||
|
||||
You should see a small set of comparable root spans rather than a framework-generated span forest:
|
||||
|
||||
- `cli.command.project`
|
||||
- `cli.command.tool`
|
||||
- `mcp.tool.write_note`
|
||||
- `mcp.tool.read_note`
|
||||
- `mcp.tool.edit_note`
|
||||
- `mcp.tool.build_context`
|
||||
- `mcp.tool.search_notes`
|
||||
- `sync.project.run`
|
||||
|
||||
You should also see correlated logs under those traces with stable fields like:
|
||||
|
||||
- `project_name`
|
||||
- `route_mode`
|
||||
- `tool_name`
|
||||
- `entrypoint`
|
||||
|
||||
### Expected nuance
|
||||
|
||||
`doctor` creates its own temporary project on purpose. That means the sync trace will usually show a different project name than the `telemetry-smoke` write/search traces. That is fine for smoke testing because the goal is to confirm:
|
||||
|
||||
- root span names are meaningful
|
||||
- scoped logs stay attached to the active trace
|
||||
- routing, tool, search, and sync phases are easy to distinguish
|
||||
|
||||
## Validation Checklist
|
||||
|
||||
We should consider the integration successful when the following are true:
|
||||
|
||||
1. With telemetry disabled, Basic Memory behaves exactly as it does today.
|
||||
2. With telemetry enabled, one user action produces one obvious root span.
|
||||
3. Logs emitted during that action are visible inside the same trace.
|
||||
4. A search in Logfire for `mcp.tool.write_note` or `sync.project.run` returns comparable spans across runs.
|
||||
5. Trace views show phase timing clearly without drowning in framework noise.
|
||||
6. Sensitive payloads are not captured by default.
|
||||
|
||||
## Immediate Implementation Direction
|
||||
|
||||
When we start coding, the first pass should be:
|
||||
|
||||
1. Add `basic_memory.telemetry`
|
||||
2. Add config/env switches for `enabled`, `send_to_logfire`, and service name
|
||||
3. Wire telemetry bootstrap into CLI, API, and MCP entrypoints
|
||||
4. Configure `loguru` to emit to both existing sinks and the Logfire handler when enabled
|
||||
5. Add manual root spans around:
|
||||
- CLI commands
|
||||
- API request handlers we care about
|
||||
- MCP tool entrypoints
|
||||
- sync root
|
||||
- search root
|
||||
6. Add child spans to the sync and routing phases only after the root span model feels clean
|
||||
|
||||
That gives us a strong foundation without repeating the earlier "turn on instrumentation everywhere" approach.
|
||||
@@ -69,24 +69,6 @@ testmon *args:
|
||||
test-smoke:
|
||||
BASIC_MEMORY_ENV=test uv run pytest -p pytest_mock -v --no-cov -m smoke test-int/mcp/test_smoke_integration.py
|
||||
|
||||
# Run graph intelligence API contract tests only
|
||||
test-graph-intel-api:
|
||||
BASIC_MEMORY_ENV=test uv run pytest -p pytest_mock -v --no-cov tests/api/v2/test_graph_intelligence_router.py
|
||||
|
||||
# Run graph intelligence MCP tests only
|
||||
test-graph-intel-mcp:
|
||||
BASIC_MEMORY_ENV=test uv run pytest -p pytest_mock -v --no-cov tests/mcp/clients/test_graph_clients.py tests/mcp/test_tool_graph_intelligence.py tests/mcp/test_tool_contracts.py
|
||||
|
||||
# Run graph intelligence CLI passthrough tests only
|
||||
test-graph-intel-cli:
|
||||
BASIC_MEMORY_ENV=test uv run pytest -p pytest_mock -v --no-cov tests/cli/test_cli_tool_graph_intelligence_json_output.py
|
||||
|
||||
# Run the full graph intelligence fast iteration slice
|
||||
test-graph-intel:
|
||||
just test-graph-intel-api
|
||||
just test-graph-intel-mcp
|
||||
just test-graph-intel-cli
|
||||
|
||||
# Fast local loop: lint, format, typecheck, impacted tests
|
||||
fast-check:
|
||||
just fix
|
||||
@@ -223,6 +205,51 @@ doctor:
|
||||
BASIC_MEMORY_CONFIG_DIR="$TMP_CONFIG" \
|
||||
./.venv/bin/python -m basic_memory.cli.main doctor --local
|
||||
|
||||
# Run an isolated Logfire smoke workflow for local trace inspection
|
||||
telemetry-smoke:
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
TMP_HOME=$(mktemp -d)
|
||||
TMP_CONFIG=$(mktemp -d)
|
||||
TMP_PROJECT=$(mktemp -d)
|
||||
export HOME="$TMP_HOME"
|
||||
export BASIC_MEMORY_ENV="${BASIC_MEMORY_ENV:-dev}"
|
||||
export BASIC_MEMORY_HOME="$TMP_PROJECT/home-root"
|
||||
export BASIC_MEMORY_CONFIG_DIR="$TMP_CONFIG"
|
||||
export BASIC_MEMORY_NO_PROMOS=1
|
||||
export BASIC_MEMORY_LOG_LEVEL="${BASIC_MEMORY_LOG_LEVEL:-INFO}"
|
||||
export BASIC_MEMORY_SEMANTIC_SEARCH_ENABLED="${BASIC_MEMORY_SEMANTIC_SEARCH_ENABLED:-false}"
|
||||
export BASIC_MEMORY_LOGFIRE_ENABLED="${BASIC_MEMORY_LOGFIRE_ENABLED:-true}"
|
||||
export BASIC_MEMORY_LOGFIRE_ENVIRONMENT="${BASIC_MEMORY_LOGFIRE_ENVIRONMENT:-telemetry-smoke}"
|
||||
if [[ -z "${BASIC_MEMORY_LOGFIRE_SEND_TO_LOGFIRE:-}" ]]; then
|
||||
if [[ -n "${LOGFIRE_TOKEN:-}" ]]; then
|
||||
export BASIC_MEMORY_LOGFIRE_SEND_TO_LOGFIRE=true
|
||||
else
|
||||
export BASIC_MEMORY_LOGFIRE_SEND_TO_LOGFIRE=false
|
||||
fi
|
||||
fi
|
||||
mkdir -p "$BASIC_MEMORY_HOME"
|
||||
echo "Telemetry smoke setup:"
|
||||
echo " logfire_enabled=$BASIC_MEMORY_LOGFIRE_ENABLED"
|
||||
echo " send_to_logfire=$BASIC_MEMORY_LOGFIRE_SEND_TO_LOGFIRE"
|
||||
echo " log_level=$BASIC_MEMORY_LOG_LEVEL"
|
||||
echo " semantic_search_enabled=$BASIC_MEMORY_SEMANTIC_SEARCH_ENABLED"
|
||||
echo " logfire_environment=$BASIC_MEMORY_LOGFIRE_ENVIRONMENT"
|
||||
echo " project_path=$TMP_PROJECT"
|
||||
./.venv/bin/python -m basic_memory.cli.main project add telemetry-smoke "$TMP_PROJECT" --default --local
|
||||
./.venv/bin/python -m basic_memory.cli.main tool write-note --title "Telemetry Smoke" --folder notes --content "hello from smoke" --project telemetry-smoke --local
|
||||
./.venv/bin/python -m basic_memory.cli.main tool read-note notes/telemetry-smoke --project telemetry-smoke --local
|
||||
./.venv/bin/python -m basic_memory.cli.main tool edit-note notes/telemetry-smoke --operation append --content $'\n\nsmoke edit line' --project telemetry-smoke --local
|
||||
./.venv/bin/python -m basic_memory.cli.main tool build-context notes/telemetry-smoke --project telemetry-smoke --local --page-size 5 --max-related 5
|
||||
./.venv/bin/python -m basic_memory.cli.main tool search-notes telemetry --project telemetry-smoke --local
|
||||
./.venv/bin/python -m basic_memory.cli.main doctor --local
|
||||
echo ""
|
||||
echo "Telemetry smoke complete."
|
||||
echo "Search Logfire for:"
|
||||
echo " service_name: basic-memory-cli"
|
||||
echo " environment: $BASIC_MEMORY_LOGFIRE_ENVIRONMENT"
|
||||
echo " span names: mcp.tool.write_note, mcp.tool.read_note, mcp.tool.edit_note, mcp.tool.build_context, mcp.tool.search_notes, sync.project.run"
|
||||
|
||||
|
||||
# Update all dependencies to latest versions
|
||||
update-deps:
|
||||
|
||||
+17
-1
@@ -54,6 +54,22 @@ Or for a one-time sync:
|
||||
basic-memory sync
|
||||
```
|
||||
|
||||
### 4. Updating Basic Memory
|
||||
|
||||
Basic Memory supports automatic updates by default for `uv tool` and Homebrew installs.
|
||||
|
||||
For manual checks and upgrades:
|
||||
|
||||
```bash
|
||||
# Check now and install if supported
|
||||
bm update
|
||||
|
||||
# Check only, do not install
|
||||
bm update --check
|
||||
```
|
||||
|
||||
To disable automatic updates, set `"auto_update": false` in `~/.basic-memory/config.json`.
|
||||
|
||||
## Configuration Options
|
||||
|
||||
### Custom Directory
|
||||
@@ -125,4 +141,4 @@ If you encounter issues:
|
||||
cat ~/.basic-memory/basic-memory.log
|
||||
```
|
||||
|
||||
For more detailed information, refer to the [full documentation](https://memory.basicmachines.co/).
|
||||
For more detailed information, refer to the [full documentation](https://docs.basicmemory.com/).
|
||||
|
||||
@@ -58,6 +58,9 @@ Documentation = "https://github.com/basicmachines-co/basic-memory#readme"
|
||||
basic-memory = "basic_memory.cli.main:app"
|
||||
bm = "basic_memory.cli.main:app"
|
||||
|
||||
[project.optional-dependencies]
|
||||
telemetry = ["logfire>=4.19.0"]
|
||||
|
||||
[build-system]
|
||||
requires = ["hatchling", "uv-dynamic-versioning>=0.7.0"]
|
||||
build-backend = "hatchling.build"
|
||||
@@ -83,6 +86,7 @@ target-version = "py312"
|
||||
|
||||
[dependency-groups]
|
||||
dev = [
|
||||
"logfire>=4.19.0",
|
||||
"gevent>=24.11.1",
|
||||
"icecream>=2.1.3",
|
||||
"pytest>=8.3.4",
|
||||
|
||||
+2
-2
@@ -6,12 +6,12 @@
|
||||
"url": "https://github.com/basicmachines-co/basic-memory.git",
|
||||
"source": "github"
|
||||
},
|
||||
"version": "0.18.5",
|
||||
"version": "0.20.2",
|
||||
"packages": [
|
||||
{
|
||||
"registryType": "pypi",
|
||||
"identifier": "basic-memory",
|
||||
"version": "0.18.5",
|
||||
"version": "0.20.2",
|
||||
"runtimeHint": "uvx",
|
||||
"runtimeArguments": [
|
||||
{"type": "positional", "value": "basic-memory"},
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
"""basic-memory - Local-first knowledge management combining Zettelkasten with knowledge graphs"""
|
||||
|
||||
# Package version - updated by release automation
|
||||
__version__ = "0.18.5"
|
||||
__version__ = "0.20.2"
|
||||
|
||||
# API version for FastAPI - independent of package version
|
||||
__api_version__ = "v0"
|
||||
|
||||
+26
-20
@@ -19,14 +19,13 @@ from basic_memory.api.v2.routers import (
|
||||
prompt_router as v2_prompt,
|
||||
importer_router as v2_importer,
|
||||
schema_router as v2_schema,
|
||||
graph_router as v2_graph,
|
||||
fcm_router as v2_fcm,
|
||||
)
|
||||
from basic_memory.api.v2.routers.project_router import (
|
||||
add_project,
|
||||
list_projects,
|
||||
synchronize_projects,
|
||||
)
|
||||
from basic_memory import telemetry
|
||||
from basic_memory.config import init_api_logging
|
||||
from basic_memory.services.exceptions import EntityAlreadyExistsError
|
||||
from basic_memory.services.initialization import initialize_app
|
||||
@@ -45,30 +44,39 @@ async def lifespan(app: FastAPI): # pragma: no cover
|
||||
set_container(container)
|
||||
app.state.container = container
|
||||
|
||||
logger.info(f"Starting Basic Memory API (mode={container.mode.name})")
|
||||
with telemetry.operation(
|
||||
"api.lifecycle.startup",
|
||||
entrypoint="api",
|
||||
mode=container.mode.name.lower(),
|
||||
):
|
||||
logger.info(f"Starting Basic Memory API (mode={container.mode.name})")
|
||||
|
||||
await initialize_app(container.config)
|
||||
await initialize_app(container.config)
|
||||
|
||||
# Cache database connections in app state for performance
|
||||
logger.info("Initializing database and caching connections...")
|
||||
engine, session_maker = await container.init_database()
|
||||
app.state.engine = engine
|
||||
app.state.session_maker = session_maker
|
||||
logger.info("Database connections cached in app state")
|
||||
# Cache database connections in app state for performance
|
||||
logger.info("Initializing database and caching connections...")
|
||||
engine, session_maker = await container.init_database()
|
||||
app.state.engine = engine
|
||||
app.state.session_maker = session_maker
|
||||
logger.info("Database connections cached in app state")
|
||||
|
||||
# Create and start sync coordinator (lifecycle centralized in coordinator)
|
||||
sync_coordinator = container.create_sync_coordinator()
|
||||
await sync_coordinator.start()
|
||||
app.state.sync_coordinator = sync_coordinator
|
||||
# Create and start sync coordinator (lifecycle centralized in coordinator)
|
||||
sync_coordinator = container.create_sync_coordinator()
|
||||
await sync_coordinator.start()
|
||||
app.state.sync_coordinator = sync_coordinator
|
||||
|
||||
# Proceed with startup
|
||||
yield
|
||||
|
||||
# Shutdown - coordinator handles clean task cancellation
|
||||
logger.info("Shutting down Basic Memory API")
|
||||
await sync_coordinator.stop()
|
||||
|
||||
await container.shutdown_database()
|
||||
with telemetry.operation(
|
||||
"api.lifecycle.shutdown",
|
||||
entrypoint="api",
|
||||
mode=container.mode.name.lower(),
|
||||
):
|
||||
logger.info("Shutting down Basic Memory API")
|
||||
await sync_coordinator.stop()
|
||||
await container.shutdown_database()
|
||||
|
||||
|
||||
# Initialize FastAPI app
|
||||
@@ -88,8 +96,6 @@ app.include_router(v2_directory, prefix="/v2/projects/{project_id}")
|
||||
app.include_router(v2_prompt, prefix="/v2/projects/{project_id}")
|
||||
app.include_router(v2_importer, prefix="/v2/projects/{project_id}")
|
||||
app.include_router(v2_schema, prefix="/v2/projects/{project_id}")
|
||||
app.include_router(v2_graph, prefix="/v2/projects/{project_id}")
|
||||
app.include_router(v2_fcm, prefix="/v2/projects/{project_id}")
|
||||
app.include_router(v2_project, prefix="/v2")
|
||||
|
||||
# Legacy web app proxy paths (compat with /proxy/projects/projects)
|
||||
|
||||
@@ -21,8 +21,6 @@ from basic_memory.api.v2.routers import (
|
||||
directory_router,
|
||||
prompt_router,
|
||||
importer_router,
|
||||
graph_router,
|
||||
fcm_router,
|
||||
)
|
||||
|
||||
__all__ = [
|
||||
@@ -34,6 +32,4 @@ __all__ = [
|
||||
"directory_router",
|
||||
"prompt_router",
|
||||
"importer_router",
|
||||
"graph_router",
|
||||
"fcm_router",
|
||||
]
|
||||
|
||||
@@ -9,8 +9,6 @@ from basic_memory.api.v2.routers.directory_router import router as directory_rou
|
||||
from basic_memory.api.v2.routers.prompt_router import router as prompt_router
|
||||
from basic_memory.api.v2.routers.importer_router import router as importer_router
|
||||
from basic_memory.api.v2.routers.schema_router import router as schema_router
|
||||
from basic_memory.api.v2.routers.graph_router import router as graph_router
|
||||
from basic_memory.api.v2.routers.fcm_router import router as fcm_router
|
||||
|
||||
__all__ = [
|
||||
"knowledge_router",
|
||||
@@ -22,6 +20,4 @@ __all__ = [
|
||||
"prompt_router",
|
||||
"importer_router",
|
||||
"schema_router",
|
||||
"graph_router",
|
||||
"fcm_router",
|
||||
]
|
||||
|
||||
@@ -1,61 +0,0 @@
|
||||
"""V2 router for FCM simulation and interop endpoints."""
|
||||
|
||||
from fastapi import APIRouter
|
||||
|
||||
from basic_memory.deps import FCMServiceV2ExternalDep, ProjectExternalIdPathDep
|
||||
from basic_memory.schemas.graph_intelligence import (
|
||||
FCMExportRequest,
|
||||
FCMExportResponse,
|
||||
FCMImportRequest,
|
||||
FCMImportResponse,
|
||||
FCMRankActionsRequest,
|
||||
FCMRankActionsResponse,
|
||||
FCMSimulateRequest,
|
||||
FCMSimulateResponse,
|
||||
)
|
||||
|
||||
router = APIRouter(prefix="/fcm", tags=["fcm-v2"])
|
||||
|
||||
|
||||
@router.post("/simulate", response_model=FCMSimulateResponse)
|
||||
async def fcm_simulate(
|
||||
request: FCMSimulateRequest,
|
||||
fcm_service: FCMServiceV2ExternalDep,
|
||||
project_id: ProjectExternalIdPathDep,
|
||||
) -> FCMSimulateResponse:
|
||||
"""Run an FCM scenario simulation."""
|
||||
_ = project_id
|
||||
return await fcm_service.simulate(request)
|
||||
|
||||
|
||||
@router.post("/rank-actions", response_model=FCMRankActionsResponse)
|
||||
async def fcm_rank_actions(
|
||||
request: FCMRankActionsRequest,
|
||||
fcm_service: FCMServiceV2ExternalDep,
|
||||
project_id: ProjectExternalIdPathDep,
|
||||
) -> FCMRankActionsResponse:
|
||||
"""Rank action candidates toward a goal."""
|
||||
_ = project_id
|
||||
return await fcm_service.rank_actions(request)
|
||||
|
||||
|
||||
@router.post("/import", response_model=FCMImportResponse)
|
||||
async def fcm_import(
|
||||
request: FCMImportRequest,
|
||||
fcm_service: FCMServiceV2ExternalDep,
|
||||
project_id: ProjectExternalIdPathDep,
|
||||
) -> FCMImportResponse:
|
||||
"""Import an FCM model using a supported interchange format."""
|
||||
_ = project_id
|
||||
return await fcm_service.import_model(request)
|
||||
|
||||
|
||||
@router.post("/export", response_model=FCMExportResponse)
|
||||
async def fcm_export(
|
||||
request: FCMExportRequest,
|
||||
fcm_service: FCMServiceV2ExternalDep,
|
||||
project_id: ProjectExternalIdPathDep,
|
||||
) -> FCMExportResponse:
|
||||
"""Export an FCM model using a supported interchange format."""
|
||||
_ = project_id
|
||||
return await fcm_service.export_model(request)
|
||||
@@ -1,71 +0,0 @@
|
||||
"""V2 router for graph intelligence endpoints."""
|
||||
|
||||
from fastapi import APIRouter, Query
|
||||
|
||||
from basic_memory.deps import (
|
||||
GraphIntelligenceServiceV2ExternalDep,
|
||||
ProjectExternalIdPathDep,
|
||||
TaskSchedulerDep,
|
||||
)
|
||||
from basic_memory.schemas.graph_intelligence import (
|
||||
GraphHealthResponse,
|
||||
GraphImpactRequest,
|
||||
GraphImpactResponse,
|
||||
GraphLineageRequest,
|
||||
GraphLineageResponse,
|
||||
GraphReindexRequest,
|
||||
GraphReindexResponse,
|
||||
)
|
||||
|
||||
router = APIRouter(prefix="/graph", tags=["graph-v2"])
|
||||
|
||||
|
||||
@router.post("/lineage", response_model=GraphLineageResponse)
|
||||
async def graph_lineage(
|
||||
request: GraphLineageRequest,
|
||||
graph_service: GraphIntelligenceServiceV2ExternalDep,
|
||||
project_id: ProjectExternalIdPathDep,
|
||||
) -> GraphLineageResponse:
|
||||
"""Build lineage paths from a start node toward an optional goal."""
|
||||
_ = project_id
|
||||
return await graph_service.lineage(request)
|
||||
|
||||
|
||||
@router.post("/impact", response_model=GraphImpactResponse)
|
||||
async def graph_impact(
|
||||
request: GraphImpactRequest,
|
||||
graph_service: GraphIntelligenceServiceV2ExternalDep,
|
||||
project_id: ProjectExternalIdPathDep,
|
||||
) -> GraphImpactResponse:
|
||||
"""Compute impact radius from a target node."""
|
||||
_ = project_id
|
||||
return await graph_service.impact(request)
|
||||
|
||||
|
||||
@router.get("/health", response_model=GraphHealthResponse)
|
||||
async def graph_health(
|
||||
graph_service: GraphIntelligenceServiceV2ExternalDep,
|
||||
project_id: ProjectExternalIdPathDep,
|
||||
scope: str | None = Query(default=None),
|
||||
timeframe: str | None = Query(default=None),
|
||||
) -> GraphHealthResponse:
|
||||
"""Report graph quality metrics and issue candidates."""
|
||||
_ = project_id
|
||||
return await graph_service.health(scope=scope, timeframe=timeframe)
|
||||
|
||||
|
||||
@router.post("/reindex", response_model=GraphReindexResponse)
|
||||
async def graph_reindex(
|
||||
request: GraphReindexRequest,
|
||||
graph_service: GraphIntelligenceServiceV2ExternalDep,
|
||||
task_scheduler: TaskSchedulerDep,
|
||||
project_id: ProjectExternalIdPathDep,
|
||||
) -> GraphReindexResponse:
|
||||
"""Queue a graph reindex operation for the current project."""
|
||||
task_scheduler.schedule(
|
||||
"reindex_graph_project",
|
||||
project_id=project_id,
|
||||
mode=request.mode,
|
||||
reason=request.reason,
|
||||
)
|
||||
return await graph_service.start_reindex_job()
|
||||
@@ -20,6 +20,7 @@ from basic_memory.deps import (
|
||||
ProjectConfigV2ExternalDep,
|
||||
AppConfigDep,
|
||||
EntityRepositoryV2ExternalDep,
|
||||
RelationRepositoryV2ExternalDep,
|
||||
ProjectExternalIdPathDep,
|
||||
TaskSchedulerDep,
|
||||
FileServiceV2ExternalDep,
|
||||
@@ -31,6 +32,9 @@ from basic_memory.schemas.v2 import (
|
||||
EntityResolveRequest,
|
||||
EntityResolveResponse,
|
||||
EntityResponseV2,
|
||||
GraphEdge,
|
||||
GraphNode,
|
||||
GraphResponse,
|
||||
MoveEntityRequestV2,
|
||||
MoveDirectoryRequestV2,
|
||||
DeleteDirectoryRequestV2,
|
||||
@@ -56,6 +60,50 @@ def _schedule_vector_sync_if_enabled(
|
||||
)
|
||||
|
||||
|
||||
## Graph endpoint
|
||||
|
||||
|
||||
@router.get("/graph", response_model=GraphResponse)
|
||||
async def get_graph(
|
||||
project_id: ProjectExternalIdPathDep,
|
||||
entity_repository: EntityRepositoryV2ExternalDep,
|
||||
relation_repository: RelationRepositoryV2ExternalDep,
|
||||
) -> GraphResponse:
|
||||
"""Return all entities and resolved relations for knowledge graph visualization.
|
||||
|
||||
Returns a flat node/edge structure optimized for rendering with graph libraries.
|
||||
Only includes resolved relations (where to_id is not null).
|
||||
"""
|
||||
logger.info("API v2 request: get_graph")
|
||||
|
||||
# Fetch all entities for this project
|
||||
entities = await entity_repository.find_all(use_load_options=False)
|
||||
nodes = [
|
||||
GraphNode(
|
||||
external_id=entity.external_id,
|
||||
title=entity.title,
|
||||
note_type=entity.note_type,
|
||||
file_path=entity.file_path,
|
||||
)
|
||||
for entity in entities
|
||||
]
|
||||
|
||||
# Fetch all resolved relations (to_id is not null) with eager-loaded entities
|
||||
relations = await relation_repository.find_all()
|
||||
edges = [
|
||||
GraphEdge(
|
||||
from_id=relation.from_entity.external_id,
|
||||
to_id=relation.to_entity.external_id,
|
||||
relation_type=relation.relation_type,
|
||||
)
|
||||
for relation in relations
|
||||
if relation.to_entity is not None
|
||||
]
|
||||
|
||||
logger.info(f"API v2 response: graph with {len(nodes)} nodes and {len(edges)} edges")
|
||||
return GraphResponse(nodes=nodes, edges=edges)
|
||||
|
||||
|
||||
## Resolution endpoint
|
||||
|
||||
|
||||
|
||||
@@ -6,6 +6,7 @@ V1 uses string-based project names which are less efficient and less stable.
|
||||
|
||||
from fastapi import APIRouter, HTTPException, Path
|
||||
|
||||
from basic_memory import telemetry
|
||||
from basic_memory.api.v2.utils import to_search_results
|
||||
from basic_memory.repository.semantic_errors import (
|
||||
SemanticDependenciesMissingError,
|
||||
@@ -47,29 +48,39 @@ async def search(
|
||||
Returns:
|
||||
SearchResponse with paginated search results
|
||||
"""
|
||||
offset = (page - 1) * page_size
|
||||
# Fetch one extra item to detect whether more pages exist (N+1 trick)
|
||||
fetch_limit = page_size + 1
|
||||
try:
|
||||
results = await search_service.search(query, limit=fetch_limit, offset=offset)
|
||||
except SemanticSearchDisabledError as exc:
|
||||
raise HTTPException(status_code=400, detail=str(exc)) from exc
|
||||
except SemanticDependenciesMissingError as exc:
|
||||
raise HTTPException(status_code=400, detail=str(exc)) from exc
|
||||
except ValueError as exc:
|
||||
raise HTTPException(status_code=400, detail=str(exc)) from exc
|
||||
|
||||
has_more = len(results) > page_size
|
||||
if has_more:
|
||||
results = results[:page_size]
|
||||
|
||||
search_results = await to_search_results(entity_service, results)
|
||||
return SearchResponse(
|
||||
results=search_results,
|
||||
current_page=page,
|
||||
with telemetry.operation(
|
||||
"api.request.search",
|
||||
entrypoint="api",
|
||||
page=page,
|
||||
page_size=page_size,
|
||||
has_more=has_more,
|
||||
)
|
||||
retrieval_mode=query.retrieval_mode.value,
|
||||
has_text_query=bool(query.text and query.text.strip()),
|
||||
has_title_query=bool(query.title),
|
||||
has_permalink_query=bool(query.permalink or query.permalink_match),
|
||||
):
|
||||
offset = (page - 1) * page_size
|
||||
# Fetch one extra item to detect whether more pages exist (N+1 trick)
|
||||
fetch_limit = page_size + 1
|
||||
try:
|
||||
results = await search_service.search(query, limit=fetch_limit, offset=offset)
|
||||
except SemanticSearchDisabledError as exc:
|
||||
raise HTTPException(status_code=400, detail=str(exc)) from exc
|
||||
except SemanticDependenciesMissingError as exc:
|
||||
raise HTTPException(status_code=400, detail=str(exc)) from exc
|
||||
except ValueError as exc:
|
||||
raise HTTPException(status_code=400, detail=str(exc)) from exc
|
||||
|
||||
has_more = len(results) > page_size
|
||||
if has_more:
|
||||
results = results[:page_size]
|
||||
|
||||
search_results = await to_search_results(entity_service, results)
|
||||
return SearchResponse(
|
||||
results=search_results,
|
||||
current_page=page,
|
||||
page_size=page_size,
|
||||
has_more=has_more,
|
||||
)
|
||||
|
||||
|
||||
@router.post("/search/reindex")
|
||||
|
||||
@@ -8,9 +8,11 @@ from typing import Optional # noqa: E402
|
||||
|
||||
import typer # noqa: E402
|
||||
|
||||
from basic_memory.cli.auto_update import maybe_run_periodic_auto_update # noqa: E402
|
||||
from basic_memory.cli.container import CliContainer, set_container # noqa: E402
|
||||
from basic_memory.cli.promo import maybe_show_cloud_promo, maybe_show_init_line # noqa: E402
|
||||
from basic_memory.config import init_cli_logging # noqa: E402
|
||||
from basic_memory import telemetry # noqa: E402
|
||||
|
||||
|
||||
def version_callback(value: bool) -> None:
|
||||
@@ -41,6 +43,14 @@ def app_callback(
|
||||
|
||||
# Initialize logging for CLI (file only, no stdout)
|
||||
init_cli_logging()
|
||||
command_name = ctx.invoked_subcommand or "root"
|
||||
ctx.with_resource(
|
||||
telemetry.operation(
|
||||
f"cli.command.{command_name}",
|
||||
entrypoint="cli",
|
||||
command_name=command_name,
|
||||
)
|
||||
)
|
||||
|
||||
# --- Composition Root ---
|
||||
# Create container and read config (single point of config access)
|
||||
@@ -52,10 +62,14 @@ def app_callback(
|
||||
# Outcome: one-time plain line printed before the subcommand runs.
|
||||
maybe_show_init_line(ctx.invoked_subcommand)
|
||||
|
||||
# Trigger: register promo as a post-command callback.
|
||||
# Why: promo output should appear after the command's own output, not before.
|
||||
# Outcome: promo panel renders below the command results (status tree, table, etc.).
|
||||
ctx.call_on_close(lambda: maybe_show_cloud_promo(ctx.invoked_subcommand))
|
||||
# Trigger: register post-command messaging callbacks.
|
||||
# Why: informational/promo/update output belongs below command results.
|
||||
# Outcome: command output remains primary, with optional follow-up notices afterwards.
|
||||
def _post_command_messages() -> None:
|
||||
maybe_show_cloud_promo(ctx.invoked_subcommand)
|
||||
maybe_run_periodic_auto_update(ctx.invoked_subcommand)
|
||||
|
||||
ctx.call_on_close(_post_command_messages)
|
||||
|
||||
# Run initialization for commands that don't use the API
|
||||
# Skip for 'mcp' command - it has its own lifespan that handles initialization
|
||||
@@ -70,6 +84,7 @@ def app_callback(
|
||||
"tool",
|
||||
"reset",
|
||||
"reindex",
|
||||
"update",
|
||||
"watch",
|
||||
}
|
||||
if (
|
||||
|
||||
@@ -0,0 +1,389 @@
|
||||
"""Automatic update checks and upgrades for the Basic Memory CLI."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import subprocess
|
||||
import sys
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
from dataclasses import dataclass
|
||||
from datetime import datetime, timedelta
|
||||
from enum import Enum
|
||||
|
||||
from loguru import logger
|
||||
from packaging.version import InvalidVersion, Version
|
||||
from rich.console import Console
|
||||
|
||||
import basic_memory
|
||||
from basic_memory.config import ConfigManager
|
||||
|
||||
PACKAGE_NAME = "basic-memory"
|
||||
PYPI_JSON_URL = "https://pypi.org/pypi/basic-memory/json"
|
||||
|
||||
PYPI_TIMEOUT_SECONDS = 5
|
||||
BREW_OUTDATED_TIMEOUT_SECONDS = 60
|
||||
UV_UPGRADE_TIMEOUT_SECONDS = 180
|
||||
BREW_UPGRADE_TIMEOUT_SECONDS = 600
|
||||
|
||||
|
||||
class InstallSource(str, Enum):
|
||||
"""How the running CLI appears to have been installed."""
|
||||
|
||||
HOMEBREW = "homebrew"
|
||||
UV_TOOL = "uv_tool"
|
||||
UVX = "uvx"
|
||||
UNKNOWN = "unknown"
|
||||
|
||||
|
||||
class AutoUpdateStatus(str, Enum):
|
||||
"""Result classification for update checks and installs."""
|
||||
|
||||
SKIPPED = "skipped"
|
||||
UP_TO_DATE = "up_to_date"
|
||||
UPDATE_AVAILABLE = "update_available"
|
||||
UPDATED = "updated"
|
||||
FAILED = "failed"
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class AutoUpdateResult:
|
||||
"""Structured result for update checks/install attempts."""
|
||||
|
||||
status: AutoUpdateStatus
|
||||
source: InstallSource
|
||||
checked: bool
|
||||
update_available: bool
|
||||
updated: bool
|
||||
latest_version: str | None = None
|
||||
message: str | None = None
|
||||
error: str | None = None
|
||||
restart_recommended: bool = False
|
||||
|
||||
|
||||
def detect_install_source(executable: str | None = None) -> InstallSource:
|
||||
"""Infer installation source from the active interpreter path."""
|
||||
active_executable = executable or sys.executable
|
||||
normalized = active_executable.lower().replace("\\", "/")
|
||||
|
||||
if "cellar/basic-memory" in normalized:
|
||||
return InstallSource.HOMEBREW
|
||||
if "uv/tools/basic-memory" in normalized:
|
||||
return InstallSource.UV_TOOL
|
||||
if "/uv/archive-" in normalized:
|
||||
return InstallSource.UVX
|
||||
return InstallSource.UNKNOWN
|
||||
|
||||
|
||||
def _is_interactive_session() -> bool:
|
||||
"""Return whether stdin/stdout are interactive terminals."""
|
||||
try:
|
||||
return sys.stdin.isatty() and sys.stdout.isatty()
|
||||
except ValueError:
|
||||
# Trigger: stdin/stdout may be closed during transport teardown.
|
||||
# Why: isatty() raises ValueError on closed descriptors.
|
||||
# Outcome: treat as non-interactive and suppress periodic output.
|
||||
return False
|
||||
|
||||
|
||||
def _run_subprocess(
|
||||
command: list[str],
|
||||
*,
|
||||
timeout_seconds: int,
|
||||
silent: bool,
|
||||
capture_output: bool,
|
||||
) -> subprocess.CompletedProcess[str]:
|
||||
"""Run a subprocess with explicit stdio behavior for protocol safety."""
|
||||
# Trigger: silent operation (MCP/background) with no need for subprocess output.
|
||||
# Why: prevent protocol/terminal pollution from child process output.
|
||||
# Outcome: stdout/stderr are discarded unless explicit capture is requested.
|
||||
use_devnull = silent and not capture_output
|
||||
stdout_target = subprocess.DEVNULL if use_devnull else subprocess.PIPE
|
||||
stderr_target = subprocess.DEVNULL if use_devnull else subprocess.PIPE
|
||||
|
||||
return subprocess.run(
|
||||
command,
|
||||
stdin=subprocess.DEVNULL,
|
||||
stdout=stdout_target,
|
||||
stderr=stderr_target,
|
||||
text=True,
|
||||
timeout=timeout_seconds,
|
||||
check=False,
|
||||
)
|
||||
|
||||
|
||||
def _version_from_pypi() -> str:
|
||||
"""Fetch the latest published package version from PyPI."""
|
||||
request = urllib.request.Request(
|
||||
PYPI_JSON_URL,
|
||||
headers={"User-Agent": f"basic-memory-cli/{basic_memory.__version__}"},
|
||||
)
|
||||
with urllib.request.urlopen(request, timeout=PYPI_TIMEOUT_SECONDS) as response:
|
||||
payload = json.loads(response.read().decode("utf-8"))
|
||||
latest = payload.get("info", {}).get("version")
|
||||
if not latest:
|
||||
raise RuntimeError("PyPI JSON response did not include info.version")
|
||||
return str(latest)
|
||||
|
||||
|
||||
def _check_homebrew_update_available(silent: bool) -> tuple[bool, str | None]:
|
||||
"""Check whether Homebrew reports an outdated basic-memory formula."""
|
||||
result = _run_subprocess(
|
||||
["brew", "outdated", "--quiet", PACKAGE_NAME],
|
||||
timeout_seconds=BREW_OUTDATED_TIMEOUT_SECONDS,
|
||||
silent=silent,
|
||||
capture_output=True,
|
||||
)
|
||||
# Trigger: brew outdated exits 1 when the formula IS outdated (with name on stdout).
|
||||
# Why: non-zero exit here means "outdated", not "error".
|
||||
# Outcome: check stdout for the package name to determine outdated status.
|
||||
stdout = (result.stdout or "").strip()
|
||||
is_outdated = PACKAGE_NAME in stdout
|
||||
return is_outdated, None
|
||||
|
||||
|
||||
def _check_pypi_update_available() -> tuple[bool, str]:
|
||||
"""Compare installed package version with PyPI latest version."""
|
||||
latest = _version_from_pypi()
|
||||
try:
|
||||
current_version = Version(basic_memory.__version__)
|
||||
latest_version = Version(latest)
|
||||
except InvalidVersion as exc:
|
||||
raise RuntimeError(
|
||||
f"Could not compare versions (current={basic_memory.__version__}, latest={latest})"
|
||||
) from exc
|
||||
|
||||
return latest_version > current_version, latest
|
||||
|
||||
|
||||
def _manual_update_hint(source: InstallSource) -> str:
|
||||
"""Return manager-appropriate manual update instructions."""
|
||||
if source == InstallSource.UV_TOOL:
|
||||
return "Run `uv tool upgrade basic-memory`."
|
||||
if source == InstallSource.HOMEBREW:
|
||||
return "Run `brew upgrade basic-memory`."
|
||||
return (
|
||||
"Automatic install is not supported for this environment. "
|
||||
"Update with your package manager (for pip: `python3 -m pip install -U basic-memory`)."
|
||||
)
|
||||
|
||||
|
||||
def _save_last_checked_timestamp(config_manager: ConfigManager, checked_at: datetime) -> None:
|
||||
"""Persist the timestamp for the most recent attempted update check."""
|
||||
config = config_manager.load_config()
|
||||
config.auto_update_last_checked_at = checked_at
|
||||
config_manager.save_config(config)
|
||||
|
||||
|
||||
def run_auto_update(
|
||||
*,
|
||||
force: bool = False,
|
||||
check_only: bool = False,
|
||||
silent: bool = False,
|
||||
config_manager: ConfigManager | None = None,
|
||||
now: datetime | None = None,
|
||||
executable: str | None = None,
|
||||
) -> AutoUpdateResult:
|
||||
"""Run update check/install flow and return a structured result."""
|
||||
manager = config_manager or ConfigManager()
|
||||
config = manager.load_config()
|
||||
source = detect_install_source(executable)
|
||||
checked_at = now or datetime.now()
|
||||
|
||||
if source == InstallSource.UVX:
|
||||
return AutoUpdateResult(
|
||||
status=AutoUpdateStatus.SKIPPED,
|
||||
source=source,
|
||||
checked=False,
|
||||
update_available=False,
|
||||
updated=False,
|
||||
message="uvx runtime detected; updates are managed by uvx cache resolution.",
|
||||
)
|
||||
|
||||
if not force and not config.auto_update:
|
||||
return AutoUpdateResult(
|
||||
status=AutoUpdateStatus.SKIPPED,
|
||||
source=source,
|
||||
checked=False,
|
||||
update_available=False,
|
||||
updated=False,
|
||||
message="Auto-update is disabled in config.",
|
||||
)
|
||||
|
||||
if not force and config.auto_update_last_checked_at is not None:
|
||||
try:
|
||||
elapsed = checked_at - config.auto_update_last_checked_at
|
||||
except TypeError:
|
||||
# Trigger: mixed naive/aware datetimes from manual config edits.
|
||||
# Why: datetime subtraction fails for mixed tz-awareness.
|
||||
# Outcome: ignore the gate once and continue with a forced check path.
|
||||
logger.warning("Auto-update interval gate skipped due to incompatible timestamp format")
|
||||
else:
|
||||
if elapsed < timedelta(seconds=config.update_check_interval):
|
||||
return AutoUpdateResult(
|
||||
status=AutoUpdateStatus.SKIPPED,
|
||||
source=source,
|
||||
checked=False,
|
||||
update_available=False,
|
||||
updated=False,
|
||||
message="Update check interval has not elapsed.",
|
||||
)
|
||||
|
||||
try:
|
||||
# --- Availability check ---
|
||||
latest_version: str | None = None
|
||||
if source == InstallSource.HOMEBREW:
|
||||
update_available, latest_version = _check_homebrew_update_available(silent=silent)
|
||||
else:
|
||||
update_available, latest_version = _check_pypi_update_available()
|
||||
|
||||
if not update_available:
|
||||
return AutoUpdateResult(
|
||||
status=AutoUpdateStatus.UP_TO_DATE,
|
||||
source=source,
|
||||
checked=True,
|
||||
update_available=False,
|
||||
updated=False,
|
||||
latest_version=latest_version,
|
||||
message=f"Basic Memory is up to date ({basic_memory.__version__}).",
|
||||
)
|
||||
|
||||
if check_only:
|
||||
return AutoUpdateResult(
|
||||
status=AutoUpdateStatus.UPDATE_AVAILABLE,
|
||||
source=source,
|
||||
checked=True,
|
||||
update_available=True,
|
||||
updated=False,
|
||||
latest_version=latest_version,
|
||||
message=(
|
||||
f"Update available (latest: {latest_version or 'unknown'}). "
|
||||
f"{_manual_update_hint(source)}"
|
||||
),
|
||||
)
|
||||
|
||||
if source == InstallSource.UNKNOWN:
|
||||
return AutoUpdateResult(
|
||||
status=AutoUpdateStatus.UPDATE_AVAILABLE,
|
||||
source=source,
|
||||
checked=True,
|
||||
update_available=True,
|
||||
updated=False,
|
||||
latest_version=latest_version,
|
||||
message=(
|
||||
f"Update available (latest: {latest_version or 'unknown'}). "
|
||||
f"{_manual_update_hint(source)}"
|
||||
),
|
||||
)
|
||||
|
||||
# --- Automatic install ---
|
||||
command = (
|
||||
["uv", "tool", "upgrade", PACKAGE_NAME]
|
||||
if source == InstallSource.UV_TOOL
|
||||
else ["brew", "upgrade", PACKAGE_NAME]
|
||||
)
|
||||
timeout = (
|
||||
UV_UPGRADE_TIMEOUT_SECONDS
|
||||
if source == InstallSource.UV_TOOL
|
||||
else BREW_UPGRADE_TIMEOUT_SECONDS
|
||||
)
|
||||
|
||||
install_result = _run_subprocess(
|
||||
command,
|
||||
timeout_seconds=timeout,
|
||||
silent=silent,
|
||||
capture_output=not silent,
|
||||
)
|
||||
if install_result.returncode != 0:
|
||||
stderr = (install_result.stderr or "").strip() if install_result.stderr else ""
|
||||
stdout = (install_result.stdout or "").strip() if install_result.stdout else ""
|
||||
detail = stderr or stdout or "update command failed"
|
||||
return AutoUpdateResult(
|
||||
status=AutoUpdateStatus.FAILED,
|
||||
source=source,
|
||||
checked=True,
|
||||
update_available=True,
|
||||
updated=False,
|
||||
latest_version=latest_version,
|
||||
message="Automatic update failed.",
|
||||
error=detail,
|
||||
)
|
||||
|
||||
return AutoUpdateResult(
|
||||
status=AutoUpdateStatus.UPDATED,
|
||||
source=source,
|
||||
checked=True,
|
||||
update_available=True,
|
||||
updated=True,
|
||||
latest_version=latest_version,
|
||||
message=(
|
||||
"Basic Memory was updated successfully. "
|
||||
"Restart running sessions to use the new version."
|
||||
),
|
||||
restart_recommended=True,
|
||||
)
|
||||
|
||||
except (
|
||||
RuntimeError,
|
||||
urllib.error.URLError,
|
||||
ValueError,
|
||||
TimeoutError,
|
||||
subprocess.SubprocessError,
|
||||
OSError,
|
||||
) as exc:
|
||||
logger.warning(f"Auto-update check failed: {exc}")
|
||||
return AutoUpdateResult(
|
||||
status=AutoUpdateStatus.FAILED,
|
||||
source=source,
|
||||
checked=True,
|
||||
update_available=False,
|
||||
updated=False,
|
||||
message="Automatic update check failed.",
|
||||
error=str(exc),
|
||||
)
|
||||
finally:
|
||||
# Trigger: we attempted a check path (including failures).
|
||||
# Why: repeated failing checks on every command create noise and unnecessary network load.
|
||||
# Outcome: next periodic check is gated by update_check_interval.
|
||||
try:
|
||||
_save_last_checked_timestamp(manager, checked_at)
|
||||
except Exception as exc: # pragma: no cover
|
||||
logger.warning(f"Failed to persist auto-update timestamp: {exc}")
|
||||
|
||||
|
||||
def maybe_run_periodic_auto_update(
|
||||
invoked_subcommand: str | None,
|
||||
*,
|
||||
config_manager: ConfigManager | None = None,
|
||||
is_interactive: bool | None = None,
|
||||
console: Console | None = None,
|
||||
) -> AutoUpdateResult | None:
|
||||
"""Run a periodic auto-update check for interactive CLI sessions."""
|
||||
interactive = _is_interactive_session() if is_interactive is None else is_interactive
|
||||
if not interactive:
|
||||
return None
|
||||
if invoked_subcommand in {None, "mcp", "update"}:
|
||||
return None
|
||||
|
||||
result = run_auto_update(
|
||||
force=False,
|
||||
check_only=False,
|
||||
silent=False,
|
||||
config_manager=config_manager,
|
||||
)
|
||||
|
||||
if result.status in {
|
||||
AutoUpdateStatus.UPDATE_AVAILABLE,
|
||||
AutoUpdateStatus.UPDATED,
|
||||
AutoUpdateStatus.FAILED,
|
||||
}:
|
||||
out = console or Console()
|
||||
if result.status == AutoUpdateStatus.UPDATED:
|
||||
out.print(f"[green]{result.message}[/green]")
|
||||
elif result.status == AutoUpdateStatus.FAILED:
|
||||
error_detail = f" {result.error}" if result.error else ""
|
||||
out.print(f"[yellow]{result.message}{error_detail}[/yellow]")
|
||||
elif result.message:
|
||||
out.print(f"[cyan]{result.message}[/cyan]")
|
||||
|
||||
return result
|
||||
@@ -8,6 +8,7 @@ from . import (
|
||||
project,
|
||||
format,
|
||||
schema,
|
||||
update,
|
||||
)
|
||||
|
||||
__all__ = [
|
||||
@@ -23,4 +24,5 @@ __all__ = [
|
||||
"project",
|
||||
"format",
|
||||
"schema",
|
||||
"update",
|
||||
]
|
||||
|
||||
@@ -54,6 +54,9 @@ async def run_doctor() -> None:
|
||||
if not status.new_project:
|
||||
raise ValueError("Failed to create doctor project")
|
||||
project_id = status.new_project.external_id
|
||||
# Use the resolved path from the server — when project_root is configured,
|
||||
# the actual project directory differs from the requested temp_path
|
||||
project_path = Path(status.new_project.path)
|
||||
console.print(f"[green]OK[/green] Created doctor project: {project_name}")
|
||||
|
||||
# --- DB -> File: create an entity via API ---
|
||||
@@ -68,7 +71,7 @@ async def run_doctor() -> None:
|
||||
)
|
||||
api_result = await knowledge_client.create_entity(api_note.model_dump(), fast=False)
|
||||
|
||||
api_file = temp_path / api_result.file_path
|
||||
api_file = project_path / api_result.file_path
|
||||
if not api_file.exists():
|
||||
raise ValueError(f"API note file missing: {api_result.file_path}")
|
||||
|
||||
@@ -79,7 +82,7 @@ async def run_doctor() -> None:
|
||||
console.print("[green]OK[/green] API write created file")
|
||||
|
||||
# --- File -> DB: write markdown file directly, then sync ---
|
||||
parser = EntityParser(temp_path)
|
||||
parser = EntityParser(project_path)
|
||||
processor = MarkdownProcessor(parser)
|
||||
manual_markdown = EntityMarkdown(
|
||||
frontmatter=EntityFrontmatter(
|
||||
@@ -93,7 +96,7 @@ async def run_doctor() -> None:
|
||||
content=f"# {manual_note_title}\n\n- [note] File to DB check",
|
||||
)
|
||||
|
||||
manual_path = temp_path / "doctor" / "manual-note.md"
|
||||
manual_path = project_path / "doctor" / "manual-note.md"
|
||||
await processor.write_file(manual_path, manual_markdown)
|
||||
console.print("[green]OK[/green] Manual file written")
|
||||
|
||||
|
||||
@@ -1,12 +1,14 @@
|
||||
"""MCP server command with streamable HTTP transport."""
|
||||
|
||||
import os
|
||||
import threading
|
||||
from typing import Any, Optional
|
||||
|
||||
import typer
|
||||
from loguru import logger
|
||||
|
||||
from basic_memory.cli.app import app
|
||||
from basic_memory.cli.auto_update import AutoUpdateStatus, run_auto_update
|
||||
from basic_memory.config import ConfigManager, init_mcp_logging
|
||||
|
||||
|
||||
@@ -80,6 +82,22 @@ def mcp(
|
||||
os.environ["BASIC_MEMORY_MCP_PROJECT"] = project_name
|
||||
logger.info(f"MCP server constrained to project: {project_name}")
|
||||
|
||||
def _run_background_auto_update() -> None:
|
||||
result = run_auto_update(force=False, check_only=False, silent=True)
|
||||
if result.restart_recommended:
|
||||
logger.info(
|
||||
"A newer Basic Memory version was installed and will apply on next restart."
|
||||
)
|
||||
elif result.status == AutoUpdateStatus.FAILED and result.error:
|
||||
logger.warning(f"MCP background auto-update failed: {result.error}")
|
||||
|
||||
# Trigger: stdio transport corresponds to local user installs.
|
||||
# Why: server transports (HTTP/SSE) run in managed environments where
|
||||
# package-manager self-upgrades are inappropriate.
|
||||
# Outcome: background auto-update runs only for local stdio MCP sessions.
|
||||
if transport == "stdio":
|
||||
threading.Thread(target=_run_background_auto_update, daemon=True).start()
|
||||
|
||||
# Run the MCP server (blocks)
|
||||
# Lifespan handles: initialization, migrations, file sync, cleanup
|
||||
logger.info(f"Starting MCP server with {transport.upper()} transport")
|
||||
|
||||
@@ -128,7 +128,7 @@ def list_projects(
|
||||
table.add_column("Cloud Path", style="green")
|
||||
table.add_column("Workspace", style="green")
|
||||
table.add_column("CLI Route", style="blue")
|
||||
table.add_column("MCP (stdio)", style="blue")
|
||||
table.add_column("MCP", style="blue")
|
||||
table.add_column("Sync", style="green")
|
||||
table.add_column("Default", style="magenta")
|
||||
|
||||
@@ -164,6 +164,11 @@ def list_projects(
|
||||
elif entry and entry.mode == ProjectMode.LOCAL and entry.path:
|
||||
local_path = format_path(normalize_project_path(entry.path))
|
||||
|
||||
# Clear local path for cloud-mode projects — only local projects
|
||||
# should display a local path
|
||||
if entry and entry.mode == ProjectMode.CLOUD:
|
||||
local_path = ""
|
||||
|
||||
cloud_path = ""
|
||||
if cloud_project is not None:
|
||||
cloud_path = normalize_project_path(cloud_project.path)
|
||||
@@ -182,7 +187,13 @@ def list_projects(
|
||||
is_default = config.default_project == project_name
|
||||
|
||||
has_sync = bool(entry and entry.local_sync_path)
|
||||
mcp_stdio_target = "local" if local_project is not None else "n/a"
|
||||
# Determine MCP transport based on project routing mode
|
||||
if entry and entry.mode == ProjectMode.CLOUD:
|
||||
mcp_transport = "https"
|
||||
elif entry is None and cloud_project is not None:
|
||||
mcp_transport = "https"
|
||||
else:
|
||||
mcp_transport = "stdio"
|
||||
|
||||
# Show workspace name (type) for cloud-sourced projects
|
||||
ws_label = ""
|
||||
@@ -195,7 +206,7 @@ def list_projects(
|
||||
"local_path": local_path,
|
||||
"cloud_path": cloud_path,
|
||||
"cli_route": cli_route,
|
||||
"mcp_stdio": mcp_stdio_target,
|
||||
"mcp_stdio": mcp_transport,
|
||||
"sync": has_sync,
|
||||
"is_default": is_default,
|
||||
}
|
||||
|
||||
@@ -16,13 +16,6 @@ from basic_memory.cli.commands.command_utils import run_with_cleanup
|
||||
from basic_memory.cli.commands.routing import force_routing, validate_routing_flags
|
||||
from basic_memory.mcp.tools import build_context as mcp_build_context
|
||||
from basic_memory.mcp.tools import edit_note as mcp_edit_note
|
||||
from basic_memory.mcp.tools import fcm_export_model as mcp_fcm_export_model
|
||||
from basic_memory.mcp.tools import fcm_import_model as mcp_fcm_import_model
|
||||
from basic_memory.mcp.tools import fcm_rank_actions as mcp_fcm_rank_actions
|
||||
from basic_memory.mcp.tools import fcm_simulate as mcp_fcm_simulate
|
||||
from basic_memory.mcp.tools import graph_health as mcp_graph_health
|
||||
from basic_memory.mcp.tools import graph_impact as mcp_graph_impact
|
||||
from basic_memory.mcp.tools import graph_lineage as mcp_graph_lineage
|
||||
from basic_memory.mcp.tools import list_memory_projects as mcp_list_projects
|
||||
from basic_memory.mcp.tools import list_workspaces as mcp_list_workspaces
|
||||
from basic_memory.mcp.tools import read_note as mcp_read_note
|
||||
@@ -47,17 +40,6 @@ def _print_json(result: Any) -> None:
|
||||
print(json.dumps(result, indent=2, ensure_ascii=True, default=str))
|
||||
|
||||
|
||||
def _parse_json_option(raw_value: Optional[str], option_name: str) -> Any:
|
||||
"""Parse a JSON CLI option with deterministic error handling."""
|
||||
if raw_value is None:
|
||||
return None
|
||||
try:
|
||||
return json.loads(raw_value)
|
||||
except json.JSONDecodeError as exc:
|
||||
typer.echo(f"Invalid JSON for {option_name}: {exc}", err=True)
|
||||
raise typer.Exit(1)
|
||||
|
||||
|
||||
# --- Commands ---
|
||||
|
||||
|
||||
@@ -384,372 +366,6 @@ def recent_activity(
|
||||
raise
|
||||
|
||||
|
||||
@tool_app.command("graph-lineage")
|
||||
def graph_lineage(
|
||||
start: Annotated[str, typer.Argument(help="Start node identifier or memory:// reference")],
|
||||
goal: Annotated[
|
||||
Optional[str],
|
||||
typer.Option("--goal", help="Optional goal node identifier for targeted lineage"),
|
||||
] = None,
|
||||
max_hops: int = typer.Option(4, "--max-hops", help="Maximum traversal hops (1-6)"),
|
||||
relation_filters: Annotated[
|
||||
Optional[List[str]],
|
||||
typer.Option("--relation-filter", help="Relation filters (repeatable)"),
|
||||
] = None,
|
||||
project: Annotated[
|
||||
Optional[str],
|
||||
typer.Option(help="The project to use. If not provided, the default project will be used."),
|
||||
] = None,
|
||||
workspace: Annotated[
|
||||
Optional[str],
|
||||
typer.Option(help="Cloud workspace tenant ID or unique name to route this request."),
|
||||
] = None,
|
||||
local: bool = typer.Option(
|
||||
False, "--local", help="Force local API routing (ignore cloud mode)"
|
||||
),
|
||||
cloud: bool = typer.Option(False, "--cloud", help="Force cloud API routing"),
|
||||
):
|
||||
"""Get graph lineage paths from a start node."""
|
||||
try:
|
||||
validate_routing_flags(local, cloud)
|
||||
with force_routing(local=local, cloud=cloud):
|
||||
result = run_with_cleanup(
|
||||
mcp_graph_lineage(
|
||||
start=start,
|
||||
goal=goal,
|
||||
max_hops=max_hops,
|
||||
relation_filters=relation_filters or [],
|
||||
project=project,
|
||||
workspace=workspace,
|
||||
output_format="json",
|
||||
)
|
||||
)
|
||||
_print_json(result)
|
||||
except ValueError as e:
|
||||
typer.echo(f"Error: {e}", err=True)
|
||||
raise typer.Exit(1)
|
||||
except Exception as e: # pragma: no cover
|
||||
if not isinstance(e, typer.Exit):
|
||||
typer.echo(f"Error during graph_lineage: {e}", err=True)
|
||||
raise typer.Exit(1)
|
||||
raise
|
||||
|
||||
|
||||
@tool_app.command("graph-impact")
|
||||
def graph_impact(
|
||||
target: Annotated[str, typer.Argument(help="Target node identifier or memory:// reference")],
|
||||
horizon: int = typer.Option(2, "--horizon", help="Impact horizon in hops (1-4)"),
|
||||
relation_filters: Annotated[
|
||||
Optional[List[str]],
|
||||
typer.Option("--relation-filter", help="Relation filters (repeatable)"),
|
||||
] = None,
|
||||
include_reasons: bool = typer.Option(
|
||||
True,
|
||||
"--include-reasons/--no-include-reasons",
|
||||
help="Include reason strings in impact output",
|
||||
),
|
||||
project: Annotated[
|
||||
Optional[str],
|
||||
typer.Option(help="The project to use. If not provided, the default project will be used."),
|
||||
] = None,
|
||||
workspace: Annotated[
|
||||
Optional[str],
|
||||
typer.Option(help="Cloud workspace tenant ID or unique name to route this request."),
|
||||
] = None,
|
||||
local: bool = typer.Option(
|
||||
False, "--local", help="Force local API routing (ignore cloud mode)"
|
||||
),
|
||||
cloud: bool = typer.Option(False, "--cloud", help="Force cloud API routing"),
|
||||
):
|
||||
"""Get impact radius for a target node."""
|
||||
try:
|
||||
validate_routing_flags(local, cloud)
|
||||
with force_routing(local=local, cloud=cloud):
|
||||
result = run_with_cleanup(
|
||||
mcp_graph_impact(
|
||||
target=target,
|
||||
horizon=horizon,
|
||||
relation_filters=relation_filters or [],
|
||||
include_reasons=include_reasons,
|
||||
project=project,
|
||||
workspace=workspace,
|
||||
output_format="json",
|
||||
)
|
||||
)
|
||||
_print_json(result)
|
||||
except ValueError as e:
|
||||
typer.echo(f"Error: {e}", err=True)
|
||||
raise typer.Exit(1)
|
||||
except Exception as e: # pragma: no cover
|
||||
if not isinstance(e, typer.Exit):
|
||||
typer.echo(f"Error during graph_impact: {e}", err=True)
|
||||
raise typer.Exit(1)
|
||||
raise
|
||||
|
||||
|
||||
@tool_app.command("graph-health")
|
||||
def graph_health(
|
||||
scope: Annotated[Optional[str], typer.Option("--scope", help="Optional scope prefix")] = None,
|
||||
timeframe: Annotated[
|
||||
Optional[str], typer.Option("--timeframe", help="Optional timeframe filter")
|
||||
] = None,
|
||||
project: Annotated[
|
||||
Optional[str],
|
||||
typer.Option(help="The project to use. If not provided, the default project will be used."),
|
||||
] = None,
|
||||
workspace: Annotated[
|
||||
Optional[str],
|
||||
typer.Option(help="Cloud workspace tenant ID or unique name to route this request."),
|
||||
] = None,
|
||||
local: bool = typer.Option(
|
||||
False, "--local", help="Force local API routing (ignore cloud mode)"
|
||||
),
|
||||
cloud: bool = typer.Option(False, "--cloud", help="Force cloud API routing"),
|
||||
):
|
||||
"""Get graph health metrics and issue candidates."""
|
||||
try:
|
||||
validate_routing_flags(local, cloud)
|
||||
with force_routing(local=local, cloud=cloud):
|
||||
result = run_with_cleanup(
|
||||
mcp_graph_health(
|
||||
scope=scope,
|
||||
timeframe=timeframe,
|
||||
project=project,
|
||||
workspace=workspace,
|
||||
output_format="json",
|
||||
)
|
||||
)
|
||||
_print_json(result)
|
||||
except ValueError as e:
|
||||
typer.echo(f"Error: {e}", err=True)
|
||||
raise typer.Exit(1)
|
||||
except Exception as e: # pragma: no cover
|
||||
if not isinstance(e, typer.Exit):
|
||||
typer.echo(f"Error during graph_health: {e}", err=True)
|
||||
raise typer.Exit(1)
|
||||
raise
|
||||
|
||||
|
||||
@tool_app.command("fcm-simulate")
|
||||
def fcm_simulate(
|
||||
actions_json: Annotated[
|
||||
str,
|
||||
typer.Option(
|
||||
"--actions-json",
|
||||
help='JSON array of actions, e.g. [{"node_id":"n1","delta":0.2}]',
|
||||
),
|
||||
],
|
||||
scenario_json: Annotated[
|
||||
Optional[str],
|
||||
typer.Option("--scenario-json", help="Optional JSON scenario object"),
|
||||
] = None,
|
||||
clamp_rules_json: Annotated[
|
||||
Optional[str],
|
||||
typer.Option("--clamp-rules-json", help="Optional JSON array of clamp rules"),
|
||||
] = None,
|
||||
project: Annotated[
|
||||
Optional[str],
|
||||
typer.Option(help="The project to use. If not provided, the default project will be used."),
|
||||
] = None,
|
||||
workspace: Annotated[
|
||||
Optional[str],
|
||||
typer.Option(help="Cloud workspace tenant ID or unique name to route this request."),
|
||||
] = None,
|
||||
local: bool = typer.Option(
|
||||
False, "--local", help="Force local API routing (ignore cloud mode)"
|
||||
),
|
||||
cloud: bool = typer.Option(False, "--cloud", help="Force cloud API routing"),
|
||||
):
|
||||
"""Run an FCM simulation."""
|
||||
actions = _parse_json_option(actions_json, "--actions-json")
|
||||
scenario = _parse_json_option(scenario_json, "--scenario-json")
|
||||
clamp_rules = _parse_json_option(clamp_rules_json, "--clamp-rules-json")
|
||||
if not isinstance(actions, list):
|
||||
typer.echo("Invalid JSON for --actions-json: expected a JSON array", err=True)
|
||||
raise typer.Exit(1)
|
||||
if scenario is not None and not isinstance(scenario, dict):
|
||||
typer.echo("Invalid JSON for --scenario-json: expected a JSON object", err=True)
|
||||
raise typer.Exit(1)
|
||||
if clamp_rules is not None and not isinstance(clamp_rules, list):
|
||||
typer.echo("Invalid JSON for --clamp-rules-json: expected a JSON array", err=True)
|
||||
raise typer.Exit(1)
|
||||
|
||||
try:
|
||||
validate_routing_flags(local, cloud)
|
||||
with force_routing(local=local, cloud=cloud):
|
||||
result = run_with_cleanup(
|
||||
mcp_fcm_simulate(
|
||||
actions=actions,
|
||||
scenario=scenario,
|
||||
clamp_rules=clamp_rules,
|
||||
project=project,
|
||||
workspace=workspace,
|
||||
output_format="json",
|
||||
)
|
||||
)
|
||||
_print_json(result)
|
||||
except ValueError as e:
|
||||
typer.echo(f"Error: {e}", err=True)
|
||||
raise typer.Exit(1)
|
||||
except Exception as e: # pragma: no cover
|
||||
if not isinstance(e, typer.Exit):
|
||||
typer.echo(f"Error during fcm_simulate: {e}", err=True)
|
||||
raise typer.Exit(1)
|
||||
raise
|
||||
|
||||
|
||||
@tool_app.command("fcm-rank-actions")
|
||||
def fcm_rank_actions(
|
||||
goal: Annotated[str, typer.Argument(help="Goal node identifier")],
|
||||
constraints_json: Annotated[
|
||||
Optional[str],
|
||||
typer.Option("--constraints-json", help="Optional JSON object of ranking constraints"),
|
||||
] = None,
|
||||
top_k: int = typer.Option(10, "--top-k", help="Number of recommendations to return"),
|
||||
project: Annotated[
|
||||
Optional[str],
|
||||
typer.Option(help="The project to use. If not provided, the default project will be used."),
|
||||
] = None,
|
||||
workspace: Annotated[
|
||||
Optional[str],
|
||||
typer.Option(help="Cloud workspace tenant ID or unique name to route this request."),
|
||||
] = None,
|
||||
local: bool = typer.Option(
|
||||
False, "--local", help="Force local API routing (ignore cloud mode)"
|
||||
),
|
||||
cloud: bool = typer.Option(False, "--cloud", help="Force cloud API routing"),
|
||||
):
|
||||
"""Rank intervention actions for an FCM goal."""
|
||||
constraints = _parse_json_option(constraints_json, "--constraints-json")
|
||||
if constraints is not None and not isinstance(constraints, dict):
|
||||
typer.echo("Invalid JSON for --constraints-json: expected a JSON object", err=True)
|
||||
raise typer.Exit(1)
|
||||
|
||||
try:
|
||||
validate_routing_flags(local, cloud)
|
||||
with force_routing(local=local, cloud=cloud):
|
||||
result = run_with_cleanup(
|
||||
mcp_fcm_rank_actions(
|
||||
goal=goal,
|
||||
constraints=constraints,
|
||||
top_k=top_k,
|
||||
project=project,
|
||||
workspace=workspace,
|
||||
output_format="json",
|
||||
)
|
||||
)
|
||||
_print_json(result)
|
||||
except ValueError as e:
|
||||
typer.echo(f"Error: {e}", err=True)
|
||||
raise typer.Exit(1)
|
||||
except Exception as e: # pragma: no cover
|
||||
if not isinstance(e, typer.Exit):
|
||||
typer.echo(f"Error during fcm_rank_actions: {e}", err=True)
|
||||
raise typer.Exit(1)
|
||||
raise
|
||||
|
||||
|
||||
@tool_app.command("fcm-import-model")
|
||||
def fcm_import_model(
|
||||
source: Annotated[str, typer.Argument(help="Source path or URI for import payload")],
|
||||
format: Annotated[
|
||||
str,
|
||||
typer.Option("--format", help="Import format (currently csv_bundle_v1)"),
|
||||
] = "csv_bundle_v1",
|
||||
merge_mode: Annotated[
|
||||
str,
|
||||
typer.Option("--merge-mode", help="Merge strategy: replace or upsert"),
|
||||
] = "upsert",
|
||||
project: Annotated[
|
||||
Optional[str],
|
||||
typer.Option(help="The project to use. If not provided, the default project will be used."),
|
||||
] = None,
|
||||
workspace: Annotated[
|
||||
Optional[str],
|
||||
typer.Option(help="Cloud workspace tenant ID or unique name to route this request."),
|
||||
] = None,
|
||||
local: bool = typer.Option(
|
||||
False, "--local", help="Force local API routing (ignore cloud mode)"
|
||||
),
|
||||
cloud: bool = typer.Option(False, "--cloud", help="Force cloud API routing"),
|
||||
):
|
||||
"""Import an FCM model."""
|
||||
try:
|
||||
validate_routing_flags(local, cloud)
|
||||
with force_routing(local=local, cloud=cloud):
|
||||
result = run_with_cleanup(
|
||||
mcp_fcm_import_model(
|
||||
source=source,
|
||||
format=format, # pyright: ignore[reportArgumentType]
|
||||
merge_mode=merge_mode, # pyright: ignore[reportArgumentType]
|
||||
project=project,
|
||||
workspace=workspace,
|
||||
output_format="json",
|
||||
)
|
||||
)
|
||||
_print_json(result)
|
||||
except ValueError as e:
|
||||
typer.echo(f"Error: {e}", err=True)
|
||||
raise typer.Exit(1)
|
||||
except Exception as e: # pragma: no cover
|
||||
if not isinstance(e, typer.Exit):
|
||||
typer.echo(f"Error during fcm_import_model: {e}", err=True)
|
||||
raise typer.Exit(1)
|
||||
raise
|
||||
|
||||
|
||||
@tool_app.command("fcm-export-model")
|
||||
def fcm_export_model(
|
||||
format: Annotated[
|
||||
str,
|
||||
typer.Option("--format", help="Export format (currently csv_bundle_v1)"),
|
||||
] = "csv_bundle_v1",
|
||||
selection_json: Annotated[
|
||||
Optional[str],
|
||||
typer.Option("--selection-json", help="Optional JSON object selection payload"),
|
||||
] = None,
|
||||
project: Annotated[
|
||||
Optional[str],
|
||||
typer.Option(help="The project to use. If not provided, the default project will be used."),
|
||||
] = None,
|
||||
workspace: Annotated[
|
||||
Optional[str],
|
||||
typer.Option(help="Cloud workspace tenant ID or unique name to route this request."),
|
||||
] = None,
|
||||
local: bool = typer.Option(
|
||||
False, "--local", help="Force local API routing (ignore cloud mode)"
|
||||
),
|
||||
cloud: bool = typer.Option(False, "--cloud", help="Force cloud API routing"),
|
||||
):
|
||||
"""Export an FCM model."""
|
||||
selection = _parse_json_option(selection_json, "--selection-json")
|
||||
if selection is not None and not isinstance(selection, dict):
|
||||
typer.echo("Invalid JSON for --selection-json: expected a JSON object", err=True)
|
||||
raise typer.Exit(1)
|
||||
|
||||
try:
|
||||
validate_routing_flags(local, cloud)
|
||||
with force_routing(local=local, cloud=cloud):
|
||||
result = run_with_cleanup(
|
||||
mcp_fcm_export_model(
|
||||
format=format, # pyright: ignore[reportArgumentType]
|
||||
selection=selection,
|
||||
project=project,
|
||||
workspace=workspace,
|
||||
output_format="json",
|
||||
)
|
||||
)
|
||||
_print_json(result)
|
||||
except ValueError as e:
|
||||
typer.echo(f"Error: {e}", err=True)
|
||||
raise typer.Exit(1)
|
||||
except Exception as e: # pragma: no cover
|
||||
if not isinstance(e, typer.Exit):
|
||||
typer.echo(f"Error during fcm_export_model: {e}", err=True)
|
||||
raise typer.Exit(1)
|
||||
raise
|
||||
|
||||
|
||||
@tool_app.command("search-notes")
|
||||
def search_notes(
|
||||
query: Annotated[
|
||||
|
||||
@@ -0,0 +1,40 @@
|
||||
"""Manual update command for Basic Memory CLI."""
|
||||
|
||||
import typer
|
||||
from rich.console import Console
|
||||
|
||||
from basic_memory.cli.app import app
|
||||
from basic_memory.cli.auto_update import AutoUpdateStatus, run_auto_update
|
||||
|
||||
console = Console()
|
||||
|
||||
|
||||
@app.command("update")
|
||||
def update(
|
||||
check: bool = typer.Option(
|
||||
False,
|
||||
"--check",
|
||||
help="Check for updates only (do not install).",
|
||||
),
|
||||
) -> None:
|
||||
"""Check for updates and install when supported."""
|
||||
result = run_auto_update(force=True, check_only=check, silent=False)
|
||||
|
||||
if result.status == AutoUpdateStatus.FAILED:
|
||||
detail = f" {result.error}" if result.error else ""
|
||||
console.print(f"[red]{result.message or 'Update failed.'}{detail}[/red]")
|
||||
raise typer.Exit(1)
|
||||
|
||||
if result.status == AutoUpdateStatus.UPDATED:
|
||||
console.print(f"[green]{result.message or 'Basic Memory updated successfully.'}[/green]")
|
||||
return
|
||||
|
||||
if result.status == AutoUpdateStatus.UP_TO_DATE:
|
||||
console.print(f"[green]{result.message or 'Basic Memory is up to date.'}[/green]")
|
||||
return
|
||||
|
||||
if result.status == AutoUpdateStatus.UPDATE_AVAILABLE:
|
||||
console.print(f"[cyan]{result.message or 'Update available.'}[/cyan]")
|
||||
return
|
||||
|
||||
console.print(f"[dim]{result.message or 'No update action was performed.'}[/dim]")
|
||||
@@ -28,6 +28,7 @@ if not _version_only_invocation(sys.argv[1:]):
|
||||
schema,
|
||||
status,
|
||||
tool,
|
||||
update,
|
||||
)
|
||||
|
||||
warnings.filterwarnings("ignore") # pragma: no cover
|
||||
|
||||
@@ -12,7 +12,7 @@ from basic_memory.config import ConfigManager
|
||||
|
||||
OSS_DISCOUNT_CODE = "BMFOSS"
|
||||
CLOUD_LEARN_MORE_URL = (
|
||||
"https://basicmemory.com?utm_source=bm-cli&utm_medium=promo&utm_campaign=cloud-upsell"
|
||||
"https://basicmemory.com?utm_source=bm-foss&utm_medium=promo&utm_campaign=cloud-upsell"
|
||||
)
|
||||
|
||||
|
||||
|
||||
+110
-9
@@ -6,14 +6,16 @@ import os
|
||||
import shutil
|
||||
from dataclasses import dataclass
|
||||
from datetime import datetime
|
||||
from enum import Enum
|
||||
from pathlib import Path
|
||||
from typing import Any, Dict, Literal, Optional, List, Tuple
|
||||
from enum import Enum
|
||||
|
||||
from loguru import logger
|
||||
from pydantic import AliasChoices, BaseModel, Field, model_validator
|
||||
from pydantic_settings import BaseSettings, SettingsConfigDict
|
||||
|
||||
from basic_memory import __version__
|
||||
from basic_memory.telemetry import configure_telemetry
|
||||
from basic_memory.utils import setup_logging, generate_permalink
|
||||
|
||||
|
||||
@@ -140,6 +142,24 @@ class BasicMemoryConfig(BaseSettings):
|
||||
# overridden by ~/.basic-memory/config.json
|
||||
log_level: str = "INFO"
|
||||
|
||||
# Optional Logfire telemetry (disabled by default)
|
||||
logfire_enabled: bool = Field(
|
||||
default=False,
|
||||
description="Enable Logfire instrumentation for local development or managed deployments.",
|
||||
)
|
||||
logfire_send_to_logfire: bool = Field(
|
||||
default=False,
|
||||
description="When true, allow Logfire to export telemetry to the configured backend.",
|
||||
)
|
||||
logfire_service_name: str = Field(
|
||||
default="basic-memory",
|
||||
description="Base service name used when constructing entrypoint-specific Logfire service names.",
|
||||
)
|
||||
logfire_environment: str | None = Field(
|
||||
default=None,
|
||||
description="Optional override for Logfire environment. Defaults to env when unset.",
|
||||
)
|
||||
|
||||
# Database configuration
|
||||
database_backend: DatabaseBackend = Field(
|
||||
default=DatabaseBackend.SQLITE,
|
||||
@@ -203,6 +223,12 @@ class BasicMemoryConfig(BaseSettings):
|
||||
ge=0.0,
|
||||
le=1.0,
|
||||
)
|
||||
default_search_type: Literal["text", "vector", "hybrid"] | None = Field(
|
||||
default=None,
|
||||
description="Default search type for search_notes when not specified per-query. "
|
||||
"Valid values: text, vector, hybrid. "
|
||||
"When unset, defaults to 'hybrid' if semantic search is enabled, otherwise 'text'.",
|
||||
)
|
||||
|
||||
# Database connection pool configuration (Postgres only)
|
||||
db_pool_size: int = Field(
|
||||
@@ -351,6 +377,22 @@ class BasicMemoryConfig(BaseSettings):
|
||||
description="Most recent cloud promo version shown in CLI.",
|
||||
)
|
||||
|
||||
auto_update: bool = Field(
|
||||
default=True,
|
||||
description="Enable automatic CLI update checks and installs when supported.",
|
||||
)
|
||||
|
||||
update_check_interval: int = Field(
|
||||
default=86400,
|
||||
description="Seconds between automatic update checks.",
|
||||
gt=0,
|
||||
)
|
||||
|
||||
auto_update_last_checked_at: Optional[datetime] = Field(
|
||||
default=None,
|
||||
description="Timestamp of the last attempted automatic update check.",
|
||||
)
|
||||
|
||||
cloud_api_key: Optional[str] = Field(
|
||||
default=None,
|
||||
description="API key for cloud access (bmc_ prefixed). Account-level, not per-project.",
|
||||
@@ -629,6 +671,12 @@ class BasicMemoryConfig(BaseSettings):
|
||||
|
||||
# Module-level cache for configuration
|
||||
_CONFIG_CACHE: Optional[BasicMemoryConfig] = None
|
||||
# Track config file mtime+size so cross-process changes (e.g. `bm project set-cloud`
|
||||
# in a separate terminal) invalidate the cache in long-lived processes like the
|
||||
# MCP stdio server. Using both mtime and size guards against coarse-granularity
|
||||
# filesystems where two writes within the same second share the same mtime.
|
||||
_CONFIG_MTIME: Optional[float] = None
|
||||
_CONFIG_SIZE: Optional[int] = None
|
||||
|
||||
|
||||
class ConfigManager:
|
||||
@@ -662,13 +710,38 @@ class ConfigManager:
|
||||
Environment variables take precedence over file config values,
|
||||
following Pydantic Settings best practices.
|
||||
|
||||
Uses module-level cache for performance across ConfigManager instances.
|
||||
Uses module-level cache with file mtime validation so that
|
||||
cross-process config changes (e.g. `bm project set-cloud` in a
|
||||
separate terminal) are picked up by long-lived processes like
|
||||
the MCP stdio server.
|
||||
"""
|
||||
global _CONFIG_CACHE
|
||||
global _CONFIG_CACHE, _CONFIG_MTIME, _CONFIG_SIZE
|
||||
|
||||
# Return cached config if available
|
||||
# Trigger: cached config exists but the on-disk file may have been
|
||||
# modified by another process (CLI command in a different terminal).
|
||||
# Why: the MCP server is long-lived; without this check it would
|
||||
# serve stale project routing forever.
|
||||
# Outcome: cheap os.stat() per access; re-read only when mtime or size differs.
|
||||
if _CONFIG_CACHE is not None:
|
||||
return _CONFIG_CACHE
|
||||
try:
|
||||
st = self.config_file.stat()
|
||||
current_mtime = st.st_mtime
|
||||
current_size = st.st_size
|
||||
except OSError:
|
||||
current_mtime = None
|
||||
current_size = None
|
||||
|
||||
if (
|
||||
current_mtime is not None
|
||||
and current_mtime == _CONFIG_MTIME
|
||||
and current_size == _CONFIG_SIZE
|
||||
):
|
||||
return _CONFIG_CACHE
|
||||
|
||||
# mtime/size changed or file gone — invalidate and fall through to re-read
|
||||
_CONFIG_CACHE = None
|
||||
_CONFIG_MTIME = None
|
||||
_CONFIG_SIZE = None
|
||||
|
||||
if self.config_file.exists():
|
||||
try:
|
||||
@@ -723,6 +796,15 @@ class ConfigManager:
|
||||
|
||||
_CONFIG_CACHE = BasicMemoryConfig(**merged_data)
|
||||
|
||||
# Record mtime+size so subsequent calls detect cross-process changes
|
||||
try:
|
||||
st = self.config_file.stat()
|
||||
_CONFIG_MTIME = st.st_mtime
|
||||
_CONFIG_SIZE = st.st_size
|
||||
except OSError:
|
||||
_CONFIG_MTIME = None
|
||||
_CONFIG_SIZE = None
|
||||
|
||||
# Re-save to normalize legacy config into current format
|
||||
if needs_resave:
|
||||
# Create backup before overwriting so users can revert if needed
|
||||
@@ -753,10 +835,12 @@ class ConfigManager:
|
||||
|
||||
def save_config(self, config: BasicMemoryConfig) -> None:
|
||||
"""Save configuration to file and invalidate cache."""
|
||||
global _CONFIG_CACHE
|
||||
global _CONFIG_CACHE, _CONFIG_MTIME, _CONFIG_SIZE
|
||||
save_basic_memory_config(self.config_file, config)
|
||||
# Invalidate cache so next load_config() reads fresh data
|
||||
_CONFIG_CACHE = None
|
||||
_CONFIG_MTIME = None
|
||||
_CONFIG_SIZE = None
|
||||
|
||||
@property
|
||||
def projects(self) -> Dict[str, str]:
|
||||
@@ -891,33 +975,50 @@ def save_basic_memory_config(file_path: Path, config: BasicMemoryConfig) -> None
|
||||
# Logging initialization functions for different entry points
|
||||
|
||||
|
||||
def init_cli_logging() -> None: # pragma: no cover
|
||||
def _configure_logfire_for_entrypoint(entrypoint: str) -> None:
|
||||
"""Configure optional Logfire telemetry for a specific entrypoint."""
|
||||
config = ConfigManager().config
|
||||
service_name = f"{config.logfire_service_name}-{entrypoint}"
|
||||
environment = config.logfire_environment or config.env
|
||||
configure_telemetry(
|
||||
service_name=service_name,
|
||||
environment=environment,
|
||||
service_version=__version__,
|
||||
enable_logfire=config.logfire_enabled,
|
||||
send_to_logfire=config.logfire_send_to_logfire,
|
||||
)
|
||||
|
||||
|
||||
def init_cli_logging() -> None:
|
||||
"""Initialize logging for CLI commands - file only.
|
||||
|
||||
CLI commands should not log to stdout to avoid interfering with
|
||||
command output and shell integration.
|
||||
"""
|
||||
log_level = os.getenv("BASIC_MEMORY_LOG_LEVEL", "INFO")
|
||||
_configure_logfire_for_entrypoint("cli")
|
||||
setup_logging(log_level=log_level, log_to_file=True)
|
||||
|
||||
|
||||
def init_mcp_logging() -> None: # pragma: no cover
|
||||
def init_mcp_logging() -> None:
|
||||
"""Initialize logging for MCP server - file only.
|
||||
|
||||
MCP server must not log to stdout as it would corrupt the
|
||||
JSON-RPC protocol communication.
|
||||
"""
|
||||
log_level = os.getenv("BASIC_MEMORY_LOG_LEVEL", "INFO")
|
||||
_configure_logfire_for_entrypoint("mcp")
|
||||
setup_logging(log_level=log_level, log_to_file=True)
|
||||
|
||||
|
||||
def init_api_logging() -> None: # pragma: no cover
|
||||
def init_api_logging() -> None:
|
||||
"""Initialize logging for API server.
|
||||
|
||||
Cloud mode (BASIC_MEMORY_CLOUD_MODE=1): stdout with structured context
|
||||
Local mode: file only
|
||||
"""
|
||||
log_level = os.getenv("BASIC_MEMORY_LOG_LEVEL", "INFO")
|
||||
_configure_logfire_for_entrypoint("api")
|
||||
cloud_mode = os.getenv("BASIC_MEMORY_CLOUD_MODE", "").lower() in ("1", "true")
|
||||
if cloud_mode:
|
||||
setup_logging(log_level=log_level, log_to_stdout=True, structured_context=True)
|
||||
|
||||
+34
-52
@@ -43,40 +43,37 @@ if sys.platform == "win32": # pragma: no cover
|
||||
_engine: Optional[AsyncEngine] = None
|
||||
_session_maker: Optional[async_sessionmaker[AsyncSession]] = None
|
||||
|
||||
# Alembic revision that enables one-time automatic embedding backfill.
|
||||
SEMANTIC_EMBEDDING_BACKFILL_REVISION = "i2c3d4e5f6g7"
|
||||
|
||||
|
||||
async def _load_applied_alembic_revisions(
|
||||
async def _needs_semantic_embedding_backfill(
|
||||
app_config: BasicMemoryConfig,
|
||||
session_maker: async_sessionmaker[AsyncSession],
|
||||
) -> set[str]:
|
||||
"""Load applied Alembic revisions from alembic_version.
|
||||
) -> bool:
|
||||
"""Check if entities exist but vector embeddings are empty.
|
||||
|
||||
Returns an empty set when the version table does not exist yet
|
||||
(fresh database before first migration).
|
||||
This is the reliable way to detect that embeddings need to be generated,
|
||||
regardless of how migrations were applied (fresh DB, upgrade, reset, etc.).
|
||||
"""
|
||||
if not app_config.semantic_search_enabled:
|
||||
return False
|
||||
|
||||
try:
|
||||
async with scoped_session(session_maker) as session:
|
||||
result = await session.execute(text("SELECT version_num FROM alembic_version"))
|
||||
return {str(row[0]) for row in result.fetchall() if row[0]}
|
||||
entity_count = (
|
||||
await session.execute(text("SELECT COUNT(*) FROM entity"))
|
||||
).scalar() or 0
|
||||
if entity_count == 0:
|
||||
return False
|
||||
|
||||
# Check if vector chunks table exists and is empty
|
||||
embedding_count = (
|
||||
await session.execute(text("SELECT COUNT(*) FROM search_vector_chunks"))
|
||||
).scalar() or 0
|
||||
|
||||
return embedding_count == 0
|
||||
except Exception as exc:
|
||||
error_message = str(exc).lower()
|
||||
if "alembic_version" in error_message and (
|
||||
"no such table" in error_message or "does not exist" in error_message
|
||||
):
|
||||
return set()
|
||||
raise
|
||||
|
||||
|
||||
def _should_run_semantic_embedding_backfill(
|
||||
revisions_before_upgrade: set[str],
|
||||
revisions_after_upgrade: set[str],
|
||||
) -> bool:
|
||||
"""Check if this migration run newly applied the backfill-trigger revision."""
|
||||
return (
|
||||
SEMANTIC_EMBEDDING_BACKFILL_REVISION in revisions_after_upgrade
|
||||
and SEMANTIC_EMBEDDING_BACKFILL_REVISION not in revisions_before_upgrade
|
||||
)
|
||||
# Table might not exist yet (pre-migration)
|
||||
logger.debug(f"Could not check embedding status: {exc}")
|
||||
return False
|
||||
|
||||
|
||||
async def _run_semantic_embedding_backfill(
|
||||
@@ -480,26 +477,9 @@ async def run_migrations(
|
||||
Note: Alembic tracks which migrations have been applied via the alembic_version table,
|
||||
so it's safe to call this multiple times - it will only run pending migrations.
|
||||
"""
|
||||
logger.debug("Running database migrations...")
|
||||
logger.info("Running database migrations...")
|
||||
temp_engine: AsyncEngine | None = None
|
||||
try:
|
||||
revisions_before_upgrade: set[str] = set()
|
||||
# Trigger: run_migrations() can be invoked before module-level session maker is set.
|
||||
# Why: we still need reliable before/after revision detection for one-time backfill.
|
||||
# Outcome: create a short-lived session maker when needed, then dispose it immediately.
|
||||
if _session_maker is None:
|
||||
precheck_engine, temp_session_maker = _create_engine_and_session(
|
||||
app_config.database_path,
|
||||
database_type,
|
||||
app_config,
|
||||
)
|
||||
try:
|
||||
revisions_before_upgrade = await _load_applied_alembic_revisions(temp_session_maker)
|
||||
finally:
|
||||
await precheck_engine.dispose()
|
||||
else:
|
||||
revisions_before_upgrade = await _load_applied_alembic_revisions(_session_maker)
|
||||
|
||||
# Get the absolute path to the alembic directory relative to this file
|
||||
alembic_dir = Path(__file__).parent / "alembic"
|
||||
config = Config()
|
||||
@@ -519,7 +499,7 @@ async def run_migrations(
|
||||
config.set_main_option("sqlalchemy.url", db_url)
|
||||
|
||||
command.upgrade(config, "head")
|
||||
logger.debug("Migrations completed successfully")
|
||||
logger.info("Migrations completed successfully")
|
||||
|
||||
# Get session maker - ensure we don't trigger recursive migration calls
|
||||
if _session_maker is None:
|
||||
@@ -541,12 +521,14 @@ async def run_migrations(
|
||||
else:
|
||||
await SQLiteSearchRepository(session_maker, 1).init_search_index()
|
||||
|
||||
revisions_after_upgrade = await _load_applied_alembic_revisions(session_maker)
|
||||
if _should_run_semantic_embedding_backfill(
|
||||
revisions_before_upgrade,
|
||||
revisions_after_upgrade,
|
||||
):
|
||||
await _run_semantic_embedding_backfill(app_config, session_maker)
|
||||
# Check if backfill is needed — actual backfill runs in background
|
||||
# from the MCP server lifespan to avoid blocking startup.
|
||||
if await _needs_semantic_embedding_backfill(app_config, session_maker):
|
||||
logger.info(
|
||||
"Semantic embeddings missing — backfill will run in background after startup"
|
||||
)
|
||||
else:
|
||||
logger.info("Semantic embeddings: up to date")
|
||||
except Exception as e: # pragma: no cover
|
||||
logger.error(f"Error running migrations: {e}")
|
||||
raise
|
||||
|
||||
@@ -131,10 +131,6 @@ from basic_memory.deps.services import (
|
||||
DirectoryServiceV2Dep,
|
||||
get_directory_service_v2_external,
|
||||
DirectoryServiceV2ExternalDep,
|
||||
get_graph_intelligence_service_v2_external,
|
||||
GraphIntelligenceServiceV2ExternalDep,
|
||||
get_fcm_service_v2_external,
|
||||
FCMServiceV2ExternalDep,
|
||||
)
|
||||
|
||||
from basic_memory.deps.importers import (
|
||||
@@ -273,10 +269,6 @@ __all__ = [
|
||||
"DirectoryServiceV2Dep",
|
||||
"get_directory_service_v2_external",
|
||||
"DirectoryServiceV2ExternalDep",
|
||||
"get_graph_intelligence_service_v2_external",
|
||||
"GraphIntelligenceServiceV2ExternalDep",
|
||||
"get_fcm_service_v2_external",
|
||||
"FCMServiceV2ExternalDep",
|
||||
# Importers
|
||||
"get_chatgpt_importer",
|
||||
"ChatGPTImporterDep",
|
||||
|
||||
@@ -39,8 +39,6 @@ from basic_memory.deps.repositories import (
|
||||
from basic_memory.markdown import EntityParser
|
||||
from basic_memory.markdown.markdown_processor import MarkdownProcessor
|
||||
from basic_memory.services import EntityService, ProjectService
|
||||
from basic_memory.services.fcm_service import FCMService
|
||||
from basic_memory.services.graph_intelligence_service import GraphIntelligenceService
|
||||
from basic_memory.services.context_service import ContextService
|
||||
from basic_memory.services.directory_service import DirectoryService
|
||||
from basic_memory.services.file_service import FileService
|
||||
@@ -360,30 +358,6 @@ async def get_context_service_v2_external(
|
||||
ContextServiceV2ExternalDep = Annotated[ContextService, Depends(get_context_service_v2_external)]
|
||||
|
||||
|
||||
# --- Graph Intelligence Service ---
|
||||
|
||||
|
||||
async def get_graph_intelligence_service_v2_external() -> GraphIntelligenceService:
|
||||
"""Create GraphIntelligenceService for v2 API (uses external_id routing)."""
|
||||
return GraphIntelligenceService()
|
||||
|
||||
|
||||
GraphIntelligenceServiceV2ExternalDep = Annotated[
|
||||
GraphIntelligenceService, Depends(get_graph_intelligence_service_v2_external)
|
||||
]
|
||||
|
||||
|
||||
# --- FCM Service ---
|
||||
|
||||
|
||||
async def get_fcm_service_v2_external() -> FCMService:
|
||||
"""Create FCMService for v2 API (uses external_id routing)."""
|
||||
return FCMService()
|
||||
|
||||
|
||||
FCMServiceV2ExternalDep = Annotated[FCMService, Depends(get_fcm_service_v2_external)]
|
||||
|
||||
|
||||
# --- Sync Service ---
|
||||
|
||||
|
||||
@@ -561,21 +535,6 @@ async def get_task_scheduler(
|
||||
async def _reindex_project(**_: Any) -> None:
|
||||
await search_service.reindex_all()
|
||||
|
||||
async def _sync_graph_entity(entity_id: int, **extra_payload: Any) -> None:
|
||||
# Trigger: graph-entity sync task is scheduled from graph lifecycle hooks.
|
||||
# Why: keep scheduler contract stable while graph index provider work lands in later phases.
|
||||
# Outcome: no-op in phase 1; task name remains valid for API and tool contracts.
|
||||
del entity_id, extra_payload
|
||||
|
||||
async def _sync_graph_project(force_full: bool = False, **_: Any) -> None:
|
||||
await _sync_project(force_full=force_full)
|
||||
|
||||
async def _reindex_graph_project(**_: Any) -> None:
|
||||
# Trigger: graph reindex requested.
|
||||
# Why: phase 1 has no dedicated graph index worker yet.
|
||||
# Outcome: run project sync path so writes stay coherent while graph provider ships.
|
||||
await _sync_project(force_full=True)
|
||||
|
||||
scheduler = LocalTaskScheduler(
|
||||
{
|
||||
"reindex_entity": _reindex_entity,
|
||||
@@ -583,9 +542,6 @@ async def get_task_scheduler(
|
||||
"sync_entity_vectors": _sync_entity_vectors,
|
||||
"sync_project": _sync_project,
|
||||
"reindex_project": _reindex_project,
|
||||
"sync_graph_entity": _sync_graph_entity,
|
||||
"sync_graph_project": _sync_graph_project,
|
||||
"reindex_graph_project": _reindex_graph_project,
|
||||
},
|
||||
test_mode=app_config.is_test_env,
|
||||
)
|
||||
|
||||
@@ -447,6 +447,11 @@ def sanitize_for_filename(text: str, replacement: str = "-") -> str:
|
||||
# compress multiple, repeated replacements
|
||||
text = re.sub(f"{re.escape(replacement)}+", replacement, text)
|
||||
|
||||
# Strip trailing periods — they cause "hi-everyone..md" double-dot filenames
|
||||
# when ".md" is appended, which triggers path traversal false positives.
|
||||
# Trailing periods are also invalid on Windows filesystems.
|
||||
text = text.strip(".")
|
||||
|
||||
return text.strip(replacement)
|
||||
|
||||
|
||||
|
||||
@@ -5,6 +5,7 @@ from typing import AsyncIterator, Callable, Optional
|
||||
from httpx import ASGITransport, AsyncClient, Timeout
|
||||
from loguru import logger
|
||||
|
||||
from basic_memory import telemetry
|
||||
from basic_memory.api.app import app as fastapi_app
|
||||
from basic_memory.config import ConfigManager, ProjectMode
|
||||
|
||||
@@ -43,21 +44,26 @@ def _asgi_client(timeout: Timeout) -> AsyncClient:
|
||||
|
||||
async def _resolve_cloud_token(config) -> str:
|
||||
"""Resolve cloud token with API key preferred, OAuth fallback."""
|
||||
token = config.cloud_api_key
|
||||
if token:
|
||||
return token
|
||||
with telemetry.span(
|
||||
"routing.resolve_cloud_credentials",
|
||||
has_api_key=bool(config.cloud_api_key),
|
||||
):
|
||||
token = config.cloud_api_key
|
||||
if token:
|
||||
return token
|
||||
|
||||
from basic_memory.cli.auth import CLIAuth
|
||||
from basic_memory.cli.auth import CLIAuth
|
||||
|
||||
auth = CLIAuth(client_id=config.cloud_client_id, authkit_domain=config.cloud_domain)
|
||||
token = await auth.get_valid_token()
|
||||
if token:
|
||||
return token
|
||||
auth = CLIAuth(client_id=config.cloud_client_id, authkit_domain=config.cloud_domain)
|
||||
token = await auth.get_valid_token()
|
||||
if token:
|
||||
return token
|
||||
|
||||
raise RuntimeError(
|
||||
"Cloud routing requested but no credentials found. "
|
||||
"Run 'bm cloud api-key save <key>' or 'bm cloud login' first."
|
||||
)
|
||||
logger.error("Cloud routing requested but no credentials were available")
|
||||
raise RuntimeError(
|
||||
"Cloud routing requested but no credentials found. "
|
||||
"Run 'bm cloud api-key save <key>' or 'bm cloud login' first."
|
||||
)
|
||||
|
||||
|
||||
@asynccontextmanager
|
||||
|
||||
@@ -18,8 +18,6 @@ from basic_memory.mcp.clients.directory import DirectoryClient
|
||||
from basic_memory.mcp.clients.resource import ResourceClient
|
||||
from basic_memory.mcp.clients.project import ProjectClient
|
||||
from basic_memory.mcp.clients.schema import SchemaClient
|
||||
from basic_memory.mcp.clients.graph import GraphClient
|
||||
from basic_memory.mcp.clients.fcm import FCMClient
|
||||
|
||||
__all__ = [
|
||||
"KnowledgeClient",
|
||||
@@ -29,6 +27,4 @@ __all__ = [
|
||||
"ResourceClient",
|
||||
"ProjectClient",
|
||||
"SchemaClient",
|
||||
"GraphClient",
|
||||
"FCMClient",
|
||||
]
|
||||
|
||||
@@ -1,56 +0,0 @@
|
||||
"""Typed client for FCM API operations."""
|
||||
|
||||
from httpx import AsyncClient
|
||||
|
||||
from basic_memory.mcp.tools.utils import call_post
|
||||
from basic_memory.schemas.graph_intelligence import (
|
||||
FCMExportRequest,
|
||||
FCMExportResponse,
|
||||
FCMImportRequest,
|
||||
FCMImportResponse,
|
||||
FCMRankActionsRequest,
|
||||
FCMRankActionsResponse,
|
||||
FCMSimulateRequest,
|
||||
FCMSimulateResponse,
|
||||
)
|
||||
|
||||
|
||||
class FCMClient:
|
||||
"""Typed client for FCM operations."""
|
||||
|
||||
def __init__(self, http_client: AsyncClient, project_id: str):
|
||||
self.http_client = http_client
|
||||
self.project_id = project_id
|
||||
self._base_path = f"/v2/projects/{project_id}/fcm"
|
||||
|
||||
async def simulate(self, request: FCMSimulateRequest) -> FCMSimulateResponse:
|
||||
response = await call_post(
|
||||
self.http_client,
|
||||
f"{self._base_path}/simulate",
|
||||
json=request.model_dump(mode="json"),
|
||||
)
|
||||
return FCMSimulateResponse.model_validate(response.json())
|
||||
|
||||
async def rank_actions(self, request: FCMRankActionsRequest) -> FCMRankActionsResponse:
|
||||
response = await call_post(
|
||||
self.http_client,
|
||||
f"{self._base_path}/rank-actions",
|
||||
json=request.model_dump(mode="json"),
|
||||
)
|
||||
return FCMRankActionsResponse.model_validate(response.json())
|
||||
|
||||
async def import_model(self, request: FCMImportRequest) -> FCMImportResponse:
|
||||
response = await call_post(
|
||||
self.http_client,
|
||||
f"{self._base_path}/import",
|
||||
json=request.model_dump(mode="json"),
|
||||
)
|
||||
return FCMImportResponse.model_validate(response.json())
|
||||
|
||||
async def export_model(self, request: FCMExportRequest) -> FCMExportResponse:
|
||||
response = await call_post(
|
||||
self.http_client,
|
||||
f"{self._base_path}/export",
|
||||
json=request.model_dump(mode="json"),
|
||||
)
|
||||
return FCMExportResponse.model_validate(response.json())
|
||||
@@ -1,62 +0,0 @@
|
||||
"""Typed client for graph intelligence API operations."""
|
||||
|
||||
from httpx import AsyncClient
|
||||
|
||||
from basic_memory.mcp.tools.utils import call_get, call_post
|
||||
from basic_memory.schemas.graph_intelligence import (
|
||||
GraphHealthResponse,
|
||||
GraphImpactRequest,
|
||||
GraphImpactResponse,
|
||||
GraphLineageRequest,
|
||||
GraphLineageResponse,
|
||||
GraphReindexRequest,
|
||||
GraphReindexResponse,
|
||||
)
|
||||
|
||||
|
||||
class GraphClient:
|
||||
"""Typed client for graph intelligence operations."""
|
||||
|
||||
def __init__(self, http_client: AsyncClient, project_id: str):
|
||||
self.http_client = http_client
|
||||
self.project_id = project_id
|
||||
self._base_path = f"/v2/projects/{project_id}/graph"
|
||||
|
||||
async def lineage(self, request: GraphLineageRequest) -> GraphLineageResponse:
|
||||
response = await call_post(
|
||||
self.http_client,
|
||||
f"{self._base_path}/lineage",
|
||||
json=request.model_dump(mode="json"),
|
||||
)
|
||||
return GraphLineageResponse.model_validate(response.json())
|
||||
|
||||
async def impact(self, request: GraphImpactRequest) -> GraphImpactResponse:
|
||||
response = await call_post(
|
||||
self.http_client,
|
||||
f"{self._base_path}/impact",
|
||||
json=request.model_dump(mode="json"),
|
||||
)
|
||||
return GraphImpactResponse.model_validate(response.json())
|
||||
|
||||
async def health(
|
||||
self, scope: str | None = None, timeframe: str | None = None
|
||||
) -> GraphHealthResponse:
|
||||
params: dict[str, str] = {}
|
||||
if scope is not None:
|
||||
params["scope"] = scope
|
||||
if timeframe is not None:
|
||||
params["timeframe"] = timeframe
|
||||
response = await call_get(
|
||||
self.http_client,
|
||||
f"{self._base_path}/health",
|
||||
params=params,
|
||||
)
|
||||
return GraphHealthResponse.model_validate(response.json())
|
||||
|
||||
async def reindex(self, request: GraphReindexRequest) -> GraphReindexResponse:
|
||||
response = await call_post(
|
||||
self.http_client,
|
||||
f"{self._base_path}/reindex",
|
||||
json=request.model_dump(mode="json"),
|
||||
)
|
||||
return GraphReindexResponse.model_validate(response.json())
|
||||
@@ -19,6 +19,7 @@ from loguru import logger
|
||||
from fastmcp import Context
|
||||
from mcp.server.fastmcp.exceptions import ToolError
|
||||
|
||||
from basic_memory import telemetry
|
||||
from basic_memory.config import BasicMemoryConfig, ConfigManager, ProjectMode
|
||||
from basic_memory.project_resolver import ProjectResolver
|
||||
from basic_memory.schemas.cloud import WorkspaceInfo, WorkspaceListResponse
|
||||
@@ -63,6 +64,28 @@ async def _resolve_default_project_from_api() -> Optional[str]:
|
||||
return None
|
||||
|
||||
|
||||
def _canonicalize_project_name(
|
||||
project_name: Optional[str],
|
||||
config: BasicMemoryConfig,
|
||||
) -> Optional[str]:
|
||||
"""Return the configured project name when the identifier matches by permalink.
|
||||
|
||||
Project routing happens before API validation, so we normalize explicit inputs
|
||||
here to keep local/cloud routing aligned with the database's case-insensitive
|
||||
project resolver.
|
||||
"""
|
||||
if project_name is None:
|
||||
return None
|
||||
|
||||
requested_permalink = generate_permalink(project_name)
|
||||
for configured_name in config.projects:
|
||||
if generate_permalink(configured_name) == requested_permalink:
|
||||
return configured_name
|
||||
|
||||
return project_name
|
||||
|
||||
|
||||
|
||||
async def resolve_project_parameter(
|
||||
project: Optional[str] = None,
|
||||
allow_discovery: bool = False,
|
||||
@@ -89,22 +112,28 @@ async def resolve_project_parameter(
|
||||
Returns:
|
||||
Resolved project name or None if no resolution possible
|
||||
"""
|
||||
# Load config for any values not explicitly provided.
|
||||
# ConfigManager reads from the local config file, which doesn't exist in cloud mode.
|
||||
# When it returns None, fall back to querying the projects API for the is_default flag.
|
||||
if default_project is None:
|
||||
with telemetry.span(
|
||||
"routing.resolve_project",
|
||||
requested_project=project,
|
||||
allow_discovery=allow_discovery,
|
||||
):
|
||||
config = ConfigManager().config
|
||||
default_project = config.default_project
|
||||
|
||||
if default_project is None:
|
||||
default_project = await _resolve_default_project_from_api()
|
||||
# Load config for any values not explicitly provided.
|
||||
# ConfigManager reads from the local config file, which doesn't exist in cloud mode.
|
||||
# When it returns None, fall back to querying the projects API for the is_default flag.
|
||||
if default_project is None:
|
||||
default_project = config.default_project
|
||||
|
||||
# Create resolver with configuration and resolve
|
||||
resolver = ProjectResolver.from_env(
|
||||
default_project=default_project,
|
||||
)
|
||||
result = resolver.resolve(project=project, allow_discovery=allow_discovery)
|
||||
return result.project
|
||||
if default_project is None:
|
||||
default_project = await _resolve_default_project_from_api()
|
||||
|
||||
# Create resolver with configuration and resolve
|
||||
resolver = ProjectResolver.from_env(
|
||||
default_project=default_project,
|
||||
)
|
||||
result = resolver.resolve(project=project, allow_discovery=allow_discovery)
|
||||
return _canonicalize_project_name(result.project, config)
|
||||
|
||||
|
||||
async def get_project_names(client: AsyncClient, headers: HeaderTypes | None = None) -> List[str]:
|
||||
@@ -177,51 +206,60 @@ async def resolve_workspace_parameter(
|
||||
context: Optional[Context] = None,
|
||||
) -> WorkspaceInfo:
|
||||
"""Resolve workspace using explicit input, session cache, and cloud discovery."""
|
||||
if context:
|
||||
cached_raw = await context.get_state("active_workspace")
|
||||
if isinstance(cached_raw, dict):
|
||||
cached_workspace = WorkspaceInfo.model_validate(cached_raw)
|
||||
if workspace is None or _workspace_matches_identifier(cached_workspace, workspace):
|
||||
logger.debug(f"Using cached workspace from context: {cached_workspace.tenant_id}")
|
||||
return cached_workspace
|
||||
with telemetry.scope(
|
||||
"routing.resolve_workspace",
|
||||
workspace_requested=workspace is not None,
|
||||
has_context=context is not None,
|
||||
):
|
||||
if context:
|
||||
cached_raw = await context.get_state("active_workspace")
|
||||
if isinstance(cached_raw, dict):
|
||||
cached_workspace = WorkspaceInfo.model_validate(cached_raw)
|
||||
if workspace is None or _workspace_matches_identifier(cached_workspace, workspace):
|
||||
logger.debug(
|
||||
f"Using cached workspace from context: {cached_workspace.tenant_id}"
|
||||
)
|
||||
return cached_workspace
|
||||
|
||||
workspaces = await get_available_workspaces(context=context)
|
||||
if not workspaces:
|
||||
raise ValueError(
|
||||
"No accessible workspaces found for this account. "
|
||||
"Ensure you have an active subscription and tenant access."
|
||||
)
|
||||
|
||||
selected_workspace: WorkspaceInfo | None = None
|
||||
|
||||
if workspace:
|
||||
matches = [item for item in workspaces if _workspace_matches_identifier(item, workspace)]
|
||||
if not matches:
|
||||
workspaces = await get_available_workspaces(context=context)
|
||||
if not workspaces:
|
||||
raise ValueError(
|
||||
f"Workspace '{workspace}' was not found.\n"
|
||||
"No accessible workspaces found for this account. "
|
||||
"Ensure you have an active subscription and tenant access."
|
||||
)
|
||||
|
||||
selected_workspace: WorkspaceInfo | None = None
|
||||
|
||||
if workspace:
|
||||
matches = [
|
||||
item for item in workspaces if _workspace_matches_identifier(item, workspace)
|
||||
]
|
||||
if not matches:
|
||||
raise ValueError(
|
||||
f"Workspace '{workspace}' was not found.\n"
|
||||
f"Available workspaces:\n{_workspace_choices(workspaces)}"
|
||||
)
|
||||
if len(matches) > 1:
|
||||
raise ValueError(
|
||||
f"Workspace name '{workspace}' matches multiple workspaces. "
|
||||
"Use tenant_id instead.\n"
|
||||
f"Available workspaces:\n{_workspace_choices(workspaces)}"
|
||||
)
|
||||
selected_workspace = matches[0]
|
||||
elif len(workspaces) == 1:
|
||||
selected_workspace = workspaces[0]
|
||||
else:
|
||||
raise ValueError(
|
||||
"Multiple workspaces are available. Ask the user which workspace to use, then retry "
|
||||
"with the 'workspace' argument set to the tenant_id or unique name.\n"
|
||||
f"Available workspaces:\n{_workspace_choices(workspaces)}"
|
||||
)
|
||||
if len(matches) > 1:
|
||||
raise ValueError(
|
||||
f"Workspace name '{workspace}' matches multiple workspaces. "
|
||||
"Use tenant_id instead.\n"
|
||||
f"Available workspaces:\n{_workspace_choices(workspaces)}"
|
||||
)
|
||||
selected_workspace = matches[0]
|
||||
elif len(workspaces) == 1:
|
||||
selected_workspace = workspaces[0]
|
||||
else:
|
||||
raise ValueError(
|
||||
"Multiple workspaces are available. Ask the user which workspace to use, then retry "
|
||||
"with the 'workspace' argument set to the tenant_id or unique name.\n"
|
||||
f"Available workspaces:\n{_workspace_choices(workspaces)}"
|
||||
)
|
||||
|
||||
if context:
|
||||
await context.set_state("active_workspace", selected_workspace.model_dump())
|
||||
logger.debug(f"Cached workspace in context: {selected_workspace.tenant_id}")
|
||||
if context:
|
||||
await context.set_state("active_workspace", selected_workspace.model_dump())
|
||||
logger.debug(f"Cached workspace in context: {selected_workspace.tenant_id}")
|
||||
|
||||
return selected_workspace
|
||||
return selected_workspace
|
||||
|
||||
|
||||
async def get_active_project(
|
||||
@@ -244,53 +282,58 @@ async def get_active_project(
|
||||
ValueError: If no project can be resolved
|
||||
HTTPError: If project doesn't exist or is inaccessible
|
||||
"""
|
||||
# Deferred import to avoid circular dependency with tools
|
||||
from basic_memory.mcp.tools.utils import call_post
|
||||
with telemetry.scope(
|
||||
"routing.validate_project",
|
||||
requested_project=project,
|
||||
has_context=context is not None,
|
||||
):
|
||||
# Deferred import to avoid circular dependency with tools
|
||||
from basic_memory.mcp.tools.utils import call_post
|
||||
|
||||
resolved_project = await resolve_project_parameter(project)
|
||||
if not resolved_project:
|
||||
project_names = await get_project_names(client, headers)
|
||||
raise ValueError(
|
||||
"No project specified. "
|
||||
"Either set 'default_project' in config, or use 'project' argument.\n"
|
||||
f"Available projects: {project_names}"
|
||||
resolved_project = await resolve_project_parameter(project)
|
||||
if not resolved_project:
|
||||
project_names = await get_project_names(client, headers)
|
||||
raise ValueError(
|
||||
"No project specified. "
|
||||
"Either set 'default_project' in config, or use 'project' argument.\n"
|
||||
f"Available projects: {project_names}"
|
||||
)
|
||||
|
||||
project = resolved_project
|
||||
|
||||
# Check if already cached in context
|
||||
if context:
|
||||
cached_raw = await context.get_state("active_project")
|
||||
if isinstance(cached_raw, dict):
|
||||
cached_project = ProjectItem.model_validate(cached_raw)
|
||||
if cached_project.name == project:
|
||||
logger.debug(f"Using cached project from context: {project}")
|
||||
return cached_project
|
||||
|
||||
# Validate project exists by calling API
|
||||
logger.debug(f"Validating project: {project}")
|
||||
response = await call_post(
|
||||
client,
|
||||
"/v2/projects/resolve",
|
||||
json={"identifier": project},
|
||||
headers=headers,
|
||||
)
|
||||
resolved = ProjectResolveResponse.model_validate(response.json())
|
||||
active_project = ProjectItem(
|
||||
id=resolved.project_id,
|
||||
external_id=resolved.external_id,
|
||||
name=resolved.name,
|
||||
path=resolved.path,
|
||||
is_default=resolved.is_default,
|
||||
)
|
||||
|
||||
project = resolved_project
|
||||
# Cache in context if available
|
||||
if context:
|
||||
await context.set_state("active_project", active_project.model_dump())
|
||||
logger.debug(f"Cached project in context: {project}")
|
||||
|
||||
# Check if already cached in context
|
||||
if context:
|
||||
cached_raw = await context.get_state("active_project")
|
||||
if isinstance(cached_raw, dict):
|
||||
cached_project = ProjectItem.model_validate(cached_raw)
|
||||
if cached_project.name == project:
|
||||
logger.debug(f"Using cached project from context: {project}")
|
||||
return cached_project
|
||||
|
||||
# Validate project exists by calling API
|
||||
logger.debug(f"Validating project: {project}")
|
||||
response = await call_post(
|
||||
client,
|
||||
"/v2/projects/resolve",
|
||||
json={"identifier": project},
|
||||
headers=headers,
|
||||
)
|
||||
resolved = ProjectResolveResponse.model_validate(response.json())
|
||||
active_project = ProjectItem(
|
||||
id=resolved.project_id,
|
||||
external_id=resolved.external_id,
|
||||
name=resolved.name,
|
||||
path=resolved.path,
|
||||
is_default=resolved.is_default,
|
||||
)
|
||||
|
||||
# Cache in context if available
|
||||
if context:
|
||||
await context.set_state("active_project", active_project.model_dump())
|
||||
logger.debug(f"Cached project in context: {project}")
|
||||
|
||||
logger.debug(f"Validated project: {active_project.name}")
|
||||
return active_project
|
||||
logger.debug(f"Validated project: {active_project.name}")
|
||||
return active_project
|
||||
|
||||
|
||||
def _split_project_prefix(path: str) -> tuple[Optional[str], str]:
|
||||
@@ -321,66 +364,77 @@ async def resolve_project_and_path(
|
||||
Tuple of (active_project, normalized_path, is_memory_url)
|
||||
"""
|
||||
is_memory_url = identifier.strip().startswith("memory://")
|
||||
if not is_memory_url:
|
||||
active_project = await get_active_project(client, project, context, headers)
|
||||
return active_project, identifier, False
|
||||
config = ConfigManager().config
|
||||
include_project = config.permalinks_include_project if is_memory_url else None
|
||||
with telemetry.scope(
|
||||
"routing.resolve_memory_url",
|
||||
is_memory_url=is_memory_url,
|
||||
requested_project=project,
|
||||
include_project_prefix=include_project,
|
||||
):
|
||||
if not is_memory_url:
|
||||
active_project = await get_active_project(client, project, context, headers)
|
||||
return active_project, identifier, False
|
||||
|
||||
normalized_path = normalize_project_reference(memory_url_path(identifier))
|
||||
project_prefix, remainder = _split_project_prefix(normalized_path)
|
||||
include_project = ConfigManager().config.permalinks_include_project
|
||||
normalized_path = normalize_project_reference(memory_url_path(identifier))
|
||||
project_prefix, remainder = _split_project_prefix(normalized_path)
|
||||
include_project = config.permalinks_include_project
|
||||
# Trigger: memory URL begins with a potential project segment
|
||||
# Why: allow project-scoped memory URLs without requiring a separate project parameter
|
||||
# Outcome: attempt to resolve the prefix as a project and route to it
|
||||
if project_prefix:
|
||||
try:
|
||||
from basic_memory.mcp.tools.utils import call_post
|
||||
|
||||
# Trigger: memory URL begins with a potential project segment
|
||||
# Why: allow project-scoped memory URLs without requiring a separate project parameter
|
||||
# Outcome: attempt to resolve the prefix as a project and route to it
|
||||
if project_prefix:
|
||||
try:
|
||||
from basic_memory.mcp.tools.utils import call_post
|
||||
|
||||
response = await call_post(
|
||||
client,
|
||||
"/v2/projects/resolve",
|
||||
json={"identifier": project_prefix},
|
||||
headers=headers,
|
||||
)
|
||||
resolved = ProjectResolveResponse.model_validate(response.json())
|
||||
except ToolError as exc:
|
||||
if "project not found" not in str(exc).lower():
|
||||
raise
|
||||
else:
|
||||
resolved_project = await resolve_project_parameter(project_prefix)
|
||||
if resolved_project and generate_permalink(resolved_project) != generate_permalink(
|
||||
project_prefix
|
||||
):
|
||||
raise ValueError(
|
||||
f"Project is constrained to '{resolved_project}', cannot use '{project_prefix}'."
|
||||
response = await call_post(
|
||||
client,
|
||||
"/v2/projects/resolve",
|
||||
json={"identifier": project_prefix},
|
||||
headers=headers,
|
||||
)
|
||||
resolved = ProjectResolveResponse.model_validate(response.json())
|
||||
except ToolError as exc:
|
||||
if "project not found" not in str(exc).lower():
|
||||
raise
|
||||
else:
|
||||
resolved_project = await resolve_project_parameter(project_prefix)
|
||||
if resolved_project and generate_permalink(resolved_project) != generate_permalink(
|
||||
project_prefix
|
||||
):
|
||||
raise ValueError(
|
||||
f"Project is constrained to '{resolved_project}', cannot use '{project_prefix}'."
|
||||
)
|
||||
|
||||
active_project = ProjectItem(
|
||||
id=resolved.project_id,
|
||||
external_id=resolved.external_id,
|
||||
name=resolved.name,
|
||||
path=resolved.path,
|
||||
is_default=resolved.is_default,
|
||||
)
|
||||
if context:
|
||||
await context.set_state("active_project", active_project.model_dump())
|
||||
active_project = ProjectItem(
|
||||
id=resolved.project_id,
|
||||
external_id=resolved.external_id,
|
||||
name=resolved.name,
|
||||
path=resolved.path,
|
||||
is_default=resolved.is_default,
|
||||
)
|
||||
if context:
|
||||
await context.set_state("active_project", active_project.model_dump())
|
||||
|
||||
resolved_path = f"{resolved.permalink}/{remainder}" if include_project else remainder
|
||||
return active_project, resolved_path, True
|
||||
resolved_path = (
|
||||
f"{resolved.permalink}/{remainder}" if include_project else remainder
|
||||
)
|
||||
return active_project, resolved_path, True
|
||||
|
||||
# Trigger: no resolvable project prefix in the memory URL
|
||||
# Why: preserve existing memory URL behavior within the active project
|
||||
# Outcome: use the active project and normalize the path for lookup
|
||||
active_project = await get_active_project(client, project, context, headers)
|
||||
resolved_path = normalized_path
|
||||
if include_project:
|
||||
# Trigger: project-prefixed permalinks are enabled and the path lacks a prefix
|
||||
# Why: ensure memory URL lookups align with canonical permalinks
|
||||
# Outcome: prefix the path with the active project's permalink
|
||||
project_prefix = active_project.permalink
|
||||
if resolved_path != project_prefix and not resolved_path.startswith(f"{project_prefix}/"):
|
||||
resolved_path = f"{project_prefix}/{resolved_path}"
|
||||
return active_project, resolved_path, True
|
||||
# Trigger: no resolvable project prefix in the memory URL
|
||||
# Why: preserve existing memory URL behavior within the active project
|
||||
# Outcome: use the active project and normalize the path for lookup
|
||||
active_project = await get_active_project(client, project, context, headers)
|
||||
resolved_path = normalized_path
|
||||
if include_project:
|
||||
# Trigger: project-prefixed permalinks are enabled and the path lacks a prefix
|
||||
# Why: ensure memory URL lookups align with canonical permalinks
|
||||
# Outcome: prefix the path with the active project's permalink
|
||||
project_prefix = active_project.permalink
|
||||
if resolved_path != project_prefix and not resolved_path.startswith(
|
||||
f"{project_prefix}/"
|
||||
):
|
||||
resolved_path = f"{project_prefix}/{resolved_path}"
|
||||
return active_project, resolved_path, True
|
||||
|
||||
|
||||
def add_project_metadata(result: str, project_name: str) -> str:
|
||||
@@ -494,9 +548,17 @@ async def get_project_client(
|
||||
# control-plane API with no valid credentials and fail with 401
|
||||
# Outcome: use the factory client directly, skip workspace resolution
|
||||
if is_factory_mode():
|
||||
async with get_client() as client:
|
||||
active_project = await get_active_project(client, resolved_project, context)
|
||||
yield client, active_project
|
||||
route_mode = "factory"
|
||||
with telemetry.scope(
|
||||
"routing.client_session",
|
||||
project_name=resolved_project,
|
||||
route_mode=route_mode,
|
||||
workspace_id=workspace,
|
||||
):
|
||||
logger.debug("Using injected client factory for project routing")
|
||||
async with get_client() as client:
|
||||
active_project = await get_active_project(client, resolved_project, context)
|
||||
yield client, active_project
|
||||
return
|
||||
|
||||
# Step 2: Check explicit routing BEFORE workspace resolution
|
||||
@@ -504,9 +566,16 @@ async def get_project_client(
|
||||
# Why: explicit flags must be deterministic — skip workspace entirely for --local
|
||||
# Outcome: route strictly based on explicit flag, no workspace network calls
|
||||
if _explicit_routing() and _force_local_mode():
|
||||
async with get_client(project_name=resolved_project) as client:
|
||||
active_project = await get_active_project(client, resolved_project, context)
|
||||
yield client, active_project
|
||||
route_mode = "explicit_local"
|
||||
with telemetry.scope(
|
||||
"routing.client_session",
|
||||
project_name=resolved_project,
|
||||
route_mode=route_mode,
|
||||
):
|
||||
logger.debug("Explicit local routing selected for project client")
|
||||
async with get_client(project_name=resolved_project) as client:
|
||||
active_project = await get_active_project(client, resolved_project, context)
|
||||
yield client, active_project
|
||||
return
|
||||
|
||||
# Step 3: Determine if cloud routing is needed
|
||||
@@ -535,28 +604,51 @@ async def get_project_client(
|
||||
if effective_workspace is None and config.default_workspace:
|
||||
effective_workspace = config.default_workspace
|
||||
|
||||
route_mode = "cloud_proxy"
|
||||
|
||||
# Priorities 4-6: if still unresolved, fall back to resolve_workspace_parameter
|
||||
# which checks context cache, auto-selects single workspace, or errors
|
||||
if effective_workspace is not None:
|
||||
# Config-resolved workspace — pass directly to get_client, skip network lookup
|
||||
async with get_client(
|
||||
with telemetry.scope(
|
||||
"routing.client_session",
|
||||
project_name=resolved_project,
|
||||
workspace=effective_workspace,
|
||||
) as client:
|
||||
active_project = await get_active_project(client, resolved_project, context)
|
||||
yield client, active_project
|
||||
route_mode=route_mode,
|
||||
workspace_id=effective_workspace,
|
||||
):
|
||||
logger.debug("Using configured workspace for cloud project routing")
|
||||
async with get_client(
|
||||
project_name=resolved_project,
|
||||
workspace=effective_workspace,
|
||||
) as client:
|
||||
active_project = await get_active_project(client, resolved_project, context)
|
||||
yield client, active_project
|
||||
else:
|
||||
# No config-based workspace — use resolve_workspace_parameter for discovery
|
||||
active_ws = await resolve_workspace_parameter(workspace=None, context=context)
|
||||
async with get_client(
|
||||
with telemetry.scope(
|
||||
"routing.client_session",
|
||||
project_name=resolved_project,
|
||||
workspace=active_ws.tenant_id,
|
||||
) as client:
|
||||
active_project = await get_active_project(client, resolved_project, context)
|
||||
yield client, active_project
|
||||
route_mode=route_mode,
|
||||
workspace_id=active_ws.tenant_id,
|
||||
):
|
||||
logger.debug("Resolved workspace dynamically for cloud project routing")
|
||||
async with get_client(
|
||||
project_name=resolved_project,
|
||||
workspace=active_ws.tenant_id,
|
||||
) as client:
|
||||
active_project = await get_active_project(client, resolved_project, context)
|
||||
yield client, active_project
|
||||
return
|
||||
|
||||
# Step 4: Local routing (default)
|
||||
async with get_client(project_name=resolved_project) as client:
|
||||
active_project = await get_active_project(client, resolved_project, context)
|
||||
yield client, active_project
|
||||
route_mode = "local_asgi"
|
||||
with telemetry.scope(
|
||||
"routing.client_session",
|
||||
project_name=resolved_project,
|
||||
route_mode=route_mode,
|
||||
):
|
||||
logger.debug("Using default local ASGI routing for project client")
|
||||
async with get_client(project_name=resolved_project) as client:
|
||||
active_project = await get_active_project(client, resolved_project, context)
|
||||
yield client, active_project
|
||||
|
||||
+130
-44
@@ -2,16 +2,70 @@
|
||||
Basic Memory FastMCP server.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import time
|
||||
from contextlib import asynccontextmanager
|
||||
|
||||
from fastmcp import FastMCP
|
||||
from loguru import logger
|
||||
from sqlalchemy import text
|
||||
from sqlalchemy.ext.asyncio import async_sessionmaker, AsyncSession
|
||||
|
||||
from basic_memory import db
|
||||
from basic_memory.cli.auth import CLIAuth
|
||||
from basic_memory.config import BasicMemoryConfig
|
||||
from basic_memory.db import (
|
||||
scoped_session,
|
||||
_needs_semantic_embedding_backfill,
|
||||
_run_semantic_embedding_backfill,
|
||||
)
|
||||
from basic_memory.mcp.container import McpContainer, set_container
|
||||
from basic_memory.services.initialization import initialize_app
|
||||
from basic_memory import telemetry
|
||||
|
||||
|
||||
async def _log_embedding_status(session_maker: async_sessionmaker[AsyncSession]) -> None:
|
||||
"""Log a clear summary of semantic embedding status at startup."""
|
||||
try:
|
||||
async with scoped_session(session_maker) as session:
|
||||
entity_count = (
|
||||
await session.execute(text("SELECT COUNT(*) FROM entity"))
|
||||
).scalar() or 0
|
||||
chunk_count = (
|
||||
await session.execute(text("SELECT COUNT(*) FROM search_vector_chunks"))
|
||||
).scalar() or 0
|
||||
embedding_count = (
|
||||
await session.execute(text("SELECT COUNT(*) FROM search_vector_embeddings_rowids"))
|
||||
).scalar() or 0
|
||||
|
||||
if entity_count == 0:
|
||||
logger.info("Semantic embeddings: no entities yet")
|
||||
elif embedding_count == 0:
|
||||
logger.warning(
|
||||
f"Semantic embeddings: EMPTY — {entity_count} entities have no embeddings. "
|
||||
"Backfill running in background..."
|
||||
)
|
||||
else:
|
||||
logger.info(
|
||||
f"Semantic embeddings: {embedding_count} embeddings "
|
||||
f"across {chunk_count} chunks for {entity_count} entities"
|
||||
)
|
||||
except Exception as exc:
|
||||
logger.debug(f"Could not check embedding status at startup: {exc}")
|
||||
|
||||
|
||||
async def _background_embedding_backfill(
|
||||
config: BasicMemoryConfig,
|
||||
session_maker: async_sessionmaker[AsyncSession],
|
||||
) -> None:
|
||||
"""Run semantic embedding backfill in the background without blocking startup."""
|
||||
try:
|
||||
if await _needs_semantic_embedding_backfill(config, session_maker):
|
||||
logger.info("Background embedding backfill starting...")
|
||||
await _run_semantic_embedding_backfill(config, session_maker)
|
||||
await _log_embedding_status(session_maker)
|
||||
except Exception as exc:
|
||||
logger.error(f"Background embedding backfill failed: {exc}")
|
||||
|
||||
|
||||
@asynccontextmanager
|
||||
@@ -29,64 +83,96 @@ async def lifespan(app: FastMCP):
|
||||
set_container(container)
|
||||
|
||||
config = container.config
|
||||
logger.info(f"Starting Basic Memory MCP server (mode={container.mode.name})")
|
||||
logger.info(
|
||||
f"Config: database_backend={config.database_backend.value}, "
|
||||
f"semantic_search_enabled={config.semantic_search_enabled}, "
|
||||
f"default_project={config.default_project}"
|
||||
)
|
||||
if config.semantic_search_enabled:
|
||||
with telemetry.operation(
|
||||
"mcp.lifecycle.startup",
|
||||
entrypoint="mcp",
|
||||
mode=container.mode.name.lower(),
|
||||
default_project=config.default_project,
|
||||
):
|
||||
logger.info(f"Starting Basic Memory MCP server (mode={container.mode.name})")
|
||||
logger.info(
|
||||
f"Semantic search: provider={config.semantic_embedding_provider}, "
|
||||
f"model={config.semantic_embedding_model}, "
|
||||
f"dimensions={config.semantic_embedding_dimensions or 'auto'}, "
|
||||
f"batch_size={config.semantic_embedding_batch_size}"
|
||||
f"Config: database_backend={config.database_backend.value}, "
|
||||
f"semantic_search_enabled={config.semantic_search_enabled}, "
|
||||
f"default_project={config.default_project}"
|
||||
)
|
||||
if config.semantic_search_enabled:
|
||||
logger.info(
|
||||
f"Semantic search: provider={config.semantic_embedding_provider}, "
|
||||
f"model={config.semantic_embedding_model}, "
|
||||
f"dimensions={config.semantic_embedding_dimensions or 'auto'}, "
|
||||
f"batch_size={config.semantic_embedding_batch_size}"
|
||||
)
|
||||
|
||||
# Log configured projects with their routing mode
|
||||
for name, entry in config.projects.items():
|
||||
default = " (default)" if name == config.default_project else ""
|
||||
logger.info(f"Project: {name} -> {entry.path} [mode={entry.mode.value}]{default}")
|
||||
# Log configured projects with their routing mode
|
||||
for name, entry in config.projects.items():
|
||||
default = " (default)" if name == config.default_project else ""
|
||||
logger.info(f"Project: {name} -> {entry.path} [mode={entry.mode.value}]{default}")
|
||||
|
||||
# Check cloud auth status (local file check, no network call)
|
||||
auth = CLIAuth(client_id=config.cloud_client_id, authkit_domain=config.cloud_domain)
|
||||
tokens = auth.load_tokens()
|
||||
if tokens is not None:
|
||||
if not auth.is_token_valid(tokens):
|
||||
expires_at = tokens.get("expires_at", 0)
|
||||
expired_ago = int(time.time() - expires_at)
|
||||
logger.warning(f"Cloud token expired {expired_ago}s ago - may need 'bm cloud login'")
|
||||
else:
|
||||
logger.info("Cloud: authenticated (OAuth token valid)")
|
||||
# Check cloud auth status (local file check, no network call)
|
||||
auth = CLIAuth(client_id=config.cloud_client_id, authkit_domain=config.cloud_domain)
|
||||
tokens = auth.load_tokens()
|
||||
if tokens is not None:
|
||||
if not auth.is_token_valid(tokens):
|
||||
expires_at = tokens.get("expires_at", 0)
|
||||
expired_ago = int(time.time() - expires_at)
|
||||
logger.warning(
|
||||
f"Cloud token expired {expired_ago}s ago - may need 'bm cloud login'"
|
||||
)
|
||||
else:
|
||||
logger.info("Cloud: authenticated (OAuth token valid)")
|
||||
|
||||
if config.cloud_api_key:
|
||||
logger.info("Cloud: API key configured")
|
||||
if config.cloud_api_key:
|
||||
logger.info("Cloud: API key configured")
|
||||
|
||||
# Track if we created the engine (vs test fixtures providing it)
|
||||
# This prevents disposing an engine provided by test fixtures when
|
||||
# multiple Client connections are made in the same test
|
||||
engine_was_none = db._engine is None
|
||||
# Track if we created the engine (vs test fixtures providing it)
|
||||
# This prevents disposing an engine provided by test fixtures when
|
||||
# multiple Client connections are made in the same test
|
||||
engine_was_none = db._engine is None
|
||||
|
||||
# Initialize app (runs migrations, reconciles projects)
|
||||
await initialize_app(container.config)
|
||||
# Initialize app (runs migrations, reconciles projects)
|
||||
await initialize_app(container.config)
|
||||
|
||||
# Create and start sync coordinator (lifecycle centralized in coordinator)
|
||||
sync_coordinator = container.create_sync_coordinator()
|
||||
await sync_coordinator.start()
|
||||
# Log embedding status so it's easy to spot in the logs
|
||||
backfill_task: asyncio.Task | None = None # type: ignore[type-arg]
|
||||
if config.semantic_search_enabled and db._session_maker is not None:
|
||||
await _log_embedding_status(db._session_maker)
|
||||
# Launch backfill in background so MCP server is ready immediately
|
||||
backfill_task = asyncio.create_task(
|
||||
_background_embedding_backfill(config, db._session_maker),
|
||||
name="embedding-backfill",
|
||||
)
|
||||
|
||||
# Create and start sync coordinator (lifecycle centralized in coordinator)
|
||||
sync_coordinator = container.create_sync_coordinator()
|
||||
await sync_coordinator.start()
|
||||
|
||||
try:
|
||||
yield
|
||||
finally:
|
||||
# Shutdown - coordinator handles clean task cancellation
|
||||
logger.debug("Shutting down Basic Memory MCP server")
|
||||
await sync_coordinator.stop()
|
||||
with telemetry.operation(
|
||||
"mcp.lifecycle.shutdown",
|
||||
entrypoint="mcp",
|
||||
mode=container.mode.name.lower(),
|
||||
):
|
||||
logger.debug("Shutting down Basic Memory MCP server")
|
||||
|
||||
# Only shutdown DB if we created it (not if test fixture provided it)
|
||||
if engine_was_none:
|
||||
await db.shutdown_db()
|
||||
logger.debug("Database connections closed")
|
||||
else: # pragma: no cover
|
||||
logger.debug("Skipping DB shutdown - engine provided externally")
|
||||
# Cancel embedding backfill if still running
|
||||
if backfill_task is not None and not backfill_task.done():
|
||||
backfill_task.cancel()
|
||||
try:
|
||||
await backfill_task
|
||||
except asyncio.CancelledError:
|
||||
logger.info("Background embedding backfill cancelled during shutdown")
|
||||
|
||||
await sync_coordinator.stop()
|
||||
|
||||
# Only shutdown DB if we created it (not if test fixture provided it)
|
||||
if engine_was_none:
|
||||
await db.shutdown_db()
|
||||
logger.debug("Database connections closed")
|
||||
else: # pragma: no cover
|
||||
logger.debug("Skipping DB shutdown - engine provided externally")
|
||||
|
||||
|
||||
mcp = FastMCP(
|
||||
|
||||
@@ -24,16 +24,6 @@ from basic_memory.mcp.tools.list_directory import list_directory
|
||||
from basic_memory.mcp.tools.edit_note import edit_note
|
||||
from basic_memory.mcp.tools.move_note import move_note
|
||||
from basic_memory.mcp.tools.workspaces import list_workspaces
|
||||
from basic_memory.mcp.tools.graph_intelligence import (
|
||||
graph_lineage,
|
||||
graph_impact,
|
||||
graph_health,
|
||||
graph_reindex,
|
||||
fcm_simulate,
|
||||
fcm_rank_actions,
|
||||
fcm_import_model,
|
||||
fcm_export_model,
|
||||
)
|
||||
from basic_memory.mcp.tools.project_management import (
|
||||
list_memory_projects,
|
||||
create_memory_project,
|
||||
@@ -54,15 +44,7 @@ __all__ = [
|
||||
"delete_note",
|
||||
"delete_project",
|
||||
"edit_note",
|
||||
"fcm_export_model",
|
||||
"fcm_import_model",
|
||||
"fcm_rank_actions",
|
||||
"fcm_simulate",
|
||||
"fetch",
|
||||
"graph_health",
|
||||
"graph_impact",
|
||||
"graph_lineage",
|
||||
"graph_reindex",
|
||||
"list_directory",
|
||||
"list_memory_projects",
|
||||
"list_workspaces",
|
||||
|
||||
@@ -6,6 +6,7 @@ from loguru import logger
|
||||
from fastmcp import Context
|
||||
|
||||
from basic_memory.config import ConfigManager
|
||||
from basic_memory import telemetry
|
||||
from basic_memory.mcp.project_context import (
|
||||
detect_project_from_url_prefix,
|
||||
get_project_client,
|
||||
@@ -190,8 +191,6 @@ async def build_context(
|
||||
if detected:
|
||||
project = detected
|
||||
|
||||
logger.info(f"Building context from {url} in project {project}")
|
||||
|
||||
# Convert string depth to integer if needed
|
||||
if isinstance(depth, str):
|
||||
try:
|
||||
@@ -203,25 +202,62 @@ async def build_context(
|
||||
|
||||
# URL is already validated and normalized by MemoryUrl type annotation
|
||||
|
||||
async with get_project_client(project, workspace, context) as (client, active_project):
|
||||
# Resolve memory:// identifier with project-prefix awareness
|
||||
_, resolved_path, _ = await resolve_project_and_path(client, url, project, context)
|
||||
with telemetry.operation(
|
||||
"mcp.tool.build_context",
|
||||
entrypoint="mcp",
|
||||
tool_name="build_context",
|
||||
requested_project=project,
|
||||
workspace_id=workspace,
|
||||
depth=depth or 1,
|
||||
timeframe=timeframe,
|
||||
page=page,
|
||||
page_size=page_size,
|
||||
max_related=max_related,
|
||||
output_format=output_format,
|
||||
is_memory_url=str(url).startswith("memory://"),
|
||||
):
|
||||
async with get_project_client(project, workspace, context) as (client, active_project):
|
||||
with telemetry.contextualize(
|
||||
project_name=active_project.name,
|
||||
workspace_id=workspace,
|
||||
tool_name="build_context",
|
||||
):
|
||||
logger.info(
|
||||
f"MCP tool call tool=build_context project={active_project.name} "
|
||||
f"url={url} depth={depth} timeframe={timeframe} output_format={output_format}"
|
||||
)
|
||||
|
||||
# Import here to avoid circular import
|
||||
from basic_memory.mcp.clients import MemoryClient
|
||||
# Resolve memory:// identifier with project-prefix awareness
|
||||
_, resolved_path, _ = await resolve_project_and_path(
|
||||
client,
|
||||
url,
|
||||
active_project.name,
|
||||
context,
|
||||
)
|
||||
|
||||
# Use typed MemoryClient for API calls
|
||||
memory_client = MemoryClient(client, active_project.external_id)
|
||||
graph = await memory_client.build_context(
|
||||
resolved_path,
|
||||
depth=depth or 1,
|
||||
timeframe=timeframe,
|
||||
page=page,
|
||||
page_size=page_size,
|
||||
max_related=max_related,
|
||||
)
|
||||
# Import here to avoid circular import
|
||||
from basic_memory.mcp.clients import MemoryClient
|
||||
|
||||
if output_format == "text":
|
||||
return _format_context_markdown(graph, active_project.name)
|
||||
# Use typed MemoryClient for API calls
|
||||
memory_client = MemoryClient(client, active_project.external_id)
|
||||
graph = await memory_client.build_context(
|
||||
resolved_path,
|
||||
depth=depth or 1,
|
||||
timeframe=timeframe,
|
||||
page=page,
|
||||
page_size=page_size,
|
||||
max_related=max_related,
|
||||
)
|
||||
|
||||
return graph.model_dump()
|
||||
logger.info(
|
||||
f"MCP tool response: tool=build_context project={active_project.name} "
|
||||
f"uri={graph.metadata.uri or resolved_path} "
|
||||
f"primary_count={graph.metadata.primary_count or 0} "
|
||||
f"related_count={graph.metadata.related_count or 0} "
|
||||
f"output_format={output_format}"
|
||||
)
|
||||
|
||||
if output_format == "text":
|
||||
return _format_context_markdown(graph, active_project.name)
|
||||
|
||||
return graph.model_dump()
|
||||
|
||||
@@ -4,12 +4,14 @@ This tool creates Obsidian canvas files (.canvas) using the JSON Canvas 1.0 spec
|
||||
"""
|
||||
|
||||
import json
|
||||
from typing import Dict, List, Any, Optional
|
||||
from typing import Annotated, Dict, List, Any, Optional
|
||||
|
||||
from loguru import logger
|
||||
from fastmcp import Context
|
||||
from pydantic import BeforeValidator
|
||||
|
||||
from basic_memory.mcp.project_context import get_project_client
|
||||
from basic_memory.utils import coerce_list
|
||||
from basic_memory.mcp.server import mcp
|
||||
from basic_memory.mcp.tools.utils import call_put, call_post, resolve_entity_id
|
||||
|
||||
@@ -19,8 +21,8 @@ from basic_memory.mcp.tools.utils import call_put, call_post, resolve_entity_id
|
||||
annotations={"destructiveHint": False, "idempotentHint": True, "openWorldHint": False},
|
||||
)
|
||||
async def canvas(
|
||||
nodes: List[Dict[str, Any]],
|
||||
edges: List[Dict[str, Any]],
|
||||
nodes: Annotated[List[Dict[str, Any]], BeforeValidator(coerce_list)],
|
||||
edges: Annotated[List[Dict[str, Any]], BeforeValidator(coerce_list)],
|
||||
title: str,
|
||||
directory: str,
|
||||
project: Optional[str] = None,
|
||||
|
||||
@@ -5,7 +5,8 @@ from loguru import logger
|
||||
from fastmcp import Context
|
||||
from mcp.server.fastmcp.exceptions import ToolError
|
||||
|
||||
from basic_memory.mcp.project_context import get_project_client
|
||||
from basic_memory.config import ConfigManager
|
||||
from basic_memory.mcp.project_context import detect_project_from_url_prefix, get_project_client
|
||||
from basic_memory.mcp.server import mcp
|
||||
|
||||
|
||||
@@ -222,6 +223,16 @@ async def delete_note(
|
||||
with suggestions for finding the correct identifier, including search
|
||||
commands and alternative formats to try.
|
||||
"""
|
||||
# Detect project from memory URL prefix before routing
|
||||
# Trigger: identifier starts with memory:// and no explicit project was provided
|
||||
# Why: only gate on memory:// to avoid misrouting plain paths like "research/note"
|
||||
# where "research" is a directory, not a project name
|
||||
# Outcome: project is set from the URL prefix, routing goes to the correct project
|
||||
if project is None and identifier.strip().startswith("memory://"):
|
||||
detected = detect_project_from_url_prefix(identifier, ConfigManager().config)
|
||||
if detected:
|
||||
project = detected
|
||||
|
||||
async with get_project_client(project, workspace, context) as (client, active_project):
|
||||
logger.debug(
|
||||
f"Deleting {'directory' if is_directory else 'note'}: {identifier} in project: {active_project.name}"
|
||||
@@ -318,7 +329,7 @@ delete_note("path/to/file.md")
|
||||
note_file_path = None
|
||||
try:
|
||||
# Resolve identifier to entity ID
|
||||
entity_id = await knowledge_client.resolve_entity(identifier)
|
||||
entity_id = await knowledge_client.resolve_entity(identifier, strict=True)
|
||||
if output_format == "json":
|
||||
entity = await knowledge_client.get_entity(entity_id)
|
||||
note_title = entity.title
|
||||
|
||||
@@ -5,7 +5,13 @@ from typing import Optional, Literal
|
||||
from loguru import logger
|
||||
from fastmcp import Context
|
||||
|
||||
from basic_memory.mcp.project_context import get_project_client, add_project_metadata
|
||||
from basic_memory.config import ConfigManager
|
||||
from basic_memory import telemetry
|
||||
from basic_memory.mcp.project_context import (
|
||||
detect_project_from_url_prefix,
|
||||
get_project_client,
|
||||
add_project_metadata,
|
||||
)
|
||||
from basic_memory.mcp.server import mcp
|
||||
from basic_memory.schemas.base import Entity
|
||||
from basic_memory.schemas.response import EntityResponse
|
||||
@@ -158,7 +164,7 @@ Error editing note '{identifier}': {error_message}
|
||||
|
||||
|
||||
@mcp.tool(
|
||||
description="Edit an existing markdown note using various operations like append, prepend, find_replace, or replace_section.",
|
||||
description="Edit an existing markdown note using various operations like append, prepend, find_replace, replace_section, insert_before_section, or insert_after_section.",
|
||||
annotations={"destructiveHint": False, "openWorldHint": False},
|
||||
)
|
||||
async def edit_note(
|
||||
@@ -190,6 +196,8 @@ async def edit_note(
|
||||
- "prepend": Add content to the beginning of the note (creates the note if it doesn't exist)
|
||||
- "find_replace": Replace occurrences of find_text with content (note must exist)
|
||||
- "replace_section": Replace content under a specific markdown header (note must exist)
|
||||
- "insert_before_section": Insert content before a section heading without consuming it (note must exist)
|
||||
- "insert_after_section": Insert content after a section heading without consuming it (note must exist)
|
||||
content: The content to add or use for replacement
|
||||
project: Project name to edit in. Optional - server will resolve using hierarchy.
|
||||
If unknown, use list_memory_projects() to discover available projects.
|
||||
@@ -253,206 +261,253 @@ async def edit_note(
|
||||
# Resolve effective default: allow MCP clients to send null for optional int field
|
||||
effective_replacements = expected_replacements if expected_replacements is not None else 1
|
||||
|
||||
async with get_project_client(project, workspace, context) as (client, active_project):
|
||||
logger.info("MCP tool call", tool="edit_note", identifier=identifier, operation=operation)
|
||||
# Detect project from memory URL prefix before routing
|
||||
# Trigger: identifier starts with memory:// and no explicit project was provided
|
||||
# Why: only gate on memory:// to avoid misrouting plain paths like "research/note"
|
||||
# where "research" is a directory, not a project name
|
||||
# Outcome: project is set from the URL prefix, routing goes to the correct project
|
||||
if project is None and identifier.strip().startswith("memory://"):
|
||||
detected = detect_project_from_url_prefix(identifier, ConfigManager().config)
|
||||
if detected:
|
||||
project = detected
|
||||
|
||||
# Validate operation
|
||||
valid_operations = ["append", "prepend", "find_replace", "replace_section"]
|
||||
if operation not in valid_operations:
|
||||
raise ValueError(
|
||||
f"Invalid operation '{operation}'. Must be one of: {', '.join(valid_operations)}"
|
||||
)
|
||||
with telemetry.operation(
|
||||
"mcp.tool.edit_note",
|
||||
entrypoint="mcp",
|
||||
tool_name="edit_note",
|
||||
requested_project=project,
|
||||
workspace_id=workspace,
|
||||
edit_operation=operation,
|
||||
output_format=output_format,
|
||||
has_section=bool(section),
|
||||
has_find_text=bool(find_text),
|
||||
expected_replacements=effective_replacements,
|
||||
):
|
||||
async with get_project_client(project, workspace, context) as (client, active_project):
|
||||
with telemetry.contextualize(
|
||||
project_name=active_project.name,
|
||||
workspace_id=workspace,
|
||||
tool_name="edit_note",
|
||||
):
|
||||
logger.info(
|
||||
f"MCP tool call tool=edit_note project={active_project.name} "
|
||||
f"identifier={identifier} operation={operation} output_format={output_format}"
|
||||
)
|
||||
|
||||
# Validate required parameters for specific operations
|
||||
if operation == "find_replace" and not find_text:
|
||||
raise ValueError("find_text parameter is required for find_replace operation")
|
||||
if operation == "replace_section" and not section:
|
||||
raise ValueError("section parameter is required for replace_section operation")
|
||||
|
||||
# Use the PATCH endpoint to edit the entity
|
||||
try:
|
||||
# Import here to avoid circular import
|
||||
from basic_memory.mcp.clients import KnowledgeClient
|
||||
|
||||
# Use typed KnowledgeClient for API calls
|
||||
knowledge_client = KnowledgeClient(client, active_project.external_id)
|
||||
|
||||
file_created = False
|
||||
entity_id = ""
|
||||
result: EntityResponse | None = None
|
||||
|
||||
# Try to resolve the entity; for append/prepend, create it if not found
|
||||
try:
|
||||
entity_id = await knowledge_client.resolve_entity(identifier)
|
||||
except Exception as resolve_error:
|
||||
# Trigger: entity does not exist yet
|
||||
# Why: append/prepend can meaningfully create a new note from the content,
|
||||
# while find_replace/replace_section require existing content to modify
|
||||
# Outcome: note is created via the same path as write_note
|
||||
error_msg = str(resolve_error).lower()
|
||||
is_not_found = "entity not found" in error_msg or "not found" in error_msg
|
||||
|
||||
if is_not_found and operation in ("append", "prepend"):
|
||||
title, directory = _parse_identifier_to_title_and_directory(identifier)
|
||||
|
||||
# Validate directory path (same security check as write_note)
|
||||
project_path = active_project.home
|
||||
if directory and not validate_project_path(directory, project_path):
|
||||
logger.warning(
|
||||
"Attempted path traversal attack blocked",
|
||||
directory=directory,
|
||||
project=active_project.name,
|
||||
)
|
||||
if output_format == "json":
|
||||
return {
|
||||
"title": title,
|
||||
"permalink": None,
|
||||
"file_path": None,
|
||||
"checksum": None,
|
||||
"operation": operation,
|
||||
"fileCreated": False,
|
||||
"error": "SECURITY_VALIDATION_ERROR",
|
||||
}
|
||||
return f"# Error\n\nDirectory path '{directory}' is not allowed - paths must stay within project boundaries"
|
||||
|
||||
entity = Entity(
|
||||
title=title,
|
||||
directory=directory,
|
||||
content_type="text/markdown",
|
||||
content=content,
|
||||
# Validate operation
|
||||
valid_operations = [
|
||||
"append",
|
||||
"prepend",
|
||||
"find_replace",
|
||||
"replace_section",
|
||||
"insert_before_section",
|
||||
"insert_after_section",
|
||||
]
|
||||
if operation not in valid_operations:
|
||||
raise ValueError(
|
||||
f"Invalid operation '{operation}'. Must be one of: {', '.join(valid_operations)}"
|
||||
)
|
||||
|
||||
# Validate required parameters for specific operations
|
||||
if operation == "find_replace" and not find_text:
|
||||
raise ValueError("find_text parameter is required for find_replace operation")
|
||||
section_ops = ("replace_section", "insert_before_section", "insert_after_section")
|
||||
if operation in section_ops and not section:
|
||||
raise ValueError("section parameter is required for section-based operations")
|
||||
|
||||
# Use the PATCH endpoint to edit the entity
|
||||
try:
|
||||
# Import here to avoid circular import
|
||||
from basic_memory.mcp.clients import KnowledgeClient
|
||||
|
||||
# Use typed KnowledgeClient for API calls
|
||||
knowledge_client = KnowledgeClient(client, active_project.external_id)
|
||||
|
||||
file_created = False
|
||||
entity_id = ""
|
||||
result: EntityResponse | None = None
|
||||
|
||||
# Try to resolve the entity; for append/prepend, create it if not found
|
||||
try:
|
||||
entity_id = await knowledge_client.resolve_entity(identifier, strict=True)
|
||||
except Exception as resolve_error:
|
||||
# Trigger: entity does not exist yet
|
||||
# Why: append/prepend can meaningfully create a new note from the content,
|
||||
# while find_replace/replace_section require existing content to modify
|
||||
# Outcome: note is created via the same path as write_note
|
||||
error_msg = str(resolve_error).lower()
|
||||
is_not_found = "entity not found" in error_msg or "not found" in error_msg
|
||||
|
||||
if is_not_found and operation in ("append", "prepend"):
|
||||
title, directory = _parse_identifier_to_title_and_directory(identifier)
|
||||
|
||||
# Validate directory path (same security check as write_note)
|
||||
project_path = active_project.home
|
||||
if directory and not validate_project_path(directory, project_path):
|
||||
logger.warning(
|
||||
"Attempted path traversal attack blocked",
|
||||
directory=directory,
|
||||
project=active_project.name,
|
||||
)
|
||||
if output_format == "json":
|
||||
return {
|
||||
"title": title,
|
||||
"permalink": None,
|
||||
"file_path": None,
|
||||
"checksum": None,
|
||||
"operation": operation,
|
||||
"fileCreated": False,
|
||||
"error": "SECURITY_VALIDATION_ERROR",
|
||||
}
|
||||
return f"# Error\n\nDirectory path '{directory}' is not allowed - paths must stay within project boundaries"
|
||||
|
||||
entity = Entity(
|
||||
title=title,
|
||||
directory=directory,
|
||||
content_type="text/markdown",
|
||||
content=content,
|
||||
)
|
||||
|
||||
logger.info(
|
||||
"Creating note via edit_note auto-create",
|
||||
title=title,
|
||||
directory=directory,
|
||||
operation=operation,
|
||||
)
|
||||
result = await knowledge_client.create_entity(
|
||||
entity.model_dump(), fast=False
|
||||
)
|
||||
file_created = True
|
||||
else:
|
||||
# find_replace/replace_section require existing content — re-raise
|
||||
raise resolve_error
|
||||
|
||||
# --- Standard edit path (entity already existed) ---
|
||||
if not file_created:
|
||||
# Prepare the edit request data
|
||||
edit_data = {
|
||||
"operation": operation,
|
||||
"content": content,
|
||||
}
|
||||
|
||||
# Add optional parameters
|
||||
if section:
|
||||
edit_data["section"] = section
|
||||
if find_text:
|
||||
edit_data["find_text"] = find_text
|
||||
if effective_replacements != 1: # Only send if different from default
|
||||
edit_data["expected_replacements"] = str(effective_replacements)
|
||||
|
||||
# Call the PATCH endpoint
|
||||
result = await knowledge_client.patch_entity(
|
||||
entity_id, edit_data, fast=False
|
||||
)
|
||||
|
||||
# --- Format response ---
|
||||
# result is always set: either by create_entity (auto-create) or patch_entity (edit)
|
||||
assert result is not None
|
||||
if file_created:
|
||||
summary = [
|
||||
f"# Created note ({operation})",
|
||||
f"project: {active_project.name}",
|
||||
f"file_path: {result.file_path}",
|
||||
f"permalink: {result.permalink}",
|
||||
f"checksum: {result.checksum[:8] if result.checksum else 'unknown'}",
|
||||
"fileCreated: true",
|
||||
]
|
||||
lines_added = len(content.split("\n"))
|
||||
summary.append(f"operation: Created note with {lines_added} lines")
|
||||
else:
|
||||
summary = [
|
||||
f"# Edited note ({operation})",
|
||||
f"project: {active_project.name}",
|
||||
f"file_path: {result.file_path}",
|
||||
f"permalink: {result.permalink}",
|
||||
f"checksum: {result.checksum[:8] if result.checksum else 'unknown'}",
|
||||
]
|
||||
|
||||
# Add operation-specific details
|
||||
if operation == "append":
|
||||
lines_added = len(content.split("\n"))
|
||||
summary.append(f"operation: Added {lines_added} lines to end of note")
|
||||
elif operation == "prepend":
|
||||
lines_added = len(content.split("\n"))
|
||||
summary.append(
|
||||
f"operation: Added {lines_added} lines to beginning of note"
|
||||
)
|
||||
elif operation == "find_replace":
|
||||
# For find_replace, we can't easily count replacements from here
|
||||
# since we don't have the original content, but the server handled it
|
||||
summary.append("operation: Find and replace operation completed")
|
||||
elif operation == "replace_section":
|
||||
summary.append(f"operation: Replaced content under section '{section}'")
|
||||
elif operation == "insert_before_section":
|
||||
summary.append(
|
||||
f"operation: Inserted content before section '{section}'"
|
||||
)
|
||||
elif operation == "insert_after_section":
|
||||
summary.append(f"operation: Inserted content after section '{section}'")
|
||||
|
||||
# Count observations by category (reuse logic from write_note)
|
||||
categories = {}
|
||||
if result.observations:
|
||||
for obs in result.observations:
|
||||
categories[obs.category] = categories.get(obs.category, 0) + 1
|
||||
|
||||
summary.append("\n## Observations")
|
||||
for category, count in sorted(categories.items()):
|
||||
summary.append(f"- {category}: {count}")
|
||||
|
||||
# Count resolved/unresolved relations
|
||||
unresolved = 0
|
||||
resolved = 0
|
||||
if result.relations:
|
||||
unresolved = sum(1 for r in result.relations if not r.to_id)
|
||||
resolved = len(result.relations) - unresolved
|
||||
|
||||
summary.append("\n## Relations")
|
||||
summary.append(f"- Resolved: {resolved}")
|
||||
if unresolved:
|
||||
summary.append(f"- Unresolved: {unresolved}")
|
||||
|
||||
logger.info(
|
||||
"Creating note via edit_note auto-create",
|
||||
title=title,
|
||||
directory=directory,
|
||||
operation=operation,
|
||||
f"MCP tool response: tool=edit_note project={active_project.name} "
|
||||
f"operation={operation} permalink={result.permalink} "
|
||||
f"observations_count={len(result.observations)} "
|
||||
f"relations_count={len(result.relations)} "
|
||||
f"file_created={str(file_created).lower()}"
|
||||
)
|
||||
result = await knowledge_client.create_entity(entity.model_dump(), fast=False)
|
||||
file_created = True
|
||||
else:
|
||||
# find_replace/replace_section require existing content — re-raise
|
||||
raise resolve_error
|
||||
|
||||
# --- Standard edit path (entity already existed) ---
|
||||
if not file_created:
|
||||
# Prepare the edit request data
|
||||
edit_data = {
|
||||
"operation": operation,
|
||||
"content": content,
|
||||
}
|
||||
if output_format == "json":
|
||||
return {
|
||||
"title": result.title,
|
||||
"permalink": result.permalink,
|
||||
"file_path": result.file_path,
|
||||
"checksum": result.checksum,
|
||||
"operation": operation,
|
||||
"fileCreated": file_created,
|
||||
}
|
||||
|
||||
# Add optional parameters
|
||||
if section:
|
||||
edit_data["section"] = section
|
||||
if find_text:
|
||||
edit_data["find_text"] = find_text
|
||||
if effective_replacements != 1: # Only send if different from default
|
||||
edit_data["expected_replacements"] = str(effective_replacements)
|
||||
summary_result = "\n".join(summary)
|
||||
return add_project_metadata(summary_result, active_project.name)
|
||||
|
||||
# Call the PATCH endpoint
|
||||
result = await knowledge_client.patch_entity(entity_id, edit_data, fast=False)
|
||||
|
||||
# --- Format response ---
|
||||
# result is always set: either by create_entity (auto-create) or patch_entity (edit)
|
||||
assert result is not None
|
||||
if file_created:
|
||||
summary = [
|
||||
f"# Created note ({operation})",
|
||||
f"project: {active_project.name}",
|
||||
f"file_path: {result.file_path}",
|
||||
f"permalink: {result.permalink}",
|
||||
f"checksum: {result.checksum[:8] if result.checksum else 'unknown'}",
|
||||
"fileCreated: true",
|
||||
]
|
||||
lines_added = len(content.split("\n"))
|
||||
summary.append(f"operation: Created note with {lines_added} lines")
|
||||
else:
|
||||
summary = [
|
||||
f"# Edited note ({operation})",
|
||||
f"project: {active_project.name}",
|
||||
f"file_path: {result.file_path}",
|
||||
f"permalink: {result.permalink}",
|
||||
f"checksum: {result.checksum[:8] if result.checksum else 'unknown'}",
|
||||
]
|
||||
|
||||
# Add operation-specific details
|
||||
if operation == "append":
|
||||
lines_added = len(content.split("\n"))
|
||||
summary.append(f"operation: Added {lines_added} lines to end of note")
|
||||
elif operation == "prepend":
|
||||
lines_added = len(content.split("\n"))
|
||||
summary.append(f"operation: Added {lines_added} lines to beginning of note")
|
||||
elif operation == "find_replace":
|
||||
# For find_replace, we can't easily count replacements from here
|
||||
# since we don't have the original content, but the server handled it
|
||||
summary.append("operation: Find and replace operation completed")
|
||||
elif operation == "replace_section":
|
||||
summary.append(f"operation: Replaced content under section '{section}'")
|
||||
|
||||
# Count observations by category (reuse logic from write_note)
|
||||
categories = {}
|
||||
if result.observations:
|
||||
for obs in result.observations:
|
||||
categories[obs.category] = categories.get(obs.category, 0) + 1
|
||||
|
||||
summary.append("\n## Observations")
|
||||
for category, count in sorted(categories.items()):
|
||||
summary.append(f"- {category}: {count}")
|
||||
|
||||
# Count resolved/unresolved relations
|
||||
unresolved = 0
|
||||
resolved = 0
|
||||
if result.relations:
|
||||
unresolved = sum(1 for r in result.relations if not r.to_id)
|
||||
resolved = len(result.relations) - unresolved
|
||||
|
||||
summary.append("\n## Relations")
|
||||
summary.append(f"- Resolved: {resolved}")
|
||||
if unresolved:
|
||||
summary.append(f"- Unresolved: {unresolved}")
|
||||
|
||||
logger.info(
|
||||
"MCP tool response",
|
||||
tool="edit_note",
|
||||
operation=operation,
|
||||
project=active_project.name,
|
||||
permalink=result.permalink,
|
||||
observations_count=len(result.observations),
|
||||
relations_count=len(result.relations),
|
||||
file_created=file_created,
|
||||
)
|
||||
|
||||
if output_format == "json":
|
||||
return {
|
||||
"title": result.title,
|
||||
"permalink": result.permalink,
|
||||
"file_path": result.file_path,
|
||||
"checksum": result.checksum,
|
||||
"operation": operation,
|
||||
"fileCreated": file_created,
|
||||
}
|
||||
|
||||
summary_result = "\n".join(summary)
|
||||
return add_project_metadata(summary_result, active_project.name)
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Error editing note: {e}")
|
||||
if output_format == "json":
|
||||
return {
|
||||
"title": None,
|
||||
"permalink": None,
|
||||
"file_path": None,
|
||||
"checksum": None,
|
||||
"operation": operation,
|
||||
"fileCreated": False,
|
||||
"error": str(e),
|
||||
}
|
||||
return _format_error_response(
|
||||
str(e),
|
||||
operation,
|
||||
identifier,
|
||||
find_text,
|
||||
effective_replacements,
|
||||
active_project.name,
|
||||
)
|
||||
except Exception as e:
|
||||
logger.error(f"Error editing note: {e}")
|
||||
if output_format == "json":
|
||||
return {
|
||||
"title": None,
|
||||
"permalink": None,
|
||||
"file_path": None,
|
||||
"checksum": None,
|
||||
"operation": operation,
|
||||
"fileCreated": False,
|
||||
"error": str(e),
|
||||
}
|
||||
return _format_error_response(
|
||||
str(e),
|
||||
operation,
|
||||
identifier,
|
||||
find_text,
|
||||
effective_replacements,
|
||||
active_project.name,
|
||||
)
|
||||
|
||||
@@ -1,271 +0,0 @@
|
||||
"""MCP tools for graph intelligence and FCM contracts."""
|
||||
|
||||
from typing import Any, Literal
|
||||
|
||||
from fastmcp import Context
|
||||
|
||||
from basic_memory.mcp.project_context import get_project_client
|
||||
from basic_memory.mcp.server import mcp
|
||||
from basic_memory.schemas.graph_intelligence import (
|
||||
FCMExportRequest,
|
||||
FCMImportRequest,
|
||||
FCMRankActionsRequest,
|
||||
FCMSimulateRequest,
|
||||
GraphImpactRequest,
|
||||
GraphLineageRequest,
|
||||
GraphReindexRequest,
|
||||
)
|
||||
|
||||
|
||||
def _format_lineage_text(result: dict[str, Any]) -> str:
|
||||
root = result["root"]["title"]
|
||||
path_count = len(result.get("paths", []))
|
||||
return f"# Graph Lineage\n\nRoot: {root}\nPaths: {path_count}"
|
||||
|
||||
|
||||
def _format_impact_text(result: dict[str, Any]) -> str:
|
||||
target = result["target"]["title"]
|
||||
affected = len(result.get("affected", []))
|
||||
return f"# Graph Impact\n\nTarget: {target}\nAffected: {affected}"
|
||||
|
||||
|
||||
def _format_health_text(result: dict[str, Any]) -> str:
|
||||
metrics = result["metrics"]
|
||||
return (
|
||||
"# Graph Health\n\n"
|
||||
f"- orphan_rate: {metrics['orphan_rate']}\n"
|
||||
f"- stale_central_nodes: {metrics['stale_central_nodes']}\n"
|
||||
f"- overloaded_hubs: {metrics['overloaded_hubs']}\n"
|
||||
f"- contradiction_candidates: {metrics['contradiction_candidates']}"
|
||||
)
|
||||
|
||||
|
||||
def _format_fcm_simulate_text(result: dict[str, Any]) -> str:
|
||||
deltas = len(result.get("deltas", []))
|
||||
converged = result["stability"]["converged"]
|
||||
return f"# FCM Simulation\n\nDeltas: {deltas}\nConverged: {converged}"
|
||||
|
||||
|
||||
def _format_fcm_rank_text(result: dict[str, Any]) -> str:
|
||||
goal = result["goal"]["label"]
|
||||
count = len(result.get("recommendations", []))
|
||||
return f"# FCM Action Ranking\n\nGoal: {goal}\nRecommendations: {count}"
|
||||
|
||||
|
||||
@mcp.tool(annotations={"readOnlyHint": True, "openWorldHint": False})
|
||||
async def graph_lineage(
|
||||
start: str,
|
||||
goal: str | None = None,
|
||||
max_hops: int = 4,
|
||||
relation_filters: list[str] | None = None,
|
||||
project: str | None = None,
|
||||
workspace: str | None = None,
|
||||
output_format: Literal["json", "text"] = "json",
|
||||
context: Context | None = None,
|
||||
) -> dict[str, Any] | str:
|
||||
"""Get lineage paths from a start node toward an optional goal."""
|
||||
from basic_memory.mcp.clients import GraphClient
|
||||
|
||||
request = GraphLineageRequest(
|
||||
start=start,
|
||||
goal=goal,
|
||||
max_hops=max_hops,
|
||||
relation_filters=relation_filters or [],
|
||||
)
|
||||
async with get_project_client(project, workspace, context) as (client, active_project):
|
||||
graph_client = GraphClient(client, active_project.external_id)
|
||||
result = await graph_client.lineage(request)
|
||||
payload = result.model_dump(mode="json")
|
||||
if output_format == "text":
|
||||
return _format_lineage_text(payload)
|
||||
return payload
|
||||
|
||||
|
||||
@mcp.tool(annotations={"readOnlyHint": True, "openWorldHint": False})
|
||||
async def graph_impact(
|
||||
target: str,
|
||||
horizon: int,
|
||||
relation_filters: list[str] | None = None,
|
||||
include_reasons: bool = True,
|
||||
project: str | None = None,
|
||||
workspace: str | None = None,
|
||||
output_format: Literal["json", "text"] = "json",
|
||||
context: Context | None = None,
|
||||
) -> dict[str, Any] | str:
|
||||
"""Get impact radius from a target node."""
|
||||
from basic_memory.mcp.clients import GraphClient
|
||||
|
||||
request = GraphImpactRequest(
|
||||
target=target,
|
||||
horizon=horizon,
|
||||
relation_filters=relation_filters or [],
|
||||
include_reasons=include_reasons,
|
||||
)
|
||||
async with get_project_client(project, workspace, context) as (client, active_project):
|
||||
graph_client = GraphClient(client, active_project.external_id)
|
||||
result = await graph_client.impact(request)
|
||||
payload = result.model_dump(mode="json")
|
||||
if output_format == "text":
|
||||
return _format_impact_text(payload)
|
||||
return payload
|
||||
|
||||
|
||||
@mcp.tool(annotations={"readOnlyHint": True, "openWorldHint": False})
|
||||
async def graph_health(
|
||||
scope: str | None = None,
|
||||
timeframe: str | None = None,
|
||||
project: str | None = None,
|
||||
workspace: str | None = None,
|
||||
output_format: Literal["json", "text"] = "json",
|
||||
context: Context | None = None,
|
||||
) -> dict[str, Any] | str:
|
||||
"""Get graph health metrics and issues."""
|
||||
from basic_memory.mcp.clients import GraphClient
|
||||
|
||||
async with get_project_client(project, workspace, context) as (client, active_project):
|
||||
graph_client = GraphClient(client, active_project.external_id)
|
||||
result = await graph_client.health(scope=scope, timeframe=timeframe)
|
||||
payload = result.model_dump(mode="json")
|
||||
if output_format == "text":
|
||||
return _format_health_text(payload)
|
||||
return payload
|
||||
|
||||
|
||||
@mcp.tool(annotations={"readOnlyHint": False, "openWorldHint": False})
|
||||
async def fcm_simulate(
|
||||
actions: list[dict[str, Any]],
|
||||
scenario: dict[str, Any] | None = None,
|
||||
clamp_rules: list[dict[str, Any]] | None = None,
|
||||
project: str | None = None,
|
||||
workspace: str | None = None,
|
||||
output_format: Literal["json", "text"] = "json",
|
||||
context: Context | None = None,
|
||||
) -> dict[str, Any] | str:
|
||||
"""Run an FCM simulation with optional scenario controls."""
|
||||
from basic_memory.mcp.clients import FCMClient
|
||||
|
||||
request = FCMSimulateRequest.model_validate(
|
||||
{
|
||||
"actions": actions,
|
||||
"scenario": scenario or {},
|
||||
"clamp_rules": clamp_rules or [],
|
||||
}
|
||||
)
|
||||
async with get_project_client(project, workspace, context) as (client, active_project):
|
||||
fcm_client = FCMClient(client, active_project.external_id)
|
||||
result = await fcm_client.simulate(request)
|
||||
payload = result.model_dump(mode="json")
|
||||
if output_format == "text":
|
||||
return _format_fcm_simulate_text(payload)
|
||||
return payload
|
||||
|
||||
|
||||
@mcp.tool(annotations={"readOnlyHint": True, "openWorldHint": False})
|
||||
async def fcm_rank_actions(
|
||||
goal: str,
|
||||
constraints: dict[str, Any] | None = None,
|
||||
top_k: int = 10,
|
||||
project: str | None = None,
|
||||
workspace: str | None = None,
|
||||
output_format: Literal["json", "text"] = "json",
|
||||
context: Context | None = None,
|
||||
) -> dict[str, Any] | str:
|
||||
"""Rank intervention actions for an FCM goal node."""
|
||||
from basic_memory.mcp.clients import FCMClient
|
||||
|
||||
request = FCMRankActionsRequest.model_validate(
|
||||
{
|
||||
"goal": goal,
|
||||
"constraints": constraints or {},
|
||||
"top_k": top_k,
|
||||
}
|
||||
)
|
||||
async with get_project_client(project, workspace, context) as (client, active_project):
|
||||
fcm_client = FCMClient(client, active_project.external_id)
|
||||
result = await fcm_client.rank_actions(request)
|
||||
payload = result.model_dump(mode="json")
|
||||
if output_format == "text":
|
||||
return _format_fcm_rank_text(payload)
|
||||
return payload
|
||||
|
||||
|
||||
@mcp.tool(annotations={"readOnlyHint": False, "openWorldHint": False})
|
||||
async def fcm_import_model(
|
||||
source: str,
|
||||
format: Literal["csv_bundle_v1"] = "csv_bundle_v1",
|
||||
merge_mode: Literal["replace", "upsert"] = "upsert",
|
||||
project: str | None = None,
|
||||
workspace: str | None = None,
|
||||
output_format: Literal["json", "text"] = "json",
|
||||
context: Context | None = None,
|
||||
) -> dict[str, Any] | str:
|
||||
"""Import an FCM model from an external source."""
|
||||
from basic_memory.mcp.clients import FCMClient
|
||||
|
||||
request = FCMImportRequest(source=source, format=format, merge_mode=merge_mode)
|
||||
async with get_project_client(project, workspace, context) as (client, active_project):
|
||||
fcm_client = FCMClient(client, active_project.external_id)
|
||||
result = await fcm_client.import_model(request)
|
||||
payload = result.model_dump(mode="json")
|
||||
if output_format == "text":
|
||||
return (
|
||||
"# FCM Import\n\n"
|
||||
f"Import ID: {payload['import_id']}\n"
|
||||
f"Nodes Loaded: {payload['nodes_loaded']}\n"
|
||||
f"Edges Loaded: {payload['edges_loaded']}"
|
||||
)
|
||||
return payload
|
||||
|
||||
|
||||
@mcp.tool(annotations={"readOnlyHint": True, "openWorldHint": False})
|
||||
async def fcm_export_model(
|
||||
format: Literal["csv_bundle_v1"] = "csv_bundle_v1",
|
||||
selection: dict[str, Any] | None = None,
|
||||
project: str | None = None,
|
||||
workspace: str | None = None,
|
||||
output_format: Literal["json", "text"] = "json",
|
||||
context: Context | None = None,
|
||||
) -> dict[str, Any] | str:
|
||||
"""Export an FCM model selection."""
|
||||
from basic_memory.mcp.clients import FCMClient
|
||||
|
||||
request = FCMExportRequest.model_validate(
|
||||
{
|
||||
"format": format,
|
||||
"selection": selection or {},
|
||||
}
|
||||
)
|
||||
async with get_project_client(project, workspace, context) as (client, active_project):
|
||||
fcm_client = FCMClient(client, active_project.external_id)
|
||||
result = await fcm_client.export_model(request)
|
||||
payload = result.model_dump(mode="json")
|
||||
if output_format == "text":
|
||||
return (
|
||||
"# FCM Export\n\n"
|
||||
f"Export ID: {payload['export_id']}\n"
|
||||
f"Node Count: {payload['node_count']}\n"
|
||||
f"Edge Count: {payload['edge_count']}"
|
||||
)
|
||||
return payload
|
||||
|
||||
|
||||
@mcp.tool(annotations={"readOnlyHint": False, "openWorldHint": False})
|
||||
async def graph_reindex(
|
||||
mode: Literal["full", "incremental"] = "incremental",
|
||||
reason: str | None = None,
|
||||
project: str | None = None,
|
||||
workspace: str | None = None,
|
||||
output_format: Literal["json", "text"] = "json",
|
||||
context: Context | None = None,
|
||||
) -> dict[str, Any] | str:
|
||||
"""Queue a graph reindex for the active project."""
|
||||
from basic_memory.mcp.clients import GraphClient
|
||||
|
||||
request = GraphReindexRequest(mode=mode, reason=reason)
|
||||
async with get_project_client(project, workspace, context) as (client, active_project):
|
||||
graph_client = GraphClient(client, active_project.external_id)
|
||||
result = await graph_client.reindex(request)
|
||||
payload = result.model_dump(mode="json")
|
||||
if output_format == "text":
|
||||
return f"# Graph Reindex\n\nJob ID: {payload['job_id']}\nStatus: {payload['status']}"
|
||||
return payload
|
||||
@@ -6,6 +6,7 @@ from typing import Optional, Literal
|
||||
|
||||
from loguru import logger
|
||||
from fastmcp import Context
|
||||
from mcp.server.fastmcp.exceptions import ToolError
|
||||
|
||||
from basic_memory.mcp.server import mcp
|
||||
from basic_memory.mcp.project_context import get_project_client
|
||||
@@ -476,8 +477,11 @@ async def move_note(
|
||||
}
|
||||
return f"# Move Failed - Invalid Parameters\n\n{error_msg}"
|
||||
async with get_project_client(project, workspace, context) as (client, active_project):
|
||||
logger.debug(
|
||||
f"Moving {'directory' if is_directory else 'note'}: {identifier} to {destination_path} in project: {active_project.name}"
|
||||
destination_target = destination_folder or destination_path
|
||||
logger.info(
|
||||
f"MCP tool call tool=move_note project={active_project.name} "
|
||||
f"identifier={identifier} destination={destination_target} "
|
||||
f"is_directory={str(is_directory).lower()}"
|
||||
)
|
||||
|
||||
# Validate destination path to prevent path traversal attacks
|
||||
@@ -637,7 +641,7 @@ move_note("path/to/file.md", "{destination_path}/file.md")
|
||||
"""Resolve and cache the source entity ID for the duration of this move."""
|
||||
nonlocal resolved_entity_id
|
||||
if resolved_entity_id is None:
|
||||
resolved_entity_id = await knowledge_client.resolve_entity(identifier)
|
||||
resolved_entity_id = await knowledge_client.resolve_entity(identifier, strict=True)
|
||||
return resolved_entity_id
|
||||
|
||||
try:
|
||||
@@ -645,8 +649,26 @@ move_note("path/to/file.md", "{destination_path}/file.md")
|
||||
source_entity = await knowledge_client.get_entity(resolved_entity_id)
|
||||
if "." in source_entity.file_path:
|
||||
source_ext = source_entity.file_path.split(".")[-1]
|
||||
except ToolError as e:
|
||||
# Trigger: strict=True resolve_entity raised because the entity was not found.
|
||||
# Why: fail fast with a formatted error instead of silently falling through
|
||||
# to extension defaults and failing later with a confusing message.
|
||||
# Outcome: move_note returns a user-facing not-found error immediately.
|
||||
logger.error(f"Move failed for '{identifier}' to '{destination_path}': {e}")
|
||||
if output_format == "json":
|
||||
return {
|
||||
"moved": False,
|
||||
"title": None,
|
||||
"permalink": None,
|
||||
"file_path": None,
|
||||
"source": identifier,
|
||||
"destination": destination_path,
|
||||
"error": str(e),
|
||||
}
|
||||
return _format_move_error_response(str(e), identifier, destination_path)
|
||||
except Exception as e:
|
||||
# If we can't fetch source metadata, continue with extension defaults.
|
||||
# If we can't fetch source metadata (e.g. get_entity or file_path parsing fails),
|
||||
# continue with extension defaults — the entity was at least resolved.
|
||||
logger.debug(f"Could not fetch source entity for extension check: {e}")
|
||||
|
||||
# --- Resolve destination_folder into destination_path ---
|
||||
@@ -815,10 +837,8 @@ move_note("{identifier}", destination_folder="notes")
|
||||
|
||||
# Log the operation
|
||||
logger.info(
|
||||
"Move note completed",
|
||||
identifier=identifier,
|
||||
destination_path=destination_path,
|
||||
project=active_project.name,
|
||||
f"MCP tool response: tool=move_note project={active_project.name} "
|
||||
f"source={identifier} destination={result.file_path} permalink={result.permalink}"
|
||||
)
|
||||
|
||||
return "\n".join(result_lines)
|
||||
|
||||
@@ -216,7 +216,7 @@ async def read_content(
|
||||
if detected:
|
||||
project = detected
|
||||
|
||||
logger.info("Reading file", path=path, project=project)
|
||||
logger.info(f"MCP tool call tool=read_content project={project} path={path}")
|
||||
|
||||
async with get_project_client(project, workspace, context) as (client, active_project):
|
||||
# Resolve path with project-prefix awareness for memory:// URLs
|
||||
@@ -260,6 +260,10 @@ async def read_content(
|
||||
# Handle text or json
|
||||
if content_type.startswith("text/") or content_type == "application/json":
|
||||
logger.debug("Processing text resource")
|
||||
logger.info(
|
||||
f"MCP tool response: tool=read_content project={active_project.name} "
|
||||
f"path={url} type=text content_type={content_type}"
|
||||
)
|
||||
return {
|
||||
"type": "text",
|
||||
"text": response.text,
|
||||
@@ -272,6 +276,10 @@ async def read_content(
|
||||
logger.debug("Processing image")
|
||||
img = PILImage.open(io.BytesIO(response.content))
|
||||
img_bytes = optimize_image(img, content_length)
|
||||
logger.info(
|
||||
f"MCP tool response: tool=read_content project={active_project.name} "
|
||||
f"path={url} type=image content_type=image/jpeg"
|
||||
)
|
||||
|
||||
return {
|
||||
"type": "image",
|
||||
@@ -291,6 +299,10 @@ async def read_content(
|
||||
"type": "error",
|
||||
"error": f"Document size {content_length} bytes exceeds maximum allowed size",
|
||||
}
|
||||
logger.info(
|
||||
f"MCP tool response: tool=read_content project={active_project.name} "
|
||||
f"path={url} type=document content_type={content_type}"
|
||||
)
|
||||
return {
|
||||
"type": "document",
|
||||
"source": {
|
||||
|
||||
@@ -8,6 +8,7 @@ import yaml
|
||||
from loguru import logger
|
||||
from fastmcp import Context
|
||||
|
||||
from basic_memory import telemetry
|
||||
from basic_memory.config import ConfigManager
|
||||
from basic_memory.mcp.project_context import (
|
||||
detect_project_from_url_prefix,
|
||||
@@ -139,186 +140,212 @@ async def read_note(
|
||||
if detected:
|
||||
project = detected
|
||||
|
||||
async with get_project_client(project, workspace, context) as (client, active_project):
|
||||
# Resolve identifier with project-prefix awareness for memory:// URLs
|
||||
_, entity_path, _ = await resolve_project_and_path(client, identifier, project, context)
|
||||
with telemetry.operation(
|
||||
"mcp.tool.read_note",
|
||||
entrypoint="mcp",
|
||||
tool_name="read_note",
|
||||
requested_project=project,
|
||||
workspace_id=workspace,
|
||||
output_format=output_format,
|
||||
page=page,
|
||||
page_size=page_size,
|
||||
include_frontmatter=include_frontmatter,
|
||||
):
|
||||
async with get_project_client(project, workspace, context) as (client, active_project):
|
||||
with telemetry.contextualize(
|
||||
project_name=active_project.name,
|
||||
workspace_id=workspace,
|
||||
tool_name="read_note",
|
||||
):
|
||||
# Resolve identifier with project-prefix awareness for memory:// URLs
|
||||
_, entity_path, _ = await resolve_project_and_path(
|
||||
client, identifier, project, context
|
||||
)
|
||||
|
||||
# Validate identifier to prevent path traversal attacks
|
||||
# For memory:// URLs, validate the extracted path (not the raw URL which
|
||||
# has a scheme prefix that confuses path validation)
|
||||
raw_path = memory_url_path(identifier) if identifier.startswith("memory://") else identifier
|
||||
processed_path = entity_path
|
||||
project_path = active_project.home
|
||||
# Validate identifier to prevent path traversal attacks
|
||||
# For memory:// URLs, validate the extracted path (not the raw URL which
|
||||
# has a scheme prefix that confuses path validation)
|
||||
raw_path = (
|
||||
memory_url_path(identifier)
|
||||
if identifier.startswith("memory://")
|
||||
else identifier
|
||||
)
|
||||
processed_path = entity_path
|
||||
project_path = active_project.home
|
||||
|
||||
if not validate_project_path(raw_path, project_path) or not validate_project_path(
|
||||
processed_path, project_path
|
||||
):
|
||||
logger.warning(
|
||||
"Attempted path traversal attack blocked",
|
||||
identifier=identifier,
|
||||
processed_path=processed_path,
|
||||
project=active_project.name,
|
||||
)
|
||||
if output_format == "json":
|
||||
return {
|
||||
"title": None,
|
||||
"permalink": None,
|
||||
"file_path": None,
|
||||
"content": None,
|
||||
"frontmatter": None,
|
||||
"error": "SECURITY_VALIDATION_ERROR",
|
||||
}
|
||||
return f"# Error\n\nIdentifier '{identifier}' is not allowed - paths must stay within project boundaries"
|
||||
|
||||
# Get the file via REST API - first try direct identifier resolution
|
||||
logger.info(
|
||||
f"Attempting to read note from Project: {active_project.name} identifier: {entity_path}"
|
||||
)
|
||||
|
||||
# Import here to avoid circular import
|
||||
from basic_memory.mcp.clients import KnowledgeClient, ResourceClient
|
||||
|
||||
# Use typed clients for API calls
|
||||
knowledge_client = KnowledgeClient(client, active_project.external_id)
|
||||
resource_client = ResourceClient(client, active_project.external_id)
|
||||
|
||||
async def _read_json_payload(entity_id: str) -> dict:
|
||||
entity = await knowledge_client.get_entity(entity_id)
|
||||
response = await resource_client.read(entity_id, page=page, page_size=page_size)
|
||||
content_text = response.text
|
||||
body_content, parsed_frontmatter = _parse_opening_frontmatter(content_text)
|
||||
return {
|
||||
"title": entity.title,
|
||||
"permalink": entity.permalink,
|
||||
"file_path": entity.file_path,
|
||||
"content": content_text if include_frontmatter else body_content,
|
||||
"frontmatter": parsed_frontmatter,
|
||||
}
|
||||
|
||||
def _empty_json_payload() -> dict:
|
||||
return {
|
||||
"title": None,
|
||||
"permalink": None,
|
||||
"file_path": None,
|
||||
"content": None,
|
||||
"frontmatter": None,
|
||||
}
|
||||
|
||||
def _search_results(payload: object) -> list[dict]:
|
||||
if not isinstance(payload, dict):
|
||||
return []
|
||||
results = payload.get("results")
|
||||
return results if isinstance(results, list) else []
|
||||
|
||||
def _result_title(item: dict) -> str:
|
||||
return str(item.get("title") or "")
|
||||
|
||||
def _result_permalink(item: dict) -> Optional[str]:
|
||||
value = item.get("permalink")
|
||||
return str(value) if value else None
|
||||
|
||||
def _result_file_path(item: dict) -> Optional[str]:
|
||||
value = item.get("file_path")
|
||||
return str(value) if value else None
|
||||
|
||||
try:
|
||||
# Try to resolve identifier to entity ID
|
||||
entity_id = await knowledge_client.resolve_entity(entity_path, strict=True)
|
||||
|
||||
# Fetch content using entity ID
|
||||
response = await resource_client.read(entity_id, page=page, page_size=page_size)
|
||||
|
||||
# If successful, return the content
|
||||
if response.status_code == 200:
|
||||
logger.info("Returning read_note result from resource: {path}", path=entity_path)
|
||||
if output_format == "json":
|
||||
return await _read_json_payload(entity_id)
|
||||
return response.text
|
||||
except Exception as e: # pragma: no cover
|
||||
logger.info(f"Direct lookup failed for '{entity_path}': {e}")
|
||||
# Continue to fallback methods
|
||||
|
||||
# Fallback 1: Try title search via API
|
||||
logger.info(f"Search title for: {identifier}")
|
||||
title_results = await search_notes(
|
||||
query=identifier,
|
||||
search_type="title",
|
||||
project=active_project.name,
|
||||
workspace=workspace,
|
||||
output_format="json",
|
||||
context=context,
|
||||
)
|
||||
|
||||
title_candidates = _search_results(title_results)
|
||||
if title_candidates:
|
||||
# Trigger: direct resolution failed and title search returned candidates.
|
||||
# Why: avoid returning unrelated notes when search yields only fuzzy matches.
|
||||
# Outcome: fetch content only when a true exact title match exists.
|
||||
result = next(
|
||||
(
|
||||
candidate
|
||||
for candidate in title_candidates
|
||||
if _is_exact_title_match(identifier, _result_title(candidate))
|
||||
),
|
||||
None,
|
||||
)
|
||||
if not result:
|
||||
logger.info(f"No exact title match found for: {identifier}")
|
||||
elif _result_permalink(result):
|
||||
try:
|
||||
# Resolve the permalink to entity ID
|
||||
entity_id = await knowledge_client.resolve_entity(
|
||||
_result_permalink(result) or "", strict=True
|
||||
if not validate_project_path(raw_path, project_path) or not validate_project_path(
|
||||
processed_path, project_path
|
||||
):
|
||||
logger.warning(
|
||||
"Attempted path traversal attack blocked",
|
||||
identifier=identifier,
|
||||
processed_path=processed_path,
|
||||
project=active_project.name,
|
||||
)
|
||||
if output_format == "json":
|
||||
return {
|
||||
"title": None,
|
||||
"permalink": None,
|
||||
"file_path": None,
|
||||
"content": None,
|
||||
"frontmatter": None,
|
||||
"error": "SECURITY_VALIDATION_ERROR",
|
||||
}
|
||||
return f"# Error\n\nIdentifier '{identifier}' is not allowed - paths must stay within project boundaries"
|
||||
|
||||
# Fetch content using the entity ID
|
||||
# Get the file via REST API - first try direct identifier resolution
|
||||
logger.info(
|
||||
f"Attempting to read note from Project: {active_project.name} identifier: {entity_path}"
|
||||
)
|
||||
|
||||
# Import here to avoid circular import
|
||||
from basic_memory.mcp.clients import KnowledgeClient, ResourceClient
|
||||
|
||||
# Use typed clients for API calls
|
||||
knowledge_client = KnowledgeClient(client, active_project.external_id)
|
||||
resource_client = ResourceClient(client, active_project.external_id)
|
||||
|
||||
async def _read_json_payload(entity_id: str) -> dict:
|
||||
entity = await knowledge_client.get_entity(entity_id)
|
||||
response = await resource_client.read(entity_id, page=page, page_size=page_size)
|
||||
content_text = response.text
|
||||
body_content, parsed_frontmatter = _parse_opening_frontmatter(content_text)
|
||||
return {
|
||||
"title": entity.title,
|
||||
"permalink": entity.permalink,
|
||||
"file_path": entity.file_path,
|
||||
"content": content_text if include_frontmatter else body_content,
|
||||
"frontmatter": parsed_frontmatter,
|
||||
}
|
||||
|
||||
def _empty_json_payload() -> dict:
|
||||
return {
|
||||
"title": None,
|
||||
"permalink": None,
|
||||
"file_path": None,
|
||||
"content": None,
|
||||
"frontmatter": None,
|
||||
}
|
||||
|
||||
def _search_results(payload: object) -> list[dict]:
|
||||
if not isinstance(payload, dict):
|
||||
return []
|
||||
results = payload.get("results")
|
||||
return results if isinstance(results, list) else []
|
||||
|
||||
def _result_title(item: dict) -> str:
|
||||
return str(item.get("title") or "")
|
||||
|
||||
def _result_permalink(item: dict) -> Optional[str]:
|
||||
value = item.get("permalink")
|
||||
return str(value) if value else None
|
||||
|
||||
def _result_file_path(item: dict) -> Optional[str]:
|
||||
value = item.get("file_path")
|
||||
return str(value) if value else None
|
||||
|
||||
try:
|
||||
# Try to resolve identifier to entity ID
|
||||
entity_id = await knowledge_client.resolve_entity(entity_path, strict=True)
|
||||
|
||||
# Fetch content using entity ID
|
||||
response = await resource_client.read(entity_id, page=page, page_size=page_size)
|
||||
|
||||
# If successful, return the content
|
||||
if response.status_code == 200:
|
||||
logger.info(
|
||||
f"Found note by exact title search: {_result_permalink(result)}"
|
||||
"Returning read_note result from resource: {path}", path=entity_path
|
||||
)
|
||||
if output_format == "json":
|
||||
return await _read_json_payload(entity_id)
|
||||
return response.text
|
||||
except Exception as e: # pragma: no cover
|
||||
logger.info(
|
||||
f"Failed to fetch content for found title match {_result_permalink(result)}: {e}"
|
||||
logger.info(f"Direct lookup failed for '{entity_path}': {e}")
|
||||
# Continue to fallback methods
|
||||
|
||||
# Fallback 1: Try title search via API
|
||||
logger.info(f"Search title for: {identifier}")
|
||||
title_results = await search_notes(
|
||||
query=identifier,
|
||||
search_type="title",
|
||||
project=active_project.name,
|
||||
workspace=workspace,
|
||||
output_format="json",
|
||||
context=context,
|
||||
)
|
||||
|
||||
title_candidates = _search_results(title_results)
|
||||
if title_candidates:
|
||||
# Trigger: direct resolution failed and title search returned candidates.
|
||||
# Why: avoid returning unrelated notes when search yields only fuzzy matches.
|
||||
# Outcome: fetch content only when a true exact title match exists.
|
||||
result = next(
|
||||
(
|
||||
candidate
|
||||
for candidate in title_candidates
|
||||
if _is_exact_title_match(identifier, _result_title(candidate))
|
||||
),
|
||||
None,
|
||||
)
|
||||
else:
|
||||
logger.info(
|
||||
f"No results in title search for: {identifier} in project {active_project.name}"
|
||||
)
|
||||
if not result:
|
||||
logger.info(f"No exact title match found for: {identifier}")
|
||||
elif _result_permalink(result):
|
||||
try:
|
||||
# Resolve the permalink to entity ID
|
||||
entity_id = await knowledge_client.resolve_entity(
|
||||
_result_permalink(result) or "", strict=True
|
||||
)
|
||||
|
||||
# Fallback 2: Text search as a last resort
|
||||
logger.info(f"Title search failed, trying text search for: {identifier}")
|
||||
text_results = await search_notes(
|
||||
query=identifier,
|
||||
search_type="text",
|
||||
project=active_project.name,
|
||||
workspace=workspace,
|
||||
output_format="json",
|
||||
context=context,
|
||||
)
|
||||
# Fetch content using the entity ID
|
||||
response = await resource_client.read(
|
||||
entity_id, page=page, page_size=page_size
|
||||
)
|
||||
|
||||
# We didn't find a direct match, construct a helpful error message
|
||||
text_candidates = _search_results(text_results)
|
||||
if not text_candidates:
|
||||
if output_format == "json":
|
||||
return _empty_json_payload()
|
||||
return format_not_found_message(active_project.name, identifier)
|
||||
if output_format == "json":
|
||||
payload = _empty_json_payload()
|
||||
payload["related_results"] = [
|
||||
{
|
||||
"title": _result_title(result),
|
||||
"permalink": _result_permalink(result),
|
||||
"file_path": _result_file_path(result),
|
||||
}
|
||||
for result in text_candidates[:5]
|
||||
]
|
||||
return payload
|
||||
return format_related_results(active_project.name, identifier, text_candidates[:5])
|
||||
if response.status_code == 200:
|
||||
logger.info(
|
||||
f"Found note by exact title search: {_result_permalink(result)}"
|
||||
)
|
||||
if output_format == "json":
|
||||
return await _read_json_payload(entity_id)
|
||||
return response.text
|
||||
except Exception as e: # pragma: no cover
|
||||
logger.info(
|
||||
f"Failed to fetch content for found title match {_result_permalink(result)}: {e}"
|
||||
)
|
||||
else:
|
||||
logger.info(
|
||||
f"No results in title search for: {identifier} in project {active_project.name}"
|
||||
)
|
||||
|
||||
# Fallback 2: Text search as a last resort
|
||||
logger.info(f"Title search failed, trying text search for: {identifier}")
|
||||
text_results = await search_notes(
|
||||
query=identifier,
|
||||
search_type="text",
|
||||
project=active_project.name,
|
||||
workspace=workspace,
|
||||
output_format="json",
|
||||
context=context,
|
||||
)
|
||||
|
||||
# We didn't find a direct match, construct a helpful error message
|
||||
text_candidates = _search_results(text_results)
|
||||
if not text_candidates:
|
||||
if output_format == "json":
|
||||
return _empty_json_payload()
|
||||
return format_not_found_message(active_project.name, identifier)
|
||||
if output_format == "json":
|
||||
payload = _empty_json_payload()
|
||||
payload["related_results"] = [
|
||||
{
|
||||
"title": _result_title(result),
|
||||
"permalink": _result_permalink(result),
|
||||
"file_path": _result_file_path(result),
|
||||
}
|
||||
for result in text_candidates[:5]
|
||||
]
|
||||
return payload
|
||||
return format_related_results(active_project.name, identifier, text_candidates[:5])
|
||||
|
||||
|
||||
def format_not_found_message(project: str | None, identifier: str) -> str:
|
||||
|
||||
@@ -160,7 +160,7 @@ def _no_notes_guidance(note_type: str, tool_name: str) -> str:
|
||||
f"## Next Steps\n\n"
|
||||
f"1. **Create notes of this type** — use `write_note` with "
|
||||
f'`note_type="{note_type}"` to create notes\n'
|
||||
f"2. **Check existing types** — use `search_notes` with `entity_types` "
|
||||
f"2. **Check existing types** — use `search_notes` with `note_types` "
|
||||
f"filter to see what types exist\n"
|
||||
f"3. **Browse content** — use `list_directory` or `recent_activity` to "
|
||||
f"see what's in the project\n"
|
||||
@@ -397,7 +397,7 @@ async def schema_infer(
|
||||
f"share a consistent structure.\n\n"
|
||||
f"## Suggestions\n"
|
||||
f"1. **Use a more specific type** — try `search_notes` with "
|
||||
f"`entity_types` filter to see what types exist\n"
|
||||
f"`note_types` filter to see what types exist\n"
|
||||
f"2. **Lower the threshold** — "
|
||||
f'`schema_infer("{note_type}", threshold=0.1)` to include '
|
||||
f"rarer fields\n"
|
||||
|
||||
@@ -2,12 +2,15 @@
|
||||
|
||||
import re
|
||||
from textwrap import dedent
|
||||
from typing import List, Optional, Dict, Any, Literal
|
||||
from typing import Annotated, List, Optional, Dict, Any, Literal
|
||||
|
||||
from loguru import logger
|
||||
from fastmcp import Context
|
||||
from pydantic import BeforeValidator
|
||||
|
||||
from basic_memory import telemetry
|
||||
from basic_memory.config import ConfigManager
|
||||
from basic_memory.utils import coerce_dict, coerce_list
|
||||
from basic_memory.mcp.container import get_container
|
||||
from basic_memory.mcp.project_context import (
|
||||
detect_project_from_url_prefix,
|
||||
@@ -23,20 +26,20 @@ from basic_memory.schemas.search import (
|
||||
)
|
||||
|
||||
|
||||
def _semantic_search_enabled_for_text_search() -> bool:
|
||||
"""Resolve semantic-search enablement in both MCP and CLI invocation paths."""
|
||||
try:
|
||||
return get_container().config.semantic_search_enabled
|
||||
except RuntimeError:
|
||||
# Trigger: MCP container is not initialized (e.g., `bm tool search-notes` direct call).
|
||||
# Why: CLI path still needs the same semantic-default behavior as MCP server path.
|
||||
# Outcome: load config directly and keep text-mode retrieval behavior consistent.
|
||||
return ConfigManager().config.semantic_search_enabled
|
||||
|
||||
|
||||
def _default_search_type() -> str:
|
||||
"""Pick default search mode from semantic-search config."""
|
||||
return "hybrid" if _semantic_search_enabled_for_text_search() else "text"
|
||||
"""Pick default search mode from config, falling back to auto-detection.
|
||||
|
||||
Priority: config default_search_type > auto-detect (hybrid if semantic enabled, else text).
|
||||
"""
|
||||
try:
|
||||
config = get_container().config
|
||||
except RuntimeError:
|
||||
config = ConfigManager().config
|
||||
|
||||
if config.default_search_type:
|
||||
return config.default_search_type
|
||||
|
||||
return "hybrid" if config.semantic_search_enabled else "text"
|
||||
|
||||
|
||||
def _format_search_error_response(
|
||||
@@ -165,7 +168,7 @@ def _format_search_error_response(
|
||||
- Remove restrictive terms: Focus on the most important keywords
|
||||
|
||||
5. **Use filtering to narrow scope**:
|
||||
- By content type: `search_notes("{project}","{query}", note_types=["note"])`
|
||||
- By note type in frontmatter: `search_notes("{project}","{query}", note_types=["note"])`
|
||||
- By recent content: `search_notes("{project}","{query}", after_date="1 week")`
|
||||
- By entity type: `search_notes("{project}","{query}", entity_types=["observation"])`
|
||||
|
||||
@@ -305,11 +308,28 @@ async def search_notes(
|
||||
page_size: int = 10,
|
||||
search_type: str | None = None,
|
||||
output_format: Literal["text", "json"] = "text",
|
||||
note_types: List[str] | None = None,
|
||||
entity_types: List[str] | None = None,
|
||||
note_types: Annotated[
|
||||
List[str] | None,
|
||||
BeforeValidator(coerce_list),
|
||||
"Filter by the 'type' field in note frontmatter (e.g. 'note', 'chapter', 'person'). "
|
||||
"Case-insensitive.",
|
||||
] = None,
|
||||
entity_types: Annotated[
|
||||
List[str] | None,
|
||||
BeforeValidator(coerce_list),
|
||||
"Filter by knowledge graph item type: 'entity' (whole notes), 'observation', or "
|
||||
"'relation'. Defaults to 'entity'. Do NOT pass schema/frontmatter types like "
|
||||
"'Chapter' here — use note_types instead.",
|
||||
] = None,
|
||||
after_date: Optional[str] = None,
|
||||
metadata_filters: Optional[Dict[str, Any]] = None,
|
||||
tags: Optional[List[str]] = None,
|
||||
metadata_filters: Annotated[
|
||||
Dict[str, Any] | None,
|
||||
BeforeValidator(coerce_dict),
|
||||
] = None,
|
||||
tags: Annotated[
|
||||
List[str] | None,
|
||||
BeforeValidator(coerce_list),
|
||||
] = None,
|
||||
status: Optional[str] = None,
|
||||
min_similarity: Optional[float] = None,
|
||||
context: Context | None = None,
|
||||
@@ -350,6 +370,7 @@ async def search_notes(
|
||||
### Search Type Examples
|
||||
- `search_notes("my-project", "Meeting", search_type="title")` - Search only in titles
|
||||
- `search_notes("work-docs", "docs/meeting-*", search_type="permalink")` - Pattern match permalinks
|
||||
Note: Permalink patterns match the full path (e.g., "project/folder/chapter-13*", not just "chapter-13*").
|
||||
- `search_notes("research", "keyword")` - Default search (hybrid when semantic is enabled,
|
||||
text when disabled)
|
||||
|
||||
@@ -436,7 +457,7 @@ async def search_notes(
|
||||
# Exact phrase search
|
||||
results = await search_notes("\"weekly standup meeting\"")
|
||||
|
||||
# Search with note type filter
|
||||
# Search with note type filter - type property in frontmatter
|
||||
results = await search_notes(
|
||||
"meeting notes",
|
||||
note_types=["note"],
|
||||
@@ -477,7 +498,8 @@ async def search_notes(
|
||||
results = await search_notes("project planning", project="my-project")
|
||||
"""
|
||||
# Avoid mutable-default-argument footguns. Treat None as "no filter".
|
||||
note_types = note_types or []
|
||||
# Lowercase note_types so "Chapter" matches the stored "chapter".
|
||||
note_types = [t.lower() for t in note_types] if note_types else []
|
||||
entity_types = entity_types or []
|
||||
|
||||
# Parse tag:<value> shorthand at tool level so it works with all search modes.
|
||||
@@ -502,124 +524,157 @@ async def search_notes(
|
||||
if detected:
|
||||
project = detected
|
||||
|
||||
async with get_project_client(project, workspace, context) as (client, active_project):
|
||||
# Handle memory:// URLs by resolving to permalink search
|
||||
is_memory_url = False
|
||||
if query is not None:
|
||||
_, resolved_query, is_memory_url = await resolve_project_and_path(
|
||||
client, query, project, context
|
||||
)
|
||||
if is_memory_url:
|
||||
query = resolved_query
|
||||
effective_search_type = search_type or _default_search_type()
|
||||
if is_memory_url:
|
||||
effective_search_type = "permalink"
|
||||
with telemetry.operation(
|
||||
"mcp.tool.search_notes",
|
||||
entrypoint="mcp",
|
||||
tool_name="search_notes",
|
||||
requested_project=project,
|
||||
workspace_id=workspace,
|
||||
search_type=search_type or "default",
|
||||
output_format=output_format,
|
||||
page=page,
|
||||
page_size=page_size,
|
||||
has_query=bool(query and query.strip()),
|
||||
note_type_filter_count=len(note_types),
|
||||
entity_type_filter_count=len(entity_types),
|
||||
has_metadata_filters=bool(metadata_filters),
|
||||
has_tags_filter=bool(tags),
|
||||
has_status_filter=bool(status),
|
||||
):
|
||||
async with get_project_client(project, workspace, context) as (client, active_project):
|
||||
with telemetry.contextualize(
|
||||
project_name=active_project.name,
|
||||
workspace_id=workspace,
|
||||
tool_name="search_notes",
|
||||
):
|
||||
# Handle memory:// URLs by resolving to permalink search
|
||||
is_memory_url = False
|
||||
if query is not None:
|
||||
_, resolved_query, is_memory_url = await resolve_project_and_path(
|
||||
client, query, project, context
|
||||
)
|
||||
if is_memory_url:
|
||||
query = resolved_query
|
||||
effective_search_type = search_type or _default_search_type()
|
||||
if is_memory_url:
|
||||
effective_search_type = "permalink"
|
||||
|
||||
try:
|
||||
# Create a SearchQuery object based on the parameters
|
||||
search_query = SearchQuery()
|
||||
try:
|
||||
# Create a SearchQuery object based on the parameters
|
||||
search_query = SearchQuery()
|
||||
|
||||
# Only map search_type to query fields when there is an actual query string.
|
||||
# When query is None/empty, skip the search mode block — filters-only path.
|
||||
effective_query = (query or "").strip()
|
||||
if effective_query:
|
||||
valid_search_types = {
|
||||
"text",
|
||||
"title",
|
||||
"permalink",
|
||||
"vector",
|
||||
"semantic",
|
||||
"hybrid",
|
||||
}
|
||||
if effective_search_type == "text":
|
||||
search_query.text = effective_query
|
||||
search_query.retrieval_mode = SearchRetrievalMode.FTS
|
||||
elif effective_search_type in ("vector", "semantic"):
|
||||
search_query.text = effective_query
|
||||
search_query.retrieval_mode = SearchRetrievalMode.VECTOR
|
||||
elif effective_search_type == "hybrid":
|
||||
search_query.text = effective_query
|
||||
search_query.retrieval_mode = SearchRetrievalMode.HYBRID
|
||||
elif effective_search_type == "title":
|
||||
search_query.title = effective_query
|
||||
elif effective_search_type == "permalink" and "*" in effective_query:
|
||||
search_query.permalink_match = effective_query
|
||||
elif effective_search_type == "permalink":
|
||||
search_query.permalink = effective_query
|
||||
else:
|
||||
raise ValueError(
|
||||
f"Invalid search_type '{effective_search_type}'. "
|
||||
f"Valid options: {', '.join(sorted(valid_search_types))}"
|
||||
# Only map search_type to query fields when there is an actual query string.
|
||||
# When query is None/empty, skip the search mode block — filters-only path.
|
||||
effective_query = (query or "").strip()
|
||||
if effective_query:
|
||||
valid_search_types = {
|
||||
"text",
|
||||
"title",
|
||||
"permalink",
|
||||
"vector",
|
||||
"semantic",
|
||||
"hybrid",
|
||||
}
|
||||
if effective_search_type == "text":
|
||||
search_query.text = effective_query
|
||||
search_query.retrieval_mode = SearchRetrievalMode.FTS
|
||||
elif effective_search_type in ("vector", "semantic"):
|
||||
search_query.text = effective_query
|
||||
search_query.retrieval_mode = SearchRetrievalMode.VECTOR
|
||||
elif effective_search_type == "hybrid":
|
||||
search_query.text = effective_query
|
||||
search_query.retrieval_mode = SearchRetrievalMode.HYBRID
|
||||
elif effective_search_type == "title":
|
||||
search_query.title = effective_query
|
||||
elif effective_search_type == "permalink" and "*" in effective_query:
|
||||
search_query.permalink_match = effective_query
|
||||
elif effective_search_type == "permalink":
|
||||
search_query.permalink = effective_query
|
||||
else:
|
||||
raise ValueError(
|
||||
f"Invalid search_type '{effective_search_type}'. "
|
||||
f"Valid options: {', '.join(sorted(valid_search_types))}"
|
||||
)
|
||||
|
||||
# Add optional filters if provided (empty lists are treated as no filter)
|
||||
if entity_types:
|
||||
search_query.entity_types = [SearchItemType(t) for t in entity_types]
|
||||
if note_types:
|
||||
search_query.note_types = note_types
|
||||
if after_date:
|
||||
search_query.after_date = after_date
|
||||
if metadata_filters:
|
||||
# Alias common column/model names to their frontmatter key equivalents.
|
||||
# Users often pass "note_type" (the entity model column) when the
|
||||
# frontmatter field is actually "type".
|
||||
_METADATA_KEY_ALIASES = {"note_type": "type"}
|
||||
metadata_filters = {
|
||||
_METADATA_KEY_ALIASES.get(k, k): v for k, v in metadata_filters.items()
|
||||
}
|
||||
search_query.metadata_filters = metadata_filters
|
||||
if tags:
|
||||
search_query.tags = tags
|
||||
if status:
|
||||
search_query.status = status
|
||||
if min_similarity is not None:
|
||||
search_query.min_similarity = min_similarity
|
||||
|
||||
# Reject searches with no criteria at all
|
||||
if search_query.no_criteria():
|
||||
return (
|
||||
"# No Search Criteria\n\n"
|
||||
"Please provide at least one of: `query`, `metadata_filters`, "
|
||||
"`tags`, `status`, `note_types`, `entity_types`, or `after_date`."
|
||||
)
|
||||
|
||||
# Default to entity-level results to avoid returning individual
|
||||
# observations/relations as separate search results (see issue #31).
|
||||
# Applied after no_criteria() so that the implicit default doesn't
|
||||
# mask a truly empty search request.
|
||||
if not search_query.entity_types:
|
||||
search_query.entity_types = [SearchItemType("entity")]
|
||||
|
||||
logger.debug(
|
||||
f"Search request: project={active_project.name} "
|
||||
f"search_type={effective_search_type} "
|
||||
f"query={effective_query or '<filters-only>'} "
|
||||
f"note_types={len(note_types)} entity_types={len(search_query.entity_types or [])} "
|
||||
f"page={page} page_size={page_size}"
|
||||
)
|
||||
# Import here to avoid circular import (tools → clients → utils → tools)
|
||||
from basic_memory.mcp.clients import SearchClient
|
||||
|
||||
# Use typed SearchClient for API calls
|
||||
search_client = SearchClient(client, active_project.external_id)
|
||||
result = await search_client.search(
|
||||
search_query.model_dump(),
|
||||
page=page,
|
||||
page_size=page_size,
|
||||
)
|
||||
logger.debug(
|
||||
f"Search response: project={active_project.name} "
|
||||
f"results={len(result.results)} has_more={str(result.has_more).lower()} "
|
||||
f"page={result.current_page} page_size={result.page_size}"
|
||||
)
|
||||
|
||||
# Add optional filters if provided (empty lists are treated as no filter)
|
||||
if entity_types:
|
||||
search_query.entity_types = [SearchItemType(t) for t in entity_types]
|
||||
if note_types:
|
||||
search_query.note_types = note_types
|
||||
if after_date:
|
||||
search_query.after_date = after_date
|
||||
if metadata_filters:
|
||||
# Alias common column/model names to their frontmatter key equivalents.
|
||||
# Users often pass "note_type" (the entity model column) when the
|
||||
# frontmatter field is actually "type".
|
||||
_METADATA_KEY_ALIASES = {"note_type": "type"}
|
||||
metadata_filters = {
|
||||
_METADATA_KEY_ALIASES.get(k, k): v for k, v in metadata_filters.items()
|
||||
}
|
||||
search_query.metadata_filters = metadata_filters
|
||||
if tags:
|
||||
search_query.tags = tags
|
||||
if status:
|
||||
search_query.status = status
|
||||
if min_similarity is not None:
|
||||
search_query.min_similarity = min_similarity
|
||||
# Check if we got no results and provide helpful guidance
|
||||
if not result.results:
|
||||
logger.debug(
|
||||
f"Search returned no results for query: {query} in project {active_project.name}"
|
||||
)
|
||||
# Don't treat this as an error, but the user might want guidance
|
||||
# We return the empty result as normal - the user can decide if they need help
|
||||
|
||||
# Reject searches with no criteria at all
|
||||
if search_query.no_criteria():
|
||||
return (
|
||||
"# No Search Criteria\n\n"
|
||||
"Please provide at least one of: `query`, `metadata_filters`, "
|
||||
"`tags`, `status`, `note_types`, `entity_types`, or `after_date`."
|
||||
)
|
||||
if output_format == "json":
|
||||
return result.model_dump(mode="json", exclude_none=True)
|
||||
|
||||
# Default to entity-level results to avoid returning individual
|
||||
# observations/relations as separate search results (see issue #31).
|
||||
# Applied after no_criteria() so that the implicit default doesn't
|
||||
# mask a truly empty search request.
|
||||
if not search_query.entity_types:
|
||||
search_query.entity_types = [SearchItemType("entity")]
|
||||
return _format_search_markdown(result, active_project.name, query)
|
||||
|
||||
logger.debug(f"Searching for {search_query} in project {active_project.name}")
|
||||
# Import here to avoid circular import (tools → clients → utils → tools)
|
||||
from basic_memory.mcp.clients import SearchClient
|
||||
|
||||
# Use typed SearchClient for API calls
|
||||
search_client = SearchClient(client, active_project.external_id)
|
||||
result = await search_client.search(
|
||||
search_query.model_dump(),
|
||||
page=page,
|
||||
page_size=page_size,
|
||||
)
|
||||
|
||||
# Check if we got no results and provide helpful guidance
|
||||
if not result.results:
|
||||
logger.debug(
|
||||
f"Search returned no results for query: {query} in project {active_project.name}"
|
||||
)
|
||||
# Don't treat this as an error, but the user might want guidance
|
||||
# We return the empty result as normal - the user can decide if they need help
|
||||
|
||||
if output_format == "json":
|
||||
return result.model_dump(mode="json", exclude_none=True)
|
||||
|
||||
return _format_search_markdown(result, active_project.name, query)
|
||||
|
||||
except Exception as e:
|
||||
logger.error(
|
||||
f"Search failed for query '{query or ''}': {e}, project: {active_project.name}"
|
||||
)
|
||||
# Return formatted error message as string for better user experience
|
||||
return _format_search_error_response(
|
||||
active_project.name, str(e), query or "", effective_search_type
|
||||
)
|
||||
except Exception as e:
|
||||
logger.error(
|
||||
f"Search failed for query '{query or ''}': {e}, project: {active_project.name}"
|
||||
)
|
||||
# Return formatted error message as string for better user experience
|
||||
return _format_search_error_response(
|
||||
active_project.name, str(e), query or "", effective_search_type
|
||||
)
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any, Dict, List, Optional
|
||||
from typing import Annotated, Any, Dict, List, Optional
|
||||
|
||||
from fastmcp import Context
|
||||
from mcp.types import ContentBlock, TextContent
|
||||
@@ -28,8 +28,17 @@ async def search_notes_ui(
|
||||
page: int = 1,
|
||||
page_size: int = 10,
|
||||
search_type: Optional[str] = None,
|
||||
note_types: List[str] | None = None,
|
||||
entity_types: List[str] | None = None,
|
||||
note_types: Annotated[
|
||||
List[str] | None,
|
||||
"Filter by the 'type' field in note frontmatter (e.g. 'note', 'chapter', 'person'). "
|
||||
"Case-insensitive.",
|
||||
] = None,
|
||||
entity_types: Annotated[
|
||||
List[str] | None,
|
||||
"Filter by knowledge graph item type: 'entity' (whole notes), 'observation', or "
|
||||
"'relation'. Defaults to 'entity'. Do NOT pass schema/frontmatter types like "
|
||||
"'Chapter' here — use note_types instead.",
|
||||
] = None,
|
||||
after_date: Optional[str] = None,
|
||||
metadata_filters: Optional[Dict[str, Any]] = None,
|
||||
tags: Optional[List[str]] = None,
|
||||
|
||||
@@ -1,16 +1,18 @@
|
||||
"""Write note tool for Basic Memory MCP server."""
|
||||
|
||||
import textwrap
|
||||
from typing import List, Union, Optional, Literal
|
||||
from typing import Annotated, List, Union, Optional, Literal
|
||||
|
||||
from loguru import logger
|
||||
from pydantic import BeforeValidator
|
||||
|
||||
from basic_memory import telemetry
|
||||
from basic_memory.config import ConfigManager
|
||||
from basic_memory.mcp.project_context import get_project_client, add_project_metadata
|
||||
from basic_memory.mcp.server import mcp
|
||||
from fastmcp import Context
|
||||
from basic_memory.schemas.base import Entity
|
||||
from basic_memory.utils import parse_tags, validate_project_path
|
||||
from basic_memory.utils import coerce_dict, parse_tags, validate_project_path
|
||||
|
||||
# Define TagType as a Union that can accept either a string or a list of strings or None
|
||||
TagType = Union[List[str], str, None]
|
||||
@@ -28,7 +30,7 @@ async def write_note(
|
||||
workspace: Optional[str] = None,
|
||||
tags: list[str] | str | None = None,
|
||||
note_type: str = "note",
|
||||
metadata: dict | None = None,
|
||||
metadata: Annotated[dict | None, BeforeValidator(coerce_dict)] = None,
|
||||
overwrite: bool | None = None,
|
||||
output_format: Literal["text", "json"] = "text",
|
||||
context: Context | None = None,
|
||||
@@ -147,161 +149,180 @@ async def write_note(
|
||||
overwrite if overwrite is not None else ConfigManager().config.write_note_overwrite_default
|
||||
)
|
||||
|
||||
async with get_project_client(project, workspace, context) as (client, active_project):
|
||||
logger.info(
|
||||
f"MCP tool call tool=write_note project={active_project.name} directory={directory}, title={title}, tags={tags}"
|
||||
)
|
||||
|
||||
# Normalize "/" to empty string for root directory (must happen before validation)
|
||||
if directory == "/":
|
||||
directory = ""
|
||||
|
||||
# Validate directory path to prevent path traversal attacks
|
||||
project_path = active_project.home
|
||||
if directory and not validate_project_path(directory, project_path):
|
||||
logger.warning(
|
||||
"Attempted path traversal attack blocked",
|
||||
directory=directory,
|
||||
project=active_project.name,
|
||||
)
|
||||
if output_format == "json":
|
||||
return {
|
||||
"title": title,
|
||||
"permalink": None,
|
||||
"file_path": None,
|
||||
"checksum": None,
|
||||
"action": "created",
|
||||
"error": "SECURITY_VALIDATION_ERROR",
|
||||
}
|
||||
return f"# Error\n\nDirectory path '{directory}' is not allowed - paths must stay within project boundaries"
|
||||
|
||||
# Process tags using the helper function
|
||||
tag_list = parse_tags(tags)
|
||||
|
||||
# Build entity_metadata from optional metadata, then explicit tags on top
|
||||
# Order matters: explicit tags parameter takes precedence over metadata["tags"]
|
||||
entity_metadata = {}
|
||||
if metadata:
|
||||
entity_metadata.update(metadata)
|
||||
if tag_list:
|
||||
entity_metadata["tags"] = tag_list
|
||||
|
||||
entity = Entity(
|
||||
title=title,
|
||||
directory=directory,
|
||||
note_type=note_type,
|
||||
content_type="text/markdown",
|
||||
content=content,
|
||||
entity_metadata=entity_metadata or None,
|
||||
)
|
||||
|
||||
# Import here to avoid circular import
|
||||
from basic_memory.mcp.clients import KnowledgeClient
|
||||
|
||||
# Use typed KnowledgeClient for API calls
|
||||
knowledge_client = KnowledgeClient(client, active_project.external_id)
|
||||
|
||||
# Try to create the entity first (optimistic create)
|
||||
logger.debug(f"Attempting to create entity permalink={entity.permalink}")
|
||||
action = "Created" # Default to created
|
||||
try:
|
||||
result = await knowledge_client.create_entity(entity.model_dump(), fast=False)
|
||||
action = "Created"
|
||||
except Exception as e:
|
||||
# If creation failed due to conflict (already exists), try to update
|
||||
if (
|
||||
"409" in str(e)
|
||||
or "conflict" in str(e).lower()
|
||||
or "already exists" in str(e).lower()
|
||||
with telemetry.operation(
|
||||
"mcp.tool.write_note",
|
||||
entrypoint="mcp",
|
||||
tool_name="write_note",
|
||||
requested_project=project,
|
||||
workspace_id=workspace,
|
||||
note_type=note_type,
|
||||
overwrite=effective_overwrite,
|
||||
output_format=output_format,
|
||||
):
|
||||
async with get_project_client(project, workspace, context) as (client, active_project):
|
||||
with telemetry.contextualize(
|
||||
project_name=active_project.name,
|
||||
workspace_id=workspace,
|
||||
tool_name="write_note",
|
||||
):
|
||||
# Guard: block overwrite unless explicitly enabled
|
||||
if not effective_overwrite:
|
||||
logger.info(
|
||||
f"MCP tool call tool=write_note project={active_project.name} directory={directory}, title={title}, tags={tags}"
|
||||
)
|
||||
|
||||
# Normalize "/" to empty string for root directory (must happen before validation)
|
||||
if directory == "/":
|
||||
directory = ""
|
||||
|
||||
# Validate directory path to prevent path traversal attacks
|
||||
project_path = active_project.home
|
||||
if directory and not validate_project_path(directory, project_path):
|
||||
logger.warning(
|
||||
f"write_note blocked: note already exists (overwrite not enabled) "
|
||||
f"permalink={entity.permalink}"
|
||||
"Attempted path traversal attack blocked",
|
||||
directory=directory,
|
||||
project=active_project.name,
|
||||
)
|
||||
if output_format == "json":
|
||||
return {
|
||||
"title": title,
|
||||
"permalink": entity.permalink,
|
||||
"permalink": None,
|
||||
"file_path": None,
|
||||
"checksum": None,
|
||||
"action": "conflict",
|
||||
"error": "NOTE_ALREADY_EXISTS",
|
||||
"action": "created",
|
||||
"error": "SECURITY_VALIDATION_ERROR",
|
||||
}
|
||||
return _format_overwrite_error(title, entity.permalink, active_project.name)
|
||||
return f"# Error\n\nDirectory path '{directory}' is not allowed - paths must stay within project boundaries"
|
||||
|
||||
logger.debug(f"Entity exists, updating instead permalink={entity.permalink}")
|
||||
# Process tags using the helper function
|
||||
tag_list = parse_tags(tags)
|
||||
|
||||
# Build entity_metadata from optional metadata, then explicit tags on top
|
||||
# Order matters: explicit tags parameter takes precedence over metadata["tags"]
|
||||
entity_metadata = {}
|
||||
if metadata:
|
||||
entity_metadata.update(metadata)
|
||||
if tag_list:
|
||||
entity_metadata["tags"] = tag_list
|
||||
|
||||
entity = Entity(
|
||||
title=title,
|
||||
directory=directory,
|
||||
note_type=note_type,
|
||||
content_type="text/markdown",
|
||||
content=content,
|
||||
entity_metadata=entity_metadata or None,
|
||||
)
|
||||
|
||||
# Import here to avoid circular import
|
||||
from basic_memory.mcp.clients import KnowledgeClient
|
||||
|
||||
# Use typed KnowledgeClient for API calls
|
||||
knowledge_client = KnowledgeClient(client, active_project.external_id)
|
||||
|
||||
# Try to create the entity first (optimistic create)
|
||||
logger.debug(f"Attempting to create entity permalink={entity.permalink}")
|
||||
action = "Created" # Default to created
|
||||
try:
|
||||
if not entity.permalink:
|
||||
raise ValueError(
|
||||
"Entity permalink is required for updates"
|
||||
) # pragma: no cover
|
||||
entity_id = await knowledge_client.resolve_entity(entity.permalink)
|
||||
result = await knowledge_client.update_entity(
|
||||
entity_id, entity.model_dump(), fast=False
|
||||
)
|
||||
action = "Updated"
|
||||
except Exception as update_error: # pragma: no cover
|
||||
# Re-raise the original error if update also fails
|
||||
raise e from update_error # pragma: no cover
|
||||
else:
|
||||
# Re-raise if it's not a conflict error
|
||||
raise # pragma: no cover
|
||||
summary = [
|
||||
f"# {action} note",
|
||||
f"project: {active_project.name}",
|
||||
f"file_path: {result.file_path}",
|
||||
f"permalink: {result.permalink}",
|
||||
f"checksum: {result.checksum[:8] if result.checksum else 'unknown'}",
|
||||
]
|
||||
result = await knowledge_client.create_entity(entity.model_dump(), fast=False)
|
||||
action = "Created"
|
||||
except Exception as e:
|
||||
# If creation failed due to conflict (already exists), try to update
|
||||
if (
|
||||
"409" in str(e)
|
||||
or "conflict" in str(e).lower()
|
||||
or "already exists" in str(e).lower()
|
||||
):
|
||||
# Guard: block overwrite unless explicitly enabled
|
||||
if not effective_overwrite:
|
||||
logger.warning(
|
||||
f"write_note blocked: note already exists (overwrite not enabled) "
|
||||
f"permalink={entity.permalink}"
|
||||
)
|
||||
if output_format == "json":
|
||||
return {
|
||||
"title": title,
|
||||
"permalink": entity.permalink,
|
||||
"file_path": None,
|
||||
"checksum": None,
|
||||
"action": "conflict",
|
||||
"error": "NOTE_ALREADY_EXISTS",
|
||||
}
|
||||
return _format_overwrite_error(
|
||||
title, entity.permalink, active_project.name
|
||||
)
|
||||
|
||||
# Count observations by category
|
||||
categories = {}
|
||||
if result.observations:
|
||||
for obs in result.observations:
|
||||
categories[obs.category] = categories.get(obs.category, 0) + 1
|
||||
logger.debug(
|
||||
f"Entity exists, updating instead permalink={entity.permalink}"
|
||||
)
|
||||
try:
|
||||
if not entity.permalink:
|
||||
raise ValueError(
|
||||
"Entity permalink is required for updates"
|
||||
) # pragma: no cover
|
||||
entity_id = await knowledge_client.resolve_entity(entity.permalink)
|
||||
result = await knowledge_client.update_entity(
|
||||
entity_id, entity.model_dump(), fast=False
|
||||
)
|
||||
action = "Updated"
|
||||
except Exception as update_error: # pragma: no cover
|
||||
# Re-raise the original error if update also fails
|
||||
raise e from update_error # pragma: no cover
|
||||
else:
|
||||
# Re-raise if it's not a conflict error
|
||||
raise # pragma: no cover
|
||||
summary = [
|
||||
f"# {action} note",
|
||||
f"project: {active_project.name}",
|
||||
f"file_path: {result.file_path}",
|
||||
f"permalink: {result.permalink}",
|
||||
f"checksum: {result.checksum[:8] if result.checksum else 'unknown'}",
|
||||
]
|
||||
|
||||
summary.append("\n## Observations")
|
||||
for category, count in sorted(categories.items()):
|
||||
summary.append(f"- {category}: {count}")
|
||||
# Count observations by category
|
||||
categories = {}
|
||||
if result.observations:
|
||||
for obs in result.observations:
|
||||
categories[obs.category] = categories.get(obs.category, 0) + 1
|
||||
|
||||
# Count resolved/unresolved relations
|
||||
unresolved = 0
|
||||
resolved = 0
|
||||
if result.relations:
|
||||
unresolved = sum(1 for r in result.relations if not r.to_id)
|
||||
resolved = len(result.relations) - unresolved
|
||||
summary.append("\n## Observations")
|
||||
for category, count in sorted(categories.items()):
|
||||
summary.append(f"- {category}: {count}")
|
||||
|
||||
summary.append("\n## Relations")
|
||||
summary.append(f"- Resolved: {resolved}")
|
||||
if unresolved:
|
||||
summary.append(f"- Unresolved: {unresolved}")
|
||||
summary.append(
|
||||
"\nNote: Unresolved relations point to entities that don't exist yet."
|
||||
)
|
||||
summary.append(
|
||||
"They will be automatically resolved when target entities are created or during sync operations."
|
||||
# Count resolved/unresolved relations
|
||||
unresolved = 0
|
||||
resolved = 0
|
||||
if result.relations:
|
||||
unresolved = sum(1 for r in result.relations if not r.to_id)
|
||||
resolved = len(result.relations) - unresolved
|
||||
|
||||
summary.append("\n## Relations")
|
||||
summary.append(f"- Resolved: {resolved}")
|
||||
if unresolved:
|
||||
summary.append(f"- Unresolved: {unresolved}")
|
||||
summary.append(
|
||||
"\nNote: Unresolved relations point to entities that don't exist yet."
|
||||
)
|
||||
summary.append(
|
||||
"They will be automatically resolved when target entities are created or during sync operations."
|
||||
)
|
||||
|
||||
if tag_list:
|
||||
summary.append(f"\n## Tags\n- {', '.join(tag_list)}")
|
||||
|
||||
# Log the response with structured data
|
||||
logger.info(
|
||||
f"MCP tool response: tool=write_note project={active_project.name} action={action} permalink={result.permalink} observations_count={len(result.observations)} relations_count={len(result.relations)} resolved_relations={resolved} unresolved_relations={unresolved}"
|
||||
)
|
||||
if output_format == "json":
|
||||
return {
|
||||
"title": result.title,
|
||||
"permalink": result.permalink,
|
||||
"file_path": result.file_path,
|
||||
"checksum": result.checksum,
|
||||
"action": action.lower(),
|
||||
}
|
||||
|
||||
if tag_list:
|
||||
summary.append(f"\n## Tags\n- {', '.join(tag_list)}")
|
||||
|
||||
# Log the response with structured data
|
||||
logger.info(
|
||||
f"MCP tool response: tool=write_note project={active_project.name} action={action} permalink={result.permalink} observations_count={len(result.observations)} relations_count={len(result.relations)} resolved_relations={resolved} unresolved_relations={unresolved}"
|
||||
)
|
||||
if output_format == "json":
|
||||
return {
|
||||
"title": result.title,
|
||||
"permalink": result.permalink,
|
||||
"file_path": result.file_path,
|
||||
"checksum": result.checksum,
|
||||
"action": action.lower(),
|
||||
}
|
||||
|
||||
summary_result = "\n".join(summary)
|
||||
return add_project_metadata(summary_result, active_project.name)
|
||||
summary_result = "\n".join(summary)
|
||||
return add_project_metadata(summary_result, active_project.name)
|
||||
|
||||
|
||||
def _format_overwrite_error(title: str, permalink: str | None, project_name: str) -> str:
|
||||
|
||||
@@ -451,21 +451,36 @@ class SearchRepositoryBase(ABC):
|
||||
return "\n\n".join(part for part in row_parts if part)
|
||||
|
||||
def _build_chunk_records(self, rows) -> list[dict[str, str]]:
|
||||
records: list[dict[str, str]] = []
|
||||
records_by_key: dict[str, dict[str, str]] = {}
|
||||
duplicate_chunk_keys = 0
|
||||
for row in rows:
|
||||
source_text = self._compose_row_source_text(row)
|
||||
chunks = self._split_text_into_chunks(source_text)
|
||||
for chunk_index, chunk_text in enumerate(chunks):
|
||||
chunk_key = f"{row.type}:{row.id}:{chunk_index}"
|
||||
source_hash = hashlib.sha256(chunk_text.encode("utf-8")).hexdigest()
|
||||
records.append(
|
||||
{
|
||||
"chunk_key": chunk_key,
|
||||
"chunk_text": chunk_text,
|
||||
"source_hash": source_hash,
|
||||
}
|
||||
)
|
||||
return records
|
||||
# Trigger: SQLite FTS5 can accumulate duplicate logical rows for the
|
||||
# same search_index id because it does not enforce relational uniqueness.
|
||||
# Why: duplicate chunk keys would schedule duplicate writes for the same
|
||||
# chunk row and eventually trip UNIQUE(rowid) in search_vector_embeddings.
|
||||
# Outcome: collapse chunk work to one deterministic record per chunk key.
|
||||
if chunk_key in records_by_key:
|
||||
duplicate_chunk_keys += 1
|
||||
records_by_key[chunk_key] = {
|
||||
"chunk_key": chunk_key,
|
||||
"chunk_text": chunk_text,
|
||||
"source_hash": source_hash,
|
||||
}
|
||||
|
||||
if duplicate_chunk_keys:
|
||||
logger.warning(
|
||||
"Collapsed duplicate vector chunk keys before embedding sync: "
|
||||
"project_id={project_id} duplicate_chunk_keys={duplicate_chunk_keys}",
|
||||
project_id=self.project_id,
|
||||
duplicate_chunk_keys=duplicate_chunk_keys,
|
||||
)
|
||||
|
||||
return list(records_by_key.values())
|
||||
|
||||
# --- Text splitting ---
|
||||
|
||||
|
||||
@@ -140,10 +140,12 @@ def validate_timeframe(timeframe: str) -> str:
|
||||
if parsed > now:
|
||||
raise ValueError("Timeframe cannot be in the future") # pragma: no cover
|
||||
|
||||
# Could format the duration back to our standard format
|
||||
days = (now - parsed).days
|
||||
# Round to nearest day to handle DST transitions where an hour shift
|
||||
# can cause e.g. "7d" to compute as 6 days + 23 hours
|
||||
total_seconds = (now - parsed).total_seconds()
|
||||
days = round(total_seconds / 86400)
|
||||
|
||||
# Could enforce reasonable limits
|
||||
# Enforce reasonable limits
|
||||
if days > 365:
|
||||
raise ValueError("Timeframe should be <= 1 year")
|
||||
|
||||
|
||||
@@ -1,318 +0,0 @@
|
||||
"""Schemas for Local+ graph intelligence and FCM contracts."""
|
||||
|
||||
from datetime import datetime
|
||||
from typing import Any, Literal
|
||||
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
|
||||
# --- Graph contracts ---
|
||||
|
||||
|
||||
class GraphLineageRequest(BaseModel):
|
||||
"""Request contract for graph lineage queries."""
|
||||
|
||||
start: str
|
||||
goal: str | None = None
|
||||
max_hops: int = Field(default=4, ge=1, le=6)
|
||||
relation_filters: list[str] = Field(default_factory=list)
|
||||
|
||||
|
||||
class GraphNodeRef(BaseModel):
|
||||
"""Minimal graph node descriptor."""
|
||||
|
||||
id: str
|
||||
title: str
|
||||
permalink: str | None = None
|
||||
|
||||
|
||||
class GraphPathEdge(BaseModel):
|
||||
"""Edge descriptor for lineage paths."""
|
||||
|
||||
relation: str
|
||||
direction: Literal["outgoing", "incoming"]
|
||||
|
||||
|
||||
class GraphLineagePath(BaseModel):
|
||||
"""Single lineage path with scores and provenance."""
|
||||
|
||||
path_id: str
|
||||
nodes: list[GraphNodeRef] = Field(default_factory=list)
|
||||
edges: list[GraphPathEdge] = Field(default_factory=list)
|
||||
deterministic_path_score: float
|
||||
confidence: float
|
||||
evidence_refs: list[str] = Field(default_factory=list)
|
||||
|
||||
|
||||
class GraphLineageResponse(BaseModel):
|
||||
"""Response contract for graph lineage queries."""
|
||||
|
||||
root: GraphNodeRef
|
||||
paths: list[GraphLineagePath] = Field(default_factory=list)
|
||||
generated_at: datetime
|
||||
|
||||
|
||||
class GraphImpactRequest(BaseModel):
|
||||
"""Request contract for impact-radius queries."""
|
||||
|
||||
target: str
|
||||
horizon: int = Field(ge=1, le=4)
|
||||
relation_filters: list[str] = Field(default_factory=list)
|
||||
include_reasons: bool = True
|
||||
|
||||
|
||||
class GraphImpactTarget(BaseModel):
|
||||
"""Impact response target descriptor."""
|
||||
|
||||
id: str
|
||||
title: str
|
||||
|
||||
|
||||
class GraphImpactItem(BaseModel):
|
||||
"""Affected node entry for impact responses."""
|
||||
|
||||
id: str
|
||||
title: str
|
||||
distance: int
|
||||
impact_score: float
|
||||
confidence: float
|
||||
reasons: list[str] = Field(default_factory=list)
|
||||
evidence_refs: list[str] = Field(default_factory=list)
|
||||
|
||||
|
||||
class GraphImpactSummary(BaseModel):
|
||||
"""Summary counters for impact responses."""
|
||||
|
||||
total_considered: int
|
||||
total_returned: int
|
||||
|
||||
|
||||
class GraphImpactResponse(BaseModel):
|
||||
"""Response contract for impact-radius queries."""
|
||||
|
||||
target: GraphImpactTarget
|
||||
affected: list[GraphImpactItem] = Field(default_factory=list)
|
||||
summary: GraphImpactSummary
|
||||
|
||||
|
||||
class GraphHealthMetrics(BaseModel):
|
||||
"""Top-level graph health metrics."""
|
||||
|
||||
orphan_rate: float
|
||||
stale_central_nodes: int
|
||||
overloaded_hubs: int
|
||||
contradiction_candidates: int
|
||||
|
||||
|
||||
class GraphHealthIssue(BaseModel):
|
||||
"""Actionable graph-health issue entry."""
|
||||
|
||||
issue_type: Literal[
|
||||
"orphan",
|
||||
"stale_central",
|
||||
"overloaded_hub",
|
||||
"contradiction_candidate",
|
||||
]
|
||||
entity_id: str
|
||||
severity: Literal["low", "medium", "high"]
|
||||
reason: str
|
||||
suggested_action: str
|
||||
confidence: float | None = None
|
||||
|
||||
|
||||
class GraphHealthResponse(BaseModel):
|
||||
"""Response contract for health checks."""
|
||||
|
||||
metrics: GraphHealthMetrics
|
||||
issues: list[GraphHealthIssue] = Field(default_factory=list)
|
||||
computed_at: datetime
|
||||
|
||||
|
||||
class GraphReindexRequest(BaseModel):
|
||||
"""Request contract for graph reindex scheduling."""
|
||||
|
||||
mode: Literal["full", "incremental"] = "incremental"
|
||||
reason: str | None = None
|
||||
|
||||
|
||||
class GraphReindexResponse(BaseModel):
|
||||
"""Response contract for graph reindex scheduling."""
|
||||
|
||||
job_id: str
|
||||
status: Literal["queued", "running", "completed", "failed"]
|
||||
scheduled_at: datetime
|
||||
|
||||
|
||||
# --- FCM contracts ---
|
||||
|
||||
|
||||
class FCMAction(BaseModel):
|
||||
"""Action delta for simulation input."""
|
||||
|
||||
node_id: str
|
||||
delta: float
|
||||
|
||||
|
||||
class FCMScenario(BaseModel):
|
||||
"""Simulation runtime configuration."""
|
||||
|
||||
steps: int = 12
|
||||
activation: Literal["tanh", "sigmoid", "bounded_linear"] = "tanh"
|
||||
decay: float = 0.05
|
||||
|
||||
|
||||
class FCMClampRule(BaseModel):
|
||||
"""Clamp bounds for selected nodes."""
|
||||
|
||||
node_id: str
|
||||
min: float
|
||||
max: float
|
||||
|
||||
|
||||
class FCMSimulateRequest(BaseModel):
|
||||
"""Request contract for FCM simulation."""
|
||||
|
||||
actions: list[FCMAction]
|
||||
scenario: FCMScenario = Field(default_factory=FCMScenario)
|
||||
clamp_rules: list[FCMClampRule] = Field(default_factory=list)
|
||||
|
||||
|
||||
class FCMNodeState(BaseModel):
|
||||
"""Node state in baseline/projected vectors."""
|
||||
|
||||
node_id: str
|
||||
state: float
|
||||
|
||||
|
||||
class FCMNodeDelta(BaseModel):
|
||||
"""Node delta entry in simulation output."""
|
||||
|
||||
node_id: str
|
||||
delta: float
|
||||
|
||||
|
||||
class FCMStability(BaseModel):
|
||||
"""Simulation stability metadata."""
|
||||
|
||||
converged: bool
|
||||
iterations_used: int
|
||||
residual: float
|
||||
|
||||
|
||||
class FCMInfluencer(BaseModel):
|
||||
"""Top influencer entry for explanation payload."""
|
||||
|
||||
source: str
|
||||
weight: float
|
||||
|
||||
|
||||
class FCMExplanation(BaseModel):
|
||||
"""Per-node explanation payload."""
|
||||
|
||||
node_id: str
|
||||
top_influencers: list[FCMInfluencer] = Field(default_factory=list)
|
||||
|
||||
|
||||
class FCMSimulateResponse(BaseModel):
|
||||
"""Response contract for FCM simulation."""
|
||||
|
||||
baseline: list[FCMNodeState] = Field(default_factory=list)
|
||||
projected: list[FCMNodeState] = Field(default_factory=list)
|
||||
deltas: list[FCMNodeDelta] = Field(default_factory=list)
|
||||
stability: FCMStability
|
||||
confidence: float
|
||||
explanations: list[FCMExplanation] = Field(default_factory=list)
|
||||
evidence_refs: list[str] = Field(default_factory=list)
|
||||
|
||||
|
||||
class FCMRankConstraints(BaseModel):
|
||||
"""Constraint set for action ranking."""
|
||||
|
||||
max_negative_impact: float | None = None
|
||||
required_tags: list[str] = Field(default_factory=list)
|
||||
disallowed_nodes: list[str] = Field(default_factory=list)
|
||||
|
||||
|
||||
class FCMRankActionsRequest(BaseModel):
|
||||
"""Request contract for FCM action ranking."""
|
||||
|
||||
goal: str
|
||||
constraints: FCMRankConstraints = Field(default_factory=FCMRankConstraints)
|
||||
top_k: int = Field(default=10, ge=1, le=25)
|
||||
|
||||
|
||||
class FCMGoalRef(BaseModel):
|
||||
"""Goal descriptor for ranking output."""
|
||||
|
||||
node_id: str
|
||||
label: str
|
||||
|
||||
|
||||
class FCMRecommendation(BaseModel):
|
||||
"""Ranked intervention candidate."""
|
||||
|
||||
action_node_id: str
|
||||
expected_goal_delta: float
|
||||
risk_penalty: float
|
||||
net_score: float
|
||||
confidence: float
|
||||
rationale: list[str] = Field(default_factory=list)
|
||||
evidence_refs: list[str] = Field(default_factory=list)
|
||||
|
||||
|
||||
class FCMRankActionsResponse(BaseModel):
|
||||
"""Response contract for action ranking."""
|
||||
|
||||
goal: FCMGoalRef
|
||||
recommendations: list[FCMRecommendation] = Field(default_factory=list)
|
||||
|
||||
|
||||
class FCMImportRequest(BaseModel):
|
||||
"""Request contract for model import."""
|
||||
|
||||
source: str
|
||||
format: Literal["csv_bundle_v1"] = "csv_bundle_v1"
|
||||
merge_mode: Literal["replace", "upsert"] = "upsert"
|
||||
|
||||
|
||||
class FCMImportResponse(BaseModel):
|
||||
"""Response contract for model import."""
|
||||
|
||||
import_id: str
|
||||
nodes_loaded: int
|
||||
edges_loaded: int
|
||||
warnings: list[str] = Field(default_factory=list)
|
||||
errors: list[str] = Field(default_factory=list)
|
||||
|
||||
|
||||
class FCMExportSelection(BaseModel):
|
||||
"""Scope selection for model export."""
|
||||
|
||||
scope: Literal["all", "tag", "subgraph"] = "all"
|
||||
tag: str | None = None
|
||||
seed_nodes: list[str] = Field(default_factory=list)
|
||||
|
||||
|
||||
class FCMExportRequest(BaseModel):
|
||||
"""Request contract for model export."""
|
||||
|
||||
format: Literal["csv_bundle_v1"] = "csv_bundle_v1"
|
||||
selection: FCMExportSelection = Field(default_factory=FCMExportSelection)
|
||||
|
||||
|
||||
class FCMExportFile(BaseModel):
|
||||
"""Single file descriptor in an export response."""
|
||||
|
||||
name: str
|
||||
path: str
|
||||
|
||||
|
||||
class FCMExportResponse(BaseModel):
|
||||
"""Response contract for model export."""
|
||||
|
||||
export_id: str
|
||||
format: Literal["csv_bundle_v1"]
|
||||
files: list[FCMExportFile] = Field(default_factory=list)
|
||||
node_count: int
|
||||
edge_count: int
|
||||
metadata: dict[str, Any] | None = None
|
||||
@@ -65,7 +65,14 @@ class EditEntityRequest(BaseModel):
|
||||
Supports various operation types for different editing scenarios.
|
||||
"""
|
||||
|
||||
operation: Literal["append", "prepend", "find_replace", "replace_section"]
|
||||
operation: Literal[
|
||||
"append",
|
||||
"prepend",
|
||||
"find_replace",
|
||||
"replace_section",
|
||||
"insert_before_section",
|
||||
"insert_after_section",
|
||||
]
|
||||
content: str
|
||||
section: Optional[str] = None
|
||||
find_text: Optional[str] = None
|
||||
@@ -75,8 +82,16 @@ class EditEntityRequest(BaseModel):
|
||||
@classmethod
|
||||
def validate_section_for_replace_section(cls, v, info):
|
||||
"""Ensure section is provided for replace_section operation."""
|
||||
if info.data.get("operation") == "replace_section" and not v:
|
||||
raise ValueError("section parameter is required for replace_section operation")
|
||||
if (
|
||||
info.data.get("operation")
|
||||
in (
|
||||
"replace_section",
|
||||
"insert_before_section",
|
||||
"insert_after_section",
|
||||
)
|
||||
and not v
|
||||
):
|
||||
raise ValueError("section parameter is required for section-based operations")
|
||||
return v
|
||||
|
||||
@field_validator("find_text")
|
||||
|
||||
@@ -10,6 +10,11 @@ from basic_memory.schemas.v2.entity import (
|
||||
ProjectResolveRequest,
|
||||
ProjectResolveResponse,
|
||||
)
|
||||
from basic_memory.schemas.v2.graph import (
|
||||
GraphEdge,
|
||||
GraphNode,
|
||||
GraphResponse,
|
||||
)
|
||||
from basic_memory.schemas.v2.resource import (
|
||||
CreateResourceRequest,
|
||||
UpdateResourceRequest,
|
||||
@@ -25,6 +30,9 @@ __all__ = [
|
||||
"DeleteDirectoryRequestV2",
|
||||
"ProjectResolveRequest",
|
||||
"ProjectResolveResponse",
|
||||
"GraphEdge",
|
||||
"GraphNode",
|
||||
"GraphResponse",
|
||||
"CreateResourceRequest",
|
||||
"UpdateResourceRequest",
|
||||
"ResourceResponse",
|
||||
|
||||
@@ -0,0 +1,31 @@
|
||||
"""Graph visualization schemas for the knowledge graph endpoint."""
|
||||
|
||||
from typing import Optional
|
||||
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
|
||||
class GraphNode(BaseModel):
|
||||
"""A node in the knowledge graph visualization."""
|
||||
|
||||
external_id: str = Field(..., description="Entity external ID (UUID)")
|
||||
title: str = Field(..., description="Entity title")
|
||||
note_type: Optional[str] = Field(None, description="Note type (e.g., note, spec, task)")
|
||||
file_path: str = Field(..., description="Relative file path")
|
||||
|
||||
|
||||
class GraphEdge(BaseModel):
|
||||
"""An edge in the knowledge graph visualization."""
|
||||
|
||||
from_id: str = Field(..., description="External ID of source entity")
|
||||
to_id: str = Field(..., description="External ID of target entity")
|
||||
relation_type: str = Field(..., description="Type of relation")
|
||||
|
||||
|
||||
class GraphResponse(BaseModel):
|
||||
"""Complete knowledge graph for visualization."""
|
||||
|
||||
nodes: list[GraphNode] = Field(default_factory=list, description="All entities as nodes")
|
||||
edges: list[GraphEdge] = Field(
|
||||
default_factory=list, description="All resolved relations as edges"
|
||||
)
|
||||
@@ -888,6 +888,14 @@ class EntityService(BaseService[EntityModel]):
|
||||
raise ValueError("section cannot be empty or whitespace only")
|
||||
return self.replace_section_content(current_content, section, content)
|
||||
|
||||
elif operation in ("insert_before_section", "insert_after_section"):
|
||||
if not section:
|
||||
raise ValueError("section is required for insert section operations")
|
||||
if not section.strip():
|
||||
raise ValueError("section cannot be empty or whitespace only")
|
||||
position = "before" if operation == "insert_before_section" else "after"
|
||||
return self.insert_relative_to_section(current_content, section, content, position)
|
||||
|
||||
else:
|
||||
raise ValueError(f"Unsupported operation: {operation}")
|
||||
|
||||
@@ -979,6 +987,73 @@ class EntityService(BaseService[EntityModel]):
|
||||
|
||||
return "\n".join(result_lines)
|
||||
|
||||
def insert_relative_to_section(
|
||||
self,
|
||||
current_content: str,
|
||||
section_header: str,
|
||||
new_content: str,
|
||||
position: str,
|
||||
) -> str:
|
||||
"""Insert content before or after a section heading without consuming it.
|
||||
|
||||
Unlike replace_section_content, this preserves the section heading and its
|
||||
existing content. The new content is inserted immediately before or after
|
||||
the heading line.
|
||||
|
||||
Args:
|
||||
current_content: The current markdown content
|
||||
section_header: The section header to anchor on (e.g., "## Section Name")
|
||||
new_content: The content to insert
|
||||
position: "before" to insert above the heading, "after" to insert below it
|
||||
|
||||
Returns:
|
||||
The updated content with new_content inserted relative to the heading
|
||||
|
||||
Raises:
|
||||
ValueError: If the section header is not found or appears more than once
|
||||
"""
|
||||
# Normalize the section header (ensure it starts with #)
|
||||
if not section_header.startswith("#"):
|
||||
section_header = "## " + section_header
|
||||
|
||||
lines = current_content.split("\n")
|
||||
matching_indices = [
|
||||
i for i, line in enumerate(lines) if line.strip() == section_header.strip()
|
||||
]
|
||||
|
||||
if len(matching_indices) == 0:
|
||||
raise ValueError(
|
||||
f"Section '{section_header}' not found in document. "
|
||||
f"Use replace_section to create a new section."
|
||||
)
|
||||
if len(matching_indices) > 1:
|
||||
raise ValueError(
|
||||
f"Multiple sections found with header '{section_header}'. "
|
||||
f"Section insertion requires unique headers."
|
||||
)
|
||||
|
||||
idx = matching_indices[0]
|
||||
|
||||
if position == "before":
|
||||
# Insert new content before the section heading
|
||||
before = lines[:idx]
|
||||
after = lines[idx:]
|
||||
# Ensure blank line separation
|
||||
insert_lines = new_content.rstrip("\n").split("\n")
|
||||
if before and before[-1].strip() != "":
|
||||
insert_lines = [""] + insert_lines
|
||||
return "\n".join(before + insert_lines + [""] + after)
|
||||
else:
|
||||
# Insert new content after the section heading line
|
||||
before = lines[: idx + 1]
|
||||
after = lines[idx + 1 :]
|
||||
insert_lines = new_content.rstrip("\n").split("\n")
|
||||
# Ensure blank line separation so inserted text doesn't merge
|
||||
# with existing section content into a single paragraph
|
||||
if after and after[0].strip() != "":
|
||||
insert_lines = insert_lines + [""]
|
||||
return "\n".join(before + insert_lines + after)
|
||||
|
||||
def _prepend_after_frontmatter(self, current_content: str, content: str) -> str:
|
||||
"""Prepend content after frontmatter, preserving frontmatter structure."""
|
||||
|
||||
|
||||
@@ -1,96 +0,0 @@
|
||||
"""Service layer for FCM contract endpoints."""
|
||||
|
||||
from uuid import uuid4
|
||||
|
||||
from basic_memory.schemas.graph_intelligence import (
|
||||
FCMExportFile,
|
||||
FCMExportRequest,
|
||||
FCMExportResponse,
|
||||
FCMGoalRef,
|
||||
FCMImportRequest,
|
||||
FCMImportResponse,
|
||||
FCMNodeDelta,
|
||||
FCMNodeState,
|
||||
FCMRankActionsRequest,
|
||||
FCMRankActionsResponse,
|
||||
FCMRecommendation,
|
||||
FCMSimulateRequest,
|
||||
FCMSimulateResponse,
|
||||
FCMStability,
|
||||
)
|
||||
|
||||
|
||||
class FCMService:
|
||||
"""FCM contract service.
|
||||
|
||||
Phase 1 keeps deterministic behavior so API and tool surfaces stabilize
|
||||
before introducing advanced simulation engines.
|
||||
"""
|
||||
|
||||
async def simulate(self, request: FCMSimulateRequest) -> FCMSimulateResponse:
|
||||
"""Return deterministic baseline/projected state vectors."""
|
||||
baseline = [FCMNodeState(node_id=action.node_id, state=0.0) for action in request.actions]
|
||||
projected = [
|
||||
FCMNodeState(node_id=action.node_id, state=action.delta) for action in request.actions
|
||||
]
|
||||
deltas = [
|
||||
FCMNodeDelta(node_id=action.node_id, delta=action.delta) for action in request.actions
|
||||
]
|
||||
return FCMSimulateResponse(
|
||||
baseline=baseline,
|
||||
projected=projected,
|
||||
deltas=deltas,
|
||||
stability=FCMStability(
|
||||
converged=True,
|
||||
iterations_used=min(request.scenario.steps, 5),
|
||||
residual=0.0,
|
||||
),
|
||||
confidence=0.5,
|
||||
explanations=[],
|
||||
evidence_refs=[],
|
||||
)
|
||||
|
||||
async def rank_actions(self, request: FCMRankActionsRequest) -> FCMRankActionsResponse:
|
||||
"""Return deterministic ranked actions for a target goal."""
|
||||
recommendations = [
|
||||
FCMRecommendation(
|
||||
action_node_id=f"{request.goal}:action:{idx + 1}",
|
||||
expected_goal_delta=0.25 - (idx * 0.01),
|
||||
risk_penalty=0.05 + (idx * 0.005),
|
||||
net_score=0.20 - (idx * 0.015),
|
||||
confidence=0.5,
|
||||
rationale=["Contract skeleton recommendation"],
|
||||
evidence_refs=[],
|
||||
)
|
||||
for idx in range(min(request.top_k, 3))
|
||||
]
|
||||
return FCMRankActionsResponse(
|
||||
goal=FCMGoalRef(node_id=request.goal, label=request.goal),
|
||||
recommendations=recommendations,
|
||||
)
|
||||
|
||||
async def import_model(self, request: FCMImportRequest) -> FCMImportResponse:
|
||||
"""Return deterministic import metadata."""
|
||||
_ = request
|
||||
return FCMImportResponse(
|
||||
import_id=str(uuid4()),
|
||||
nodes_loaded=0,
|
||||
edges_loaded=0,
|
||||
warnings=[],
|
||||
errors=[],
|
||||
)
|
||||
|
||||
async def export_model(self, request: FCMExportRequest) -> FCMExportResponse:
|
||||
"""Return deterministic export metadata and file descriptors."""
|
||||
scope = request.selection.scope
|
||||
return FCMExportResponse(
|
||||
export_id=str(uuid4()),
|
||||
format=request.format,
|
||||
files=[
|
||||
FCMExportFile(name="nodes.csv", path=f"/tmp/{scope}-nodes.csv"),
|
||||
FCMExportFile(name="edges.csv", path=f"/tmp/{scope}-edges.csv"),
|
||||
],
|
||||
node_count=0,
|
||||
edge_count=0,
|
||||
metadata={"scope": scope},
|
||||
)
|
||||
@@ -1,122 +0,0 @@
|
||||
"""Service layer for graph intelligence contract endpoints."""
|
||||
|
||||
from datetime import datetime, timezone
|
||||
from uuid import uuid4
|
||||
|
||||
from basic_memory.schemas.graph_intelligence import (
|
||||
GraphHealthMetrics,
|
||||
GraphHealthResponse,
|
||||
GraphImpactItem,
|
||||
GraphImpactRequest,
|
||||
GraphImpactResponse,
|
||||
GraphImpactSummary,
|
||||
GraphImpactTarget,
|
||||
GraphLineagePath,
|
||||
GraphLineageRequest,
|
||||
GraphLineageResponse,
|
||||
GraphNodeRef,
|
||||
GraphPathEdge,
|
||||
GraphReindexResponse,
|
||||
)
|
||||
|
||||
|
||||
def _normalize_memory_ref(value: str) -> str:
|
||||
"""Normalize user input into a memory:// reference string."""
|
||||
if value.startswith("memory://"):
|
||||
return value
|
||||
return f"memory://{value}"
|
||||
|
||||
|
||||
def _normalize_node_id(value: str) -> str:
|
||||
"""Return a stable node id for contract skeleton outputs."""
|
||||
return value.removeprefix("memory://")
|
||||
|
||||
|
||||
class GraphIntelligenceService:
|
||||
"""Graph intelligence contract service.
|
||||
|
||||
Phase 1 behavior is intentionally deterministic and lightweight so routing,
|
||||
clients, and contract tests can ship before deeper traversal engines.
|
||||
"""
|
||||
|
||||
async def lineage(self, request: GraphLineageRequest) -> GraphLineageResponse:
|
||||
"""Return a deterministic lineage payload for the requested root/goal."""
|
||||
root_ref = _normalize_memory_ref(request.start)
|
||||
root = GraphNodeRef(
|
||||
id=_normalize_node_id(root_ref),
|
||||
title=_normalize_node_id(root_ref),
|
||||
permalink=_normalize_node_id(root_ref),
|
||||
)
|
||||
|
||||
nodes = [root]
|
||||
edges: list[GraphPathEdge] = []
|
||||
if request.goal:
|
||||
goal_ref = _normalize_memory_ref(request.goal)
|
||||
nodes.append(
|
||||
GraphNodeRef(
|
||||
id=_normalize_node_id(goal_ref),
|
||||
title=_normalize_node_id(goal_ref),
|
||||
permalink=_normalize_node_id(goal_ref),
|
||||
)
|
||||
)
|
||||
edges.append(GraphPathEdge(relation="related_to", direction="outgoing"))
|
||||
|
||||
path = GraphLineagePath(
|
||||
path_id=f"path-{uuid4()}",
|
||||
nodes=nodes,
|
||||
edges=edges,
|
||||
deterministic_path_score=1.0 if request.goal else 0.5,
|
||||
confidence=0.5,
|
||||
evidence_refs=[root_ref],
|
||||
)
|
||||
|
||||
return GraphLineageResponse(
|
||||
root=root,
|
||||
paths=[path],
|
||||
generated_at=datetime.now(timezone.utc),
|
||||
)
|
||||
|
||||
async def impact(self, request: GraphImpactRequest) -> GraphImpactResponse:
|
||||
"""Return a deterministic impact preview payload."""
|
||||
target_id = _normalize_node_id(_normalize_memory_ref(request.target))
|
||||
affected = [
|
||||
GraphImpactItem(
|
||||
id=f"{target_id}:neighbor:1",
|
||||
title=f"{target_id} dependent",
|
||||
distance=min(request.horizon, 1),
|
||||
impact_score=0.55,
|
||||
confidence=0.5,
|
||||
reasons=["Connected via typed relation in contract skeleton"],
|
||||
evidence_refs=[_normalize_memory_ref(request.target)],
|
||||
)
|
||||
]
|
||||
if not request.include_reasons:
|
||||
affected[0].reasons = []
|
||||
|
||||
return GraphImpactResponse(
|
||||
target=GraphImpactTarget(id=target_id, title=target_id),
|
||||
affected=affected,
|
||||
summary=GraphImpactSummary(total_considered=1, total_returned=1),
|
||||
)
|
||||
|
||||
async def health(self, scope: str | None, timeframe: str | None) -> GraphHealthResponse:
|
||||
"""Return deterministic baseline health metrics."""
|
||||
_ = (scope, timeframe)
|
||||
return GraphHealthResponse(
|
||||
metrics=GraphHealthMetrics(
|
||||
orphan_rate=0.0,
|
||||
stale_central_nodes=0,
|
||||
overloaded_hubs=0,
|
||||
contradiction_candidates=0,
|
||||
),
|
||||
issues=[],
|
||||
computed_at=datetime.now(timezone.utc),
|
||||
)
|
||||
|
||||
async def start_reindex_job(self) -> GraphReindexResponse:
|
||||
"""Create reindex job metadata for queued responses."""
|
||||
return GraphReindexResponse(
|
||||
job_id=str(uuid4()),
|
||||
status="queued",
|
||||
scheduled_at=datetime.now(timezone.utc),
|
||||
)
|
||||
@@ -11,6 +11,7 @@ from typing import TYPE_CHECKING, Dict, Optional, Sequence
|
||||
|
||||
from loguru import logger
|
||||
from sqlalchemy import text
|
||||
from sqlalchemy.exc import OperationalError as SAOperationalError
|
||||
|
||||
from basic_memory.models import Project
|
||||
from basic_memory.repository.project_repository import ProjectRepository
|
||||
@@ -996,64 +997,101 @@ class ProjectService:
|
||||
)
|
||||
|
||||
# --- Count queries (tables exist) ---
|
||||
# Filter by entity existence to exclude stale rows from deleted entities
|
||||
# that remain in derived search tables (search_index, search_vector_chunks)
|
||||
entity_exists = "AND entity_id IN (SELECT id FROM entity WHERE project_id = :project_id)"
|
||||
# Same filter for aliased chunks table (used in JOIN queries below)
|
||||
chunk_entity_exists = (
|
||||
"AND c.entity_id IN (SELECT id FROM entity WHERE project_id = :project_id)"
|
||||
)
|
||||
|
||||
si_result = await self.repository.execute_query(
|
||||
text(
|
||||
"SELECT COUNT(DISTINCT entity_id) FROM search_index WHERE project_id = :project_id"
|
||||
"SELECT COUNT(DISTINCT entity_id) FROM search_index "
|
||||
f"WHERE project_id = :project_id {entity_exists}"
|
||||
),
|
||||
{"project_id": project_id},
|
||||
)
|
||||
total_indexed_entities = si_result.scalar() or 0
|
||||
|
||||
chunks_result = await self.repository.execute_query(
|
||||
text("SELECT COUNT(*) FROM search_vector_chunks WHERE project_id = :project_id"),
|
||||
{"project_id": project_id},
|
||||
)
|
||||
total_chunks = chunks_result.scalar() or 0
|
||||
|
||||
entities_with_chunks_result = await self.repository.execute_query(
|
||||
text(
|
||||
"SELECT COUNT(DISTINCT entity_id) FROM search_vector_chunks "
|
||||
"WHERE project_id = :project_id"
|
||||
),
|
||||
{"project_id": project_id},
|
||||
)
|
||||
total_entities_with_chunks = entities_with_chunks_result.scalar() or 0
|
||||
|
||||
# Embeddings count — join pattern differs between SQLite and Postgres
|
||||
if is_postgres:
|
||||
embeddings_sql = text(
|
||||
"SELECT COUNT(*) FROM search_vector_chunks c "
|
||||
"JOIN search_vector_embeddings e ON e.chunk_id = c.id "
|
||||
"WHERE c.project_id = :project_id"
|
||||
)
|
||||
else:
|
||||
embeddings_sql = text(
|
||||
"SELECT COUNT(*) FROM search_vector_chunks c "
|
||||
"JOIN search_vector_embeddings e ON e.rowid = c.id "
|
||||
"WHERE c.project_id = :project_id"
|
||||
try:
|
||||
chunks_result = await self.repository.execute_query(
|
||||
text(
|
||||
"SELECT COUNT(*) FROM search_vector_chunks "
|
||||
f"WHERE project_id = :project_id {entity_exists}"
|
||||
),
|
||||
{"project_id": project_id},
|
||||
)
|
||||
total_chunks = chunks_result.scalar() or 0
|
||||
|
||||
embeddings_result = await self.repository.execute_query(
|
||||
embeddings_sql, {"project_id": project_id}
|
||||
)
|
||||
total_embeddings = embeddings_result.scalar() or 0
|
||||
|
||||
# Orphaned chunks (chunks without embeddings — indicates interrupted indexing)
|
||||
if is_postgres:
|
||||
orphan_sql = text(
|
||||
"SELECT COUNT(*) FROM search_vector_chunks c "
|
||||
"LEFT JOIN search_vector_embeddings e ON e.chunk_id = c.id "
|
||||
"WHERE c.project_id = :project_id AND e.chunk_id IS NULL"
|
||||
)
|
||||
else:
|
||||
orphan_sql = text(
|
||||
"SELECT COUNT(*) FROM search_vector_chunks c "
|
||||
"LEFT JOIN search_vector_embeddings e ON e.rowid = c.id "
|
||||
"WHERE c.project_id = :project_id AND e.rowid IS NULL"
|
||||
entities_with_chunks_result = await self.repository.execute_query(
|
||||
text(
|
||||
"SELECT COUNT(DISTINCT entity_id) FROM search_vector_chunks "
|
||||
f"WHERE project_id = :project_id {entity_exists}"
|
||||
),
|
||||
{"project_id": project_id},
|
||||
)
|
||||
total_entities_with_chunks = entities_with_chunks_result.scalar() or 0
|
||||
|
||||
orphan_result = await self.repository.execute_query(orphan_sql, {"project_id": project_id})
|
||||
orphaned_chunks = orphan_result.scalar() or 0
|
||||
# Embeddings count — join pattern differs between SQLite and Postgres
|
||||
if is_postgres:
|
||||
embeddings_sql = text(
|
||||
"SELECT COUNT(*) FROM search_vector_chunks c "
|
||||
"JOIN search_vector_embeddings e ON e.chunk_id = c.id "
|
||||
f"WHERE c.project_id = :project_id {chunk_entity_exists}"
|
||||
)
|
||||
else:
|
||||
embeddings_sql = text(
|
||||
"SELECT COUNT(*) FROM search_vector_chunks c "
|
||||
"JOIN search_vector_embeddings e ON e.rowid = c.id "
|
||||
f"WHERE c.project_id = :project_id {chunk_entity_exists}"
|
||||
)
|
||||
|
||||
embeddings_result = await self.repository.execute_query(
|
||||
embeddings_sql, {"project_id": project_id}
|
||||
)
|
||||
total_embeddings = embeddings_result.scalar() or 0
|
||||
|
||||
# Orphaned chunks (chunks without embeddings — indicates interrupted indexing)
|
||||
if is_postgres:
|
||||
orphan_sql = text(
|
||||
"SELECT COUNT(*) FROM search_vector_chunks c "
|
||||
"LEFT JOIN search_vector_embeddings e ON e.chunk_id = c.id "
|
||||
f"WHERE c.project_id = :project_id AND e.chunk_id IS NULL {chunk_entity_exists}"
|
||||
)
|
||||
else:
|
||||
orphan_sql = text(
|
||||
"SELECT COUNT(*) FROM search_vector_chunks c "
|
||||
"LEFT JOIN search_vector_embeddings e ON e.rowid = c.id "
|
||||
f"WHERE c.project_id = :project_id AND e.rowid IS NULL {chunk_entity_exists}"
|
||||
)
|
||||
|
||||
orphan_result = await self.repository.execute_query(
|
||||
orphan_sql, {"project_id": project_id}
|
||||
)
|
||||
orphaned_chunks = orphan_result.scalar() or 0
|
||||
except SAOperationalError as exc:
|
||||
# Trigger: sqlite_master can list vec0 virtual tables even when sqlite-vec
|
||||
# is not loaded in the current Python runtime.
|
||||
# Why: project info should degrade gracefully instead of crashing on stats queries.
|
||||
# Outcome: report vector tables as unavailable and point the user to install the
|
||||
# missing dependency before rebuilding embeddings.
|
||||
if is_postgres or "no such module: vec0" not in str(exc).lower():
|
||||
raise
|
||||
|
||||
return EmbeddingStatus(
|
||||
semantic_search_enabled=True,
|
||||
embedding_provider=provider,
|
||||
embedding_model=model,
|
||||
embedding_dimensions=dimensions,
|
||||
total_indexed_entities=total_indexed_entities,
|
||||
vector_tables_exist=False,
|
||||
reindex_recommended=True,
|
||||
reindex_reason=(
|
||||
"SQLite vector tables exist but sqlite-vec is unavailable in this Python "
|
||||
"environment — install/update basic-memory, then run: bm reindex --embeddings"
|
||||
),
|
||||
)
|
||||
|
||||
# --- Reindex recommendation logic (priority order) ---
|
||||
reindex_recommended = False
|
||||
|
||||
@@ -5,12 +5,12 @@ import re
|
||||
from datetime import datetime
|
||||
from typing import List, Optional, Set, Dict, Any
|
||||
|
||||
|
||||
from dateparser import parse
|
||||
from fastapi import BackgroundTasks
|
||||
from loguru import logger
|
||||
from sqlalchemy import text
|
||||
|
||||
from basic_memory import telemetry
|
||||
from basic_memory.models import Entity
|
||||
from basic_memory.repository import EntityRepository
|
||||
from basic_memory.repository.search_repository import (
|
||||
@@ -152,8 +152,6 @@ class SearchService:
|
||||
logger.debug("no criteria passed to query")
|
||||
return []
|
||||
|
||||
logger.trace(f"Searching with query: {query}")
|
||||
|
||||
after_date = (
|
||||
(
|
||||
query.after_date
|
||||
@@ -176,21 +174,32 @@ class SearchService:
|
||||
retrieval_mode = query.retrieval_mode or SearchRetrievalMode.FTS
|
||||
strict_search_text = query.text
|
||||
|
||||
# First pass: preserve existing strict search behavior.
|
||||
results = await self.repository.search(
|
||||
search_text=strict_search_text,
|
||||
permalink=query.permalink,
|
||||
permalink_match=query.permalink_match,
|
||||
title=query.title,
|
||||
note_types=query.note_types,
|
||||
search_item_types=query.entity_types,
|
||||
after_date=after_date,
|
||||
metadata_filters=metadata_filters,
|
||||
retrieval_mode=retrieval_mode,
|
||||
min_similarity=query.min_similarity,
|
||||
with telemetry.scope(
|
||||
"search.execute",
|
||||
retrieval_mode=retrieval_mode.value,
|
||||
has_text_query=bool(strict_search_text),
|
||||
has_title_query=bool(query.title),
|
||||
has_permalink_query=bool(query.permalink or query.permalink_match),
|
||||
has_metadata_filters=bool(metadata_filters),
|
||||
limit=limit,
|
||||
offset=offset,
|
||||
)
|
||||
):
|
||||
logger.trace(f"Searching with query: {query}")
|
||||
# First pass: preserve existing strict search behavior.
|
||||
results = await self.repository.search(
|
||||
search_text=strict_search_text,
|
||||
permalink=query.permalink,
|
||||
permalink_match=query.permalink_match,
|
||||
title=query.title,
|
||||
note_types=query.note_types,
|
||||
search_item_types=query.entity_types,
|
||||
after_date=after_date,
|
||||
metadata_filters=metadata_filters,
|
||||
retrieval_mode=retrieval_mode,
|
||||
min_similarity=query.min_similarity,
|
||||
limit=limit,
|
||||
offset=offset,
|
||||
)
|
||||
|
||||
# Trigger: strict FTS with plain multi-term text returned no results.
|
||||
# Why: natural-language queries often include stopwords that over-constrain implicit AND.
|
||||
@@ -209,20 +218,27 @@ class SearchService:
|
||||
"Strict FTS returned 0 results; retrying relaxed FTS query "
|
||||
f"strict='{strict_search_text}' relaxed='{relaxed_search_text}'"
|
||||
)
|
||||
return await self.repository.search(
|
||||
search_text=relaxed_search_text,
|
||||
permalink=query.permalink,
|
||||
permalink_match=query.permalink_match,
|
||||
title=query.title,
|
||||
note_types=query.note_types,
|
||||
search_item_types=query.entity_types,
|
||||
after_date=after_date,
|
||||
metadata_filters=metadata_filters,
|
||||
retrieval_mode=retrieval_mode,
|
||||
min_similarity=query.min_similarity,
|
||||
with telemetry.scope(
|
||||
"search.relaxed_fts_retry",
|
||||
retrieval_mode=retrieval_mode.value,
|
||||
token_count=len(self._tokenize_fts_text(strict_search_text)),
|
||||
limit=limit,
|
||||
offset=offset,
|
||||
)
|
||||
):
|
||||
return await self.repository.search(
|
||||
search_text=relaxed_search_text,
|
||||
permalink=query.permalink,
|
||||
permalink_match=query.permalink_match,
|
||||
title=query.title,
|
||||
note_types=query.note_types,
|
||||
search_item_types=query.entity_types,
|
||||
after_date=after_date,
|
||||
metadata_filters=metadata_filters,
|
||||
retrieval_mode=retrieval_mode,
|
||||
min_similarity=query.min_similarity,
|
||||
limit=limit,
|
||||
offset=offset,
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
def _tokenize_fts_text(search_text: str) -> list[str]:
|
||||
@@ -403,6 +419,11 @@ class SearchService:
|
||||
"""
|
||||
entities = await self.entity_repository.find_all()
|
||||
entity_ids = [entity.id for entity in entities]
|
||||
|
||||
# Clean up stale rows in search_index and search_vector_chunks
|
||||
# that reference entity_ids no longer in the entity table
|
||||
await self._purge_stale_search_rows()
|
||||
|
||||
batch_result = await self.repository.sync_entity_vectors_batch(
|
||||
entity_ids,
|
||||
progress_callback=progress_callback,
|
||||
@@ -419,6 +440,52 @@ class SearchService:
|
||||
|
||||
return stats
|
||||
|
||||
async def _purge_stale_search_rows(self) -> None:
|
||||
"""Remove rows from search_index and search_vector_chunks for deleted entities.
|
||||
|
||||
Trigger: entities are deleted but their derived search rows remain
|
||||
Why: stale rows inflate embedding coverage stats in project info
|
||||
Outcome: search tables only contain rows for entities that still exist
|
||||
"""
|
||||
from basic_memory.repository.sqlite_search_repository import SQLiteSearchRepository
|
||||
from sqlalchemy import text
|
||||
|
||||
project_id = self.repository.project_id
|
||||
stale_entity_filter = (
|
||||
"entity_id NOT IN (SELECT id FROM entity WHERE project_id = :project_id)"
|
||||
)
|
||||
params = {"project_id": project_id}
|
||||
|
||||
# Delete stale search_index rows
|
||||
await self.repository.execute_query(
|
||||
text(
|
||||
f"DELETE FROM search_index WHERE project_id = :project_id AND {stale_entity_filter}"
|
||||
),
|
||||
params,
|
||||
)
|
||||
|
||||
# SQLite vec has no CASCADE — must delete embeddings before chunks
|
||||
if isinstance(self.repository, SQLiteSearchRepository):
|
||||
await self.repository.execute_query(
|
||||
text(
|
||||
"DELETE FROM search_vector_embeddings WHERE rowid IN ("
|
||||
"SELECT id FROM search_vector_chunks "
|
||||
f"WHERE project_id = :project_id AND {stale_entity_filter})"
|
||||
),
|
||||
params,
|
||||
)
|
||||
|
||||
# Postgres CASCADE handles embedding deletion automatically
|
||||
await self.repository.execute_query(
|
||||
text(
|
||||
f"DELETE FROM search_vector_chunks "
|
||||
f"WHERE project_id = :project_id AND {stale_entity_filter}"
|
||||
),
|
||||
params,
|
||||
)
|
||||
|
||||
logger.info("Purged stale search rows for deleted entities", project_id=project_id)
|
||||
|
||||
async def index_entity_file(
|
||||
self,
|
||||
entity: Entity,
|
||||
|
||||
@@ -15,6 +15,7 @@ import aiofiles.os
|
||||
from loguru import logger
|
||||
from sqlalchemy.exc import IntegrityError
|
||||
|
||||
from basic_memory import telemetry
|
||||
from basic_memory import db
|
||||
from basic_memory.config import BasicMemoryConfig, ConfigManager
|
||||
from basic_memory.file_utils import has_frontmatter
|
||||
@@ -36,6 +37,7 @@ from basic_memory.services.search_service import SearchService
|
||||
|
||||
# Circuit breaker configuration
|
||||
MAX_CONSECUTIVE_FAILURES = 3
|
||||
SLOW_FILE_SYNC_WARNING_MS = 500
|
||||
|
||||
|
||||
@dataclass
|
||||
@@ -265,112 +267,161 @@ class SyncService:
|
||||
|
||||
start_time = time.time()
|
||||
sync_start_timestamp = time.time() # Capture at start for watermark
|
||||
logger.info(f"Sync operation started for directory: {directory} (force_full={force_full})")
|
||||
|
||||
# initial paths from db to sync
|
||||
# path -> checksum
|
||||
report = await self.scan(directory, force_full=force_full)
|
||||
|
||||
# order of sync matters to resolve relations effectively
|
||||
logger.info(
|
||||
f"Sync changes detected: new_files={len(report.new)}, modified_files={len(report.modified)}, "
|
||||
+ f"deleted_files={len(report.deleted)}, moved_files={len(report.moves)}"
|
||||
)
|
||||
|
||||
# sync moves first
|
||||
for old_path, new_path in report.moves.items():
|
||||
# in the case where a file has been deleted and replaced by another file
|
||||
# it will show up in the move and modified lists, so handle it in modified
|
||||
if new_path in report.modified:
|
||||
report.modified.remove(new_path)
|
||||
logger.debug(
|
||||
f"File marked as moved and modified: old_path={old_path}, new_path={new_path}"
|
||||
)
|
||||
else:
|
||||
await self.handle_move(old_path, new_path)
|
||||
|
||||
# deleted next
|
||||
for path in report.deleted:
|
||||
await self.handle_delete(path)
|
||||
|
||||
# then new and modified
|
||||
for path in report.new:
|
||||
entity, _ = await self.sync_file(path, new=True)
|
||||
|
||||
# Track if file was skipped
|
||||
if entity is None and await self._should_skip_file(path):
|
||||
failure_info = self._file_failures[path]
|
||||
report.skipped_files.append(
|
||||
SkippedFile(
|
||||
path=path,
|
||||
reason=failure_info.last_error,
|
||||
failure_count=failure_info.count,
|
||||
first_failed=failure_info.first_failure,
|
||||
)
|
||||
)
|
||||
|
||||
for path in report.modified:
|
||||
entity, _ = await self.sync_file(path, new=False)
|
||||
|
||||
# Track if file was skipped
|
||||
if entity is None and await self._should_skip_file(path):
|
||||
failure_info = self._file_failures[path]
|
||||
report.skipped_files.append(
|
||||
SkippedFile(
|
||||
path=path,
|
||||
reason=failure_info.last_error,
|
||||
failure_count=failure_info.count,
|
||||
first_failed=failure_info.first_failure,
|
||||
)
|
||||
)
|
||||
|
||||
# Only resolve relations if there were actual changes
|
||||
# If no files changed, no new unresolved relations could have been created
|
||||
if report.total > 0:
|
||||
await self.resolve_relations()
|
||||
else:
|
||||
logger.info("Skipping relation resolution - no file changes detected")
|
||||
|
||||
# Update scan watermark after successful sync
|
||||
# Use the timestamp from sync start (not end) to ensure we catch files
|
||||
# created during the sync on the next iteration
|
||||
current_file_count = await self._quick_count_files(directory)
|
||||
if self.entity_repository.project_id is not None:
|
||||
project = await self.project_repository.find_by_id(self.entity_repository.project_id)
|
||||
if project:
|
||||
await self.project_repository.update(
|
||||
project.id,
|
||||
{
|
||||
"last_scan_timestamp": sync_start_timestamp,
|
||||
"last_file_count": current_file_count,
|
||||
},
|
||||
)
|
||||
logger.debug(
|
||||
f"Updated scan watermark: timestamp={sync_start_timestamp}, "
|
||||
f"file_count={current_file_count}"
|
||||
)
|
||||
|
||||
duration_ms = int((time.time() - start_time) * 1000)
|
||||
|
||||
# Log summary with skipped files if any
|
||||
if report.skipped_files:
|
||||
logger.warning(
|
||||
f"Sync completed with {len(report.skipped_files)} skipped files: "
|
||||
f"directory={directory}, total_changes={report.total}, "
|
||||
f"skipped={len(report.skipped_files)}, duration_ms={duration_ms}"
|
||||
)
|
||||
for skipped in report.skipped_files:
|
||||
logger.warning(
|
||||
f"Skipped file: path={skipped.path}, "
|
||||
f"failures={skipped.failure_count}, reason={skipped.reason}"
|
||||
)
|
||||
else:
|
||||
with telemetry.operation(
|
||||
"sync.project.run",
|
||||
project_name=project_name,
|
||||
force_full=force_full,
|
||||
):
|
||||
logger.info(
|
||||
f"Sync operation completed: directory={directory}, "
|
||||
f"total_changes={report.total}, duration_ms={duration_ms}"
|
||||
f"Sync operation started for directory: {directory} (force_full={force_full})"
|
||||
)
|
||||
|
||||
return report
|
||||
# initial paths from db to sync
|
||||
# path -> checksum
|
||||
with telemetry.scope("sync.project.scan", force_full=force_full):
|
||||
report = await self.scan(directory, force_full=force_full)
|
||||
|
||||
# order of sync matters to resolve relations effectively
|
||||
logger.info(
|
||||
f"Sync changes detected: new_files={len(report.new)}, modified_files={len(report.modified)}, "
|
||||
+ f"deleted_files={len(report.deleted)}, moved_files={len(report.moves)}"
|
||||
)
|
||||
|
||||
with telemetry.scope(
|
||||
"sync.project.apply_changes",
|
||||
new_count=len(report.new),
|
||||
modified_count=len(report.modified),
|
||||
deleted_count=len(report.deleted),
|
||||
move_count=len(report.moves),
|
||||
):
|
||||
# sync moves first
|
||||
for old_path, new_path in report.moves.items():
|
||||
# in the case where a file has been deleted and replaced by another file
|
||||
# it will show up in the move and modified lists, so handle it in modified
|
||||
if new_path in report.modified:
|
||||
report.modified.remove(new_path)
|
||||
logger.debug(
|
||||
f"File marked as moved and modified: old_path={old_path}, new_path={new_path}"
|
||||
)
|
||||
else:
|
||||
await self.handle_move(old_path, new_path)
|
||||
|
||||
# deleted next
|
||||
for path in report.deleted:
|
||||
await self.handle_delete(path)
|
||||
|
||||
# then new and modified — collect entity IDs for batch vector embedding
|
||||
synced_entity_ids: list[int] = []
|
||||
|
||||
for path in report.new:
|
||||
entity, _ = await self.sync_file(path, new=True)
|
||||
|
||||
if entity is not None:
|
||||
synced_entity_ids.append(entity.id)
|
||||
# Track if file was skipped
|
||||
elif await self._should_skip_file(path):
|
||||
failure_info = self._file_failures[path]
|
||||
report.skipped_files.append(
|
||||
SkippedFile(
|
||||
path=path,
|
||||
reason=failure_info.last_error,
|
||||
failure_count=failure_info.count,
|
||||
first_failed=failure_info.first_failure,
|
||||
)
|
||||
)
|
||||
|
||||
for path in report.modified:
|
||||
entity, _ = await self.sync_file(path, new=False)
|
||||
|
||||
if entity is not None:
|
||||
synced_entity_ids.append(entity.id)
|
||||
# Track if file was skipped
|
||||
elif await self._should_skip_file(path):
|
||||
failure_info = self._file_failures[path]
|
||||
report.skipped_files.append(
|
||||
SkippedFile(
|
||||
path=path,
|
||||
reason=failure_info.last_error,
|
||||
failure_count=failure_info.count,
|
||||
first_failed=failure_info.first_failure,
|
||||
)
|
||||
)
|
||||
|
||||
# Only resolve relations if there were actual changes
|
||||
# If no files changed, no new unresolved relations could have been created
|
||||
if report.total > 0:
|
||||
with telemetry.scope("sync.project.resolve_relations", relation_scope="all_pending"):
|
||||
await self.resolve_relations()
|
||||
else:
|
||||
logger.info("Skipping relation resolution - no file changes detected")
|
||||
|
||||
# Batch-generate vector embeddings for all synced entities
|
||||
if synced_entity_ids and self.app_config.semantic_search_enabled:
|
||||
try:
|
||||
with telemetry.scope(
|
||||
"sync.project.sync_embeddings",
|
||||
entity_count=len(synced_entity_ids),
|
||||
):
|
||||
logger.info(
|
||||
f"Generating semantic embeddings for {len(synced_entity_ids)} entities..."
|
||||
)
|
||||
batch_result = await self.search_service.sync_entity_vectors_batch(
|
||||
synced_entity_ids
|
||||
)
|
||||
logger.info(
|
||||
f"Semantic embeddings complete: "
|
||||
f"synced={batch_result.entities_synced}, "
|
||||
f"failed={batch_result.entities_failed}"
|
||||
)
|
||||
except SemanticDependenciesMissingError:
|
||||
logger.warning(
|
||||
"Semantic search dependencies missing — vector embeddings skipped. "
|
||||
"Run 'bm reindex --embeddings' after resolving the dependency issue."
|
||||
)
|
||||
|
||||
# Update scan watermark after successful sync
|
||||
# Use the timestamp from sync start (not end) to ensure we catch files
|
||||
# created during the sync on the next iteration
|
||||
with telemetry.scope("sync.project.update_watermark"):
|
||||
current_file_count = await self._quick_count_files(directory)
|
||||
if self.entity_repository.project_id is not None:
|
||||
project = await self.project_repository.find_by_id(
|
||||
self.entity_repository.project_id
|
||||
)
|
||||
if project:
|
||||
await self.project_repository.update(
|
||||
project.id,
|
||||
{
|
||||
"last_scan_timestamp": sync_start_timestamp,
|
||||
"last_file_count": current_file_count,
|
||||
},
|
||||
)
|
||||
logger.debug(
|
||||
f"Updated scan watermark: timestamp={sync_start_timestamp}, "
|
||||
f"file_count={current_file_count}"
|
||||
)
|
||||
|
||||
duration_ms = int((time.time() - start_time) * 1000)
|
||||
|
||||
# Log summary with skipped files if any
|
||||
if report.skipped_files:
|
||||
logger.warning(
|
||||
f"Sync completed with {len(report.skipped_files)} skipped files: "
|
||||
f"directory={directory}, total_changes={report.total}, "
|
||||
f"skipped={len(report.skipped_files)}, duration_ms={duration_ms}"
|
||||
)
|
||||
for skipped in report.skipped_files:
|
||||
logger.warning(
|
||||
f"Skipped file: path={skipped.path}, "
|
||||
f"failures={skipped.failure_count}, reason={skipped.reason}"
|
||||
)
|
||||
else:
|
||||
logger.info(
|
||||
f"Sync operation completed: directory={directory}, "
|
||||
f"total_changes={report.total}, duration_ms={duration_ms}"
|
||||
)
|
||||
|
||||
return report
|
||||
|
||||
async def scan(self, directory, force_full: bool = False):
|
||||
"""Smart scan using watermark and file count for large project optimization.
|
||||
@@ -407,171 +458,180 @@ class SyncService:
|
||||
if project is None:
|
||||
raise ValueError(f"Project not found: {self.entity_repository.project_id}")
|
||||
|
||||
# Step 1: Quick file count
|
||||
logger.debug("Counting files in directory")
|
||||
current_count = await self._quick_count_files(directory)
|
||||
logger.debug(f"Found {current_count} files in directory")
|
||||
with telemetry.scope("sync.project.select_scan_strategy", force_full=force_full):
|
||||
# Step 1: Quick file count
|
||||
logger.debug("Counting files in directory")
|
||||
current_count = await self._quick_count_files(directory)
|
||||
logger.debug(f"Found {current_count} files in directory")
|
||||
|
||||
# Step 2: Determine scan strategy based on watermark and file count
|
||||
if force_full:
|
||||
# User explicitly requested full scan → bypass watermark optimization
|
||||
scan_type = "full_forced"
|
||||
logger.info("Force full scan requested, bypassing watermark optimization")
|
||||
file_paths_to_scan = await self._scan_directory_full(directory)
|
||||
# Step 2: Determine scan strategy based on watermark and file count
|
||||
if force_full:
|
||||
# User explicitly requested full scan → bypass watermark optimization
|
||||
scan_type = "full_forced"
|
||||
logger.info("Force full scan requested, bypassing watermark optimization")
|
||||
scan_coro = self._scan_directory_full(directory)
|
||||
|
||||
elif project.last_file_count is None:
|
||||
# First sync ever → full scan
|
||||
scan_type = "full_initial"
|
||||
logger.info("First sync for this project, performing full scan")
|
||||
file_paths_to_scan = await self._scan_directory_full(directory)
|
||||
elif project.last_file_count is None:
|
||||
# First sync ever → full scan
|
||||
scan_type = "full_initial"
|
||||
logger.info("First sync for this project, performing full scan")
|
||||
scan_coro = self._scan_directory_full(directory)
|
||||
|
||||
elif current_count < project.last_file_count:
|
||||
# Files deleted → need full scan to detect which ones
|
||||
scan_type = "full_deletions"
|
||||
logger.info(
|
||||
f"File count decreased ({project.last_file_count} → {current_count}), "
|
||||
f"running full scan to detect deletions"
|
||||
)
|
||||
file_paths_to_scan = await self._scan_directory_full(directory)
|
||||
elif current_count < project.last_file_count:
|
||||
# Files deleted → need full scan to detect which ones
|
||||
scan_type = "full_deletions"
|
||||
logger.info(
|
||||
f"File count decreased ({project.last_file_count} → {current_count}), "
|
||||
f"running full scan to detect deletions"
|
||||
)
|
||||
scan_coro = self._scan_directory_full(directory)
|
||||
|
||||
elif project.last_scan_timestamp is not None:
|
||||
# Incremental scan: only files modified since last scan
|
||||
scan_type = "incremental"
|
||||
logger.debug(
|
||||
f"Running incremental scan for files modified since {project.last_scan_timestamp}"
|
||||
)
|
||||
file_paths_to_scan = await self._scan_directory_modified_since(
|
||||
directory, project.last_scan_timestamp
|
||||
)
|
||||
logger.debug(
|
||||
f"Incremental scan found {len(file_paths_to_scan)} potentially changed files"
|
||||
)
|
||||
elif project.last_scan_timestamp is not None:
|
||||
# Incremental scan: only files modified since last scan
|
||||
scan_type = "incremental"
|
||||
logger.debug(
|
||||
f"Running incremental scan for files modified since {project.last_scan_timestamp}"
|
||||
)
|
||||
scan_coro = self._scan_directory_modified_since(
|
||||
directory, project.last_scan_timestamp
|
||||
)
|
||||
|
||||
else:
|
||||
# Fallback to full scan (no watermark available)
|
||||
scan_type = "full_fallback"
|
||||
logger.warning("No scan watermark available, falling back to full scan")
|
||||
file_paths_to_scan = await self._scan_directory_full(directory)
|
||||
|
||||
# Step 3: Process each file with mtime-based comparison
|
||||
scanned_paths: Set[str] = set()
|
||||
changed_checksums: Dict[str, str] = {}
|
||||
|
||||
logger.debug(f"Processing {len(file_paths_to_scan)} files with mtime-based comparison")
|
||||
|
||||
for rel_path in file_paths_to_scan:
|
||||
scanned_paths.add(rel_path)
|
||||
|
||||
# Get file stats
|
||||
abs_path = directory / rel_path
|
||||
if not abs_path.exists():
|
||||
# File was deleted between scan and now (race condition)
|
||||
continue
|
||||
|
||||
stat_info = abs_path.stat()
|
||||
|
||||
# Indexed lookup - single file query (not full table scan)
|
||||
db_entity = await self.entity_repository.get_by_file_path(rel_path)
|
||||
|
||||
if db_entity is None:
|
||||
# New file - need checksum for move detection
|
||||
checksum = await self.file_service.compute_checksum(rel_path)
|
||||
report.new.add(rel_path)
|
||||
changed_checksums[rel_path] = checksum
|
||||
logger.trace(f"New file detected: {rel_path}")
|
||||
continue
|
||||
|
||||
# File exists in DB - check if mtime/size changed
|
||||
db_mtime = db_entity.mtime
|
||||
db_size = db_entity.size
|
||||
fs_mtime = stat_info.st_mtime
|
||||
fs_size = stat_info.st_size
|
||||
|
||||
# Compare mtime and size (like rsync/rclone)
|
||||
# Allow small epsilon for float comparison (0.01s = 10ms)
|
||||
mtime_changed = db_mtime is None or abs(fs_mtime - db_mtime) > 0.01
|
||||
size_changed = db_size is None or fs_size != db_size
|
||||
|
||||
if mtime_changed or size_changed:
|
||||
# File modified - compute checksum
|
||||
checksum = await self.file_service.compute_checksum(rel_path)
|
||||
db_checksum = db_entity.checksum
|
||||
|
||||
# Only mark as modified if checksum actually differs
|
||||
# (handles cases where mtime changed but content didn't, e.g., git operations)
|
||||
if checksum != db_checksum:
|
||||
report.modified.add(rel_path)
|
||||
changed_checksums[rel_path] = checksum
|
||||
logger.trace(
|
||||
f"Modified file detected: {rel_path}, "
|
||||
f"mtime_changed={mtime_changed}, size_changed={size_changed}"
|
||||
)
|
||||
else:
|
||||
# File unchanged - no checksum needed
|
||||
logger.trace(f"File unchanged (mtime/size match): {rel_path}")
|
||||
# Fallback to full scan (no watermark available)
|
||||
scan_type = "full_fallback"
|
||||
logger.warning("No scan watermark available, falling back to full scan")
|
||||
scan_coro = self._scan_directory_full(directory)
|
||||
|
||||
# Step 4: Detect moves (for both full and incremental scans)
|
||||
# Check if any "new" files are actually moves by matching checksums
|
||||
for new_path in list(report.new): # Use list() to allow modification during iteration
|
||||
new_checksum = changed_checksums.get(new_path)
|
||||
if not new_checksum:
|
||||
continue
|
||||
with telemetry.scope("sync.project.filesystem_scan", scan_type=scan_type):
|
||||
file_paths_to_scan = await scan_coro
|
||||
if scan_type == "incremental":
|
||||
logger.debug(
|
||||
f"Incremental scan found {len(file_paths_to_scan)} potentially changed files"
|
||||
)
|
||||
|
||||
# Look for existing entity with same checksum but different path
|
||||
# This could be a move or a copy
|
||||
existing_entities = await self.entity_repository.find_by_checksum(new_checksum)
|
||||
# Step 3: Process each file with mtime-based comparison
|
||||
scanned_paths: Set[str] = set()
|
||||
changed_checksums: Dict[str, str] = {}
|
||||
|
||||
for candidate in existing_entities:
|
||||
if candidate.file_path == new_path:
|
||||
# Same path, skip (shouldn't happen for "new" files but be safe)
|
||||
logger.debug(f"Processing {len(file_paths_to_scan)} files with mtime-based comparison")
|
||||
|
||||
for rel_path in file_paths_to_scan:
|
||||
scanned_paths.add(rel_path)
|
||||
|
||||
# Get file stats
|
||||
abs_path = directory / rel_path
|
||||
if not abs_path.exists():
|
||||
# File was deleted between scan and now (race condition)
|
||||
continue
|
||||
|
||||
# Check if the old path still exists on disk
|
||||
old_path_abs = directory / candidate.file_path
|
||||
if old_path_abs.exists():
|
||||
# Original still exists → this is a copy, not a move
|
||||
logger.trace(
|
||||
f"File copy detected (not move): {candidate.file_path} copied to {new_path}"
|
||||
)
|
||||
stat_info = abs_path.stat()
|
||||
|
||||
# Indexed lookup - single file query (not full table scan)
|
||||
db_entity = await self.entity_repository.get_by_file_path(rel_path)
|
||||
|
||||
if db_entity is None:
|
||||
# New file - need checksum for move detection
|
||||
checksum = await self.file_service.compute_checksum(rel_path)
|
||||
report.new.add(rel_path)
|
||||
changed_checksums[rel_path] = checksum
|
||||
logger.trace(f"New file detected: {rel_path}")
|
||||
continue
|
||||
|
||||
# Original doesn't exist → this is a move!
|
||||
report.moves[candidate.file_path] = new_path
|
||||
report.new.remove(new_path)
|
||||
logger.trace(f"Move detected: {candidate.file_path} -> {new_path}")
|
||||
break # Only match first candidate
|
||||
# File exists in DB - check if mtime/size changed
|
||||
db_mtime = db_entity.mtime
|
||||
db_size = db_entity.size
|
||||
fs_mtime = stat_info.st_mtime
|
||||
fs_size = stat_info.st_size
|
||||
|
||||
# Step 5: Detect deletions (only for full scans)
|
||||
# Incremental scans can't reliably detect deletions since they only see modified files
|
||||
if scan_type in ("full_initial", "full_deletions", "full_fallback", "full_forced"):
|
||||
# Use optimized query for just file paths (not full entities)
|
||||
db_file_paths = await self.entity_repository.get_all_file_paths()
|
||||
logger.debug(f"Found {len(db_file_paths)} db paths for deletion detection")
|
||||
# Compare mtime and size (like rsync/rclone)
|
||||
# Allow small epsilon for float comparison (0.01s = 10ms)
|
||||
mtime_changed = db_mtime is None or abs(fs_mtime - db_mtime) > 0.01
|
||||
size_changed = db_size is None or fs_size != db_size
|
||||
|
||||
for db_path in db_file_paths:
|
||||
if db_path not in scanned_paths:
|
||||
# File in DB but not on filesystem
|
||||
# Check if it was already detected as a move
|
||||
if db_path in report.moves:
|
||||
# Already handled as a move, skip
|
||||
if mtime_changed or size_changed:
|
||||
# File modified - compute checksum
|
||||
checksum = await self.file_service.compute_checksum(rel_path)
|
||||
db_checksum = db_entity.checksum
|
||||
|
||||
# Only mark as modified if checksum actually differs
|
||||
# (handles cases where mtime changed but content didn't, e.g., git operations)
|
||||
if checksum != db_checksum:
|
||||
report.modified.add(rel_path)
|
||||
changed_checksums[rel_path] = checksum
|
||||
logger.trace(
|
||||
f"Modified file detected: {rel_path}, "
|
||||
f"mtime_changed={mtime_changed}, size_changed={size_changed}"
|
||||
)
|
||||
else:
|
||||
# File unchanged - no checksum needed
|
||||
logger.trace(f"File unchanged (mtime/size match): {rel_path}")
|
||||
|
||||
# Step 4: Detect moves (for both full and incremental scans)
|
||||
# Check if any "new" files are actually moves by matching checksums
|
||||
with telemetry.scope("sync.project.detect_moves", new_count=len(report.new)):
|
||||
for new_path in list(
|
||||
report.new
|
||||
): # Use list() to allow modification during iteration
|
||||
new_checksum = changed_checksums.get(new_path)
|
||||
if not new_checksum:
|
||||
continue
|
||||
|
||||
# File was deleted
|
||||
report.deleted.add(db_path)
|
||||
logger.trace(f"Deleted file detected: {db_path}")
|
||||
# Look for existing entity with same checksum but different path
|
||||
# This could be a move or a copy
|
||||
existing_entities = await self.entity_repository.find_by_checksum(new_checksum)
|
||||
|
||||
# Store checksums for files that need syncing
|
||||
report.checksums = changed_checksums
|
||||
for candidate in existing_entities:
|
||||
if candidate.file_path == new_path:
|
||||
# Same path, skip (shouldn't happen for "new" files but be safe)
|
||||
continue
|
||||
|
||||
scan_duration_ms = int((time.time() - scan_start_time) * 1000)
|
||||
# Check if the old path still exists on disk
|
||||
old_path_abs = directory / candidate.file_path
|
||||
if old_path_abs.exists():
|
||||
# Original still exists → this is a copy, not a move
|
||||
logger.trace(
|
||||
f"File copy detected (not move): {candidate.file_path} copied to {new_path}"
|
||||
)
|
||||
continue
|
||||
|
||||
logger.info(
|
||||
f"Completed {scan_type} scan for directory {directory} in {scan_duration_ms}ms, "
|
||||
f"found {report.total} changes (new={len(report.new)}, "
|
||||
f"modified={len(report.modified)}, deleted={len(report.deleted)}, "
|
||||
f"moves={len(report.moves)})"
|
||||
)
|
||||
return report
|
||||
# Original doesn't exist → this is a move!
|
||||
report.moves[candidate.file_path] = new_path
|
||||
report.new.remove(new_path)
|
||||
logger.trace(f"Move detected: {candidate.file_path} -> {new_path}")
|
||||
break # Only match first candidate
|
||||
|
||||
# Step 5: Detect deletions (only for full scans)
|
||||
# Incremental scans can't reliably detect deletions since they only see modified files
|
||||
if scan_type in ("full_initial", "full_deletions", "full_fallback", "full_forced"):
|
||||
with telemetry.scope("sync.project.detect_deletions", scan_type=scan_type):
|
||||
# Use optimized query for just file paths (not full entities)
|
||||
db_file_paths = await self.entity_repository.get_all_file_paths()
|
||||
logger.debug(f"Found {len(db_file_paths)} db paths for deletion detection")
|
||||
|
||||
for db_path in db_file_paths:
|
||||
if db_path not in scanned_paths:
|
||||
# File in DB but not on filesystem
|
||||
# Check if it was already detected as a move
|
||||
if db_path in report.moves:
|
||||
# Already handled as a move, skip
|
||||
continue
|
||||
|
||||
# File was deleted
|
||||
report.deleted.add(db_path)
|
||||
logger.trace(f"Deleted file detected: {db_path}")
|
||||
|
||||
# Store checksums for files that need syncing
|
||||
report.checksums = changed_checksums
|
||||
|
||||
scan_duration_ms = int((time.time() - scan_start_time) * 1000)
|
||||
|
||||
logger.info(
|
||||
f"Completed {scan_type} scan for directory {directory} in {scan_duration_ms}ms, "
|
||||
f"found {report.total} changes (new={len(report.new)}, "
|
||||
f"modified={len(report.modified)}, deleted={len(report.deleted)}, "
|
||||
f"moves={len(report.moves)})"
|
||||
)
|
||||
return report
|
||||
|
||||
async def sync_file(
|
||||
self, path: str, new: bool = True
|
||||
@@ -590,12 +650,14 @@ class SyncService:
|
||||
logger.warning(f"Skipping file due to repeated failures: {path}")
|
||||
return None, None
|
||||
|
||||
try:
|
||||
logger.debug(
|
||||
f"Syncing file path={path} is_new={new} is_markdown={self.file_service.is_markdown(path)}"
|
||||
)
|
||||
start_time = time.time()
|
||||
is_markdown = self.file_service.is_markdown(path)
|
||||
file_kind = "markdown" if is_markdown else "regular"
|
||||
|
||||
if self.file_service.is_markdown(path):
|
||||
try:
|
||||
logger.debug(f"Syncing file path={path} is_new={new} is_markdown={is_markdown}")
|
||||
|
||||
if is_markdown:
|
||||
entity, checksum = await self.sync_markdown_file(path, new)
|
||||
else:
|
||||
entity, checksum = await self.sync_regular_file(path, new)
|
||||
@@ -620,33 +682,63 @@ class SyncService:
|
||||
logger.debug(
|
||||
f"File sync completed, path={path}, entity_id={entity.id}, checksum={checksum[:8]}"
|
||||
)
|
||||
duration_ms = int((time.time() - start_time) * 1000)
|
||||
if duration_ms >= SLOW_FILE_SYNC_WARNING_MS:
|
||||
logger.warning(
|
||||
f"Slow file sync detected: path={path}, file_kind={file_kind}, duration_ms={duration_ms}"
|
||||
)
|
||||
return entity, checksum
|
||||
|
||||
except FileNotFoundError:
|
||||
# File exists in database but not on filesystem
|
||||
# This indicates a database/filesystem inconsistency - treat as deletion
|
||||
logger.warning(
|
||||
f"File not found during sync, treating as deletion: path={path}. "
|
||||
"This may indicate a race condition or manual file deletion."
|
||||
)
|
||||
await self.handle_delete(path)
|
||||
with telemetry.scope(
|
||||
"sync.file.failure",
|
||||
failure_type="file_not_found",
|
||||
path=path,
|
||||
file_kind=file_kind,
|
||||
is_new=new,
|
||||
is_fatal=False,
|
||||
):
|
||||
logger.warning(
|
||||
f"File not found during sync, treating as deletion: path={path}. "
|
||||
"This may indicate a race condition or manual file deletion."
|
||||
)
|
||||
await self.handle_delete(path)
|
||||
return None, None
|
||||
|
||||
except Exception as e:
|
||||
failure_type = type(e).__name__
|
||||
# Check if this is a fatal error (or caused by one)
|
||||
# Fatal errors like project deletion should terminate sync immediately
|
||||
if isinstance(e, SyncFatalError) or isinstance(
|
||||
e.__cause__, SyncFatalError
|
||||
): # pragma: no cover
|
||||
logger.error(f"Fatal sync error encountered, terminating sync: path={path}")
|
||||
with telemetry.scope(
|
||||
"sync.file.failure",
|
||||
failure_type=failure_type,
|
||||
path=path,
|
||||
file_kind=file_kind,
|
||||
is_new=new,
|
||||
is_fatal=True,
|
||||
):
|
||||
logger.error(f"Fatal sync error encountered, terminating sync: path={path}")
|
||||
raise
|
||||
|
||||
# Otherwise treat as recoverable file-level error
|
||||
error_msg = str(e)
|
||||
logger.error(f"Failed to sync file: path={path}, error={error_msg}")
|
||||
with telemetry.scope(
|
||||
"sync.file.failure",
|
||||
failure_type=failure_type,
|
||||
path=path,
|
||||
file_kind=file_kind,
|
||||
is_new=new,
|
||||
is_fatal=False,
|
||||
):
|
||||
logger.error(f"Failed to sync file: path={path}, error={error_msg}")
|
||||
|
||||
# Record failure for circuit breaker
|
||||
await self._record_failure(path, error_msg)
|
||||
# Record failure for circuit breaker
|
||||
await self._record_failure(path, error_msg)
|
||||
|
||||
return None, None
|
||||
|
||||
@@ -1040,24 +1132,36 @@ class SyncService:
|
||||
# update search index only on successful resolution
|
||||
await self.search_service.index_entity(resolved_entity)
|
||||
except IntegrityError:
|
||||
# IntegrityError means a relation with this (from_id, to_id, relation_type)
|
||||
# already exists. The UPDATE was rolled back, so our unresolved relation
|
||||
# (to_id=NULL) still exists in the database. We delete it because:
|
||||
# 1. It's redundant - a resolved relation already captures this relationship
|
||||
# 2. If we don't delete it, future syncs will try to resolve it again
|
||||
# and get the same IntegrityError
|
||||
logger.debug(
|
||||
"Deleting duplicate unresolved relation "
|
||||
f"relation_id={relation.id} "
|
||||
f"from_id={relation.from_id} "
|
||||
f"to_name={relation.to_name} "
|
||||
f"resolved_to_id={resolved_entity.id}"
|
||||
)
|
||||
try:
|
||||
await self.relation_repository.delete(relation.id)
|
||||
except Exception as e:
|
||||
# Log but don't fail - the relation may have been deleted already
|
||||
logger.debug(f"Could not delete duplicate relation {relation.id}: {e}")
|
||||
with telemetry.scope(
|
||||
"sync.relation.resolve_conflict",
|
||||
relation_id=relation.id,
|
||||
relation_type=relation.relation_type,
|
||||
):
|
||||
# IntegrityError means a relation with this (from_id, to_id, relation_type)
|
||||
# already exists. The UPDATE was rolled back, so our unresolved relation
|
||||
# (to_id=NULL) still exists in the database. We delete it because:
|
||||
# 1. It's redundant - a resolved version already captures this relationship
|
||||
# 2. If we don't delete it, future syncs will try to resolve it again
|
||||
# and get the same IntegrityError
|
||||
logger.debug(
|
||||
"Deleting duplicate unresolved relation "
|
||||
f"relation_id={relation.id} "
|
||||
f"from_id={relation.from_id} "
|
||||
f"to_name={relation.to_name} "
|
||||
f"resolved_to_id={resolved_entity.id}"
|
||||
)
|
||||
try:
|
||||
await self.relation_repository.delete(relation.id)
|
||||
except Exception as e:
|
||||
with telemetry.scope(
|
||||
"sync.relation.cleanup_failure",
|
||||
relation_id=relation.id,
|
||||
relation_type=relation.relation_type,
|
||||
):
|
||||
# Log but don't fail - the relation may have been deleted already
|
||||
logger.debug(
|
||||
f"Could not delete duplicate relation {relation.id}: {e}"
|
||||
)
|
||||
|
||||
async def _quick_count_files(self, directory: Path) -> int:
|
||||
"""Fast file count using find command.
|
||||
|
||||
@@ -0,0 +1,209 @@
|
||||
"""Optional Logfire telemetry helpers for Basic Memory.
|
||||
|
||||
Telemetry is disabled by default. When enabled, this module configures Logfire,
|
||||
exposes a `loguru` handler for trace-aware logging, and provides lightweight
|
||||
helpers for manual spans and logger context binding.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from contextlib import contextmanager
|
||||
from contextvars import ContextVar
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Any, Iterator
|
||||
|
||||
from loguru import logger
|
||||
|
||||
REPOSITORY_URL = "https://github.com/basicmachines-co/basic-memory"
|
||||
ROOT_PATH = "src/basic_memory"
|
||||
|
||||
|
||||
def _load_logfire() -> Any | None:
|
||||
"""Load the optional logfire dependency lazily."""
|
||||
try:
|
||||
import logfire
|
||||
except ImportError:
|
||||
return None
|
||||
return logfire
|
||||
|
||||
|
||||
@dataclass
|
||||
class TelemetryState:
|
||||
"""Process-local Logfire configuration state."""
|
||||
|
||||
enabled: bool = False
|
||||
configured: bool = False
|
||||
service_name: str | None = None
|
||||
environment: str | None = None
|
||||
send_to_logfire: bool = False
|
||||
warnings: list[str] = field(default_factory=list)
|
||||
|
||||
|
||||
_STATE = TelemetryState()
|
||||
_LOGFIRE_HANDLER: dict[str, Any] | None = None
|
||||
_ACTIVE_LOG_CONTEXT: ContextVar[dict[str, Any]] = ContextVar("basic_memory_log_context", default={})
|
||||
|
||||
|
||||
def reset_telemetry_state() -> None:
|
||||
"""Reset process-local telemetry state.
|
||||
|
||||
Primarily used by tests.
|
||||
"""
|
||||
global _LOGFIRE_HANDLER
|
||||
_STATE.enabled = False
|
||||
_STATE.configured = False
|
||||
_STATE.service_name = None
|
||||
_STATE.environment = None
|
||||
_STATE.send_to_logfire = False
|
||||
_STATE.warnings.clear()
|
||||
_LOGFIRE_HANDLER = None
|
||||
_ACTIVE_LOG_CONTEXT.set({})
|
||||
|
||||
|
||||
def _filter_attributes(attrs: dict[str, Any]) -> dict[str, Any]:
|
||||
"""Drop null attributes so span and log payloads stay compact."""
|
||||
return {key: value for key, value in attrs.items() if value is not None}
|
||||
|
||||
|
||||
def _current_log_context() -> dict[str, Any]:
|
||||
"""Return the currently active telemetry context for this execution flow."""
|
||||
return dict(_ACTIVE_LOG_CONTEXT.get())
|
||||
|
||||
|
||||
def configure_telemetry(
|
||||
service_name: str,
|
||||
*,
|
||||
environment: str,
|
||||
service_version: str | None = None,
|
||||
enable_logfire: bool = False,
|
||||
send_to_logfire: bool = False,
|
||||
log_level: str = "INFO",
|
||||
) -> bool:
|
||||
"""Configure optional Logfire instrumentation for the current process."""
|
||||
global _LOGFIRE_HANDLER
|
||||
|
||||
reset_telemetry_state()
|
||||
_STATE.service_name = service_name
|
||||
_STATE.environment = environment
|
||||
_STATE.send_to_logfire = send_to_logfire
|
||||
_STATE.enabled = enable_logfire
|
||||
|
||||
if not enable_logfire:
|
||||
return False
|
||||
|
||||
logfire = _load_logfire()
|
||||
if logfire is None:
|
||||
_STATE.enabled = False
|
||||
_STATE.warnings.append(
|
||||
"Logfire telemetry was enabled but the 'logfire' package is not installed. "
|
||||
"Telemetry remains disabled."
|
||||
)
|
||||
return False
|
||||
|
||||
configure_kwargs = {
|
||||
"service_name": service_name,
|
||||
"environment": environment,
|
||||
"code_source": logfire.CodeSource(
|
||||
repository=REPOSITORY_URL,
|
||||
revision=service_version or "",
|
||||
root_path=ROOT_PATH,
|
||||
),
|
||||
"min_level": log_level.lower(),
|
||||
"send_to_logfire": send_to_logfire,
|
||||
}
|
||||
|
||||
try:
|
||||
logfire.configure(**configure_kwargs)
|
||||
except TypeError:
|
||||
configure_kwargs.pop("send_to_logfire", None)
|
||||
logfire.configure(**configure_kwargs)
|
||||
except Exception as exc: # pragma: no cover
|
||||
_STATE.enabled = False # pragma: no cover
|
||||
_STATE.warnings.append(f"Failed to configure Logfire telemetry: {exc}") # pragma: no cover
|
||||
return False # pragma: no cover
|
||||
|
||||
_LOGFIRE_HANDLER = logfire.loguru_handler()
|
||||
_STATE.configured = True
|
||||
return True
|
||||
|
||||
|
||||
def telemetry_enabled() -> bool:
|
||||
"""Return True when telemetry is both enabled and configured."""
|
||||
return _STATE.enabled and _STATE.configured
|
||||
|
||||
|
||||
def get_logfire_handler() -> dict[str, Any] | None:
|
||||
"""Return the active Logfire `loguru` handler, if any."""
|
||||
return _LOGFIRE_HANDLER
|
||||
|
||||
|
||||
def pop_telemetry_warnings() -> list[str]:
|
||||
"""Return and clear pending telemetry warnings."""
|
||||
warnings = list(_STATE.warnings)
|
||||
_STATE.warnings.clear()
|
||||
return warnings
|
||||
|
||||
|
||||
def bind_telemetry_context(**attrs: Any):
|
||||
"""Bind stable telemetry attributes onto the shared Loguru logger."""
|
||||
merged_attrs = _current_log_context()
|
||||
merged_attrs.update(_filter_attributes(attrs))
|
||||
return logger.bind(**merged_attrs)
|
||||
|
||||
|
||||
@contextmanager
|
||||
def contextualize(**attrs: Any) -> Iterator[None]:
|
||||
"""Apply stable telemetry attributes to all Loguru calls in this scope."""
|
||||
filtered_attrs = _filter_attributes(attrs)
|
||||
merged_attrs = _current_log_context()
|
||||
merged_attrs.update(filtered_attrs)
|
||||
context_token = _ACTIVE_LOG_CONTEXT.set(merged_attrs)
|
||||
|
||||
try:
|
||||
with logger.contextualize(**filtered_attrs):
|
||||
yield
|
||||
finally:
|
||||
_ACTIVE_LOG_CONTEXT.reset(context_token)
|
||||
|
||||
|
||||
@contextmanager
|
||||
def scope(name: str, **attrs: Any) -> Iterator[None]:
|
||||
"""Create a span and bind the same stable attributes into Loguru context."""
|
||||
with contextualize(**attrs):
|
||||
with span(name, **attrs):
|
||||
yield
|
||||
|
||||
|
||||
# Alias: `operation` signals a root-level boundary (entrypoint, tool invocation),
|
||||
# while `scope` signals a nested phase. The distinction is convention only.
|
||||
operation = scope
|
||||
|
||||
|
||||
@contextmanager
|
||||
def span(name: str, **attrs: Any) -> Iterator[None]:
|
||||
"""Create a manual Logfire span when telemetry is enabled."""
|
||||
if not telemetry_enabled():
|
||||
yield
|
||||
return
|
||||
|
||||
logfire = _load_logfire()
|
||||
if logfire is None: # pragma: no cover
|
||||
yield # pragma: no cover
|
||||
return # pragma: no cover
|
||||
|
||||
with logfire.span(name, **_filter_attributes(attrs)):
|
||||
yield
|
||||
|
||||
|
||||
__all__ = [
|
||||
"bind_telemetry_context",
|
||||
"contextualize",
|
||||
"configure_telemetry",
|
||||
"get_logfire_handler",
|
||||
"operation",
|
||||
"pop_telemetry_warnings",
|
||||
"reset_telemetry_state",
|
||||
"scope",
|
||||
"span",
|
||||
"telemetry_enabled",
|
||||
]
|
||||
@@ -1,5 +1,6 @@
|
||||
"""Utility functions for basic-memory."""
|
||||
|
||||
import json
|
||||
import os
|
||||
|
||||
import logging
|
||||
@@ -7,11 +8,13 @@ import re
|
||||
import sys
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
from typing import Protocol, Union, runtime_checkable, List, Optional
|
||||
from typing import Any, Protocol, Union, runtime_checkable, List, Optional
|
||||
|
||||
from loguru import logger
|
||||
from unidecode import unidecode
|
||||
|
||||
from basic_memory import telemetry
|
||||
|
||||
|
||||
def normalize_project_path(path: str) -> str:
|
||||
"""Normalize project path by stripping mount point prefix.
|
||||
@@ -66,6 +69,7 @@ class PathLike(Protocol):
|
||||
# In type annotations, use Union[Path, str] instead of FilePath for now
|
||||
# This preserves compatibility with existing code while we migrate
|
||||
FilePath = Union[Path, str]
|
||||
WINDOWS_LOG_FILE_RETENTION = 5
|
||||
|
||||
|
||||
def generate_permalink(file_path: Union[Path, str, PathLike], split_extension: bool = True) -> str:
|
||||
@@ -250,7 +254,7 @@ def setup_logging(
|
||||
log_to_file: bool = False,
|
||||
log_to_stdout: bool = False,
|
||||
structured_context: bool = False,
|
||||
) -> None: # pragma: no cover
|
||||
) -> None:
|
||||
"""Configure logging with explicit settings.
|
||||
|
||||
This function provides a simple, explicit interface for configuring logging.
|
||||
@@ -273,8 +277,14 @@ def setup_logging(
|
||||
|
||||
# Add file handler with rotation
|
||||
if log_to_file:
|
||||
log_path = Path.home() / ".basic-memory" / "basic-memory.log"
|
||||
# Trigger: Windows does not allow renaming an open file held by another process.
|
||||
# Why: multiple basic-memory processes can share the same log directory at once.
|
||||
# Outcome: use per-process log files on Windows so log rotation stays local.
|
||||
log_filename = f"basic-memory-{os.getpid()}.log" if os.name == "nt" else "basic-memory.log"
|
||||
log_path = Path.home() / ".basic-memory" / log_filename
|
||||
log_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
if os.name == "nt":
|
||||
_cleanup_windows_log_files(log_path.parent, log_path.name)
|
||||
# Keep logging synchronous (enqueue=False) to avoid background logging threads.
|
||||
# Background threads are a common source of "hang on exit" issues in CLI/test runs.
|
||||
logger.add(
|
||||
@@ -292,6 +302,11 @@ def setup_logging(
|
||||
if log_to_stdout:
|
||||
logger.add(sys.stderr, level=log_level, backtrace=True, diagnose=True, colorize=True)
|
||||
|
||||
# Add Logfire sink when telemetry bootstrap enabled it for this process.
|
||||
logfire_handler = telemetry.get_logfire_handler()
|
||||
if logfire_handler is not None:
|
||||
logger.add(**logfire_handler)
|
||||
|
||||
# Bind structured context for cloud observability
|
||||
if structured_context:
|
||||
logger.configure(
|
||||
@@ -307,6 +322,31 @@ def setup_logging(
|
||||
logging.getLogger("httpx").setLevel(logging.WARNING)
|
||||
logging.getLogger("watchfiles.main").setLevel(logging.WARNING)
|
||||
|
||||
for warning_message in telemetry.pop_telemetry_warnings():
|
||||
logger.warning(warning_message)
|
||||
|
||||
|
||||
def _cleanup_windows_log_files(log_dir: Path, current_log_name: str) -> None:
|
||||
"""Trim stale per-process Windows log files so the directory stays bounded."""
|
||||
stale_logs = [
|
||||
path
|
||||
for path in log_dir.glob("basic-memory-*.log*")
|
||||
if path.is_file() and path.name != current_log_name
|
||||
]
|
||||
|
||||
if len(stale_logs) <= WINDOWS_LOG_FILE_RETENTION - 1:
|
||||
return
|
||||
|
||||
# Trigger: per-process log filenames avoid Windows rename contention but fragment retention.
|
||||
# Why: loguru retention applies per sink, not across the whole basic-memory log directory.
|
||||
# Outcome: keep only the newest stale PID logs so repeated CLI/server launches stay bounded.
|
||||
stale_logs.sort(key=lambda path: path.stat().st_mtime, reverse=True)
|
||||
for stale_log in stale_logs[WINDOWS_LOG_FILE_RETENTION - 1 :]:
|
||||
try:
|
||||
stale_log.unlink()
|
||||
except OSError:
|
||||
logger.debug("Failed to delete stale Windows log file: {path}", path=stale_log)
|
||||
|
||||
|
||||
def parse_tags(tags: Union[List[str], str, None]) -> List[str]:
|
||||
"""Parse tags from various input formats into a consistent list.
|
||||
@@ -356,6 +396,36 @@ def parse_tags(tags: Union[List[str], str, None]) -> List[str]:
|
||||
return []
|
||||
|
||||
|
||||
def coerce_list(v: Any) -> Any:
|
||||
"""Coerce string input to list for MCP clients that serialize lists as strings."""
|
||||
if v is None:
|
||||
return v
|
||||
if isinstance(v, str):
|
||||
try:
|
||||
parsed = json.loads(v)
|
||||
if isinstance(parsed, list):
|
||||
return parsed
|
||||
except (json.JSONDecodeError, TypeError):
|
||||
pass
|
||||
# Single string value — wrap in a list
|
||||
return [v]
|
||||
return v
|
||||
|
||||
|
||||
def coerce_dict(v: Any) -> Any:
|
||||
"""Coerce string input to dict for MCP clients that serialize dicts as strings."""
|
||||
if v is None:
|
||||
return v
|
||||
if isinstance(v, str):
|
||||
try:
|
||||
parsed = json.loads(v)
|
||||
if isinstance(parsed, dict):
|
||||
return parsed
|
||||
except (json.JSONDecodeError, TypeError):
|
||||
pass
|
||||
return v
|
||||
|
||||
|
||||
def normalize_newlines(multiline: str) -> str:
|
||||
"""Replace any \r\n, \r, or \n with the native newline.
|
||||
|
||||
@@ -443,12 +513,23 @@ def valid_project_path_value(path: str):
|
||||
if not path:
|
||||
return True
|
||||
|
||||
# Check for obvious path traversal patterns first
|
||||
if ".." in path or "~" in path:
|
||||
# Check for tilde (home directory expansion)
|
||||
if "~" in path:
|
||||
return False
|
||||
|
||||
# Check for Windows-style path traversal (even on Unix systems)
|
||||
if "\\.." in path or path.startswith("\\"):
|
||||
# Check for ".." as a path segment (path traversal), not as a substring.
|
||||
# Filenames like "hi-everyone..md" are legitimate and must not be blocked.
|
||||
# Also block segments like ".. " and ".. ." because Windows normalizes
|
||||
# trailing dots and spaces away, making them equivalent to "..".
|
||||
segments = path.replace("\\", "/").split("/")
|
||||
if any(
|
||||
seg == ".." or (len(seg) > 2 and seg[:2] == ".." and all(c in ". " for c in seg[2:]))
|
||||
for seg in segments
|
||||
):
|
||||
return False
|
||||
|
||||
# Check for Windows-style leading backslash
|
||||
if path.startswith("\\"):
|
||||
return False
|
||||
|
||||
# Block absolute paths (Unix-style starting with / or Windows-style with drive letters)
|
||||
|
||||
@@ -208,7 +208,7 @@ def test_edit_note_replace_section_fails_without_section(
|
||||
)
|
||||
|
||||
assert result.exit_code != 0
|
||||
assert "section parameter is required for replace_section operation" in result.output
|
||||
assert "section parameter is required for section-based operations" in result.output
|
||||
|
||||
|
||||
def test_edit_note_append_creates_nonexistent_note_cli(
|
||||
|
||||
@@ -258,6 +258,8 @@ def config_manager(app_config: BasicMemoryConfig, config_home) -> ConfigManager:
|
||||
from basic_memory import config as config_module
|
||||
|
||||
config_module._CONFIG_CACHE = None
|
||||
config_module._CONFIG_MTIME = None
|
||||
config_module._CONFIG_SIZE = None
|
||||
|
||||
config_manager = ConfigManager()
|
||||
# Update its paths to use the test directory
|
||||
|
||||
@@ -307,8 +307,13 @@ async def test_delete_note_by_file_path(mcp_server, app, test_project):
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_delete_note_case_insensitive(mcp_server, app, test_project):
|
||||
"""Test that note deletion is case insensitive for titles."""
|
||||
async def test_delete_note_rejects_case_mismatch(mcp_server, app, test_project):
|
||||
"""Test that delete_note with wrong case does not fuzzy-match to an existing note.
|
||||
|
||||
Strict resolution (#649) prevents destructive operations from silently
|
||||
resolving to a different note via fuzzy search. Case-mismatched titles
|
||||
should be rejected, not resolved to the nearest match.
|
||||
"""
|
||||
|
||||
async with Client(mcp_server) as client:
|
||||
# Create a note with mixed case
|
||||
@@ -323,7 +328,7 @@ async def test_delete_note_case_insensitive(mcp_server, app, test_project):
|
||||
},
|
||||
)
|
||||
|
||||
# Try to delete with different case
|
||||
# Try to delete with different case — should NOT find the note
|
||||
delete_result = await client.call_tool(
|
||||
"delete_note",
|
||||
{
|
||||
@@ -332,8 +337,28 @@ async def test_delete_note_case_insensitive(mcp_server, app, test_project):
|
||||
},
|
||||
)
|
||||
|
||||
# Should return True for successful deletion
|
||||
assert "true" in delete_result.content[0].text.lower()
|
||||
# Should return False (not found) — strict mode rejects fuzzy matches
|
||||
assert "false" in delete_result.content[0].text.lower()
|
||||
|
||||
# Verify the note still exists using the exact title
|
||||
read_result = await client.call_tool(
|
||||
"read_note",
|
||||
{
|
||||
"project": test_project.name,
|
||||
"identifier": "CamelCase Note Title",
|
||||
},
|
||||
)
|
||||
assert "Testing case sensitivity" in read_result.content[0].text
|
||||
|
||||
# Delete with exact title should succeed
|
||||
delete_result2 = await client.call_tool(
|
||||
"delete_note",
|
||||
{
|
||||
"project": test_project.name,
|
||||
"identifier": "CamelCase Note Title",
|
||||
},
|
||||
)
|
||||
assert "true" in delete_result2.content[0].text.lower()
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
|
||||
@@ -710,3 +710,81 @@ async def test_edit_note_using_different_identifiers(mcp_server, app, test_proje
|
||||
assert "Edited by title." in content
|
||||
assert "Edited by permalink." in content
|
||||
assert "Edited by folder/title." in content
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_edit_note_append_autocreate_does_not_fuzzy_match(mcp_server, app, test_project):
|
||||
"""Reproduces #649: edit_note append must auto-create, not fuzzy-match to an existing note.
|
||||
|
||||
Creates two notes, then attempts to append to a nonexistent identifier.
|
||||
The tool should create a new note, and neither existing note should be modified.
|
||||
"""
|
||||
|
||||
async with Client(mcp_server) as client:
|
||||
# Create two notes that could be fuzzy-matched
|
||||
await client.call_tool(
|
||||
"write_note",
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "Routing Test A",
|
||||
"directory": "test",
|
||||
"content": "# Routing Test A\n\nContent A.",
|
||||
},
|
||||
)
|
||||
await client.call_tool(
|
||||
"write_note",
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "Routing Test B",
|
||||
"directory": "test",
|
||||
"content": "# Routing Test B\n\nContent B.",
|
||||
},
|
||||
)
|
||||
|
||||
# Attempt to edit a nonexistent note — should error, not silently edit A or B
|
||||
edit_result = await client.call_tool(
|
||||
"edit_note",
|
||||
{
|
||||
"project": test_project.name,
|
||||
"identifier": "Routing Test NONEXISTENT",
|
||||
"operation": "append",
|
||||
"content": "\n\nThis should NOT appear in any note.",
|
||||
},
|
||||
)
|
||||
|
||||
edit_text = edit_result.content[0].text
|
||||
# append to nonexistent creates a new note — verify it did NOT edit A or B
|
||||
assert "Created note (append)" in edit_text
|
||||
assert "fileCreated: true" in edit_text
|
||||
|
||||
# Verify neither A nor B was modified
|
||||
read_a = await client.call_tool(
|
||||
"read_note",
|
||||
{"project": test_project.name, "identifier": "Routing Test A"},
|
||||
)
|
||||
content_a = read_a.content[0].text
|
||||
assert "Content A" in content_a
|
||||
assert "This should NOT appear" not in content_a
|
||||
|
||||
read_b = await client.call_tool(
|
||||
"read_note",
|
||||
{"project": test_project.name, "identifier": "Routing Test B"},
|
||||
)
|
||||
content_b = read_b.content[0].text
|
||||
assert "Content B" in content_b
|
||||
assert "This should NOT appear" not in content_b
|
||||
|
||||
# Now test find_replace on nonexistent — should error
|
||||
edit_result2 = await client.call_tool(
|
||||
"edit_note",
|
||||
{
|
||||
"project": test_project.name,
|
||||
"identifier": "Routing Test NONEXISTENT AGAIN",
|
||||
"operation": "find_replace",
|
||||
"content": "replaced",
|
||||
"find_text": "Content",
|
||||
},
|
||||
)
|
||||
|
||||
error_text = edit_result2.content[0].text
|
||||
assert "Edit Failed" in error_text
|
||||
|
||||
@@ -716,3 +716,56 @@ async def test_move_note_destination_folder_mutually_exclusive(mcp_server, app,
|
||||
error_text = move_result.content[0].text
|
||||
assert "# Move Failed - Invalid Parameters" in error_text
|
||||
assert "Cannot specify both" in error_text
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_move_note_strict_resolution_rejects_fuzzy_match(mcp_server, app, test_project):
|
||||
"""move_note must not fuzzy-match a nonexistent identifier to an existing note (#649)."""
|
||||
|
||||
async with Client(mcp_server) as client:
|
||||
# Create two notes that could be fuzzy-matched
|
||||
await client.call_tool(
|
||||
"write_note",
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "Move Strict Test A",
|
||||
"directory": "test",
|
||||
"content": "# Move Strict Test A\n\nContent A.",
|
||||
},
|
||||
)
|
||||
await client.call_tool(
|
||||
"write_note",
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "Move Strict Test B",
|
||||
"directory": "test",
|
||||
"content": "# Move Strict Test B\n\nContent B.",
|
||||
},
|
||||
)
|
||||
|
||||
# Attempt to move a nonexistent note — should error, not move A or B
|
||||
move_result = await client.call_tool(
|
||||
"move_note",
|
||||
{
|
||||
"project": test_project.name,
|
||||
"identifier": "Move Strict Test NONEXISTENT",
|
||||
"destination_path": "archive/Moved.md",
|
||||
},
|
||||
)
|
||||
|
||||
assert len(move_result.content) == 1
|
||||
error_text = move_result.content[0].text
|
||||
assert "# Move Failed" in error_text
|
||||
|
||||
# Verify neither A nor B was moved
|
||||
read_a = await client.call_tool(
|
||||
"read_note",
|
||||
{"project": test_project.name, "identifier": "Move Strict Test A"},
|
||||
)
|
||||
assert "Content A" in read_a.content[0].text
|
||||
|
||||
read_b = await client.call_tool(
|
||||
"read_note",
|
||||
{"project": test_project.name, "identifier": "Move Strict Test B"},
|
||||
)
|
||||
assert "Content B" in read_b.content[0].text
|
||||
|
||||
@@ -0,0 +1,165 @@
|
||||
"""Integration tests for MCP tools accepting string-serialized list/dict params.
|
||||
|
||||
Goes through the full FastMCP Client → validate_call → tool function path,
|
||||
which is where Pydantic rejects strings for list/dict params.
|
||||
"""
|
||||
|
||||
import pytest
|
||||
from fastmcp import Client
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_search_notes_entity_types_as_string(mcp_server, app, test_project):
|
||||
"""search_notes should accept entity_types as a JSON string via MCP protocol."""
|
||||
async with Client(mcp_server) as client:
|
||||
await client.call_tool(
|
||||
"write_note",
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "Entity Type Coerce Test",
|
||||
"directory": "test",
|
||||
"content": "# Test\nContent for entity type coercion",
|
||||
},
|
||||
)
|
||||
|
||||
# MCP client sends entity_types as a string
|
||||
result = await client.call_tool(
|
||||
"search_notes",
|
||||
{
|
||||
"project": test_project.name,
|
||||
"query": "coercion",
|
||||
"entity_types": '["entity"]',
|
||||
},
|
||||
)
|
||||
text = result.content[0].text
|
||||
assert "Search Failed" not in text
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_search_notes_note_types_as_string(mcp_server, app, test_project):
|
||||
"""search_notes should accept note_types as a JSON string via MCP protocol."""
|
||||
async with Client(mcp_server) as client:
|
||||
await client.call_tool(
|
||||
"write_note",
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "Note Type Coerce Test",
|
||||
"directory": "test",
|
||||
"content": "# Test\nContent for note type coercion",
|
||||
},
|
||||
)
|
||||
|
||||
result = await client.call_tool(
|
||||
"search_notes",
|
||||
{
|
||||
"project": test_project.name,
|
||||
"query": "coercion",
|
||||
"note_types": '["note"]',
|
||||
},
|
||||
)
|
||||
text = result.content[0].text
|
||||
assert "Search Failed" not in text
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_search_notes_tags_as_string(mcp_server, app, test_project):
|
||||
"""search_notes should accept tags as a JSON string via MCP protocol."""
|
||||
async with Client(mcp_server) as client:
|
||||
await client.call_tool(
|
||||
"write_note",
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "Tags Coerce Test",
|
||||
"directory": "test",
|
||||
"content": "# Test\nTagged content for coercion",
|
||||
"tags": "alpha",
|
||||
},
|
||||
)
|
||||
|
||||
result = await client.call_tool(
|
||||
"search_notes",
|
||||
{
|
||||
"project": test_project.name,
|
||||
"query": "tagged",
|
||||
"tags": '["alpha"]',
|
||||
},
|
||||
)
|
||||
text = result.content[0].text
|
||||
assert "Search Failed" not in text
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_search_notes_metadata_filters_as_string(mcp_server, app, test_project):
|
||||
"""search_notes should accept metadata_filters as a JSON string via MCP protocol."""
|
||||
async with Client(mcp_server) as client:
|
||||
await client.call_tool(
|
||||
"write_note",
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "Metadata Coerce Test",
|
||||
"directory": "test",
|
||||
"content": "# Test\nMetadata content for coercion",
|
||||
},
|
||||
)
|
||||
|
||||
result = await client.call_tool(
|
||||
"search_notes",
|
||||
{
|
||||
"project": test_project.name,
|
||||
"query": "metadata",
|
||||
"metadata_filters": '{"type": "note"}',
|
||||
},
|
||||
)
|
||||
text = result.content[0].text
|
||||
assert "Search Failed" not in text
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_write_note_metadata_as_string(mcp_server, app, test_project):
|
||||
"""write_note should accept metadata as a JSON string via MCP protocol."""
|
||||
async with Client(mcp_server) as client:
|
||||
result = await client.call_tool(
|
||||
"write_note",
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "String Metadata Note",
|
||||
"directory": "test",
|
||||
"content": "# Test\nWith string metadata",
|
||||
"metadata": '{"priority": "high"}',
|
||||
},
|
||||
)
|
||||
text = result.content[0].text
|
||||
assert "Created note" in text or "Updated note" in text
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_canvas_nodes_edges_as_string(mcp_server, app, test_project):
|
||||
"""canvas should accept nodes and edges as JSON strings via MCP protocol."""
|
||||
import json
|
||||
|
||||
nodes = [
|
||||
{
|
||||
"id": "n1",
|
||||
"type": "text",
|
||||
"text": "Hello",
|
||||
"x": 0,
|
||||
"y": 0,
|
||||
"width": 200,
|
||||
"height": 100,
|
||||
}
|
||||
]
|
||||
edges = [{"id": "e1", "fromNode": "n1", "toNode": "n1", "label": "self"}]
|
||||
|
||||
async with Client(mcp_server) as client:
|
||||
result = await client.call_tool(
|
||||
"canvas",
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "Coerce Canvas Test",
|
||||
"directory": "test",
|
||||
"nodes": json.dumps(nodes),
|
||||
"edges": json.dumps(edges),
|
||||
},
|
||||
)
|
||||
text = result.content[0].text
|
||||
assert "Created" in text or "Updated" in text
|
||||
@@ -1,120 +0,0 @@
|
||||
"""Tests for v2 graph intelligence and FCM routers."""
|
||||
|
||||
import pytest
|
||||
from httpx import AsyncClient
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_graph_lineage_contract(client: AsyncClient, v2_project_url: str):
|
||||
response = await client.post(
|
||||
f"{v2_project_url}/graph/lineage",
|
||||
json={"start": "memory://specs/search"},
|
||||
)
|
||||
|
||||
assert response.status_code == 200
|
||||
data = response.json()
|
||||
assert set(["root", "paths", "generated_at"]).issubset(data.keys())
|
||||
assert data["root"]["id"] == "specs/search"
|
||||
assert isinstance(data["paths"], list)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_graph_impact_contract(client: AsyncClient, v2_project_url: str):
|
||||
response = await client.post(
|
||||
f"{v2_project_url}/graph/impact",
|
||||
json={"target": "memory://specs/search", "horizon": 2},
|
||||
)
|
||||
|
||||
assert response.status_code == 200
|
||||
data = response.json()
|
||||
assert set(["target", "affected", "summary"]).issubset(data.keys())
|
||||
assert data["summary"]["total_considered"] >= data["summary"]["total_returned"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_graph_health_contract(client: AsyncClient, v2_project_url: str):
|
||||
response = await client.get(
|
||||
f"{v2_project_url}/graph/health",
|
||||
params={"scope": "specs", "timeframe": "30d"},
|
||||
)
|
||||
|
||||
assert response.status_code == 200
|
||||
data = response.json()
|
||||
assert set(["metrics", "issues", "computed_at"]).issubset(data.keys())
|
||||
assert "orphan_rate" in data["metrics"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_graph_reindex_schedules_task(
|
||||
client: AsyncClient,
|
||||
v2_project_url: str,
|
||||
task_scheduler_spy: list[dict[str, object]],
|
||||
):
|
||||
response = await client.post(
|
||||
f"{v2_project_url}/graph/reindex",
|
||||
json={"mode": "full", "reason": "contract test"},
|
||||
)
|
||||
|
||||
assert response.status_code == 200
|
||||
data = response.json()
|
||||
assert data["status"] == "queued"
|
||||
assert data["job_id"]
|
||||
|
||||
assert task_scheduler_spy
|
||||
last = task_scheduler_spy[-1]
|
||||
assert last["task_name"] == "reindex_graph_project"
|
||||
assert last["payload"]["mode"] == "full"
|
||||
assert last["payload"]["reason"] == "contract test"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_fcm_simulate_contract(client: AsyncClient, v2_project_url: str):
|
||||
response = await client.post(
|
||||
f"{v2_project_url}/fcm/simulate",
|
||||
json={"actions": [{"node_id": "test-node", "delta": 0.2}]},
|
||||
)
|
||||
|
||||
assert response.status_code == 200
|
||||
data = response.json()
|
||||
assert set(["baseline", "projected", "deltas", "stability", "confidence"]).issubset(data.keys())
|
||||
assert data["stability"]["converged"] is True
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_fcm_rank_actions_contract(client: AsyncClient, v2_project_url: str):
|
||||
response = await client.post(
|
||||
f"{v2_project_url}/fcm/rank-actions",
|
||||
json={"goal": "reduce-regressions", "top_k": 2},
|
||||
)
|
||||
|
||||
assert response.status_code == 200
|
||||
data = response.json()
|
||||
assert set(["goal", "recommendations"]).issubset(data.keys())
|
||||
assert len(data["recommendations"]) <= 2
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_fcm_import_contract(client: AsyncClient, v2_project_url: str):
|
||||
response = await client.post(
|
||||
f"{v2_project_url}/fcm/import",
|
||||
json={"source": "/tmp/model.csv", "format": "csv_bundle_v1"},
|
||||
)
|
||||
|
||||
assert response.status_code == 200
|
||||
data = response.json()
|
||||
assert set(["import_id", "nodes_loaded", "edges_loaded", "warnings", "errors"]).issubset(
|
||||
data.keys()
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_fcm_export_contract(client: AsyncClient, v2_project_url: str):
|
||||
response = await client.post(
|
||||
f"{v2_project_url}/fcm/export",
|
||||
json={"format": "csv_bundle_v1", "selection": {"scope": "all"}},
|
||||
)
|
||||
|
||||
assert response.status_code == 200
|
||||
data = response.json()
|
||||
assert set(["export_id", "format", "files", "node_count", "edge_count"]).issubset(data.keys())
|
||||
assert len(data["files"]) == 2
|
||||
@@ -0,0 +1,62 @@
|
||||
"""Telemetry coverage for the v2 search router."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import importlib
|
||||
from contextlib import contextmanager
|
||||
|
||||
import pytest
|
||||
|
||||
from basic_memory.schemas.search import SearchQuery
|
||||
|
||||
search_router_module = importlib.import_module("basic_memory.api.v2.routers.search_router")
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_search_router_wraps_request_in_manual_operation() -> None:
|
||||
operations: list[tuple[str, dict]] = []
|
||||
|
||||
class FakeSearchService:
|
||||
async def search(self, query, *, limit, offset):
|
||||
return []
|
||||
|
||||
@contextmanager
|
||||
def fake_operation(name: str, **attrs):
|
||||
operations.append((name, attrs))
|
||||
yield
|
||||
|
||||
async def fake_to_search_results(entity_service, results):
|
||||
return []
|
||||
|
||||
original_operation = search_router_module.telemetry.operation
|
||||
original_to_search_results = search_router_module.to_search_results
|
||||
search_router_module.telemetry.operation = fake_operation
|
||||
search_router_module.to_search_results = fake_to_search_results
|
||||
try:
|
||||
response = await search_router_module.search(
|
||||
SearchQuery(text="hello world"),
|
||||
FakeSearchService(),
|
||||
object(),
|
||||
project_id="project-123",
|
||||
page=2,
|
||||
page_size=5,
|
||||
)
|
||||
finally:
|
||||
search_router_module.telemetry.operation = original_operation
|
||||
search_router_module.to_search_results = original_to_search_results
|
||||
|
||||
assert response.current_page == 2
|
||||
assert operations == [
|
||||
(
|
||||
"api.request.search",
|
||||
{
|
||||
"entrypoint": "api",
|
||||
"page": 2,
|
||||
"page_size": 5,
|
||||
"retrieval_mode": "fts",
|
||||
"has_text_query": True,
|
||||
"has_title_query": False,
|
||||
"has_permalink_query": False,
|
||||
},
|
||||
)
|
||||
]
|
||||
@@ -25,6 +25,8 @@ def isolated_home(tmp_path, monkeypatch) -> Path:
|
||||
from basic_memory import config as config_module
|
||||
|
||||
config_module._CONFIG_CACHE = None
|
||||
config_module._CONFIG_MTIME = None
|
||||
config_module._CONFIG_SIZE = None
|
||||
|
||||
monkeypatch.setenv("HOME", str(tmp_path))
|
||||
if os.name == "nt":
|
||||
|
||||
@@ -0,0 +1,399 @@
|
||||
"""Tests for CLI auto-update behavior."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import subprocess
|
||||
from datetime import datetime, timedelta, timezone
|
||||
from io import StringIO
|
||||
|
||||
from rich.console import Console
|
||||
|
||||
from basic_memory.cli.auto_update import (
|
||||
AutoUpdateResult,
|
||||
AutoUpdateStatus,
|
||||
InstallSource,
|
||||
_check_homebrew_update_available,
|
||||
_is_interactive_session,
|
||||
detect_install_source,
|
||||
maybe_run_periodic_auto_update,
|
||||
run_auto_update,
|
||||
)
|
||||
from basic_memory.config import BasicMemoryConfig
|
||||
|
||||
|
||||
class StubConfigManager:
|
||||
"""Simple in-memory ConfigManager stub for updater tests."""
|
||||
|
||||
def __init__(self, config: BasicMemoryConfig):
|
||||
self._config = config
|
||||
self.save_calls = 0
|
||||
|
||||
def load_config(self) -> BasicMemoryConfig:
|
||||
return self._config
|
||||
|
||||
def save_config(self, config: BasicMemoryConfig) -> None:
|
||||
self._config = config
|
||||
self.save_calls += 1
|
||||
|
||||
|
||||
def _capture_console() -> tuple[Console, StringIO]:
|
||||
"""Create a Console that writes to an in-memory buffer."""
|
||||
buf = StringIO()
|
||||
return Console(file=buf, force_terminal=True), buf
|
||||
|
||||
|
||||
def _base_config(tmp_path) -> BasicMemoryConfig:
|
||||
return BasicMemoryConfig(projects={"main": {"path": str(tmp_path / "main")}})
|
||||
|
||||
|
||||
def _result(
|
||||
status: AutoUpdateStatus,
|
||||
*,
|
||||
message: str | None,
|
||||
error: str | None = None,
|
||||
) -> AutoUpdateResult:
|
||||
return AutoUpdateResult(
|
||||
status=status,
|
||||
source=InstallSource.UV_TOOL,
|
||||
checked=True,
|
||||
update_available=status in {AutoUpdateStatus.UPDATE_AVAILABLE, AutoUpdateStatus.UPDATED},
|
||||
updated=status == AutoUpdateStatus.UPDATED,
|
||||
latest_version="9.9.9",
|
||||
message=message,
|
||||
error=error,
|
||||
restart_recommended=status == AutoUpdateStatus.UPDATED,
|
||||
)
|
||||
|
||||
|
||||
def test_detect_install_source_variants():
|
||||
assert (
|
||||
detect_install_source("/opt/homebrew/Cellar/basic-memory/0.18.0/bin/python")
|
||||
== InstallSource.HOMEBREW
|
||||
)
|
||||
assert (
|
||||
detect_install_source("/Users/me/.local/share/uv/tools/basic-memory/bin/python")
|
||||
== InstallSource.UV_TOOL
|
||||
)
|
||||
assert (
|
||||
detect_install_source("/Users/me/.cache/uv/archive-v0/abc123/bin/python")
|
||||
== InstallSource.UVX
|
||||
)
|
||||
assert (
|
||||
detect_install_source("/Users/me/Library/Caches/uv/archive-v0/abc123/bin/python")
|
||||
== InstallSource.UVX
|
||||
)
|
||||
assert detect_install_source("/usr/local/bin/python3") == InstallSource.UNKNOWN
|
||||
|
||||
|
||||
def test_interval_gate_skips_check_when_recent(tmp_path):
|
||||
config = _base_config(tmp_path)
|
||||
config.auto_update_last_checked_at = datetime.now() - timedelta(seconds=30)
|
||||
config.update_check_interval = 3600
|
||||
manager = StubConfigManager(config)
|
||||
|
||||
result = run_auto_update(config_manager=manager)
|
||||
|
||||
assert result.status == AutoUpdateStatus.SKIPPED
|
||||
assert result.checked is False
|
||||
assert manager.save_calls == 0
|
||||
|
||||
|
||||
def test_auto_update_disabled_skips_periodic(tmp_path):
|
||||
config = _base_config(tmp_path)
|
||||
config.auto_update = False
|
||||
manager = StubConfigManager(config)
|
||||
|
||||
result = run_auto_update(config_manager=manager)
|
||||
|
||||
assert result.status == AutoUpdateStatus.SKIPPED
|
||||
assert result.checked is False
|
||||
|
||||
|
||||
def test_force_bypasses_auto_update_disabled(monkeypatch, tmp_path):
|
||||
config = _base_config(tmp_path)
|
||||
config.auto_update = False
|
||||
manager = StubConfigManager(config)
|
||||
|
||||
monkeypatch.setattr(
|
||||
"basic_memory.cli.auto_update._check_pypi_update_available",
|
||||
lambda: (False, "0.0.0"),
|
||||
)
|
||||
|
||||
result = run_auto_update(
|
||||
force=True,
|
||||
config_manager=manager,
|
||||
executable="/Users/me/.local/share/uv/tools/basic-memory/bin/python",
|
||||
)
|
||||
|
||||
assert result.status == AutoUpdateStatus.UP_TO_DATE
|
||||
assert result.checked is True
|
||||
assert manager.save_calls == 1
|
||||
|
||||
|
||||
def test_check_homebrew_update_available_exit_code_1_means_outdated(monkeypatch):
|
||||
"""brew outdated exits 1 when the formula is outdated, not on error."""
|
||||
|
||||
def _fake_run(command, **kwargs):
|
||||
return subprocess.CompletedProcess(
|
||||
command, 1, stdout="basicmachines-co/basic-memory/basic-memory\n", stderr=""
|
||||
)
|
||||
|
||||
monkeypatch.setattr("basic_memory.cli.auto_update._run_subprocess", _fake_run)
|
||||
is_outdated, _ = _check_homebrew_update_available(silent=False)
|
||||
assert is_outdated is True
|
||||
|
||||
|
||||
def test_check_homebrew_update_available_exit_code_0_means_up_to_date(monkeypatch):
|
||||
"""brew outdated exits 0 when the formula is up to date."""
|
||||
|
||||
def _fake_run(command, **kwargs):
|
||||
return subprocess.CompletedProcess(command, 0, stdout="", stderr="")
|
||||
|
||||
monkeypatch.setattr("basic_memory.cli.auto_update._run_subprocess", _fake_run)
|
||||
is_outdated, _ = _check_homebrew_update_available(silent=False)
|
||||
assert is_outdated is False
|
||||
|
||||
|
||||
def test_homebrew_outdated_triggers_upgrade(monkeypatch, tmp_path):
|
||||
config = _base_config(tmp_path)
|
||||
manager = StubConfigManager(config)
|
||||
|
||||
monkeypatch.setattr(
|
||||
"basic_memory.cli.auto_update._check_homebrew_update_available",
|
||||
lambda silent: (True, None),
|
||||
)
|
||||
calls: list[list[str]] = []
|
||||
|
||||
def _fake_run_subprocess(command, **kwargs):
|
||||
calls.append(command)
|
||||
return subprocess.CompletedProcess(command, 0, stdout="", stderr="")
|
||||
|
||||
monkeypatch.setattr("basic_memory.cli.auto_update._run_subprocess", _fake_run_subprocess)
|
||||
|
||||
result = run_auto_update(
|
||||
config_manager=manager,
|
||||
executable="/opt/homebrew/Cellar/basic-memory/0.18.0/bin/python",
|
||||
)
|
||||
|
||||
assert result.status == AutoUpdateStatus.UPDATED
|
||||
assert calls == [["brew", "upgrade", "basic-memory"]]
|
||||
|
||||
|
||||
def test_uv_tool_pypi_check_triggers_upgrade(monkeypatch, tmp_path):
|
||||
config = _base_config(tmp_path)
|
||||
manager = StubConfigManager(config)
|
||||
|
||||
monkeypatch.setattr(
|
||||
"basic_memory.cli.auto_update._check_pypi_update_available",
|
||||
lambda: (True, "9.9.9"),
|
||||
)
|
||||
calls: list[list[str]] = []
|
||||
|
||||
def _fake_run_subprocess(command, **kwargs):
|
||||
calls.append(command)
|
||||
return subprocess.CompletedProcess(command, 0, stdout="", stderr="")
|
||||
|
||||
monkeypatch.setattr("basic_memory.cli.auto_update._run_subprocess", _fake_run_subprocess)
|
||||
|
||||
result = run_auto_update(
|
||||
config_manager=manager,
|
||||
executable="/Users/me/.local/share/uv/tools/basic-memory/bin/python",
|
||||
)
|
||||
|
||||
assert result.status == AutoUpdateStatus.UPDATED
|
||||
assert result.latest_version == "9.9.9"
|
||||
assert calls == [["uv", "tool", "upgrade", "basic-memory"]]
|
||||
|
||||
|
||||
def test_unknown_manager_returns_manual_update_guidance(monkeypatch, tmp_path):
|
||||
config = _base_config(tmp_path)
|
||||
manager = StubConfigManager(config)
|
||||
monkeypatch.setattr(
|
||||
"basic_memory.cli.auto_update._check_pypi_update_available",
|
||||
lambda: (True, "9.9.9"),
|
||||
)
|
||||
|
||||
result = run_auto_update(
|
||||
force=True,
|
||||
config_manager=manager,
|
||||
executable="/usr/local/bin/python3",
|
||||
)
|
||||
|
||||
assert result.status == AutoUpdateStatus.UPDATE_AVAILABLE
|
||||
assert result.updated is False
|
||||
assert "Automatic install is not supported" in (result.message or "")
|
||||
|
||||
|
||||
def test_uvx_runtime_is_skipped(monkeypatch, tmp_path):
|
||||
config = _base_config(tmp_path)
|
||||
manager = StubConfigManager(config)
|
||||
|
||||
result = run_auto_update(
|
||||
config_manager=manager,
|
||||
executable="/Users/me/.cache/uv/archive-v0/abc123/bin/python",
|
||||
)
|
||||
|
||||
assert result.status == AutoUpdateStatus.SKIPPED
|
||||
assert result.source == InstallSource.UVX
|
||||
assert result.checked is False
|
||||
assert manager.save_calls == 0
|
||||
|
||||
|
||||
def test_mcp_silent_mode_suppresses_subprocess_output(monkeypatch, tmp_path):
|
||||
config = _base_config(tmp_path)
|
||||
manager = StubConfigManager(config)
|
||||
monkeypatch.setattr(
|
||||
"basic_memory.cli.auto_update._check_pypi_update_available",
|
||||
lambda: (True, "9.9.9"),
|
||||
)
|
||||
|
||||
captured_kwargs: list[dict] = []
|
||||
|
||||
def _fake_run_subprocess(command, **kwargs):
|
||||
captured_kwargs.append(kwargs)
|
||||
return subprocess.CompletedProcess(command, 0, stdout="", stderr="")
|
||||
|
||||
monkeypatch.setattr("basic_memory.cli.auto_update._run_subprocess", _fake_run_subprocess)
|
||||
|
||||
result = run_auto_update(
|
||||
config_manager=manager,
|
||||
executable="/Users/me/.local/share/uv/tools/basic-memory/bin/python",
|
||||
silent=True,
|
||||
)
|
||||
|
||||
assert result.status == AutoUpdateStatus.UPDATED
|
||||
assert captured_kwargs
|
||||
assert captured_kwargs[0]["silent"] is True
|
||||
assert captured_kwargs[0]["capture_output"] is False
|
||||
|
||||
|
||||
def test_subprocess_oserror_is_non_fatal(monkeypatch, tmp_path):
|
||||
config = _base_config(tmp_path)
|
||||
manager = StubConfigManager(config)
|
||||
monkeypatch.setattr(
|
||||
"basic_memory.cli.auto_update._check_pypi_update_available",
|
||||
lambda: (True, "9.9.9"),
|
||||
)
|
||||
|
||||
def _raise_oserror(command, **kwargs):
|
||||
raise FileNotFoundError(command[0])
|
||||
|
||||
monkeypatch.setattr("basic_memory.cli.auto_update._run_subprocess", _raise_oserror)
|
||||
|
||||
result = run_auto_update(
|
||||
config_manager=manager,
|
||||
executable="/Users/me/.local/share/uv/tools/basic-memory/bin/python",
|
||||
)
|
||||
|
||||
assert result.status == AutoUpdateStatus.FAILED
|
||||
assert result.checked is True
|
||||
|
||||
|
||||
def test_mixed_timezone_timestamp_does_not_crash_interval_gate(monkeypatch, tmp_path):
|
||||
config = _base_config(tmp_path)
|
||||
config.auto_update_last_checked_at = datetime.now(timezone.utc)
|
||||
manager = StubConfigManager(config)
|
||||
|
||||
monkeypatch.setattr(
|
||||
"basic_memory.cli.auto_update._check_pypi_update_available",
|
||||
lambda: (False, "0.0.0"),
|
||||
)
|
||||
|
||||
result = run_auto_update(
|
||||
config_manager=manager,
|
||||
executable="/Users/me/.local/share/uv/tools/basic-memory/bin/python",
|
||||
)
|
||||
|
||||
assert result.status == AutoUpdateStatus.UP_TO_DATE
|
||||
assert result.checked is True
|
||||
|
||||
|
||||
def test_maybe_run_periodic_auto_update_non_interactive_has_no_console_output():
|
||||
console, buf = _capture_console()
|
||||
result = maybe_run_periodic_auto_update(
|
||||
"status",
|
||||
is_interactive=False,
|
||||
console=console,
|
||||
)
|
||||
assert result is None
|
||||
assert buf.getvalue() == ""
|
||||
|
||||
|
||||
def test_maybe_run_periodic_auto_update_prints_updated(monkeypatch):
|
||||
console, buf = _capture_console()
|
||||
monkeypatch.setattr(
|
||||
"basic_memory.cli.auto_update.run_auto_update",
|
||||
lambda **kwargs: _result(
|
||||
AutoUpdateStatus.UPDATED,
|
||||
message="Basic Memory was updated successfully.",
|
||||
),
|
||||
)
|
||||
|
||||
result = maybe_run_periodic_auto_update("status", is_interactive=True, console=console)
|
||||
assert result is not None
|
||||
assert result.status == AutoUpdateStatus.UPDATED
|
||||
assert "updated successfully" in buf.getvalue().lower()
|
||||
|
||||
|
||||
def test_maybe_run_periodic_auto_update_prints_available(monkeypatch):
|
||||
console, buf = _capture_console()
|
||||
monkeypatch.setattr(
|
||||
"basic_memory.cli.auto_update.run_auto_update",
|
||||
lambda **kwargs: _result(
|
||||
AutoUpdateStatus.UPDATE_AVAILABLE,
|
||||
message="Update available (latest: 9.9.9).",
|
||||
),
|
||||
)
|
||||
|
||||
result = maybe_run_periodic_auto_update("status", is_interactive=True, console=console)
|
||||
assert result is not None
|
||||
assert result.status == AutoUpdateStatus.UPDATE_AVAILABLE
|
||||
assert "update available" in buf.getvalue().lower()
|
||||
|
||||
|
||||
def test_maybe_run_periodic_auto_update_prints_failed_with_error(monkeypatch):
|
||||
console, buf = _capture_console()
|
||||
monkeypatch.setattr(
|
||||
"basic_memory.cli.auto_update.run_auto_update",
|
||||
lambda **kwargs: _result(
|
||||
AutoUpdateStatus.FAILED,
|
||||
message="Automatic update check failed.",
|
||||
error="network timeout",
|
||||
),
|
||||
)
|
||||
|
||||
result = maybe_run_periodic_auto_update("status", is_interactive=True, console=console)
|
||||
assert result is not None
|
||||
assert result.status == AutoUpdateStatus.FAILED
|
||||
output = buf.getvalue().lower()
|
||||
assert "automatic update check failed" in output
|
||||
assert "network timeout" in output
|
||||
|
||||
|
||||
def test_maybe_run_periodic_auto_update_uses_interactive_probe_when_not_overridden(monkeypatch):
|
||||
console, buf = _capture_console()
|
||||
monkeypatch.setattr("basic_memory.cli.auto_update._is_interactive_session", lambda: True)
|
||||
monkeypatch.setattr(
|
||||
"basic_memory.cli.auto_update.run_auto_update",
|
||||
lambda **kwargs: _result(
|
||||
AutoUpdateStatus.UP_TO_DATE,
|
||||
message="Basic Memory is up to date.",
|
||||
),
|
||||
)
|
||||
|
||||
result = maybe_run_periodic_auto_update("status", console=console)
|
||||
assert result is not None
|
||||
assert result.status == AutoUpdateStatus.UP_TO_DATE
|
||||
# UP_TO_DATE is intentionally silent for periodic checks.
|
||||
assert buf.getvalue() == ""
|
||||
|
||||
|
||||
def test_is_interactive_session_handles_closed_stdio(monkeypatch):
|
||||
class _BrokenStream:
|
||||
def isatty(self) -> bool:
|
||||
raise ValueError("I/O operation on closed file")
|
||||
|
||||
monkeypatch.setattr("basic_memory.cli.auto_update.sys.stdin", _BrokenStream())
|
||||
monkeypatch.setattr("basic_memory.cli.auto_update.sys.stdout", _BrokenStream())
|
||||
|
||||
assert _is_interactive_session() is False
|
||||
@@ -0,0 +1,49 @@
|
||||
"""Telemetry coverage for CLI command boundaries."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from basic_memory.cli import app as cli_app
|
||||
|
||||
|
||||
class FakeContext:
|
||||
"""Small Typer-like context for callback testing."""
|
||||
|
||||
def __init__(self, invoked_subcommand: str | None) -> None:
|
||||
self.invoked_subcommand = invoked_subcommand
|
||||
self.resources: list[object] = []
|
||||
self.close_callbacks: list[object] = []
|
||||
|
||||
def with_resource(self, resource: object) -> None:
|
||||
self.resources.append(resource)
|
||||
|
||||
def call_on_close(self, callback) -> None:
|
||||
self.close_callbacks.append(callback)
|
||||
|
||||
|
||||
def test_app_callback_registers_command_operation(monkeypatch) -> None:
|
||||
operations: list[tuple[str, dict]] = []
|
||||
resource = object()
|
||||
|
||||
monkeypatch.setattr(cli_app, "init_cli_logging", lambda: None)
|
||||
monkeypatch.setattr(cli_app.CliContainer, "create", staticmethod(lambda: object()))
|
||||
monkeypatch.setattr(cli_app, "set_container", lambda container: None)
|
||||
monkeypatch.setattr(cli_app, "maybe_show_init_line", lambda command_name: None)
|
||||
monkeypatch.setattr(cli_app, "maybe_show_cloud_promo", lambda command_name: None)
|
||||
monkeypatch.setattr(cli_app, "maybe_run_periodic_auto_update", lambda command_name: None)
|
||||
|
||||
def fake_operation(name: str, **attrs):
|
||||
operations.append((name, attrs))
|
||||
return resource
|
||||
|
||||
monkeypatch.setattr(cli_app.telemetry, "operation", fake_operation)
|
||||
|
||||
ctx = FakeContext(invoked_subcommand="status")
|
||||
cli_app.app_callback(ctx, version=None)
|
||||
|
||||
assert ctx.resources == [resource]
|
||||
assert operations == [
|
||||
(
|
||||
"cli.command.status",
|
||||
{"entrypoint": "cli", "command_name": "status"},
|
||||
)
|
||||
]
|
||||
@@ -1,184 +0,0 @@
|
||||
"""Tests for graph/FCM CLI tool JSON passthrough commands."""
|
||||
|
||||
import json
|
||||
from unittest.mock import AsyncMock, patch
|
||||
|
||||
from typer.testing import CliRunner
|
||||
|
||||
from basic_memory.cli.main import app as cli_app
|
||||
|
||||
runner = CliRunner()
|
||||
|
||||
|
||||
@patch(
|
||||
"basic_memory.cli.commands.tool.mcp_graph_lineage",
|
||||
new_callable=AsyncMock,
|
||||
return_value={
|
||||
"root": {"id": "specs/search"},
|
||||
"paths": [],
|
||||
"generated_at": "2026-03-05T00:00:00Z",
|
||||
},
|
||||
)
|
||||
def test_graph_lineage_json_output(mock_tool):
|
||||
result = runner.invoke(cli_app, ["tool", "graph-lineage", "memory://specs/search"])
|
||||
assert result.exit_code == 0, f"CLI failed: {result.output}"
|
||||
data = json.loads(result.output)
|
||||
assert data["root"]["id"] == "specs/search"
|
||||
assert mock_tool.call_args.kwargs["output_format"] == "json"
|
||||
|
||||
|
||||
@patch(
|
||||
"basic_memory.cli.commands.tool.mcp_graph_impact",
|
||||
new_callable=AsyncMock,
|
||||
return_value={
|
||||
"target": {"id": "specs/search", "title": "specs/search"},
|
||||
"affected": [],
|
||||
"summary": {"total_considered": 0, "total_returned": 0},
|
||||
},
|
||||
)
|
||||
def test_graph_impact_passthrough(mock_tool):
|
||||
result = runner.invoke(
|
||||
cli_app,
|
||||
[
|
||||
"tool",
|
||||
"graph-impact",
|
||||
"memory://specs/search",
|
||||
"--horizon",
|
||||
"3",
|
||||
"--relation-filter",
|
||||
"depends_on",
|
||||
],
|
||||
)
|
||||
assert result.exit_code == 0, f"CLI failed: {result.output}"
|
||||
assert mock_tool.call_args.kwargs["horizon"] == 3
|
||||
assert mock_tool.call_args.kwargs["relation_filters"] == ["depends_on"]
|
||||
assert mock_tool.call_args.kwargs["output_format"] == "json"
|
||||
|
||||
|
||||
@patch(
|
||||
"basic_memory.cli.commands.tool.mcp_graph_health",
|
||||
new_callable=AsyncMock,
|
||||
return_value={
|
||||
"metrics": {
|
||||
"orphan_rate": 0.0,
|
||||
"stale_central_nodes": 0,
|
||||
"overloaded_hubs": 0,
|
||||
"contradiction_candidates": 0,
|
||||
},
|
||||
"issues": [],
|
||||
"computed_at": "2026-03-05T00:00:00Z",
|
||||
},
|
||||
)
|
||||
def test_graph_health_json_output(mock_tool):
|
||||
result = runner.invoke(
|
||||
cli_app,
|
||||
["tool", "graph-health", "--scope", "specs", "--timeframe", "30d"],
|
||||
)
|
||||
assert result.exit_code == 0, f"CLI failed: {result.output}"
|
||||
data = json.loads(result.output)
|
||||
assert "metrics" in data
|
||||
assert mock_tool.call_args.kwargs["scope"] == "specs"
|
||||
assert mock_tool.call_args.kwargs["timeframe"] == "30d"
|
||||
|
||||
|
||||
@patch(
|
||||
"basic_memory.cli.commands.tool.mcp_fcm_simulate",
|
||||
new_callable=AsyncMock,
|
||||
return_value={
|
||||
"baseline": [],
|
||||
"projected": [],
|
||||
"deltas": [],
|
||||
"stability": {"converged": True, "iterations_used": 1, "residual": 0.0},
|
||||
"confidence": 0.5,
|
||||
},
|
||||
)
|
||||
def test_fcm_simulate_json_output(mock_tool):
|
||||
result = runner.invoke(
|
||||
cli_app,
|
||||
[
|
||||
"tool",
|
||||
"fcm-simulate",
|
||||
"--actions-json",
|
||||
'[{"node_id":"n1","delta":0.2}]',
|
||||
"--scenario-json",
|
||||
'{"steps":8}',
|
||||
],
|
||||
)
|
||||
assert result.exit_code == 0, f"CLI failed: {result.output}"
|
||||
assert mock_tool.call_args.kwargs["actions"] == [{"node_id": "n1", "delta": 0.2}]
|
||||
assert mock_tool.call_args.kwargs["scenario"] == {"steps": 8}
|
||||
|
||||
|
||||
def test_fcm_simulate_invalid_actions_json():
|
||||
result = runner.invoke(
|
||||
cli_app,
|
||||
["tool", "fcm-simulate", "--actions-json", '{"node_id":"n1","delta":0.2}'],
|
||||
)
|
||||
assert result.exit_code == 1
|
||||
assert "expected a JSON array" in result.output
|
||||
|
||||
|
||||
@patch(
|
||||
"basic_memory.cli.commands.tool.mcp_fcm_rank_actions",
|
||||
new_callable=AsyncMock,
|
||||
return_value={"goal": {"node_id": "g1", "label": "g1"}, "recommendations": []},
|
||||
)
|
||||
def test_fcm_rank_actions_passthrough(mock_tool):
|
||||
result = runner.invoke(
|
||||
cli_app,
|
||||
["tool", "fcm-rank-actions", "g1", "--constraints-json", '{"required_tags":["risk"]}'],
|
||||
)
|
||||
assert result.exit_code == 0, f"CLI failed: {result.output}"
|
||||
assert mock_tool.call_args.kwargs["constraints"] == {"required_tags": ["risk"]}
|
||||
assert mock_tool.call_args.kwargs["output_format"] == "json"
|
||||
|
||||
|
||||
@patch(
|
||||
"basic_memory.cli.commands.tool.mcp_fcm_import_model",
|
||||
new_callable=AsyncMock,
|
||||
return_value={
|
||||
"import_id": "imp-1",
|
||||
"nodes_loaded": 0,
|
||||
"edges_loaded": 0,
|
||||
"warnings": [],
|
||||
"errors": [],
|
||||
},
|
||||
)
|
||||
def test_fcm_import_model_json_output(mock_tool):
|
||||
result = runner.invoke(
|
||||
cli_app,
|
||||
["tool", "fcm-import-model", "/tmp/model.csv", "--format", "csv_bundle_v1"],
|
||||
)
|
||||
assert result.exit_code == 0, f"CLI failed: {result.output}"
|
||||
data = json.loads(result.output)
|
||||
assert data["import_id"] == "imp-1"
|
||||
assert mock_tool.call_args.kwargs["output_format"] == "json"
|
||||
|
||||
|
||||
@patch(
|
||||
"basic_memory.cli.commands.tool.mcp_fcm_export_model",
|
||||
new_callable=AsyncMock,
|
||||
return_value={
|
||||
"export_id": "exp-1",
|
||||
"format": "csv_bundle_v1",
|
||||
"files": [],
|
||||
"node_count": 0,
|
||||
"edge_count": 0,
|
||||
},
|
||||
)
|
||||
def test_fcm_export_model_json_output(mock_tool):
|
||||
result = runner.invoke(
|
||||
cli_app,
|
||||
[
|
||||
"tool",
|
||||
"fcm-export-model",
|
||||
"--format",
|
||||
"csv_bundle_v1",
|
||||
"--selection-json",
|
||||
'{"scope":"all"}',
|
||||
],
|
||||
)
|
||||
assert result.exit_code == 0, f"CLI failed: {result.output}"
|
||||
data = json.loads(result.output)
|
||||
assert data["export_id"] == "exp-1"
|
||||
assert mock_tool.call_args.kwargs["selection"] == {"scope": "all"}
|
||||
@@ -350,6 +350,8 @@ def write_config(tmp_path, monkeypatch):
|
||||
from basic_memory import config as config_module
|
||||
|
||||
config_module._CONFIG_CACHE = None
|
||||
config_module._CONFIG_MTIME = None
|
||||
config_module._CONFIG_SIZE = None
|
||||
|
||||
config_dir = tmp_path / ".basic-memory"
|
||||
config_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
@@ -27,6 +27,8 @@ def mock_config(tmp_path, monkeypatch):
|
||||
from basic_memory import config as config_module
|
||||
|
||||
config_module._CONFIG_CACHE = None
|
||||
config_module._CONFIG_MTIME = None
|
||||
config_module._CONFIG_SIZE = None
|
||||
|
||||
config_dir = tmp_path / ".basic-memory"
|
||||
config_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
@@ -29,6 +29,8 @@ def write_config(tmp_path, monkeypatch):
|
||||
from basic_memory import config as config_module
|
||||
|
||||
config_module._CONFIG_CACHE = None
|
||||
config_module._CONFIG_MTIME = None
|
||||
config_module._CONFIG_SIZE = None
|
||||
|
||||
config_dir = tmp_path / ".basic-memory"
|
||||
config_dir.mkdir(parents=True, exist_ok=True)
|
||||
@@ -122,15 +124,16 @@ def test_project_list_shows_local_cloud_presence_and_routes(
|
||||
assert "Local Path" in result.stdout
|
||||
assert "Cloud Path" in result.stdout
|
||||
assert "CLI Route" in result.stdout
|
||||
assert "MCP (stdio)" in result.stdout
|
||||
assert "MCP" in result.stdout
|
||||
|
||||
lines = result.stdout.splitlines()
|
||||
alpha_line = next(line for line in lines if "│ alpha" in line)
|
||||
beta_line = next(line for line in lines if "│ beta" in line)
|
||||
|
||||
assert "local" in alpha_line # CLI route for alpha
|
||||
assert "stdio" in alpha_line # Local projects use stdio transport
|
||||
assert "cloud" in beta_line # CLI route for beta
|
||||
assert "n/a" in beta_line # MCP stdio route is unavailable for cloud-only projects
|
||||
assert "https" in beta_line # Cloud projects use HTTPS transport
|
||||
assert "alpha-local" in result.stdout
|
||||
assert "/alpha" in result.stdout
|
||||
assert "/beta" in result.stdout
|
||||
|
||||
@@ -22,6 +22,8 @@ def mock_config(tmp_path, monkeypatch):
|
||||
from basic_memory import config as config_module
|
||||
|
||||
config_module._CONFIG_CACHE = None
|
||||
config_module._CONFIG_MTIME = None
|
||||
config_module._CONFIG_SIZE = None
|
||||
|
||||
config_dir = tmp_path / ".basic-memory"
|
||||
config_dir.mkdir(parents=True, exist_ok=True)
|
||||
@@ -68,6 +70,8 @@ class TestSetCloud:
|
||||
from basic_memory import config as config_module
|
||||
|
||||
config_module._CONFIG_CACHE = None
|
||||
config_module._CONFIG_MTIME = None
|
||||
config_module._CONFIG_SIZE = None
|
||||
|
||||
config_dir = tmp_path / ".basic-memory"
|
||||
config_dir.mkdir(parents=True, exist_ok=True)
|
||||
@@ -91,6 +95,8 @@ class TestSetCloud:
|
||||
from basic_memory import config as config_module
|
||||
|
||||
config_module._CONFIG_CACHE = None
|
||||
config_module._CONFIG_MTIME = None
|
||||
config_module._CONFIG_SIZE = None
|
||||
|
||||
config_dir = tmp_path / ".basic-memory"
|
||||
config_dir.mkdir(parents=True, exist_ok=True)
|
||||
@@ -161,11 +167,15 @@ class TestSetLocal:
|
||||
|
||||
# Manually set workspace_id on the project
|
||||
config_module._CONFIG_CACHE = None
|
||||
config_module._CONFIG_MTIME = None
|
||||
config_module._CONFIG_SIZE = None
|
||||
config_data = json.loads(mock_config.read_text())
|
||||
config_data["projects"]["research"]["mode"] = "cloud"
|
||||
config_data["projects"]["research"]["workspace_id"] = "11111111-1111-1111-1111-111111111111"
|
||||
mock_config.write_text(json.dumps(config_data, indent=2))
|
||||
config_module._CONFIG_CACHE = None
|
||||
config_module._CONFIG_MTIME = None
|
||||
config_module._CONFIG_SIZE = None
|
||||
|
||||
# Set back to local
|
||||
result = runner.invoke(app, ["project", "set-local", "research"])
|
||||
@@ -173,6 +183,8 @@ class TestSetLocal:
|
||||
|
||||
# Verify workspace_id was cleared
|
||||
config_module._CONFIG_CACHE = None
|
||||
config_module._CONFIG_MTIME = None
|
||||
config_module._CONFIG_SIZE = None
|
||||
updated_data = json.loads(mock_config.read_text())
|
||||
assert updated_data["projects"]["research"]["workspace_id"] is None
|
||||
assert updated_data["projects"]["research"]["mode"] == "local"
|
||||
@@ -187,6 +199,8 @@ class TestSetCloudWithWorkspace:
|
||||
from basic_memory.schemas.cloud import WorkspaceInfo
|
||||
|
||||
config_module._CONFIG_CACHE = None
|
||||
config_module._CONFIG_MTIME = None
|
||||
config_module._CONFIG_SIZE = None
|
||||
|
||||
async def fake_get_available_workspaces():
|
||||
return [
|
||||
@@ -210,6 +224,8 @@ class TestSetCloudWithWorkspace:
|
||||
|
||||
# Verify workspace_id was persisted
|
||||
config_module._CONFIG_CACHE = None
|
||||
config_module._CONFIG_MTIME = None
|
||||
config_module._CONFIG_SIZE = None
|
||||
updated_data = json.loads(mock_config.read_text())
|
||||
assert (
|
||||
updated_data["projects"]["research"]["workspace_id"]
|
||||
@@ -222,6 +238,8 @@ class TestSetCloudWithWorkspace:
|
||||
from basic_memory.schemas.cloud import WorkspaceInfo
|
||||
|
||||
config_module._CONFIG_CACHE = None
|
||||
config_module._CONFIG_MTIME = None
|
||||
config_module._CONFIG_SIZE = None
|
||||
|
||||
async def fake_get_available_workspaces():
|
||||
return [
|
||||
@@ -249,17 +267,23 @@ class TestSetCloudWithWorkspace:
|
||||
from basic_memory import config as config_module
|
||||
|
||||
config_module._CONFIG_CACHE = None
|
||||
config_module._CONFIG_MTIME = None
|
||||
config_module._CONFIG_SIZE = None
|
||||
|
||||
# Set default_workspace in config
|
||||
config_data = json.loads(mock_config.read_text())
|
||||
config_data["default_workspace"] = "global-default-tenant-id"
|
||||
mock_config.write_text(json.dumps(config_data, indent=2))
|
||||
config_module._CONFIG_CACHE = None
|
||||
config_module._CONFIG_MTIME = None
|
||||
config_module._CONFIG_SIZE = None
|
||||
|
||||
result = runner.invoke(app, ["project", "set-cloud", "research"])
|
||||
assert result.exit_code == 0
|
||||
|
||||
# Verify workspace_id was set from default
|
||||
config_module._CONFIG_CACHE = None
|
||||
config_module._CONFIG_MTIME = None
|
||||
config_module._CONFIG_SIZE = None
|
||||
updated_data = json.loads(mock_config.read_text())
|
||||
assert updated_data["projects"]["research"]["workspace_id"] == "global-default-tenant-id"
|
||||
|
||||
@@ -0,0 +1,90 @@
|
||||
"""Tests for `bm update` command."""
|
||||
|
||||
from typer.testing import CliRunner
|
||||
|
||||
from basic_memory.cli.app import app
|
||||
from basic_memory.cli.auto_update import AutoUpdateResult, AutoUpdateStatus, InstallSource
|
||||
|
||||
|
||||
def _result(
|
||||
status: AutoUpdateStatus,
|
||||
*,
|
||||
message: str | None,
|
||||
error: str | None = None,
|
||||
) -> AutoUpdateResult:
|
||||
return AutoUpdateResult(
|
||||
status=status,
|
||||
source=InstallSource.UV_TOOL,
|
||||
checked=True,
|
||||
update_available=status in {AutoUpdateStatus.UPDATE_AVAILABLE, AutoUpdateStatus.UPDATED},
|
||||
updated=status == AutoUpdateStatus.UPDATED,
|
||||
latest_version="9.9.9",
|
||||
message=message,
|
||||
error=error,
|
||||
restart_recommended=status == AutoUpdateStatus.UPDATED,
|
||||
)
|
||||
|
||||
|
||||
def test_update_command_applies_upgrade(monkeypatch):
|
||||
runner = CliRunner()
|
||||
|
||||
monkeypatch.setattr(
|
||||
"basic_memory.cli.commands.update.run_auto_update",
|
||||
lambda **kwargs: _result(
|
||||
AutoUpdateStatus.UPDATED,
|
||||
message="Basic Memory was updated successfully.",
|
||||
),
|
||||
)
|
||||
|
||||
result = runner.invoke(app, ["update"])
|
||||
assert result.exit_code == 0
|
||||
assert "updated successfully" in result.stdout.lower()
|
||||
|
||||
|
||||
def test_update_command_check_only_shows_available(monkeypatch):
|
||||
runner = CliRunner()
|
||||
|
||||
monkeypatch.setattr(
|
||||
"basic_memory.cli.commands.update.run_auto_update",
|
||||
lambda **kwargs: _result(
|
||||
AutoUpdateStatus.UPDATE_AVAILABLE,
|
||||
message="Update available (latest: 9.9.9). Run `uv tool upgrade basic-memory`.",
|
||||
),
|
||||
)
|
||||
|
||||
result = runner.invoke(app, ["update", "--check"])
|
||||
assert result.exit_code == 0
|
||||
assert "update available" in result.stdout.lower()
|
||||
|
||||
|
||||
def test_update_command_reports_up_to_date(monkeypatch):
|
||||
runner = CliRunner()
|
||||
|
||||
monkeypatch.setattr(
|
||||
"basic_memory.cli.commands.update.run_auto_update",
|
||||
lambda **kwargs: _result(
|
||||
AutoUpdateStatus.UP_TO_DATE,
|
||||
message="Basic Memory is up to date.",
|
||||
),
|
||||
)
|
||||
|
||||
result = runner.invoke(app, ["update"])
|
||||
assert result.exit_code == 0
|
||||
assert "up to date" in result.stdout.lower()
|
||||
|
||||
|
||||
def test_update_command_failure_exits_nonzero(monkeypatch):
|
||||
runner = CliRunner()
|
||||
|
||||
monkeypatch.setattr(
|
||||
"basic_memory.cli.commands.update.run_auto_update",
|
||||
lambda **kwargs: _result(
|
||||
AutoUpdateStatus.FAILED,
|
||||
message="Automatic update failed.",
|
||||
error="network timeout",
|
||||
),
|
||||
)
|
||||
|
||||
result = runner.invoke(app, ["update"])
|
||||
assert result.exit_code == 1
|
||||
assert "automatic update failed" in result.stdout.lower()
|
||||
@@ -76,6 +76,8 @@ class TestWorkspaceSetDefault:
|
||||
monkeypatch.setenv("HOME", str(temp_path))
|
||||
monkeypatch.setenv("BASIC_MEMORY_CONFIG_DIR", str(config_dir))
|
||||
basic_memory.config._CONFIG_CACHE = None
|
||||
basic_memory.config._CONFIG_MTIME = None
|
||||
basic_memory.config._CONFIG_SIZE = None
|
||||
|
||||
config_manager = ConfigManager()
|
||||
test_config = BasicMemoryConfig(
|
||||
@@ -106,6 +108,8 @@ class TestWorkspaceSetDefault:
|
||||
|
||||
# Verify config was updated
|
||||
basic_memory.config._CONFIG_CACHE = None
|
||||
basic_memory.config._CONFIG_MTIME = None
|
||||
basic_memory.config._CONFIG_SIZE = None
|
||||
config = ConfigManager().config
|
||||
assert config.default_workspace == "11111111-1111-1111-1111-111111111111"
|
||||
|
||||
|
||||
@@ -88,6 +88,12 @@ def anyio_backend():
|
||||
return "asyncio"
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def suppress_logfire_no_config_warning(monkeypatch) -> None:
|
||||
"""Keep tests focused on behavior instead of Logfire bootstrap warnings."""
|
||||
monkeypatch.setenv("LOGFIRE_IGNORE_NO_CONFIG", "1")
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def project_root() -> Path:
|
||||
return Path(__file__).parent.parent
|
||||
@@ -138,6 +144,8 @@ def config_manager(app_config: BasicMemoryConfig, config_home: Path, monkeypatch
|
||||
from basic_memory import config as config_module
|
||||
|
||||
config_module._CONFIG_CACHE = None
|
||||
config_module._CONFIG_MTIME = None
|
||||
config_module._CONFIG_SIZE = None
|
||||
|
||||
# Create a new ConfigManager that uses the test home directory
|
||||
config_manager = ConfigManager()
|
||||
|
||||
@@ -1,216 +0,0 @@
|
||||
"""Tests for graph and FCM typed clients."""
|
||||
|
||||
from unittest.mock import MagicMock
|
||||
|
||||
import pytest
|
||||
|
||||
from basic_memory.mcp.clients import FCMClient, GraphClient
|
||||
|
||||
|
||||
class TestGraphClient:
|
||||
def test_init(self):
|
||||
mock_http = MagicMock()
|
||||
client = GraphClient(mock_http, "project-123")
|
||||
assert client.http_client is mock_http
|
||||
assert client.project_id == "project-123"
|
||||
assert client._base_path == "/v2/projects/project-123/graph"
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_lineage(self, monkeypatch):
|
||||
from basic_memory.mcp.clients import graph as graph_mod
|
||||
from basic_memory.schemas.graph_intelligence import GraphLineageRequest
|
||||
|
||||
mock_response = MagicMock()
|
||||
mock_response.json.return_value = {
|
||||
"root": {"id": "specs/search", "title": "specs/search", "permalink": "specs/search"},
|
||||
"paths": [],
|
||||
"generated_at": "2026-03-05T00:00:00+00:00",
|
||||
}
|
||||
|
||||
async def mock_call_post(client, url, **kwargs):
|
||||
assert "/v2/projects/proj-123/graph/lineage" in url
|
||||
return mock_response
|
||||
|
||||
monkeypatch.setattr(graph_mod, "call_post", mock_call_post)
|
||||
|
||||
client = GraphClient(MagicMock(), "proj-123")
|
||||
result = await client.lineage(GraphLineageRequest(start="memory://specs/search"))
|
||||
assert result.root.id == "specs/search"
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_impact(self, monkeypatch):
|
||||
from basic_memory.mcp.clients import graph as graph_mod
|
||||
from basic_memory.schemas.graph_intelligence import GraphImpactRequest
|
||||
|
||||
mock_response = MagicMock()
|
||||
mock_response.json.return_value = {
|
||||
"target": {"id": "specs/search", "title": "specs/search"},
|
||||
"affected": [],
|
||||
"summary": {"total_considered": 0, "total_returned": 0},
|
||||
}
|
||||
|
||||
async def mock_call_post(client, url, **kwargs):
|
||||
assert "/v2/projects/proj-123/graph/impact" in url
|
||||
return mock_response
|
||||
|
||||
monkeypatch.setattr(graph_mod, "call_post", mock_call_post)
|
||||
|
||||
client = GraphClient(MagicMock(), "proj-123")
|
||||
result = await client.impact(GraphImpactRequest(target="memory://specs/search", horizon=2))
|
||||
assert result.summary.total_returned == 0
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_health(self, monkeypatch):
|
||||
from basic_memory.mcp.clients import graph as graph_mod
|
||||
|
||||
mock_response = MagicMock()
|
||||
mock_response.json.return_value = {
|
||||
"metrics": {
|
||||
"orphan_rate": 0.0,
|
||||
"stale_central_nodes": 0,
|
||||
"overloaded_hubs": 0,
|
||||
"contradiction_candidates": 0,
|
||||
},
|
||||
"issues": [],
|
||||
"computed_at": "2026-03-05T00:00:00+00:00",
|
||||
}
|
||||
|
||||
async def mock_call_get(client, url, **kwargs):
|
||||
assert "/v2/projects/proj-123/graph/health" in url
|
||||
assert kwargs["params"]["scope"] == "specs"
|
||||
return mock_response
|
||||
|
||||
monkeypatch.setattr(graph_mod, "call_get", mock_call_get)
|
||||
|
||||
client = GraphClient(MagicMock(), "proj-123")
|
||||
result = await client.health(scope="specs", timeframe="30d")
|
||||
assert result.metrics.orphan_rate == 0.0
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_reindex(self, monkeypatch):
|
||||
from basic_memory.mcp.clients import graph as graph_mod
|
||||
from basic_memory.schemas.graph_intelligence import GraphReindexRequest
|
||||
|
||||
mock_response = MagicMock()
|
||||
mock_response.json.return_value = {
|
||||
"job_id": "job-123",
|
||||
"status": "queued",
|
||||
"scheduled_at": "2026-03-05T00:00:00+00:00",
|
||||
}
|
||||
|
||||
async def mock_call_post(client, url, **kwargs):
|
||||
assert "/v2/projects/proj-123/graph/reindex" in url
|
||||
return mock_response
|
||||
|
||||
monkeypatch.setattr(graph_mod, "call_post", mock_call_post)
|
||||
|
||||
client = GraphClient(MagicMock(), "proj-123")
|
||||
result = await client.reindex(GraphReindexRequest(mode="full"))
|
||||
assert result.status == "queued"
|
||||
|
||||
|
||||
class TestFCMClient:
|
||||
def test_init(self):
|
||||
mock_http = MagicMock()
|
||||
client = FCMClient(mock_http, "project-123")
|
||||
assert client.http_client is mock_http
|
||||
assert client.project_id == "project-123"
|
||||
assert client._base_path == "/v2/projects/project-123/fcm"
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_simulate(self, monkeypatch):
|
||||
from basic_memory.mcp.clients import fcm as fcm_mod
|
||||
from basic_memory.schemas.graph_intelligence import FCMSimulateRequest
|
||||
|
||||
mock_response = MagicMock()
|
||||
mock_response.json.return_value = {
|
||||
"baseline": [{"node_id": "n1", "state": 0.0}],
|
||||
"projected": [{"node_id": "n1", "state": 0.2}],
|
||||
"deltas": [{"node_id": "n1", "delta": 0.2}],
|
||||
"stability": {"converged": True, "iterations_used": 3, "residual": 0.0},
|
||||
"confidence": 0.5,
|
||||
"explanations": [],
|
||||
"evidence_refs": [],
|
||||
}
|
||||
|
||||
async def mock_call_post(client, url, **kwargs):
|
||||
assert "/v2/projects/proj-123/fcm/simulate" in url
|
||||
return mock_response
|
||||
|
||||
monkeypatch.setattr(fcm_mod, "call_post", mock_call_post)
|
||||
|
||||
request = FCMSimulateRequest(actions=[{"node_id": "n1", "delta": 0.2}])
|
||||
result = await FCMClient(MagicMock(), "proj-123").simulate(request)
|
||||
assert result.stability.converged is True
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_rank_actions(self, monkeypatch):
|
||||
from basic_memory.mcp.clients import fcm as fcm_mod
|
||||
from basic_memory.schemas.graph_intelligence import FCMRankActionsRequest
|
||||
|
||||
mock_response = MagicMock()
|
||||
mock_response.json.return_value = {
|
||||
"goal": {"node_id": "g1", "label": "g1"},
|
||||
"recommendations": [],
|
||||
}
|
||||
|
||||
async def mock_call_post(client, url, **kwargs):
|
||||
assert "/v2/projects/proj-123/fcm/rank-actions" in url
|
||||
return mock_response
|
||||
|
||||
monkeypatch.setattr(fcm_mod, "call_post", mock_call_post)
|
||||
|
||||
request = FCMRankActionsRequest(goal="g1")
|
||||
result = await FCMClient(MagicMock(), "proj-123").rank_actions(request)
|
||||
assert result.goal.node_id == "g1"
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_import_model(self, monkeypatch):
|
||||
from basic_memory.mcp.clients import fcm as fcm_mod
|
||||
from basic_memory.schemas.graph_intelligence import FCMImportRequest
|
||||
|
||||
mock_response = MagicMock()
|
||||
mock_response.json.return_value = {
|
||||
"import_id": "imp-1",
|
||||
"nodes_loaded": 0,
|
||||
"edges_loaded": 0,
|
||||
"warnings": [],
|
||||
"errors": [],
|
||||
}
|
||||
|
||||
async def mock_call_post(client, url, **kwargs):
|
||||
assert "/v2/projects/proj-123/fcm/import" in url
|
||||
return mock_response
|
||||
|
||||
monkeypatch.setattr(fcm_mod, "call_post", mock_call_post)
|
||||
|
||||
request = FCMImportRequest(source="/tmp/model.csv")
|
||||
result = await FCMClient(MagicMock(), "proj-123").import_model(request)
|
||||
assert result.import_id == "imp-1"
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_export_model(self, monkeypatch):
|
||||
from basic_memory.mcp.clients import fcm as fcm_mod
|
||||
from basic_memory.schemas.graph_intelligence import FCMExportRequest
|
||||
|
||||
mock_response = MagicMock()
|
||||
mock_response.json.return_value = {
|
||||
"export_id": "exp-1",
|
||||
"format": "csv_bundle_v1",
|
||||
"files": [
|
||||
{"name": "nodes.csv", "path": "/tmp/nodes.csv"},
|
||||
{"name": "edges.csv", "path": "/tmp/edges.csv"},
|
||||
],
|
||||
"node_count": 0,
|
||||
"edge_count": 0,
|
||||
}
|
||||
|
||||
async def mock_call_post(client, url, **kwargs):
|
||||
assert "/v2/projects/proj-123/fcm/export" in url
|
||||
return mock_response
|
||||
|
||||
monkeypatch.setattr(fcm_mod, "call_post", mock_call_post)
|
||||
|
||||
request = FCMExportRequest()
|
||||
result = await FCMClient(MagicMock(), "proj-123").export_model(request)
|
||||
assert result.format == "csv_bundle_v1"
|
||||
@@ -0,0 +1,46 @@
|
||||
"""Telemetry coverage for async client auth failures."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import importlib
|
||||
from contextlib import contextmanager
|
||||
from types import SimpleNamespace
|
||||
|
||||
import pytest
|
||||
|
||||
async_client_module = importlib.import_module("basic_memory.mcp.async_client")
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_resolve_cloud_token_emits_failure_span(monkeypatch) -> None:
|
||||
spans: list[tuple[str, dict]] = []
|
||||
error_messages: list[str] = []
|
||||
|
||||
class FakeAuth:
|
||||
def __init__(self, client_id: str, authkit_domain: str) -> None:
|
||||
self.client_id = client_id
|
||||
self.authkit_domain = authkit_domain
|
||||
|
||||
async def get_valid_token(self):
|
||||
return None
|
||||
|
||||
@contextmanager
|
||||
def fake_span(name: str, **attrs):
|
||||
spans.append((name, attrs))
|
||||
yield
|
||||
|
||||
monkeypatch.setattr(async_client_module.telemetry, "span", fake_span)
|
||||
monkeypatch.setattr("basic_memory.cli.auth.CLIAuth", FakeAuth)
|
||||
monkeypatch.setattr(async_client_module.logger, "error", error_messages.append)
|
||||
|
||||
config = SimpleNamespace(
|
||||
cloud_api_key=None,
|
||||
cloud_client_id="client-123",
|
||||
cloud_domain="auth.example.com",
|
||||
)
|
||||
|
||||
with pytest.raises(RuntimeError, match="no credentials found"):
|
||||
await async_client_module._resolve_cloud_token(config)
|
||||
|
||||
assert spans == [("routing.resolve_cloud_credentials", {"has_api_key": False})]
|
||||
assert error_messages == ["Cloud routing requested but no credentials were available"]
|
||||
@@ -94,6 +94,26 @@ async def test_uses_explicit_project_when_no_env(config_manager, monkeypatch):
|
||||
assert await resolve_project_parameter(project="explicit-project") == "explicit-project"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_canonicalizes_case_insensitive_project_reference(
|
||||
config_manager, config_home, monkeypatch
|
||||
):
|
||||
from basic_memory.config import ProjectEntry
|
||||
from basic_memory.mcp.project_context import resolve_project_parameter
|
||||
|
||||
cfg = config_manager.load_config()
|
||||
project_name = "Personal-Project"
|
||||
project_path = config_home / "personal-project"
|
||||
project_path.mkdir(parents=True, exist_ok=True)
|
||||
cfg.projects[project_name] = ProjectEntry(path=str(project_path))
|
||||
config_manager.save_config(cfg)
|
||||
|
||||
monkeypatch.delenv("BASIC_MEMORY_MCP_PROJECT", raising=False)
|
||||
|
||||
assert await resolve_project_parameter(project="personal-project") == project_name
|
||||
assert await resolve_project_parameter(project="PERSONAL-PROJECT") == project_name
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_uses_default_project(config_manager, config_home, monkeypatch):
|
||||
from basic_memory.mcp.project_context import resolve_project_parameter
|
||||
@@ -363,6 +383,7 @@ class TestDetectProjectFromUrlPrefix:
|
||||
assert result == "My Research"
|
||||
|
||||
|
||||
|
||||
class TestGetProjectClientRoutingOrder:
|
||||
"""Test that get_project_client respects explicit routing before workspace resolution."""
|
||||
|
||||
|
||||
@@ -0,0 +1,94 @@
|
||||
"""Telemetry coverage for project routing helpers."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import importlib
|
||||
from contextlib import contextmanager
|
||||
|
||||
import pytest
|
||||
|
||||
from basic_memory.config import ProjectEntry
|
||||
from basic_memory.schemas.cloud import WorkspaceInfo
|
||||
|
||||
project_context = importlib.import_module("basic_memory.mcp.project_context")
|
||||
|
||||
|
||||
class _ContextState:
|
||||
"""Minimal FastMCP context-state stub for unit tests."""
|
||||
|
||||
def __init__(self) -> None:
|
||||
self._state: dict[str, object] = {}
|
||||
|
||||
async def get_state(self, key: str):
|
||||
return self._state.get(key)
|
||||
|
||||
async def set_state(self, key: str, value: object, **kwargs) -> None:
|
||||
self._state[key] = value
|
||||
|
||||
|
||||
def _capture_telemetry():
|
||||
spans: list[tuple[str, dict]] = []
|
||||
contexts: list[dict] = []
|
||||
|
||||
@contextmanager
|
||||
def fake_span(name: str, **attrs):
|
||||
spans.append((name, attrs))
|
||||
yield
|
||||
|
||||
@contextmanager
|
||||
def fake_contextualize(**attrs):
|
||||
contexts.append(attrs)
|
||||
yield
|
||||
|
||||
return spans, contexts, fake_span, fake_contextualize
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_resolve_workspace_parameter_emits_routing_span(monkeypatch) -> None:
|
||||
spans, _, fake_span, _ = _capture_telemetry()
|
||||
context = _ContextState()
|
||||
workspace = WorkspaceInfo(
|
||||
tenant_id="11111111-1111-1111-1111-111111111111",
|
||||
workspace_type="personal",
|
||||
name="Personal",
|
||||
role="owner",
|
||||
)
|
||||
|
||||
async def fake_get_available_workspaces(context=None):
|
||||
return [workspace]
|
||||
|
||||
monkeypatch.setattr(project_context.telemetry, "span", fake_span)
|
||||
monkeypatch.setattr(project_context, "get_available_workspaces", fake_get_available_workspaces)
|
||||
|
||||
resolved = await project_context.resolve_workspace_parameter(context=context)
|
||||
|
||||
assert resolved.tenant_id == workspace.tenant_id
|
||||
assert spans == [
|
||||
(
|
||||
"routing.resolve_workspace",
|
||||
{"workspace_requested": False, "has_context": True},
|
||||
)
|
||||
]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_get_project_client_contextualizes_route_mode(config_manager, monkeypatch) -> None:
|
||||
spans, contexts, fake_span, fake_contextualize = _capture_telemetry()
|
||||
|
||||
config = config_manager.load_config()
|
||||
(config_manager.config_dir.parent / "main").mkdir(parents=True, exist_ok=True)
|
||||
config.projects["main"] = ProjectEntry(path=str(config_manager.config_dir.parent / "main"))
|
||||
config_manager.save_config(config)
|
||||
|
||||
monkeypatch.setattr(project_context.telemetry, "span", fake_span)
|
||||
monkeypatch.setattr(project_context.telemetry, "contextualize", fake_contextualize)
|
||||
|
||||
with pytest.raises(Exception):
|
||||
async with project_context.get_project_client(project="main"):
|
||||
pass
|
||||
|
||||
span_names = [name for name, _ in spans]
|
||||
assert "routing.resolve_project" in span_names
|
||||
assert "routing.client_session" in span_names
|
||||
assert "routing.validate_project" in span_names
|
||||
assert {"project_name": "main", "route_mode": "local_asgi"} in contexts
|
||||
@@ -0,0 +1,33 @@
|
||||
"""Telemetry coverage for MCP server lifecycle spans."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from contextlib import contextmanager
|
||||
|
||||
import pytest
|
||||
|
||||
from basic_memory.mcp.server import lifespan, mcp
|
||||
import basic_memory.mcp.server as server_module
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_mcp_lifespan_wraps_startup_and_shutdown(config_manager) -> None:
|
||||
operations: list[tuple[str, dict]] = []
|
||||
|
||||
@contextmanager
|
||||
def fake_operation(name: str, **attrs):
|
||||
operations.append((name, attrs))
|
||||
yield
|
||||
|
||||
original_operation = server_module.telemetry.operation
|
||||
server_module.telemetry.operation = fake_operation
|
||||
try:
|
||||
async with lifespan(mcp):
|
||||
pass
|
||||
finally:
|
||||
server_module.telemetry.operation = original_operation
|
||||
|
||||
assert [name for name, _ in operations] == [
|
||||
"mcp.lifecycle.startup",
|
||||
"mcp.lifecycle.shutdown",
|
||||
]
|
||||
@@ -35,31 +35,7 @@ EXPECTED_TOOL_SIGNATURES: dict[str, list[str]] = {
|
||||
"expected_replacements",
|
||||
"output_format",
|
||||
],
|
||||
"fcm_export_model": ["format", "selection", "project", "workspace", "output_format"],
|
||||
"fcm_import_model": ["source", "format", "merge_mode", "project", "workspace", "output_format"],
|
||||
"fcm_rank_actions": ["goal", "constraints", "top_k", "project", "workspace", "output_format"],
|
||||
"fcm_simulate": ["actions", "scenario", "clamp_rules", "project", "workspace", "output_format"],
|
||||
"fetch": ["id"],
|
||||
"graph_health": ["scope", "timeframe", "project", "workspace", "output_format"],
|
||||
"graph_impact": [
|
||||
"target",
|
||||
"horizon",
|
||||
"relation_filters",
|
||||
"include_reasons",
|
||||
"project",
|
||||
"workspace",
|
||||
"output_format",
|
||||
],
|
||||
"graph_lineage": [
|
||||
"start",
|
||||
"goal",
|
||||
"max_hops",
|
||||
"relation_filters",
|
||||
"project",
|
||||
"workspace",
|
||||
"output_format",
|
||||
],
|
||||
"graph_reindex": ["mode", "reason", "project", "workspace", "output_format"],
|
||||
"list_directory": ["dir_name", "depth", "file_name_glob", "project", "workspace"],
|
||||
"list_memory_projects": ["output_format", "workspace"],
|
||||
"list_workspaces": ["output_format"],
|
||||
@@ -137,15 +113,7 @@ TOOL_FUNCTIONS: dict[str, object] = {
|
||||
"delete_note": tools.delete_note,
|
||||
"delete_project": tools.delete_project,
|
||||
"edit_note": tools.edit_note,
|
||||
"fcm_export_model": tools.fcm_export_model,
|
||||
"fcm_import_model": tools.fcm_import_model,
|
||||
"fcm_rank_actions": tools.fcm_rank_actions,
|
||||
"fcm_simulate": tools.fcm_simulate,
|
||||
"fetch": tools.fetch,
|
||||
"graph_health": tools.graph_health,
|
||||
"graph_impact": tools.graph_impact,
|
||||
"graph_lineage": tools.graph_lineage,
|
||||
"graph_reindex": tools.graph_reindex,
|
||||
"list_directory": tools.list_directory,
|
||||
"list_memory_projects": tools.list_memory_projects,
|
||||
"list_workspaces": tools.list_workspaces,
|
||||
|
||||
@@ -1,6 +1,12 @@
|
||||
"""Tests for delete_note MCP tool."""
|
||||
|
||||
from basic_memory.mcp.tools.delete_note import _format_delete_error_response
|
||||
from unittest.mock import patch
|
||||
|
||||
import pytest
|
||||
|
||||
from basic_memory.mcp.tools.delete_note import delete_note, _format_delete_error_response
|
||||
from basic_memory.mcp.tools.read_note import read_note
|
||||
from basic_memory.mcp.tools.write_note import write_note
|
||||
|
||||
|
||||
class TestDeleteNoteErrorFormatting:
|
||||
@@ -94,5 +100,78 @@ class TestDeleteNoteErrorFormatting:
|
||||
assert "folder/note-title" in result # Permalink format
|
||||
|
||||
|
||||
# Integration tests removed to focus on error formatting coverage
|
||||
# The error formatting tests above provide the necessary coverage for MCP tool error messaging
|
||||
@pytest.mark.asyncio
|
||||
async def test_delete_note_rejects_fuzzy_match(client, test_project):
|
||||
"""delete_note must reject nonexistent identifiers, not fuzzy-match to a similar note."""
|
||||
await write_note(
|
||||
project=test_project.name,
|
||||
title="Delete Target Note",
|
||||
directory="test",
|
||||
content="# Delete Target Note\nShould not be deleted.",
|
||||
)
|
||||
|
||||
# Attempt to delete a nonexistent note — should return False, not silently delete the existing note
|
||||
result = await delete_note(
|
||||
project=test_project.name,
|
||||
identifier="Delete Target NONEXISTENT",
|
||||
)
|
||||
|
||||
# Should indicate not found (False or error string)
|
||||
assert result is False or (isinstance(result, str) and "not found" in result.lower())
|
||||
|
||||
# Verify the existing note was NOT deleted
|
||||
content = await read_note("Delete Target Note", project=test_project.name)
|
||||
assert "Should not be deleted" in content
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_delete_note_detects_project_from_memory_url(client, test_project):
|
||||
"""delete_note should detect project from memory:// URL prefix when project=None."""
|
||||
# Create a note to delete
|
||||
await write_note(
|
||||
project=test_project.name,
|
||||
title="Delete URL Note",
|
||||
directory="test",
|
||||
content="# Delete URL Note\nContent to delete.",
|
||||
)
|
||||
|
||||
# Delete using memory:// URL with project=None — should auto-detect project
|
||||
# The note may or may not be found (depends on URL resolution), but the key
|
||||
# assertion is that routing goes to the correct project
|
||||
result = await delete_note(
|
||||
identifier=f"memory://{test_project.name}/test/delete-url-note",
|
||||
project=None,
|
||||
)
|
||||
|
||||
# Result is True (deleted) or False (not found by that URL) — either is acceptable.
|
||||
# The important thing is it didn't error and routed to the correct project.
|
||||
assert isinstance(result, bool)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_delete_note_skips_detection_for_plain_path(client, test_project):
|
||||
"""delete_note should NOT call detect_project_from_url_prefix for plain path identifiers.
|
||||
|
||||
A plain path like 'research/note' should not be misrouted to a project
|
||||
named 'research' — the 'research' segment is a directory, not a project.
|
||||
"""
|
||||
with patch("basic_memory.mcp.tools.delete_note.detect_project_from_url_prefix") as mock_detect:
|
||||
# Use a plain path (no memory:// prefix) — detection should not be called
|
||||
await delete_note(
|
||||
identifier="test/nonexistent-note",
|
||||
project=None,
|
||||
)
|
||||
|
||||
mock_detect.assert_not_called()
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_delete_note_skips_detection_when_project_provided(client, test_project):
|
||||
"""delete_note should skip URL detection when project is explicitly provided."""
|
||||
with patch("basic_memory.mcp.tools.delete_note.detect_project_from_url_prefix") as mock_detect:
|
||||
await delete_note(
|
||||
identifier=f"memory://{test_project.name}/test/some-note",
|
||||
project=test_project.name,
|
||||
)
|
||||
|
||||
mock_detect.assert_not_called()
|
||||
|
||||
@@ -1,8 +1,11 @@
|
||||
"""Tests for the edit_note MCP tool."""
|
||||
|
||||
from unittest.mock import patch
|
||||
|
||||
import pytest
|
||||
|
||||
from basic_memory.mcp.tools.edit_note import edit_note
|
||||
from basic_memory.mcp.tools.read_note import read_note
|
||||
from basic_memory.mcp.tools.write_note import write_note
|
||||
|
||||
|
||||
@@ -320,7 +323,7 @@ async def test_edit_note_replace_section_missing_section(client, test_project):
|
||||
content="new content",
|
||||
)
|
||||
|
||||
assert "section parameter is required for replace_section operation" in str(exc_info.value)
|
||||
assert "section parameter is required for section-based operations" in str(exc_info.value)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@@ -611,3 +614,218 @@ async def test_edit_note_preserves_permalink_when_frontmatter_missing(client, te
|
||||
assert f"permalink: {test_project.name}/test/test-note" in second_result
|
||||
assert f"[Session: Using project '{test_project.name}']" in second_result
|
||||
# The edit should succeed without validation errors
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_edit_note_find_replace_rejects_fuzzy_match(client, test_project):
|
||||
"""find_replace must reject nonexistent identifiers, not fuzzy-match to a similar note."""
|
||||
# Create two notes that could be fuzzy-matched
|
||||
await write_note(
|
||||
project=test_project.name,
|
||||
title="Routing Test A",
|
||||
directory="test",
|
||||
content="# Routing Test A\nContent A.",
|
||||
)
|
||||
await write_note(
|
||||
project=test_project.name,
|
||||
title="Routing Test B",
|
||||
directory="test",
|
||||
content="# Routing Test B\nContent B.",
|
||||
)
|
||||
|
||||
# Attempt to edit a nonexistent note — should error, not silently edit A or B
|
||||
result = await edit_note(
|
||||
project=test_project.name,
|
||||
identifier="Routing Test NONEXISTENT",
|
||||
operation="find_replace",
|
||||
content="replaced",
|
||||
find_text="Content",
|
||||
)
|
||||
|
||||
assert isinstance(result, str)
|
||||
assert "# Edit Failed" in result
|
||||
|
||||
# Verify neither A nor B was modified
|
||||
content_a = await read_note("Routing Test A", project=test_project.name)
|
||||
assert "Content A" in content_a
|
||||
content_b = await read_note("Routing Test B", project=test_project.name)
|
||||
assert "Content B" in content_b
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_edit_note_append_autocreate_not_fuzzy_match(client, test_project):
|
||||
"""append to a nonexistent note should auto-create it, not fuzzy-match an existing note."""
|
||||
await write_note(
|
||||
project=test_project.name,
|
||||
title="Existing Note Alpha",
|
||||
directory="test",
|
||||
content="# Existing Note Alpha\nOriginal content.",
|
||||
)
|
||||
|
||||
# Append to a nonexistent note — should create a new note, not edit "Existing Note Alpha"
|
||||
result = await edit_note(
|
||||
project=test_project.name,
|
||||
identifier="Existing Note ZZZZZ",
|
||||
operation="append",
|
||||
content="# New Note\nBrand new content.",
|
||||
)
|
||||
|
||||
assert isinstance(result, str)
|
||||
assert "Created note (append)" in result
|
||||
assert "fileCreated: true" in result
|
||||
|
||||
# Verify original note was NOT modified
|
||||
content = await read_note("Existing Note Alpha", project=test_project.name)
|
||||
assert "Original content" in content
|
||||
assert "Brand new content" not in content
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_edit_note_insert_before_section_operation(client, test_project):
|
||||
"""Test inserting content before a section heading."""
|
||||
# Create initial note with sections
|
||||
await write_note(
|
||||
project=test_project.name,
|
||||
title="Insert Before Doc",
|
||||
directory="docs",
|
||||
content="# Doc\n\n## Overview\nOverview content.\n\n## Details\nDetail content.",
|
||||
)
|
||||
|
||||
result = await edit_note(
|
||||
project=test_project.name,
|
||||
identifier="docs/insert-before-doc",
|
||||
operation="insert_before_section",
|
||||
content="--- inserted divider ---",
|
||||
section="## Details",
|
||||
)
|
||||
|
||||
assert isinstance(result, str)
|
||||
assert "Edited note (insert_before_section)" in result
|
||||
assert f"project: {test_project.name}" in result
|
||||
assert "Inserted content before section '## Details'" in result
|
||||
assert f"[Session: Using project '{test_project.name}']" in result
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_edit_note_insert_after_section_operation(client, test_project):
|
||||
"""Test inserting content after a section heading."""
|
||||
# Create initial note with sections
|
||||
await write_note(
|
||||
project=test_project.name,
|
||||
title="Insert After Doc",
|
||||
directory="docs",
|
||||
content="# Doc\n\n## Overview\nOverview content.\n\n## Details\nDetail content.",
|
||||
)
|
||||
|
||||
result = await edit_note(
|
||||
project=test_project.name,
|
||||
identifier="docs/insert-after-doc",
|
||||
operation="insert_after_section",
|
||||
content="Inserted after overview heading",
|
||||
section="## Overview",
|
||||
)
|
||||
|
||||
assert isinstance(result, str)
|
||||
assert "Edited note (insert_after_section)" in result
|
||||
assert f"project: {test_project.name}" in result
|
||||
assert "Inserted content after section '## Overview'" in result
|
||||
assert f"[Session: Using project '{test_project.name}']" in result
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_edit_note_insert_before_section_missing_section(client, test_project):
|
||||
"""Test insert_before_section without section parameter raises ValueError."""
|
||||
await write_note(
|
||||
project=test_project.name,
|
||||
title="Test Note",
|
||||
directory="test",
|
||||
content="# Test\nContent here.",
|
||||
)
|
||||
|
||||
with pytest.raises(ValueError, match="section parameter is required"):
|
||||
await edit_note(
|
||||
project=test_project.name,
|
||||
identifier="test/test-note",
|
||||
operation="insert_before_section",
|
||||
content="new content",
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_edit_note_insert_before_section_not_found(client, test_project):
|
||||
"""Test insert_before_section when section doesn't exist returns error."""
|
||||
await write_note(
|
||||
project=test_project.name,
|
||||
title="Test Note",
|
||||
directory="test",
|
||||
content="# Test\n\n## Existing\nContent here.",
|
||||
)
|
||||
|
||||
result = await edit_note(
|
||||
project=test_project.name,
|
||||
identifier="test/test-note",
|
||||
operation="insert_before_section",
|
||||
content="new content",
|
||||
section="## Nonexistent",
|
||||
)
|
||||
|
||||
assert isinstance(result, str)
|
||||
assert "# Edit Failed" in result
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_edit_note_detects_project_from_memory_url(client, test_project):
|
||||
"""edit_note should detect project from memory:// URL prefix when project=None."""
|
||||
# Create a note first
|
||||
await write_note(
|
||||
project=test_project.name,
|
||||
title="URL Detection Note",
|
||||
directory="test",
|
||||
content="# URL Detection Note\nOriginal content.",
|
||||
)
|
||||
|
||||
# Edit using memory:// URL with project=None — should auto-detect project
|
||||
# The memory URL uses the permalink (which includes project prefix)
|
||||
result = await edit_note(
|
||||
identifier=f"memory://{test_project.name}/test/url-detection-note",
|
||||
operation="append",
|
||||
content="\nAppended via memory URL.",
|
||||
project=None,
|
||||
)
|
||||
|
||||
assert isinstance(result, str)
|
||||
# Should route to the correct project and succeed (either edit or create)
|
||||
assert f"project: {test_project.name}" in result
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_edit_note_skips_detection_for_plain_path(client, test_project):
|
||||
"""edit_note should NOT call detect_project_from_url_prefix for plain path identifiers.
|
||||
|
||||
A plain path like 'research/note' should not be misrouted to a project
|
||||
named 'research' — the 'research' segment is a directory, not a project.
|
||||
"""
|
||||
with patch("basic_memory.mcp.tools.edit_note.detect_project_from_url_prefix") as mock_detect:
|
||||
# Use a plain path (no memory:// prefix) — detection should not be called
|
||||
await edit_note(
|
||||
identifier="test/some-note",
|
||||
operation="append",
|
||||
content="content",
|
||||
project=None,
|
||||
)
|
||||
|
||||
mock_detect.assert_not_called()
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_edit_note_skips_detection_when_project_provided(client, test_project):
|
||||
"""edit_note should skip URL detection when project is explicitly provided."""
|
||||
with patch("basic_memory.mcp.tools.edit_note.detect_project_from_url_prefix") as mock_detect:
|
||||
await edit_note(
|
||||
identifier=f"memory://{test_project.name}/test/some-note",
|
||||
operation="append",
|
||||
content="content",
|
||||
project=test_project.name,
|
||||
)
|
||||
|
||||
mock_detect.assert_not_called()
|
||||
|
||||
@@ -1,114 +0,0 @@
|
||||
"""Tests for graph intelligence MCP tools."""
|
||||
|
||||
import pytest
|
||||
|
||||
from basic_memory.mcp.tools import (
|
||||
fcm_export_model,
|
||||
fcm_import_model,
|
||||
fcm_rank_actions,
|
||||
fcm_simulate,
|
||||
graph_health,
|
||||
graph_impact,
|
||||
graph_lineage,
|
||||
graph_reindex,
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_graph_lineage_json_and_text_modes(app, test_project):
|
||||
json_result = await graph_lineage(
|
||||
start="memory://specs/search",
|
||||
project=test_project.name,
|
||||
output_format="json",
|
||||
)
|
||||
assert isinstance(json_result, dict)
|
||||
assert set(["root", "paths", "generated_at"]).issubset(json_result.keys())
|
||||
|
||||
text_result = await graph_lineage(
|
||||
start="memory://specs/search",
|
||||
project=test_project.name,
|
||||
output_format="text",
|
||||
)
|
||||
assert isinstance(text_result, str)
|
||||
assert "Graph Lineage" in text_result
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_graph_impact_and_health(app, test_project):
|
||||
impact = await graph_impact(
|
||||
target="memory://specs/search",
|
||||
horizon=2,
|
||||
project=test_project.name,
|
||||
output_format="json",
|
||||
)
|
||||
assert isinstance(impact, dict)
|
||||
assert set(["target", "affected", "summary"]).issubset(impact.keys())
|
||||
|
||||
health = await graph_health(
|
||||
scope="specs",
|
||||
timeframe="30d",
|
||||
project=test_project.name,
|
||||
output_format="json",
|
||||
)
|
||||
assert isinstance(health, dict)
|
||||
assert set(["metrics", "issues", "computed_at"]).issubset(health.keys())
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_graph_reindex(app, test_project):
|
||||
result = await graph_reindex(project=test_project.name, output_format="json")
|
||||
assert isinstance(result, dict)
|
||||
assert result["status"] == "queued"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_fcm_simulate_and_rank_actions(app, test_project):
|
||||
simulation = await fcm_simulate(
|
||||
actions=[{"node_id": "n1", "delta": 0.2}],
|
||||
project=test_project.name,
|
||||
output_format="json",
|
||||
)
|
||||
assert isinstance(simulation, dict)
|
||||
assert set(["baseline", "projected", "deltas", "stability", "confidence"]).issubset(
|
||||
simulation.keys()
|
||||
)
|
||||
|
||||
ranking = await fcm_rank_actions(
|
||||
goal="reduce-regressions",
|
||||
top_k=2,
|
||||
project=test_project.name,
|
||||
output_format="json",
|
||||
)
|
||||
assert isinstance(ranking, dict)
|
||||
assert set(["goal", "recommendations"]).issubset(ranking.keys())
|
||||
assert len(ranking["recommendations"]) <= 2
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_fcm_import_export_json_and_text(app, test_project):
|
||||
imported = await fcm_import_model(
|
||||
source="/tmp/model.csv",
|
||||
format="csv_bundle_v1",
|
||||
project=test_project.name,
|
||||
output_format="json",
|
||||
)
|
||||
assert isinstance(imported, dict)
|
||||
assert "import_id" in imported
|
||||
|
||||
exported_json = await fcm_export_model(
|
||||
format="csv_bundle_v1",
|
||||
selection={"scope": "all"},
|
||||
project=test_project.name,
|
||||
output_format="json",
|
||||
)
|
||||
assert isinstance(exported_json, dict)
|
||||
assert set(["export_id", "files", "node_count", "edge_count"]).issubset(exported_json.keys())
|
||||
|
||||
exported_text = await fcm_export_model(
|
||||
format="csv_bundle_v1",
|
||||
selection={"scope": "all"},
|
||||
project=test_project.name,
|
||||
output_format="text",
|
||||
)
|
||||
assert isinstance(exported_text, str)
|
||||
assert "FCM Export" in exported_text
|
||||
@@ -590,6 +590,31 @@ async def test_move_note_preserves_frontmatter(app, client, test_project):
|
||||
assert "Content with custom metadata" in content
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_move_note_rejects_fuzzy_match(client, test_project):
|
||||
"""move_note must reject nonexistent identifiers, not fuzzy-match to a similar note."""
|
||||
await write_note(
|
||||
project=test_project.name,
|
||||
title="Move Target Note",
|
||||
directory="source",
|
||||
content="# Move Target Note\nShould not be moved.",
|
||||
)
|
||||
|
||||
# Attempt to move a nonexistent note — should error, not silently move the existing note
|
||||
result = await move_note(
|
||||
project=test_project.name,
|
||||
identifier="Move Target NONEXISTENT",
|
||||
destination_path="target/Moved.md",
|
||||
)
|
||||
|
||||
assert isinstance(result, str)
|
||||
assert "# Move Failed" in result
|
||||
|
||||
# Verify the existing note was NOT moved
|
||||
content = await read_note("Move Target Note", project=test_project.name)
|
||||
assert "Should not be moved" in content
|
||||
|
||||
|
||||
class TestMoveNoteErrorFormatting:
|
||||
"""Test move note error formatting for better user experience."""
|
||||
|
||||
|
||||
@@ -788,6 +788,7 @@ async def test_search_notes_defaults_to_hybrid_when_semantic_enabled(monkeypatch
|
||||
@dataclass
|
||||
class StubConfig:
|
||||
semantic_search_enabled: bool = True
|
||||
default_search_type: str | None = None
|
||||
|
||||
@dataclass
|
||||
class StubContainer:
|
||||
@@ -849,6 +850,7 @@ async def test_search_notes_defaults_to_fts_when_semantic_disabled(monkeypatch):
|
||||
@dataclass
|
||||
class StubConfig:
|
||||
semantic_search_enabled: bool = False
|
||||
default_search_type: str | None = None
|
||||
|
||||
@dataclass
|
||||
class StubContainer:
|
||||
@@ -909,6 +911,7 @@ async def test_search_notes_explicit_text_stays_fts_when_semantic_enabled(monkey
|
||||
@dataclass
|
||||
class StubConfig:
|
||||
semantic_search_enabled: bool = True
|
||||
default_search_type: str | None = None
|
||||
|
||||
@dataclass
|
||||
class StubContainer:
|
||||
@@ -976,7 +979,11 @@ async def test_search_notes_defaults_to_hybrid_when_container_not_initialized(mo
|
||||
lambda: type(
|
||||
"StubConfigManager",
|
||||
(),
|
||||
{"config": type("Cfg", (), {"semantic_search_enabled": True})()},
|
||||
{
|
||||
"config": type(
|
||||
"Cfg", (), {"semantic_search_enabled": True, "default_search_type": None}
|
||||
)()
|
||||
},
|
||||
)(),
|
||||
)
|
||||
|
||||
@@ -1037,7 +1044,11 @@ async def test_search_notes_defaults_to_fts_when_container_not_initialized_and_s
|
||||
lambda: type(
|
||||
"StubConfigManager",
|
||||
(),
|
||||
{"config": type("Cfg", (), {"semantic_search_enabled": False})()},
|
||||
{
|
||||
"config": type(
|
||||
"Cfg", (), {"semantic_search_enabled": False, "default_search_type": None}
|
||||
)()
|
||||
},
|
||||
)(),
|
||||
)
|
||||
|
||||
@@ -1146,6 +1157,54 @@ async def test_search_notes_explicit_entity_types_overrides_default(monkeypatch)
|
||||
assert captured_payload["entity_types"] == ["observation"]
|
||||
|
||||
|
||||
# --- Tests for note_types case-insensitivity ------------------------------------
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_search_notes_note_types_lowercased(monkeypatch):
|
||||
"""note_types values are lowercased so 'Chapter' matches stored 'chapter'."""
|
||||
import importlib
|
||||
|
||||
search_mod = importlib.import_module("basic_memory.mcp.tools.search")
|
||||
clients_mod = importlib.import_module("basic_memory.mcp.clients")
|
||||
|
||||
class StubProject:
|
||||
name = "test-project"
|
||||
external_id = "test-external-id"
|
||||
|
||||
@asynccontextmanager
|
||||
async def fake_get_project_client(*args, **kwargs):
|
||||
yield (object(), StubProject())
|
||||
|
||||
async def fake_resolve_project_and_path(
|
||||
client, identifier, project=None, context=None, headers=None
|
||||
):
|
||||
return StubProject(), identifier, False
|
||||
|
||||
captured_payload: dict = {}
|
||||
|
||||
class MockSearchClient:
|
||||
def __init__(self, *args, **kwargs):
|
||||
pass
|
||||
|
||||
async def search(self, payload, page, page_size):
|
||||
captured_payload.update(payload)
|
||||
return SearchResponse(results=[], current_page=page, page_size=page_size)
|
||||
|
||||
monkeypatch.setattr(search_mod, "get_project_client", fake_get_project_client)
|
||||
monkeypatch.setattr(search_mod, "resolve_project_and_path", fake_resolve_project_and_path)
|
||||
monkeypatch.setattr(clients_mod, "SearchClient", MockSearchClient)
|
||||
|
||||
await search_mod.search_notes(
|
||||
project="test-project",
|
||||
query="test",
|
||||
note_types=["Chapter", "Person"],
|
||||
)
|
||||
|
||||
# note_types should be lowercased
|
||||
assert captured_payload["note_types"] == ["chapter", "person"]
|
||||
|
||||
|
||||
# --- Tests for tag: prefix parsing (issue #30) ---------------------------------
|
||||
|
||||
|
||||
@@ -1519,3 +1578,54 @@ async def test_search_notes_metadata_filters_preserves_non_aliased_keys(monkeypa
|
||||
|
||||
# "note_type" aliased to "type", "priority" passes through unchanged
|
||||
assert captured_payload["metadata_filters"] == {"type": "spec", "priority": "high"}
|
||||
|
||||
|
||||
def test_default_search_type_uses_config_value():
|
||||
"""_default_search_type should return config.default_search_type when set."""
|
||||
import sys
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
search_module = sys.modules["basic_memory.mcp.tools.search"]
|
||||
|
||||
mock_config = MagicMock()
|
||||
mock_config.default_search_type = "vector"
|
||||
mock_config.semantic_search_enabled = True
|
||||
mock_container = MagicMock()
|
||||
mock_container.config = mock_config
|
||||
|
||||
with patch.object(search_module, "get_container", return_value=mock_container):
|
||||
assert search_module._default_search_type() == "vector"
|
||||
|
||||
|
||||
def test_default_search_type_falls_back_to_hybrid_when_semantic_enabled():
|
||||
"""When default_search_type is None and semantic is enabled, default to hybrid."""
|
||||
import sys
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
search_module = sys.modules["basic_memory.mcp.tools.search"]
|
||||
|
||||
mock_config = MagicMock()
|
||||
mock_config.default_search_type = None
|
||||
mock_config.semantic_search_enabled = True
|
||||
mock_container = MagicMock()
|
||||
mock_container.config = mock_config
|
||||
|
||||
with patch.object(search_module, "get_container", return_value=mock_container):
|
||||
assert search_module._default_search_type() == "hybrid"
|
||||
|
||||
|
||||
def test_default_search_type_falls_back_to_text_when_semantic_disabled():
|
||||
"""When default_search_type is None and semantic is disabled, default to text."""
|
||||
import sys
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
search_module = sys.modules["basic_memory.mcp.tools.search"]
|
||||
|
||||
mock_config = MagicMock()
|
||||
mock_config.default_search_type = None
|
||||
mock_config.semantic_search_enabled = False
|
||||
mock_container = MagicMock()
|
||||
mock_container.config = mock_config
|
||||
|
||||
with patch.object(search_module, "get_container", return_value=mock_container):
|
||||
assert search_module._default_search_type() == "text"
|
||||
|
||||
@@ -0,0 +1,309 @@
|
||||
"""Telemetry coverage for MCP tool root spans."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import importlib
|
||||
from contextlib import contextmanager
|
||||
|
||||
import pytest
|
||||
|
||||
build_context_module = importlib.import_module("basic_memory.mcp.tools.build_context")
|
||||
edit_note_module = importlib.import_module("basic_memory.mcp.tools.edit_note")
|
||||
read_note_module = importlib.import_module("basic_memory.mcp.tools.read_note")
|
||||
search_module = importlib.import_module("basic_memory.mcp.tools.search")
|
||||
write_note_module = importlib.import_module("basic_memory.mcp.tools.write_note")
|
||||
|
||||
|
||||
def _recording_contexts():
|
||||
operations: list[tuple[str, dict]] = []
|
||||
contexts: list[dict] = []
|
||||
|
||||
@contextmanager
|
||||
def fake_operation(name: str, **attrs):
|
||||
operations.append((name, attrs))
|
||||
yield
|
||||
|
||||
@contextmanager
|
||||
def fake_contextualize(**attrs):
|
||||
contexts.append(attrs)
|
||||
yield
|
||||
|
||||
return operations, contexts, fake_operation, fake_contextualize
|
||||
|
||||
|
||||
def _contains_context(contexts: list[dict], expected: dict) -> bool:
|
||||
return any(expected.items() <= context.items() for context in contexts)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_write_note_emits_root_operation_and_project_context(
|
||||
app, test_project, monkeypatch
|
||||
) -> None:
|
||||
operations, contexts, fake_operation, fake_contextualize = _recording_contexts()
|
||||
monkeypatch.setattr(write_note_module.telemetry, "operation", fake_operation)
|
||||
monkeypatch.setattr(write_note_module.telemetry, "contextualize", fake_contextualize)
|
||||
|
||||
await write_note_module.write_note(
|
||||
project=test_project.name,
|
||||
title="Telemetry Note",
|
||||
directory="notes",
|
||||
content="Telemetry content",
|
||||
output_format="json",
|
||||
)
|
||||
|
||||
assert operations == [
|
||||
(
|
||||
"mcp.tool.write_note",
|
||||
{
|
||||
"entrypoint": "mcp",
|
||||
"tool_name": "write_note",
|
||||
"requested_project": test_project.name,
|
||||
"workspace_id": None,
|
||||
"note_type": "note",
|
||||
"overwrite": False,
|
||||
"output_format": "json",
|
||||
},
|
||||
)
|
||||
]
|
||||
assert _contains_context(
|
||||
contexts,
|
||||
{
|
||||
"project_name": test_project.name,
|
||||
"route_mode": "local_asgi",
|
||||
},
|
||||
)
|
||||
assert _contains_context(
|
||||
contexts,
|
||||
{
|
||||
"project_name": test_project.name,
|
||||
"tool_name": "write_note",
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_read_note_emits_root_operation_and_project_context(
|
||||
app, test_project, monkeypatch
|
||||
) -> None:
|
||||
await write_note_module.write_note(
|
||||
project=test_project.name,
|
||||
title="Readable Telemetry Note",
|
||||
directory="notes",
|
||||
content="Readable telemetry content",
|
||||
)
|
||||
|
||||
operations, contexts, fake_operation, fake_contextualize = _recording_contexts()
|
||||
monkeypatch.setattr(read_note_module.telemetry, "operation", fake_operation)
|
||||
monkeypatch.setattr(read_note_module.telemetry, "contextualize", fake_contextualize)
|
||||
|
||||
await read_note_module.read_note(
|
||||
"notes/readable-telemetry-note",
|
||||
project=test_project.name,
|
||||
output_format="json",
|
||||
include_frontmatter=True,
|
||||
)
|
||||
|
||||
assert operations == [
|
||||
(
|
||||
"mcp.tool.read_note",
|
||||
{
|
||||
"entrypoint": "mcp",
|
||||
"tool_name": "read_note",
|
||||
"requested_project": test_project.name,
|
||||
"workspace_id": None,
|
||||
"output_format": "json",
|
||||
"page": 1,
|
||||
"page_size": 10,
|
||||
"include_frontmatter": True,
|
||||
},
|
||||
)
|
||||
]
|
||||
assert _contains_context(
|
||||
contexts,
|
||||
{
|
||||
"project_name": test_project.name,
|
||||
"route_mode": "local_asgi",
|
||||
},
|
||||
)
|
||||
assert _contains_context(
|
||||
contexts,
|
||||
{
|
||||
"project_name": test_project.name,
|
||||
"tool_name": "read_note",
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_search_notes_emits_root_operation_and_project_context(
|
||||
app, test_project, monkeypatch
|
||||
) -> None:
|
||||
await write_note_module.write_note(
|
||||
project=test_project.name,
|
||||
title="Searchable Telemetry Note",
|
||||
directory="notes",
|
||||
content="Telemetry search content",
|
||||
tags=["telemetry"],
|
||||
)
|
||||
|
||||
operations, contexts, fake_operation, fake_contextualize = _recording_contexts()
|
||||
monkeypatch.setattr(search_module.telemetry, "operation", fake_operation)
|
||||
monkeypatch.setattr(search_module.telemetry, "contextualize", fake_contextualize)
|
||||
|
||||
await search_module.search_notes(
|
||||
project=test_project.name,
|
||||
query="telemetry",
|
||||
search_type="text",
|
||||
tags=["telemetry"],
|
||||
output_format="json",
|
||||
)
|
||||
|
||||
assert operations[0] == (
|
||||
"mcp.tool.search_notes",
|
||||
{
|
||||
"entrypoint": "mcp",
|
||||
"tool_name": "search_notes",
|
||||
"requested_project": test_project.name,
|
||||
"workspace_id": None,
|
||||
"search_type": "text",
|
||||
"output_format": "json",
|
||||
"page": 1,
|
||||
"page_size": 10,
|
||||
"has_query": True,
|
||||
"note_type_filter_count": 0,
|
||||
"entity_type_filter_count": 0,
|
||||
"has_metadata_filters": False,
|
||||
"has_tags_filter": True,
|
||||
"has_status_filter": False,
|
||||
},
|
||||
)
|
||||
assert ("api.request.search",) == (operations[1][0],)
|
||||
assert _contains_context(
|
||||
contexts,
|
||||
{
|
||||
"project_name": test_project.name,
|
||||
"route_mode": "local_asgi",
|
||||
},
|
||||
)
|
||||
assert _contains_context(
|
||||
contexts,
|
||||
{
|
||||
"project_name": test_project.name,
|
||||
"tool_name": "search_notes",
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_edit_note_emits_root_operation_and_project_context(
|
||||
app, test_project, monkeypatch
|
||||
) -> None:
|
||||
await write_note_module.write_note(
|
||||
project=test_project.name,
|
||||
title="Editable Telemetry Note",
|
||||
directory="notes",
|
||||
content="Original telemetry content",
|
||||
)
|
||||
|
||||
operations, contexts, fake_operation, fake_contextualize = _recording_contexts()
|
||||
monkeypatch.setattr(edit_note_module.telemetry, "operation", fake_operation)
|
||||
monkeypatch.setattr(edit_note_module.telemetry, "contextualize", fake_contextualize)
|
||||
|
||||
await edit_note_module.edit_note(
|
||||
"notes/editable-telemetry-note",
|
||||
operation="append",
|
||||
content="\n\nAppended telemetry content",
|
||||
project=test_project.name,
|
||||
output_format="json",
|
||||
)
|
||||
|
||||
assert operations == [
|
||||
(
|
||||
"mcp.tool.edit_note",
|
||||
{
|
||||
"entrypoint": "mcp",
|
||||
"tool_name": "edit_note",
|
||||
"requested_project": test_project.name,
|
||||
"workspace_id": None,
|
||||
"edit_operation": "append",
|
||||
"output_format": "json",
|
||||
"has_section": False,
|
||||
"has_find_text": False,
|
||||
"expected_replacements": 1,
|
||||
},
|
||||
)
|
||||
]
|
||||
assert _contains_context(
|
||||
contexts,
|
||||
{
|
||||
"project_name": test_project.name,
|
||||
"route_mode": "local_asgi",
|
||||
},
|
||||
)
|
||||
assert _contains_context(
|
||||
contexts,
|
||||
{
|
||||
"project_name": test_project.name,
|
||||
"tool_name": "edit_note",
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_build_context_emits_root_operation_and_project_context(
|
||||
app, test_project, monkeypatch
|
||||
) -> None:
|
||||
await write_note_module.write_note(
|
||||
project=test_project.name,
|
||||
title="Context Telemetry Note",
|
||||
directory="notes",
|
||||
content="Context telemetry content",
|
||||
)
|
||||
|
||||
operations, contexts, fake_operation, fake_contextualize = _recording_contexts()
|
||||
monkeypatch.setattr(build_context_module.telemetry, "operation", fake_operation)
|
||||
monkeypatch.setattr(build_context_module.telemetry, "contextualize", fake_contextualize)
|
||||
|
||||
await build_context_module.build_context(
|
||||
url="memory://notes/context-telemetry-note",
|
||||
project=test_project.name,
|
||||
depth=2,
|
||||
timeframe="7d",
|
||||
page=1,
|
||||
page_size=5,
|
||||
max_related=3,
|
||||
output_format="json",
|
||||
)
|
||||
|
||||
assert operations == [
|
||||
(
|
||||
"mcp.tool.build_context",
|
||||
{
|
||||
"entrypoint": "mcp",
|
||||
"tool_name": "build_context",
|
||||
"requested_project": test_project.name,
|
||||
"workspace_id": None,
|
||||
"depth": 2,
|
||||
"timeframe": "7d",
|
||||
"page": 1,
|
||||
"page_size": 5,
|
||||
"max_related": 3,
|
||||
"output_format": "json",
|
||||
"is_memory_url": True,
|
||||
},
|
||||
)
|
||||
]
|
||||
assert _contains_context(
|
||||
contexts,
|
||||
{
|
||||
"project_name": test_project.name,
|
||||
"route_mode": "local_asgi",
|
||||
},
|
||||
)
|
||||
assert _contains_context(
|
||||
contexts,
|
||||
{
|
||||
"project_name": test_project.name,
|
||||
"tool_name": "build_context",
|
||||
},
|
||||
)
|
||||
@@ -1257,6 +1257,11 @@ class TestWriteNoteOverwriteGuard:
|
||||
# Set config to allow overwrites by default
|
||||
app_config.write_note_overwrite_default = True
|
||||
config_module._CONFIG_CACHE = app_config
|
||||
# Pin mtime+size to the on-disk file so the cache guard sees a match
|
||||
# and keeps our injected config instead of re-reading from disk.
|
||||
_st = config_manager.config_file.stat()
|
||||
config_module._CONFIG_MTIME = _st.st_mtime
|
||||
config_module._CONFIG_SIZE = _st.st_size
|
||||
|
||||
try:
|
||||
await write_note(
|
||||
@@ -1281,6 +1286,9 @@ class TestWriteNoteOverwriteGuard:
|
||||
# Restore config
|
||||
app_config.write_note_overwrite_default = False
|
||||
config_module._CONFIG_CACHE = app_config
|
||||
_st = config_manager.config_file.stat()
|
||||
config_module._CONFIG_MTIME = _st.st_mtime
|
||||
config_module._CONFIG_SIZE = _st.st_size
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_write_note_new_note_unaffected(self, app, test_project):
|
||||
|
||||
@@ -237,6 +237,29 @@ class TestBuildChunkRecords:
|
||||
records = self.repo._build_chunk_records(rows)
|
||||
assert any("99" in r["chunk_key"] for r in records)
|
||||
|
||||
def test_duplicate_rows_collapse_to_unique_chunk_keys(self):
|
||||
rows = [
|
||||
_make_row(
|
||||
row_type=SearchItemType.ENTITY.value,
|
||||
title="Spec",
|
||||
permalink="spec",
|
||||
content_snippet="shared content",
|
||||
row_id=77,
|
||||
),
|
||||
_make_row(
|
||||
row_type=SearchItemType.ENTITY.value,
|
||||
title="Spec",
|
||||
permalink="spec",
|
||||
content_snippet="shared content",
|
||||
row_id=77,
|
||||
),
|
||||
]
|
||||
|
||||
records = self.repo._build_chunk_records(rows)
|
||||
|
||||
assert len(records) == 1
|
||||
assert records[0]["chunk_key"] == "entity:77:0"
|
||||
|
||||
|
||||
# --- SQLite SemanticSearchDisabledError ---
|
||||
|
||||
|
||||
@@ -345,7 +345,7 @@ def test_edit_entity_request_find_replace_empty_find_text():
|
||||
def test_edit_entity_request_replace_section_empty_section():
|
||||
"""Test that replace_section operation requires non-empty section parameter."""
|
||||
with pytest.raises(
|
||||
ValueError, match="section parameter is required for replace_section operation"
|
||||
ValueError, match="section parameter is required for section-based operations"
|
||||
):
|
||||
EditEntityRequest.model_validate(
|
||||
{
|
||||
@@ -356,6 +356,46 @@ def test_edit_entity_request_replace_section_empty_section():
|
||||
)
|
||||
|
||||
|
||||
def test_edit_entity_request_insert_before_section():
|
||||
"""Test insert_before_section is a valid operation."""
|
||||
edit_request = EditEntityRequest.model_validate(
|
||||
{
|
||||
"operation": "insert_before_section",
|
||||
"content": "content to insert",
|
||||
"section": "## Target Section",
|
||||
}
|
||||
)
|
||||
assert edit_request.operation == "insert_before_section"
|
||||
assert edit_request.section == "## Target Section"
|
||||
|
||||
|
||||
def test_edit_entity_request_insert_after_section():
|
||||
"""Test insert_after_section is a valid operation."""
|
||||
edit_request = EditEntityRequest.model_validate(
|
||||
{
|
||||
"operation": "insert_after_section",
|
||||
"content": "content to insert",
|
||||
"section": "## Target Section",
|
||||
}
|
||||
)
|
||||
assert edit_request.operation == "insert_after_section"
|
||||
assert edit_request.section == "## Target Section"
|
||||
|
||||
|
||||
def test_edit_entity_request_insert_before_section_empty_section():
|
||||
"""Test that insert_before_section requires non-empty section parameter."""
|
||||
with pytest.raises(
|
||||
ValueError, match="section parameter is required for section-based operations"
|
||||
):
|
||||
EditEntityRequest.model_validate(
|
||||
{
|
||||
"operation": "insert_before_section",
|
||||
"content": "content",
|
||||
"section": "",
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
# New tests for timeframe parsing functions
|
||||
class TestTimeframeParsing:
|
||||
"""Test cases for parse_timeframe() and validate_timeframe() functions."""
|
||||
@@ -391,7 +431,7 @@ class TestTimeframeParsing:
|
||||
result_1d = parse_timeframe("1d")
|
||||
expected_1d = now - timedelta(days=1)
|
||||
diff = abs((result_1d - expected_1d).total_seconds())
|
||||
assert diff < 3600 # Within 1 hour tolerance (accounts for DST transitions)
|
||||
assert diff <= 3610 # Within 1 hour tolerance + execution margin (DST transitions)
|
||||
assert result_1d.tzinfo is not None
|
||||
|
||||
# Test yesterday - should be yesterday at same time
|
||||
@@ -404,7 +444,7 @@ class TestTimeframeParsing:
|
||||
result_week = parse_timeframe("1 week ago")
|
||||
expected_week = now - timedelta(weeks=1)
|
||||
diff = abs((result_week - expected_week).total_seconds())
|
||||
assert diff < 3600 # Within 1 hour tolerance
|
||||
assert diff <= 3610 # Within 1 hour tolerance + execution margin (DST transitions)
|
||||
assert result_week.tzinfo is not None
|
||||
|
||||
def test_parse_timeframe_invalid(self):
|
||||
|
||||
@@ -1402,6 +1402,267 @@ async def test_edit_entity_replace_section_strips_duplicate_header(
|
||||
assert "## Another Section" in file_content # Other sections preserved
|
||||
|
||||
|
||||
# Insert before/after section tests
|
||||
@pytest.mark.asyncio
|
||||
async def test_edit_entity_insert_before_section(
|
||||
entity_service: EntityService, file_service: FileService
|
||||
):
|
||||
"""Test inserting content before a section heading."""
|
||||
content = dedent("""
|
||||
# Main Title
|
||||
|
||||
## Section 1
|
||||
Section 1 content
|
||||
|
||||
## Section 2
|
||||
Section 2 content
|
||||
""").strip()
|
||||
|
||||
entity = await entity_service.create_entity(
|
||||
EntitySchema(
|
||||
title="Insert Before Test",
|
||||
directory="docs",
|
||||
note_type="note",
|
||||
content=content,
|
||||
)
|
||||
)
|
||||
|
||||
updated = await entity_service.edit_entity(
|
||||
identifier=entity.permalink,
|
||||
operation="insert_before_section",
|
||||
content="Inserted before section 2",
|
||||
section="## Section 2",
|
||||
)
|
||||
|
||||
file_path = file_service.get_entity_path(updated)
|
||||
file_content, _ = await file_service.read_file(file_path)
|
||||
assert "Inserted before section 2" in file_content
|
||||
assert "## Section 2" in file_content
|
||||
assert "Section 2 content" in file_content
|
||||
# Inserted content should appear before the section heading
|
||||
assert file_content.index("Inserted before section 2") < file_content.index("## Section 2")
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_edit_entity_insert_after_section(
|
||||
entity_service: EntityService, file_service: FileService
|
||||
):
|
||||
"""Test inserting content after a section heading."""
|
||||
content = dedent("""
|
||||
# Main Title
|
||||
|
||||
## Section 1
|
||||
Section 1 content
|
||||
|
||||
## Section 2
|
||||
Section 2 content
|
||||
""").strip()
|
||||
|
||||
entity = await entity_service.create_entity(
|
||||
EntitySchema(
|
||||
title="Insert After Test",
|
||||
directory="docs",
|
||||
note_type="note",
|
||||
content=content,
|
||||
)
|
||||
)
|
||||
|
||||
updated = await entity_service.edit_entity(
|
||||
identifier=entity.permalink,
|
||||
operation="insert_after_section",
|
||||
content="Inserted after section 1 heading",
|
||||
section="## Section 1",
|
||||
)
|
||||
|
||||
file_path = file_service.get_entity_path(updated)
|
||||
file_content, _ = await file_service.read_file(file_path)
|
||||
assert "Inserted after section 1 heading" in file_content
|
||||
assert "## Section 1" in file_content
|
||||
assert "Section 1 content" in file_content
|
||||
# Inserted content should appear after the heading but content is also preserved
|
||||
assert file_content.index("## Section 1") < file_content.index(
|
||||
"Inserted after section 1 heading"
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_edit_entity_insert_before_section_not_found(entity_service: EntityService):
|
||||
"""Test insert_before_section raises ValueError when section not found."""
|
||||
entity = await entity_service.create_entity(
|
||||
EntitySchema(
|
||||
title="Test Note",
|
||||
directory="test",
|
||||
note_type="note",
|
||||
content="# Main Title\n\nSome content",
|
||||
)
|
||||
)
|
||||
|
||||
with pytest.raises(ValueError, match="Section '## Missing' not found"):
|
||||
await entity_service.edit_entity(
|
||||
identifier=entity.permalink,
|
||||
operation="insert_before_section",
|
||||
content="new content",
|
||||
section="## Missing",
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_edit_entity_insert_after_section_not_found(entity_service: EntityService):
|
||||
"""Test insert_after_section raises ValueError when section not found."""
|
||||
entity = await entity_service.create_entity(
|
||||
EntitySchema(
|
||||
title="Test Note",
|
||||
directory="test",
|
||||
note_type="note",
|
||||
content="# Main Title\n\nSome content",
|
||||
)
|
||||
)
|
||||
|
||||
with pytest.raises(ValueError, match="Section '## Missing' not found"):
|
||||
await entity_service.edit_entity(
|
||||
identifier=entity.permalink,
|
||||
operation="insert_after_section",
|
||||
content="new content",
|
||||
section="## Missing",
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_edit_entity_insert_before_section_multiple_sections_error(
|
||||
entity_service: EntityService,
|
||||
):
|
||||
"""Test insert_before_section raises ValueError with duplicate sections."""
|
||||
entity = await entity_service.create_entity(
|
||||
EntitySchema(
|
||||
title="Test Note",
|
||||
directory="test",
|
||||
note_type="note",
|
||||
content="# Title\n\n## Dup\nFirst\n\n## Dup\nSecond",
|
||||
)
|
||||
)
|
||||
|
||||
with pytest.raises(ValueError, match="Multiple sections found"):
|
||||
await entity_service.edit_entity(
|
||||
identifier=entity.permalink,
|
||||
operation="insert_before_section",
|
||||
content="new content",
|
||||
section="## Dup",
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_edit_entity_insert_before_section_missing_section_param(
|
||||
entity_service: EntityService,
|
||||
):
|
||||
"""Test insert_before_section raises ValueError when section param is missing."""
|
||||
entity = await entity_service.create_entity(
|
||||
EntitySchema(
|
||||
title="Test Note",
|
||||
directory="test",
|
||||
note_type="note",
|
||||
content="# Title\n\nContent",
|
||||
)
|
||||
)
|
||||
|
||||
with pytest.raises(ValueError, match="section is required"):
|
||||
await entity_service.edit_entity(
|
||||
identifier=entity.permalink,
|
||||
operation="insert_before_section",
|
||||
content="new content",
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_edit_entity_insert_before_section_empty_section(entity_service: EntityService):
|
||||
"""Test insert_before_section raises ValueError when section is empty/whitespace."""
|
||||
entity = await entity_service.create_entity(
|
||||
EntitySchema(
|
||||
title="Test Note",
|
||||
directory="test",
|
||||
note_type="note",
|
||||
content="# Title\n\nContent",
|
||||
)
|
||||
)
|
||||
|
||||
with pytest.raises(ValueError, match="section cannot be empty"):
|
||||
await entity_service.edit_entity(
|
||||
identifier=entity.permalink,
|
||||
operation="insert_before_section",
|
||||
content="new content",
|
||||
section=" ",
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_edit_entity_insert_after_section_at_end_of_document(
|
||||
entity_service: EntityService, file_service: FileService
|
||||
):
|
||||
"""Test inserting after the last section in a document."""
|
||||
content = dedent("""
|
||||
# Main Title
|
||||
|
||||
## Only Section
|
||||
Some content here
|
||||
""").strip()
|
||||
|
||||
entity = await entity_service.create_entity(
|
||||
EntitySchema(
|
||||
title="Insert End Test",
|
||||
directory="docs",
|
||||
note_type="note",
|
||||
content=content,
|
||||
)
|
||||
)
|
||||
|
||||
updated = await entity_service.edit_entity(
|
||||
identifier=entity.permalink,
|
||||
operation="insert_after_section",
|
||||
content="Inserted after the last section heading",
|
||||
section="## Only Section",
|
||||
)
|
||||
|
||||
file_path = file_service.get_entity_path(updated)
|
||||
file_content, _ = await file_service.read_file(file_path)
|
||||
assert "Inserted after the last section heading" in file_content
|
||||
assert "## Only Section" in file_content
|
||||
assert "Some content here" in file_content
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_edit_entity_insert_after_section_preserves_paragraph_separation(
|
||||
entity_service: EntityService, file_service: FileService
|
||||
):
|
||||
"""Test that insert_after_section adds blank line so inserted text doesn't merge
|
||||
with existing section content into a single markdown paragraph."""
|
||||
content = dedent("""
|
||||
# Main Title
|
||||
|
||||
## Section
|
||||
Existing paragraph text
|
||||
""").strip()
|
||||
|
||||
entity = await entity_service.create_entity(
|
||||
EntitySchema(
|
||||
title="Paragraph Sep Test",
|
||||
directory="docs",
|
||||
note_type="note",
|
||||
content=content,
|
||||
)
|
||||
)
|
||||
|
||||
updated = await entity_service.edit_entity(
|
||||
identifier=entity.permalink,
|
||||
operation="insert_after_section",
|
||||
content="Inserted line",
|
||||
section="## Section",
|
||||
)
|
||||
|
||||
file_path = file_service.get_entity_path(updated)
|
||||
file_content, _ = await file_service.read_file(file_path)
|
||||
# The inserted line and existing content should be separated by a blank line
|
||||
assert "Inserted line\n\nExisting paragraph text" in file_content
|
||||
|
||||
|
||||
# Move entity tests
|
||||
@pytest.mark.asyncio
|
||||
async def test_move_entity_success(
|
||||
|
||||
@@ -200,10 +200,10 @@ async def test_initialize_app_no_precedence_warning_when_not_conflicting(
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_run_migrations_triggers_embedding_backfill_on_new_revision(
|
||||
async def test_run_migrations_triggers_embedding_backfill_when_entities_exist_but_no_embeddings(
|
||||
monkeypatch, app_config: BasicMemoryConfig
|
||||
):
|
||||
"""When the trigger revision is newly applied, run automatic embedding backfill once."""
|
||||
"""run_migrations checks for missing embeddings (actual backfill runs in background from MCP)."""
|
||||
|
||||
class StubSearchRepository:
|
||||
def __init__(self, *args, **kwargs):
|
||||
@@ -224,29 +224,24 @@ async def test_run_migrations_triggers_embedding_backfill_on_new_revision(
|
||||
monkeypatch.setattr("basic_memory.db.SQLiteSearchRepository", StubSearchRepository)
|
||||
monkeypatch.setattr("basic_memory.db.PostgresSearchRepository", StubSearchRepository)
|
||||
|
||||
load_revisions_mock = AsyncMock(
|
||||
side_effect=[
|
||||
set(),
|
||||
{db.SEMANTIC_EMBEDDING_BACKFILL_REVISION},
|
||||
]
|
||||
needs_backfill_mock = AsyncMock(return_value=True)
|
||||
monkeypatch.setattr(
|
||||
"basic_memory.db._needs_semantic_embedding_backfill", needs_backfill_mock
|
||||
)
|
||||
backfill_mock = AsyncMock()
|
||||
monkeypatch.setattr("basic_memory.db._load_applied_alembic_revisions", load_revisions_mock)
|
||||
monkeypatch.setattr("basic_memory.db._run_semantic_embedding_backfill", backfill_mock)
|
||||
|
||||
await db.run_migrations(app_config)
|
||||
|
||||
assert load_revisions_mock.await_count == 2
|
||||
backfill_mock.assert_awaited_once_with(app_config, session_marker)
|
||||
# Verifies the check runs — backfill itself is launched by MCP lifespan
|
||||
needs_backfill_mock.assert_awaited_once_with(app_config, session_marker)
|
||||
finally:
|
||||
db._session_maker = original_session_maker # pyright: ignore [reportPrivateUsage]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_run_migrations_skips_embedding_backfill_when_revision_already_applied(
|
||||
async def test_run_migrations_skips_embedding_backfill_when_embeddings_already_exist(
|
||||
monkeypatch, app_config: BasicMemoryConfig
|
||||
):
|
||||
"""If the trigger revision was already present before upgrade, skip backfill."""
|
||||
"""When embeddings already exist, no backfill is needed."""
|
||||
|
||||
class StubSearchRepository:
|
||||
def __init__(self, *args, **kwargs):
|
||||
@@ -267,20 +262,14 @@ async def test_run_migrations_skips_embedding_backfill_when_revision_already_app
|
||||
monkeypatch.setattr("basic_memory.db.SQLiteSearchRepository", StubSearchRepository)
|
||||
monkeypatch.setattr("basic_memory.db.PostgresSearchRepository", StubSearchRepository)
|
||||
|
||||
load_revisions_mock = AsyncMock(
|
||||
side_effect=[
|
||||
{db.SEMANTIC_EMBEDDING_BACKFILL_REVISION},
|
||||
{db.SEMANTIC_EMBEDDING_BACKFILL_REVISION},
|
||||
]
|
||||
needs_backfill_mock = AsyncMock(return_value=False)
|
||||
monkeypatch.setattr(
|
||||
"basic_memory.db._needs_semantic_embedding_backfill", needs_backfill_mock
|
||||
)
|
||||
backfill_mock = AsyncMock()
|
||||
monkeypatch.setattr("basic_memory.db._load_applied_alembic_revisions", load_revisions_mock)
|
||||
monkeypatch.setattr("basic_memory.db._run_semantic_embedding_backfill", backfill_mock)
|
||||
|
||||
await db.run_migrations(app_config)
|
||||
|
||||
assert load_revisions_mock.await_count == 2
|
||||
assert backfill_mock.await_count == 0
|
||||
needs_backfill_mock.assert_awaited_once_with(app_config, session_marker)
|
||||
finally:
|
||||
db._session_maker = original_session_maker # pyright: ignore [reportPrivateUsage]
|
||||
|
||||
@@ -378,3 +367,58 @@ async def test_semantic_embedding_backfill_skips_when_semantic_disabled(
|
||||
app_config.semantic_search_enabled = False
|
||||
await db._run_semantic_embedding_backfill(app_config, session_maker) # pyright: ignore [reportPrivateUsage]
|
||||
assert called is False
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_needs_semantic_embedding_backfill_true_when_entities_exist_no_embeddings(
|
||||
app_config: BasicMemoryConfig,
|
||||
session_maker,
|
||||
test_project,
|
||||
):
|
||||
"""Should return True when entities exist but vector chunks table is empty."""
|
||||
from basic_memory.repository.entity_repository import EntityRepository
|
||||
|
||||
entity_repository = EntityRepository(session_maker, project_id=test_project.id)
|
||||
await entity_repository.create(
|
||||
{
|
||||
"title": "Test Entity",
|
||||
"note_type": "note",
|
||||
"entity_metadata": {},
|
||||
"content_type": "text/markdown",
|
||||
"file_path": "test/backfill-check.md",
|
||||
"permalink": "test/backfill-check",
|
||||
"project_id": test_project.id,
|
||||
"created_at": datetime.now(),
|
||||
"updated_at": datetime.now(),
|
||||
}
|
||||
)
|
||||
|
||||
# Clear any embeddings left by other tests in the shared DB
|
||||
async with db.scoped_session(session_maker) as session:
|
||||
await session.execute(db.text("DELETE FROM search_vector_chunks"))
|
||||
|
||||
app_config.semantic_search_enabled = True
|
||||
result = await db._needs_semantic_embedding_backfill(app_config, session_maker) # pyright: ignore [reportPrivateUsage]
|
||||
assert result is True
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_needs_semantic_embedding_backfill_false_when_no_entities(
|
||||
app_config: BasicMemoryConfig,
|
||||
session_maker,
|
||||
):
|
||||
"""Should return False when no entities exist (nothing to backfill)."""
|
||||
app_config.semantic_search_enabled = True
|
||||
result = await db._needs_semantic_embedding_backfill(app_config, session_maker) # pyright: ignore [reportPrivateUsage]
|
||||
assert result is False
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_needs_semantic_embedding_backfill_false_when_semantic_disabled(
|
||||
app_config: BasicMemoryConfig,
|
||||
session_maker,
|
||||
):
|
||||
"""Should return False when semantic search is disabled."""
|
||||
app_config.semantic_search_enabled = False
|
||||
result = await db._needs_semantic_embedding_backfill(app_config, session_maker) # pyright: ignore [reportPrivateUsage]
|
||||
assert result is False
|
||||
|
||||
@@ -778,6 +778,8 @@ async def test_add_project_with_project_root_sanitizes_paths(
|
||||
from basic_memory import config as config_module
|
||||
|
||||
config_module._CONFIG_CACHE = None
|
||||
config_module._CONFIG_MTIME = None
|
||||
config_module._CONFIG_SIZE = None
|
||||
|
||||
test_cases = [
|
||||
# (project_name, user_path, expected_sanitized_name)
|
||||
@@ -845,6 +847,8 @@ async def test_add_project_with_project_root_rejects_escape_attempts(
|
||||
from basic_memory import config as config_module
|
||||
|
||||
config_module._CONFIG_CACHE = None
|
||||
config_module._CONFIG_MTIME = None
|
||||
config_module._CONFIG_SIZE = None
|
||||
|
||||
# All of these should succeed by being sanitized to paths under project_root
|
||||
# The sanitization removes dangerous patterns, so they don't escape
|
||||
@@ -931,6 +935,8 @@ async def test_add_project_with_project_root_normalizes_case(
|
||||
from basic_memory import config as config_module
|
||||
|
||||
config_module._CONFIG_CACHE = None
|
||||
config_module._CONFIG_MTIME = None
|
||||
config_module._CONFIG_SIZE = None
|
||||
|
||||
test_cases = [
|
||||
# (input_path, expected_normalized_path)
|
||||
@@ -985,6 +991,8 @@ async def test_add_project_with_project_root_detects_case_collisions(
|
||||
from basic_memory import config as config_module
|
||||
|
||||
config_module._CONFIG_CACHE = None
|
||||
config_module._CONFIG_MTIME = None
|
||||
config_module._CONFIG_SIZE = None
|
||||
|
||||
# First, create a project with lowercase path
|
||||
first_project = "documents-project"
|
||||
@@ -1159,6 +1167,8 @@ async def test_add_project_nested_validation_with_project_root(
|
||||
from basic_memory import config as config_module
|
||||
|
||||
config_module._CONFIG_CACHE = None
|
||||
config_module._CONFIG_MTIME = None
|
||||
config_module._CONFIG_SIZE = None
|
||||
|
||||
parent_project_name = f"cloud-parent-{os.urandom(4).hex()}"
|
||||
child_project_name = f"cloud-child-{os.urandom(4).hex()}"
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user