Compare commits

..

1 Commits

Author SHA1 Message Date
phernandez a026412a13 docs: capture graph intelligence phase status and plan
Signed-off-by: phernandez <paul@basicmachines.co>
2026-03-05 10:00:49 -06:00
37 changed files with 2065 additions and 1360 deletions
+4 -147
View File
@@ -2,156 +2,13 @@
## Unreleased
## v0.19.1 (2026-03-08)
### Bug Fixes
- **#649**: Enforce strict entity resolution in destructive MCP tools (`edit_note`, `move_note`, `delete_note`)
- Prevents fuzzy-match fallback from silently editing/moving/deleting the wrong note
- DST-related timeframe validation fix (round instead of truncate days)
### Features
- **#648**: Add `insert_before_section` and `insert_after_section` edit operations
- Add `GET /knowledge/graph` endpoint for full graph visualization
### Dependencies
- Bump authlib from 1.6.6 to 1.6.7
## v0.19.0 (2026-03-07)
### Highlights
- **Semantic vector search** for SQLite and Postgres with FastEmbed embeddings
- **Schema system** for validating and inferring knowledge base structure
- **Per-project cloud routing** with API key authentication
- **Upgraded to FastMCP 3.0** with tool annotations
- **CLI overhaul** with JSON output, workspace awareness, and project dashboard
### Features
- **#550**: Add semantic vector search for SQLite and Postgres
- FastEmbed-based embeddings with automatic backfill
- Hybrid search combining full-text and vector similarity
- Score-based fusion replacing RRF for better ranking
- `min_similarity` override for tuning search precision
- Semantic dependencies are now default, with optional extras fallback
- **#549**: Schema system for Basic Memory
- `schema_infer` — infer schema from existing notes
- `schema_validate` — validate notes against a schema definition
- `schema_diff` — compare schemas across projects
- Frontmatter validation support (#597)
- Read schema definitions from file instead of stale DB metadata (#635)
- **#555**: Per-project local/cloud routing with API key auth
- Individual projects route through cloud while others stay local
- `basic-memory cloud set-key` and `basic-memory project set-cloud/set-local`
- Stdio MCP honors per-project cloud routing (#590)
- **#598**: Upgrade FastMCP 2.12.3 to 3.0.1 with tool annotations
- **#585**: Add JSON output mode for MCP tools (default text)
- `--json` output for CLI commands for scripting and CI
- **#576**: Add workspace selection flow for MCP and CLI
- Workspace-aware cloud project listing
- CLI refactoring for workspace support
- **#544**: Project-prefixed permalinks and memory URL routing
- **#632**: Add overwrite guard to `write_note` tool
- **#614**: `edit_note` append/prepend auto-creates note if not found
- **#609**: Richer content context in search results
- Return matched chunk text in search results (#601)
- Improved content hit rate
- **#602**: Add `created_by` and `last_updated_by` user tracking to Entity
- **#600**: Rename `entity_type` to `note_type` across codebase
- **#574**: Add `display_name` and `is_private` to ProjectItem
- **#569**: Expose `external_id` in EntityResponse and link resolver
- **#567**: Isolate default SQLite DB by config dir
- **#560**: Enable `default_project_mode` by default
- **#559**: Add `basic-memory watch` CLI command
- **#546**: Add cloud discovery touchpoints to CLI and MCP
- **#572**: CLI analytics via Umami event collector
- Replace project info with htop-inspired dashboard
- Merge `search_by_metadata` into `search_notes` with optional query
- Add `--strip-frontmatter` to `basic-memory tool read-note`
- Add `destination_folder` parameter to `move_note` tool
### Bug Fixes
- **#644**: Fix default project resolution in cloud mode
- ChatGPT search/fetch tools broken in cloud mode
- `resolve_project_parameter` falls back to projects API
- **#638**: Restore API backward compatibility for v0.18.x clients
- **#637**: Create backup before config migration overwrites old format
- **#636**: `list_workspaces` bypasses factory pattern on cloud MCP server
- **#631**: `build_context` related_results schema validation failure
- **#613**: Reduce excessive log volume by demoting per-request noise to DEBUG
- **#612**: Handle quoted picoschema enum strings in YAML frontmatter
- **#607**: Guard against closed streams in promo and missing vector tables
- **#606**: Accept null for `expected_replacements` in `edit_note`
- **#595**: `recent_activity` dedup and pagination across MCP tools
- **#593**: Backend-specific distance-to-similarity conversion
- **#582**: Use LinkResolver fallback in `build_context` for flexible identifier matching
- **#577**: Replace RRF with score-based fusion in hybrid search
- **#575**: Remove hardcoded "main" default from `default_project`
- **#534**: Speed up `bm --version` startup
- Fix semantic embeddings not generated on fresh DB or upgrade
- Clarify `search_notes` parameter naming and fix `note_types` case sensitivity
- Parse `tag:` prefix at MCP tool level to avoid hybrid search failure
- Cap sqlite-vec knn k parameter at 4096 limit
- Parameterize SQL queries in search repository type filters
- Coerce list frontmatter values to strings for title and type fields
- Avoid `Post(**metadata)` crash when frontmatter contains 'content' or 'handler' keys
- Upgrade cryptography and python-multipart for security advisories
### Internal
- **#594**: Add `ty` as supplemental type checker
- Batched vector sync orchestration across repositories
- FastEmbed parallel guardrails and provider caching
- Improved cloud CLI status and error messages
- CI coverage and Postgres test fixes
- Default behavior is unchanged: `content` still includes raw markdown with frontmatter.
- With `--strip-frontmatter`, both text and JSON modes return body-only markdown content.
- JSON output now includes an additive `frontmatter` field with parsed YAML metadata (or `null`
when no valid opening frontmatter block exists).
## v0.18.5 (2026-02-13)
-12
View File
@@ -23,18 +23,6 @@ Basic Memory lets you build persistent knowledge through natural conversations w
Claude, while keeping everything in simple Markdown files on your computer. It uses the Model Context Protocol (MCP) to
enable any compatible LLM to read and write to your local knowledge base.
## What's New in v0.19.0
- **Semantic Vector Search** — find notes by meaning, not just keywords. Combines full-text and vector similarity for hybrid search with FastEmbed embeddings.
- **Schema System** — infer, validate, and diff the structure of your knowledge base with `schema_infer`, `schema_validate`, and `schema_diff` tools.
- **Per-Project Cloud Routing** — route individual projects through the cloud while others stay local, using API key authentication (`basic-memory project set-cloud`).
- **FastMCP 3.0** — upgraded to FastMCP 3.0 with tool annotations for better client integration.
- **CLI Overhaul** — JSON output mode (`--json`) for scripting, workspace-aware commands, and an htop-inspired project dashboard.
- **Smarter Editing** — `edit_note` append/prepend auto-creates notes if they don't exist; `write_note` has an overwrite guard to prevent accidental data loss.
- **Richer Search Results** — matched chunk text returned in search results for better context.
See the full [CHANGELOG](CHANGELOG.md) for details.
- Website: [basicmemory.com](https://basicmemory.com?utm_source=github&utm_medium=referral&utm_campaign=readme)
- Documentation: [docs.basicmemory.com](https://docs.basicmemory.com?utm_source=github&utm_medium=referral&utm_campaign=readme)
- Community: [Discord](https://discord.gg/tyvKNccgqN?utm_source=github&utm_medium=referral&utm_campaign=readme)
@@ -0,0 +1,594 @@
# SPEC-LOCAL-GRAPH-INTELLIGENCE-IMPLEMENTATION-PLAN
**Status:** Draft (Decision-Complete)
**Date:** 2026-03-05
**Owner:** Basic Memory Engineering
**Implementation Status (2026-03-05):** Phase 1 contract skeleton implemented in `basic-memory` branch `codex/graph-intelligence-phase1`.
**Related Specs:**
1. `/docs/specs/SPEC-LOCAL-GRAPH-INTELLIGENCE-MASTER.md`
2. `/docs/specs/SPEC-LOCAL-GRAPH-INTELLIGENCE-TECHNICAL-ADDENDUM.md`
3. `/docs/specs/SPEC-LOCAL-GRAPH-INTELLIGENCE.md`
## Scope and Intent
This document is the execution handoff for Local+ Graph Intelligence.
It defines exactly how we will deliver graph and FCM capabilities inside the existing Basic Memory architecture:
1. FastAPI-first business logic.
2. MCP and CLI as thin facades.
3. Local/cloud contract parity.
4. Tight build-test-iterate loop for fast delivery.
This file is intentionally implementation-oriented and does not duplicate pricing narrative from the master spec.
## Architecture Alignment (FastAPI-first, MCP/CLI facade)
Locked architecture alignment for implementation:
1. MCP tools remain thin proxy facades.
2. CLI `bm tool` commands call MCP tools in JSON mode.
3. Core logic lives in FastAPI routers and services.
4. Cloud and local share the same REST contracts.
5. Per-project routing continues through existing project client patterns.
Execution mapping:
1. API routers define public contracts in `/graph` and `/fcm` domains.
2. Services own traversal, scoring, simulation, and fallback logic.
3. Repositories and index providers own data access and graph index operations.
4. MCP typed clients call REST endpoints and return JSON-first tool output.
5. CLI passthrough executes tool calls and prints machine-friendly JSON.
## Locked Decisions
1. SQLite remains operational source for entities, relations, embeddings, and project state.
2. Markdown remains source of truth.
3. Oxigraph/pyoxigraph is the derived graph index for deep traversal.
4. FCM simulation runs in Python service layer; it is not delegated to graph DB query engines.
5. Graph index is rebuildable and disposable; stale index never blocks user workflows.
6. FCM model and scenario artifacts persist in app database.
7. Local+ features are gated by config flags first; entitlement wiring follows later.
8. Graph-first vertical slices ship before deep FCM expansion.
9. Atomic tools ship first; orchestration workflows are deferred.
10. Existing `build_context` and `search_notes` remain backward compatible with no breaking change.
## Progress Snapshot (as of 2026-03-05)
Completed in Phase 1:
1. Added `/graph` and `/fcm` v2 routers with all required contract endpoints.
2. Added graph/FCM request and response schemas for all public API contracts.
3. Added service-layer implementations for graph and FCM contract endpoints.
4. Added typed MCP clients for graph and FCM API calls.
5. Added MCP tools: `graph_lineage`, `graph_impact`, `graph_health`, `graph_reindex`, `fcm_simulate`, `fcm_rank_actions`, `fcm_import_model`, `fcm_export_model`.
6. Added CLI passthrough commands under `bm tool ...` for all planned graph/FCM operations.
7. Added scheduler task names for graph lifecycle: `sync_graph_entity`, `sync_graph_project`, `reindex_graph_project`.
8. Added focused tests for API, MCP clients/tools, and CLI graph/FCM passthrough.
9. Added fast-loop `just` targets: `test-graph-intel-api`, `test-graph-intel-mcp`, `test-graph-intel-cli`, `test-graph-intel`.
Validation completed:
1. `just test-graph-intel` passes.
2. `ruff check` passes on changed files.
3. `pyright` passes on changed files.
Still pending after Phase 1:
1. SQL-backed traversal/scoring for graph `lineage`, `impact`, and `health`.
2. Oxigraph provider integration and stale-index catch-up flow.
3. Persistent FCM model/scenario state and interop round-trip guarantees.
4. Config-flag and entitlement gating at API/tool boundaries.
5. Performance instrumentation and p95 envelope enforcement.
## Delivery Phases
### Phase 1: Contract skeleton
Status: Completed (2026-03-05)
Deliverables:
1. Add `/graph` and `/fcm` API routers with request/response schemas.
2. Add typed MCP clients for graph and FCM endpoints.
3. Add MCP tool passthrough commands for all new operations.
4. Add CLI `bm tool` passthrough commands mirroring MCP surface.
5. Add minimal smoke tests for route reachability and schema validation.
Exit criteria:
1. All endpoints return structured success and error envelopes.
2. MCP/CLI paths execute end-to-end with stubbed service responses.
### Phase 2: Graph capabilities on SQL-backed logic
Status: Next active phase
Deliverables:
1. Implement `lineage`, `impact`, and `health` in service layer using SQL-backed traversal and scoring.
2. Add provenance/evidence linking in graph outputs.
3. Add deterministic graph-health calculations for fixed snapshots.
Exit criteria:
1. `graph_lineage`, `graph_impact`, and `graph_health` pass contract tests.
2. SQL fallback behavior is explicit and covered by tests.
### Phase 3: Oxigraph derived index provider
Status: Planned
Deliverables:
1. Introduce Oxigraph provider behind graph-query interface.
2. Add lazy catch-up jobs and project-wide reindex operation.
3. Preserve SQL fallback when index is missing or stale.
Exit criteria:
1. Stale index path serves results via SQL and schedules catch-up.
2. Index rebuild can be triggered and completed without data loss.
### Phase 4: FCM import/simulate/rank/export
Status: Planned (contract endpoints complete, full behavior pending)
Deliverables:
1. Implement CSV-first import/export contracts.
2. Implement deterministic simulation core with convergence metadata.
3. Implement action ranking with evidence references and confidence output.
4. Persist scenario inputs and result artifacts.
Exit criteria:
1. Research flow scenario passes: import -> simulate -> rank -> export.
2. Interop round-trip preserves node/edge counts and signed weights.
### Phase 5: Hardening
Status: Planned
Deliverables:
1. Performance tuning against published latency envelopes.
2. Local/cloud parity tests for semantics and error behavior.
3. MCP prompt/docs updates for new graph and FCM tools.
4. Operational docs for reindex, fallback, and troubleshooting.
Exit criteria:
1. `just check` passes before merge.
2. Acceptance criteria in this document are fully met.
## API and Interface Additions
### Shared API conventions
1. All endpoints are project-scoped under `/v2/projects/{project_id}`.
2. Request and response bodies are JSON-first and agent-friendly.
3. Success envelope is endpoint-specific payload with deterministic fields and optional probabilistic fields.
4. Error envelope:
```json
{
"error": {
"code": "INVALID_ARGUMENT|NOT_FOUND|INDEX_NOT_READY|MODEL_INVALID|RESOURCE_LIMIT_EXCEEDED|INTERNAL_ERROR",
"message": "string",
"details": {}
}
}
```
5. Latency and scale targets are p95 targets for local default hardware profile.
### 1) `POST /v2/projects/{project_id}/graph/lineage`
Purpose: explain decision lineage and supporting evidence paths.
Request schema:
```json
{
"start": "string",
"goal": "string|null",
"max_hops": 4,
"relation_filters": ["string"]
}
```
Response schema:
```json
{
"root": {"id": "string", "title": "string", "permalink": "string"},
"paths": [
{
"path_id": "string",
"nodes": [{"id": "string", "title": "string"}],
"edges": [{"relation": "string", "direction": "outgoing|incoming"}],
"deterministic_path_score": 0.0,
"confidence": 0.0,
"evidence_refs": ["memory://..."]
}
],
"generated_at": "RFC3339"
}
```
Deterministic fields: `root`, `paths.nodes`, `paths.edges`, `deterministic_path_score`, `generated_at`.
Probabilistic fields: `confidence`.
Latency target: p95 <= 450ms with `max_hops<=4`.
Scale envelope: up to 50k nodes and 300k edges.
### 2) `POST /v2/projects/{project_id}/graph/impact`
Purpose: preview impact radius before edits or decisions.
Request schema:
```json
{
"target": "string",
"horizon": 2,
"relation_filters": ["string"],
"include_reasons": true
}
```
Response schema:
```json
{
"target": {"id": "string", "title": "string"},
"affected": [
{
"id": "string",
"title": "string",
"distance": 1,
"impact_score": 0.0,
"confidence": 0.0,
"reasons": ["string"],
"evidence_refs": ["memory://..."]
}
],
"summary": {"total_considered": 0, "total_returned": 0}
}
```
Deterministic fields: membership, distance, summary counts.
Probabilistic fields: `impact_score`, `confidence`.
Latency target: p95 <= 650ms for `horizon<=3`.
Scale envelope: default 200 results, hard cap 1000 with pagination token.
### 3) `GET /v2/projects/{project_id}/graph/health`
Purpose: report deterministic graph quality and actionable issues.
Query params:
1. `scope` optional directory prefix.
2. `timeframe` optional window like `30d`.
Response schema:
```json
{
"metrics": {
"orphan_rate": 0.0,
"stale_central_nodes": 0,
"overloaded_hubs": 0,
"contradiction_candidates": 0
},
"issues": [
{
"issue_type": "orphan|stale_central|overloaded_hub|contradiction_candidate",
"entity_id": "string",
"severity": "low|medium|high",
"reason": "string",
"suggested_action": "string",
"confidence": 0.0
}
],
"computed_at": "RFC3339"
}
```
Deterministic fields: `metrics`, issue membership for fixed snapshot.
Probabilistic fields: contradiction confidence when applicable.
Latency target: p95 <= 1500ms project-wide, <= 700ms scoped.
### 4) `POST /v2/projects/{project_id}/graph/reindex`
Purpose: force project-wide graph index rebuild.
Request schema:
```json
{
"mode": "full|incremental",
"reason": "string|null"
}
```
Response schema:
```json
{
"job_id": "string",
"status": "queued|running|completed|failed",
"scheduled_at": "RFC3339"
}
```
Deterministic fields: job metadata and status transitions.
Probabilistic fields: none.
Latency target: enqueue response p95 <= 120ms.
### 5) `POST /v2/projects/{project_id}/fcm/simulate`
Purpose: run FCM scenario simulation.
Request schema:
```json
{
"actions": [{"node_id": "string", "delta": 0.2}],
"scenario": {
"steps": 12,
"activation": "tanh|sigmoid|bounded_linear",
"decay": 0.05
},
"clamp_rules": [{"node_id": "string", "min": -1.0, "max": 1.0}]
}
```
Response schema:
```json
{
"baseline": [{"node_id": "string", "state": 0.0}],
"projected": [{"node_id": "string", "state": 0.0}],
"deltas": [{"node_id": "string", "delta": 0.0}],
"stability": {"converged": true, "iterations_used": 0, "residual": 0.0},
"confidence": 0.0,
"explanations": [{"node_id": "string", "top_influencers": [{"source": "string", "weight": 0.0}]}],
"evidence_refs": ["memory://..."]
}
```
Deterministic fields: baseline, projected, deltas, stability for fixed model and params.
Probabilistic fields: confidence.
Latency target: p95 <= 1000ms for <=500 nodes and <=5000 edges.
### 6) `POST /v2/projects/{project_id}/fcm/rank-actions`
Purpose: rank candidate interventions by expected outcome and risk.
Request schema:
```json
{
"goal": "string",
"constraints": {
"max_negative_impact": 0.25,
"required_tags": ["string"],
"disallowed_nodes": ["string"]
},
"top_k": 10
}
```
Response schema:
```json
{
"goal": {"node_id": "string", "label": "string"},
"recommendations": [
{
"action_node_id": "string",
"expected_goal_delta": 0.0,
"risk_penalty": 0.0,
"net_score": 0.0,
"confidence": 0.0,
"rationale": ["string"],
"evidence_refs": ["memory://..."]
}
]
}
```
Deterministic fields: candidate set and constraint compliance.
Probabilistic fields: expected delta, penalty, net score, confidence.
Latency target: p95 <= 1500ms for top-10 from <=100 candidates.
### 7) `POST /v2/projects/{project_id}/fcm/import`
Purpose: import FCM model from CSV-first contract.
Request schema:
```json
{
"source": "string",
"format": "csv_bundle_v1",
"merge_mode": "replace|upsert"
}
```
Response schema:
```json
{
"import_id": "string",
"nodes_loaded": 0,
"edges_loaded": 0,
"warnings": ["string"],
"errors": ["string"]
}
```
Deterministic fields: counts and validation diagnostics.
Probabilistic fields: none.
Latency target: p95 <= 2500ms for 10k edges import.
### 8) `POST /v2/projects/{project_id}/fcm/export`
Purpose: export FCM model for interoperability.
Request schema:
```json
{
"format": "csv_bundle_v1",
"selection": {
"scope": "all|tag|subgraph",
"tag": "string|null",
"seed_nodes": ["string"]
}
}
```
Response schema:
```json
{
"export_id": "string",
"format": "csv_bundle_v1",
"files": [{"name": "nodes.csv", "path": "string"}, {"name": "edges.csv", "path": "string"}],
"node_count": 0,
"edge_count": 0
}
```
Deterministic fields: file names and counts for fixed selection.
Probabilistic fields: none.
Latency target: p95 <= 1800ms for 50k edges export.
## Data Model and Storage Boundaries
1. SQLite is mandatory operational source for entities, relations, embeddings, and project metadata.
2. Oxigraph stores derived knowledge graph index only.
3. FCM state persists in app database with scenario artifacts and run history.
4. Graph index is rebuildable and disposable by design.
5. Markdown files remain canonical source of truth.
Implementation data boundaries:
1. Knowledge graph schema tracks descriptive nodes and typed edges plus provenance.
2. FCM schema tracks signed weighted causal edges and node states.
3. Provenance model requires `evidence_refs`, `confidence`, and `updated_at`.
4. Scenario model stores interventions, constraints, run parameters, and output deltas.
5. Interop schema starts with CSV-first Mental Modeler contract.
## Background Jobs and Index Lifecycle
Scheduler tasks to add:
1. `sync_graph_entity`
2. `sync_graph_project`
3. `reindex_graph_project`
Lifecycle rules:
1. Note writes, edits, moves, and deletes schedule graph-index sync tasks.
2. Scheduling pattern mirrors existing vector sync behavior.
3. On stale or missing graph index, request path serves via SQL fallback and schedules catch-up.
4. Reindex is idempotent and safe to rerun.
5. Index version metadata is tracked per project for staleness checks.
Operational behaviors:
1. Foreground requests never block on full reindex completion.
2. Background job failures surface in health endpoints with actionable status.
3. Reindex job can run incremental or full mode.
4. Phase 1 note: scheduler task names and reindex enqueue path are implemented; write/edit/move/delete sync hooks still need explicit wiring.
## MCP and CLI Surface
New MCP tools:
1. `graph_lineage`
2. `graph_impact`
3. `graph_health`
4. `fcm_simulate`
5. `fcm_rank_actions`
6. `fcm_import_model`
7. `fcm_export_model`
CLI passthrough additions:
1. `bm tool graph-lineage ...`
2. `bm tool graph-impact ...`
3. `bm tool graph-health ...`
4. `bm tool fcm-simulate ...`
5. `bm tool fcm-rank-actions ...`
6. `bm tool fcm-import-model ...`
7. `bm tool fcm-export-model ...`
Output conventions:
1. Default output is JSON for MCP and CLI.
2. MCP supports optional `output_format="text"` for human-readable summaries.
3. CLI remains JSON-first to keep agent integration deterministic.
## Test Strategy (fast loop + gates)
### Slice-by-slice loop
For each vertical slice, implement in this order:
1. API contract and schema.
2. Typed MCP client.
3. MCP tool passthrough.
4. CLI passthrough.
5. Focused tests for API/MCP/CLI.
Fast checks per slice:
1. Targeted `pytest` for changed API, MCP, and CLI modules.
2. `just fast-check`.
3. `just doctor`.
4. `just test-graph-intel` for graph/FCM-only iteration loop.
Milestone gates:
1. SQLite unit and integration pass first.
2. Selective Postgres parity tests for new graph and FCM contracts.
3. Full `just check` before merge.
### Required test cases and scenarios
1. Casual user impact preview before note edit.
2. Decision audit: lineage plus evidence references explain recommendation.
3. Graph health deterministic output for fixed snapshot.
4. Research flow: import model -> simulate -> rank -> export.
5. Sparse and contradictory graph input degrades gracefully.
6. Interop round-trip preserves node/edge counts and signed weights.
7. Local and cloud parity on contract semantics and error model.
8. Stale index fallback path returns valid response and schedules catch-up.
## Rollout and Feature Flagging
Rollout controls:
1. Gate graph and FCM endpoints behind config flags first.
2. Add entitlement enforcement after behavior and reliability stabilize.
3. Keep existing tools and endpoints fully backward compatible.
Suggested flags:
1. `feature_graph_intelligence_enabled`
2. `feature_fcm_enabled`
3. `feature_graph_oxigraph_provider_enabled`
4. `feature_graph_sql_fallback_enabled`
Rollout sequence:
1. Enable contract skeleton in dev.
2. Enable graph features for internal alpha users.
3. Enable Oxigraph provider with fallback-on by default.
4. Enable FCM import/simulate/rank/export for research alpha users.
5. Promote to Local+ beta when acceptance criteria are met.
## Risks and Mitigations
1. Risk: graph query complexity increases p95 latency.
Mitigation: strict query caps, fallback path, and performance budgets per endpoint.
2. Risk: stale index produces confusing outputs.
Mitigation: explicit staleness checks, SQL fallback, and background catch-up scheduling.
3. Risk: FCM recommendations appear opaque.
Mitigation: require evidence references, confidence fields, and deterministic simulation metadata.
4. Risk: local/cloud contract drift.
Mitigation: shared schemas, contract tests, and parity checks in CI gates.
5. Risk: integration surface grows faster than team can validate.
Mitigation: phase gates and vertical-slice completion before opening next phase.
## Improvement Backlog (Post-Phase 1)
1. Refactor `bm tool` graph/FCM commands into a dedicated CLI module to reduce `tool.py` size and improve maintainability.
2. Consolidate repeated MCP text-formatting helpers for graph/FCM outputs.
3. Replace deterministic placeholder graph behavior with SQL-backed lineage/impact/health implementations.
4. Add explicit config/entitlement enforcement for graph/FCM endpoints and tools.
5. Add performance telemetry and p95 reporting for graph and FCM routes.
6. Add parity and degradation tests for stale-index fallback and contradictory/sparse inputs.
## Acceptance Criteria
1. All required sections in this document are complete with no unresolved decisions.
2. API/interface contracts are implementation-ready with request, response, error, latency, and scale details.
3. Architecture alignment is explicit: FastAPI logic core, MCP/CLI facades, shared local/cloud contracts.
4. Delivery phases define concrete outputs and exit criteria.
5. Test strategy includes tight iteration loop and milestone gates.
6. Required scenario matrix is covered in test plan and mapped to implementation phases.
7. Rollout plan includes feature flags and backward compatibility guarantees.
8. An implementer can execute this plan without additional architecture clarification.
## Assumptions and Defaults
1. Config-flag gating first; entitlement wiring later.
2. Graph-first vertical slices before deep FCM expansion.
3. Atomic tools first; orchestration layer deferred.
4. JSON-first contracts for agent usability.
5. No breaking changes to existing `build_context` and `search_notes`.
## Out of Scope
1. Implementation details unrelated to graph/FCM delivery phases in this document.
2. Migration execution.
3. Pricing and positioning rewrites.
4. Cloud infrastructure changes in this phase.
@@ -0,0 +1,782 @@
# SPEC-LOCAL-GRAPH-INTELLIGENCE-MASTER: Local+ Graph Intelligence Blueprint
**Status:** Draft (Iteration 2, Decision-Complete)
**Date:** 2026-03-05
**Owner:** Basic Memory
**Primary Audience:** Internal build team (Product, Engineering, GTM)
**Current Phase (2026-03-05):** Implementation Plan Phase 1 is complete; Phase 2 (SQL-backed graph logic) is the active engineering phase.
**Related Specs:**
1. `/docs/specs/SPEC-LOCAL-GRAPH-INTELLIGENCE.md`
2. `/docs/specs/SPEC-LOCAL-GRAPH-INTELLIGENCE-TECHNICAL-ADDENDUM.md`
3. `/docs/specs/SPEC-LOCAL-GRAPH-INTELLIGENCE-IMPLEMENTATION-PLAN.md`
Reading guide:
1. Sections 1-5 define the business and product decisions.
2. Sections 6-10 define architecture and interface contracts.
3. Sections 11-14 define pricing, rollout, and decision gates for execution.
## 1) Executive Thesis
Basic Memory will ship **Local+ Graph Intelligence** as a premium local capability that upgrades the product from retrieval to decision support.
Positioning statement:
1. "Keep your local workflow. Add decision intelligence as complexity grows."
2. The product sells safer decisions and explainable recommendations, not graph database mechanics.
Locked thesis decisions:
1. SQLite will remain the operational core.
2. Markdown will remain source of truth.
3. Graph and FCM indexes will be derived and rebuildable.
4. Oxigraph/pyoxigraph will be the v1 graph index path.
5. FCM simulation will run in a Python service layer.
6. SurrealDB and FalkorDB will not be core dependencies in v1 due license-roadmap mismatch.
7. Product messaging will sell outcomes (safer decisions, explainable recommendations), not database internals.
## 2) Problem and Opportunity
Current state after v0.19:
1. Recursive SQL traversal can retrieve connected notes but becomes expensive and noisy after a few hops.
2. Users still do manual synthesis for impact analysis, decision lineage, and contradiction resolution.
3. Researchers need causal reasoning and scenario modeling, not only graph navigation.
Opportunity:
1. Deliver a premium local tier that materially improves decision quality while keeping data local.
2. Create a bridge from knowledge graph navigation to causal simulation (FCM).
3. Open a research-heavy market segment that values explainability and model interoperability.
Business opportunity:
1. Add a middle tier between free OSS and cloud subscription.
2. Preserve an upgrade path to hosted collaboration for research teams later.
3. Differentiate Basic Memory for research-grade workflows without forcing cloud adoption.
## 3) User Segments and Jobs-to-be-Done
| Segment | Primary Job-to-be-Done | Pain Today | Value Trigger |
|---|---|---|---|
| Casual local builder | Avoid breaking related notes when editing | Hidden dependencies and rework | Impact preview before edits |
| Solo technical founder | Keep architecture and decision context coherent | Context overload and drift | Decision lineage + impact radius |
| Research user | Model and test intervention strategies | No integrated causal simulation with notes | FCM simulation + action ranking |
| Product/research lead | Synthesize evidence quickly across many docs | Fragmented understanding | Path exploration + priority briefs |
## 4) Product Outcomes (not feature list)
Local+ Graph Intelligence will optimize for these outcomes:
1. **Change Safety:** users catch downstream impacts before they edit.
2. **Decision Clarity:** users can explain why an answer or recommendation was produced.
3. **Knowledge Health:** users keep larger graphs coherent with less manual audit work.
4. **Research Leverage:** users run scenario-level reasoning tied to explicit evidence.
Outcome metrics (for 30-day retained Local+ cohorts):
1. Median time-to-understanding for complex topics decreases by at least 35% for active Local+ users.
2. User-reported surprise side effects after note edits decrease by at least 30%.
3. At least 60% of active Local+ users invoke graph intelligence features weekly.
4. At least 40% of research-profile Local+ users invoke one FCM workflow weekly.
## 5) Feature Set v1/v1.5/v2
### v1 (post-v0.19 launch scope)
Included:
1. Decision Lineage
2. Impact Radius
3. Path Explorer (guided)
4. Graph Health (orphans, stale-central nodes, overloaded hubs)
5. CSV FCM import/export (nodes and edges)
6. FCM simulation for explicit action scenarios
7. FCM action ranking with evidence-linked rationale
Excluded:
1. Native Mental Modeler project format write support
2. Team governance and shared model policy controls
3. Cloud-only enhancements
### v1.5
Included:
1. Contradiction Watch with reconciliation queue
2. Priority Briefs (graph + FCM leverage summary)
3. Stronger uncertainty propagation in FCM scoring
4. Cloud execution optionality for heavy simulation jobs
### v2
Included:
1. Team-shared model governance
2. Hosted collaboration features for research teams
3. Optional native model translators beyond CSV baseline
Cut line policy:
1. If a capability cannot meet explainability requirements, it moves to v1.5+.
2. If a capability requires cloud to function, it cannot be marked v1.
3. If a capability cannot meet local performance envelopes, it cannot be promoted into default workflows.
## 6) Technical Architecture (Two-Graph Model)
### High-level architecture
```mermaid
flowchart LR
A[Markdown Files Source of Truth] --> B[Parser + Sync Pipeline]
B --> C[SQLite Operational Store]
B --> D[Derived Knowledge Graph Index Oxigraph]
C --> E[Graph Intelligence Service]
D --> E
E --> F[Lineage Impact Path Health APIs]
C --> G[FCM Service Python]
D --> G
G --> H[Simulation Ranking Interop APIs]
```
### Two-graph model
1. **Knowledge Graph (descriptive):** notes, decisions, concepts, and typed relations.
2. **FCM Graph (causal):** signed weighted influence links between goals, drivers, risks, and interventions.
### Core architectural decisions
1. SQLite is authoritative for entities, observations, relations, metadata, embeddings, and project state.
2. Oxigraph is a derived index for multi-hop graph traversal and graph-pattern retrieval.
3. FCM calculations run in Python using explicit model state and deterministic numerical steps.
4. Local mode runs fully offline.
5. Cloud mode can execute the same contracts via adjunct services while Neon remains system of record.
## 7) Backend Decision and Trade-Offs
### Final recommendation
Use this stack for v1:
1. SQLite (existing): primary operational store.
2. Oxigraph/pyoxigraph: derived knowledge graph index.
3. Python FCM service: causal simulation and ranking.
Decision rationale:
1. This preserves local-first UX while enabling deeper traversal and causal simulation.
2. This avoids restrictive licensing dependencies in the core product path.
3. This keeps a clean cloud portability path where Neon remains the hosted system of record.
### Trade-off matrix
| Option | Strengths | Risks | Decision |
|---|---|---|---|
| SQLite + Oxigraph + Python FCM | Local-first, permissive licensing, clear service boundaries, cloud-portable | Requires translation layer for query ergonomics | **Adopt v1** |
| Apache AGE on Postgres | SQL+graph in one engine, good cloud-side graph semantics | Neon extension support uncertainty, weaker local/cloud parity with SQLite local baseline | Defer |
| SurrealDB | Strong integrated multi-model experience | BSL posture conflicts with future hosted/open strategy timing | Reject for v1 core |
| FalkorDB | Graph performance and Redis ecosystem familiarity | SSPL posture conflicts with hosted/open strategy | Reject for v1 core |
## 8) Public APIs / Interfaces
All APIs are proposed MCP tool contracts for Local+ mode.
### Common conventions
1. `project` parameter is optional and follows existing Basic Memory project routing.
2. Deterministic fields are reproducible with identical inputs and index state.
3. Probabilistic fields are model-derived scores and include confidence metadata.
4. Error model uses structured codes and fail-fast behavior.
Shared error codes:
1. `INVALID_ARGUMENT`
2. `NOT_FOUND`
3. `MODEL_INVALID`
4. `INDEX_NOT_READY`
5. `RESOURCE_LIMIT_EXCEEDED`
6. `INTERNAL_ERROR`
---
### 8.1 `graph_lineage(start, goal?)`
**Input schema:**
```json
{
"start": "string (required, permalink or memory URL)",
"goal": "string (optional, concept or decision target)",
"max_hops": "integer (optional, default 4, range 1-6)",
"relation_filters": ["string"],
"project": "string (optional)"
}
```
**Output schema:**
```json
{
"root": {"id": "string", "title": "string", "permalink": "string"},
"paths": [
{
"path_id": "string",
"nodes": [{"id": "string", "title": "string"}],
"edges": [{"relation": "string", "direction": "outgoing|incoming"}],
"deterministic_path_score": 0.0,
"confidence": 0.0,
"evidence_refs": ["memory://..."]
}
],
"generated_at": "RFC3339"
}
```
**Deterministic fields:** root, nodes, edges, deterministic path score, generated timestamp.
**Probabilistic fields:** confidence.
**Latency target:** p95 <= 450ms for `max_hops<=4`, graph envelope up to 50k nodes / 300k edges.
**Scale envelope:**
1. Tested local baseline: 50k nodes, 300k edges.
2. Expected degradation: path expansion can exceed latency target when candidate paths > 20k.
---
### 8.2 `graph_impact(target, horizon, relation_filters?)`
**Input schema:**
```json
{
"target": "string (required)",
"horizon": "integer (required, range 1-4)",
"relation_filters": ["string"],
"include_reasons": "boolean (default true)",
"project": "string (optional)"
}
```
**Output schema:**
```json
{
"target": {"id": "string", "title": "string"},
"affected": [
{
"id": "string",
"title": "string",
"distance": 2,
"impact_score": 0.0,
"confidence": 0.0,
"reasons": ["string"]
}
],
"summary": {"total_considered": 0, "total_returned": 0}
}
```
**Deterministic fields:** membership, distance, summary counts.
**Probabilistic fields:** impact score, confidence.
**Latency target:** p95 <= 650ms for `horizon<=3` under baseline envelope.
**Scale envelope:**
1. `affected` default cap: 200 items.
2. Hard cap: 1000 items with pagination token.
---
### 8.3 `graph_health(scope?, timeframe?)`
**Input schema:**
```json
{
"scope": "string (optional, directory prefix or project-wide)",
"timeframe": "string (optional, e.g. 30d, 90d)",
"project": "string (optional)"
}
```
**Output schema:**
```json
{
"metrics": {
"orphan_rate": 0.0,
"stale_central_nodes": 0,
"overloaded_hubs": 0,
"contradiction_candidates": 0
},
"issues": [
{
"issue_type": "orphan|stale_central|overloaded_hub|contradiction_candidate",
"entity_id": "string",
"severity": "low|medium|high",
"reason": "string",
"suggested_action": "string"
}
],
"computed_at": "RFC3339"
}
```
**Deterministic fields:** metrics and issue list membership for a fixed graph snapshot.
**Probabilistic fields:** contradiction candidate confidence when present.
**Latency target:** p95 <= 1500ms project-wide; <= 700ms for scoped directory mode.
**Scale envelope:** project-wide scans tested to 50k nodes.
---
### 8.4 `fcm_simulate(actions, scenario?, clamp_rules?)`
**Input schema:**
```json
{
"actions": [
{"node_id": "string", "delta": 0.2}
],
"scenario": {
"steps": 12,
"activation": "tanh|sigmoid|bounded_linear",
"decay": 0.05
},
"clamp_rules": [
{"node_id": "string", "min": -1.0, "max": 1.0}
],
"project": "string (optional)"
}
```
**Output schema:**
```json
{
"baseline": [{"node_id": "string", "state": 0.12}],
"projected": [{"node_id": "string", "state": 0.43}],
"deltas": [{"node_id": "string", "delta": 0.31}],
"stability": {
"converged": true,
"iterations_used": 9,
"residual": 0.002
},
"confidence": 0.0,
"explanations": [
{"node_id": "string", "top_influencers": [{"source": "string", "weight": 0.7}]}
]
}
```
**Deterministic fields:** baseline, projected, deltas, convergence metadata for fixed model and parameters.
**Probabilistic fields:** confidence (derived from edge confidence and evidence coverage).
**Latency target:** p95 <= 1000ms for up to 500 nodes / 5000 edges and <=12 steps.
**Scale envelope:**
1. Soft limit: 2000 nodes / 20000 edges.
2. Over soft limit: return `RESOURCE_LIMIT_EXCEEDED` with remediation guidance.
---
### 8.5 `fcm_rank_actions(goal, constraints?, top_k?)`
**Input schema:**
```json
{
"goal": "string (required node_id)",
"constraints": {
"max_negative_impact": 0.25,
"required_tags": ["string"],
"disallowed_nodes": ["string"]
},
"top_k": "integer (default 10, range 1-25)",
"project": "string (optional)"
}
```
**Output schema:**
```json
{
"goal": {"node_id": "string", "label": "string"},
"recommendations": [
{
"action_node_id": "string",
"expected_goal_delta": 0.0,
"risk_penalty": 0.0,
"net_score": 0.0,
"confidence": 0.0,
"rationale": ["string"],
"evidence_refs": ["memory://..."]
}
]
}
```
**Deterministic fields:** candidate action set, constraints compliance.
**Probabilistic fields:** expected goal delta, risk penalty, net score, confidence.
**Latency target:** p95 <= 1500ms for top 10 from up to 100 candidate actions.
**Scale envelope:**
1. Candidate actions hard cap: 1000.
2. For larger sets, require pre-filtering via tags/scope.
---
### 8.6 `fcm_import_model(source, format)`
**Input schema:**
```json
{
"source": "string (required path or URI)",
"format": "csv_bundle_v1 (required)",
"merge_mode": "replace|upsert (default upsert)",
"project": "string (optional)"
}
```
**Output schema:**
```json
{
"import_id": "string",
"nodes_loaded": 0,
"edges_loaded": 0,
"warnings": ["string"],
"errors": ["string"]
}
```
**Deterministic fields:** load counts and validation results.
**Probabilistic fields:** none.
**Latency target:** p95 <= 2500ms for 10k edges CSV bundle.
**Scale envelope:**
1. Maximum CSV rows per import: 250k.
2. Above limit returns `RESOURCE_LIMIT_EXCEEDED`.
---
### 8.7 `fcm_export_model(format, selection?)`
**Input schema:**
```json
{
"format": "csv_bundle_v1 (required)",
"selection": {
"scope": "all|tag|subgraph",
"tag": "string (optional)",
"seed_nodes": ["string"]
},
"project": "string (optional)"
}
```
**Output schema:**
```json
{
"export_id": "string",
"format": "csv_bundle_v1",
"files": [
{"name": "nodes.csv", "path": "string"},
{"name": "edges.csv", "path": "string"}
],
"node_count": 0,
"edge_count": 0
}
```
**Deterministic fields:** file set and row counts for fixed selection.
**Probabilistic fields:** none.
**Latency target:** p95 <= 1800ms for 50k edges export.
**Scale envelope:**
1. Max export rows: 500k total.
2. Pagination or scoped export required above cap.
## 9) Data Model and Storage Boundaries
### 9.1 Knowledge graph schema (descriptive)
`KnowledgeNode`:
1. `id: str`
2. `kind: note|decision|spec|concept|person|project`
3. `title: str`
4. `permalink: str`
5. `tags: list[str]`
6. `updated_at: datetime`
`KnowledgeEdge`:
1. `id: str`
2. `src_id: str`
3. `dst_id: str`
4. `relation: str`
5. `directionality: directed|bidirectional`
6. `evidence_refs: list[str]`
7. `confidence: float [0,1]`
8. `updated_at: datetime`
### 9.2 FCM schema (causal signed weighted)
`FCMNode`:
1. `id: str`
2. `label: str`
3. `node_type: goal|driver|risk|intervention|context`
4. `state: float [-1,1]`
5. `clamp_min: float`
6. `clamp_max: float`
7. `metadata: map`
`FCMEdge`:
1. `id: str`
2. `source_id: str`
3. `target_id: str`
4. `weight: float [-1,1]`
5. `confidence: float [0,1]`
6. `time_decay: float [0,1]`
7. `evidence_refs: list[str]`
8. `updated_at: datetime`
### 9.3 Provenance model
`ProvenanceRecord`:
1. `entity_id: str`
2. `evidence_refs: list[str]`
3. `confidence: float [0,1]`
4. `updated_at: datetime`
5. `source_type: extracted|user_authored|imported`
### 9.4 Scenario model
`Scenario`:
1. `id: str`
2. `name: str`
3. `interventions: list[{node_id, delta}]`
4. `constraints: list[{node_id, min, max}]`
5. `steps: int`
6. `activation: tanh|sigmoid|bounded_linear`
7. `created_at: datetime`
8. `created_by: str`
`ScenarioResult`:
1. `scenario_id: str`
2. `converged: bool`
3. `iterations_used: int`
4. `residual: float`
5. `goal_deltas: list[{node_id, delta}]`
6. `confidence: float [0,1]`
### 9.5 Storage boundaries
| Layer | System of Record | Purpose | Rebuildable |
|---|---|---|---|
| Markdown files | File system | Canonical knowledge content | No |
| SQLite entities/relations/embeddings | SQLite | Operational queries and project state | Yes (from markdown + embedding pipeline) |
| Knowledge graph triples | Oxigraph | Fast graph traversal and pattern queries | Yes |
| FCM model and snapshots | SQLite + optional artifacts | Causal model state and scenario history | Yes (from imports and authored model definitions) |
## 10) Mental Modeler Interoperability
### v1 interoperability contract
Format: `csv_bundle_v1`
1. `nodes.csv`
2. `edges.csv`
3. Optional `scenarios.csv`
`nodes.csv` required columns:
1. `node_id`
2. `label`
3. `node_type`
4. `state`
5. `clamp_min`
6. `clamp_max`
`edges.csv` required columns:
1. `edge_id`
2. `source_id`
3. `target_id`
4. `weight`
5. `confidence`
6. `evidence_refs` (semicolon-delimited)
### Import rules
1. Missing required columns fail with `MODEL_INVALID`.
2. Unknown node types fail fast.
3. Weight and confidence ranges are strictly validated.
4. Import returns warnings for dangling evidence references.
### Export rules
1. Preserve stable IDs for round-trip compatibility.
2. Preserve signed weights exactly.
3. Preserve confidence values exactly.
4. Non-portable metadata is emitted to `metadata.json` sidecar when present.
### Native file translators
1. Native project-format translation is deferred to v2.
2. CSV remains the guaranteed compatibility baseline in v1 and v1.5.
## 11) Pricing and Packaging
### Tier structure
| Tier | Price Monthly | Price Annual | Beta Price (25% off) | Target Persona | Core Value |
|---|---:|---:|---:|---|---|
| OSS Local | $0 | $0 | $0 | Casual local users | Retrieval and memory basics |
| Local+ Graph Intelligence | $9 | $90 | $6.75 monthly / $67.50 annual | Founders, consultants, researchers | Safer changes + explainable graph + FCM simulation |
| Cloud Pro (current anchor) | $19 | $190 | $14.25 monthly / $142.50 annual | Users who need hosted sync and cloud workflows | Managed cloud + sync + collaboration path |
Pricing principles:
1. Local+ is intentionally priced between free OSS and cloud to capture users who need deeper intelligence but not hosted sync.
2. Cloud Pro remains the hosted convenience anchor and future collaboration path.
3. Local+ must stand on standalone local value and cannot depend on cloud features.
### Feature gate mapping
| Capability | OSS Local | Local+ | Cloud Pro |
|---|---|---|---|
| Search and basic context tools | Yes | Yes | Yes |
| Decision Lineage | No | Yes | Yes |
| Impact Radius | No | Yes | Yes |
| Graph Health | No | Yes | Yes |
| FCM simulate + rank | No | Yes | Yes |
| CSV model import/export | No | Yes | Yes |
| Hosted collaboration controls | No | No | Future add-on |
### Packaging decisions
1. Local+ remains fully local-capable and does not require cloud auth to run.
2. Cloud Pro remains the hosted convenience and collaboration anchor.
3. Future hosted research add-on will layer on Cloud Pro after v2 readiness.
## 12) Rollout Strategy
### 12.1 Document production iterations (locked)
Iteration 1 (draft complete):
1. Complete all 15 sections in one pass.
2. Include v1/v1.5/v2 cut lines.
3. Include pricing and scenario definitions.
4. Ensure no placeholders.
Iteration 2 (hardening and decision lock):
1. Resolve cross-section contradictions.
2. Convert uncertain language to locked decisions.
3. Add measurable acceptance criteria and risk owners.
4. Finalize execution-ready API contracts.
### 12.2 Product rollout phases
Phase A: Foundation release (v1)
1. Graph lineage, impact, health.
2. CSV model import/export.
3. FCM simulation and ranking.
4. Advanced mode UX gating for research-grade controls.
Phase B: Quality and confidence (v1.5)
1. Contradiction Watch.
2. Priority Briefs.
3. Improved uncertainty propagation.
4. Confidence calibration pass using real-world model feedback.
Phase C: Team expansion (v2)
1. Hosted team governance.
2. Shared model controls.
3. Extended translator support.
### 12.3 Go/No-Go release gates
Gate to ship v1 default workflows:
1. p95 latency targets are met within the declared scale envelopes.
2. Scenario round-trip fidelity tests pass for CSV import/export.
3. Every recommendation and simulation path exposes evidence references and confidence.
Gate to ship v1.5:
1. Contradiction Watch precision is acceptable for default-on use.
2. Confidence calibration reduces false-confidence reports in user testing.
Gate to ship v2 team features:
1. Clear willingness-to-pay signal from team and research buyers.
2. Cloud execution path preserves local-cloud semantic parity for core contracts.
## 13) Risks, Counterarguments, and Mitigations
| Risk | Counterargument | Severity | Likelihood | Mitigation | Owner |
|---|---|---|---|---|---|
| "This is just better search" | Positioning can collapse into technical jargon | High | Medium | Lead with decision safety and explainability outcomes in product copy and onboarding | Product Lead |
| FCM feels opaque or invented | Users distrust black-box scoring | High | Medium | Require evidence refs and confidence disclosure on every recommendation | Applied AI Lead |
| Local performance regressions | Multi-hop and simulation can feel slow on laptops | Medium | Medium | Enforce envelopes, caps, and fail-fast limit errors with guidance | Engineering Lead |
| Research features overwhelm casual users | UX complexity can reduce adoption | Medium | High | Default to guided flows and hide advanced controls behind explicit advanced mode | Design Lead |
| License/roadmap conflict if backend changes | Later swap to restrictive engines creates GTM risk | High | Low | Lock permissive v1 stack and require leadership sign-off for any license-restricted dependency | Product + Legal |
| Interop mismatch with external tools | Round-trip drift harms trust with researchers | Medium | Medium | Validate node and edge parity in import/export tests and version interop schema | Integrations Lead |
| Pricing confusion between Local+ and Cloud Pro | Buyers may not understand which tier fits | Medium | Medium | Publish explicit tier comparison focused on local intelligence vs hosted collaboration | GTM Lead |
## 14) Acceptance Criteria
### 14.1 Document acceptance criteria
1. Product, technical, and pricing decisions are explicit and unambiguous.
2. API contracts include input/output schemas, error models, deterministic versus probabilistic fields, and performance envelopes.
3. v1/v1.5/v2 cut lines are explicit and consistent.
4. Risk register includes severity, likelihood, owner, and mitigation.
5. Document can be handed to implementation without additional architecture decisions.
6. Leadership can use this document directly for pricing and positioning decisions.
### 14.2 Product acceptance criteria for v1 delivery
1. `graph_lineage`, `graph_impact`, `graph_health`, `fcm_simulate`, `fcm_rank_actions`, `fcm_import_model`, and `fcm_export_model` are available as Local+ contracts.
2. Local mode executes all v1 contracts without cloud dependency.
3. API p95 latency targets are met within defined scale envelopes.
4. Every ranked or simulated output includes evidence-linked rationale and confidence.
5. CSV round-trip preserves node count, edge count, and signed weights exactly.
### 14.3 Scenario test matrix (required)
1. **Casual local user impact check**
Expected pass:
`graph_impact` returns ranked affected notes with reasons before a note edit.
2. **Research workflow simulation**
Expected pass:
User imports a model bundle, runs `fcm_simulate`, and receives converged deltas and rationale.
3. **Decision audit traceability**
Expected pass:
`graph_lineage` returns path and evidence references that explain recommendation origin.
4. **Cloud and local parity**
Expected pass:
Identical query inputs return semantically equivalent outputs in local and cloud modes with local fallback behavior when cloud is unavailable.
5. **Sparse or contradictory graph behavior**
Expected pass:
System degrades gracefully with explicit uncertainty and does not fabricate high-confidence recommendations.
6. **Interop round-trip fidelity**
Expected pass:
`fcm_export_model` then `fcm_import_model` preserves node and edge counts and signed weights without mutation.
## 15) Appendix (license notes, terminology, examples)
### 15.1 License notes (verified 2026-03-05)
1. SurrealDB core licensing is published under BSL 1.1 with DBaaS-related restrictions in its conversion window.
2. FalkorDB is published under SSPLv1.
3. pyoxigraph is dual-licensed Apache-2.0 or MIT.
4. Neon extension catalog currently does not list Apache AGE as a supported extension.
These notes support the v1 dependency decisions in this document.
### 15.2 Terminology
1. **Knowledge graph:** descriptive relation graph derived from markdown knowledge.
2. **FCM:** fuzzy cognitive model with signed weighted causal edges.
3. **Deterministic field:** reproducible output field from fixed input and fixed model/index snapshot.
4. **Probabilistic field:** score influenced by confidence weights and model uncertainty.
### 15.3 Assumptions and defaults
1. Markdown remains source of truth.
2. SQLite remains mandatory baseline.
3. Graph and FCM capabilities are premium Local+ features, not OSS defaults.
4. Research-heavy features are advanced mode, while mainstream UX stays guided.
5. Mental Modeler interoperability starts with CSV contract first; native translator support is deferred.
### 15.4 Out of scope for this document
1. Implementation code changes.
2. Database migrations.
3. Cloud infrastructure edits.
4. Full instrumentation pilot plan as the primary artifact.
### 15.5 External references
1. [SurrealDB licensing](https://surrealdb.com/license)
2. [FalkorDB licensing](https://docs.falkordb.com/References/license.html)
3. [pyoxigraph package and license](https://pypi.org/project/pyoxigraph/)
4. [Neon Postgres extension catalog](https://neon.com/docs/extensions/pg-extensions)
@@ -0,0 +1,299 @@
# SPEC-LOCAL-GRAPH-INTELLIGENCE: Technical Addendum (Graph + FCM)
**Status:** Draft
**Date:** 2026-03-05
**Owner:** Basic Memory
**Current Phase (2026-03-05):** Contract skeleton implementation is complete; next active phase is SQL-backed graph capabilities.
Related product spec:
`/docs/specs/SPEC-LOCAL-GRAPH-INTELLIGENCE.md`
Related execution spec:
`/docs/specs/SPEC-LOCAL-GRAPH-INTELLIGENCE-IMPLEMENTATION-PLAN.md`
## Why This Addendum Exists
The product spec defines user value. This addendum defines the technical shape that can deliver that value without
breaking local-first principles.
This addendum also introduces a second graph layer:
1. Knowledge graph for relationships between notes, entities, and decisions.
2. Fuzzy Cognitive Model (FCM) graph for weighted causal reasoning over actions and outcomes.
Both are derived from markdown and optional user-provided models.
## Strategic Reality Check
This is a strong idea if we stage it correctly.
It is not a pipe dream if we avoid one trap: building a big "graph platform" before proving users repeatedly use
decision simulation workflows.
The correct strategy is:
1. Launch high-precision graph insights first.
2. Add FCM scoring where it changes user behavior (not as a novelty dashboard).
3. Expand to hosted/team workflows only after local usage proves repeat value.
## Constraints and Design Principles
1. SQLite remains the operational source for entities, observations, relations, and embeddings.
2. Markdown remains source of truth.
3. Graph indexes are derived, rebuildable, and disposable.
4. Premium local mode must run fully offline.
5. Cloud deployment should support both single-tenant and SaaS later.
6. Avoid licenses that constrain hosted/open-source strategy.
## Backend Recommendation
### Primary Recommendation
Use a dual-store architecture:
1. SQLite (existing): operational data, metadata filters, embeddings, and most retrieval.
2. Oxigraph/pyoxigraph (new): derived graph index for graph traversal and graph-pattern queries.
3. Python simulation layer (new): FCM state propagation, scenario runs, and decision scoring.
Why this is the best fit:
1. Permissive licensing profile.
2. Works locally with low footprint.
3. Cloud-compatible as sidecar service while keeping Neon Postgres as core cloud store.
4. Clear boundary between graph query and numeric simulation concerns.
### Candidate Trade-Offs
#### Oxigraph/pyoxigraph
Pros:
1. Lightweight local embedding.
2. Good fit for derived-index strategy.
3. Strong path for standards-based graph representation.
Cons:
1. SPARQL fluency is less common than SQL/Cypher.
2. Requires a translation layer so product features are not query-language-coupled.
#### Apache AGE (Postgres extension)
Pros:
1. SQL + graph in one engine.
2. Attractive for cloud-side graph operations.
Cons:
1. Neon support is uncertain for this extension.
2. Local/cloud parity is harder if local uses SQLite.
#### SurrealDB / FalkorDB
Pros:
1. Strong graph-oriented developer experience.
Cons:
1. License posture is misaligned with a future hosted/open-source roadmap unless commercial terms are accepted.
Decision:
Do not make these core dependencies for v1 of Local+ Graph Intelligence.
## Two-Graph Model
### A) Knowledge Graph (Descriptive)
Node examples:
1. Note
2. Decision
3. Spec
4. Person
5. Project
6. Concept
Edge examples:
1. `depends_on`
2. `informed_by`
3. `contradicts`
4. `supports`
5. `implements`
6. `derived_from`
Purpose:
Power navigation, lineage, path explanation, impact radius, and health checks.
### B) FCM Graph (Causal, Signed, Weighted)
Node examples:
1. Goal: "Reduce regressions"
2. Driver: "Test coverage"
3. Risk: "Scope creep"
4. Intervention: "Add review gate"
5. Context variable: "Team bandwidth"
Edge attributes:
1. `weight` in [-1.0, 1.0]
2. `confidence` in [0.0, 1.0]
3. `evidence_refs` (links to notes/specs)
4. `time_decay` (optional)
Purpose:
Power scenario simulation and action ranking, not generic retrieval.
## Premium Feature Mapping to Architecture
### Decision Lineage
Backed by:
1. Knowledge graph path queries.
2. Evidence references stored on edges.
### Impact Radius
Backed by:
1. Multi-hop neighborhood expansion with relation-type weights.
2. Risk ranking using centrality + recency + confidence.
### Contradiction Watch
Backed by:
1. Candidate contradiction edges.
2. Confidence-scored reconciliation queue.
### Priority Briefs
Backed by:
1. Health metrics (orphan rate, stale-central nodes, unresolved contradictions).
2. Optional FCM "top leverage actions" summary.
### New Premium Feature: Action Simulator
Backed by:
1. FCM scenario runs over selected action nodes.
2. Ranked interventions with expected positive/negative downstream effects.
3. Explicit rationale graph for every recommendation.
## Mental Modeler Interop Plan
Goal:
Make Basic Memory the AI-enabled operating layer around existing researcher workflows, not a replacement for their tools.
Interoperability phases:
1. Import/export edge lists and node tables via CSV as the baseline interchange.
2. Preserve concept IDs and metadata so round-trips remain stable.
3. Add translator support for native model files if/when schema contracts are validated with partner data.
Validation requirement:
1. Round-trip tests must preserve node count, edge count, and signed weights.
2. Confidence/evidence metadata may be Basic Memory extensions and should degrade gracefully when exported.
## Suggested Tool/API Surface (Product-Facing)
1. `graph_lineage(start, goal?)`
Returns explainable evidence paths.
2. `graph_impact(target, horizon=2..4)`
Returns ranked affected nodes with reasons.
3. `graph_health()`
Returns actionable graph quality issues.
4. `fcm_simulate(actions, scenario?)`
Returns projected effects and uncertainty.
5. `fcm_rank_actions(goal, constraints?)`
Returns top candidate actions with trade-offs.
6. `fcm_import_model(source)` / `fcm_export_model(format)`
Handles interop with external cognitive mapping workflows.
## Local and Cloud Deployment Shape
### Local (Primary)
1. SQLite + local embeddings.
2. Oxigraph as local sidecar/index library.
3. FCM simulation in process.
### Cloud (Future-Compatible)
1. Neon Postgres remains system of record in hosted mode.
2. Graph index service runs per tenant or shared multi-tenant with strict tenancy boundaries.
3. FCM simulation service can run stateless workers reading graph snapshots.
Principle:
Do not require cloud to run premium local features.
## Rollout Plan With Go/No-Go Gates
### Phase 0: Proof of Utility (4-6 weeks)
Deliver:
1. Decision Lineage
2. Impact Radius
3. CSV FCM import + `fcm_simulate` prototype
Gate to continue:
1. Repeated weekly usage by pilot users.
2. Users report changed decisions, not just curiosity clicks.
### Phase 1: Productized Local+ Beta
Deliver:
1. Graph health workflow
2. Contradiction Watch
3. Action ranking with explicit rationale
Gate to continue:
1. Retention of graph features after first month.
2. Measured reduction in "surprise side effects" after edits.
### Phase 2: Hosted Expansion
Deliver:
1. Optional cloud execution for heavy simulations.
2. Team-shared model governance.
Gate to continue:
1. Clear willingness to pay for hosted collaboration.
## Risks and Mitigations
Risk: FCM outputs feel "made up."
Mitigation: Require evidence links and confidence scoring in every recommendation.
Risk: Research-heavy feature alienates casual users.
Mitigation: Keep FCM features in an advanced mode; default to concise guidance workflows.
Risk: Overengineering early graph stack.
Mitigation: Keep derived-index architecture and strict phase gates tied to behavior change.
Risk: Interop friction with external tooling.
Mitigation: Start with transparent CSV contract and strict round-trip validation.
## Candid Recommendation
Pursue this. It is a high-upside differentiation path for Local+ if executed with staged validation.
The key is to sell outcomes:
1. "Safer decisions"
2. "Explainable recommendations"
3. "Faster synthesis for complex research"
Avoid selling "graph DB" as the product. That is implementation detail.
+262
View File
@@ -0,0 +1,262 @@
# SPEC-LOCAL-GRAPH-INTELLIGENCE: Premium Local Graph Intelligence
**Status:** Draft
**Date:** 2026-03-05
**Owner:** Basic Memory
**Current Phase (2026-03-05):** Phase 1 contract foundation shipped; engineering is now executing SQL-backed Phase 2 graph logic.
Companion technical addendum:
`/docs/specs/SPEC-LOCAL-GRAPH-INTELLIGENCE-TECHNICAL-ADDENDUM.md`
## Summary
Add a premium local feature that turns Basic Memory from "search and recall" into "explain and guide."
The value is not a new database. The value is better decisions for local users:
1. Understand why something matters.
2. See what will be affected before making a change.
3. Detect weak spots in the knowledge base early.
4. Navigate complex knowledge intentionally instead of loading everything.
This feature is additive. Existing local workflows remain intact.
## Positioning
Core message:
"Your notes do more than store knowledge. They reveal consequences, lineage, and blind spots."
Local user promise:
1. Keep files local.
2. Keep markdown as source of truth.
3. Get advanced graph intelligence as an opt-in premium capability.
## Problem
Today, deep graph navigation is possible but often expensive in context size and hard to steer for complex questions.
Users can find information, but they still do manual synthesis to answer:
1. What changed because of this note?
2. Why did we decide this?
3. What might break if I update this?
4. Which parts of the graph are stale, isolated, or contradictory?
The cost is time, cognitive load, and missed risk.
## Goals
1. Provide clear, explainable graph insights that users can act on.
2. Make deep navigation feel guided, not overwhelming.
3. Help users prevent mistakes before they happen.
4. Create premium local value that is easy to understand and justify.
5. Keep feature behavior transparent and trustworthy.
## Non-Goals
1. Replacing SQLite as the primary operational store.
2. Changing markdown as source of truth.
3. Forcing users to learn graph query languages.
4. Building a cloud-only feature set.
5. Turning Basic Memory into an enterprise BI product.
## Product Frame: From Retrieval to Reasoning
The feature should be framed as a shift in user outcome:
1. Retrieval: "Find me the note."
2. Reasoning: "Show me the path, impact, and confidence around this note."
This is the main narrative upgrade for premium local users.
## Premium Value Pillars
### 1) Decision Confidence
Users can see decision lineage:
1. What evidence supported a decision.
2. Which notes/specs informed it.
3. How that decision evolved over time.
### 2) Change Safety
Users can run impact-aware workflows:
1. Estimate blast radius before editing.
2. Surface downstream dependencies.
3. Prioritize what to review first.
### 3) Knowledge Quality
Users can maintain graph health:
1. Detect orphaned notes.
2. Detect overloaded hub notes.
3. Detect stale but high-centrality notes.
4. Detect likely contradictions.
### 4) Guided Navigation
Users can explore deeper relationships without context explosion:
1. Follow promising branches.
2. Stop when confidence is sufficient.
3. Avoid "load everything and hope."
## Feature Catalog (Value-First)
### A. Decision Lineage
What users get:
1. A clear "why chain" for important conclusions.
2. Traceable connections to supporting notes.
3. Better handoffs and historical understanding.
### B. Impact Radius
What users get:
1. A ranked list of likely affected notes before edits.
2. Safer refactors for docs, plans, and architecture.
3. Reduced accidental drift and inconsistency.
### C. Knowledge Health Dashboard
What users get:
1. Weekly health signals for the graph.
2. Actionable cleanup targets.
3. Better long-term memory quality with less manual auditing.
### D. Path Explorer
What users get:
1. "Show me how A connects to B" style explanations.
2. Multiple candidate paths with confidence cues.
3. Better discovery across large note collections.
### E. Contradiction Watch
What users get:
1. Early warnings for conflicting statements.
2. Suggested reconciliation workflow.
3. Higher trust in the knowledge base.
### F. Priority Briefs
What users get:
1. Periodic "what matters now" graph summaries.
2. Focused recommendations, not noisy activity dumps.
3. Better focus for solo builders and small teams.
## User Personas and Why They Pay
### Solo Technical Founder
Pain:
Cannot hold full architecture and decision history in working memory.
Premium value:
Impact Radius + Decision Lineage prevent rework and regressions.
### Product/Research Lead
Pain:
Knowledge is fragmented across specs, notes, and decisions.
Premium value:
Path Explorer + Priority Briefs compress synthesis time.
### Consultant/Fractional Operator
Pain:
Frequent context switching across domains and clients.
Premium value:
Knowledge Health + Decision Lineage speed onboarding and reporting.
## Packaging Direction
Suggested packaging:
1. OSS Local: existing search + context tools.
2. Local+ Graph Intelligence: advanced graph insight features listed above.
3. Future Team Add-On: shared policies, shared graph health views, shared lineage views.
Core upsell line:
"Keep your local workflow. Add graph intelligence when complexity grows."
## Experience Principles
1. Explainability first.
Every advanced result should show "why this was suggested."
2. Actionability over novelty.
Insights should lead to concrete next steps, not abstract charts.
3. Progressive disclosure.
Start with concise summaries, expand on demand.
4. Deterministic where possible.
Users should trust repeated runs of the same workflow.
5. Respect local-first expectations.
No surprise cloud dependency in premium local mode.
## Success Criteria (Product)
1. Users can describe the benefit in one sentence:
"It shows me what matters and what breaks before I change things."
2. Premium users report lower time-to-understanding for complex topics.
3. Premium users report fewer "surprise side effects" after edits.
4. Premium users keep larger knowledge graphs healthy with less manual effort.
5. Feature adoption is driven by outcomes, not by curiosity-only usage.
## Risks and Mitigations
Risk: Feature sounds like "just better search."
Mitigation: Lead messaging with decision confidence and change safety, not traversal depth.
Risk: Feature feels too advanced for normal users.
Mitigation: Package as guided insights and reports, not as a query language.
Risk: Insight quality feels noisy.
Mitigation: Focus launch scope on high-precision insight types and transparent rationale.
Risk: Value is hard to prove.
Mitigation: Track user-facing outcomes (time saved, risk avoided, cleanup completed).
## Rollout Narrative
Phase 1: "Safer Changes"
1. Impact Radius
2. Decision Lineage
Phase 2: "Health and Clarity"
1. Knowledge Health Dashboard
2. Contradiction Watch
Phase 3: "Strategic Navigation"
1. Path Explorer
2. Priority Briefs
## One-Line Positioning Options
1. "Local notes, strategic intelligence."
2. "Know what changed, why it matters, and what it affects."
3. "From note-taking to decision support."
## Open Questions
1. Which two features best define the paid tier at launch?
2. Which insight types should be guaranteed deterministic in v1?
3. Should Priority Briefs be bundled or separate as an add-on?
4. What is the simplest in-product education flow for first-time premium users?
+2 -2
View File
@@ -6,12 +6,12 @@
"url": "https://github.com/basicmachines-co/basic-memory.git",
"source": "github"
},
"version": "0.19.1",
"version": "0.18.5",
"packages": [
{
"registryType": "pypi",
"identifier": "basic-memory",
"version": "0.19.1",
"version": "0.18.5",
"runtimeHint": "uvx",
"runtimeArguments": [
{"type": "positional", "value": "basic-memory"},
+1 -1
View File
@@ -1,7 +1,7 @@
"""basic-memory - Local-first knowledge management combining Zettelkasten with knowledge graphs"""
# Package version - updated by release automation
__version__ = "0.19.1"
__version__ = "0.18.5"
# API version for FastAPI - independent of package version
__api_version__ = "v0"
@@ -20,7 +20,6 @@ from basic_memory.deps import (
ProjectConfigV2ExternalDep,
AppConfigDep,
EntityRepositoryV2ExternalDep,
RelationRepositoryV2ExternalDep,
ProjectExternalIdPathDep,
TaskSchedulerDep,
FileServiceV2ExternalDep,
@@ -32,9 +31,6 @@ from basic_memory.schemas.v2 import (
EntityResolveRequest,
EntityResolveResponse,
EntityResponseV2,
GraphEdge,
GraphNode,
GraphResponse,
MoveEntityRequestV2,
MoveDirectoryRequestV2,
DeleteDirectoryRequestV2,
@@ -60,50 +56,6 @@ def _schedule_vector_sync_if_enabled(
)
## Graph endpoint
@router.get("/graph", response_model=GraphResponse)
async def get_graph(
project_id: ProjectExternalIdPathDep,
entity_repository: EntityRepositoryV2ExternalDep,
relation_repository: RelationRepositoryV2ExternalDep,
) -> GraphResponse:
"""Return all entities and resolved relations for knowledge graph visualization.
Returns a flat node/edge structure optimized for rendering with graph libraries.
Only includes resolved relations (where to_id is not null).
"""
logger.info("API v2 request: get_graph")
# Fetch all entities for this project
entities = await entity_repository.find_all(use_load_options=False)
nodes = [
GraphNode(
external_id=entity.external_id,
title=entity.title,
note_type=entity.note_type,
file_path=entity.file_path,
)
for entity in entities
]
# Fetch all resolved relations (to_id is not null) with eager-loaded entities
relations = await relation_repository.find_all()
edges = [
GraphEdge(
from_id=relation.from_entity.external_id,
to_id=relation.to_entity.external_id,
relation_type=relation.relation_type,
)
for relation in relations
if relation.to_entity is not None
]
logger.info(f"API v2 response: graph with {len(nodes)} nodes and {len(edges)} edges")
return GraphResponse(nodes=nodes, edges=edges)
## Resolution endpoint
+1 -1
View File
@@ -12,7 +12,7 @@ from basic_memory.config import ConfigManager
OSS_DISCOUNT_CODE = "BMFOSS"
CLOUD_LEARN_MORE_URL = (
"https://basicmemory.com?utm_source=bm-foss&utm_medium=promo&utm_campaign=cloud-upsell"
"https://basicmemory.com?utm_source=bm-cli&utm_medium=promo&utm_campaign=cloud-upsell"
)
+52 -34
View File
@@ -43,37 +43,40 @@ if sys.platform == "win32": # pragma: no cover
_engine: Optional[AsyncEngine] = None
_session_maker: Optional[async_sessionmaker[AsyncSession]] = None
# Alembic revision that enables one-time automatic embedding backfill.
SEMANTIC_EMBEDDING_BACKFILL_REVISION = "i2c3d4e5f6g7"
async def _needs_semantic_embedding_backfill(
app_config: BasicMemoryConfig,
async def _load_applied_alembic_revisions(
session_maker: async_sessionmaker[AsyncSession],
) -> bool:
"""Check if entities exist but vector embeddings are empty.
) -> set[str]:
"""Load applied Alembic revisions from alembic_version.
This is the reliable way to detect that embeddings need to be generated,
regardless of how migrations were applied (fresh DB, upgrade, reset, etc.).
Returns an empty set when the version table does not exist yet
(fresh database before first migration).
"""
if not app_config.semantic_search_enabled:
return False
try:
async with scoped_session(session_maker) as session:
entity_count = (
await session.execute(text("SELECT COUNT(*) FROM entity"))
).scalar() or 0
if entity_count == 0:
return False
# Check if vector chunks table exists and is empty
embedding_count = (
await session.execute(text("SELECT COUNT(*) FROM search_vector_chunks"))
).scalar() or 0
return embedding_count == 0
result = await session.execute(text("SELECT version_num FROM alembic_version"))
return {str(row[0]) for row in result.fetchall() if row[0]}
except Exception as exc:
# Table might not exist yet (pre-migration)
logger.debug(f"Could not check embedding status: {exc}")
return False
error_message = str(exc).lower()
if "alembic_version" in error_message and (
"no such table" in error_message or "does not exist" in error_message
):
return set()
raise
def _should_run_semantic_embedding_backfill(
revisions_before_upgrade: set[str],
revisions_after_upgrade: set[str],
) -> bool:
"""Check if this migration run newly applied the backfill-trigger revision."""
return (
SEMANTIC_EMBEDDING_BACKFILL_REVISION in revisions_after_upgrade
and SEMANTIC_EMBEDDING_BACKFILL_REVISION not in revisions_before_upgrade
)
async def _run_semantic_embedding_backfill(
@@ -477,9 +480,26 @@ async def run_migrations(
Note: Alembic tracks which migrations have been applied via the alembic_version table,
so it's safe to call this multiple times - it will only run pending migrations.
"""
logger.info("Running database migrations...")
logger.debug("Running database migrations...")
temp_engine: AsyncEngine | None = None
try:
revisions_before_upgrade: set[str] = set()
# Trigger: run_migrations() can be invoked before module-level session maker is set.
# Why: we still need reliable before/after revision detection for one-time backfill.
# Outcome: create a short-lived session maker when needed, then dispose it immediately.
if _session_maker is None:
precheck_engine, temp_session_maker = _create_engine_and_session(
app_config.database_path,
database_type,
app_config,
)
try:
revisions_before_upgrade = await _load_applied_alembic_revisions(temp_session_maker)
finally:
await precheck_engine.dispose()
else:
revisions_before_upgrade = await _load_applied_alembic_revisions(_session_maker)
# Get the absolute path to the alembic directory relative to this file
alembic_dir = Path(__file__).parent / "alembic"
config = Config()
@@ -499,7 +519,7 @@ async def run_migrations(
config.set_main_option("sqlalchemy.url", db_url)
command.upgrade(config, "head")
logger.info("Migrations completed successfully")
logger.debug("Migrations completed successfully")
# Get session maker - ensure we don't trigger recursive migration calls
if _session_maker is None:
@@ -521,14 +541,12 @@ async def run_migrations(
else:
await SQLiteSearchRepository(session_maker, 1).init_search_index()
# Check if backfill is needed — actual backfill runs in background
# from the MCP server lifespan to avoid blocking startup.
if await _needs_semantic_embedding_backfill(app_config, session_maker):
logger.info(
"Semantic embeddings missing — backfill will run in background after startup"
)
else:
logger.info("Semantic embeddings: up to date")
revisions_after_upgrade = await _load_applied_alembic_revisions(session_maker)
if _should_run_semantic_embedding_backfill(
revisions_before_upgrade,
revisions_after_upgrade,
):
await _run_semantic_embedding_backfill(app_config, session_maker)
except Exception as e: # pragma: no cover
logger.error(f"Error running migrations: {e}")
raise
-72
View File
@@ -2,71 +2,18 @@
Basic Memory FastMCP server.
"""
import asyncio
import time
from contextlib import asynccontextmanager
from fastmcp import FastMCP
from loguru import logger
from sqlalchemy import text
from sqlalchemy.ext.asyncio import async_sessionmaker, AsyncSession
from basic_memory import db
from basic_memory.cli.auth import CLIAuth
from basic_memory.config import BasicMemoryConfig
from basic_memory.db import (
scoped_session,
_needs_semantic_embedding_backfill,
_run_semantic_embedding_backfill,
)
from basic_memory.mcp.container import McpContainer, set_container
from basic_memory.services.initialization import initialize_app
async def _log_embedding_status(session_maker: async_sessionmaker[AsyncSession]) -> None:
"""Log a clear summary of semantic embedding status at startup."""
try:
async with scoped_session(session_maker) as session:
entity_count = (
await session.execute(text("SELECT COUNT(*) FROM entity"))
).scalar() or 0
chunk_count = (
await session.execute(text("SELECT COUNT(*) FROM search_vector_chunks"))
).scalar() or 0
embedding_count = (
await session.execute(text("SELECT COUNT(*) FROM search_vector_embeddings_rowids"))
).scalar() or 0
if entity_count == 0:
logger.info("Semantic embeddings: no entities yet")
elif embedding_count == 0:
logger.warning(
f"Semantic embeddings: EMPTY — {entity_count} entities have no embeddings. "
"Backfill running in background..."
)
else:
logger.info(
f"Semantic embeddings: {embedding_count} embeddings "
f"across {chunk_count} chunks for {entity_count} entities"
)
except Exception as exc:
logger.debug(f"Could not check embedding status at startup: {exc}")
async def _background_embedding_backfill(
config: BasicMemoryConfig,
session_maker: async_sessionmaker[AsyncSession],
) -> None:
"""Run semantic embedding backfill in the background without blocking startup."""
try:
if await _needs_semantic_embedding_backfill(config, session_maker):
logger.info("Background embedding backfill starting...")
await _run_semantic_embedding_backfill(config, session_maker)
await _log_embedding_status(session_maker)
except Exception as exc:
logger.error(f"Background embedding backfill failed: {exc}")
@asynccontextmanager
async def lifespan(app: FastMCP):
"""Lifecycle manager for the MCP server.
@@ -123,16 +70,6 @@ async def lifespan(app: FastMCP):
# Initialize app (runs migrations, reconciles projects)
await initialize_app(container.config)
# Log embedding status so it's easy to spot in the logs
backfill_task: asyncio.Task | None = None # type: ignore[type-arg]
if config.semantic_search_enabled and db._session_maker is not None:
await _log_embedding_status(db._session_maker)
# Launch backfill in background so MCP server is ready immediately
backfill_task = asyncio.create_task(
_background_embedding_backfill(config, db._session_maker),
name="embedding-backfill",
)
# Create and start sync coordinator (lifecycle centralized in coordinator)
sync_coordinator = container.create_sync_coordinator()
await sync_coordinator.start()
@@ -142,15 +79,6 @@ async def lifespan(app: FastMCP):
finally:
# Shutdown - coordinator handles clean task cancellation
logger.debug("Shutting down Basic Memory MCP server")
# Cancel embedding backfill if still running
if backfill_task is not None and not backfill_task.done():
backfill_task.cancel()
try:
await backfill_task
except asyncio.CancelledError:
logger.info("Background embedding backfill cancelled during shutdown")
await sync_coordinator.stop()
# Only shutdown DB if we created it (not if test fixture provided it)
+1 -1
View File
@@ -318,7 +318,7 @@ delete_note("path/to/file.md")
note_file_path = None
try:
# Resolve identifier to entity ID
entity_id = await knowledge_client.resolve_entity(identifier, strict=True)
entity_id = await knowledge_client.resolve_entity(identifier)
if output_format == "json":
entity = await knowledge_client.get_entity(entity_id)
note_title = entity.title
+5 -19
View File
@@ -158,7 +158,7 @@ Error editing note '{identifier}': {error_message}
@mcp.tool(
description="Edit an existing markdown note using various operations like append, prepend, find_replace, replace_section, insert_before_section, or insert_after_section.",
description="Edit an existing markdown note using various operations like append, prepend, find_replace, or replace_section.",
annotations={"destructiveHint": False, "openWorldHint": False},
)
async def edit_note(
@@ -190,8 +190,6 @@ async def edit_note(
- "prepend": Add content to the beginning of the note (creates the note if it doesn't exist)
- "find_replace": Replace occurrences of find_text with content (note must exist)
- "replace_section": Replace content under a specific markdown header (note must exist)
- "insert_before_section": Insert content before a section heading without consuming it (note must exist)
- "insert_after_section": Insert content after a section heading without consuming it (note must exist)
content: The content to add or use for replacement
project: Project name to edit in. Optional - server will resolve using hierarchy.
If unknown, use list_memory_projects() to discover available projects.
@@ -259,14 +257,7 @@ async def edit_note(
logger.info("MCP tool call", tool="edit_note", identifier=identifier, operation=operation)
# Validate operation
valid_operations = [
"append",
"prepend",
"find_replace",
"replace_section",
"insert_before_section",
"insert_after_section",
]
valid_operations = ["append", "prepend", "find_replace", "replace_section"]
if operation not in valid_operations:
raise ValueError(
f"Invalid operation '{operation}'. Must be one of: {', '.join(valid_operations)}"
@@ -275,9 +266,8 @@ async def edit_note(
# Validate required parameters for specific operations
if operation == "find_replace" and not find_text:
raise ValueError("find_text parameter is required for find_replace operation")
section_ops = ("replace_section", "insert_before_section", "insert_after_section")
if operation in section_ops and not section:
raise ValueError("section parameter is required for section-based operations")
if operation == "replace_section" and not section:
raise ValueError("section parameter is required for replace_section operation")
# Use the PATCH endpoint to edit the entity
try:
@@ -293,7 +283,7 @@ async def edit_note(
# Try to resolve the entity; for append/prepend, create it if not found
try:
entity_id = await knowledge_client.resolve_entity(identifier, strict=True)
entity_id = await knowledge_client.resolve_entity(identifier)
except Exception as resolve_error:
# Trigger: entity does not exist yet
# Why: append/prepend can meaningfully create a new note from the content,
@@ -399,10 +389,6 @@ async def edit_note(
summary.append("operation: Find and replace operation completed")
elif operation == "replace_section":
summary.append(f"operation: Replaced content under section '{section}'")
elif operation == "insert_before_section":
summary.append(f"operation: Inserted content before section '{section}'")
elif operation == "insert_after_section":
summary.append(f"operation: Inserted content after section '{section}'")
# Count observations by category (reuse logic from write_note)
categories = {}
+2 -21
View File
@@ -6,7 +6,6 @@ from typing import Optional, Literal
from loguru import logger
from fastmcp import Context
from mcp.server.fastmcp.exceptions import ToolError
from basic_memory.mcp.server import mcp
from basic_memory.mcp.project_context import get_project_client
@@ -638,7 +637,7 @@ move_note("path/to/file.md", "{destination_path}/file.md")
"""Resolve and cache the source entity ID for the duration of this move."""
nonlocal resolved_entity_id
if resolved_entity_id is None:
resolved_entity_id = await knowledge_client.resolve_entity(identifier, strict=True)
resolved_entity_id = await knowledge_client.resolve_entity(identifier)
return resolved_entity_id
try:
@@ -646,26 +645,8 @@ move_note("path/to/file.md", "{destination_path}/file.md")
source_entity = await knowledge_client.get_entity(resolved_entity_id)
if "." in source_entity.file_path:
source_ext = source_entity.file_path.split(".")[-1]
except ToolError as e:
# Trigger: strict=True resolve_entity raised because the entity was not found.
# Why: fail fast with a formatted error instead of silently falling through
# to extension defaults and failing later with a confusing message.
# Outcome: move_note returns a user-facing not-found error immediately.
logger.error(f"Move failed for '{identifier}' to '{destination_path}': {e}")
if output_format == "json":
return {
"moved": False,
"title": None,
"permalink": None,
"file_path": None,
"source": identifier,
"destination": destination_path,
"error": str(e),
}
return _format_move_error_response(str(e), identifier, destination_path)
except Exception as e:
# If we can't fetch source metadata (e.g. get_entity or file_path parsing fails),
# continue with extension defaults — the entity was at least resolved.
# If we can't fetch source metadata, continue with extension defaults.
logger.debug(f"Could not fetch source entity for extension check: {e}")
# --- Resolve destination_folder into destination_path ---
+2 -2
View File
@@ -160,7 +160,7 @@ def _no_notes_guidance(note_type: str, tool_name: str) -> str:
f"## Next Steps\n\n"
f"1. **Create notes of this type** — use `write_note` with "
f'`note_type="{note_type}"` to create notes\n'
f"2. **Check existing types** — use `search_notes` with `note_types` "
f"2. **Check existing types** — use `search_notes` with `entity_types` "
f"filter to see what types exist\n"
f"3. **Browse content** — use `list_directory` or `recent_activity` to "
f"see what's in the project\n"
@@ -397,7 +397,7 @@ async def schema_infer(
f"share a consistent structure.\n\n"
f"## Suggestions\n"
f"1. **Use a more specific type** — try `search_notes` with "
f"`note_types` filter to see what types exist\n"
f"`entity_types` filter to see what types exist\n"
f"2. **Lower the threshold** — "
f'`schema_infer("{note_type}", threshold=0.1)` to include '
f"rarer fields\n"
+6 -17
View File
@@ -2,7 +2,7 @@
import re
from textwrap import dedent
from typing import Annotated, List, Optional, Dict, Any, Literal
from typing import List, Optional, Dict, Any, Literal
from loguru import logger
from fastmcp import Context
@@ -165,7 +165,7 @@ def _format_search_error_response(
- Remove restrictive terms: Focus on the most important keywords
5. **Use filtering to narrow scope**:
- By note type in frontmatter: `search_notes("{project}","{query}", note_types=["note"])`
- By content type: `search_notes("{project}","{query}", note_types=["note"])`
- By recent content: `search_notes("{project}","{query}", after_date="1 week")`
- By entity type: `search_notes("{project}","{query}", entity_types=["observation"])`
@@ -305,17 +305,8 @@ async def search_notes(
page_size: int = 10,
search_type: str | None = None,
output_format: Literal["text", "json"] = "text",
note_types: Annotated[
List[str] | None,
"Filter by the 'type' field in note frontmatter (e.g. 'note', 'chapter', 'person'). "
"Case-insensitive.",
] = None,
entity_types: Annotated[
List[str] | None,
"Filter by knowledge graph item type: 'entity' (whole notes), 'observation', or "
"'relation'. Defaults to 'entity'. Do NOT pass schema/frontmatter types like "
"'Chapter' here — use note_types instead.",
] = None,
note_types: List[str] | None = None,
entity_types: List[str] | None = None,
after_date: Optional[str] = None,
metadata_filters: Optional[Dict[str, Any]] = None,
tags: Optional[List[str]] = None,
@@ -359,7 +350,6 @@ async def search_notes(
### Search Type Examples
- `search_notes("my-project", "Meeting", search_type="title")` - Search only in titles
- `search_notes("work-docs", "docs/meeting-*", search_type="permalink")` - Pattern match permalinks
Note: Permalink patterns match the full path (e.g., "project/folder/chapter-13*", not just "chapter-13*").
- `search_notes("research", "keyword")` - Default search (hybrid when semantic is enabled,
text when disabled)
@@ -446,7 +436,7 @@ async def search_notes(
# Exact phrase search
results = await search_notes("\"weekly standup meeting\"")
# Search with note type filter - type property in frontmatter
# Search with note type filter
results = await search_notes(
"meeting notes",
note_types=["note"],
@@ -487,8 +477,7 @@ async def search_notes(
results = await search_notes("project planning", project="my-project")
"""
# Avoid mutable-default-argument footguns. Treat None as "no filter".
# Lowercase note_types so "Chapter" matches the stored "chapter".
note_types = [t.lower() for t in note_types] if note_types else []
note_types = note_types or []
entity_types = entity_types or []
# Parse tag:<value> shorthand at tool level so it works with all search modes.
+3 -12
View File
@@ -2,7 +2,7 @@
from __future__ import annotations
from typing import Annotated, Any, Dict, List, Optional
from typing import Any, Dict, List, Optional
from fastmcp import Context
from mcp.types import ContentBlock, TextContent
@@ -28,17 +28,8 @@ async def search_notes_ui(
page: int = 1,
page_size: int = 10,
search_type: Optional[str] = None,
note_types: Annotated[
List[str] | None,
"Filter by the 'type' field in note frontmatter (e.g. 'note', 'chapter', 'person'). "
"Case-insensitive.",
] = None,
entity_types: Annotated[
List[str] | None,
"Filter by knowledge graph item type: 'entity' (whole notes), 'observation', or "
"'relation'. Defaults to 'entity'. Do NOT pass schema/frontmatter types like "
"'Chapter' here — use note_types instead.",
] = None,
note_types: List[str] | None = None,
entity_types: List[str] | None = None,
after_date: Optional[str] = None,
metadata_filters: Optional[Dict[str, Any]] = None,
tags: Optional[List[str]] = None,
+3 -5
View File
@@ -140,12 +140,10 @@ def validate_timeframe(timeframe: str) -> str:
if parsed > now:
raise ValueError("Timeframe cannot be in the future") # pragma: no cover
# Round to nearest day to handle DST transitions where an hour shift
# can cause e.g. "7d" to compute as 6 days + 23 hours
total_seconds = (now - parsed).total_seconds()
days = round(total_seconds / 86400)
# Could format the duration back to our standard format
days = (now - parsed).days
# Enforce reasonable limits
# Could enforce reasonable limits
if days > 365:
raise ValueError("Timeframe should be <= 1 year")
+3 -18
View File
@@ -65,14 +65,7 @@ class EditEntityRequest(BaseModel):
Supports various operation types for different editing scenarios.
"""
operation: Literal[
"append",
"prepend",
"find_replace",
"replace_section",
"insert_before_section",
"insert_after_section",
]
operation: Literal["append", "prepend", "find_replace", "replace_section"]
content: str
section: Optional[str] = None
find_text: Optional[str] = None
@@ -82,16 +75,8 @@ class EditEntityRequest(BaseModel):
@classmethod
def validate_section_for_replace_section(cls, v, info):
"""Ensure section is provided for replace_section operation."""
if (
info.data.get("operation")
in (
"replace_section",
"insert_before_section",
"insert_after_section",
)
and not v
):
raise ValueError("section parameter is required for section-based operations")
if info.data.get("operation") == "replace_section" and not v:
raise ValueError("section parameter is required for replace_section operation")
return v
@field_validator("find_text")
-8
View File
@@ -10,11 +10,6 @@ from basic_memory.schemas.v2.entity import (
ProjectResolveRequest,
ProjectResolveResponse,
)
from basic_memory.schemas.v2.graph import (
GraphEdge,
GraphNode,
GraphResponse,
)
from basic_memory.schemas.v2.resource import (
CreateResourceRequest,
UpdateResourceRequest,
@@ -30,9 +25,6 @@ __all__ = [
"DeleteDirectoryRequestV2",
"ProjectResolveRequest",
"ProjectResolveResponse",
"GraphEdge",
"GraphNode",
"GraphResponse",
"CreateResourceRequest",
"UpdateResourceRequest",
"ResourceResponse",
-31
View File
@@ -1,31 +0,0 @@
"""Graph visualization schemas for the knowledge graph endpoint."""
from typing import Optional
from pydantic import BaseModel, Field
class GraphNode(BaseModel):
"""A node in the knowledge graph visualization."""
external_id: str = Field(..., description="Entity external ID (UUID)")
title: str = Field(..., description="Entity title")
note_type: Optional[str] = Field(None, description="Note type (e.g., note, spec, task)")
file_path: str = Field(..., description="Relative file path")
class GraphEdge(BaseModel):
"""An edge in the knowledge graph visualization."""
from_id: str = Field(..., description="External ID of source entity")
to_id: str = Field(..., description="External ID of target entity")
relation_type: str = Field(..., description="Type of relation")
class GraphResponse(BaseModel):
"""Complete knowledge graph for visualization."""
nodes: list[GraphNode] = Field(default_factory=list, description="All entities as nodes")
edges: list[GraphEdge] = Field(
default_factory=list, description="All resolved relations as edges"
)
@@ -888,14 +888,6 @@ class EntityService(BaseService[EntityModel]):
raise ValueError("section cannot be empty or whitespace only")
return self.replace_section_content(current_content, section, content)
elif operation in ("insert_before_section", "insert_after_section"):
if not section:
raise ValueError("section is required for insert section operations")
if not section.strip():
raise ValueError("section cannot be empty or whitespace only")
position = "before" if operation == "insert_before_section" else "after"
return self.insert_relative_to_section(current_content, section, content, position)
else:
raise ValueError(f"Unsupported operation: {operation}")
@@ -987,73 +979,6 @@ class EntityService(BaseService[EntityModel]):
return "\n".join(result_lines)
def insert_relative_to_section(
self,
current_content: str,
section_header: str,
new_content: str,
position: str,
) -> str:
"""Insert content before or after a section heading without consuming it.
Unlike replace_section_content, this preserves the section heading and its
existing content. The new content is inserted immediately before or after
the heading line.
Args:
current_content: The current markdown content
section_header: The section header to anchor on (e.g., "## Section Name")
new_content: The content to insert
position: "before" to insert above the heading, "after" to insert below it
Returns:
The updated content with new_content inserted relative to the heading
Raises:
ValueError: If the section header is not found or appears more than once
"""
# Normalize the section header (ensure it starts with #)
if not section_header.startswith("#"):
section_header = "## " + section_header
lines = current_content.split("\n")
matching_indices = [
i for i, line in enumerate(lines) if line.strip() == section_header.strip()
]
if len(matching_indices) == 0:
raise ValueError(
f"Section '{section_header}' not found in document. "
f"Use replace_section to create a new section."
)
if len(matching_indices) > 1:
raise ValueError(
f"Multiple sections found with header '{section_header}'. "
f"Section insertion requires unique headers."
)
idx = matching_indices[0]
if position == "before":
# Insert new content before the section heading
before = lines[:idx]
after = lines[idx:]
# Ensure blank line separation
insert_lines = new_content.rstrip("\n").split("\n")
if before and before[-1].strip() != "":
insert_lines = [""] + insert_lines
return "\n".join(before + insert_lines + [""] + after)
else:
# Insert new content after the section heading line
before = lines[: idx + 1]
after = lines[idx + 1 :]
insert_lines = new_content.rstrip("\n").split("\n")
# Ensure blank line separation so inserted text doesn't merge
# with existing section content into a single paragraph
if after and after[0].strip() != "":
insert_lines = insert_lines + [""]
return "\n".join(before + insert_lines + after)
def _prepend_after_frontmatter(self, current_content: str, content: str) -> str:
"""Prepend content after frontmatter, preserving frontmatter structure."""
+3 -29
View File
@@ -293,16 +293,12 @@ class SyncService:
for path in report.deleted:
await self.handle_delete(path)
# then new and modified — collect entity IDs for batch vector embedding
synced_entity_ids: list[int] = []
# then new and modified
for path in report.new:
entity, _ = await self.sync_file(path, new=True)
if entity is not None:
synced_entity_ids.append(entity.id)
# Track if file was skipped
elif await self._should_skip_file(path):
if entity is None and await self._should_skip_file(path):
failure_info = self._file_failures[path]
report.skipped_files.append(
SkippedFile(
@@ -316,10 +312,8 @@ class SyncService:
for path in report.modified:
entity, _ = await self.sync_file(path, new=False)
if entity is not None:
synced_entity_ids.append(entity.id)
# Track if file was skipped
elif await self._should_skip_file(path):
if entity is None and await self._should_skip_file(path):
failure_info = self._file_failures[path]
report.skipped_files.append(
SkippedFile(
@@ -337,26 +331,6 @@ class SyncService:
else:
logger.info("Skipping relation resolution - no file changes detected")
# Batch-generate vector embeddings for all synced entities
if synced_entity_ids and self.app_config.semantic_search_enabled:
try:
logger.info(
f"Generating semantic embeddings for {len(synced_entity_ids)} entities..."
)
batch_result = await self.search_service.sync_entity_vectors_batch(
synced_entity_ids
)
logger.info(
f"Semantic embeddings complete: "
f"synced={batch_result.entities_synced}, "
f"failed={batch_result.entities_failed}"
)
except SemanticDependenciesMissingError:
logger.warning(
"Semantic search dependencies missing — vector embeddings skipped. "
"Run 'bm reindex --embeddings' after resolving the dependency issue."
)
# Update scan watermark after successful sync
# Use the timestamp from sync start (not end) to ensure we catch files
# created during the sync on the next iteration
@@ -208,7 +208,7 @@ def test_edit_note_replace_section_fails_without_section(
)
assert result.exit_code != 0
assert "section parameter is required for section-based operations" in result.output
assert "section parameter is required for replace_section operation" in result.output
def test_edit_note_append_creates_nonexistent_note_cli(
+5 -30
View File
@@ -307,13 +307,8 @@ async def test_delete_note_by_file_path(mcp_server, app, test_project):
@pytest.mark.asyncio
async def test_delete_note_rejects_case_mismatch(mcp_server, app, test_project):
"""Test that delete_note with wrong case does not fuzzy-match to an existing note.
Strict resolution (#649) prevents destructive operations from silently
resolving to a different note via fuzzy search. Case-mismatched titles
should be rejected, not resolved to the nearest match.
"""
async def test_delete_note_case_insensitive(mcp_server, app, test_project):
"""Test that note deletion is case insensitive for titles."""
async with Client(mcp_server) as client:
# Create a note with mixed case
@@ -328,7 +323,7 @@ async def test_delete_note_rejects_case_mismatch(mcp_server, app, test_project):
},
)
# Try to delete with different case — should NOT find the note
# Try to delete with different case
delete_result = await client.call_tool(
"delete_note",
{
@@ -337,28 +332,8 @@ async def test_delete_note_rejects_case_mismatch(mcp_server, app, test_project):
},
)
# Should return False (not found) — strict mode rejects fuzzy matches
assert "false" in delete_result.content[0].text.lower()
# Verify the note still exists using the exact title
read_result = await client.call_tool(
"read_note",
{
"project": test_project.name,
"identifier": "CamelCase Note Title",
},
)
assert "Testing case sensitivity" in read_result.content[0].text
# Delete with exact title should succeed
delete_result2 = await client.call_tool(
"delete_note",
{
"project": test_project.name,
"identifier": "CamelCase Note Title",
},
)
assert "true" in delete_result2.content[0].text.lower()
# Should return True for successful deletion
assert "true" in delete_result.content[0].text.lower()
@pytest.mark.asyncio
@@ -710,81 +710,3 @@ async def test_edit_note_using_different_identifiers(mcp_server, app, test_proje
assert "Edited by title." in content
assert "Edited by permalink." in content
assert "Edited by folder/title." in content
@pytest.mark.asyncio
async def test_edit_note_append_autocreate_does_not_fuzzy_match(mcp_server, app, test_project):
"""Reproduces #649: edit_note append must auto-create, not fuzzy-match to an existing note.
Creates two notes, then attempts to append to a nonexistent identifier.
The tool should create a new note, and neither existing note should be modified.
"""
async with Client(mcp_server) as client:
# Create two notes that could be fuzzy-matched
await client.call_tool(
"write_note",
{
"project": test_project.name,
"title": "Routing Test A",
"directory": "test",
"content": "# Routing Test A\n\nContent A.",
},
)
await client.call_tool(
"write_note",
{
"project": test_project.name,
"title": "Routing Test B",
"directory": "test",
"content": "# Routing Test B\n\nContent B.",
},
)
# Attempt to edit a nonexistent note — should error, not silently edit A or B
edit_result = await client.call_tool(
"edit_note",
{
"project": test_project.name,
"identifier": "Routing Test NONEXISTENT",
"operation": "append",
"content": "\n\nThis should NOT appear in any note.",
},
)
edit_text = edit_result.content[0].text
# append to nonexistent creates a new note — verify it did NOT edit A or B
assert "Created note (append)" in edit_text
assert "fileCreated: true" in edit_text
# Verify neither A nor B was modified
read_a = await client.call_tool(
"read_note",
{"project": test_project.name, "identifier": "Routing Test A"},
)
content_a = read_a.content[0].text
assert "Content A" in content_a
assert "This should NOT appear" not in content_a
read_b = await client.call_tool(
"read_note",
{"project": test_project.name, "identifier": "Routing Test B"},
)
content_b = read_b.content[0].text
assert "Content B" in content_b
assert "This should NOT appear" not in content_b
# Now test find_replace on nonexistent — should error
edit_result2 = await client.call_tool(
"edit_note",
{
"project": test_project.name,
"identifier": "Routing Test NONEXISTENT AGAIN",
"operation": "find_replace",
"content": "replaced",
"find_text": "Content",
},
)
error_text = edit_result2.content[0].text
assert "Edit Failed" in error_text
@@ -716,56 +716,3 @@ async def test_move_note_destination_folder_mutually_exclusive(mcp_server, app,
error_text = move_result.content[0].text
assert "# Move Failed - Invalid Parameters" in error_text
assert "Cannot specify both" in error_text
@pytest.mark.asyncio
async def test_move_note_strict_resolution_rejects_fuzzy_match(mcp_server, app, test_project):
"""move_note must not fuzzy-match a nonexistent identifier to an existing note (#649)."""
async with Client(mcp_server) as client:
# Create two notes that could be fuzzy-matched
await client.call_tool(
"write_note",
{
"project": test_project.name,
"title": "Move Strict Test A",
"directory": "test",
"content": "# Move Strict Test A\n\nContent A.",
},
)
await client.call_tool(
"write_note",
{
"project": test_project.name,
"title": "Move Strict Test B",
"directory": "test",
"content": "# Move Strict Test B\n\nContent B.",
},
)
# Attempt to move a nonexistent note — should error, not move A or B
move_result = await client.call_tool(
"move_note",
{
"project": test_project.name,
"identifier": "Move Strict Test NONEXISTENT",
"destination_path": "archive/Moved.md",
},
)
assert len(move_result.content) == 1
error_text = move_result.content[0].text
assert "# Move Failed" in error_text
# Verify neither A nor B was moved
read_a = await client.call_tool(
"read_note",
{"project": test_project.name, "identifier": "Move Strict Test A"},
)
assert "Content A" in read_a.content[0].text
read_b = await client.call_tool(
"read_note",
{"project": test_project.name, "identifier": "Move Strict Test B"},
)
assert "Content B" in read_b.content[0].text
-2
View File
@@ -31,7 +31,6 @@ async def test_returns_none_when_no_default_and_no_project(config_manager, monke
config_manager.save_config(cfg)
monkeypatch.delenv("BASIC_MEMORY_MCP_PROJECT", raising=False)
# Prevent API fallback from returning a project via stale dependency overrides
async def _no_api_fallback():
return None
@@ -118,7 +117,6 @@ async def test_returns_none_when_no_default(config_manager, monkeypatch):
config_manager.save_config(cfg)
monkeypatch.delenv("BASIC_MEMORY_MCP_PROJECT", raising=False)
# Prevent API fallback from returning a project via stale dependency overrides
async def _no_api_fallback():
return None
+3 -27
View File
@@ -1,10 +1,6 @@
"""Tests for delete_note MCP tool."""
import pytest
from basic_memory.mcp.tools.delete_note import delete_note, _format_delete_error_response
from basic_memory.mcp.tools.read_note import read_note
from basic_memory.mcp.tools.write_note import write_note
from basic_memory.mcp.tools.delete_note import _format_delete_error_response
class TestDeleteNoteErrorFormatting:
@@ -98,25 +94,5 @@ class TestDeleteNoteErrorFormatting:
assert "folder/note-title" in result # Permalink format
@pytest.mark.asyncio
async def test_delete_note_rejects_fuzzy_match(client, test_project):
"""delete_note must reject nonexistent identifiers, not fuzzy-match to a similar note."""
await write_note(
project=test_project.name,
title="Delete Target Note",
directory="test",
content="# Delete Target Note\nShould not be deleted.",
)
# Attempt to delete a nonexistent note — should return False, not silently delete the existing note
result = await delete_note(
project=test_project.name,
identifier="Delete Target NONEXISTENT",
)
# Should indicate not found (False or error string)
assert result is False or (isinstance(result, str) and "not found" in result.lower())
# Verify the existing note was NOT deleted
content = await read_note("Delete Target Note", project=test_project.name)
assert "Should not be deleted" in content
# Integration tests removed to focus on error formatting coverage
# The error formatting tests above provide the necessary coverage for MCP tool error messaging
+1 -160
View File
@@ -1,10 +1,8 @@
"""Tests for the edit_note MCP tool."""
import pytest
from basic_memory.mcp.tools.edit_note import edit_note
from basic_memory.mcp.tools.read_note import read_note
from basic_memory.mcp.tools.write_note import write_note
@@ -322,7 +320,7 @@ async def test_edit_note_replace_section_missing_section(client, test_project):
content="new content",
)
assert "section parameter is required for section-based operations" in str(exc_info.value)
assert "section parameter is required for replace_section operation" in str(exc_info.value)
@pytest.mark.asyncio
@@ -613,160 +611,3 @@ async def test_edit_note_preserves_permalink_when_frontmatter_missing(client, te
assert f"permalink: {test_project.name}/test/test-note" in second_result
assert f"[Session: Using project '{test_project.name}']" in second_result
# The edit should succeed without validation errors
@pytest.mark.asyncio
async def test_edit_note_find_replace_rejects_fuzzy_match(client, test_project):
"""find_replace must reject nonexistent identifiers, not fuzzy-match to a similar note."""
# Create two notes that could be fuzzy-matched
await write_note(
project=test_project.name,
title="Routing Test A",
directory="test",
content="# Routing Test A\nContent A.",
)
await write_note(
project=test_project.name,
title="Routing Test B",
directory="test",
content="# Routing Test B\nContent B.",
)
# Attempt to edit a nonexistent note — should error, not silently edit A or B
result = await edit_note(
project=test_project.name,
identifier="Routing Test NONEXISTENT",
operation="find_replace",
content="replaced",
find_text="Content",
)
assert isinstance(result, str)
assert "# Edit Failed" in result
# Verify neither A nor B was modified
content_a = await read_note("Routing Test A", project=test_project.name)
assert "Content A" in content_a
content_b = await read_note("Routing Test B", project=test_project.name)
assert "Content B" in content_b
@pytest.mark.asyncio
async def test_edit_note_append_autocreate_not_fuzzy_match(client, test_project):
"""append to a nonexistent note should auto-create it, not fuzzy-match an existing note."""
await write_note(
project=test_project.name,
title="Existing Note Alpha",
directory="test",
content="# Existing Note Alpha\nOriginal content.",
)
# Append to a nonexistent note — should create a new note, not edit "Existing Note Alpha"
result = await edit_note(
project=test_project.name,
identifier="Existing Note ZZZZZ",
operation="append",
content="# New Note\nBrand new content.",
)
assert isinstance(result, str)
assert "Created note (append)" in result
assert "fileCreated: true" in result
# Verify original note was NOT modified
content = await read_note("Existing Note Alpha", project=test_project.name)
assert "Original content" in content
assert "Brand new content" not in content
@pytest.mark.asyncio
async def test_edit_note_insert_before_section_operation(client, test_project):
"""Test inserting content before a section heading."""
# Create initial note with sections
await write_note(
project=test_project.name,
title="Insert Before Doc",
directory="docs",
content="# Doc\n\n## Overview\nOverview content.\n\n## Details\nDetail content.",
)
result = await edit_note(
project=test_project.name,
identifier="docs/insert-before-doc",
operation="insert_before_section",
content="--- inserted divider ---",
section="## Details",
)
assert isinstance(result, str)
assert "Edited note (insert_before_section)" in result
assert f"project: {test_project.name}" in result
assert "Inserted content before section '## Details'" in result
assert f"[Session: Using project '{test_project.name}']" in result
@pytest.mark.asyncio
async def test_edit_note_insert_after_section_operation(client, test_project):
"""Test inserting content after a section heading."""
# Create initial note with sections
await write_note(
project=test_project.name,
title="Insert After Doc",
directory="docs",
content="# Doc\n\n## Overview\nOverview content.\n\n## Details\nDetail content.",
)
result = await edit_note(
project=test_project.name,
identifier="docs/insert-after-doc",
operation="insert_after_section",
content="Inserted after overview heading",
section="## Overview",
)
assert isinstance(result, str)
assert "Edited note (insert_after_section)" in result
assert f"project: {test_project.name}" in result
assert "Inserted content after section '## Overview'" in result
assert f"[Session: Using project '{test_project.name}']" in result
@pytest.mark.asyncio
async def test_edit_note_insert_before_section_missing_section(client, test_project):
"""Test insert_before_section without section parameter raises ValueError."""
await write_note(
project=test_project.name,
title="Test Note",
directory="test",
content="# Test\nContent here.",
)
with pytest.raises(ValueError, match="section parameter is required"):
await edit_note(
project=test_project.name,
identifier="test/test-note",
operation="insert_before_section",
content="new content",
)
@pytest.mark.asyncio
async def test_edit_note_insert_before_section_not_found(client, test_project):
"""Test insert_before_section when section doesn't exist returns error."""
await write_note(
project=test_project.name,
title="Test Note",
directory="test",
content="# Test\n\n## Existing\nContent here.",
)
result = await edit_note(
project=test_project.name,
identifier="test/test-note",
operation="insert_before_section",
content="new content",
section="## Nonexistent",
)
assert isinstance(result, str)
assert "# Edit Failed" in result
-25
View File
@@ -590,31 +590,6 @@ async def test_move_note_preserves_frontmatter(app, client, test_project):
assert "Content with custom metadata" in content
@pytest.mark.asyncio
async def test_move_note_rejects_fuzzy_match(client, test_project):
"""move_note must reject nonexistent identifiers, not fuzzy-match to a similar note."""
await write_note(
project=test_project.name,
title="Move Target Note",
directory="source",
content="# Move Target Note\nShould not be moved.",
)
# Attempt to move a nonexistent note — should error, not silently move the existing note
result = await move_note(
project=test_project.name,
identifier="Move Target NONEXISTENT",
destination_path="target/Moved.md",
)
assert isinstance(result, str)
assert "# Move Failed" in result
# Verify the existing note was NOT moved
content = await read_note("Move Target Note", project=test_project.name)
assert "Should not be moved" in content
class TestMoveNoteErrorFormatting:
"""Test move note error formatting for better user experience."""
-48
View File
@@ -1146,54 +1146,6 @@ async def test_search_notes_explicit_entity_types_overrides_default(monkeypatch)
assert captured_payload["entity_types"] == ["observation"]
# --- Tests for note_types case-insensitivity ------------------------------------
@pytest.mark.asyncio
async def test_search_notes_note_types_lowercased(monkeypatch):
"""note_types values are lowercased so 'Chapter' matches stored 'chapter'."""
import importlib
search_mod = importlib.import_module("basic_memory.mcp.tools.search")
clients_mod = importlib.import_module("basic_memory.mcp.clients")
class StubProject:
name = "test-project"
external_id = "test-external-id"
@asynccontextmanager
async def fake_get_project_client(*args, **kwargs):
yield (object(), StubProject())
async def fake_resolve_project_and_path(
client, identifier, project=None, context=None, headers=None
):
return StubProject(), identifier, False
captured_payload: dict = {}
class MockSearchClient:
def __init__(self, *args, **kwargs):
pass
async def search(self, payload, page, page_size):
captured_payload.update(payload)
return SearchResponse(results=[], current_page=page, page_size=page_size)
monkeypatch.setattr(search_mod, "get_project_client", fake_get_project_client)
monkeypatch.setattr(search_mod, "resolve_project_and_path", fake_resolve_project_and_path)
monkeypatch.setattr(clients_mod, "SearchClient", MockSearchClient)
await search_mod.search_notes(
project="test-project",
query="test",
note_types=["Chapter", "Person"],
)
# note_types should be lowercased
assert captured_payload["note_types"] == ["chapter", "person"]
# --- Tests for tag: prefix parsing (issue #30) ---------------------------------
+3 -43
View File
@@ -345,7 +345,7 @@ def test_edit_entity_request_find_replace_empty_find_text():
def test_edit_entity_request_replace_section_empty_section():
"""Test that replace_section operation requires non-empty section parameter."""
with pytest.raises(
ValueError, match="section parameter is required for section-based operations"
ValueError, match="section parameter is required for replace_section operation"
):
EditEntityRequest.model_validate(
{
@@ -356,46 +356,6 @@ def test_edit_entity_request_replace_section_empty_section():
)
def test_edit_entity_request_insert_before_section():
"""Test insert_before_section is a valid operation."""
edit_request = EditEntityRequest.model_validate(
{
"operation": "insert_before_section",
"content": "content to insert",
"section": "## Target Section",
}
)
assert edit_request.operation == "insert_before_section"
assert edit_request.section == "## Target Section"
def test_edit_entity_request_insert_after_section():
"""Test insert_after_section is a valid operation."""
edit_request = EditEntityRequest.model_validate(
{
"operation": "insert_after_section",
"content": "content to insert",
"section": "## Target Section",
}
)
assert edit_request.operation == "insert_after_section"
assert edit_request.section == "## Target Section"
def test_edit_entity_request_insert_before_section_empty_section():
"""Test that insert_before_section requires non-empty section parameter."""
with pytest.raises(
ValueError, match="section parameter is required for section-based operations"
):
EditEntityRequest.model_validate(
{
"operation": "insert_before_section",
"content": "content",
"section": "",
}
)
# New tests for timeframe parsing functions
class TestTimeframeParsing:
"""Test cases for parse_timeframe() and validate_timeframe() functions."""
@@ -431,7 +391,7 @@ class TestTimeframeParsing:
result_1d = parse_timeframe("1d")
expected_1d = now - timedelta(days=1)
diff = abs((result_1d - expected_1d).total_seconds())
assert diff <= 3610 # Within 1 hour tolerance + execution margin (DST transitions)
assert diff < 3600 # Within 1 hour tolerance (accounts for DST transitions)
assert result_1d.tzinfo is not None
# Test yesterday - should be yesterday at same time
@@ -444,7 +404,7 @@ class TestTimeframeParsing:
result_week = parse_timeframe("1 week ago")
expected_week = now - timedelta(weeks=1)
diff = abs((result_week - expected_week).total_seconds())
assert diff <= 3610 # Within 1 hour tolerance + execution margin (DST transitions)
assert diff < 3600 # Within 1 hour tolerance
assert result_week.tzinfo is not None
def test_parse_timeframe_invalid(self):
-261
View File
@@ -1402,267 +1402,6 @@ async def test_edit_entity_replace_section_strips_duplicate_header(
assert "## Another Section" in file_content # Other sections preserved
# Insert before/after section tests
@pytest.mark.asyncio
async def test_edit_entity_insert_before_section(
entity_service: EntityService, file_service: FileService
):
"""Test inserting content before a section heading."""
content = dedent("""
# Main Title
## Section 1
Section 1 content
## Section 2
Section 2 content
""").strip()
entity = await entity_service.create_entity(
EntitySchema(
title="Insert Before Test",
directory="docs",
note_type="note",
content=content,
)
)
updated = await entity_service.edit_entity(
identifier=entity.permalink,
operation="insert_before_section",
content="Inserted before section 2",
section="## Section 2",
)
file_path = file_service.get_entity_path(updated)
file_content, _ = await file_service.read_file(file_path)
assert "Inserted before section 2" in file_content
assert "## Section 2" in file_content
assert "Section 2 content" in file_content
# Inserted content should appear before the section heading
assert file_content.index("Inserted before section 2") < file_content.index("## Section 2")
@pytest.mark.asyncio
async def test_edit_entity_insert_after_section(
entity_service: EntityService, file_service: FileService
):
"""Test inserting content after a section heading."""
content = dedent("""
# Main Title
## Section 1
Section 1 content
## Section 2
Section 2 content
""").strip()
entity = await entity_service.create_entity(
EntitySchema(
title="Insert After Test",
directory="docs",
note_type="note",
content=content,
)
)
updated = await entity_service.edit_entity(
identifier=entity.permalink,
operation="insert_after_section",
content="Inserted after section 1 heading",
section="## Section 1",
)
file_path = file_service.get_entity_path(updated)
file_content, _ = await file_service.read_file(file_path)
assert "Inserted after section 1 heading" in file_content
assert "## Section 1" in file_content
assert "Section 1 content" in file_content
# Inserted content should appear after the heading but content is also preserved
assert file_content.index("## Section 1") < file_content.index(
"Inserted after section 1 heading"
)
@pytest.mark.asyncio
async def test_edit_entity_insert_before_section_not_found(entity_service: EntityService):
"""Test insert_before_section raises ValueError when section not found."""
entity = await entity_service.create_entity(
EntitySchema(
title="Test Note",
directory="test",
note_type="note",
content="# Main Title\n\nSome content",
)
)
with pytest.raises(ValueError, match="Section '## Missing' not found"):
await entity_service.edit_entity(
identifier=entity.permalink,
operation="insert_before_section",
content="new content",
section="## Missing",
)
@pytest.mark.asyncio
async def test_edit_entity_insert_after_section_not_found(entity_service: EntityService):
"""Test insert_after_section raises ValueError when section not found."""
entity = await entity_service.create_entity(
EntitySchema(
title="Test Note",
directory="test",
note_type="note",
content="# Main Title\n\nSome content",
)
)
with pytest.raises(ValueError, match="Section '## Missing' not found"):
await entity_service.edit_entity(
identifier=entity.permalink,
operation="insert_after_section",
content="new content",
section="## Missing",
)
@pytest.mark.asyncio
async def test_edit_entity_insert_before_section_multiple_sections_error(
entity_service: EntityService,
):
"""Test insert_before_section raises ValueError with duplicate sections."""
entity = await entity_service.create_entity(
EntitySchema(
title="Test Note",
directory="test",
note_type="note",
content="# Title\n\n## Dup\nFirst\n\n## Dup\nSecond",
)
)
with pytest.raises(ValueError, match="Multiple sections found"):
await entity_service.edit_entity(
identifier=entity.permalink,
operation="insert_before_section",
content="new content",
section="## Dup",
)
@pytest.mark.asyncio
async def test_edit_entity_insert_before_section_missing_section_param(
entity_service: EntityService,
):
"""Test insert_before_section raises ValueError when section param is missing."""
entity = await entity_service.create_entity(
EntitySchema(
title="Test Note",
directory="test",
note_type="note",
content="# Title\n\nContent",
)
)
with pytest.raises(ValueError, match="section is required"):
await entity_service.edit_entity(
identifier=entity.permalink,
operation="insert_before_section",
content="new content",
)
@pytest.mark.asyncio
async def test_edit_entity_insert_before_section_empty_section(entity_service: EntityService):
"""Test insert_before_section raises ValueError when section is empty/whitespace."""
entity = await entity_service.create_entity(
EntitySchema(
title="Test Note",
directory="test",
note_type="note",
content="# Title\n\nContent",
)
)
with pytest.raises(ValueError, match="section cannot be empty"):
await entity_service.edit_entity(
identifier=entity.permalink,
operation="insert_before_section",
content="new content",
section=" ",
)
@pytest.mark.asyncio
async def test_edit_entity_insert_after_section_at_end_of_document(
entity_service: EntityService, file_service: FileService
):
"""Test inserting after the last section in a document."""
content = dedent("""
# Main Title
## Only Section
Some content here
""").strip()
entity = await entity_service.create_entity(
EntitySchema(
title="Insert End Test",
directory="docs",
note_type="note",
content=content,
)
)
updated = await entity_service.edit_entity(
identifier=entity.permalink,
operation="insert_after_section",
content="Inserted after the last section heading",
section="## Only Section",
)
file_path = file_service.get_entity_path(updated)
file_content, _ = await file_service.read_file(file_path)
assert "Inserted after the last section heading" in file_content
assert "## Only Section" in file_content
assert "Some content here" in file_content
@pytest.mark.asyncio
async def test_edit_entity_insert_after_section_preserves_paragraph_separation(
entity_service: EntityService, file_service: FileService
):
"""Test that insert_after_section adds blank line so inserted text doesn't merge
with existing section content into a single markdown paragraph."""
content = dedent("""
# Main Title
## Section
Existing paragraph text
""").strip()
entity = await entity_service.create_entity(
EntitySchema(
title="Paragraph Sep Test",
directory="docs",
note_type="note",
content=content,
)
)
updated = await entity_service.edit_entity(
identifier=entity.permalink,
operation="insert_after_section",
content="Inserted line",
section="## Section",
)
file_path = file_service.get_entity_path(updated)
file_content, _ = await file_service.read_file(file_path)
# The inserted line and existing content should be separated by a blank line
assert "Inserted line\n\nExisting paragraph text" in file_content
# Move entity tests
@pytest.mark.asyncio
async def test_move_entity_success(
+24 -68
View File
@@ -200,10 +200,10 @@ async def test_initialize_app_no_precedence_warning_when_not_conflicting(
@pytest.mark.asyncio
async def test_run_migrations_triggers_embedding_backfill_when_entities_exist_but_no_embeddings(
async def test_run_migrations_triggers_embedding_backfill_on_new_revision(
monkeypatch, app_config: BasicMemoryConfig
):
"""run_migrations checks for missing embeddings (actual backfill runs in background from MCP)."""
"""When the trigger revision is newly applied, run automatic embedding backfill once."""
class StubSearchRepository:
def __init__(self, *args, **kwargs):
@@ -224,24 +224,29 @@ async def test_run_migrations_triggers_embedding_backfill_when_entities_exist_bu
monkeypatch.setattr("basic_memory.db.SQLiteSearchRepository", StubSearchRepository)
monkeypatch.setattr("basic_memory.db.PostgresSearchRepository", StubSearchRepository)
needs_backfill_mock = AsyncMock(return_value=True)
monkeypatch.setattr(
"basic_memory.db._needs_semantic_embedding_backfill", needs_backfill_mock
load_revisions_mock = AsyncMock(
side_effect=[
set(),
{db.SEMANTIC_EMBEDDING_BACKFILL_REVISION},
]
)
backfill_mock = AsyncMock()
monkeypatch.setattr("basic_memory.db._load_applied_alembic_revisions", load_revisions_mock)
monkeypatch.setattr("basic_memory.db._run_semantic_embedding_backfill", backfill_mock)
await db.run_migrations(app_config)
# Verifies the check runs — backfill itself is launched by MCP lifespan
needs_backfill_mock.assert_awaited_once_with(app_config, session_marker)
assert load_revisions_mock.await_count == 2
backfill_mock.assert_awaited_once_with(app_config, session_marker)
finally:
db._session_maker = original_session_maker # pyright: ignore [reportPrivateUsage]
@pytest.mark.asyncio
async def test_run_migrations_skips_embedding_backfill_when_embeddings_already_exist(
async def test_run_migrations_skips_embedding_backfill_when_revision_already_applied(
monkeypatch, app_config: BasicMemoryConfig
):
"""When embeddings already exist, no backfill is needed."""
"""If the trigger revision was already present before upgrade, skip backfill."""
class StubSearchRepository:
def __init__(self, *args, **kwargs):
@@ -262,14 +267,20 @@ async def test_run_migrations_skips_embedding_backfill_when_embeddings_already_e
monkeypatch.setattr("basic_memory.db.SQLiteSearchRepository", StubSearchRepository)
monkeypatch.setattr("basic_memory.db.PostgresSearchRepository", StubSearchRepository)
needs_backfill_mock = AsyncMock(return_value=False)
monkeypatch.setattr(
"basic_memory.db._needs_semantic_embedding_backfill", needs_backfill_mock
load_revisions_mock = AsyncMock(
side_effect=[
{db.SEMANTIC_EMBEDDING_BACKFILL_REVISION},
{db.SEMANTIC_EMBEDDING_BACKFILL_REVISION},
]
)
backfill_mock = AsyncMock()
monkeypatch.setattr("basic_memory.db._load_applied_alembic_revisions", load_revisions_mock)
monkeypatch.setattr("basic_memory.db._run_semantic_embedding_backfill", backfill_mock)
await db.run_migrations(app_config)
needs_backfill_mock.assert_awaited_once_with(app_config, session_marker)
assert load_revisions_mock.await_count == 2
assert backfill_mock.await_count == 0
finally:
db._session_maker = original_session_maker # pyright: ignore [reportPrivateUsage]
@@ -367,58 +378,3 @@ async def test_semantic_embedding_backfill_skips_when_semantic_disabled(
app_config.semantic_search_enabled = False
await db._run_semantic_embedding_backfill(app_config, session_maker) # pyright: ignore [reportPrivateUsage]
assert called is False
@pytest.mark.asyncio
async def test_needs_semantic_embedding_backfill_true_when_entities_exist_no_embeddings(
app_config: BasicMemoryConfig,
session_maker,
test_project,
):
"""Should return True when entities exist but vector chunks table is empty."""
from basic_memory.repository.entity_repository import EntityRepository
entity_repository = EntityRepository(session_maker, project_id=test_project.id)
await entity_repository.create(
{
"title": "Test Entity",
"note_type": "note",
"entity_metadata": {},
"content_type": "text/markdown",
"file_path": "test/backfill-check.md",
"permalink": "test/backfill-check",
"project_id": test_project.id,
"created_at": datetime.now(),
"updated_at": datetime.now(),
}
)
# Clear any embeddings left by other tests in the shared DB
async with db.scoped_session(session_maker) as session:
await session.execute(db.text("DELETE FROM search_vector_chunks"))
app_config.semantic_search_enabled = True
result = await db._needs_semantic_embedding_backfill(app_config, session_maker) # pyright: ignore [reportPrivateUsage]
assert result is True
@pytest.mark.asyncio
async def test_needs_semantic_embedding_backfill_false_when_no_entities(
app_config: BasicMemoryConfig,
session_maker,
):
"""Should return False when no entities exist (nothing to backfill)."""
app_config.semantic_search_enabled = True
result = await db._needs_semantic_embedding_backfill(app_config, session_maker) # pyright: ignore [reportPrivateUsage]
assert result is False
@pytest.mark.asyncio
async def test_needs_semantic_embedding_backfill_false_when_semantic_disabled(
app_config: BasicMemoryConfig,
session_maker,
):
"""Should return False when semantic search is disabled."""
app_config.semantic_search_enabled = False
result = await db._needs_semantic_embedding_backfill(app_config, session_maker) # pyright: ignore [reportPrivateUsage]
assert result is False
Generated
+3 -9
View File
@@ -142,14 +142,14 @@ wheels = [
[[package]]
name = "authlib"
version = "1.6.7"
version = "1.6.6"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "cryptography" },
]
sdist = { url = "https://files.pythonhosted.org/packages/49/dc/ed1681bf1339dd6ea1ce56136bad4baabc6f7ad466e375810702b0237047/authlib-1.6.7.tar.gz", hash = "sha256:dbf10100011d1e1b34048c9d120e83f13b35d69a826ae762b93d2fb5aafc337b", size = 164950, upload-time = "2026-02-06T14:04:14.171Z" }
sdist = { url = "https://files.pythonhosted.org/packages/bb/9b/b1661026ff24bc641b76b78c5222d614776b0c085bcfdac9bd15a1cb4b35/authlib-1.6.6.tar.gz", hash = "sha256:45770e8e056d0f283451d9996fbb59b70d45722b45d854d58f32878d0a40c38e", size = 164894, upload-time = "2025-12-12T08:01:41.464Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/f8/00/3ed12264094ec91f534fae429945efbaa9f8c666f3aa7061cc3b2a26a0cd/authlib-1.6.7-py2.py3-none-any.whl", hash = "sha256:c637340d9a02789d2efa1d003a7437d10d3e565237bcb5fcbc6c134c7b95bab0", size = 244115, upload-time = "2026-02-06T14:04:12.141Z" },
{ url = "https://files.pythonhosted.org/packages/54/51/321e821856452f7386c4e9df866f196720b1ad0c5ea1623ea7399969ae3b/authlib-1.6.6-py2.py3-none-any.whl", hash = "sha256:7d9e9bc535c13974313a87f53e8430eb6ea3d1cf6ae4f6efcd793f2e949143fd", size = 244005, upload-time = "2025-12-12T08:01:40.209Z" },
]
[[package]]
@@ -304,16 +304,10 @@ sdist = { url = "https://files.pythonhosted.org/packages/92/88/b8527e1b00c1811db
wheels = [
{ url = "https://files.pythonhosted.org/packages/d3/25/79c98ebe12df31548ba4eaf44db11b7cad6b3e7b4203718335620939083c/caio-0.9.25-cp312-cp312-macosx_10_13_universal2.whl", hash = "sha256:fb7ff95af4c31ad3f03179149aab61097a71fd85e05f89b4786de0359dffd044", size = 36983, upload-time = "2025-12-26T15:21:36.075Z" },
{ url = "https://files.pythonhosted.org/packages/a3/2b/21288691f16d479945968a0a4f2856818c1c5be56881d51d4dac9b255d26/caio-0.9.25-cp312-cp312-manylinux2010_x86_64.manylinux2014_x86_64.manylinux_2_12_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:97084e4e30dfa598449d874c4d8e0c8d5ea17d2f752ef5e48e150ff9d240cd64", size = 82012, upload-time = "2025-12-26T15:22:20.983Z" },
{ url = "https://files.pythonhosted.org/packages/03/c4/8a1b580875303500a9c12b9e0af58cb82e47f5bcf888c2457742a138273c/caio-0.9.25-cp312-cp312-manylinux_2_34_aarch64.whl", hash = "sha256:4fa69eba47e0f041b9d4f336e2ad40740681c43e686b18b191b6c5f4c5544bfb", size = 81502, upload-time = "2026-03-04T22:08:22.381Z" },
{ url = "https://files.pythonhosted.org/packages/d1/1c/0fe770b8ffc8362c48134d1592d653a81a3d8748d764bec33864db36319d/caio-0.9.25-cp312-cp312-manylinux_2_34_x86_64.whl", hash = "sha256:6bebf6f079f1341d19f7386db9b8b1f07e8cc15ae13bfdaff573371ba0575d69", size = 80200, upload-time = "2026-03-04T22:08:23.382Z" },
{ url = "https://files.pythonhosted.org/packages/31/57/5e6ff127e6f62c9f15d989560435c642144aa4210882f9494204bc892305/caio-0.9.25-cp313-cp313-macosx_10_13_universal2.whl", hash = "sha256:d6c2a3411af97762a2b03840c3cec2f7f728921ff8adda53d7ea2315a8563451", size = 36979, upload-time = "2025-12-26T15:21:35.484Z" },
{ url = "https://files.pythonhosted.org/packages/a3/9f/f21af50e72117eb528c422d4276cbac11fb941b1b812b182e0a9c70d19c5/caio-0.9.25-cp313-cp313-manylinux2010_x86_64.manylinux2014_x86_64.manylinux_2_12_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:0998210a4d5cd5cb565b32ccfe4e53d67303f868a76f212e002a8554692870e6", size = 81900, upload-time = "2025-12-26T15:22:21.919Z" },
{ url = "https://files.pythonhosted.org/packages/9c/12/c39ae2a4037cb10ad5eb3578eb4d5f8c1a2575c62bba675f3406b7ef0824/caio-0.9.25-cp313-cp313-manylinux_2_34_aarch64.whl", hash = "sha256:1a177d4777141b96f175fe2c37a3d96dec7911ed9ad5f02bac38aaa1c936611f", size = 81523, upload-time = "2026-03-04T22:08:25.187Z" },
{ url = "https://files.pythonhosted.org/packages/22/59/f8f2e950eb4f1a5a3883e198dca514b9d475415cb6cd7b78b9213a0dd45a/caio-0.9.25-cp313-cp313-manylinux_2_34_x86_64.whl", hash = "sha256:9ed3cfb28c0e99fec5e208c934e5c157d0866aa9c32aa4dc5e9b6034af6286b7", size = 80243, upload-time = "2026-03-04T22:08:26.449Z" },
{ url = "https://files.pythonhosted.org/packages/69/ca/a08fdc7efdcc24e6a6131a93c85be1f204d41c58f474c42b0670af8c016b/caio-0.9.25-cp314-cp314-macosx_10_15_universal2.whl", hash = "sha256:fab6078b9348e883c80a5e14b382e6ad6aabbc4429ca034e76e730cf464269db", size = 36978, upload-time = "2025-12-26T15:21:41.055Z" },
{ url = "https://files.pythonhosted.org/packages/5e/6c/d4d24f65e690213c097174d26eda6831f45f4734d9d036d81790a27e7b78/caio-0.9.25-cp314-cp314-manylinux2010_x86_64.manylinux2014_x86_64.manylinux_2_12_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:44a6b58e52d488c75cfaa5ecaa404b2b41cc965e6c417e03251e868ecd5b6d77", size = 81832, upload-time = "2025-12-26T15:22:22.757Z" },
{ url = "https://files.pythonhosted.org/packages/87/a4/e534cf7d2d0e8d880e25dd61e8d921ffcfe15bd696734589826f5a2df727/caio-0.9.25-cp314-cp314-manylinux_2_34_aarch64.whl", hash = "sha256:628a630eb7fb22381dd8e3c8ab7f59e854b9c806639811fc3f4310c6bd711d79", size = 81565, upload-time = "2026-03-04T22:08:27.483Z" },
{ url = "https://files.pythonhosted.org/packages/3f/ed/bf81aeac1d290017e5e5ac3e880fd56ee15e50a6d0353986799d1bc5cfd5/caio-0.9.25-cp314-cp314-manylinux_2_34_x86_64.whl", hash = "sha256:0ba16aa605ccb174665357fc729cf500679c2d94d5f1458a6f0d5ca48f2060a7", size = 80071, upload-time = "2026-03-04T22:08:28.751Z" },
{ url = "https://files.pythonhosted.org/packages/86/93/1f76c8d1bafe3b0614e06b2195784a3765bbf7b0a067661af9e2dd47fc33/caio-0.9.25-py3-none-any.whl", hash = "sha256:06c0bb02d6b929119b1cfbe1ca403c768b2013a369e2db46bfa2a5761cf82e40", size = 19087, upload-time = "2025-12-26T15:22:00.221Z" },
]