mirror of
https://github.com/basicmachines-co/basic-memory
synced 2026-06-21 13:47:35 +00:00
Compare commits
1 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| e5a1f7e683 |
+67
-13
@@ -92,7 +92,7 @@ jobs:
|
||||
run: |
|
||||
uv pip install -e ".[dev]"
|
||||
|
||||
- name: Run tests
|
||||
- name: Run tests (SQLite Unit)
|
||||
run: |
|
||||
just test-unit-sqlite
|
||||
|
||||
@@ -139,7 +139,7 @@ jobs:
|
||||
run: |
|
||||
uv pip install -e ".[dev]"
|
||||
|
||||
- name: Run tests
|
||||
- name: Run tests (SQLite Integration)
|
||||
run: |
|
||||
just test-int-sqlite
|
||||
|
||||
@@ -150,10 +150,7 @@ jobs:
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
include:
|
||||
- python-version: "3.12"
|
||||
- python-version: "3.13"
|
||||
- python-version: "3.14"
|
||||
python-version: [ "3.12", "3.13", "3.14" ]
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
# Note: No services section needed - testcontainers handles Postgres in Docker
|
||||
@@ -183,7 +180,7 @@ jobs:
|
||||
run: |
|
||||
uv pip install -e ".[dev]"
|
||||
|
||||
- name: Run tests
|
||||
- name: Run tests (Postgres Unit)
|
||||
run: |
|
||||
just test-unit-postgres
|
||||
|
||||
@@ -194,10 +191,7 @@ jobs:
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
include:
|
||||
- python-version: "3.12"
|
||||
- python-version: "3.13"
|
||||
- python-version: "3.14"
|
||||
python-version: [ "3.12", "3.13", "3.14" ]
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
# Note: No services section needed - testcontainers handles Postgres in Docker
|
||||
@@ -227,7 +221,7 @@ jobs:
|
||||
run: |
|
||||
uv pip install -e ".[dev]"
|
||||
|
||||
- name: Run tests
|
||||
- name: Run tests (Postgres Integration)
|
||||
run: |
|
||||
just test-int-postgres
|
||||
|
||||
@@ -262,6 +256,66 @@ jobs:
|
||||
run: |
|
||||
uv pip install -e ".[dev]"
|
||||
|
||||
- name: Run tests
|
||||
- name: Run tests (Semantic)
|
||||
run: |
|
||||
just test-semantic
|
||||
|
||||
coverage:
|
||||
name: Coverage Summary (combined, Python 3.12)
|
||||
timeout-minutes: 60
|
||||
needs:
|
||||
- static-checks
|
||||
- test-sqlite-unit
|
||||
- test-sqlite-integration
|
||||
- test-postgres-unit
|
||||
- test-postgres-integration
|
||||
- test-semantic
|
||||
if: ${{ github.event_name == 'push' && github.ref == 'refs/heads/main' }}
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: true
|
||||
|
||||
- name: Set up Python 3.12
|
||||
uses: actions/setup-python@v4
|
||||
with:
|
||||
python-version: "3.12"
|
||||
cache: "pip"
|
||||
|
||||
- name: Install uv
|
||||
run: |
|
||||
pip install uv
|
||||
|
||||
- uses: extractions/setup-just@v3
|
||||
|
||||
- name: Create virtual env
|
||||
run: |
|
||||
uv venv
|
||||
|
||||
- name: Install dependencies
|
||||
run: |
|
||||
uv pip install -e ".[dev]"
|
||||
|
||||
- name: Run combined coverage (SQLite + Postgres)
|
||||
run: |
|
||||
just coverage
|
||||
|
||||
- name: Add coverage report to job summary
|
||||
if: always()
|
||||
run: |
|
||||
{
|
||||
echo "## Coverage"
|
||||
echo ""
|
||||
echo '```'
|
||||
uv run coverage report -m
|
||||
echo '```'
|
||||
} >> "$GITHUB_STEP_SUMMARY"
|
||||
|
||||
- name: Upload HTML coverage report
|
||||
if: always()
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: htmlcov
|
||||
path: htmlcov/
|
||||
|
||||
@@ -13,7 +13,7 @@
|
||||
- **Cloud is optional.** The local-first open-source workflow continues as always.
|
||||
- **OSS discount:** use code `BMFOSS` for 20% off for 3 months.
|
||||
|
||||
[Sign up now →](https://basicmemory.com?utm_source=github&utm_medium=referral&utm_campaign=readme)
|
||||
[Sign up now →](https://basicmemory.com)
|
||||
|
||||
with a 7 day free trial
|
||||
|
||||
@@ -23,9 +23,8 @@ Basic Memory lets you build persistent knowledge through natural conversations w
|
||||
Claude, while keeping everything in simple Markdown files on your computer. It uses the Model Context Protocol (MCP) to
|
||||
enable any compatible LLM to read and write to your local knowledge base.
|
||||
|
||||
- Website: [basicmemory.com](https://basicmemory.com?utm_source=github&utm_medium=referral&utm_campaign=readme)
|
||||
- Documentation: [docs.basicmemory.com](https://docs.basicmemory.com?utm_source=github&utm_medium=referral&utm_campaign=readme)
|
||||
- Community: [Discord](https://discord.gg/tyvKNccgqN?utm_source=github&utm_medium=referral&utm_campaign=readme)
|
||||
- Website: https://basicmemory.com
|
||||
- Documentation: https://docs.basicmemory.com
|
||||
|
||||
## Pick up your conversation right where you left off
|
||||
|
||||
@@ -439,7 +438,8 @@ list_directory(dir_name, depth) - Browse directory contents with filtering
|
||||
**Search & Discovery:**
|
||||
```
|
||||
search(query, page, page_size) - Search across your knowledge base
|
||||
search_notes(query, page, page_size, search_type, types, entity_types, after_date, metadata_filters, tags, status, project) - Search with filters (query is optional for filter-only searches)
|
||||
search_notes(query, page, page_size, search_type, types, entity_types, after_date, metadata_filters, tags, status, project) - Search with filters
|
||||
search_by_metadata(filters, limit, offset, project) - Structured frontmatter search
|
||||
```
|
||||
|
||||
**Project Management:**
|
||||
@@ -476,13 +476,13 @@ canvas(nodes, edges, title, folder) - Generate knowledge visualizations
|
||||
|
||||
## Futher info
|
||||
|
||||
See the [Documentation](https://docs.basicmemory.com?utm_source=github&utm_medium=referral&utm_campaign=readme) for more info, including:
|
||||
See the [Documentation](https://docs.basicmemory.com) for more info, including:
|
||||
|
||||
- [Complete User Guide](https://docs.basicmemory.com/user-guide/?utm_source=github&utm_medium=referral&utm_campaign=readme)
|
||||
- [CLI tools](https://docs.basicmemory.com/guides/cli-reference/?utm_source=github&utm_medium=referral&utm_campaign=readme)
|
||||
- [Cloud CLI and Sync](https://docs.basicmemory.com/guides/cloud-cli/?utm_source=github&utm_medium=referral&utm_campaign=readme)
|
||||
- [Managing multiple Projects](https://docs.basicmemory.com/guides/cli-reference/?utm_source=github&utm_medium=referral&utm_campaign=readme#project)
|
||||
- [Importing data from OpenAI/Claude Projects](https://docs.basicmemory.com/guides/cli-reference/?utm_source=github&utm_medium=referral&utm_campaign=readme#import)
|
||||
- [Complete User Guide](https://docs.basicmemory.com/user-guide/)
|
||||
- [CLI tools](https://docs.basicmemory.com/guides/cli-reference/)
|
||||
- [Cloud CLI and Sync](https://docs.basicmemory.com/guides/cloud-cli/)
|
||||
- [Managing multiple Projects](https://docs.basicmemory.com/guides/cli-reference/#project)
|
||||
- [Importing data from OpenAI/Claude Projects](https://docs.basicmemory.com/guides/cli-reference/#import)
|
||||
|
||||
## Telemetry
|
||||
|
||||
@@ -635,4 +635,4 @@ and submitting PRs.
|
||||
</picture>
|
||||
</a>
|
||||
|
||||
Built with ♥️ by [Basic Machines](https://basicmachines.co?utm_source=github&utm_medium=referral&utm_campaign=readme)
|
||||
Built with ♥️ by Basic Machines
|
||||
|
||||
@@ -427,8 +427,6 @@ await write_note(
|
||||
)
|
||||
```
|
||||
|
||||
> **Important**: `write_note` errors if the note already exists. Use `edit_note` for incremental changes, or pass `overwrite=True` to replace.
|
||||
|
||||
**Well-structured note**:
|
||||
|
||||
```python
|
||||
@@ -762,9 +760,6 @@ notes = await read_note(
|
||||
identifier="memory://specs/*",
|
||||
project="main"
|
||||
)
|
||||
|
||||
# Cross-project URL (auto-routes to the correct project)
|
||||
note = await read_note(identifier="memory://research/specs/api-design")
|
||||
```
|
||||
|
||||
```python
|
||||
@@ -1065,19 +1060,16 @@ results = await search_notes(
|
||||
project="main"
|
||||
)
|
||||
|
||||
# Metadata-only search (no query needed)
|
||||
results = await search_notes(
|
||||
metadata_filters={"type": "spec", "status": "in-progress"},
|
||||
# Metadata-only search
|
||||
results = await search_by_metadata(
|
||||
filters={"type": "spec", "status": "in-progress"},
|
||||
project="main"
|
||||
)
|
||||
```
|
||||
|
||||
### Search Types
|
||||
|
||||
Available types: `"text"`, `"title"`, `"permalink"`, `"vector"`/`"semantic"`, `"hybrid"`.
|
||||
Default is `"hybrid"` when semantic search is enabled, `"text"` otherwise.
|
||||
|
||||
**Text search**:
|
||||
**Text search (default)**:
|
||||
|
||||
```python
|
||||
# Full-text search across all content
|
||||
@@ -1088,52 +1080,17 @@ results = await search_notes(
|
||||
)
|
||||
```
|
||||
|
||||
**Title and permalink search**:
|
||||
|
||||
```python
|
||||
# Search by title only
|
||||
results = await search_notes(query="API Design", search_type="title", project="main")
|
||||
|
||||
# Search by permalink
|
||||
results = await search_notes(query="specs/api-design", search_type="permalink", project="main")
|
||||
```
|
||||
|
||||
**Semantic/vector search**:
|
||||
**Semantic search**:
|
||||
|
||||
```python
|
||||
# Semantic/vector search (if enabled)
|
||||
results = await search_notes(
|
||||
query="user login security",
|
||||
search_type="semantic", # or "vector"
|
||||
project="main"
|
||||
)
|
||||
|
||||
# Override similarity threshold
|
||||
results = await search_notes(
|
||||
query="user login security",
|
||||
search_type="semantic",
|
||||
min_similarity=0.5,
|
||||
project="main"
|
||||
)
|
||||
```
|
||||
|
||||
**Hybrid search** (combines text + semantic):
|
||||
|
||||
```python
|
||||
results = await search_notes(
|
||||
query="authentication best practices",
|
||||
search_type="hybrid",
|
||||
project="main"
|
||||
)
|
||||
```
|
||||
|
||||
**Tag shorthand in query**:
|
||||
|
||||
```python
|
||||
# Use tag: prefix as shorthand
|
||||
results = await search_notes(query="tag:security", project="main")
|
||||
```
|
||||
|
||||
### Search Response
|
||||
|
||||
**Result structure**:
|
||||
@@ -2204,31 +2161,6 @@ active_project = projects[0]["name"]
|
||||
results = await search_notes(query="test", project=active_project)
|
||||
```
|
||||
|
||||
### Note Already Exists
|
||||
|
||||
**Error**: `write_note` called for a note that already exists
|
||||
|
||||
**Solution**:
|
||||
|
||||
```python
|
||||
# Preferred: use edit_note for incremental updates
|
||||
await edit_note(
|
||||
identifier="Existing Topic",
|
||||
operation="append",
|
||||
content="\n- [update] new information",
|
||||
project="main"
|
||||
)
|
||||
|
||||
# Alternative: replace the entire note
|
||||
await write_note(
|
||||
title="Existing Topic",
|
||||
content="# Existing Topic\n...",
|
||||
folder="notes",
|
||||
overwrite=True,
|
||||
project="main"
|
||||
)
|
||||
```
|
||||
|
||||
### Entity Not Found
|
||||
|
||||
**Error**: Note doesn't exist
|
||||
@@ -2784,15 +2716,14 @@ await write_note(
|
||||
|
||||
### Content Management
|
||||
|
||||
**write_note(title, content, folder, tags, note_type, overwrite, project)**
|
||||
- Create new markdown notes (errors if note already exists unless overwrite=True)
|
||||
**write_note(title, content, folder, tags, note_type, project)**
|
||||
- Create or update markdown notes
|
||||
- Parameters:
|
||||
- `title` (required): Note title
|
||||
- `content` (required): Markdown content
|
||||
- `folder` (required): Destination folder
|
||||
- `tags` (optional): List of tags
|
||||
- `note_type` (optional): Type of note (stored in frontmatter). Can be "note", "person", "meeting", "guide", etc.
|
||||
- `overwrite` (optional): Set to True to replace an existing note (default: error if exists)
|
||||
- `project` (required unless default_project_mode): Target project
|
||||
- Returns: Created/updated entity with permalink
|
||||
- Example:
|
||||
@@ -2959,20 +2890,19 @@ contents = await list_directory(
|
||||
|
||||
### Search & Discovery
|
||||
|
||||
**search_notes(query, page, page_size, search_type, types, entity_types, after_date, metadata_filters, tags, status, min_similarity, project)**
|
||||
**search_notes(query, page, page_size, search_type, types, entity_types, after_date, metadata_filters, tags, status, project)**
|
||||
- Search across knowledge base
|
||||
- Parameters:
|
||||
- `query` (optional): Search query (not required for filter-only searches)
|
||||
- `query` (required): Search query
|
||||
- `page` (optional): Page number (default: 1)
|
||||
- `page_size` (optional): Results per page (default: 10)
|
||||
- `search_type` (optional): "text", "title", "permalink", "vector"/"semantic", "hybrid" (default: "hybrid" when semantic enabled, "text" otherwise)
|
||||
- `search_type` (optional): "text" or "semantic"
|
||||
- `types` (optional): Entity type filter
|
||||
- `entity_types` (optional): Observation category filter
|
||||
- `after_date` (optional): Date filter (ISO format)
|
||||
- `metadata_filters` (optional): Structured frontmatter filters (dict, supports `$in`, `$gt`, `$gte`, `$lt`, `$lte`, `$between` operators)
|
||||
- `tags` (optional): Frontmatter tags filter (list); also available via `tag:` query shorthand
|
||||
- `metadata_filters` (optional): Structured frontmatter filters (dict)
|
||||
- `tags` (optional): Frontmatter tags filter (list)
|
||||
- `status` (optional): Frontmatter status filter (string)
|
||||
- `min_similarity` (optional): Override similarity threshold for vector/hybrid search
|
||||
- `project` (required unless default_project_mode): Target project
|
||||
- Returns: Matching entities with scores
|
||||
- Example:
|
||||
@@ -2985,11 +2915,18 @@ results = await search_notes(
|
||||
)
|
||||
```
|
||||
|
||||
**Metadata-only search (via search_notes)**
|
||||
- Use `search_notes` with `metadata_filters` and no `query` for metadata-only searches:
|
||||
**search_by_metadata(filters, limit, offset, project)**
|
||||
- Metadata-only search using structured frontmatter
|
||||
- Parameters:
|
||||
- `filters` (required): Dict of field -> value (supports $in, $gt/$gte/$lt/$lte, $between)
|
||||
- `limit` (optional): Max results (default: 20)
|
||||
- `offset` (optional): Pagination offset (default: 0)
|
||||
- `project` (required unless default_project_mode): Target project
|
||||
- Returns: Matching entities
|
||||
- Example:
|
||||
```python
|
||||
results = await search_notes(
|
||||
metadata_filters={"type": "spec", "status": "in-progress"},
|
||||
results = await search_by_metadata(
|
||||
filters={"type": "spec", "status": "in-progress"},
|
||||
project="main"
|
||||
)
|
||||
```
|
||||
@@ -3041,15 +2978,6 @@ await delete_project(project_name="old-project")
|
||||
status = await sync_status(project="main")
|
||||
```
|
||||
|
||||
**list_workspaces()**
|
||||
- List available workspaces (cloud)
|
||||
- Parameters: None
|
||||
- Returns: List of workspaces with metadata
|
||||
- Example:
|
||||
```python
|
||||
workspaces = await list_workspaces()
|
||||
```
|
||||
|
||||
### Visualization
|
||||
|
||||
**canvas(nodes, edges, title, folder, project)**
|
||||
@@ -3318,8 +3246,8 @@ await edit_note(
|
||||
project="main"
|
||||
)
|
||||
|
||||
# When full rewrite is needed, use overwrite=True
|
||||
await write_note(title="Note", content="...", folder="notes", overwrite=True)
|
||||
# Avoid: Complete rewrite
|
||||
# (unless necessary for major restructuring)
|
||||
```
|
||||
|
||||
### 14. Tagging Strategy
|
||||
|
||||
+47
-13
@@ -2,9 +2,14 @@
|
||||
|
||||
Basic Memory automatically indexes custom frontmatter fields so you can query them with structured filters. Any YAML key in a note's frontmatter beyond the standard set (`title`, `type`, `tags`, `permalink`, `schema`) is stored as `entity_metadata` and becomes searchable.
|
||||
|
||||
## Querying with `search_notes`
|
||||
## Two Ways to Query
|
||||
|
||||
`search_notes` is the single search tool for all queries — text, metadata filters, or both. The `query` parameter is optional, so you can use metadata filters alone without passing an empty string.
|
||||
| Tool | Use When |
|
||||
|------|----------|
|
||||
| `search_by_metadata` | You only need metadata filters (no text query) |
|
||||
| `search_notes` | You want to combine a text query with metadata filters |
|
||||
|
||||
Both tools accept the same filter syntax.
|
||||
|
||||
## Filter Syntax
|
||||
|
||||
@@ -83,16 +88,45 @@ This queries the `version` key inside a `schema` object in frontmatter.
|
||||
- `$in` and array-contains require non-empty lists.
|
||||
- `$between` requires exactly two values `[min, max]`.
|
||||
|
||||
## MCP Tool — `search_notes`
|
||||
## MCP Tools
|
||||
|
||||
`search_notes` is the single search tool for text queries, metadata filters, or both. The `query` parameter is optional.
|
||||
### `search_by_metadata` — metadata-only search
|
||||
|
||||
Searches entities by structured frontmatter metadata without a text query. Results are scoped to entity-level items.
|
||||
|
||||
**Parameters:**
|
||||
|
||||
| Parameter | Type | Required | Description |
|
||||
|-----------|------|----------|-------------|
|
||||
| `filters` | dict | Yes | Metadata filter dictionary (see syntax above) |
|
||||
| `project` | string | No | Project to search in (uses default if omitted) |
|
||||
| `limit` | int | No | Max results (default 20) |
|
||||
| `offset` | int | No | Skip N results for pagination (default 0) |
|
||||
|
||||
**Example:**
|
||||
|
||||
```python
|
||||
# Find all notes with status "in-progress"
|
||||
await search_by_metadata({"status": "in-progress"})
|
||||
|
||||
# Find high-priority specs in the research project
|
||||
await search_by_metadata(
|
||||
{"type": "spec", "priority": {"$in": ["high", "critical"]}},
|
||||
project="research",
|
||||
limit=10,
|
||||
)
|
||||
```
|
||||
|
||||
### `search_notes` with metadata — combined text + metadata
|
||||
|
||||
The `search_notes` tool accepts `metadata_filters`, `tags`, and `status` parameters alongside the text `query`. This lets you combine full-text search with structured filtering.
|
||||
|
||||
**Relevant parameters:**
|
||||
|
||||
| Parameter | Type | Description |
|
||||
|-----------|------|-------------|
|
||||
| `query` | string (optional) | Text search query. Omit for filter-only searches. |
|
||||
| `metadata_filters` | dict | Structured filter dict (see syntax above) |
|
||||
| `query` | string | Text search query (can be empty when using only filters) |
|
||||
| `metadata_filters` | dict | Structured filter dict (same syntax as `search_by_metadata`) |
|
||||
| `tags` | list[str] | Convenience shorthand — merged into `metadata_filters["tags"]` |
|
||||
| `status` | string | Convenience shorthand — merged into `metadata_filters["status"]` |
|
||||
|
||||
@@ -104,8 +138,8 @@ This queries the `version` key inside a `schema` object in frontmatter.
|
||||
# Text search filtered by metadata
|
||||
await search_notes("authentication", metadata_filters={"status": "draft"})
|
||||
|
||||
# Filter-only search (no query needed)
|
||||
await search_notes(metadata_filters={"type": "spec"})
|
||||
# Filter-only search (empty query)
|
||||
await search_notes("", metadata_filters={"type": "spec"})
|
||||
|
||||
# Combine text, tags shortcut, and metadata
|
||||
await search_notes(
|
||||
@@ -116,7 +150,7 @@ await search_notes(
|
||||
|
||||
# Convenience shortcuts
|
||||
await search_notes("planning", status="active")
|
||||
await search_notes(tags=["tier1", "alpha"])
|
||||
await search_notes("", tags=["tier1", "alpha"])
|
||||
```
|
||||
|
||||
## Tag Search Shortcuts
|
||||
@@ -226,19 +260,19 @@ confidence: 0.6
|
||||
|
||||
```python
|
||||
# Find all in-progress specs
|
||||
await search_notes(metadata_filters={"status": "in-progress", "type": "spec"})
|
||||
await search_by_metadata({"status": "in-progress", "type": "spec"})
|
||||
# → Auth Design
|
||||
|
||||
# Find high-confidence specs
|
||||
await search_notes(metadata_filters={"confidence": {"$gt": 0.7}})
|
||||
await search_by_metadata({"confidence": {"$gt": 0.7}})
|
||||
# → Auth Design (confidence: 0.85)
|
||||
|
||||
# Find specs with priority high or medium
|
||||
await search_notes(metadata_filters={"priority": {"$in": ["high", "medium"]}})
|
||||
await search_by_metadata({"priority": {"$in": ["high", "medium"]}})
|
||||
# → Auth Design, Search Redesign
|
||||
|
||||
# Find specs in a confidence range
|
||||
await search_notes(metadata_filters={"confidence": {"$between": [0.5, 0.9]}})
|
||||
await search_by_metadata({"confidence": {"$between": [0.5, 0.9]}})
|
||||
# → Auth Design (0.85), Search Redesign (0.6)
|
||||
|
||||
# Find notes tagged with security
|
||||
|
||||
+14
-89
@@ -4,7 +4,7 @@
|
||||
|
||||
v0.19.0 is a major release that introduces semantic vector search, a schema validation system,
|
||||
project-prefixed permalinks, per-project cloud routing, and a significant upgrade to FastMCP 3.0.
|
||||
It includes 90+ commits since v0.18.0 spanning new features, architectural improvements, and
|
||||
It includes 66 commits since v0.18.0 spanning new features, architectural improvements, and
|
||||
stability fixes across both SQLite and Postgres backends.
|
||||
|
||||
---
|
||||
@@ -16,7 +16,6 @@ stability fixes across both SQLite and Postgres backends.
|
||||
Full vector and hybrid search for SQLite (via sqlite-vec) and Postgres (via pgvector).
|
||||
|
||||
- **Hybrid search mode** combines full-text search (FTS) with vector similarity for best results
|
||||
- **Score-based fusion** replaces RRF for hybrid ranking — `max(vec, fts) + 0.3 * min(vec, fts)` preserves dominant signals and rewards dual-source agreement (#577)
|
||||
- **Default search mode** is now `hybrid` when semantic search is enabled, `text` when disabled
|
||||
- Embedding providers: FastEmbed (local, default) or OpenAI API
|
||||
- Configurable similarity threshold via `semantic_min_similarity` (default 0.55)
|
||||
@@ -24,7 +23,6 @@ Full vector and hybrid search for SQLite (via sqlite-vec) and Postgres (via pgve
|
||||
- Auto-backfill: existing entities get embeddings generated on first startup
|
||||
- Backend-specific distance-to-similarity conversion (cosine for SQLite, inner product for Postgres)
|
||||
- FTS fallback: if semantic dependencies are missing, search gracefully degrades to text-only
|
||||
- sqlite-vec knn `k` parameter capped at 4096 to prevent backend errors
|
||||
|
||||
**Configuration:**
|
||||
```json
|
||||
@@ -86,29 +84,6 @@ Cloud projects can target specific workspaces for multi-tenant environments.
|
||||
|
||||
## New Tools and Capabilities
|
||||
|
||||
### Dashboard (`bm project info`)
|
||||
|
||||
`bm project info` now displays an htop-inspired compact dashboard with:
|
||||
|
||||
- Horizontal bar charts for note types (top 5)
|
||||
- Embedding coverage bar with Unicode block characters
|
||||
- Colored status dots for at-a-glance health
|
||||
- `EmbeddingStatus` schema and `get_embedding_status()` service method for programmatic access
|
||||
|
||||
### Unified Metadata Search
|
||||
|
||||
`search_by_metadata` has been merged into `search_notes` — one tool for all searches.
|
||||
`query` is now optional, so you can search purely by frontmatter metadata.
|
||||
|
||||
```
|
||||
search_notes(metadata_filters={"status": "in-progress"})
|
||||
search_notes(metadata_filters={"tags": ["security", "oauth"]})
|
||||
search_notes(metadata_filters={"priority": {"$in": ["high", "critical"]}})
|
||||
search_notes(metadata_filters={"schema.confidence": {"$gt": 0.7}})
|
||||
search_notes(tags=["security"]) # convenience shorthand
|
||||
search_notes(status="draft") # convenience shorthand
|
||||
```
|
||||
|
||||
### JSON Output Mode
|
||||
|
||||
All MCP tools now support `output_format="json"` for machine-readable responses.
|
||||
@@ -117,6 +92,17 @@ All MCP tools now support `output_format="json"` for machine-readable responses.
|
||||
- `build_context` defaults to `"json"` with slimmed payloads (redundant fields stripped)
|
||||
- CLI tool commands support `--format json` flag
|
||||
|
||||
### Structured Metadata Search
|
||||
|
||||
New `search_by_metadata` tool for searching by frontmatter fields.
|
||||
|
||||
```
|
||||
search_by_metadata({"status": "in-progress"})
|
||||
search_by_metadata({"tags": ["security", "oauth"]})
|
||||
search_by_metadata({"priority": {"$in": ["high", "critical"]}})
|
||||
search_by_metadata({"schema.confidence": {"$gt": 0.7}})
|
||||
```
|
||||
|
||||
### `tag:` Search Shorthand
|
||||
|
||||
Search by tag using convenient shorthand syntax.
|
||||
@@ -130,32 +116,14 @@ search_notes("tag:coffee AND tag:brewing")
|
||||
|
||||
Entities now track `created_by` and `last_updated_by` fields for attribution.
|
||||
|
||||
### Improved Search Result Content (#609)
|
||||
### Matched Chunk Text in Search
|
||||
|
||||
Search results now surface more relevant context:
|
||||
|
||||
- `matched_chunk_text` populated for FTS-only hybrid results (no more fallback to truncated content)
|
||||
- `TOP_CHUNKS_PER_RESULT` increased from 3 to 5, catching answers deeper in large notes (~2700 → ~4500 chars)
|
||||
- `CONTENT_DISPLAY_LIMIT` doubled from 2000 to 4000 chars for results without matched chunks
|
||||
|
||||
### `write_note` Overwrite Guard (#632)
|
||||
|
||||
`write_note` is now non-idempotent by default. If a note already exists, the tool returns an
|
||||
error instead of silently overwriting. Pass `overwrite=True` to replace, or use `edit_note`
|
||||
for incremental updates. Config option `write_note_overwrite_default` restores the old upsert
|
||||
behavior.
|
||||
Search results now include `matched_chunk` field showing the specific text that matched.
|
||||
|
||||
---
|
||||
|
||||
## Architecture Changes
|
||||
|
||||
### Score-Based Hybrid Fusion (#577)
|
||||
|
||||
RRF (Reciprocal Rank Fusion) compressed all fused scores to ~0.016, destroying ranking
|
||||
differentiation. The new formula `max(vec, fts) + FUSION_BONUS * min(vec, fts)` preserves
|
||||
dominant signals and rewards dual-source agreement. Zero-score results now produce zero
|
||||
fused score instead of receiving a 0.1 weight floor.
|
||||
|
||||
### FastMCP 3.0 Upgrade
|
||||
|
||||
Upgraded from FastMCP 2.12.3 to 3.0.1.
|
||||
@@ -208,19 +176,6 @@ Docker-internal paths that don't exist locally.
|
||||
All `bm tool` subcommands support `--format json` for machine-readable output, enabling
|
||||
integration with scripts and plugins.
|
||||
|
||||
### `--json` for Top-Level CLI Commands
|
||||
|
||||
Five additional CLI commands now support `--json` for machine-readable output:
|
||||
|
||||
- `bm status --json` — sync report with new/modified/deleted/moved files and skipped files
|
||||
- `bm project list --json` — structured project list with name, paths, routing mode, and defaults
|
||||
- `bm schema validate --json` — validation report with per-note pass/fail, warnings, and errors
|
||||
- `bm schema infer --json` — field frequency analysis and suggested schema definition
|
||||
- `bm schema diff --json` — drift report with new fields, dropped fields, and cardinality changes
|
||||
|
||||
This complements the existing `bm project info --json` and `bm tool --format json` support,
|
||||
making all major CLI commands scriptable for CI pipelines and automation.
|
||||
|
||||
### Cloud Promo and Analytics
|
||||
|
||||
- Cloud promo panel shown on first run or version bump with OSS discount code
|
||||
@@ -233,7 +188,6 @@ making all major CLI commands scriptable for CI pipelines and automation.
|
||||
|
||||
## Bug Fixes
|
||||
|
||||
- **#577**: RRF fusion compressed all hybrid scores to ~0.016, destroying ranking differentiation
|
||||
- **#582**: build_context returns empty results on valid note identifiers
|
||||
- **#575**: Remove hardcoded "main" default from default_project
|
||||
- **#595**: recent_activity dedup and pagination across MCP tools
|
||||
@@ -246,19 +200,6 @@ making all major CLI commands scriptable for CI pipelines and automation.
|
||||
- **#533**: Fix recent_activity prompt defaults
|
||||
- **#530**: Prevent spurious `metadata: {}` in frontmatter output
|
||||
- **#601**: Return matched chunk text in search results
|
||||
- **#606**: Accept `null` for `expected_replacements` in `edit_note`
|
||||
- **#579, #607**: Guard against closed streams in promo panel and missing vector tables on shutdown
|
||||
- **#609**: FTS-only hybrid results missing `matched_chunk_text`; content limits too conservative
|
||||
- **#631**: `build_context` related_results schema validation failure — replaced fragile `_slim_context()` stripping with Pydantic `exclude=True` field config
|
||||
- **#630**: Skip workspace resolution when client factory is active — prevents 401 errors in cloud MCP server mode
|
||||
- **#30**: `tag:` prefix query fails with hybrid search — moved tag prefix parsing to MCP tool level so it works with all search modes
|
||||
- **#31**: `search_notes` returns cluttered observation/relation-level results — now defaults to entity-level results
|
||||
- **#28**: `schema_infer` and `schema_diff` return raw Pydantic models as "undefined" in LLM output — added markdown formatters
|
||||
- Fix `schema_validate` identifier resolution (now uses LinkResolver) and text rendering (markdown formatter)
|
||||
- **#634**: `schema_validate` and `schema_diff` use stale database metadata instead of reading schema definitions from file — now reads frontmatter directly from the file with fallback to database metadata
|
||||
- Fix `Post(**metadata)` crash when frontmatter contains `content` or `handler` keys
|
||||
- Fix list-valued frontmatter fields (`title`, `type`) crashing on `.strip()` — now coerced to strings
|
||||
- Cap sqlite-vec knn `k` parameter at 4096 to prevent backend errors
|
||||
- Parameterize SQL queries in search repository type filters
|
||||
- Double-default display in project list
|
||||
- `ensure_frontmatter_on_sync` default changed to `True`
|
||||
@@ -267,13 +208,6 @@ making all major CLI commands scriptable for CI pipelines and automation.
|
||||
|
||||
---
|
||||
|
||||
## Security
|
||||
|
||||
- Upgrade `cryptography` for CVE advisory
|
||||
- Upgrade `python-multipart` for security advisory
|
||||
|
||||
---
|
||||
|
||||
## Internal / Developer
|
||||
|
||||
- **#598**: Upgrade FastMCP 2.12.3 → 3.0.1 with tool annotations
|
||||
@@ -283,8 +217,6 @@ making all major CLI commands scriptable for CI pipelines and automation.
|
||||
- **#596**: Fix CLI runtime defects and audit regressions
|
||||
- CLI refactoring and workspace-aware cloud project listing
|
||||
- Split and speed up PR test matrix in CI
|
||||
- Fix CI: collect coverage from test jobs instead of re-running all tests
|
||||
- Create `search_vector_chunks` in test fixtures for Postgres compatibility
|
||||
|
||||
---
|
||||
|
||||
@@ -303,16 +235,9 @@ making all major CLI commands scriptable for CI pipelines and automation.
|
||||
- **Semantic search dependencies** are now included by default. If sqlite-vec fails to load,
|
||||
search gracefully falls back to FTS. Run `bm reindex --embeddings` to generate embeddings
|
||||
for existing content.
|
||||
- **Hybrid search scoring** has changed from RRF to score-based fusion. Search result ordering
|
||||
may differ — results should be more accurate with better score differentiation.
|
||||
- **`search_by_metadata`** is removed as a standalone tool. Use `search_notes` with
|
||||
`metadata_filters` instead (same parameters, same behavior).
|
||||
- **Project-prefixed permalinks** are enabled by default. Existing notes keep their current
|
||||
permalinks until modified. Set `permalinks_include_project: false` to disable.
|
||||
- **Frontmatter on sync** is now enabled by default. Files without frontmatter will have it
|
||||
added on next sync. Set `ensure_frontmatter_on_sync: false` to preserve old behavior.
|
||||
- **Config migration** runs automatically for cloud projects with bisync — `local_sync_path`
|
||||
is promoted to `path` so filesystem operations work correctly.
|
||||
- **`write_note` is no longer idempotent** — calls to `write_note` for existing notes now
|
||||
return an error unless `overwrite=True` is passed. Use `edit_note` for incremental changes,
|
||||
or set `write_note_overwrite_default: true` in config to restore the old behavior.
|
||||
|
||||
@@ -165,13 +165,13 @@ Returns results ranked by cosine similarity. Individual observations and relatio
|
||||
|
||||
### `hybrid`
|
||||
|
||||
Combines FTS and vector results using score-based fusion. This is generally the best mode when you want both keyword precision and semantic recall.
|
||||
Combines FTS and vector results using reciprocal rank fusion (RRF). This is generally the best mode when you want both keyword precision and semantic recall.
|
||||
|
||||
```python
|
||||
search_notes("authentication security", search_type="hybrid")
|
||||
```
|
||||
|
||||
Score-based fusion uses the formula `max(vec, fts) + bonus * min(vec, fts)` to preserve the dominant signal while rewarding results found by both methods.
|
||||
RRF merges the two ranked lists so that items appearing in both get a score boost, while items found by only one method still appear.
|
||||
|
||||
### When to Use Which
|
||||
|
||||
@@ -236,14 +236,14 @@ Each chunk has a `source_hash` (SHA-256 of the chunk text). On re-sync, unchange
|
||||
|
||||
### Hybrid Fusion
|
||||
|
||||
Hybrid search uses score-based fusion to merge FTS and vector results:
|
||||
Hybrid search uses reciprocal rank fusion (RRF) to merge FTS and vector results:
|
||||
|
||||
1. Run FTS search to get keyword-ranked results; normalize scores to [0, 1]
|
||||
2. Run vector search to get similarity-ranked results (already [0, 1])
|
||||
3. For each result, compute: `fused = max(vec_score, fts_score) + 0.3 * min(vec_score, fts_score)`
|
||||
1. Run FTS search to get keyword-ranked results
|
||||
2. Run vector search to get similarity-ranked results
|
||||
3. For each result, compute: `score = 1/(k + fts_rank) + 1/(k + vector_rank)` where `k = 60`
|
||||
4. Sort by fused score
|
||||
|
||||
The dominant signal (whichever source scored higher) is preserved, and dual-source agreement adds a bonus. Unlike rank-based fusion, this approach retains score magnitude — a strong vector match stays strong even without an FTS hit.
|
||||
Items found by both methods get a natural score boost. Items found by only one method still appear but rank lower.
|
||||
|
||||
### Observation-Level Results
|
||||
|
||||
|
||||
@@ -69,24 +69,6 @@ testmon *args:
|
||||
test-smoke:
|
||||
BASIC_MEMORY_ENV=test uv run pytest -p pytest_mock -v --no-cov -m smoke test-int/mcp/test_smoke_integration.py
|
||||
|
||||
# Run graph intelligence API contract tests only
|
||||
test-graph-intel-api:
|
||||
BASIC_MEMORY_ENV=test uv run pytest -p pytest_mock -v --no-cov tests/api/v2/test_graph_intelligence_router.py
|
||||
|
||||
# Run graph intelligence MCP tests only
|
||||
test-graph-intel-mcp:
|
||||
BASIC_MEMORY_ENV=test uv run pytest -p pytest_mock -v --no-cov tests/mcp/clients/test_graph_clients.py tests/mcp/test_tool_graph_intelligence.py tests/mcp/test_tool_contracts.py
|
||||
|
||||
# Run graph intelligence CLI passthrough tests only
|
||||
test-graph-intel-cli:
|
||||
BASIC_MEMORY_ENV=test uv run pytest -p pytest_mock -v --no-cov tests/cli/test_cli_tool_graph_intelligence_json_output.py
|
||||
|
||||
# Run the full graph intelligence fast iteration slice
|
||||
test-graph-intel:
|
||||
just test-graph-intel-api
|
||||
just test-graph-intel-mcp
|
||||
just test-graph-intel-cli
|
||||
|
||||
# Fast local loop: lint, format, typecheck, impacted tests
|
||||
fast-check:
|
||||
just fix
|
||||
|
||||
@@ -22,7 +22,10 @@ def table_exists(connection, table_name: str) -> bool:
|
||||
"""Check if a table exists (idempotent migration support)."""
|
||||
if connection.dialect.name == "postgresql":
|
||||
result = connection.execute(
|
||||
text("SELECT 1 FROM information_schema.tables WHERE table_name = :table_name"),
|
||||
text(
|
||||
"SELECT 1 FROM information_schema.tables "
|
||||
"WHERE table_name = :table_name"
|
||||
),
|
||||
{"table_name": table_name},
|
||||
)
|
||||
return result.fetchone() is not None
|
||||
|
||||
@@ -19,8 +19,6 @@ from basic_memory.api.v2.routers import (
|
||||
prompt_router as v2_prompt,
|
||||
importer_router as v2_importer,
|
||||
schema_router as v2_schema,
|
||||
graph_router as v2_graph,
|
||||
fcm_router as v2_fcm,
|
||||
)
|
||||
from basic_memory.api.v2.routers.project_router import (
|
||||
add_project,
|
||||
@@ -88,8 +86,6 @@ app.include_router(v2_directory, prefix="/v2/projects/{project_id}")
|
||||
app.include_router(v2_prompt, prefix="/v2/projects/{project_id}")
|
||||
app.include_router(v2_importer, prefix="/v2/projects/{project_id}")
|
||||
app.include_router(v2_schema, prefix="/v2/projects/{project_id}")
|
||||
app.include_router(v2_graph, prefix="/v2/projects/{project_id}")
|
||||
app.include_router(v2_fcm, prefix="/v2/projects/{project_id}")
|
||||
app.include_router(v2_project, prefix="/v2")
|
||||
|
||||
# Legacy web app proxy paths (compat with /proxy/projects/projects)
|
||||
|
||||
@@ -21,8 +21,6 @@ from basic_memory.api.v2.routers import (
|
||||
directory_router,
|
||||
prompt_router,
|
||||
importer_router,
|
||||
graph_router,
|
||||
fcm_router,
|
||||
)
|
||||
|
||||
__all__ = [
|
||||
@@ -34,6 +32,4 @@ __all__ = [
|
||||
"directory_router",
|
||||
"prompt_router",
|
||||
"importer_router",
|
||||
"graph_router",
|
||||
"fcm_router",
|
||||
]
|
||||
|
||||
@@ -9,8 +9,6 @@ from basic_memory.api.v2.routers.directory_router import router as directory_rou
|
||||
from basic_memory.api.v2.routers.prompt_router import router as prompt_router
|
||||
from basic_memory.api.v2.routers.importer_router import router as importer_router
|
||||
from basic_memory.api.v2.routers.schema_router import router as schema_router
|
||||
from basic_memory.api.v2.routers.graph_router import router as graph_router
|
||||
from basic_memory.api.v2.routers.fcm_router import router as fcm_router
|
||||
|
||||
__all__ = [
|
||||
"knowledge_router",
|
||||
@@ -22,6 +20,4 @@ __all__ = [
|
||||
"prompt_router",
|
||||
"importer_router",
|
||||
"schema_router",
|
||||
"graph_router",
|
||||
"fcm_router",
|
||||
]
|
||||
|
||||
@@ -1,61 +0,0 @@
|
||||
"""V2 router for FCM simulation and interop endpoints."""
|
||||
|
||||
from fastapi import APIRouter
|
||||
|
||||
from basic_memory.deps import FCMServiceV2ExternalDep, ProjectExternalIdPathDep
|
||||
from basic_memory.schemas.graph_intelligence import (
|
||||
FCMExportRequest,
|
||||
FCMExportResponse,
|
||||
FCMImportRequest,
|
||||
FCMImportResponse,
|
||||
FCMRankActionsRequest,
|
||||
FCMRankActionsResponse,
|
||||
FCMSimulateRequest,
|
||||
FCMSimulateResponse,
|
||||
)
|
||||
|
||||
router = APIRouter(prefix="/fcm", tags=["fcm-v2"])
|
||||
|
||||
|
||||
@router.post("/simulate", response_model=FCMSimulateResponse)
|
||||
async def fcm_simulate(
|
||||
request: FCMSimulateRequest,
|
||||
fcm_service: FCMServiceV2ExternalDep,
|
||||
project_id: ProjectExternalIdPathDep,
|
||||
) -> FCMSimulateResponse:
|
||||
"""Run an FCM scenario simulation."""
|
||||
_ = project_id
|
||||
return await fcm_service.simulate(request)
|
||||
|
||||
|
||||
@router.post("/rank-actions", response_model=FCMRankActionsResponse)
|
||||
async def fcm_rank_actions(
|
||||
request: FCMRankActionsRequest,
|
||||
fcm_service: FCMServiceV2ExternalDep,
|
||||
project_id: ProjectExternalIdPathDep,
|
||||
) -> FCMRankActionsResponse:
|
||||
"""Rank action candidates toward a goal."""
|
||||
_ = project_id
|
||||
return await fcm_service.rank_actions(request)
|
||||
|
||||
|
||||
@router.post("/import", response_model=FCMImportResponse)
|
||||
async def fcm_import(
|
||||
request: FCMImportRequest,
|
||||
fcm_service: FCMServiceV2ExternalDep,
|
||||
project_id: ProjectExternalIdPathDep,
|
||||
) -> FCMImportResponse:
|
||||
"""Import an FCM model using a supported interchange format."""
|
||||
_ = project_id
|
||||
return await fcm_service.import_model(request)
|
||||
|
||||
|
||||
@router.post("/export", response_model=FCMExportResponse)
|
||||
async def fcm_export(
|
||||
request: FCMExportRequest,
|
||||
fcm_service: FCMServiceV2ExternalDep,
|
||||
project_id: ProjectExternalIdPathDep,
|
||||
) -> FCMExportResponse:
|
||||
"""Export an FCM model using a supported interchange format."""
|
||||
_ = project_id
|
||||
return await fcm_service.export_model(request)
|
||||
@@ -1,71 +0,0 @@
|
||||
"""V2 router for graph intelligence endpoints."""
|
||||
|
||||
from fastapi import APIRouter, Query
|
||||
|
||||
from basic_memory.deps import (
|
||||
GraphIntelligenceServiceV2ExternalDep,
|
||||
ProjectExternalIdPathDep,
|
||||
TaskSchedulerDep,
|
||||
)
|
||||
from basic_memory.schemas.graph_intelligence import (
|
||||
GraphHealthResponse,
|
||||
GraphImpactRequest,
|
||||
GraphImpactResponse,
|
||||
GraphLineageRequest,
|
||||
GraphLineageResponse,
|
||||
GraphReindexRequest,
|
||||
GraphReindexResponse,
|
||||
)
|
||||
|
||||
router = APIRouter(prefix="/graph", tags=["graph-v2"])
|
||||
|
||||
|
||||
@router.post("/lineage", response_model=GraphLineageResponse)
|
||||
async def graph_lineage(
|
||||
request: GraphLineageRequest,
|
||||
graph_service: GraphIntelligenceServiceV2ExternalDep,
|
||||
project_id: ProjectExternalIdPathDep,
|
||||
) -> GraphLineageResponse:
|
||||
"""Build lineage paths from a start node toward an optional goal."""
|
||||
_ = project_id
|
||||
return await graph_service.lineage(request)
|
||||
|
||||
|
||||
@router.post("/impact", response_model=GraphImpactResponse)
|
||||
async def graph_impact(
|
||||
request: GraphImpactRequest,
|
||||
graph_service: GraphIntelligenceServiceV2ExternalDep,
|
||||
project_id: ProjectExternalIdPathDep,
|
||||
) -> GraphImpactResponse:
|
||||
"""Compute impact radius from a target node."""
|
||||
_ = project_id
|
||||
return await graph_service.impact(request)
|
||||
|
||||
|
||||
@router.get("/health", response_model=GraphHealthResponse)
|
||||
async def graph_health(
|
||||
graph_service: GraphIntelligenceServiceV2ExternalDep,
|
||||
project_id: ProjectExternalIdPathDep,
|
||||
scope: str | None = Query(default=None),
|
||||
timeframe: str | None = Query(default=None),
|
||||
) -> GraphHealthResponse:
|
||||
"""Report graph quality metrics and issue candidates."""
|
||||
_ = project_id
|
||||
return await graph_service.health(scope=scope, timeframe=timeframe)
|
||||
|
||||
|
||||
@router.post("/reindex", response_model=GraphReindexResponse)
|
||||
async def graph_reindex(
|
||||
request: GraphReindexRequest,
|
||||
graph_service: GraphIntelligenceServiceV2ExternalDep,
|
||||
task_scheduler: TaskSchedulerDep,
|
||||
project_id: ProjectExternalIdPathDep,
|
||||
) -> GraphReindexResponse:
|
||||
"""Queue a graph reindex operation for the current project."""
|
||||
task_scheduler.schedule(
|
||||
"reindex_graph_project",
|
||||
project_id=project_id,
|
||||
mode=request.mode,
|
||||
reason=request.reason,
|
||||
)
|
||||
return await graph_service.start_reindex_job()
|
||||
@@ -130,7 +130,7 @@ async def resolve_identifier(
|
||||
resolution_method=resolution_method,
|
||||
)
|
||||
|
||||
logger.debug(
|
||||
logger.info(
|
||||
f"API v2 response: resolved '{data.identifier}' to external_id={result.external_id} via {resolution_method}"
|
||||
)
|
||||
|
||||
|
||||
@@ -48,7 +48,7 @@ async def list_projects(
|
||||
A list of all projects with metadata
|
||||
"""
|
||||
projects = await project_service.list_projects()
|
||||
default_project = await project_service.get_default_project_name()
|
||||
default_project = project_service.default_project
|
||||
|
||||
project_items = [
|
||||
ProjectItem(
|
||||
|
||||
@@ -10,15 +10,9 @@ Flow: Entity loaded with eager observations/relations -> convert to tuples -> co
|
||||
|
||||
from pathlib import Path as FilePath
|
||||
|
||||
import frontmatter
|
||||
from fastapi import APIRouter, Path, Query
|
||||
from loguru import logger
|
||||
|
||||
from basic_memory.deps import (
|
||||
EntityRepositoryV2ExternalDep,
|
||||
FileServiceV2ExternalDep,
|
||||
LinkResolverV2ExternalDep,
|
||||
)
|
||||
from basic_memory.deps import EntityRepositoryV2ExternalDep
|
||||
from basic_memory.models.knowledge import Entity
|
||||
from basic_memory.schemas.schema import (
|
||||
ValidationReport,
|
||||
@@ -73,54 +67,11 @@ def _entity_to_note_data(entity: Entity) -> NoteData:
|
||||
|
||||
|
||||
def _entity_frontmatter(entity: Entity) -> dict:
|
||||
"""Build a frontmatter dict from an entity's database metadata.
|
||||
|
||||
Used for the notes being validated — their type and schema ref are
|
||||
unlikely to change between syncs.
|
||||
"""
|
||||
fm = dict(entity.entity_metadata) if entity.entity_metadata else {}
|
||||
"""Build a frontmatter dict from an entity for schema resolution."""
|
||||
frontmatter = dict(entity.entity_metadata) if entity.entity_metadata else {}
|
||||
if entity.note_type:
|
||||
fm.setdefault("type", entity.note_type)
|
||||
return fm
|
||||
|
||||
|
||||
async def _schema_frontmatter_from_file(
|
||||
file_service: FileServiceV2ExternalDep,
|
||||
entity: Entity,
|
||||
) -> dict:
|
||||
"""Read a schema entity's frontmatter directly from its file.
|
||||
|
||||
Schema definitions (field declarations, validation mode) are the source
|
||||
of truth for validation. Reading from the file ensures schema-validate
|
||||
always uses the latest settings, even when the file watcher hasn't
|
||||
synced changes to entity_metadata in the database.
|
||||
"""
|
||||
try:
|
||||
content = await file_service.read_file_content(entity.file_path)
|
||||
post = frontmatter.loads(content)
|
||||
metadata = dict(post.metadata)
|
||||
|
||||
# Trigger: file is mid-edit and missing required schema fields
|
||||
# Why: parse_schema_note() raises ValueError for missing entity/schema,
|
||||
# which would turn validation into a 500 response
|
||||
# Outcome: fall back to last-known-good database metadata
|
||||
if not metadata.get("entity") or not isinstance(metadata.get("schema"), dict):
|
||||
logger.warning(
|
||||
"Schema file has incomplete frontmatter, falling back to database metadata",
|
||||
file_path=entity.file_path,
|
||||
)
|
||||
return _entity_frontmatter(entity)
|
||||
|
||||
return metadata
|
||||
except Exception:
|
||||
# Trigger: file is missing, unreadable, or has malformed frontmatter
|
||||
# Why: fall back to database metadata rather than failing validation entirely
|
||||
# Outcome: behaves like before this change — uses potentially stale data
|
||||
logger.warning(
|
||||
"Failed to read schema file, falling back to database metadata",
|
||||
file_path=entity.file_path,
|
||||
)
|
||||
return _entity_frontmatter(entity)
|
||||
frontmatter.setdefault("type", entity.note_type)
|
||||
return frontmatter
|
||||
|
||||
|
||||
# --- Validation ---
|
||||
@@ -129,8 +80,6 @@ async def _schema_frontmatter_from_file(
|
||||
@router.post("/schema/validate", response_model=ValidationReport)
|
||||
async def validate_schema(
|
||||
entity_repository: EntityRepositoryV2ExternalDep,
|
||||
file_service: FileServiceV2ExternalDep,
|
||||
link_resolver: LinkResolverV2ExternalDep,
|
||||
project_id: str = Path(..., description="Project external UUID"),
|
||||
note_type: str | None = Query(None, description="Note type to validate"),
|
||||
identifier: str | None = Query(None, description="Specific note identifier"),
|
||||
@@ -139,20 +88,14 @@ async def validate_schema(
|
||||
|
||||
Validates a specific note (by identifier) or all notes of a given type.
|
||||
Returns warnings/errors based on the schema's validation mode.
|
||||
|
||||
Schema definitions are read directly from their files to ensure the
|
||||
latest settings (validation mode, field declarations) are always used,
|
||||
even when file changes haven't been synced to the database yet.
|
||||
"""
|
||||
results: list[NoteValidationResponse] = []
|
||||
|
||||
# --- Single note validation ---
|
||||
if identifier:
|
||||
# Resolve identifier flexibly (permalink, title, path, fuzzy)
|
||||
# to match how read_note and other tools resolve identifiers
|
||||
entity = await link_resolver.resolve_link(identifier)
|
||||
entity = await entity_repository.get_by_permalink(identifier)
|
||||
if not entity:
|
||||
return ValidationReport(note_type=note_type, total_notes=0, total_entities=0)
|
||||
return ValidationReport(note_type=note_type, total_notes=0, results=[])
|
||||
|
||||
frontmatter = _entity_frontmatter(entity)
|
||||
schema_ref = frontmatter.get("schema")
|
||||
@@ -163,12 +106,12 @@ async def validate_schema(
|
||||
query,
|
||||
allow_reference_match=isinstance(schema_ref, str) and query == schema_ref,
|
||||
)
|
||||
return [await _schema_frontmatter_from_file(file_service, e) for e in entities]
|
||||
return [_entity_frontmatter(e) for e in entities]
|
||||
|
||||
schema_def = await resolve_schema(frontmatter, search_fn)
|
||||
if schema_def:
|
||||
result = validate_note(
|
||||
entity.title or entity.permalink or identifier,
|
||||
entity.permalink or identifier,
|
||||
schema_def,
|
||||
_entity_observations(entity),
|
||||
_entity_relations(entity),
|
||||
@@ -178,8 +121,7 @@ async def validate_schema(
|
||||
|
||||
return ValidationReport(
|
||||
note_type=note_type or entity.note_type,
|
||||
total_notes=len(results),
|
||||
total_entities=1,
|
||||
total_notes=1,
|
||||
valid_count=1 if (results and results[0].passed) else 0,
|
||||
warning_count=sum(len(r.warnings) for r in results),
|
||||
error_count=sum(len(r.errors) for r in results),
|
||||
@@ -199,12 +141,12 @@ async def validate_schema(
|
||||
query,
|
||||
allow_reference_match=isinstance(schema_ref, str) and query == schema_ref,
|
||||
)
|
||||
return [await _schema_frontmatter_from_file(file_service, e) for e in entities]
|
||||
return [_entity_frontmatter(e) for e in entities]
|
||||
|
||||
schema_def = await resolve_schema(frontmatter, search_fn)
|
||||
if schema_def:
|
||||
result = validate_note(
|
||||
entity.title or entity.permalink or entity.file_path,
|
||||
entity.permalink or entity.file_path,
|
||||
schema_def,
|
||||
_entity_observations(entity),
|
||||
_entity_relations(entity),
|
||||
@@ -273,7 +215,6 @@ async def infer_schema_endpoint(
|
||||
@router.get("/schema/diff/{note_type}", response_model=DriftReport)
|
||||
async def diff_schema_endpoint(
|
||||
entity_repository: EntityRepositoryV2ExternalDep,
|
||||
file_service: FileServiceV2ExternalDep,
|
||||
note_type: str = Path(..., description="Note type to check for drift"),
|
||||
project_id: str = Path(..., description="Project external UUID"),
|
||||
):
|
||||
@@ -286,7 +227,7 @@ async def diff_schema_endpoint(
|
||||
|
||||
async def search_fn(query: str) -> list[dict]:
|
||||
entities = await _find_schema_entities(entity_repository, query)
|
||||
return [await _schema_frontmatter_from_file(file_service, e) for e in entities]
|
||||
return [_entity_frontmatter(e) for e in entities]
|
||||
|
||||
# Resolve schema by note type
|
||||
schema_frontmatter = {"type": note_type}
|
||||
|
||||
@@ -25,7 +25,7 @@ import basic_memory
|
||||
# Configuration — defaults baked in, overridable via environment
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
_DEFAULT_UMAMI_HOST = "https://api-gateway.umami.dev"
|
||||
_DEFAULT_UMAMI_HOST = "https://cloud.umami.is"
|
||||
_DEFAULT_UMAMI_SITE_ID = "f6479898-ebaf-4e60-bce2-6dc60a3f6c5c"
|
||||
|
||||
|
||||
@@ -76,9 +76,7 @@ def track(event_name: str, data: Optional[dict] = None) -> None:
|
||||
host = _umami_host()
|
||||
site_id = _umami_site_id()
|
||||
|
||||
# Umami v2 /api/send requires "type" at top level alongside "payload"
|
||||
payload = {
|
||||
"type": "event",
|
||||
"payload": {
|
||||
"hostname": "cli.basicmemory.com",
|
||||
"language": "en",
|
||||
@@ -89,7 +87,7 @@ def track(event_name: str, data: Optional[dict] = None) -> None:
|
||||
"version": basic_memory.__version__,
|
||||
**(data or {}),
|
||||
},
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
def _send():
|
||||
@@ -99,16 +97,11 @@ def track(event_name: str, data: Optional[dict] = None) -> None:
|
||||
data=json.dumps(payload).encode("utf-8"),
|
||||
headers={
|
||||
"Content-Type": "application/json",
|
||||
# Umami's bot detection rejects non-browser User-Agents
|
||||
"User-Agent": "Mozilla/5.0 (compatible; BasicMemoryCLI/"
|
||||
f"{basic_memory.__version__})",
|
||||
"User-Agent": f"basic-memory-cli/{basic_memory.__version__}",
|
||||
},
|
||||
)
|
||||
urllib.request.urlopen(req, timeout=3)
|
||||
except Exception:
|
||||
pass # Never break the CLI for analytics
|
||||
|
||||
# Non-daemon so the process waits for the request to complete.
|
||||
# The 3s urllib timeout caps the worst-case exit delay.
|
||||
t = threading.Thread(target=_send)
|
||||
t.start()
|
||||
threading.Thread(target=_send, daemon=True).start()
|
||||
|
||||
@@ -85,13 +85,13 @@ def logout():
|
||||
|
||||
@cloud_app.command("status")
|
||||
def status() -> None:
|
||||
"""Check cloud authentication and connection status."""
|
||||
"""Check cloud authentication state and cloud instance health."""
|
||||
config_manager = ConfigManager()
|
||||
config = config_manager.load_config()
|
||||
auth = CLIAuth(client_id=config.cloud_client_id, authkit_domain=config.cloud_domain)
|
||||
tokens = auth.load_tokens()
|
||||
|
||||
console.print("[bold blue]Cloud Status[/bold blue]")
|
||||
console.print("[bold blue]Cloud Authentication Status[/bold blue]")
|
||||
console.print(f" Host: {config.cloud_host}")
|
||||
console.print(
|
||||
f" API Key: {'[green]configured[/green]' if config.cloud_api_key else '[yellow]not set[/yellow]'}"
|
||||
@@ -99,12 +99,17 @@ def status() -> None:
|
||||
|
||||
oauth_status = "[yellow]not logged in[/yellow]"
|
||||
if tokens:
|
||||
if auth.is_token_valid(tokens):
|
||||
oauth_status = "[green]token valid[/green]"
|
||||
else:
|
||||
oauth_status = "[yellow]token expired[/yellow]"
|
||||
oauth_status = (
|
||||
"[green]token valid[/green]"
|
||||
if auth.is_token_valid(tokens)
|
||||
else "[yellow]token expired[/yellow]"
|
||||
)
|
||||
console.print(f" OAuth: {oauth_status}")
|
||||
|
||||
# Get cloud configuration
|
||||
_, _, host_url = get_cloud_config()
|
||||
host_url = host_url.rstrip("/")
|
||||
|
||||
has_credentials = bool(config.cloud_api_key) or tokens is not None
|
||||
if not has_credentials:
|
||||
console.print(
|
||||
@@ -112,20 +117,33 @@ def status() -> None:
|
||||
)
|
||||
return
|
||||
|
||||
# Quick connection check — just verify we can reach the cloud
|
||||
_, _, host_url = get_cloud_config()
|
||||
host_url = host_url.rstrip("/")
|
||||
|
||||
try:
|
||||
run_with_cleanup(make_api_request(method="GET", url=f"{host_url}/proxy/health"))
|
||||
console.print("\n[green]Cloud connected[/green]")
|
||||
except CloudAPIError:
|
||||
console.print("\n[yellow]Cloud not connected[/yellow]")
|
||||
console.print("\n[blue]Checking cloud instance health...[/blue]")
|
||||
|
||||
# Make API request to check health
|
||||
response = run_with_cleanup(make_api_request(method="GET", url=f"{host_url}/proxy/health"))
|
||||
|
||||
health_data = response.json()
|
||||
|
||||
console.print("[green]Cloud instance is healthy[/green]")
|
||||
|
||||
# Display status details
|
||||
if "status" in health_data:
|
||||
console.print(f" Status: {health_data['status']}")
|
||||
if "version" in health_data:
|
||||
console.print(f" Version: {health_data['version']}")
|
||||
if "timestamp" in health_data:
|
||||
console.print(f" Timestamp: {health_data['timestamp']}")
|
||||
|
||||
console.print("\n[dim]To sync projects, use: bm project bisync --name <project>[/dim]")
|
||||
|
||||
except CloudAPIError as e:
|
||||
console.print(f"[yellow]Cloud health check failed: {e}[/yellow]")
|
||||
console.print(
|
||||
"[dim]Try re-authenticating with 'bm cloud login' or 'bm cloud api-key save'.[/dim]"
|
||||
"[dim]Try re-authenticating with 'bm cloud login' or setting API key with 'bm cloud api-key save'.[/dim]"
|
||||
)
|
||||
except Exception:
|
||||
console.print("\n[yellow]Cloud not connected[/yellow]")
|
||||
except Exception as e:
|
||||
console.print(f"[yellow]Unexpected health check error: {e}[/yellow]")
|
||||
|
||||
|
||||
@cloud_app.command("setup")
|
||||
|
||||
@@ -11,7 +11,7 @@ from sqlalchemy.exc import OperationalError
|
||||
from basic_memory import db
|
||||
from basic_memory.cli.app import app
|
||||
from basic_memory.cli.commands.command_utils import run_with_cleanup
|
||||
from basic_memory.config import ConfigManager, ProjectMode
|
||||
from basic_memory.config import ConfigManager
|
||||
from basic_memory.repository import ProjectRepository
|
||||
from basic_memory.services.initialization import reconcile_projects_with_config
|
||||
from basic_memory.sync.sync_service import get_sync_service
|
||||
@@ -169,16 +169,7 @@ async def _reindex(app_config, search: bool, embeddings: bool, project: str | No
|
||||
if project:
|
||||
projects = [p for p in projects if p.name == project]
|
||||
if not projects:
|
||||
# Check if it's a cloud-only project — those can't be reindexed locally
|
||||
project_mode = app_config.get_project_mode(project)
|
||||
if project_mode == ProjectMode.CLOUD:
|
||||
console.print(
|
||||
f"[yellow]Project '{project}' is a cloud project.[/yellow]\n"
|
||||
"Reindexing is a local operation — cloud projects are "
|
||||
"indexed on the server."
|
||||
)
|
||||
else:
|
||||
console.print(f"[red]Project '{project}' not found.[/red]")
|
||||
console.print(f"[red]Project '{project}' not found.[/red]")
|
||||
raise typer.Exit(1)
|
||||
|
||||
for proj in projects:
|
||||
|
||||
@@ -61,7 +61,6 @@ def list_projects(
|
||||
local: bool = typer.Option(False, "--local", help="Force local routing for this command"),
|
||||
cloud: bool = typer.Option(False, "--cloud", help="Force cloud API routing"),
|
||||
workspace: str = typer.Option(None, "--workspace", help="Cloud workspace name or tenant_id"),
|
||||
json_output: bool = typer.Option(False, "--json", help="Output in JSON format"),
|
||||
) -> None:
|
||||
"""List Basic Memory projects from local and (when available) cloud."""
|
||||
try:
|
||||
@@ -97,9 +96,13 @@ def list_projects(
|
||||
|
||||
if _has_cloud_credentials(config):
|
||||
try:
|
||||
with console.status("[bold blue]Fetching cloud projects...", spinner="dots"):
|
||||
with console.status(
|
||||
"[bold blue]Fetching cloud projects...", spinner="dots"
|
||||
):
|
||||
with force_routing(cloud=True):
|
||||
cloud_result = run_with_cleanup(_list_projects(effective_workspace))
|
||||
cloud_result = run_with_cleanup(
|
||||
_list_projects(effective_workspace)
|
||||
)
|
||||
except Exception as exc: # pragma: no cover
|
||||
cloud_error = exc
|
||||
|
||||
@@ -110,7 +113,9 @@ def list_projects(
|
||||
try:
|
||||
from basic_memory.mcp.project_context import get_available_workspaces
|
||||
|
||||
with console.status("[bold blue]Resolving workspace...", spinner="dots"):
|
||||
with console.status(
|
||||
"[bold blue]Resolving workspace...", spinner="dots"
|
||||
):
|
||||
workspaces = run_with_cleanup(get_available_workspaces())
|
||||
matched = next(
|
||||
(ws for ws in workspaces if ws.tenant_id == effective_workspace),
|
||||
@@ -148,8 +153,6 @@ def list_projects(
|
||||
project_names_by_permalink[permalink] = project.name
|
||||
cloud_projects_by_permalink[permalink] = project
|
||||
|
||||
# --- Build unified project list ---
|
||||
project_rows: list[dict] = []
|
||||
for permalink in sorted(project_names_by_permalink):
|
||||
project_name = project_names_by_permalink[permalink]
|
||||
local_project = local_projects_by_permalink.get(permalink)
|
||||
@@ -179,9 +182,9 @@ def list_projects(
|
||||
else:
|
||||
cli_route = ProjectMode.LOCAL.value
|
||||
|
||||
is_default = config.default_project == project_name
|
||||
is_default = "[X]" if config.default_project == project_name else ""
|
||||
|
||||
has_sync = bool(entry and entry.local_sync_path)
|
||||
has_sync = "[X]" if entry and entry.local_sync_path else ""
|
||||
mcp_stdio_target = "local" if local_project is not None else "n/a"
|
||||
|
||||
# Show workspace name (type) for cloud-sourced projects
|
||||
@@ -189,48 +192,24 @@ def list_projects(
|
||||
if cloud_project is not None and cloud_ws_name:
|
||||
ws_label = f"{cloud_ws_name} ({cloud_ws_type})" if cloud_ws_type else cloud_ws_name
|
||||
|
||||
row_data = {
|
||||
"name": project_name,
|
||||
"permalink": permalink,
|
||||
"local_path": local_path,
|
||||
"cloud_path": cloud_path,
|
||||
"cli_route": cli_route,
|
||||
"mcp_stdio": mcp_stdio_target,
|
||||
"sync": has_sync,
|
||||
"is_default": is_default,
|
||||
}
|
||||
if ws_label:
|
||||
row_data["workspace"] = cloud_ws_name or ""
|
||||
if cloud_ws_type:
|
||||
row_data["workspace_type"] = cloud_ws_type
|
||||
row = [
|
||||
project_name,
|
||||
local_path,
|
||||
cloud_path,
|
||||
ws_label,
|
||||
cli_route,
|
||||
mcp_stdio_target,
|
||||
has_sync,
|
||||
is_default,
|
||||
]
|
||||
|
||||
project_rows.append(row_data)
|
||||
|
||||
# --- JSON output ---
|
||||
if json_output:
|
||||
print(json.dumps({"projects": project_rows}, indent=2, default=str))
|
||||
return
|
||||
|
||||
# --- Rich table output ---
|
||||
for row_data in project_rows:
|
||||
table.add_row(
|
||||
row_data["name"],
|
||||
row_data["local_path"],
|
||||
row_data["cloud_path"],
|
||||
row_data.get("workspace", "")
|
||||
+ (f" ({row_data['workspace_type']})" if row_data.get("workspace_type") else ""),
|
||||
row_data["cli_route"],
|
||||
row_data["mcp_stdio"],
|
||||
"[X]" if row_data["sync"] else "",
|
||||
"[X]" if row_data["is_default"] else "",
|
||||
)
|
||||
table.add_row(*row)
|
||||
|
||||
console.print(table)
|
||||
if cloud_error is not None:
|
||||
console.print(f"[yellow]Cloud project discovery failed: {cloud_error}[/yellow]")
|
||||
console.print(
|
||||
"[dim]Showing local projects only. "
|
||||
"Run 'bm cloud login' or 'bm cloud api-key save <key>' if this is a credentials issue.[/dim]"
|
||||
"[yellow]Cloud project discovery failed. "
|
||||
"Showing local projects only. Run 'bm cloud login' or 'bm cloud api-key save <key>'.[/yellow]"
|
||||
)
|
||||
except Exception as e:
|
||||
console.print(f"[red]Error listing projects: {str(e)}[/red]")
|
||||
|
||||
@@ -173,7 +173,6 @@ def validate(
|
||||
typer.Option(help="The project name."),
|
||||
] = None,
|
||||
strict: bool = typer.Option(False, "--strict", help="Exit with error on validation failures"),
|
||||
json_output: bool = typer.Option(False, "--json", help="Output in JSON format"),
|
||||
local: bool = typer.Option(
|
||||
False, "--local", help="Force local API routing (ignore cloud mode)"
|
||||
),
|
||||
@@ -184,7 +183,6 @@ def validate(
|
||||
TARGET can be a note path (e.g., people/ada-lovelace.md) or a note type
|
||||
(e.g., person). If omitted, validates all notes that have schemas.
|
||||
|
||||
Use --json for machine-readable output.
|
||||
Use --strict to exit with error code 1 if any validation errors are found.
|
||||
Use --local to force local routing when cloud mode is enabled.
|
||||
Use --cloud to force cloud routing when cloud mode is disabled.
|
||||
@@ -213,19 +211,12 @@ def validate(
|
||||
|
||||
# Handle error responses
|
||||
if isinstance(result, dict) and "error" in result:
|
||||
if json_output:
|
||||
print(json.dumps(result, indent=2, default=str))
|
||||
else:
|
||||
console.print(f"[yellow]{result['error']}[/yellow]")
|
||||
console.print(f"[yellow]{result['error']}[/yellow]")
|
||||
return
|
||||
|
||||
# output_format="json" guarantees a dict return
|
||||
assert isinstance(result, dict)
|
||||
|
||||
if json_output:
|
||||
print(json.dumps(result, indent=2, default=str))
|
||||
else:
|
||||
_render_validate_table(result)
|
||||
_render_validate_table(result)
|
||||
|
||||
if strict and result.get("error_count", 0) > 0:
|
||||
raise typer.Exit(1)
|
||||
@@ -254,7 +245,6 @@ def infer(
|
||||
0.25, "--threshold", help="Minimum frequency for optional fields (0-1)"
|
||||
),
|
||||
save: bool = typer.Option(False, "--save", help="Save inferred schema to schema/ directory"),
|
||||
json_output: bool = typer.Option(False, "--json", help="Output in JSON format"),
|
||||
local: bool = typer.Option(
|
||||
False, "--local", help="Force local API routing (ignore cloud mode)"
|
||||
),
|
||||
@@ -268,7 +258,6 @@ def infer(
|
||||
Fields present in 95%+ of notes become required. Fields above the
|
||||
threshold (default 25%) become optional. Fields below threshold are excluded.
|
||||
|
||||
Use --json for machine-readable output.
|
||||
Use --local to force local routing when cloud mode is enabled.
|
||||
Use --cloud to force cloud routing when cloud mode is disabled.
|
||||
"""
|
||||
@@ -288,10 +277,7 @@ def infer(
|
||||
|
||||
# Handle error responses
|
||||
if isinstance(result, dict) and "error" in result:
|
||||
if json_output:
|
||||
print(json.dumps(result, indent=2, default=str))
|
||||
else:
|
||||
console.print(f"[yellow]{result['error']}[/yellow]")
|
||||
console.print(f"[yellow]{result['error']}[/yellow]")
|
||||
return
|
||||
|
||||
# output_format="json" guarantees a dict return
|
||||
@@ -299,16 +285,10 @@ def infer(
|
||||
|
||||
# Handle zero notes
|
||||
if result.get("notes_analyzed", 0) == 0:
|
||||
if json_output:
|
||||
print(json.dumps(result, indent=2, default=str))
|
||||
else:
|
||||
console.print(f"[yellow]No notes found with type: {note_type}[/yellow]")
|
||||
console.print(f"[yellow]No notes found with type: {note_type}[/yellow]")
|
||||
return
|
||||
|
||||
if json_output:
|
||||
print(json.dumps(result, indent=2, default=str))
|
||||
else:
|
||||
_render_infer_table(result)
|
||||
_render_infer_table(result)
|
||||
|
||||
if save:
|
||||
console.print(
|
||||
@@ -336,7 +316,6 @@ def diff(
|
||||
Optional[str],
|
||||
typer.Option(help="The project name."),
|
||||
] = None,
|
||||
json_output: bool = typer.Option(False, "--json", help="Output in JSON format"),
|
||||
local: bool = typer.Option(
|
||||
False, "--local", help="Force local API routing (ignore cloud mode)"
|
||||
),
|
||||
@@ -348,7 +327,6 @@ def diff(
|
||||
are actually structured. Identifies new fields,
|
||||
dropped fields, and cardinality changes.
|
||||
|
||||
Use --json for machine-readable output.
|
||||
Use --local to force local routing when cloud mode is enabled.
|
||||
Use --cloud to force cloud routing when cloud mode is disabled.
|
||||
"""
|
||||
@@ -367,19 +345,12 @@ def diff(
|
||||
|
||||
# Handle error responses
|
||||
if isinstance(result, dict) and "error" in result:
|
||||
if json_output:
|
||||
print(json.dumps(result, indent=2, default=str))
|
||||
else:
|
||||
console.print(f"[yellow]{result['error']}[/yellow]")
|
||||
console.print(f"[yellow]{result['error']}[/yellow]")
|
||||
return
|
||||
|
||||
# output_format="json" guarantees a dict return
|
||||
assert isinstance(result, dict)
|
||||
|
||||
if json_output:
|
||||
print(json.dumps(result, indent=2, default=str))
|
||||
else:
|
||||
_render_diff_output(result)
|
||||
_render_diff_output(result)
|
||||
except ValueError as e:
|
||||
console.print(f"[red]Error: {e}[/red]")
|
||||
raise typer.Exit(1)
|
||||
|
||||
@@ -1,6 +1,5 @@
|
||||
"""Status command for basic-memory CLI."""
|
||||
|
||||
import json
|
||||
from typing import Set, Dict
|
||||
from typing import Annotated, Optional
|
||||
|
||||
@@ -142,20 +141,21 @@ def display_changes(
|
||||
console.print(Panel(tree, expand=False))
|
||||
|
||||
|
||||
async def run_status(
|
||||
project: Optional[str] = None,
|
||||
) -> tuple[str, SyncReportResponse]:
|
||||
"""Fetch sync status of files vs database.
|
||||
|
||||
Returns (project_name, sync_report) for the caller to render.
|
||||
"""
|
||||
async def run_status(project: Optional[str] = None, verbose: bool = False): # pragma: no cover
|
||||
"""Check sync status of files vs database."""
|
||||
# Resolve default project so get_client() can route per-project
|
||||
project = project or ConfigManager().default_project
|
||||
|
||||
async with get_client(project_name=project) as client:
|
||||
project_item = await get_active_project(client, project, None)
|
||||
sync_report = await ProjectClient(client).get_status(project_item.external_id)
|
||||
return project_item.name, sync_report
|
||||
try:
|
||||
async with get_client(project_name=project) as client:
|
||||
project_item = await get_active_project(client, project, None)
|
||||
sync_report = await ProjectClient(client).get_status(project_item.external_id)
|
||||
|
||||
display_changes(project_item.name, "Status", sync_report, verbose)
|
||||
|
||||
except (ValueError, ToolError) as e:
|
||||
console.print(f"[red]Error: {e}[/red]")
|
||||
raise typer.Exit(1)
|
||||
|
||||
|
||||
@app.command()
|
||||
@@ -165,7 +165,6 @@ def status(
|
||||
typer.Option(help="The project name."),
|
||||
] = None,
|
||||
verbose: bool = typer.Option(False, "--verbose", "-v", help="Show detailed file information"),
|
||||
json_output: bool = typer.Option(False, "--json", help="Output in JSON format"),
|
||||
local: bool = typer.Option(
|
||||
False, "--local", help="Force local API routing (ignore cloud mode)"
|
||||
),
|
||||
@@ -173,7 +172,6 @@ def status(
|
||||
):
|
||||
"""Show sync status between files and database.
|
||||
|
||||
Use --json for machine-readable output.
|
||||
Use --local to force local routing when cloud mode is enabled.
|
||||
Use --cloud to force cloud routing when cloud mode is disabled.
|
||||
"""
|
||||
@@ -189,24 +187,11 @@ def status(
|
||||
if not local and not cloud:
|
||||
local = True
|
||||
with force_routing(local=local, cloud=cloud):
|
||||
project_name, sync_report = run_with_cleanup(run_status(project))
|
||||
|
||||
if json_output:
|
||||
print(json.dumps(sync_report.model_dump(mode="json"), indent=2, default=str))
|
||||
else:
|
||||
display_changes(project_name, "Status", sync_report, verbose)
|
||||
except (ValueError, ToolError) as e:
|
||||
if json_output:
|
||||
print(json.dumps({"error": str(e)}, indent=2))
|
||||
else:
|
||||
console.print(f"[red]Error: {e}[/red]")
|
||||
run_with_cleanup(run_status(project, verbose)) # pragma: no cover
|
||||
except ValueError as e:
|
||||
console.print(f"[red]Error: {e}[/red]")
|
||||
raise typer.Exit(code=1)
|
||||
except typer.Exit:
|
||||
raise
|
||||
except Exception as e:
|
||||
logger.error(f"Error checking status: {e}")
|
||||
if json_output:
|
||||
print(json.dumps({"error": str(e)}, indent=2))
|
||||
else:
|
||||
typer.echo(f"Error checking status: {e}", err=True)
|
||||
typer.echo(f"Error checking status: {e}", err=True)
|
||||
raise typer.Exit(code=1) # pragma: no cover
|
||||
|
||||
@@ -16,13 +16,6 @@ from basic_memory.cli.commands.command_utils import run_with_cleanup
|
||||
from basic_memory.cli.commands.routing import force_routing, validate_routing_flags
|
||||
from basic_memory.mcp.tools import build_context as mcp_build_context
|
||||
from basic_memory.mcp.tools import edit_note as mcp_edit_note
|
||||
from basic_memory.mcp.tools import fcm_export_model as mcp_fcm_export_model
|
||||
from basic_memory.mcp.tools import fcm_import_model as mcp_fcm_import_model
|
||||
from basic_memory.mcp.tools import fcm_rank_actions as mcp_fcm_rank_actions
|
||||
from basic_memory.mcp.tools import fcm_simulate as mcp_fcm_simulate
|
||||
from basic_memory.mcp.tools import graph_health as mcp_graph_health
|
||||
from basic_memory.mcp.tools import graph_impact as mcp_graph_impact
|
||||
from basic_memory.mcp.tools import graph_lineage as mcp_graph_lineage
|
||||
from basic_memory.mcp.tools import list_memory_projects as mcp_list_projects
|
||||
from basic_memory.mcp.tools import list_workspaces as mcp_list_workspaces
|
||||
from basic_memory.mcp.tools import read_note as mcp_read_note
|
||||
@@ -47,17 +40,6 @@ def _print_json(result: Any) -> None:
|
||||
print(json.dumps(result, indent=2, ensure_ascii=True, default=str))
|
||||
|
||||
|
||||
def _parse_json_option(raw_value: Optional[str], option_name: str) -> Any:
|
||||
"""Parse a JSON CLI option with deterministic error handling."""
|
||||
if raw_value is None:
|
||||
return None
|
||||
try:
|
||||
return json.loads(raw_value)
|
||||
except json.JSONDecodeError as exc:
|
||||
typer.echo(f"Invalid JSON for {option_name}: {exc}", err=True)
|
||||
raise typer.Exit(1)
|
||||
|
||||
|
||||
# --- Commands ---
|
||||
|
||||
|
||||
@@ -384,372 +366,6 @@ def recent_activity(
|
||||
raise
|
||||
|
||||
|
||||
@tool_app.command("graph-lineage")
|
||||
def graph_lineage(
|
||||
start: Annotated[str, typer.Argument(help="Start node identifier or memory:// reference")],
|
||||
goal: Annotated[
|
||||
Optional[str],
|
||||
typer.Option("--goal", help="Optional goal node identifier for targeted lineage"),
|
||||
] = None,
|
||||
max_hops: int = typer.Option(4, "--max-hops", help="Maximum traversal hops (1-6)"),
|
||||
relation_filters: Annotated[
|
||||
Optional[List[str]],
|
||||
typer.Option("--relation-filter", help="Relation filters (repeatable)"),
|
||||
] = None,
|
||||
project: Annotated[
|
||||
Optional[str],
|
||||
typer.Option(help="The project to use. If not provided, the default project will be used."),
|
||||
] = None,
|
||||
workspace: Annotated[
|
||||
Optional[str],
|
||||
typer.Option(help="Cloud workspace tenant ID or unique name to route this request."),
|
||||
] = None,
|
||||
local: bool = typer.Option(
|
||||
False, "--local", help="Force local API routing (ignore cloud mode)"
|
||||
),
|
||||
cloud: bool = typer.Option(False, "--cloud", help="Force cloud API routing"),
|
||||
):
|
||||
"""Get graph lineage paths from a start node."""
|
||||
try:
|
||||
validate_routing_flags(local, cloud)
|
||||
with force_routing(local=local, cloud=cloud):
|
||||
result = run_with_cleanup(
|
||||
mcp_graph_lineage(
|
||||
start=start,
|
||||
goal=goal,
|
||||
max_hops=max_hops,
|
||||
relation_filters=relation_filters or [],
|
||||
project=project,
|
||||
workspace=workspace,
|
||||
output_format="json",
|
||||
)
|
||||
)
|
||||
_print_json(result)
|
||||
except ValueError as e:
|
||||
typer.echo(f"Error: {e}", err=True)
|
||||
raise typer.Exit(1)
|
||||
except Exception as e: # pragma: no cover
|
||||
if not isinstance(e, typer.Exit):
|
||||
typer.echo(f"Error during graph_lineage: {e}", err=True)
|
||||
raise typer.Exit(1)
|
||||
raise
|
||||
|
||||
|
||||
@tool_app.command("graph-impact")
|
||||
def graph_impact(
|
||||
target: Annotated[str, typer.Argument(help="Target node identifier or memory:// reference")],
|
||||
horizon: int = typer.Option(2, "--horizon", help="Impact horizon in hops (1-4)"),
|
||||
relation_filters: Annotated[
|
||||
Optional[List[str]],
|
||||
typer.Option("--relation-filter", help="Relation filters (repeatable)"),
|
||||
] = None,
|
||||
include_reasons: bool = typer.Option(
|
||||
True,
|
||||
"--include-reasons/--no-include-reasons",
|
||||
help="Include reason strings in impact output",
|
||||
),
|
||||
project: Annotated[
|
||||
Optional[str],
|
||||
typer.Option(help="The project to use. If not provided, the default project will be used."),
|
||||
] = None,
|
||||
workspace: Annotated[
|
||||
Optional[str],
|
||||
typer.Option(help="Cloud workspace tenant ID or unique name to route this request."),
|
||||
] = None,
|
||||
local: bool = typer.Option(
|
||||
False, "--local", help="Force local API routing (ignore cloud mode)"
|
||||
),
|
||||
cloud: bool = typer.Option(False, "--cloud", help="Force cloud API routing"),
|
||||
):
|
||||
"""Get impact radius for a target node."""
|
||||
try:
|
||||
validate_routing_flags(local, cloud)
|
||||
with force_routing(local=local, cloud=cloud):
|
||||
result = run_with_cleanup(
|
||||
mcp_graph_impact(
|
||||
target=target,
|
||||
horizon=horizon,
|
||||
relation_filters=relation_filters or [],
|
||||
include_reasons=include_reasons,
|
||||
project=project,
|
||||
workspace=workspace,
|
||||
output_format="json",
|
||||
)
|
||||
)
|
||||
_print_json(result)
|
||||
except ValueError as e:
|
||||
typer.echo(f"Error: {e}", err=True)
|
||||
raise typer.Exit(1)
|
||||
except Exception as e: # pragma: no cover
|
||||
if not isinstance(e, typer.Exit):
|
||||
typer.echo(f"Error during graph_impact: {e}", err=True)
|
||||
raise typer.Exit(1)
|
||||
raise
|
||||
|
||||
|
||||
@tool_app.command("graph-health")
|
||||
def graph_health(
|
||||
scope: Annotated[Optional[str], typer.Option("--scope", help="Optional scope prefix")] = None,
|
||||
timeframe: Annotated[
|
||||
Optional[str], typer.Option("--timeframe", help="Optional timeframe filter")
|
||||
] = None,
|
||||
project: Annotated[
|
||||
Optional[str],
|
||||
typer.Option(help="The project to use. If not provided, the default project will be used."),
|
||||
] = None,
|
||||
workspace: Annotated[
|
||||
Optional[str],
|
||||
typer.Option(help="Cloud workspace tenant ID or unique name to route this request."),
|
||||
] = None,
|
||||
local: bool = typer.Option(
|
||||
False, "--local", help="Force local API routing (ignore cloud mode)"
|
||||
),
|
||||
cloud: bool = typer.Option(False, "--cloud", help="Force cloud API routing"),
|
||||
):
|
||||
"""Get graph health metrics and issue candidates."""
|
||||
try:
|
||||
validate_routing_flags(local, cloud)
|
||||
with force_routing(local=local, cloud=cloud):
|
||||
result = run_with_cleanup(
|
||||
mcp_graph_health(
|
||||
scope=scope,
|
||||
timeframe=timeframe,
|
||||
project=project,
|
||||
workspace=workspace,
|
||||
output_format="json",
|
||||
)
|
||||
)
|
||||
_print_json(result)
|
||||
except ValueError as e:
|
||||
typer.echo(f"Error: {e}", err=True)
|
||||
raise typer.Exit(1)
|
||||
except Exception as e: # pragma: no cover
|
||||
if not isinstance(e, typer.Exit):
|
||||
typer.echo(f"Error during graph_health: {e}", err=True)
|
||||
raise typer.Exit(1)
|
||||
raise
|
||||
|
||||
|
||||
@tool_app.command("fcm-simulate")
|
||||
def fcm_simulate(
|
||||
actions_json: Annotated[
|
||||
str,
|
||||
typer.Option(
|
||||
"--actions-json",
|
||||
help='JSON array of actions, e.g. [{"node_id":"n1","delta":0.2}]',
|
||||
),
|
||||
],
|
||||
scenario_json: Annotated[
|
||||
Optional[str],
|
||||
typer.Option("--scenario-json", help="Optional JSON scenario object"),
|
||||
] = None,
|
||||
clamp_rules_json: Annotated[
|
||||
Optional[str],
|
||||
typer.Option("--clamp-rules-json", help="Optional JSON array of clamp rules"),
|
||||
] = None,
|
||||
project: Annotated[
|
||||
Optional[str],
|
||||
typer.Option(help="The project to use. If not provided, the default project will be used."),
|
||||
] = None,
|
||||
workspace: Annotated[
|
||||
Optional[str],
|
||||
typer.Option(help="Cloud workspace tenant ID or unique name to route this request."),
|
||||
] = None,
|
||||
local: bool = typer.Option(
|
||||
False, "--local", help="Force local API routing (ignore cloud mode)"
|
||||
),
|
||||
cloud: bool = typer.Option(False, "--cloud", help="Force cloud API routing"),
|
||||
):
|
||||
"""Run an FCM simulation."""
|
||||
actions = _parse_json_option(actions_json, "--actions-json")
|
||||
scenario = _parse_json_option(scenario_json, "--scenario-json")
|
||||
clamp_rules = _parse_json_option(clamp_rules_json, "--clamp-rules-json")
|
||||
if not isinstance(actions, list):
|
||||
typer.echo("Invalid JSON for --actions-json: expected a JSON array", err=True)
|
||||
raise typer.Exit(1)
|
||||
if scenario is not None and not isinstance(scenario, dict):
|
||||
typer.echo("Invalid JSON for --scenario-json: expected a JSON object", err=True)
|
||||
raise typer.Exit(1)
|
||||
if clamp_rules is not None and not isinstance(clamp_rules, list):
|
||||
typer.echo("Invalid JSON for --clamp-rules-json: expected a JSON array", err=True)
|
||||
raise typer.Exit(1)
|
||||
|
||||
try:
|
||||
validate_routing_flags(local, cloud)
|
||||
with force_routing(local=local, cloud=cloud):
|
||||
result = run_with_cleanup(
|
||||
mcp_fcm_simulate(
|
||||
actions=actions,
|
||||
scenario=scenario,
|
||||
clamp_rules=clamp_rules,
|
||||
project=project,
|
||||
workspace=workspace,
|
||||
output_format="json",
|
||||
)
|
||||
)
|
||||
_print_json(result)
|
||||
except ValueError as e:
|
||||
typer.echo(f"Error: {e}", err=True)
|
||||
raise typer.Exit(1)
|
||||
except Exception as e: # pragma: no cover
|
||||
if not isinstance(e, typer.Exit):
|
||||
typer.echo(f"Error during fcm_simulate: {e}", err=True)
|
||||
raise typer.Exit(1)
|
||||
raise
|
||||
|
||||
|
||||
@tool_app.command("fcm-rank-actions")
|
||||
def fcm_rank_actions(
|
||||
goal: Annotated[str, typer.Argument(help="Goal node identifier")],
|
||||
constraints_json: Annotated[
|
||||
Optional[str],
|
||||
typer.Option("--constraints-json", help="Optional JSON object of ranking constraints"),
|
||||
] = None,
|
||||
top_k: int = typer.Option(10, "--top-k", help="Number of recommendations to return"),
|
||||
project: Annotated[
|
||||
Optional[str],
|
||||
typer.Option(help="The project to use. If not provided, the default project will be used."),
|
||||
] = None,
|
||||
workspace: Annotated[
|
||||
Optional[str],
|
||||
typer.Option(help="Cloud workspace tenant ID or unique name to route this request."),
|
||||
] = None,
|
||||
local: bool = typer.Option(
|
||||
False, "--local", help="Force local API routing (ignore cloud mode)"
|
||||
),
|
||||
cloud: bool = typer.Option(False, "--cloud", help="Force cloud API routing"),
|
||||
):
|
||||
"""Rank intervention actions for an FCM goal."""
|
||||
constraints = _parse_json_option(constraints_json, "--constraints-json")
|
||||
if constraints is not None and not isinstance(constraints, dict):
|
||||
typer.echo("Invalid JSON for --constraints-json: expected a JSON object", err=True)
|
||||
raise typer.Exit(1)
|
||||
|
||||
try:
|
||||
validate_routing_flags(local, cloud)
|
||||
with force_routing(local=local, cloud=cloud):
|
||||
result = run_with_cleanup(
|
||||
mcp_fcm_rank_actions(
|
||||
goal=goal,
|
||||
constraints=constraints,
|
||||
top_k=top_k,
|
||||
project=project,
|
||||
workspace=workspace,
|
||||
output_format="json",
|
||||
)
|
||||
)
|
||||
_print_json(result)
|
||||
except ValueError as e:
|
||||
typer.echo(f"Error: {e}", err=True)
|
||||
raise typer.Exit(1)
|
||||
except Exception as e: # pragma: no cover
|
||||
if not isinstance(e, typer.Exit):
|
||||
typer.echo(f"Error during fcm_rank_actions: {e}", err=True)
|
||||
raise typer.Exit(1)
|
||||
raise
|
||||
|
||||
|
||||
@tool_app.command("fcm-import-model")
|
||||
def fcm_import_model(
|
||||
source: Annotated[str, typer.Argument(help="Source path or URI for import payload")],
|
||||
format: Annotated[
|
||||
str,
|
||||
typer.Option("--format", help="Import format (currently csv_bundle_v1)"),
|
||||
] = "csv_bundle_v1",
|
||||
merge_mode: Annotated[
|
||||
str,
|
||||
typer.Option("--merge-mode", help="Merge strategy: replace or upsert"),
|
||||
] = "upsert",
|
||||
project: Annotated[
|
||||
Optional[str],
|
||||
typer.Option(help="The project to use. If not provided, the default project will be used."),
|
||||
] = None,
|
||||
workspace: Annotated[
|
||||
Optional[str],
|
||||
typer.Option(help="Cloud workspace tenant ID or unique name to route this request."),
|
||||
] = None,
|
||||
local: bool = typer.Option(
|
||||
False, "--local", help="Force local API routing (ignore cloud mode)"
|
||||
),
|
||||
cloud: bool = typer.Option(False, "--cloud", help="Force cloud API routing"),
|
||||
):
|
||||
"""Import an FCM model."""
|
||||
try:
|
||||
validate_routing_flags(local, cloud)
|
||||
with force_routing(local=local, cloud=cloud):
|
||||
result = run_with_cleanup(
|
||||
mcp_fcm_import_model(
|
||||
source=source,
|
||||
format=format, # pyright: ignore[reportArgumentType]
|
||||
merge_mode=merge_mode, # pyright: ignore[reportArgumentType]
|
||||
project=project,
|
||||
workspace=workspace,
|
||||
output_format="json",
|
||||
)
|
||||
)
|
||||
_print_json(result)
|
||||
except ValueError as e:
|
||||
typer.echo(f"Error: {e}", err=True)
|
||||
raise typer.Exit(1)
|
||||
except Exception as e: # pragma: no cover
|
||||
if not isinstance(e, typer.Exit):
|
||||
typer.echo(f"Error during fcm_import_model: {e}", err=True)
|
||||
raise typer.Exit(1)
|
||||
raise
|
||||
|
||||
|
||||
@tool_app.command("fcm-export-model")
|
||||
def fcm_export_model(
|
||||
format: Annotated[
|
||||
str,
|
||||
typer.Option("--format", help="Export format (currently csv_bundle_v1)"),
|
||||
] = "csv_bundle_v1",
|
||||
selection_json: Annotated[
|
||||
Optional[str],
|
||||
typer.Option("--selection-json", help="Optional JSON object selection payload"),
|
||||
] = None,
|
||||
project: Annotated[
|
||||
Optional[str],
|
||||
typer.Option(help="The project to use. If not provided, the default project will be used."),
|
||||
] = None,
|
||||
workspace: Annotated[
|
||||
Optional[str],
|
||||
typer.Option(help="Cloud workspace tenant ID or unique name to route this request."),
|
||||
] = None,
|
||||
local: bool = typer.Option(
|
||||
False, "--local", help="Force local API routing (ignore cloud mode)"
|
||||
),
|
||||
cloud: bool = typer.Option(False, "--cloud", help="Force cloud API routing"),
|
||||
):
|
||||
"""Export an FCM model."""
|
||||
selection = _parse_json_option(selection_json, "--selection-json")
|
||||
if selection is not None and not isinstance(selection, dict):
|
||||
typer.echo("Invalid JSON for --selection-json: expected a JSON object", err=True)
|
||||
raise typer.Exit(1)
|
||||
|
||||
try:
|
||||
validate_routing_flags(local, cloud)
|
||||
with force_routing(local=local, cloud=cloud):
|
||||
result = run_with_cleanup(
|
||||
mcp_fcm_export_model(
|
||||
format=format, # pyright: ignore[reportArgumentType]
|
||||
selection=selection,
|
||||
project=project,
|
||||
workspace=workspace,
|
||||
output_format="json",
|
||||
)
|
||||
)
|
||||
_print_json(result)
|
||||
except ValueError as e:
|
||||
typer.echo(f"Error: {e}", err=True)
|
||||
raise typer.Exit(1)
|
||||
except Exception as e: # pragma: no cover
|
||||
if not isinstance(e, typer.Exit):
|
||||
typer.echo(f"Error during fcm_export_model: {e}", err=True)
|
||||
raise typer.Exit(1)
|
||||
raise
|
||||
|
||||
|
||||
@tool_app.command("search-notes")
|
||||
def search_notes(
|
||||
query: Annotated[
|
||||
@@ -869,7 +485,7 @@ def search_notes(
|
||||
with force_routing(local=local, cloud=cloud):
|
||||
result = run_with_cleanup(
|
||||
mcp_search(
|
||||
query=query or None,
|
||||
query=query or "",
|
||||
project=project,
|
||||
workspace=workspace,
|
||||
search_type=search_type,
|
||||
|
||||
@@ -27,9 +27,10 @@ def _is_interactive_session() -> bool:
|
||||
try:
|
||||
return sys.stdin.isatty() and sys.stdout.isatty()
|
||||
except ValueError:
|
||||
# Trigger: stdin/stdout already closed (e.g., MCP stdio transport shutdown)
|
||||
# Why: isatty() raises ValueError on closed file descriptors
|
||||
# Outcome: treat as non-interactive, suppressing promo output
|
||||
# Trigger: stdin/stdout already closed (e.g., MCP stdio transport shutdown).
|
||||
# Why: isatty() raises ValueError on closed file objects; treat as non-interactive
|
||||
# so the promo is suppressed rather than crashing the shutdown path.
|
||||
# Outcome: promo skipped for this invocation.
|
||||
return False
|
||||
|
||||
|
||||
|
||||
@@ -3,7 +3,6 @@
|
||||
import importlib.util
|
||||
import json
|
||||
import os
|
||||
import shutil
|
||||
from dataclasses import dataclass
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
@@ -173,25 +172,6 @@ class BasicMemoryConfig(BaseSettings):
|
||||
description="Batch size for embedding generation.",
|
||||
gt=0,
|
||||
)
|
||||
semantic_embedding_sync_batch_size: int = Field(
|
||||
default=64,
|
||||
description="Batch size for vector sync orchestration flushes.",
|
||||
gt=0,
|
||||
)
|
||||
semantic_embedding_cache_dir: str | None = Field(
|
||||
default=None,
|
||||
description="Optional cache directory for FastEmbed model artifacts.",
|
||||
)
|
||||
semantic_embedding_threads: int | None = Field(
|
||||
default=None,
|
||||
description="Optional FastEmbed runtime thread count override.",
|
||||
gt=0,
|
||||
)
|
||||
semantic_embedding_parallel: int | None = Field(
|
||||
default=None,
|
||||
description="Optional FastEmbed embed() parallelism override.",
|
||||
gt=0,
|
||||
)
|
||||
semantic_vector_k: int = Field(
|
||||
default=100,
|
||||
description="Vector candidate count for vector and hybrid retrieval.",
|
||||
@@ -265,16 +245,6 @@ class BasicMemoryConfig(BaseSettings):
|
||||
description="Disable automatic permalink generation in frontmatter. When enabled, new notes won't have permalinks added and sync won't update permalinks. Existing permalinks will still work for reading.",
|
||||
)
|
||||
|
||||
write_note_overwrite_default: bool = Field(
|
||||
default=False,
|
||||
description=(
|
||||
"Default value for write_note's overwrite parameter. "
|
||||
"When False (default), write_note errors if note already exists. "
|
||||
"Set to True to restore pre-v0.20 upsert behavior. "
|
||||
"Env: BASIC_MEMORY_WRITE_NOTE_OVERWRITE_DEFAULT"
|
||||
),
|
||||
)
|
||||
|
||||
ensure_frontmatter_on_sync: bool = Field(
|
||||
default=True,
|
||||
description="Ensure markdown files have frontmatter during sync by adding derived title/type/permalink when missing. When combined with disable_permalinks=True, this setting takes precedence for missing-frontmatter files and still writes permalinks.",
|
||||
@@ -725,10 +695,7 @@ class ConfigManager:
|
||||
|
||||
# Re-save to normalize legacy config into current format
|
||||
if needs_resave:
|
||||
# Create backup before overwriting so users can revert if needed
|
||||
backup_path = self.config_file.with_suffix(".json.bak")
|
||||
shutil.copy2(self.config_file, backup_path)
|
||||
logger.info(f"Migrating config to current format (backup: {backup_path})")
|
||||
logger.info("Migrating config to current format")
|
||||
save_basic_memory_config(self.config_file, _CONFIG_CACHE)
|
||||
|
||||
return _CONFIG_CACHE
|
||||
|
||||
@@ -128,13 +128,8 @@ async def _run_semantic_embedding_backfill(
|
||||
project_id=project_id,
|
||||
app_config=app_config,
|
||||
)
|
||||
batch_result = await search_repository.sync_entity_vectors_batch(entity_ids)
|
||||
if batch_result.entities_failed > 0:
|
||||
logger.warning(
|
||||
"Automatic semantic embedding backfill encountered entity failures: "
|
||||
f"project={project_name}, failed={batch_result.entities_failed}, "
|
||||
f"failed_entity_ids={batch_result.failed_entity_ids}"
|
||||
)
|
||||
for entity_id in entity_ids:
|
||||
await search_repository.sync_entity_vectors(entity_id)
|
||||
|
||||
logger.info(
|
||||
"Automatic semantic embedding backfill complete: "
|
||||
@@ -480,7 +475,7 @@ async def run_migrations(
|
||||
Note: Alembic tracks which migrations have been applied via the alembic_version table,
|
||||
so it's safe to call this multiple times - it will only run pending migrations.
|
||||
"""
|
||||
logger.debug("Running database migrations...")
|
||||
logger.info("Running database migrations...")
|
||||
temp_engine: AsyncEngine | None = None
|
||||
try:
|
||||
revisions_before_upgrade: set[str] = set()
|
||||
@@ -519,7 +514,7 @@ async def run_migrations(
|
||||
config.set_main_option("sqlalchemy.url", db_url)
|
||||
|
||||
command.upgrade(config, "head")
|
||||
logger.debug("Migrations completed successfully")
|
||||
logger.info("Migrations completed successfully")
|
||||
|
||||
# Get session maker - ensure we don't trigger recursive migration calls
|
||||
if _session_maker is None:
|
||||
|
||||
@@ -131,10 +131,6 @@ from basic_memory.deps.services import (
|
||||
DirectoryServiceV2Dep,
|
||||
get_directory_service_v2_external,
|
||||
DirectoryServiceV2ExternalDep,
|
||||
get_graph_intelligence_service_v2_external,
|
||||
GraphIntelligenceServiceV2ExternalDep,
|
||||
get_fcm_service_v2_external,
|
||||
FCMServiceV2ExternalDep,
|
||||
)
|
||||
|
||||
from basic_memory.deps.importers import (
|
||||
@@ -273,10 +269,6 @@ __all__ = [
|
||||
"DirectoryServiceV2Dep",
|
||||
"get_directory_service_v2_external",
|
||||
"DirectoryServiceV2ExternalDep",
|
||||
"get_graph_intelligence_service_v2_external",
|
||||
"GraphIntelligenceServiceV2ExternalDep",
|
||||
"get_fcm_service_v2_external",
|
||||
"FCMServiceV2ExternalDep",
|
||||
# Importers
|
||||
"get_chatgpt_importer",
|
||||
"ChatGPTImporterDep",
|
||||
|
||||
@@ -39,8 +39,6 @@ from basic_memory.deps.repositories import (
|
||||
from basic_memory.markdown import EntityParser
|
||||
from basic_memory.markdown.markdown_processor import MarkdownProcessor
|
||||
from basic_memory.services import EntityService, ProjectService
|
||||
from basic_memory.services.fcm_service import FCMService
|
||||
from basic_memory.services.graph_intelligence_service import GraphIntelligenceService
|
||||
from basic_memory.services.context_service import ContextService
|
||||
from basic_memory.services.directory_service import DirectoryService
|
||||
from basic_memory.services.file_service import FileService
|
||||
@@ -360,30 +358,6 @@ async def get_context_service_v2_external(
|
||||
ContextServiceV2ExternalDep = Annotated[ContextService, Depends(get_context_service_v2_external)]
|
||||
|
||||
|
||||
# --- Graph Intelligence Service ---
|
||||
|
||||
|
||||
async def get_graph_intelligence_service_v2_external() -> GraphIntelligenceService:
|
||||
"""Create GraphIntelligenceService for v2 API (uses external_id routing)."""
|
||||
return GraphIntelligenceService()
|
||||
|
||||
|
||||
GraphIntelligenceServiceV2ExternalDep = Annotated[
|
||||
GraphIntelligenceService, Depends(get_graph_intelligence_service_v2_external)
|
||||
]
|
||||
|
||||
|
||||
# --- FCM Service ---
|
||||
|
||||
|
||||
async def get_fcm_service_v2_external() -> FCMService:
|
||||
"""Create FCMService for v2 API (uses external_id routing)."""
|
||||
return FCMService()
|
||||
|
||||
|
||||
FCMServiceV2ExternalDep = Annotated[FCMService, Depends(get_fcm_service_v2_external)]
|
||||
|
||||
|
||||
# --- Sync Service ---
|
||||
|
||||
|
||||
@@ -561,21 +535,6 @@ async def get_task_scheduler(
|
||||
async def _reindex_project(**_: Any) -> None:
|
||||
await search_service.reindex_all()
|
||||
|
||||
async def _sync_graph_entity(entity_id: int, **extra_payload: Any) -> None:
|
||||
# Trigger: graph-entity sync task is scheduled from graph lifecycle hooks.
|
||||
# Why: keep scheduler contract stable while graph index provider work lands in later phases.
|
||||
# Outcome: no-op in phase 1; task name remains valid for API and tool contracts.
|
||||
del entity_id, extra_payload
|
||||
|
||||
async def _sync_graph_project(force_full: bool = False, **_: Any) -> None:
|
||||
await _sync_project(force_full=force_full)
|
||||
|
||||
async def _reindex_graph_project(**_: Any) -> None:
|
||||
# Trigger: graph reindex requested.
|
||||
# Why: phase 1 has no dedicated graph index worker yet.
|
||||
# Outcome: run project sync path so writes stay coherent while graph provider ships.
|
||||
await _sync_project(force_full=True)
|
||||
|
||||
scheduler = LocalTaskScheduler(
|
||||
{
|
||||
"reindex_entity": _reindex_entity,
|
||||
@@ -583,9 +542,6 @@ async def get_task_scheduler(
|
||||
"sync_entity_vectors": _sync_entity_vectors,
|
||||
"sync_project": _sync_project,
|
||||
"reindex_project": _reindex_project,
|
||||
"sync_graph_entity": _sync_graph_entity,
|
||||
"sync_graph_project": _sync_graph_project,
|
||||
"reindex_graph_project": _reindex_graph_project,
|
||||
},
|
||||
test_mode=app_config.is_test_env,
|
||||
)
|
||||
|
||||
@@ -88,22 +88,6 @@ def normalize_frontmatter_value(value: Any) -> Any:
|
||||
return value
|
||||
|
||||
|
||||
def _coerce_to_string(value: Any) -> str:
|
||||
"""Coerce a frontmatter value to a string.
|
||||
|
||||
YAML can parse scalar-looking fields as lists when the author uses block
|
||||
sequence syntax. For fields like ``title`` and ``type`` that *must* be
|
||||
strings, this helper converts lists to a comma-separated string and any
|
||||
other non-string type via ``str()``.
|
||||
"""
|
||||
if isinstance(value, str):
|
||||
return value
|
||||
if isinstance(value, list):
|
||||
# Join list items, converting each to string first
|
||||
return ", ".join(str(item) for item in value)
|
||||
return str(value)
|
||||
|
||||
|
||||
def normalize_frontmatter_metadata(metadata: dict) -> dict:
|
||||
"""Normalize all values in frontmatter metadata dict.
|
||||
|
||||
@@ -249,15 +233,9 @@ class EntityParser:
|
||||
|
||||
content = strip_bom(content)
|
||||
|
||||
# Parse frontmatter with proper error handling for malformed YAML.
|
||||
# We use frontmatter.parse() instead of frontmatter.loads() because
|
||||
# loads() does Post(content, handler, **metadata), which crashes when
|
||||
# the YAML contains reserved keys like 'content' or 'handler'.
|
||||
# See basic-memory-cloud#375.
|
||||
# Parse frontmatter with proper error handling for malformed YAML
|
||||
try:
|
||||
fm_metadata, fm_content = frontmatter.parse(content)
|
||||
post = frontmatter.Post(fm_content)
|
||||
post.metadata.update(fm_metadata)
|
||||
post = frontmatter.loads(content)
|
||||
except yaml.YAMLError as e:
|
||||
logger.warning(
|
||||
f"Failed to parse YAML frontmatter in {file_path}: {e}. "
|
||||
@@ -270,21 +248,14 @@ class EntityParser:
|
||||
# Normalize frontmatter values
|
||||
metadata = normalize_frontmatter_metadata(post.metadata)
|
||||
|
||||
# Ensure required string fields are always strings.
|
||||
# YAML can parse these as lists when authors use block sequence syntax
|
||||
# (e.g. "title:\n - My Title"), causing 'list' has no attribute 'strip'
|
||||
# downstream. See basic-memory-cloud#376.
|
||||
# Ensure required fields have defaults
|
||||
title = metadata.get("title")
|
||||
if title is not None:
|
||||
title = _coerce_to_string(title)
|
||||
if not title or title == "None":
|
||||
metadata["title"] = file_path.stem
|
||||
else:
|
||||
metadata["title"] = title
|
||||
|
||||
note_type = metadata.get("type")
|
||||
if note_type is not None:
|
||||
note_type = _coerce_to_string(note_type)
|
||||
metadata["type"] = note_type if note_type is not None else "note"
|
||||
|
||||
tags = parse_tags(metadata.get("tags", [])) # pyright: ignore
|
||||
|
||||
@@ -154,13 +154,13 @@ async def get_client(
|
||||
# Outcome: route strictly based on explicit flag.
|
||||
if _explicit_routing():
|
||||
if _force_local_mode():
|
||||
logger.debug("Explicit local routing enabled - using ASGI client")
|
||||
logger.info("Explicit local routing enabled - using ASGI client")
|
||||
async with _asgi_client(timeout) as client:
|
||||
yield client
|
||||
return
|
||||
|
||||
if _force_cloud_mode():
|
||||
logger.debug("Explicit cloud routing enabled - using cloud proxy client")
|
||||
logger.info("Explicit cloud routing enabled - using cloud proxy client")
|
||||
async with _cloud_client(config, timeout, workspace=workspace) as client:
|
||||
yield client
|
||||
return
|
||||
@@ -172,7 +172,7 @@ async def get_client(
|
||||
if project_name is not None and not _explicit_routing():
|
||||
project_mode = config.get_project_mode(project_name)
|
||||
if project_mode == ProjectMode.CLOUD:
|
||||
logger.debug(f"Project '{project_name}' is cloud mode - using cloud proxy client")
|
||||
logger.info(f"Project '{project_name}' is cloud mode - using cloud proxy client")
|
||||
try:
|
||||
async with _cloud_client(config, timeout, workspace=workspace) as client:
|
||||
yield client
|
||||
@@ -183,13 +183,13 @@ async def get_client(
|
||||
) from exc
|
||||
return
|
||||
|
||||
logger.debug(f"Project '{project_name}' is local mode - using ASGI client")
|
||||
logger.info(f"Project '{project_name}' is local mode - using ASGI client")
|
||||
async with _asgi_client(timeout) as client:
|
||||
yield client
|
||||
return
|
||||
|
||||
# --- Default fallback ---
|
||||
logger.debug("Default routing - using ASGI client for local Basic Memory API")
|
||||
logger.info("Default routing - using ASGI client for local Basic Memory API")
|
||||
async with _asgi_client(timeout) as client:
|
||||
yield client
|
||||
|
||||
|
||||
@@ -18,8 +18,6 @@ from basic_memory.mcp.clients.directory import DirectoryClient
|
||||
from basic_memory.mcp.clients.resource import ResourceClient
|
||||
from basic_memory.mcp.clients.project import ProjectClient
|
||||
from basic_memory.mcp.clients.schema import SchemaClient
|
||||
from basic_memory.mcp.clients.graph import GraphClient
|
||||
from basic_memory.mcp.clients.fcm import FCMClient
|
||||
|
||||
__all__ = [
|
||||
"KnowledgeClient",
|
||||
@@ -29,6 +27,4 @@ __all__ = [
|
||||
"ResourceClient",
|
||||
"ProjectClient",
|
||||
"SchemaClient",
|
||||
"GraphClient",
|
||||
"FCMClient",
|
||||
]
|
||||
|
||||
@@ -1,56 +0,0 @@
|
||||
"""Typed client for FCM API operations."""
|
||||
|
||||
from httpx import AsyncClient
|
||||
|
||||
from basic_memory.mcp.tools.utils import call_post
|
||||
from basic_memory.schemas.graph_intelligence import (
|
||||
FCMExportRequest,
|
||||
FCMExportResponse,
|
||||
FCMImportRequest,
|
||||
FCMImportResponse,
|
||||
FCMRankActionsRequest,
|
||||
FCMRankActionsResponse,
|
||||
FCMSimulateRequest,
|
||||
FCMSimulateResponse,
|
||||
)
|
||||
|
||||
|
||||
class FCMClient:
|
||||
"""Typed client for FCM operations."""
|
||||
|
||||
def __init__(self, http_client: AsyncClient, project_id: str):
|
||||
self.http_client = http_client
|
||||
self.project_id = project_id
|
||||
self._base_path = f"/v2/projects/{project_id}/fcm"
|
||||
|
||||
async def simulate(self, request: FCMSimulateRequest) -> FCMSimulateResponse:
|
||||
response = await call_post(
|
||||
self.http_client,
|
||||
f"{self._base_path}/simulate",
|
||||
json=request.model_dump(mode="json"),
|
||||
)
|
||||
return FCMSimulateResponse.model_validate(response.json())
|
||||
|
||||
async def rank_actions(self, request: FCMRankActionsRequest) -> FCMRankActionsResponse:
|
||||
response = await call_post(
|
||||
self.http_client,
|
||||
f"{self._base_path}/rank-actions",
|
||||
json=request.model_dump(mode="json"),
|
||||
)
|
||||
return FCMRankActionsResponse.model_validate(response.json())
|
||||
|
||||
async def import_model(self, request: FCMImportRequest) -> FCMImportResponse:
|
||||
response = await call_post(
|
||||
self.http_client,
|
||||
f"{self._base_path}/import",
|
||||
json=request.model_dump(mode="json"),
|
||||
)
|
||||
return FCMImportResponse.model_validate(response.json())
|
||||
|
||||
async def export_model(self, request: FCMExportRequest) -> FCMExportResponse:
|
||||
response = await call_post(
|
||||
self.http_client,
|
||||
f"{self._base_path}/export",
|
||||
json=request.model_dump(mode="json"),
|
||||
)
|
||||
return FCMExportResponse.model_validate(response.json())
|
||||
@@ -1,62 +0,0 @@
|
||||
"""Typed client for graph intelligence API operations."""
|
||||
|
||||
from httpx import AsyncClient
|
||||
|
||||
from basic_memory.mcp.tools.utils import call_get, call_post
|
||||
from basic_memory.schemas.graph_intelligence import (
|
||||
GraphHealthResponse,
|
||||
GraphImpactRequest,
|
||||
GraphImpactResponse,
|
||||
GraphLineageRequest,
|
||||
GraphLineageResponse,
|
||||
GraphReindexRequest,
|
||||
GraphReindexResponse,
|
||||
)
|
||||
|
||||
|
||||
class GraphClient:
|
||||
"""Typed client for graph intelligence operations."""
|
||||
|
||||
def __init__(self, http_client: AsyncClient, project_id: str):
|
||||
self.http_client = http_client
|
||||
self.project_id = project_id
|
||||
self._base_path = f"/v2/projects/{project_id}/graph"
|
||||
|
||||
async def lineage(self, request: GraphLineageRequest) -> GraphLineageResponse:
|
||||
response = await call_post(
|
||||
self.http_client,
|
||||
f"{self._base_path}/lineage",
|
||||
json=request.model_dump(mode="json"),
|
||||
)
|
||||
return GraphLineageResponse.model_validate(response.json())
|
||||
|
||||
async def impact(self, request: GraphImpactRequest) -> GraphImpactResponse:
|
||||
response = await call_post(
|
||||
self.http_client,
|
||||
f"{self._base_path}/impact",
|
||||
json=request.model_dump(mode="json"),
|
||||
)
|
||||
return GraphImpactResponse.model_validate(response.json())
|
||||
|
||||
async def health(
|
||||
self, scope: str | None = None, timeframe: str | None = None
|
||||
) -> GraphHealthResponse:
|
||||
params: dict[str, str] = {}
|
||||
if scope is not None:
|
||||
params["scope"] = scope
|
||||
if timeframe is not None:
|
||||
params["timeframe"] = timeframe
|
||||
response = await call_get(
|
||||
self.http_client,
|
||||
f"{self._base_path}/health",
|
||||
params=params,
|
||||
)
|
||||
return GraphHealthResponse.model_validate(response.json())
|
||||
|
||||
async def reindex(self, request: GraphReindexRequest) -> GraphReindexResponse:
|
||||
response = await call_post(
|
||||
self.http_client,
|
||||
f"{self._base_path}/reindex",
|
||||
json=request.model_dump(mode="json"),
|
||||
)
|
||||
return GraphReindexResponse.model_validate(response.json())
|
||||
@@ -9,7 +9,7 @@ compatibility with existing MCP tools.
|
||||
"""
|
||||
|
||||
from contextlib import asynccontextmanager
|
||||
from typing import AsyncIterator, Awaitable, Callable, Optional, List, Tuple
|
||||
from typing import AsyncIterator, Optional, List, Tuple
|
||||
|
||||
from httpx import AsyncClient
|
||||
from httpx._types import (
|
||||
@@ -27,41 +27,6 @@ from basic_memory.schemas.v2 import ProjectResolveResponse
|
||||
from basic_memory.schemas.memory import memory_url_path
|
||||
from basic_memory.utils import generate_permalink, normalize_project_reference
|
||||
|
||||
# --- Workspace provider injection ---
|
||||
# Mirrors the set_client_factory() pattern in async_client.py.
|
||||
# The cloud MCP server sets a provider that queries its own database directly,
|
||||
# avoiding the control-plane HTTP round-trip that requires local credentials.
|
||||
_workspace_provider: Optional[Callable[[], Awaitable[list[WorkspaceInfo]]]] = None
|
||||
|
||||
|
||||
def set_workspace_provider(provider: Callable[[], Awaitable[list[WorkspaceInfo]]]) -> None:
|
||||
"""Override workspace discovery (for cloud app, testing, etc)."""
|
||||
global _workspace_provider
|
||||
_workspace_provider = provider
|
||||
|
||||
|
||||
async def _resolve_default_project_from_api() -> Optional[str]:
|
||||
"""Query the projects API for the default project.
|
||||
|
||||
Used as a fallback when ConfigManager has no local config (cloud mode).
|
||||
"""
|
||||
from basic_memory.mcp.async_client import get_client
|
||||
|
||||
try:
|
||||
async with get_client() as client:
|
||||
response = await client.get("/v2/projects/")
|
||||
if response.status_code == 200:
|
||||
project_list = ProjectList.model_validate(response.json())
|
||||
if project_list.default_project:
|
||||
return project_list.default_project
|
||||
# Fallback: find project with is_default=True
|
||||
for p in project_list.projects:
|
||||
if p.is_default:
|
||||
return p.name
|
||||
except Exception:
|
||||
pass
|
||||
return None
|
||||
|
||||
|
||||
async def resolve_project_parameter(
|
||||
project: Optional[str] = None,
|
||||
@@ -89,16 +54,11 @@ async def resolve_project_parameter(
|
||||
Returns:
|
||||
Resolved project name or None if no resolution possible
|
||||
"""
|
||||
# Load config for any values not explicitly provided.
|
||||
# ConfigManager reads from the local config file, which doesn't exist in cloud mode.
|
||||
# When it returns None, fall back to querying the projects API for the is_default flag.
|
||||
# Load config for any values not explicitly provided
|
||||
if default_project is None:
|
||||
config = ConfigManager().config
|
||||
default_project = config.default_project
|
||||
|
||||
if default_project is None:
|
||||
default_project = await _resolve_default_project_from_api()
|
||||
|
||||
# Create resolver with configuration and resolve
|
||||
resolver = ProjectResolver.from_env(
|
||||
default_project=default_project,
|
||||
@@ -143,19 +103,6 @@ async def get_available_workspaces(context: Optional[Context] = None) -> list[Wo
|
||||
if isinstance(cached_raw, list):
|
||||
return [WorkspaceInfo.model_validate(item) for item in cached_raw]
|
||||
|
||||
# Trigger: workspace provider was injected (e.g., by cloud MCP server)
|
||||
# Why: the cloud server IS the cloud — it can query its own database
|
||||
# directly instead of making an HTTP round-trip that requires local credentials
|
||||
# Outcome: use provider result, cache in context, skip control-plane client
|
||||
if _workspace_provider is not None:
|
||||
workspaces = await _workspace_provider()
|
||||
if context:
|
||||
await context.set_state(
|
||||
"available_workspaces",
|
||||
[ws.model_dump() for ws in workspaces],
|
||||
)
|
||||
return workspaces
|
||||
|
||||
from basic_memory.mcp.async_client import get_cloud_control_plane_client
|
||||
from basic_memory.mcp.tools.utils import call_get
|
||||
|
||||
@@ -472,7 +419,6 @@ async def get_project_client(
|
||||
_explicit_routing,
|
||||
_force_local_mode,
|
||||
get_client,
|
||||
is_factory_mode,
|
||||
)
|
||||
|
||||
# Step 1: Resolve project name from config (no network call)
|
||||
@@ -487,18 +433,6 @@ async def get_project_client(
|
||||
f"Available projects: {project_names}"
|
||||
)
|
||||
|
||||
# Step 1b: Factory injection (in-process cloud server)
|
||||
# Trigger: set_client_factory() was called (e.g., by cloud MCP server)
|
||||
# Why: the transport layer already resolved workspace and tenant context;
|
||||
# attempting cloud workspace resolution here would call the production
|
||||
# control-plane API with no valid credentials and fail with 401
|
||||
# Outcome: use the factory client directly, skip workspace resolution
|
||||
if is_factory_mode():
|
||||
async with get_client() as client:
|
||||
active_project = await get_active_project(client, resolved_project, context)
|
||||
yield client, active_project
|
||||
return
|
||||
|
||||
# Step 2: Check explicit routing BEFORE workspace resolution
|
||||
# Trigger: CLI passed --local or --cloud
|
||||
# Why: explicit flags must be deterministic — skip workspace entirely for --local
|
||||
|
||||
@@ -13,6 +13,7 @@ from pydantic import Field
|
||||
from basic_memory.mcp.server import mcp
|
||||
from basic_memory.mcp.tools.recent_activity import recent_activity
|
||||
from basic_memory.mcp.tools.search import search_notes
|
||||
from basic_memory.schemas.search import SearchResponse
|
||||
|
||||
|
||||
@mcp.prompt(
|
||||
@@ -41,12 +42,15 @@ async def continue_conversation(
|
||||
logger.info(f"Continuing session, topic: {topic}, timeframe: {timeframe}")
|
||||
|
||||
if topic:
|
||||
# Use json format to get structured data for result counting and branching
|
||||
result = await search_notes(query=topic, after_date=timeframe, output_format="json")
|
||||
# Search for the topic using the search tool directly
|
||||
result = await search_notes(query=topic, after_date=timeframe)
|
||||
|
||||
if isinstance(result, dict):
|
||||
if isinstance(result, SearchResponse):
|
||||
context_text = _format_continuation_results(result, topic)
|
||||
result_count = len(result.results)
|
||||
elif isinstance(result, dict):
|
||||
results = result.get("results", [])
|
||||
context_text = _format_continuation_results(results, topic)
|
||||
context_text = str(result)
|
||||
result_count = len(results)
|
||||
else:
|
||||
# Error string
|
||||
@@ -107,24 +111,23 @@ async def continue_conversation(
|
||||
return prompt
|
||||
|
||||
|
||||
def _format_continuation_results(results: list[dict], topic: str) -> str:
|
||||
"""Format search result dicts for conversation continuation context."""
|
||||
if not results:
|
||||
def _format_continuation_results(result: SearchResponse, topic: str) -> str:
|
||||
"""Format search results for conversation continuation context."""
|
||||
if not result.results:
|
||||
return f"No previous context found for '{topic}'."
|
||||
|
||||
lines = [f"## Previous Context for '{topic}'\n"]
|
||||
|
||||
for item in results:
|
||||
title = item.get("title", "Untitled")
|
||||
permalink = item.get("permalink", "")
|
||||
for item in result.results:
|
||||
title = item.title or "Untitled"
|
||||
permalink = item.permalink or ""
|
||||
|
||||
lines.append(f"### {title}")
|
||||
if permalink:
|
||||
lines.append(f"permalink: {permalink}")
|
||||
lines.append(f'Read with: `read_note("{permalink}")`')
|
||||
content = item.get("content")
|
||||
if content:
|
||||
content = content[:300] + "..." if len(content) > 300 else content
|
||||
lines.append(f"Read with: `read_note(\"{permalink}\")`")
|
||||
if item.content:
|
||||
content = item.content[:300] + "..." if len(item.content) > 300 else item.content
|
||||
lines.append(f"\n{content}")
|
||||
lines.append("")
|
||||
|
||||
|
||||
@@ -11,6 +11,7 @@ from pydantic import Field
|
||||
|
||||
from basic_memory.mcp.server import mcp
|
||||
from basic_memory.mcp.tools.search import search_notes
|
||||
from basic_memory.schemas.search import SearchResponse
|
||||
|
||||
|
||||
@mcp.prompt(
|
||||
@@ -38,14 +39,18 @@ async def search_prompt(
|
||||
"""
|
||||
logger.info(f"Searching knowledge base, query: {query}, timeframe: {timeframe}")
|
||||
|
||||
# Use json format to get structured data for result counting and formatting
|
||||
result = await search_notes(query=query, after_date=timeframe, output_format="json")
|
||||
# Call the search tool directly — it returns SearchResponse, dict, or error string
|
||||
result = await search_notes(query=query, after_date=timeframe)
|
||||
|
||||
# Format the tool output into a prompt with guidance
|
||||
if isinstance(result, dict):
|
||||
if isinstance(result, SearchResponse):
|
||||
result_count = len(result.results)
|
||||
result_text = _format_search_results(result, query)
|
||||
elif isinstance(result, dict):
|
||||
# json output format
|
||||
results = result.get("results", [])
|
||||
result_count = len(results)
|
||||
result_text = _format_search_results(results, query)
|
||||
result_text = str(result)
|
||||
else:
|
||||
# Error string from search tool
|
||||
result_count = 0
|
||||
@@ -71,27 +76,28 @@ async def search_prompt(
|
||||
""")
|
||||
|
||||
|
||||
def _format_search_results(results: list[dict], query: str) -> str:
|
||||
"""Format search result dicts into readable markdown."""
|
||||
if not results:
|
||||
def _format_search_results(result: SearchResponse, query: str) -> str:
|
||||
"""Format SearchResponse into readable markdown."""
|
||||
if not result.results:
|
||||
return f"No results found for '{query}'."
|
||||
|
||||
lines = [f"Found {len(results)} results:\n"]
|
||||
lines = [f"Found {len(result.results)} results:\n"]
|
||||
|
||||
for item in results:
|
||||
title = item.get("title", "Untitled")
|
||||
permalink = item.get("permalink", "")
|
||||
score = item.get("score")
|
||||
score_text = f" (score: {score:.2f})" if score else ""
|
||||
for item in result.results:
|
||||
title = item.title or "Untitled"
|
||||
permalink = item.permalink or ""
|
||||
score = f" (score: {item.score:.2f})" if item.score else ""
|
||||
|
||||
lines.append(f"- **{title}**{score_text}")
|
||||
lines.append(f"- **{title}**{score}")
|
||||
if permalink:
|
||||
lines.append(f" permalink: {permalink}")
|
||||
content = item.get("content")
|
||||
if content:
|
||||
if item.content:
|
||||
# Truncate content snippet
|
||||
content = content[:200] + "..." if len(content) > 200 else content
|
||||
content = item.content[:200] + "..." if len(item.content) > 200 else item.content
|
||||
lines.append(f" {content}")
|
||||
lines.append("")
|
||||
|
||||
if result.has_more:
|
||||
lines.append("*More results available. Use page=2 to see next page.*")
|
||||
|
||||
return "\n".join(lines)
|
||||
|
||||
@@ -57,16 +57,6 @@ await write_note(
|
||||
)
|
||||
```
|
||||
|
||||
> **Important**: `write_note` errors if the note already exists. Use `edit_note` for incremental changes, or pass `overwrite=True` to replace.
|
||||
|
||||
```python
|
||||
# Preferred: update an existing note incrementally
|
||||
await edit_note(identifier="Topic", operation="append", content="\n- [category] new fact")
|
||||
|
||||
# Alternative: replace the entire note
|
||||
await write_note(title="Topic", content="...", folder="notes", overwrite=True)
|
||||
```
|
||||
|
||||
### Reading Knowledge
|
||||
|
||||
```python
|
||||
@@ -80,27 +70,11 @@ content = await read_note("memory://folder/topic", project="main")
|
||||
### Searching
|
||||
|
||||
```python
|
||||
# Basic text search
|
||||
results = await search_notes(query="authentication", project="main")
|
||||
|
||||
# Search types: "text" (default), "title", "permalink", "vector"/"semantic", "hybrid"
|
||||
# Default is "hybrid" when semantic search is enabled, "text" otherwise
|
||||
results = await search_notes(query="auth flow", search_type="hybrid")
|
||||
|
||||
# Tag shorthand in query (multiple tags: "tag:x AND tag:y" or "tag:x tag:y")
|
||||
results = await search_notes(query="tag:security")
|
||||
results = await search_notes(query="tag:coffee AND tag:brewing")
|
||||
|
||||
# Filter-only search (no query needed)
|
||||
results = await search_notes(tags=["security", "auth"], status="active")
|
||||
|
||||
# Metadata filters with operators: $in, $gt, $gte, $lt, $lte, $between
|
||||
results = await search_notes(
|
||||
metadata_filters={"priority": {"$in": ["high", "critical"]}}
|
||||
query="authentication",
|
||||
project="main",
|
||||
page_size=10
|
||||
)
|
||||
|
||||
# Override similarity threshold for vector/hybrid search
|
||||
results = await search_notes(query="auth", search_type="hybrid", min_similarity=0.5)
|
||||
```
|
||||
|
||||
### Building Context
|
||||
@@ -188,8 +162,6 @@ activity = await recent_activity(project="main")
|
||||
- 2-3 relations per note
|
||||
- Meaningful categories and relation types
|
||||
|
||||
**Prefer `edit_note` for updates** — use `write_note` only for new notes.
|
||||
|
||||
**Search before creating:**
|
||||
```python
|
||||
# Find existing entities to reference
|
||||
@@ -229,14 +201,6 @@ except:
|
||||
results = await search_notes(query="test", project=projects[0].name)
|
||||
```
|
||||
|
||||
**Note already exists:**
|
||||
```python
|
||||
# write_note returns an error if the note exists — use edit_note or overwrite
|
||||
await edit_note(identifier="Existing Topic", operation="append", content="\n- [update] new info")
|
||||
# Or replace entirely:
|
||||
await write_note(title="Existing Topic", content="...", folder="notes", overwrite=True)
|
||||
```
|
||||
|
||||
**Forward references:**
|
||||
```python
|
||||
# Check response for unresolved relations
|
||||
@@ -292,14 +256,13 @@ context = await build_context(url=f"memory://{results[0].permalink}", project="m
|
||||
|
||||
| Tool | Purpose | Key Params |
|
||||
|------|---------|------------|
|
||||
| `write_note` | Create new | title, content, folder, project, overwrite |
|
||||
| `write_note` | Create/update | title, content, folder, project |
|
||||
| `read_note` | Read content | identifier, project |
|
||||
| `edit_note` | Modify existing | identifier, operation, content, project |
|
||||
| `search_notes` | Find notes | query, search_type, tags, metadata_filters, project |
|
||||
| `search_notes` | Find notes | query, project |
|
||||
| `build_context` | Graph traversal | url, depth, project |
|
||||
| `recent_activity` | Recent changes | timeframe, project |
|
||||
| `list_memory_projects` | Show projects | (none) |
|
||||
| `list_workspaces` | Show workspaces | (none) |
|
||||
|
||||
## memory:// URL Format
|
||||
|
||||
@@ -307,7 +270,6 @@ context = await build_context(url=f"memory://{results[0].permalink}", project="m
|
||||
- `memory://folder/title` - By folder + title
|
||||
- `memory://permalink` - By permalink
|
||||
- `memory://folder/*` - All in folder
|
||||
- `memory://project-name/folder/title` - Cross-project (auto-routes to the correct project)
|
||||
|
||||
For full documentation: https://docs.basicmemory.com
|
||||
|
||||
|
||||
@@ -18,22 +18,12 @@ from basic_memory.mcp.tools.view_note import view_note
|
||||
from basic_memory.mcp.tools.write_note import write_note
|
||||
from basic_memory.mcp.tools.cloud_info import cloud_info
|
||||
from basic_memory.mcp.tools.release_notes import release_notes
|
||||
from basic_memory.mcp.tools.search import search_notes
|
||||
from basic_memory.mcp.tools.search import search_notes, search_by_metadata
|
||||
from basic_memory.mcp.tools.canvas import canvas
|
||||
from basic_memory.mcp.tools.list_directory import list_directory
|
||||
from basic_memory.mcp.tools.edit_note import edit_note
|
||||
from basic_memory.mcp.tools.move_note import move_note
|
||||
from basic_memory.mcp.tools.workspaces import list_workspaces
|
||||
from basic_memory.mcp.tools.graph_intelligence import (
|
||||
graph_lineage,
|
||||
graph_impact,
|
||||
graph_health,
|
||||
graph_reindex,
|
||||
fcm_simulate,
|
||||
fcm_rank_actions,
|
||||
fcm_import_model,
|
||||
fcm_export_model,
|
||||
)
|
||||
from basic_memory.mcp.tools.project_management import (
|
||||
list_memory_projects,
|
||||
create_memory_project,
|
||||
@@ -54,15 +44,7 @@ __all__ = [
|
||||
"delete_note",
|
||||
"delete_project",
|
||||
"edit_note",
|
||||
"fcm_export_model",
|
||||
"fcm_import_model",
|
||||
"fcm_rank_actions",
|
||||
"fcm_simulate",
|
||||
"fetch",
|
||||
"graph_health",
|
||||
"graph_impact",
|
||||
"graph_lineage",
|
||||
"graph_reindex",
|
||||
"list_directory",
|
||||
"list_memory_projects",
|
||||
"list_workspaces",
|
||||
@@ -76,6 +58,7 @@ __all__ = [
|
||||
"schema_infer",
|
||||
"schema_validate",
|
||||
"search",
|
||||
"search_by_metadata",
|
||||
"search_notes",
|
||||
# "search_notes_ui",
|
||||
"view_note",
|
||||
|
||||
@@ -22,6 +22,74 @@ from basic_memory.schemas.memory import (
|
||||
RelationSummary,
|
||||
)
|
||||
|
||||
# --- Fields to strip from each model (redundant with parent entity) ---
|
||||
|
||||
_OBSERVATION_STRIP = {
|
||||
"observation_id",
|
||||
"entity_id",
|
||||
"entity_external_id",
|
||||
"title",
|
||||
"file_path",
|
||||
"created_at",
|
||||
}
|
||||
_RELATION_STRIP = {
|
||||
"relation_id",
|
||||
"entity_id",
|
||||
"from_entity_id",
|
||||
"from_entity_external_id",
|
||||
"to_entity_id",
|
||||
"to_entity_external_id",
|
||||
"title",
|
||||
"file_path",
|
||||
"created_at",
|
||||
}
|
||||
_ENTITY_STRIP = {"entity_id", "created_at"}
|
||||
_METADATA_STRIP = {"total_results", "generated_at"}
|
||||
|
||||
|
||||
def _slim_summary(summary: EntitySummary | RelationSummary | ObservationSummary) -> dict:
|
||||
"""Strip redundant fields from a summary model based on its type."""
|
||||
if isinstance(summary, ObservationSummary):
|
||||
strip = _OBSERVATION_STRIP
|
||||
elif isinstance(summary, RelationSummary):
|
||||
strip = _RELATION_STRIP
|
||||
else:
|
||||
strip = _ENTITY_STRIP
|
||||
|
||||
data = summary.model_dump()
|
||||
for key in strip:
|
||||
data.pop(key, None)
|
||||
return data
|
||||
|
||||
|
||||
def _slim_context(graph: GraphContext) -> dict:
|
||||
"""Transform GraphContext into a slimmed dict, stripping redundant fields.
|
||||
|
||||
Reduces payload size ~40% by removing fields on nested objects that
|
||||
duplicate information already present on the parent entity (IDs,
|
||||
timestamps, file paths).
|
||||
"""
|
||||
slimmed_results = []
|
||||
for result in graph.results:
|
||||
slimmed_results.append(
|
||||
{
|
||||
"primary_result": _slim_summary(result.primary_result),
|
||||
"observations": [_slim_summary(obs) for obs in result.observations],
|
||||
"related_results": [_slim_summary(rel) for rel in result.related_results],
|
||||
}
|
||||
)
|
||||
|
||||
metadata = graph.metadata.model_dump()
|
||||
for key in _METADATA_STRIP:
|
||||
metadata.pop(key, None)
|
||||
|
||||
return {
|
||||
"results": slimmed_results,
|
||||
"metadata": metadata,
|
||||
"page": graph.page,
|
||||
"page_size": graph.page_size,
|
||||
}
|
||||
|
||||
|
||||
def _format_entity_block(result: ContextResult) -> str:
|
||||
"""Format a single context result as a markdown block."""
|
||||
@@ -126,7 +194,7 @@ def _format_context_markdown(graph: GraphContext, project: str) -> str:
|
||||
- Or standard formats like "7d", "24h"
|
||||
|
||||
Format options:
|
||||
- "json" (default): Structured JSON with internal fields excluded
|
||||
- "json" (default): Slimmed JSON with redundant fields removed
|
||||
- "text": Compact markdown text for LLM consumption
|
||||
""",
|
||||
annotations={"readOnlyHint": True, "openWorldHint": False},
|
||||
@@ -163,12 +231,12 @@ async def build_context(
|
||||
page: Page number of results to return (default: 1)
|
||||
page_size: Number of results to return per page (default: 10)
|
||||
max_related: Maximum number of related results to return (default: 10)
|
||||
output_format: Response format - "json" for structured JSON dict,
|
||||
output_format: Response format - "json" for slimmed JSON dict,
|
||||
"text" for compact markdown text
|
||||
context: Optional FastMCP context for performance caching.
|
||||
|
||||
Returns:
|
||||
dict (output_format="json"): Structured JSON with internal fields excluded
|
||||
dict (output_format="json"): Slimmed JSON with redundant fields removed
|
||||
str (output_format="text"): Compact markdown representation
|
||||
|
||||
Examples:
|
||||
@@ -224,4 +292,4 @@ async def build_context(
|
||||
if output_format == "text":
|
||||
return _format_context_markdown(graph, active_project.name)
|
||||
|
||||
return graph.model_dump()
|
||||
return _slim_context(graph)
|
||||
|
||||
@@ -7,13 +7,13 @@ a list containing a single `{"type": "text", "text": "{...json...}"}` item.
|
||||
|
||||
import json
|
||||
from typing import Any, Dict, List, Optional
|
||||
|
||||
from fastmcp import Context
|
||||
from loguru import logger
|
||||
from fastmcp import Context
|
||||
|
||||
from basic_memory.mcp.server import mcp
|
||||
from basic_memory.mcp.tools.read_note import read_note
|
||||
from basic_memory.mcp.tools.search import search_notes
|
||||
from basic_memory.mcp.tools.read_note import read_note
|
||||
from basic_memory.config import ConfigManager
|
||||
from basic_memory.schemas.search import SearchResponse, SearchResult
|
||||
|
||||
|
||||
@@ -113,12 +113,16 @@ async def search(
|
||||
logger.info(f"ChatGPT search request: query='{query}'")
|
||||
|
||||
try:
|
||||
# Let search_notes resolve the default project via get_project_client(),
|
||||
# which works in both local mode (ConfigManager) and cloud mode (database).
|
||||
# ChatGPT tools don't expose project parameter, so use default project
|
||||
config = ConfigManager().config
|
||||
default_project = config.default_project
|
||||
|
||||
# Call underlying search_notes with sensible defaults for ChatGPT
|
||||
results = await search_notes(
|
||||
query=query,
|
||||
project=default_project, # Use default project for ChatGPT
|
||||
page=1,
|
||||
page_size=10,
|
||||
page_size=10, # Reasonable default for ChatGPT consumption
|
||||
output_format="json",
|
||||
context=context,
|
||||
)
|
||||
@@ -176,13 +180,17 @@ async def fetch(
|
||||
logger.info(f"ChatGPT fetch request: id='{id}'")
|
||||
|
||||
try:
|
||||
# Let read_note resolve the default project via get_project_client(),
|
||||
# which works in both local mode (ConfigManager) and cloud mode (database).
|
||||
# ChatGPT tools don't expose project parameter, so use default project
|
||||
config = ConfigManager().config
|
||||
default_project = config.default_project
|
||||
|
||||
# Call underlying read_note function (default output_format="text" returns str)
|
||||
content = str(
|
||||
await read_note(
|
||||
identifier=id,
|
||||
project=default_project, # Use default project for ChatGPT
|
||||
page=1,
|
||||
page_size=10,
|
||||
page_size=10, # Default pagination
|
||||
context=context,
|
||||
)
|
||||
)
|
||||
|
||||
@@ -7,36 +7,6 @@ from fastmcp import Context
|
||||
|
||||
from basic_memory.mcp.project_context import get_project_client, add_project_metadata
|
||||
from basic_memory.mcp.server import mcp
|
||||
from basic_memory.schemas.base import Entity
|
||||
from basic_memory.schemas.response import EntityResponse
|
||||
from basic_memory.utils import validate_project_path
|
||||
|
||||
|
||||
def _parse_identifier_to_title_and_directory(identifier: str) -> tuple[str, str]:
|
||||
"""Parse an identifier into (title, directory) for creating a new note.
|
||||
|
||||
Strips memory:// prefix if present, then splits on the last '/' to
|
||||
separate the directory path from the note title.
|
||||
|
||||
Examples:
|
||||
"conversations/my-note" → ("my-note", "conversations")
|
||||
"my-note" → ("my-note", "")
|
||||
"a/b/c/my-note" → ("my-note", "a/b/c")
|
||||
"memory://a/b/note" → ("note", "a/b")
|
||||
"""
|
||||
cleaned = identifier
|
||||
if cleaned.startswith("memory://"):
|
||||
cleaned = cleaned[len("memory://") :]
|
||||
|
||||
if "/" in cleaned:
|
||||
last_slash = cleaned.rfind("/")
|
||||
directory = cleaned[:last_slash]
|
||||
title = cleaned[last_slash + 1 :]
|
||||
else:
|
||||
directory = ""
|
||||
title = cleaned
|
||||
|
||||
return title, directory
|
||||
|
||||
|
||||
def _format_error_response(
|
||||
@@ -49,19 +19,15 @@ def _format_error_response(
|
||||
) -> str:
|
||||
"""Format helpful error responses for edit_note failures that guide the AI to retry successfully."""
|
||||
|
||||
# Entity not found errors — only reachable for find_replace/replace_section
|
||||
# because append/prepend auto-create the note when it doesn't exist
|
||||
# Entity not found errors
|
||||
if "Entity not found" in error_message or "entity not found" in error_message.lower():
|
||||
return f"""# Edit Failed - Note Not Found
|
||||
|
||||
The note with identifier '{identifier}' could not be found. The `find_replace` and `replace_section` operations require an existing note with content to modify.
|
||||
|
||||
**Tip:** `append` and `prepend` operations automatically create the note if it doesn't exist.
|
||||
The note with identifier '{identifier}' could not be found. Edit operations require an exact match (no fuzzy matching).
|
||||
|
||||
## Suggestions to try:
|
||||
1. **Use append/prepend instead**: These operations will create the note automatically if it doesn't exist
|
||||
2. **Search for the note first**: Use `search_notes("{project or "project-name"}", "{identifier.split("/")[-1]}")` to find similar notes with exact identifiers
|
||||
3. **Try different exact identifier formats**:
|
||||
1. **Search for the note first**: Use `search_notes("{project or "project-name"}", "{identifier.split("/")[-1]}")` to find similar notes with exact identifiers
|
||||
2. **Try different exact identifier formats**:
|
||||
- If you used a permalink like "folder/note-title", try the exact title: "{identifier.split("/")[-1].replace("-", " ").title()}"
|
||||
- If you used a title, try the exact permalink format: "{identifier.lower().replace(" ", "-")}"
|
||||
- Use `read_note("{project or "project-name"}", "{identifier}")` first to verify the note exists and get the exact identifier
|
||||
@@ -169,7 +135,7 @@ async def edit_note(
|
||||
workspace: Optional[str] = None,
|
||||
section: Optional[str] = None,
|
||||
find_text: Optional[str] = None,
|
||||
expected_replacements: Optional[int] = None,
|
||||
expected_replacements: int = 1,
|
||||
output_format: Literal["text", "json"] = "text",
|
||||
context: Context | None = None,
|
||||
) -> str | dict:
|
||||
@@ -186,10 +152,10 @@ async def edit_note(
|
||||
Must be an exact match - fuzzy matching is not supported for edit operations.
|
||||
Use search_notes() or read_note() first to find the correct identifier if uncertain.
|
||||
operation: The editing operation to perform:
|
||||
- "append": Add content to the end of the note (creates the note if it doesn't exist)
|
||||
- "prepend": Add content to the beginning of the note (creates the note if it doesn't exist)
|
||||
- "find_replace": Replace occurrences of find_text with content (note must exist)
|
||||
- "replace_section": Replace content under a specific markdown header (note must exist)
|
||||
- "append": Add content to the end of the note
|
||||
- "prepend": Add content to the beginning of the note
|
||||
- "find_replace": Replace occurrences of find_text with content
|
||||
- "replace_section": Replace content under a specific markdown header
|
||||
content: The content to add or use for replacement
|
||||
project: Project name to edit in. Optional - server will resolve using hierarchy.
|
||||
If unknown, use list_memory_projects() to discover available projects.
|
||||
@@ -250,9 +216,6 @@ async def edit_note(
|
||||
search_notes() first to find the correct identifier. The tool provides detailed
|
||||
error messages with suggestions if operations fail.
|
||||
"""
|
||||
# Resolve effective default: allow MCP clients to send null for optional int field
|
||||
effective_replacements = expected_replacements if expected_replacements is not None else 1
|
||||
|
||||
async with get_project_client(project, workspace, context) as (client, active_project):
|
||||
logger.info("MCP tool call", tool="edit_note", identifier=identifier, operation=operation)
|
||||
|
||||
@@ -277,118 +240,48 @@ async def edit_note(
|
||||
# Use typed KnowledgeClient for API calls
|
||||
knowledge_client = KnowledgeClient(client, active_project.external_id)
|
||||
|
||||
file_created = False
|
||||
entity_id = ""
|
||||
result: EntityResponse | None = None
|
||||
# Resolve identifier to entity ID
|
||||
entity_id = await knowledge_client.resolve_entity(identifier)
|
||||
|
||||
# Try to resolve the entity; for append/prepend, create it if not found
|
||||
try:
|
||||
entity_id = await knowledge_client.resolve_entity(identifier)
|
||||
except Exception as resolve_error:
|
||||
# Trigger: entity does not exist yet
|
||||
# Why: append/prepend can meaningfully create a new note from the content,
|
||||
# while find_replace/replace_section require existing content to modify
|
||||
# Outcome: note is created via the same path as write_note
|
||||
error_msg = str(resolve_error).lower()
|
||||
is_not_found = "entity not found" in error_msg or "not found" in error_msg
|
||||
# Prepare the edit request data
|
||||
edit_data = {
|
||||
"operation": operation,
|
||||
"content": content,
|
||||
}
|
||||
|
||||
if is_not_found and operation in ("append", "prepend"):
|
||||
title, directory = _parse_identifier_to_title_and_directory(identifier)
|
||||
# Add optional parameters
|
||||
if section:
|
||||
edit_data["section"] = section
|
||||
if find_text:
|
||||
edit_data["find_text"] = find_text
|
||||
if expected_replacements != 1: # Only send if different from default
|
||||
edit_data["expected_replacements"] = str(expected_replacements)
|
||||
|
||||
# Validate directory path (same security check as write_note)
|
||||
project_path = active_project.home
|
||||
if directory and not validate_project_path(directory, project_path):
|
||||
logger.warning(
|
||||
"Attempted path traversal attack blocked",
|
||||
directory=directory,
|
||||
project=active_project.name,
|
||||
)
|
||||
if output_format == "json":
|
||||
return {
|
||||
"title": title,
|
||||
"permalink": None,
|
||||
"file_path": None,
|
||||
"checksum": None,
|
||||
"operation": operation,
|
||||
"fileCreated": False,
|
||||
"error": "SECURITY_VALIDATION_ERROR",
|
||||
}
|
||||
return f"# Error\n\nDirectory path '{directory}' is not allowed - paths must stay within project boundaries"
|
||||
# Call the PATCH endpoint
|
||||
result = await knowledge_client.patch_entity(entity_id, edit_data, fast=False)
|
||||
|
||||
entity = Entity(
|
||||
title=title,
|
||||
directory=directory,
|
||||
content_type="text/markdown",
|
||||
content=content,
|
||||
)
|
||||
# Format summary
|
||||
summary = [
|
||||
f"# Edited note ({operation})",
|
||||
f"project: {active_project.name}",
|
||||
f"file_path: {result.file_path}",
|
||||
f"permalink: {result.permalink}",
|
||||
f"checksum: {result.checksum[:8] if result.checksum else 'unknown'}",
|
||||
]
|
||||
|
||||
logger.info(
|
||||
"Creating note via edit_note auto-create",
|
||||
title=title,
|
||||
directory=directory,
|
||||
operation=operation,
|
||||
)
|
||||
result = await knowledge_client.create_entity(entity.model_dump(), fast=False)
|
||||
file_created = True
|
||||
else:
|
||||
# find_replace/replace_section require existing content — re-raise
|
||||
raise resolve_error
|
||||
|
||||
# --- Standard edit path (entity already existed) ---
|
||||
if not file_created:
|
||||
# Prepare the edit request data
|
||||
edit_data = {
|
||||
"operation": operation,
|
||||
"content": content,
|
||||
}
|
||||
|
||||
# Add optional parameters
|
||||
if section:
|
||||
edit_data["section"] = section
|
||||
if find_text:
|
||||
edit_data["find_text"] = find_text
|
||||
if effective_replacements != 1: # Only send if different from default
|
||||
edit_data["expected_replacements"] = str(effective_replacements)
|
||||
|
||||
# Call the PATCH endpoint
|
||||
result = await knowledge_client.patch_entity(entity_id, edit_data, fast=False)
|
||||
|
||||
# --- Format response ---
|
||||
# result is always set: either by create_entity (auto-create) or patch_entity (edit)
|
||||
assert result is not None
|
||||
if file_created:
|
||||
summary = [
|
||||
f"# Created note ({operation})",
|
||||
f"project: {active_project.name}",
|
||||
f"file_path: {result.file_path}",
|
||||
f"permalink: {result.permalink}",
|
||||
f"checksum: {result.checksum[:8] if result.checksum else 'unknown'}",
|
||||
"fileCreated: true",
|
||||
]
|
||||
# Add operation-specific details
|
||||
if operation == "append":
|
||||
lines_added = len(content.split("\n"))
|
||||
summary.append(f"operation: Created note with {lines_added} lines")
|
||||
else:
|
||||
summary = [
|
||||
f"# Edited note ({operation})",
|
||||
f"project: {active_project.name}",
|
||||
f"file_path: {result.file_path}",
|
||||
f"permalink: {result.permalink}",
|
||||
f"checksum: {result.checksum[:8] if result.checksum else 'unknown'}",
|
||||
]
|
||||
|
||||
# Add operation-specific details
|
||||
if operation == "append":
|
||||
lines_added = len(content.split("\n"))
|
||||
summary.append(f"operation: Added {lines_added} lines to end of note")
|
||||
elif operation == "prepend":
|
||||
lines_added = len(content.split("\n"))
|
||||
summary.append(f"operation: Added {lines_added} lines to beginning of note")
|
||||
elif operation == "find_replace":
|
||||
# For find_replace, we can't easily count replacements from here
|
||||
# since we don't have the original content, but the server handled it
|
||||
summary.append("operation: Find and replace operation completed")
|
||||
elif operation == "replace_section":
|
||||
summary.append(f"operation: Replaced content under section '{section}'")
|
||||
summary.append(f"operation: Added {lines_added} lines to end of note")
|
||||
elif operation == "prepend":
|
||||
lines_added = len(content.split("\n"))
|
||||
summary.append(f"operation: Added {lines_added} lines to beginning of note")
|
||||
elif operation == "find_replace":
|
||||
# For find_replace, we can't easily count replacements from here
|
||||
# since we don't have the original content, but the server handled it
|
||||
summary.append("operation: Find and replace operation completed")
|
||||
elif operation == "replace_section":
|
||||
summary.append(f"operation: Replaced content under section '{section}'")
|
||||
|
||||
# Count observations by category (reuse logic from write_note)
|
||||
categories = {}
|
||||
@@ -420,7 +313,6 @@ async def edit_note(
|
||||
permalink=result.permalink,
|
||||
observations_count=len(result.observations),
|
||||
relations_count=len(result.relations),
|
||||
file_created=file_created,
|
||||
)
|
||||
|
||||
if output_format == "json":
|
||||
@@ -430,7 +322,6 @@ async def edit_note(
|
||||
"file_path": result.file_path,
|
||||
"checksum": result.checksum,
|
||||
"operation": operation,
|
||||
"fileCreated": file_created,
|
||||
}
|
||||
|
||||
summary_result = "\n".join(summary)
|
||||
@@ -445,14 +336,8 @@ async def edit_note(
|
||||
"file_path": None,
|
||||
"checksum": None,
|
||||
"operation": operation,
|
||||
"fileCreated": False,
|
||||
"error": str(e),
|
||||
}
|
||||
return _format_error_response(
|
||||
str(e),
|
||||
operation,
|
||||
identifier,
|
||||
find_text,
|
||||
effective_replacements,
|
||||
active_project.name,
|
||||
str(e), operation, identifier, find_text, expected_replacements, active_project.name
|
||||
)
|
||||
|
||||
@@ -1,271 +0,0 @@
|
||||
"""MCP tools for graph intelligence and FCM contracts."""
|
||||
|
||||
from typing import Any, Literal
|
||||
|
||||
from fastmcp import Context
|
||||
|
||||
from basic_memory.mcp.project_context import get_project_client
|
||||
from basic_memory.mcp.server import mcp
|
||||
from basic_memory.schemas.graph_intelligence import (
|
||||
FCMExportRequest,
|
||||
FCMImportRequest,
|
||||
FCMRankActionsRequest,
|
||||
FCMSimulateRequest,
|
||||
GraphImpactRequest,
|
||||
GraphLineageRequest,
|
||||
GraphReindexRequest,
|
||||
)
|
||||
|
||||
|
||||
def _format_lineage_text(result: dict[str, Any]) -> str:
|
||||
root = result["root"]["title"]
|
||||
path_count = len(result.get("paths", []))
|
||||
return f"# Graph Lineage\n\nRoot: {root}\nPaths: {path_count}"
|
||||
|
||||
|
||||
def _format_impact_text(result: dict[str, Any]) -> str:
|
||||
target = result["target"]["title"]
|
||||
affected = len(result.get("affected", []))
|
||||
return f"# Graph Impact\n\nTarget: {target}\nAffected: {affected}"
|
||||
|
||||
|
||||
def _format_health_text(result: dict[str, Any]) -> str:
|
||||
metrics = result["metrics"]
|
||||
return (
|
||||
"# Graph Health\n\n"
|
||||
f"- orphan_rate: {metrics['orphan_rate']}\n"
|
||||
f"- stale_central_nodes: {metrics['stale_central_nodes']}\n"
|
||||
f"- overloaded_hubs: {metrics['overloaded_hubs']}\n"
|
||||
f"- contradiction_candidates: {metrics['contradiction_candidates']}"
|
||||
)
|
||||
|
||||
|
||||
def _format_fcm_simulate_text(result: dict[str, Any]) -> str:
|
||||
deltas = len(result.get("deltas", []))
|
||||
converged = result["stability"]["converged"]
|
||||
return f"# FCM Simulation\n\nDeltas: {deltas}\nConverged: {converged}"
|
||||
|
||||
|
||||
def _format_fcm_rank_text(result: dict[str, Any]) -> str:
|
||||
goal = result["goal"]["label"]
|
||||
count = len(result.get("recommendations", []))
|
||||
return f"# FCM Action Ranking\n\nGoal: {goal}\nRecommendations: {count}"
|
||||
|
||||
|
||||
@mcp.tool(annotations={"readOnlyHint": True, "openWorldHint": False})
|
||||
async def graph_lineage(
|
||||
start: str,
|
||||
goal: str | None = None,
|
||||
max_hops: int = 4,
|
||||
relation_filters: list[str] | None = None,
|
||||
project: str | None = None,
|
||||
workspace: str | None = None,
|
||||
output_format: Literal["json", "text"] = "json",
|
||||
context: Context | None = None,
|
||||
) -> dict[str, Any] | str:
|
||||
"""Get lineage paths from a start node toward an optional goal."""
|
||||
from basic_memory.mcp.clients import GraphClient
|
||||
|
||||
request = GraphLineageRequest(
|
||||
start=start,
|
||||
goal=goal,
|
||||
max_hops=max_hops,
|
||||
relation_filters=relation_filters or [],
|
||||
)
|
||||
async with get_project_client(project, workspace, context) as (client, active_project):
|
||||
graph_client = GraphClient(client, active_project.external_id)
|
||||
result = await graph_client.lineage(request)
|
||||
payload = result.model_dump(mode="json")
|
||||
if output_format == "text":
|
||||
return _format_lineage_text(payload)
|
||||
return payload
|
||||
|
||||
|
||||
@mcp.tool(annotations={"readOnlyHint": True, "openWorldHint": False})
|
||||
async def graph_impact(
|
||||
target: str,
|
||||
horizon: int,
|
||||
relation_filters: list[str] | None = None,
|
||||
include_reasons: bool = True,
|
||||
project: str | None = None,
|
||||
workspace: str | None = None,
|
||||
output_format: Literal["json", "text"] = "json",
|
||||
context: Context | None = None,
|
||||
) -> dict[str, Any] | str:
|
||||
"""Get impact radius from a target node."""
|
||||
from basic_memory.mcp.clients import GraphClient
|
||||
|
||||
request = GraphImpactRequest(
|
||||
target=target,
|
||||
horizon=horizon,
|
||||
relation_filters=relation_filters or [],
|
||||
include_reasons=include_reasons,
|
||||
)
|
||||
async with get_project_client(project, workspace, context) as (client, active_project):
|
||||
graph_client = GraphClient(client, active_project.external_id)
|
||||
result = await graph_client.impact(request)
|
||||
payload = result.model_dump(mode="json")
|
||||
if output_format == "text":
|
||||
return _format_impact_text(payload)
|
||||
return payload
|
||||
|
||||
|
||||
@mcp.tool(annotations={"readOnlyHint": True, "openWorldHint": False})
|
||||
async def graph_health(
|
||||
scope: str | None = None,
|
||||
timeframe: str | None = None,
|
||||
project: str | None = None,
|
||||
workspace: str | None = None,
|
||||
output_format: Literal["json", "text"] = "json",
|
||||
context: Context | None = None,
|
||||
) -> dict[str, Any] | str:
|
||||
"""Get graph health metrics and issues."""
|
||||
from basic_memory.mcp.clients import GraphClient
|
||||
|
||||
async with get_project_client(project, workspace, context) as (client, active_project):
|
||||
graph_client = GraphClient(client, active_project.external_id)
|
||||
result = await graph_client.health(scope=scope, timeframe=timeframe)
|
||||
payload = result.model_dump(mode="json")
|
||||
if output_format == "text":
|
||||
return _format_health_text(payload)
|
||||
return payload
|
||||
|
||||
|
||||
@mcp.tool(annotations={"readOnlyHint": False, "openWorldHint": False})
|
||||
async def fcm_simulate(
|
||||
actions: list[dict[str, Any]],
|
||||
scenario: dict[str, Any] | None = None,
|
||||
clamp_rules: list[dict[str, Any]] | None = None,
|
||||
project: str | None = None,
|
||||
workspace: str | None = None,
|
||||
output_format: Literal["json", "text"] = "json",
|
||||
context: Context | None = None,
|
||||
) -> dict[str, Any] | str:
|
||||
"""Run an FCM simulation with optional scenario controls."""
|
||||
from basic_memory.mcp.clients import FCMClient
|
||||
|
||||
request = FCMSimulateRequest.model_validate(
|
||||
{
|
||||
"actions": actions,
|
||||
"scenario": scenario or {},
|
||||
"clamp_rules": clamp_rules or [],
|
||||
}
|
||||
)
|
||||
async with get_project_client(project, workspace, context) as (client, active_project):
|
||||
fcm_client = FCMClient(client, active_project.external_id)
|
||||
result = await fcm_client.simulate(request)
|
||||
payload = result.model_dump(mode="json")
|
||||
if output_format == "text":
|
||||
return _format_fcm_simulate_text(payload)
|
||||
return payload
|
||||
|
||||
|
||||
@mcp.tool(annotations={"readOnlyHint": True, "openWorldHint": False})
|
||||
async def fcm_rank_actions(
|
||||
goal: str,
|
||||
constraints: dict[str, Any] | None = None,
|
||||
top_k: int = 10,
|
||||
project: str | None = None,
|
||||
workspace: str | None = None,
|
||||
output_format: Literal["json", "text"] = "json",
|
||||
context: Context | None = None,
|
||||
) -> dict[str, Any] | str:
|
||||
"""Rank intervention actions for an FCM goal node."""
|
||||
from basic_memory.mcp.clients import FCMClient
|
||||
|
||||
request = FCMRankActionsRequest.model_validate(
|
||||
{
|
||||
"goal": goal,
|
||||
"constraints": constraints or {},
|
||||
"top_k": top_k,
|
||||
}
|
||||
)
|
||||
async with get_project_client(project, workspace, context) as (client, active_project):
|
||||
fcm_client = FCMClient(client, active_project.external_id)
|
||||
result = await fcm_client.rank_actions(request)
|
||||
payload = result.model_dump(mode="json")
|
||||
if output_format == "text":
|
||||
return _format_fcm_rank_text(payload)
|
||||
return payload
|
||||
|
||||
|
||||
@mcp.tool(annotations={"readOnlyHint": False, "openWorldHint": False})
|
||||
async def fcm_import_model(
|
||||
source: str,
|
||||
format: Literal["csv_bundle_v1"] = "csv_bundle_v1",
|
||||
merge_mode: Literal["replace", "upsert"] = "upsert",
|
||||
project: str | None = None,
|
||||
workspace: str | None = None,
|
||||
output_format: Literal["json", "text"] = "json",
|
||||
context: Context | None = None,
|
||||
) -> dict[str, Any] | str:
|
||||
"""Import an FCM model from an external source."""
|
||||
from basic_memory.mcp.clients import FCMClient
|
||||
|
||||
request = FCMImportRequest(source=source, format=format, merge_mode=merge_mode)
|
||||
async with get_project_client(project, workspace, context) as (client, active_project):
|
||||
fcm_client = FCMClient(client, active_project.external_id)
|
||||
result = await fcm_client.import_model(request)
|
||||
payload = result.model_dump(mode="json")
|
||||
if output_format == "text":
|
||||
return (
|
||||
"# FCM Import\n\n"
|
||||
f"Import ID: {payload['import_id']}\n"
|
||||
f"Nodes Loaded: {payload['nodes_loaded']}\n"
|
||||
f"Edges Loaded: {payload['edges_loaded']}"
|
||||
)
|
||||
return payload
|
||||
|
||||
|
||||
@mcp.tool(annotations={"readOnlyHint": True, "openWorldHint": False})
|
||||
async def fcm_export_model(
|
||||
format: Literal["csv_bundle_v1"] = "csv_bundle_v1",
|
||||
selection: dict[str, Any] | None = None,
|
||||
project: str | None = None,
|
||||
workspace: str | None = None,
|
||||
output_format: Literal["json", "text"] = "json",
|
||||
context: Context | None = None,
|
||||
) -> dict[str, Any] | str:
|
||||
"""Export an FCM model selection."""
|
||||
from basic_memory.mcp.clients import FCMClient
|
||||
|
||||
request = FCMExportRequest.model_validate(
|
||||
{
|
||||
"format": format,
|
||||
"selection": selection or {},
|
||||
}
|
||||
)
|
||||
async with get_project_client(project, workspace, context) as (client, active_project):
|
||||
fcm_client = FCMClient(client, active_project.external_id)
|
||||
result = await fcm_client.export_model(request)
|
||||
payload = result.model_dump(mode="json")
|
||||
if output_format == "text":
|
||||
return (
|
||||
"# FCM Export\n\n"
|
||||
f"Export ID: {payload['export_id']}\n"
|
||||
f"Node Count: {payload['node_count']}\n"
|
||||
f"Edge Count: {payload['edge_count']}"
|
||||
)
|
||||
return payload
|
||||
|
||||
|
||||
@mcp.tool(annotations={"readOnlyHint": False, "openWorldHint": False})
|
||||
async def graph_reindex(
|
||||
mode: Literal["full", "incremental"] = "incremental",
|
||||
reason: str | None = None,
|
||||
project: str | None = None,
|
||||
workspace: str | None = None,
|
||||
output_format: Literal["json", "text"] = "json",
|
||||
context: Context | None = None,
|
||||
) -> dict[str, Any] | str:
|
||||
"""Queue a graph reindex for the active project."""
|
||||
from basic_memory.mcp.clients import GraphClient
|
||||
|
||||
request = GraphReindexRequest(mode=mode, reason=reason)
|
||||
async with get_project_client(project, workspace, context) as (client, active_project):
|
||||
graph_client = GraphClient(client, active_project.external_id)
|
||||
result = await graph_client.reindex(request)
|
||||
payload = result.model_dump(mode="json")
|
||||
if output_format == "text":
|
||||
return f"# Graph Reindex\n\nJob ID: {payload['job_id']}\nStatus: {payload['status']}"
|
||||
return payload
|
||||
@@ -11,141 +11,7 @@ from fastmcp import Context
|
||||
|
||||
from basic_memory.mcp.project_context import get_project_client
|
||||
from basic_memory.mcp.server import mcp
|
||||
from basic_memory.schemas.schema import DriftReport, InferenceReport, ValidationReport
|
||||
|
||||
|
||||
def _format_validation_report(report: ValidationReport) -> str:
|
||||
"""Render a ValidationReport as readable markdown.
|
||||
|
||||
Produces output the LLM can display directly instead of trying to
|
||||
interpret raw JSON, which leads to "undefined — invalid" rendering.
|
||||
"""
|
||||
lines: list[str] = []
|
||||
|
||||
# --- Header ---
|
||||
type_label = report.note_type or "all"
|
||||
lines.append(f"# Schema Validation: {type_label}")
|
||||
lines.append("")
|
||||
lines.append(
|
||||
f"Notes: {report.total_notes} | Valid: {report.valid_count} "
|
||||
f"| Warnings: {report.warning_count} | Errors: {report.error_count}"
|
||||
)
|
||||
lines.append("")
|
||||
|
||||
# --- Per-note results ---
|
||||
for r in report.results:
|
||||
status = "valid" if r.passed else "INVALID"
|
||||
lines.append(f"- **{r.note_identifier}** — {status}")
|
||||
for w in r.warnings:
|
||||
lines.append(f" - warning: {w}")
|
||||
for e in r.errors:
|
||||
lines.append(f" - error: {e}")
|
||||
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def _format_inference_report(report: InferenceReport) -> str:
|
||||
"""Render an InferenceReport as readable markdown.
|
||||
|
||||
Without this formatter the LLM receives raw JSON and renders
|
||||
field names as "undefined".
|
||||
"""
|
||||
lines: list[str] = []
|
||||
|
||||
# --- Header ---
|
||||
lines.append(f"# Schema Inference: {report.note_type}")
|
||||
lines.append("")
|
||||
lines.append(f"Notes analyzed: {report.notes_analyzed}")
|
||||
lines.append("")
|
||||
|
||||
# --- Suggested schema YAML ---
|
||||
if report.suggested_schema:
|
||||
lines.append("## Suggested Schema")
|
||||
lines.append("")
|
||||
lines.append("```yaml")
|
||||
lines.append("---")
|
||||
lines.append(f"title: {report.note_type.title()}")
|
||||
lines.append("type: schema")
|
||||
lines.append(f"entity: {report.note_type}")
|
||||
lines.append("version: 1")
|
||||
lines.append("schema:")
|
||||
for field_name, field_def in report.suggested_schema.items():
|
||||
lines.append(f" {field_name}: {field_def}")
|
||||
lines.append("---")
|
||||
lines.append("```")
|
||||
lines.append("")
|
||||
|
||||
# --- Field frequency table ---
|
||||
if report.field_frequencies:
|
||||
lines.append("## Field Frequencies")
|
||||
lines.append("")
|
||||
for f in report.field_frequencies:
|
||||
pct = f"{f.percentage:.0%}"
|
||||
req_marker = "required" if f.name in report.suggested_required else "optional"
|
||||
samples = ", ".join(f.sample_values[:3]) if f.sample_values else ""
|
||||
sample_str = f" (e.g. {samples})" if samples else ""
|
||||
lines.append(
|
||||
f"- **{f.name}** ({f.source}) — {pct} ({f.count}/{f.total}) "
|
||||
f"[{req_marker}]{sample_str}"
|
||||
)
|
||||
lines.append("")
|
||||
|
||||
# --- Excluded fields ---
|
||||
if report.excluded:
|
||||
lines.append("## Excluded (below threshold)")
|
||||
lines.append("")
|
||||
for name in report.excluded:
|
||||
lines.append(f"- {name}")
|
||||
lines.append("")
|
||||
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def _format_drift_report(report: DriftReport) -> str:
|
||||
"""Render a DriftReport as readable markdown.
|
||||
|
||||
Without this formatter the LLM receives raw JSON and renders
|
||||
field names as "undefined".
|
||||
"""
|
||||
lines: list[str] = []
|
||||
|
||||
# --- Header ---
|
||||
lines.append(f"# Schema Drift: {report.note_type}")
|
||||
lines.append("")
|
||||
|
||||
has_drift = report.new_fields or report.dropped_fields or report.cardinality_changes
|
||||
|
||||
if not has_drift:
|
||||
lines.append("No drift detected — schema matches actual usage.")
|
||||
return "\n".join(lines)
|
||||
|
||||
# --- New fields ---
|
||||
if report.new_fields:
|
||||
lines.append("## New Fields (in notes but not in schema)")
|
||||
lines.append("")
|
||||
for f in report.new_fields:
|
||||
pct = f"{f.percentage:.0%}"
|
||||
lines.append(f"- **{f.name}** ({f.source}) — {pct} ({f.count}/{f.total})")
|
||||
lines.append("")
|
||||
|
||||
# --- Dropped fields ---
|
||||
if report.dropped_fields:
|
||||
lines.append("## Dropped Fields (in schema but rare in notes)")
|
||||
lines.append("")
|
||||
for f in report.dropped_fields:
|
||||
pct = f"{f.percentage:.0%}"
|
||||
lines.append(f"- **{f.name}** ({f.source}) — {pct} ({f.count}/{f.total})")
|
||||
lines.append("")
|
||||
|
||||
# --- Cardinality changes ---
|
||||
if report.cardinality_changes:
|
||||
lines.append("## Cardinality Changes")
|
||||
lines.append("")
|
||||
for change in report.cardinality_changes:
|
||||
lines.append(f"- {change}")
|
||||
lines.append("")
|
||||
|
||||
return "\n".join(lines)
|
||||
from basic_memory.schemas.schema import ValidationReport, InferenceReport, DriftReport
|
||||
|
||||
|
||||
def _no_notes_guidance(note_type: str, tool_name: str) -> str:
|
||||
@@ -276,25 +142,24 @@ async def schema_validate(
|
||||
# Trigger: no entities of this type exist in the project
|
||||
# Why: can't validate notes that don't exist yet
|
||||
# Outcome: return guidance on creating notes of this type
|
||||
effective_type = note_type or result.note_type or "unknown"
|
||||
if result.total_entities == 0:
|
||||
if note_type and result.total_entities == 0:
|
||||
if output_format == "json":
|
||||
return {"error": f"No notes found of type '{effective_type}'"}
|
||||
return _no_notes_guidance(effective_type, "schema_validate")
|
||||
return {"error": f"No notes found of type '{note_type}'"}
|
||||
return _no_notes_guidance(note_type, "schema_validate")
|
||||
|
||||
# --- No schema guard ---
|
||||
# Trigger: entities exist but none were validated (no schema found)
|
||||
# Why: notes of this type exist but no schema was found, so none were validated
|
||||
# Outcome: return guidance on how to create a schema
|
||||
if result.total_notes == 0:
|
||||
if note_type and result.total_notes == 0:
|
||||
if output_format == "json":
|
||||
return {"error": f"No schema found for type '{effective_type}'"}
|
||||
return _no_schema_guidance(effective_type, "schema_validate")
|
||||
return {"error": f"No schema found for type '{note_type}'"}
|
||||
return _no_schema_guidance(note_type, "schema_validate")
|
||||
|
||||
if output_format == "json":
|
||||
return result.model_dump(mode="json", exclude_none=True)
|
||||
|
||||
return _format_validation_report(result)
|
||||
return result
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Schema validation failed: {e}, project: {active_project.name}")
|
||||
@@ -321,7 +186,7 @@ async def schema_infer(
|
||||
workspace: Optional[str] = None,
|
||||
output_format: Literal["text", "json"] = "text",
|
||||
context: Context | None = None,
|
||||
) -> str | dict:
|
||||
) -> InferenceReport | str | dict:
|
||||
"""Analyze existing notes and suggest a schema definition.
|
||||
|
||||
Examines observation categories and relation types across all notes
|
||||
@@ -409,7 +274,7 @@ async def schema_infer(
|
||||
if output_format == "json":
|
||||
return result.model_dump(mode="json", exclude_none=True)
|
||||
|
||||
return _format_inference_report(result)
|
||||
return result
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Schema inference failed: {e}, project: {active_project.name}")
|
||||
@@ -435,7 +300,7 @@ async def schema_diff(
|
||||
workspace: Optional[str] = None,
|
||||
output_format: Literal["text", "json"] = "text",
|
||||
context: Context | None = None,
|
||||
) -> str | dict:
|
||||
) -> DriftReport | str | dict:
|
||||
"""Detect drift between a schema definition and actual note usage.
|
||||
|
||||
Compares the existing schema for a note type against how notes of
|
||||
@@ -497,7 +362,7 @@ async def schema_diff(
|
||||
if output_format == "json":
|
||||
return result.model_dump(mode="json", exclude_none=True)
|
||||
|
||||
return _format_drift_report(result)
|
||||
return result
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Schema diff failed: {e}, project: {active_project.name}")
|
||||
|
||||
@@ -1,6 +1,5 @@
|
||||
"""Search tools for Basic Memory MCP server."""
|
||||
|
||||
import re
|
||||
from textwrap import dedent
|
||||
from typing import List, Optional, Dict, Any, Literal
|
||||
|
||||
@@ -251,46 +250,6 @@ Error searching for '{query}': {error_message}
|
||||
- **Patterns**: `tag:example`, `category:observation`"""
|
||||
|
||||
|
||||
def _format_search_markdown(result: SearchResponse, project: str, query: str | None) -> str:
|
||||
"""Format SearchResponse as compact markdown text.
|
||||
|
||||
Produces a human-readable markdown representation suitable for LLM
|
||||
consumption when structured data isn't needed.
|
||||
"""
|
||||
if not result.results:
|
||||
return f"No results found for '{query or ''}' in project '{project}'."
|
||||
|
||||
parts = []
|
||||
|
||||
# --- Header ---
|
||||
if query:
|
||||
parts.append(f"# Search Results: {query}")
|
||||
else:
|
||||
parts.append("# Search Results")
|
||||
parts.append(f"*project: {project}*")
|
||||
parts.append("")
|
||||
|
||||
# --- Result blocks ---
|
||||
for r in result.results:
|
||||
parts.append(f"### {r.title}")
|
||||
parts.append(f"- permalink: {r.permalink}")
|
||||
parts.append(f"- score: {r.score:.4f}")
|
||||
if r.matched_chunk:
|
||||
parts.append(f"- match: {r.matched_chunk[:200]}")
|
||||
parts.append("")
|
||||
|
||||
# --- Footer with pagination ---
|
||||
parts.append("---")
|
||||
count = len(result.results)
|
||||
parts.append(
|
||||
f"*{count} result{'s' if count != 1 else ''}"
|
||||
f" | page {result.current_page}, page_size {result.page_size}"
|
||||
f"{' | more available' if result.has_more else ''}*"
|
||||
)
|
||||
|
||||
return "\n".join(parts)
|
||||
|
||||
|
||||
@mcp.tool(
|
||||
description="Search across all content in the knowledge base with advanced syntax support.",
|
||||
# TODO: re-enable once MCP client rendering is working
|
||||
@@ -298,7 +257,7 @@ def _format_search_markdown(result: SearchResponse, project: str, query: str | N
|
||||
annotations={"readOnlyHint": True, "openWorldHint": False},
|
||||
)
|
||||
async def search_notes(
|
||||
query: Optional[str] = None,
|
||||
query: str,
|
||||
project: Optional[str] = None,
|
||||
workspace: Optional[str] = None,
|
||||
page: int = 1,
|
||||
@@ -313,7 +272,7 @@ async def search_notes(
|
||||
status: Optional[str] = None,
|
||||
min_similarity: Optional[float] = None,
|
||||
context: Context | None = None,
|
||||
) -> dict | str:
|
||||
) -> SearchResponse | dict | str:
|
||||
"""Search across all content in the knowledge base with comprehensive syntax support.
|
||||
|
||||
This tool searches the knowledge base using full-text search, pattern matching,
|
||||
@@ -343,9 +302,9 @@ async def search_notes(
|
||||
- `search_notes("work-project", "category:observation")` - Filter by observation categories
|
||||
- `search_notes("team-docs", "author:username")` - Find content by author (if metadata available)
|
||||
|
||||
**Note:** `tag:` shorthand is automatically converted to a `tags` filter, so it works
|
||||
with any search type (text, hybrid, vector). You can also use the `tags` parameter
|
||||
directly: `search_notes("project", "query", tags=["my-tag"])`
|
||||
**Note:** `tag:` shorthand requires `search_type="text"` when semantic search is enabled
|
||||
(the default is hybrid). Alternatively, use the `tags` parameter for tag filtering with
|
||||
any search type: `search_notes("project", "query", tags=["my-tag"])`
|
||||
|
||||
### Search Type Examples
|
||||
- `search_notes("my-project", "Meeting", search_type="title")` - Search only in titles
|
||||
@@ -374,10 +333,8 @@ async def search_notes(
|
||||
- Nested keys use dot notation (e.g., `"schema.confidence"`).
|
||||
|
||||
### Filter-only Searches
|
||||
Omit `query` (or pass None) when only using structured filters:
|
||||
- `search_notes(metadata_filters={"type": "spec"}, project="my-project")`
|
||||
- `search_notes(tags=["security"], project="my-project")`
|
||||
- `search_notes(status="draft", project="my-project")`
|
||||
You can pass an empty query string when only using structured filters:
|
||||
- `search_notes("my-project", "", metadata_filters={"type": "spec"})`
|
||||
|
||||
### Convenience Filters
|
||||
`tags` and `status` are shorthand for metadata_filters. If the same key exists in
|
||||
@@ -390,8 +347,7 @@ async def search_notes(
|
||||
- `search_notes("archive", "docs/2024-*", search_type="permalink")` - Year-based permalink search
|
||||
|
||||
Args:
|
||||
query: Optional search query string (supports boolean operators, phrases, patterns).
|
||||
Omit or pass None for filter-only searches using metadata_filters, tags, or status.
|
||||
query: The search query string (supports boolean operators, phrases, patterns)
|
||||
project: Project name to search in. Optional - server will resolve using hierarchy.
|
||||
If unknown, use list_memory_projects() to discover available projects.
|
||||
page: The page number of results to return (default 1)
|
||||
@@ -413,8 +369,7 @@ async def search_notes(
|
||||
context: Optional FastMCP context for performance caching.
|
||||
|
||||
Returns:
|
||||
Formatted markdown text (output_format="text"), dict (output_format="json"),
|
||||
or helpful error guidance string if search fails
|
||||
SearchResponse with results and pagination info, or helpful error guidance if search fails
|
||||
|
||||
Examples:
|
||||
# Basic text search
|
||||
@@ -480,77 +435,48 @@ async def search_notes(
|
||||
note_types = note_types or []
|
||||
entity_types = entity_types or []
|
||||
|
||||
# Parse tag:<value> shorthand at tool level so it works with all search modes.
|
||||
# Handles "tag:security", "tag:coffee tag:brewing", "tag:coffee AND tag:brewing".
|
||||
# Without this, hybrid/vector modes fail because they require non-empty text,
|
||||
# but the service-layer tag: parser clears the text after the mode is set.
|
||||
if query and "tag:" in query.lower():
|
||||
# Extract tag values, splitting comma-separated lists (e.g. "tag:coffee,brewing")
|
||||
raw_values = re.findall(r"tag:(\S+)", query, flags=re.IGNORECASE)
|
||||
tag_values = [v for raw in raw_values for v in raw.split(",") if v]
|
||||
if tag_values:
|
||||
# Merge with any explicitly provided tags
|
||||
tags = list(set((tags or []) + tag_values))
|
||||
# Remove tag: tokens and boolean connectors, keep remaining text as query
|
||||
remainder = re.sub(r"tag:\S+", "", query, flags=re.IGNORECASE)
|
||||
remainder = re.sub(r"\b(AND|OR|NOT)\b", "", remainder).strip()
|
||||
query = remainder or None
|
||||
|
||||
# Detect project from memory URL prefix before routing
|
||||
if project is None and query is not None:
|
||||
if project is None:
|
||||
detected = detect_project_from_url_prefix(query, ConfigManager().config)
|
||||
if detected:
|
||||
project = detected
|
||||
|
||||
async with get_project_client(project, workspace, context) as (client, active_project):
|
||||
# Handle memory:// URLs by resolving to permalink search
|
||||
is_memory_url = False
|
||||
if query is not None:
|
||||
_, resolved_query, is_memory_url = await resolve_project_and_path(
|
||||
client, query, project, context
|
||||
)
|
||||
if is_memory_url:
|
||||
query = resolved_query
|
||||
_, resolved_query, is_memory_url = await resolve_project_and_path(
|
||||
client, query, project, context
|
||||
)
|
||||
effective_search_type = search_type or _default_search_type()
|
||||
if is_memory_url:
|
||||
query = resolved_query
|
||||
effective_search_type = "permalink"
|
||||
|
||||
try:
|
||||
# Create a SearchQuery object based on the parameters
|
||||
search_query = SearchQuery()
|
||||
|
||||
# Only map search_type to query fields when there is an actual query string.
|
||||
# When query is None/empty, skip the search mode block — filters-only path.
|
||||
effective_query = (query or "").strip()
|
||||
if effective_query:
|
||||
valid_search_types = {
|
||||
"text",
|
||||
"title",
|
||||
"permalink",
|
||||
"vector",
|
||||
"semantic",
|
||||
"hybrid",
|
||||
}
|
||||
if effective_search_type == "text":
|
||||
search_query.text = effective_query
|
||||
search_query.retrieval_mode = SearchRetrievalMode.FTS
|
||||
elif effective_search_type in ("vector", "semantic"):
|
||||
search_query.text = effective_query
|
||||
search_query.retrieval_mode = SearchRetrievalMode.VECTOR
|
||||
elif effective_search_type == "hybrid":
|
||||
search_query.text = effective_query
|
||||
search_query.retrieval_mode = SearchRetrievalMode.HYBRID
|
||||
elif effective_search_type == "title":
|
||||
search_query.title = effective_query
|
||||
elif effective_search_type == "permalink" and "*" in effective_query:
|
||||
search_query.permalink_match = effective_query
|
||||
elif effective_search_type == "permalink":
|
||||
search_query.permalink = effective_query
|
||||
else:
|
||||
raise ValueError(
|
||||
f"Invalid search_type '{effective_search_type}'. "
|
||||
f"Valid options: {', '.join(sorted(valid_search_types))}"
|
||||
)
|
||||
# Map search_type to the appropriate query field and retrieval mode
|
||||
valid_search_types = {"text", "title", "permalink", "vector", "semantic", "hybrid"}
|
||||
if effective_search_type == "text":
|
||||
search_query.text = query
|
||||
search_query.retrieval_mode = SearchRetrievalMode.FTS
|
||||
elif effective_search_type in ("vector", "semantic"):
|
||||
search_query.text = query
|
||||
search_query.retrieval_mode = SearchRetrievalMode.VECTOR
|
||||
elif effective_search_type == "hybrid":
|
||||
search_query.text = query
|
||||
search_query.retrieval_mode = SearchRetrievalMode.HYBRID
|
||||
elif effective_search_type == "title":
|
||||
search_query.title = query
|
||||
elif effective_search_type == "permalink" and "*" in query:
|
||||
search_query.permalink_match = query
|
||||
elif effective_search_type == "permalink":
|
||||
search_query.permalink = query
|
||||
else:
|
||||
raise ValueError(
|
||||
f"Invalid search_type '{effective_search_type}'. "
|
||||
f"Valid options: {', '.join(sorted(valid_search_types))}"
|
||||
)
|
||||
|
||||
# Add optional filters if provided (empty lists are treated as no filter)
|
||||
if entity_types:
|
||||
@@ -560,13 +486,6 @@ async def search_notes(
|
||||
if after_date:
|
||||
search_query.after_date = after_date
|
||||
if metadata_filters:
|
||||
# Alias common column/model names to their frontmatter key equivalents.
|
||||
# Users often pass "note_type" (the entity model column) when the
|
||||
# frontmatter field is actually "type".
|
||||
_METADATA_KEY_ALIASES = {"note_type": "type"}
|
||||
metadata_filters = {
|
||||
_METADATA_KEY_ALIASES.get(k, k): v for k, v in metadata_filters.items()
|
||||
}
|
||||
search_query.metadata_filters = metadata_filters
|
||||
if tags:
|
||||
search_query.tags = tags
|
||||
@@ -575,22 +494,7 @@ async def search_notes(
|
||||
if min_similarity is not None:
|
||||
search_query.min_similarity = min_similarity
|
||||
|
||||
# Reject searches with no criteria at all
|
||||
if search_query.no_criteria():
|
||||
return (
|
||||
"# No Search Criteria\n\n"
|
||||
"Please provide at least one of: `query`, `metadata_filters`, "
|
||||
"`tags`, `status`, `note_types`, `entity_types`, or `after_date`."
|
||||
)
|
||||
|
||||
# Default to entity-level results to avoid returning individual
|
||||
# observations/relations as separate search results (see issue #31).
|
||||
# Applied after no_criteria() so that the implicit default doesn't
|
||||
# mask a truly empty search request.
|
||||
if not search_query.entity_types:
|
||||
search_query.entity_types = [SearchItemType("entity")]
|
||||
|
||||
logger.debug(f"Searching for {search_query} in project {active_project.name}")
|
||||
logger.info(f"Searching for {search_query} in project {active_project.name}")
|
||||
# Import here to avoid circular import (tools → clients → utils → tools)
|
||||
from basic_memory.mcp.clients import SearchClient
|
||||
|
||||
@@ -604,7 +508,7 @@ async def search_notes(
|
||||
|
||||
# Check if we got no results and provide helpful guidance
|
||||
if not result.results:
|
||||
logger.debug(
|
||||
logger.info(
|
||||
f"Search returned no results for query: {query} in project {active_project.name}"
|
||||
)
|
||||
# Don't treat this as an error, but the user might want guidance
|
||||
@@ -613,13 +517,91 @@ async def search_notes(
|
||||
if output_format == "json":
|
||||
return result.model_dump(mode="json", exclude_none=True)
|
||||
|
||||
return _format_search_markdown(result, active_project.name, query)
|
||||
return result
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Search failed for query '{query}': {e}, project: {active_project.name}")
|
||||
# Return formatted error message as string for better user experience
|
||||
return _format_search_error_response(
|
||||
active_project.name, str(e), query, effective_search_type
|
||||
)
|
||||
|
||||
|
||||
@mcp.tool(
|
||||
description="Search entities by structured frontmatter metadata.",
|
||||
annotations={"readOnlyHint": True, "openWorldHint": False},
|
||||
)
|
||||
async def search_by_metadata(
|
||||
filters: Dict[str, Any],
|
||||
project: Optional[str] = None,
|
||||
workspace: Optional[str] = None,
|
||||
limit: int = 20,
|
||||
offset: int = 0,
|
||||
context: Context | None = None,
|
||||
) -> SearchResponse | str:
|
||||
"""Search entities by structured frontmatter metadata.
|
||||
|
||||
Args:
|
||||
filters: Dictionary of metadata filters (e.g., {"status": "in-progress"})
|
||||
project: Project name to search in. Optional - server will resolve using hierarchy.
|
||||
limit: Maximum number of results to return
|
||||
offset: Number of results to skip (for pagination)
|
||||
context: Optional FastMCP context for performance caching.
|
||||
|
||||
Returns:
|
||||
SearchResponse with results, or helpful error guidance if search fails
|
||||
"""
|
||||
if limit <= 0:
|
||||
return "# Error\n\n`limit` must be greater than 0."
|
||||
|
||||
# Build a structured-only search query
|
||||
search_query = SearchQuery()
|
||||
search_query.metadata_filters = filters
|
||||
search_query.entity_types = [SearchItemType.ENTITY]
|
||||
|
||||
# Convert offset/limit to page/page_size (API uses paging)
|
||||
page_size = limit
|
||||
page = (offset // limit) + 1
|
||||
offset_within_page = offset % limit
|
||||
|
||||
async with get_project_client(project, workspace, context) as (client, active_project):
|
||||
logger.info(
|
||||
f"Structured search in project {active_project.name} filters={filters} limit={limit} offset={offset}"
|
||||
)
|
||||
|
||||
try:
|
||||
from basic_memory.mcp.clients import SearchClient
|
||||
|
||||
search_client = SearchClient(client, active_project.external_id)
|
||||
result = await search_client.search(
|
||||
search_query.model_dump(),
|
||||
page=page,
|
||||
page_size=page_size,
|
||||
)
|
||||
|
||||
# Apply offset within page, fetch next page if needed
|
||||
if offset_within_page:
|
||||
remaining = result.results[offset_within_page:]
|
||||
if len(remaining) < limit:
|
||||
next_page = page + 1
|
||||
extra = await search_client.search(
|
||||
search_query.model_dump(),
|
||||
page=next_page,
|
||||
page_size=page_size,
|
||||
)
|
||||
remaining.extend(extra.results[: max(0, limit - len(remaining))])
|
||||
result = SearchResponse(
|
||||
results=remaining[:limit],
|
||||
current_page=page,
|
||||
page_size=page_size,
|
||||
)
|
||||
|
||||
return result
|
||||
|
||||
except Exception as e:
|
||||
logger.error(
|
||||
f"Search failed for query '{query or ''}': {e}, project: {active_project.name}"
|
||||
f"Metadata search failed for filters '{filters}': {e}, project: {active_project.name}"
|
||||
)
|
||||
# Return formatted error message as string for better user experience
|
||||
return _format_search_error_response(
|
||||
active_project.name, str(e), query or "", effective_search_type
|
||||
active_project.name, str(e), str(filters), "metadata"
|
||||
)
|
||||
|
||||
@@ -1,11 +1,9 @@
|
||||
"""Write note tool for Basic Memory MCP server."""
|
||||
|
||||
import textwrap
|
||||
from typing import List, Union, Optional, Literal
|
||||
|
||||
from loguru import logger
|
||||
|
||||
from basic_memory.config import ConfigManager
|
||||
from basic_memory.mcp.project_context import get_project_client, add_project_metadata
|
||||
from basic_memory.mcp.server import mcp
|
||||
from fastmcp import Context
|
||||
@@ -17,8 +15,8 @@ TagType = Union[List[str], str, None]
|
||||
|
||||
|
||||
@mcp.tool(
|
||||
description="Create a markdown note. If the note already exists, returns an error by default — pass overwrite=True to replace.",
|
||||
annotations={"destructiveHint": True, "idempotentHint": False, "openWorldHint": False},
|
||||
description="Create or update a markdown note. Returns a markdown formatted summary of the semantic content.",
|
||||
annotations={"destructiveHint": False, "idempotentHint": True, "openWorldHint": False},
|
||||
)
|
||||
async def write_note(
|
||||
title: str,
|
||||
@@ -29,15 +27,12 @@ async def write_note(
|
||||
tags: list[str] | str | None = None,
|
||||
note_type: str = "note",
|
||||
metadata: dict | None = None,
|
||||
overwrite: bool | None = None,
|
||||
output_format: Literal["text", "json"] = "text",
|
||||
context: Context | None = None,
|
||||
) -> str | dict:
|
||||
"""Write a markdown note to the knowledge base.
|
||||
|
||||
Creates a markdown note with semantic observations and relations.
|
||||
If the note already exists, returns an error by default. Pass overwrite=True
|
||||
to replace the existing note. For incremental updates, use edit_note instead.
|
||||
Creates or updates a markdown note with semantic observations and relations.
|
||||
|
||||
Project Resolution:
|
||||
Server resolves projects using a unified priority chain (same in local and cloud modes):
|
||||
@@ -79,8 +74,6 @@ async def write_note(
|
||||
metadata: Optional dict of extra frontmatter fields merged into entity_metadata.
|
||||
Useful for schema notes or any note that needs custom YAML frontmatter
|
||||
beyond title/type/tags. Nested dicts are supported.
|
||||
overwrite: If True, replace existing note on conflict. If False, error on conflict.
|
||||
If None (default), consult write_note_overwrite_default config setting.
|
||||
output_format: "text" returns the existing markdown summary. "json" returns
|
||||
machine-readable metadata.
|
||||
context: Optional FastMCP context for performance caching.
|
||||
@@ -113,13 +106,12 @@ async def write_note(
|
||||
note_type="guide"
|
||||
)
|
||||
|
||||
# Overwrite an existing note explicitly
|
||||
# Update existing note (same title/directory)
|
||||
write_note(
|
||||
project="my-research",
|
||||
title="Meeting Notes",
|
||||
directory="meetings",
|
||||
content="# Weekly Standup\\n\\n- [decision] Use PostgreSQL instead #tech",
|
||||
overwrite=True
|
||||
content="# Weekly Standup\\n\\n- [decision] Use PostgreSQL instead #tech"
|
||||
)
|
||||
|
||||
# Create a schema note with custom frontmatter via metadata
|
||||
@@ -140,13 +132,6 @@ async def write_note(
|
||||
HTTPError: If project doesn't exist or is inaccessible
|
||||
SecurityError: If directory path attempts path traversal
|
||||
"""
|
||||
# Resolve overwrite flag: explicit parameter > config default
|
||||
# Trigger: caller omitted the parameter (None)
|
||||
# Why: lets users set a global default without breaking per-call overrides
|
||||
effective_overwrite = (
|
||||
overwrite if overwrite is not None else ConfigManager().config.write_note_overwrite_default
|
||||
)
|
||||
|
||||
async with get_project_client(project, workspace, context) as (client, active_project):
|
||||
logger.info(
|
||||
f"MCP tool call tool=write_note project={active_project.name} directory={directory}, title={title}, tags={tags}"
|
||||
@@ -214,23 +199,6 @@ async def write_note(
|
||||
or "conflict" in str(e).lower()
|
||||
or "already exists" in str(e).lower()
|
||||
):
|
||||
# Guard: block overwrite unless explicitly enabled
|
||||
if not effective_overwrite:
|
||||
logger.warning(
|
||||
f"write_note blocked: note already exists (overwrite not enabled) "
|
||||
f"permalink={entity.permalink}"
|
||||
)
|
||||
if output_format == "json":
|
||||
return {
|
||||
"title": title,
|
||||
"permalink": entity.permalink,
|
||||
"file_path": None,
|
||||
"checksum": None,
|
||||
"action": "conflict",
|
||||
"error": "NOTE_ALREADY_EXISTS",
|
||||
}
|
||||
return _format_overwrite_error(title, entity.permalink, active_project.name)
|
||||
|
||||
logger.debug(f"Entity exists, updating instead permalink={entity.permalink}")
|
||||
try:
|
||||
if not entity.permalink:
|
||||
@@ -302,23 +270,3 @@ async def write_note(
|
||||
|
||||
summary_result = "\n".join(summary)
|
||||
return add_project_metadata(summary_result, active_project.name)
|
||||
|
||||
|
||||
def _format_overwrite_error(title: str, permalink: str | None, project_name: str) -> str:
|
||||
"""Format a helpful error when write_note is blocked by the overwrite guard."""
|
||||
return textwrap.dedent(f"""\
|
||||
# Error: Note already exists
|
||||
|
||||
**"{title}"** already exists (permalink: `{permalink}`).
|
||||
|
||||
`write_note` does not overwrite by default. Choose an option:
|
||||
|
||||
| Goal | Action |
|
||||
|------|--------|
|
||||
| Append content | `edit_note("{permalink}", operation="append", content="...")` |
|
||||
| Prepend content | `edit_note("{permalink}", operation="prepend", content="...")` |
|
||||
| Replace a section | `edit_note("{permalink}", operation="replace_section", section="...", content="...")` |
|
||||
| Full replace | `write_note("{title}", ..., overwrite=True)` |
|
||||
| Inspect first | `read_note("{permalink}")` |
|
||||
|
||||
Project: {project_name}""")
|
||||
|
||||
@@ -93,27 +93,6 @@ CREATE VIRTUAL TABLE IF NOT EXISTS search_index USING fts5(
|
||||
);
|
||||
""")
|
||||
|
||||
# Postgres semantic chunk metadata table.
|
||||
# Matches the Alembic migration (h1b2c3d4e5f6) schema.
|
||||
# Used by tests to create the table without running full migrations.
|
||||
CREATE_POSTGRES_SEARCH_VECTOR_CHUNKS_TABLE = DDL("""
|
||||
CREATE TABLE IF NOT EXISTS search_vector_chunks (
|
||||
id BIGSERIAL PRIMARY KEY,
|
||||
entity_id INTEGER NOT NULL,
|
||||
project_id INTEGER NOT NULL,
|
||||
chunk_key TEXT NOT NULL,
|
||||
chunk_text TEXT NOT NULL,
|
||||
source_hash TEXT NOT NULL,
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||||
UNIQUE (project_id, entity_id, chunk_key)
|
||||
)
|
||||
""")
|
||||
|
||||
CREATE_POSTGRES_SEARCH_VECTOR_CHUNKS_INDEX = DDL("""
|
||||
CREATE INDEX IF NOT EXISTS idx_search_vector_chunks_project_entity
|
||||
ON search_vector_chunks (project_id, entity_id)
|
||||
""")
|
||||
|
||||
# Local semantic chunk metadata table for SQLite.
|
||||
# Embedding vectors live in sqlite-vec virtual table keyed by this table rowid.
|
||||
CREATE_SQLITE_SEARCH_VECTOR_CHUNKS = DDL("""
|
||||
|
||||
@@ -1,34 +1,8 @@
|
||||
"""Factory for creating configured semantic embedding providers."""
|
||||
|
||||
from threading import Lock
|
||||
|
||||
from basic_memory.config import BasicMemoryConfig
|
||||
from basic_memory.repository.embedding_provider import EmbeddingProvider
|
||||
|
||||
type ProviderCacheKey = tuple[str, str, int | None, int, str | None, int | None, int | None]
|
||||
|
||||
_EMBEDDING_PROVIDER_CACHE: dict[ProviderCacheKey, EmbeddingProvider] = {}
|
||||
_EMBEDDING_PROVIDER_CACHE_LOCK = Lock()
|
||||
|
||||
|
||||
def _provider_cache_key(app_config: BasicMemoryConfig) -> ProviderCacheKey:
|
||||
"""Build a stable cache key from provider-relevant semantic embedding config."""
|
||||
return (
|
||||
app_config.semantic_embedding_provider.strip().lower(),
|
||||
app_config.semantic_embedding_model,
|
||||
app_config.semantic_embedding_dimensions,
|
||||
app_config.semantic_embedding_batch_size,
|
||||
app_config.semantic_embedding_cache_dir,
|
||||
app_config.semantic_embedding_threads,
|
||||
app_config.semantic_embedding_parallel,
|
||||
)
|
||||
|
||||
|
||||
def reset_embedding_provider_cache() -> None:
|
||||
"""Clear process-level embedding provider cache (used by tests)."""
|
||||
with _EMBEDDING_PROVIDER_CACHE_LOCK:
|
||||
_EMBEDDING_PROVIDER_CACHE.clear()
|
||||
|
||||
|
||||
def create_embedding_provider(app_config: BasicMemoryConfig) -> EmbeddingProvider:
|
||||
"""Create an embedding provider based on semantic config.
|
||||
@@ -36,50 +10,32 @@ def create_embedding_provider(app_config: BasicMemoryConfig) -> EmbeddingProvide
|
||||
When semantic_embedding_dimensions is set in config, it overrides
|
||||
the provider's default dimensions (384 for FastEmbed, 1536 for OpenAI).
|
||||
"""
|
||||
cache_key = _provider_cache_key(app_config)
|
||||
with _EMBEDDING_PROVIDER_CACHE_LOCK:
|
||||
if cached_provider := _EMBEDDING_PROVIDER_CACHE.get(cache_key):
|
||||
return cached_provider
|
||||
|
||||
provider_name = app_config.semantic_embedding_provider.strip().lower()
|
||||
extra_kwargs: dict = {}
|
||||
if app_config.semantic_embedding_dimensions is not None:
|
||||
extra_kwargs["dimensions"] = app_config.semantic_embedding_dimensions
|
||||
|
||||
provider: EmbeddingProvider
|
||||
if provider_name == "fastembed":
|
||||
# Deferred import: fastembed (and its onnxruntime dep) may not be installed
|
||||
from basic_memory.repository.fastembed_provider import FastEmbedEmbeddingProvider
|
||||
|
||||
if app_config.semantic_embedding_cache_dir is not None:
|
||||
extra_kwargs["cache_dir"] = app_config.semantic_embedding_cache_dir
|
||||
if app_config.semantic_embedding_threads is not None:
|
||||
extra_kwargs["threads"] = app_config.semantic_embedding_threads
|
||||
if app_config.semantic_embedding_parallel is not None:
|
||||
extra_kwargs["parallel"] = app_config.semantic_embedding_parallel
|
||||
|
||||
provider = FastEmbedEmbeddingProvider(
|
||||
return FastEmbedEmbeddingProvider(
|
||||
model_name=app_config.semantic_embedding_model,
|
||||
batch_size=app_config.semantic_embedding_batch_size,
|
||||
**extra_kwargs,
|
||||
)
|
||||
elif provider_name == "openai":
|
||||
|
||||
if provider_name == "openai":
|
||||
# Deferred import: openai may not be installed
|
||||
from basic_memory.repository.openai_provider import OpenAIEmbeddingProvider
|
||||
|
||||
model_name = app_config.semantic_embedding_model or "text-embedding-3-small"
|
||||
if model_name == "bge-small-en-v1.5":
|
||||
model_name = "text-embedding-3-small"
|
||||
provider = OpenAIEmbeddingProvider(
|
||||
return OpenAIEmbeddingProvider(
|
||||
model_name=model_name,
|
||||
batch_size=app_config.semantic_embedding_batch_size,
|
||||
**extra_kwargs,
|
||||
)
|
||||
else:
|
||||
raise ValueError(f"Unsupported semantic embedding provider: {provider_name}")
|
||||
|
||||
with _EMBEDDING_PROVIDER_CACHE_LOCK:
|
||||
if cached_provider := _EMBEDDING_PROVIDER_CACHE.get(cache_key):
|
||||
return cached_provider
|
||||
_EMBEDDING_PROVIDER_CACHE[cache_key] = provider
|
||||
return provider
|
||||
raise ValueError(f"Unsupported semantic embedding provider: {provider_name}")
|
||||
|
||||
@@ -5,8 +5,6 @@ from __future__ import annotations
|
||||
import asyncio
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from loguru import logger
|
||||
|
||||
from basic_memory.repository.embedding_provider import EmbeddingProvider
|
||||
from basic_memory.repository.semantic_errors import SemanticDependenciesMissingError
|
||||
|
||||
@@ -21,25 +19,16 @@ class FastEmbedEmbeddingProvider(EmbeddingProvider):
|
||||
"bge-small-en-v1.5": "BAAI/bge-small-en-v1.5",
|
||||
}
|
||||
|
||||
def _effective_parallel(self) -> int | None:
|
||||
return self.parallel if self.parallel is not None and self.parallel > 1 else None
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
model_name: str = "bge-small-en-v1.5",
|
||||
*,
|
||||
batch_size: int = 64,
|
||||
dimensions: int = 384,
|
||||
cache_dir: str | None = None,
|
||||
threads: int | None = None,
|
||||
parallel: int | None = None,
|
||||
) -> None:
|
||||
self.model_name = model_name
|
||||
self.dimensions = dimensions
|
||||
self.batch_size = batch_size
|
||||
self.cache_dir = cache_dir
|
||||
self.threads = threads
|
||||
self.parallel = parallel
|
||||
self._model: TextEmbedding | None = None
|
||||
self._model_lock = asyncio.Lock()
|
||||
|
||||
@@ -63,29 +52,9 @@ class FastEmbedEmbeddingProvider(EmbeddingProvider):
|
||||
"pip install -U basic-memory"
|
||||
) from exc
|
||||
resolved_model_name = self._MODEL_ALIASES.get(self.model_name, self.model_name)
|
||||
if self.cache_dir is not None and self.threads is not None:
|
||||
return TextEmbedding(
|
||||
model_name=resolved_model_name,
|
||||
cache_dir=self.cache_dir,
|
||||
threads=self.threads,
|
||||
)
|
||||
if self.cache_dir is not None:
|
||||
return TextEmbedding(model_name=resolved_model_name, cache_dir=self.cache_dir)
|
||||
if self.threads is not None:
|
||||
return TextEmbedding(model_name=resolved_model_name, threads=self.threads)
|
||||
return TextEmbedding(model_name=resolved_model_name)
|
||||
|
||||
self._model = await asyncio.to_thread(_create_model)
|
||||
logger.info(
|
||||
"FastEmbed model loaded: model_name={model_name} batch_size={batch_size} "
|
||||
"threads={threads} configured_parallel={configured_parallel} "
|
||||
"effective_parallel={effective_parallel}",
|
||||
model_name=self._MODEL_ALIASES.get(self.model_name, self.model_name),
|
||||
batch_size=self.batch_size,
|
||||
threads=self.threads,
|
||||
configured_parallel=self.parallel,
|
||||
effective_parallel=self._effective_parallel(),
|
||||
)
|
||||
return self._model
|
||||
|
||||
async def embed_documents(self, texts: list[str]) -> list[list[float]]:
|
||||
@@ -93,23 +62,9 @@ class FastEmbedEmbeddingProvider(EmbeddingProvider):
|
||||
return []
|
||||
|
||||
model = await self._load_model()
|
||||
effective_parallel = self._effective_parallel()
|
||||
logger.debug(
|
||||
"FastEmbed embed_documents call: text_count={text_count} batch_size={batch_size} "
|
||||
"threads={threads} configured_parallel={configured_parallel} "
|
||||
"effective_parallel={effective_parallel}",
|
||||
text_count=len(texts),
|
||||
batch_size=self.batch_size,
|
||||
threads=self.threads,
|
||||
configured_parallel=self.parallel,
|
||||
effective_parallel=effective_parallel,
|
||||
)
|
||||
|
||||
def _embed_batch() -> list[list[float]]:
|
||||
embed_kwargs: dict[str, int] = {"batch_size": self.batch_size}
|
||||
if effective_parallel is not None:
|
||||
embed_kwargs["parallel"] = effective_parallel
|
||||
vectors = list(model.embed(texts, **embed_kwargs))
|
||||
vectors = list(model.embed(texts, batch_size=self.batch_size))
|
||||
normalized: list[list[float]] = []
|
||||
for vector in vectors:
|
||||
values = vector.tolist() if hasattr(vector, "tolist") else vector
|
||||
|
||||
@@ -2,10 +2,9 @@
|
||||
|
||||
from typing import Dict, List, Sequence
|
||||
|
||||
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.ext.asyncio import async_sessionmaker
|
||||
from sqlalchemy.orm import selectinload
|
||||
from sqlalchemy.orm.interfaces import LoaderOption
|
||||
|
||||
from basic_memory.models import Observation
|
||||
from basic_memory.repository.repository import Repository
|
||||
@@ -23,10 +22,6 @@ class ObservationRepository(Repository[Observation]):
|
||||
"""
|
||||
super().__init__(session_maker, Observation, project_id=project_id)
|
||||
|
||||
def get_load_options(self) -> List[LoaderOption]:
|
||||
"""Eager-load parent entity to prevent N+1 if obs.entity is accessed."""
|
||||
return [selectinload(Observation.entity)]
|
||||
|
||||
async def find_by_entity(self, entity_id: int) -> Sequence[Observation]:
|
||||
"""Find all observations for a specific entity."""
|
||||
query = select(Observation).filter(Observation.entity_id == entity_id)
|
||||
|
||||
@@ -58,9 +58,6 @@ class PostgresSearchRepository(SearchRepositoryBase):
|
||||
self._semantic_enabled = self._app_config.semantic_search_enabled
|
||||
self._semantic_vector_k = self._app_config.semantic_vector_k
|
||||
self._semantic_min_similarity = self._app_config.semantic_min_similarity
|
||||
self._semantic_embedding_sync_batch_size = (
|
||||
self._app_config.semantic_embedding_sync_batch_size
|
||||
)
|
||||
self._embedding_provider = embedding_provider
|
||||
self._vector_dimensions = 384
|
||||
self._vector_tables_initialized = False
|
||||
@@ -270,7 +267,7 @@ class PostgresSearchRepository(SearchRepositoryBase):
|
||||
if self._vector_tables_initialized:
|
||||
return
|
||||
|
||||
logger.debug("Ensuring Postgres vector tables exist for semantic search")
|
||||
logger.info("Ensuring Postgres vector tables exist for semantic search")
|
||||
|
||||
async with self._vector_tables_lock:
|
||||
if self._vector_tables_initialized:
|
||||
@@ -361,7 +358,7 @@ class PostgresSearchRepository(SearchRepositoryBase):
|
||||
)
|
||||
await session.commit()
|
||||
|
||||
logger.debug(f"Postgres vector tables ready (dimensions={self._vector_dimensions})")
|
||||
logger.info(f"Postgres vector tables ready (dimensions={self._vector_dimensions})")
|
||||
self._vector_tables_initialized = True
|
||||
|
||||
async def _get_existing_embedding_dims(self, session: AsyncSession) -> int | None:
|
||||
@@ -692,7 +689,9 @@ class PostgresSearchRepository(SearchRepositoryBase):
|
||||
for idx, note_type in enumerate(note_types):
|
||||
param_name = f"note_type_{idx}"
|
||||
params[param_name] = json.dumps({"note_type": note_type})
|
||||
type_conditions.append(f"search_index.metadata @> CAST(:{param_name} AS jsonb)")
|
||||
type_conditions.append(
|
||||
f"search_index.metadata @> CAST(:{param_name} AS jsonb)"
|
||||
)
|
||||
conditions.append(f"({' OR '.join(type_conditions)})")
|
||||
|
||||
# Handle date filter
|
||||
|
||||
@@ -41,7 +41,7 @@ class SearchIndexRow:
|
||||
# Matched chunk text from vector search (the actual content that matched the query)
|
||||
matched_chunk_text: Optional[str] = None
|
||||
|
||||
CONTENT_DISPLAY_LIMIT = 4000
|
||||
CONTENT_DISPLAY_LIMIT = 250
|
||||
|
||||
@property
|
||||
def content(self):
|
||||
|
||||
@@ -7,7 +7,7 @@ The actual repository implementations are backend-specific:
|
||||
"""
|
||||
|
||||
from datetime import datetime
|
||||
from typing import Any, Callable, List, Optional, Protocol
|
||||
from typing import List, Optional, Protocol
|
||||
|
||||
from sqlalchemy import Result
|
||||
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
|
||||
@@ -15,7 +15,6 @@ from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
|
||||
from basic_memory.config import BasicMemoryConfig, ConfigManager, DatabaseBackend
|
||||
from basic_memory.repository.postgres_search_repository import PostgresSearchRepository
|
||||
from basic_memory.repository.search_index_row import SearchIndexRow
|
||||
from basic_memory.repository.search_repository_base import VectorSyncBatchResult
|
||||
from basic_memory.repository.sqlite_search_repository import SQLiteSearchRepository
|
||||
from basic_memory.schemas.search import SearchItemType, SearchRetrievalMode
|
||||
|
||||
@@ -70,14 +69,6 @@ class SearchRepository(Protocol):
|
||||
"""Sync semantic vector chunks for an entity."""
|
||||
...
|
||||
|
||||
async def sync_entity_vectors_batch(
|
||||
self,
|
||||
entity_ids: list[int],
|
||||
progress_callback: Optional[Callable[[int, int, int], Any]] = None,
|
||||
) -> VectorSyncBatchResult:
|
||||
"""Sync semantic vector chunks for a batch of entities."""
|
||||
...
|
||||
|
||||
async def execute_query(self, query, params: dict) -> Result:
|
||||
"""Execute a raw SQL query."""
|
||||
...
|
||||
|
||||
@@ -5,9 +5,9 @@ import json
|
||||
import re
|
||||
import time
|
||||
from abc import ABC, abstractmethod
|
||||
from dataclasses import dataclass, field, replace
|
||||
from dataclasses import replace
|
||||
from datetime import datetime
|
||||
from typing import Any, Callable, Dict, List, Optional
|
||||
from typing import Any, Dict, List, Optional
|
||||
|
||||
from loguru import logger
|
||||
from sqlalchemy import Executable, Result, text
|
||||
@@ -25,67 +25,20 @@ from basic_memory.schemas.search import SearchItemType, SearchRetrievalMode
|
||||
# --- Semantic search constants ---
|
||||
|
||||
VECTOR_FILTER_SCAN_LIMIT = 50000
|
||||
FUSION_BONUS = 0.3
|
||||
FTS_GATE_THRESHOLD = 0.0
|
||||
RRF_K = 60
|
||||
MAX_VECTOR_CHUNK_CHARS = 900
|
||||
VECTOR_CHUNK_OVERLAP_CHARS = 120
|
||||
TOP_CHUNKS_PER_RESULT = 5
|
||||
SMALL_NOTE_CONTENT_LIMIT = 2000
|
||||
HEADER_LINE_PATTERN = re.compile(r"^\s*#{1,6}\s+")
|
||||
BULLET_PATTERN = re.compile(r"^[\-\*]\s+")
|
||||
|
||||
|
||||
@dataclass
|
||||
class VectorSyncBatchResult:
|
||||
"""Aggregate result for batched semantic vector sync runs."""
|
||||
|
||||
entities_total: int
|
||||
entities_synced: int
|
||||
entities_failed: int
|
||||
failed_entity_ids: list[int] = field(default_factory=list)
|
||||
embedding_jobs_total: int = 0
|
||||
embed_seconds_total: float = 0.0
|
||||
write_seconds_total: float = 0.0
|
||||
|
||||
|
||||
@dataclass
|
||||
class _PreparedEntityVectorSync:
|
||||
"""Prepared chunk mutations + embedding jobs for one entity."""
|
||||
|
||||
entity_id: int
|
||||
sync_start: float
|
||||
source_rows_count: int
|
||||
embedding_jobs: list[tuple[int, str]]
|
||||
|
||||
|
||||
@dataclass
|
||||
class _PendingEmbeddingJob:
|
||||
"""Pending embedding write entry with entity ownership metadata."""
|
||||
|
||||
entity_id: int
|
||||
chunk_row_id: int
|
||||
chunk_text: str
|
||||
|
||||
|
||||
@dataclass
|
||||
class _EntitySyncRuntime:
|
||||
"""Per-entity runtime counters used while flushes are in flight."""
|
||||
|
||||
sync_start: float
|
||||
source_rows_count: int
|
||||
embedding_jobs_count: int
|
||||
remaining_jobs: int
|
||||
embed_seconds: float = 0.0
|
||||
write_seconds: float = 0.0
|
||||
|
||||
|
||||
class SearchRepositoryBase(ABC):
|
||||
"""Abstract base class for backend-specific search repository implementations.
|
||||
|
||||
This class defines the common interface that all search repositories must implement,
|
||||
regardless of whether they use SQLite FTS5 or Postgres tsvector for full-text search.
|
||||
|
||||
Shared semantic search logic (chunking, embedding orchestration, hybrid score-based fusion)
|
||||
Shared semantic search logic (chunking, embedding orchestration, hybrid RRF fusion)
|
||||
lives here. Backend-specific operations are delegated to abstract hooks.
|
||||
|
||||
Concrete implementations:
|
||||
@@ -98,7 +51,6 @@ class SearchRepositoryBase(ABC):
|
||||
_semantic_vector_k: int
|
||||
_semantic_min_similarity: float
|
||||
_embedding_provider: Optional[EmbeddingProvider]
|
||||
_semantic_embedding_sync_batch_size: int
|
||||
_vector_dimensions: int
|
||||
_vector_tables_initialized: bool
|
||||
|
||||
@@ -608,205 +560,15 @@ class SearchRepositoryBase(ABC):
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
async def sync_entity_vectors(self, entity_id: int) -> None:
|
||||
"""Sync semantic chunk rows + embeddings for a single entity."""
|
||||
await self._sync_entity_vectors_internal(
|
||||
[entity_id],
|
||||
progress_callback=None,
|
||||
continue_on_error=False,
|
||||
)
|
||||
"""Sync semantic chunk rows + embeddings for a single entity.
|
||||
|
||||
async def sync_entity_vectors_batch(
|
||||
self,
|
||||
entity_ids: list[int],
|
||||
progress_callback: Optional[Callable[[int, int, int], Any]] = None,
|
||||
) -> VectorSyncBatchResult:
|
||||
"""Sync semantic chunk rows + embeddings for a batch of entities."""
|
||||
return await self._sync_entity_vectors_internal(
|
||||
entity_ids,
|
||||
progress_callback=progress_callback,
|
||||
continue_on_error=True,
|
||||
)
|
||||
|
||||
async def _sync_entity_vectors_internal(
|
||||
self,
|
||||
entity_ids: list[int],
|
||||
progress_callback: Optional[Callable[[int, int, int], Any]],
|
||||
continue_on_error: bool,
|
||||
) -> VectorSyncBatchResult:
|
||||
"""Run shared vector sync orchestration for one or many entities."""
|
||||
This is the shared orchestration logic. Backend-specific SQL operations
|
||||
are delegated to abstract hooks (_delete_entity_chunks, _write_embeddings, etc.).
|
||||
"""
|
||||
self._assert_semantic_available()
|
||||
await self._ensure_vector_tables()
|
||||
assert self._embedding_provider is not None
|
||||
|
||||
total_entities = len(entity_ids)
|
||||
result = VectorSyncBatchResult(
|
||||
entities_total=total_entities,
|
||||
entities_synced=0,
|
||||
entities_failed=0,
|
||||
)
|
||||
if total_entities == 0:
|
||||
return result
|
||||
|
||||
logger.info(
|
||||
"Vector batch sync start: project_id={project_id} entities_total={entities_total} "
|
||||
"sync_batch_size={sync_batch_size}",
|
||||
project_id=self.project_id,
|
||||
entities_total=total_entities,
|
||||
sync_batch_size=self._semantic_embedding_sync_batch_size,
|
||||
)
|
||||
|
||||
pending_jobs: list[_PendingEmbeddingJob] = []
|
||||
entity_runtime: dict[int, _EntitySyncRuntime] = {}
|
||||
failed_entity_ids: set[int] = set()
|
||||
synced_entity_ids: set[int] = set()
|
||||
|
||||
for index, entity_id in enumerate(entity_ids):
|
||||
if progress_callback is not None:
|
||||
progress_callback(entity_id, index, total_entities)
|
||||
|
||||
try:
|
||||
prepared = await self._prepare_entity_vector_jobs(entity_id)
|
||||
except Exception as exc:
|
||||
if not continue_on_error:
|
||||
raise
|
||||
failed_entity_ids.add(entity_id)
|
||||
logger.warning(
|
||||
"Vector batch sync entity prepare failed: project_id={project_id} "
|
||||
"entity_id={entity_id} error={error}",
|
||||
project_id=self.project_id,
|
||||
entity_id=entity_id,
|
||||
error=str(exc),
|
||||
)
|
||||
continue
|
||||
|
||||
embedding_jobs_count = len(prepared.embedding_jobs)
|
||||
result.embedding_jobs_total += embedding_jobs_count
|
||||
|
||||
if embedding_jobs_count == 0:
|
||||
synced_entity_ids.add(entity_id)
|
||||
total_seconds = time.perf_counter() - prepared.sync_start
|
||||
self._log_vector_sync_complete(
|
||||
entity_id=entity_id,
|
||||
total_seconds=total_seconds,
|
||||
embed_seconds=0.0,
|
||||
write_seconds=0.0,
|
||||
source_rows_count=prepared.source_rows_count,
|
||||
embedding_jobs_count=0,
|
||||
)
|
||||
continue
|
||||
|
||||
entity_runtime[entity_id] = _EntitySyncRuntime(
|
||||
sync_start=prepared.sync_start,
|
||||
source_rows_count=prepared.source_rows_count,
|
||||
embedding_jobs_count=embedding_jobs_count,
|
||||
remaining_jobs=embedding_jobs_count,
|
||||
)
|
||||
pending_jobs.extend(
|
||||
_PendingEmbeddingJob(
|
||||
entity_id=entity_id, chunk_row_id=row_id, chunk_text=chunk_text
|
||||
)
|
||||
for row_id, chunk_text in prepared.embedding_jobs
|
||||
)
|
||||
|
||||
while len(pending_jobs) >= self._semantic_embedding_sync_batch_size:
|
||||
flush_jobs = pending_jobs[: self._semantic_embedding_sync_batch_size]
|
||||
pending_jobs = pending_jobs[self._semantic_embedding_sync_batch_size :]
|
||||
try:
|
||||
embed_seconds, write_seconds = await self._flush_embedding_jobs(
|
||||
flush_jobs=flush_jobs,
|
||||
entity_runtime=entity_runtime,
|
||||
synced_entity_ids=synced_entity_ids,
|
||||
)
|
||||
result.embed_seconds_total += embed_seconds
|
||||
result.write_seconds_total += write_seconds
|
||||
except Exception as exc:
|
||||
if not continue_on_error:
|
||||
raise
|
||||
affected_entity_ids = sorted({job.entity_id for job in flush_jobs})
|
||||
failed_entity_ids.update(affected_entity_ids)
|
||||
for failed_entity_id in affected_entity_ids:
|
||||
entity_runtime.pop(failed_entity_id, None)
|
||||
logger.warning(
|
||||
"Vector batch sync flush failed: project_id={project_id} "
|
||||
"affected_entities={affected_entities} chunk_count={chunk_count} error={error}",
|
||||
project_id=self.project_id,
|
||||
affected_entities=affected_entity_ids,
|
||||
chunk_count=len(flush_jobs),
|
||||
error=str(exc),
|
||||
)
|
||||
|
||||
if pending_jobs:
|
||||
flush_jobs = list(pending_jobs)
|
||||
pending_jobs = []
|
||||
try:
|
||||
embed_seconds, write_seconds = await self._flush_embedding_jobs(
|
||||
flush_jobs=flush_jobs,
|
||||
entity_runtime=entity_runtime,
|
||||
synced_entity_ids=synced_entity_ids,
|
||||
)
|
||||
result.embed_seconds_total += embed_seconds
|
||||
result.write_seconds_total += write_seconds
|
||||
except Exception as exc:
|
||||
if not continue_on_error:
|
||||
raise
|
||||
affected_entity_ids = sorted({job.entity_id for job in flush_jobs})
|
||||
failed_entity_ids.update(affected_entity_ids)
|
||||
for failed_entity_id in affected_entity_ids:
|
||||
entity_runtime.pop(failed_entity_id, None)
|
||||
logger.warning(
|
||||
"Vector batch sync final flush failed: project_id={project_id} "
|
||||
"affected_entities={affected_entities} chunk_count={chunk_count} error={error}",
|
||||
project_id=self.project_id,
|
||||
affected_entities=affected_entity_ids,
|
||||
chunk_count=len(flush_jobs),
|
||||
error=str(exc),
|
||||
)
|
||||
|
||||
# Trigger: this should never happen after all flushes succeed.
|
||||
# Why: remaining jobs mean runtime tracking drifted from queued jobs.
|
||||
# Outcome: fail-safe marks these entities as failed to avoid false positives.
|
||||
if entity_runtime:
|
||||
orphan_runtime_entities = sorted(entity_runtime.keys())
|
||||
failed_entity_ids.update(orphan_runtime_entities)
|
||||
logger.warning(
|
||||
"Vector batch sync left unfinished entities after flushes: "
|
||||
"project_id={project_id} unfinished_entities={unfinished_entities}",
|
||||
project_id=self.project_id,
|
||||
unfinished_entities=orphan_runtime_entities,
|
||||
)
|
||||
|
||||
# Keep result counters aligned with successful/failed terminal states.
|
||||
synced_entity_ids.difference_update(failed_entity_ids)
|
||||
result.failed_entity_ids = sorted(failed_entity_ids)
|
||||
result.entities_failed = len(result.failed_entity_ids)
|
||||
result.entities_synced = len(synced_entity_ids)
|
||||
|
||||
logger.info(
|
||||
"Vector batch sync complete: project_id={project_id} entities_total={entities_total} "
|
||||
"entities_synced={entities_synced} entities_failed={entities_failed} "
|
||||
"embedding_jobs_total={embedding_jobs_total} embed_seconds_total={embed_seconds_total:.3f} "
|
||||
"write_seconds_total={write_seconds_total:.3f}",
|
||||
project_id=self.project_id,
|
||||
entities_total=result.entities_total,
|
||||
entities_synced=result.entities_synced,
|
||||
entities_failed=result.entities_failed,
|
||||
embedding_jobs_total=result.embedding_jobs_total,
|
||||
embed_seconds_total=result.embed_seconds_total,
|
||||
write_seconds_total=result.write_seconds_total,
|
||||
)
|
||||
|
||||
return result
|
||||
|
||||
async def _prepare_entity_vector_jobs(self, entity_id: int) -> _PreparedEntityVectorSync:
|
||||
"""Prepare chunk mutations and embedding jobs for one entity."""
|
||||
sync_start = time.perf_counter()
|
||||
|
||||
logger.info(
|
||||
"Vector sync start: project_id={project_id} entity_id={entity_id}",
|
||||
project_id=self.project_id,
|
||||
entity_id=entity_id,
|
||||
)
|
||||
|
||||
async with db.scoped_session(self.session_maker) as session:
|
||||
await self._prepare_vector_session(session)
|
||||
|
||||
@@ -832,49 +594,18 @@ class SearchRepositoryBase(ABC):
|
||||
},
|
||||
)
|
||||
rows = row_result.fetchall()
|
||||
source_rows_count = len(rows)
|
||||
built_chunk_records_count = 0
|
||||
|
||||
# No search_index rows → delete all chunk/embedding data for this entity.
|
||||
if not rows:
|
||||
logger.info(
|
||||
"Vector sync source prepared: project_id={project_id} entity_id={entity_id} "
|
||||
"source_rows_count={source_rows_count} "
|
||||
"built_chunk_records_count={built_chunk_records_count}",
|
||||
project_id=self.project_id,
|
||||
entity_id=entity_id,
|
||||
source_rows_count=source_rows_count,
|
||||
built_chunk_records_count=built_chunk_records_count,
|
||||
)
|
||||
await self._delete_entity_chunks(session, entity_id)
|
||||
await session.commit()
|
||||
return _PreparedEntityVectorSync(
|
||||
entity_id=entity_id,
|
||||
sync_start=sync_start,
|
||||
source_rows_count=source_rows_count,
|
||||
embedding_jobs=[],
|
||||
)
|
||||
return
|
||||
|
||||
chunk_records = self._build_chunk_records(rows)
|
||||
built_chunk_records_count = len(chunk_records)
|
||||
logger.info(
|
||||
"Vector sync source prepared: project_id={project_id} entity_id={entity_id} "
|
||||
"source_rows_count={source_rows_count} "
|
||||
"built_chunk_records_count={built_chunk_records_count}",
|
||||
project_id=self.project_id,
|
||||
entity_id=entity_id,
|
||||
source_rows_count=source_rows_count,
|
||||
built_chunk_records_count=built_chunk_records_count,
|
||||
)
|
||||
if not chunk_records:
|
||||
await self._delete_entity_chunks(session, entity_id)
|
||||
await session.commit()
|
||||
return _PreparedEntityVectorSync(
|
||||
entity_id=entity_id,
|
||||
sync_start=sync_start,
|
||||
source_rows_count=source_rows_count,
|
||||
embedding_jobs=[],
|
||||
)
|
||||
return
|
||||
|
||||
# --- Diff existing chunks against incoming ---
|
||||
existing_rows_result = await session.execute(
|
||||
@@ -886,7 +617,6 @@ class SearchRepositoryBase(ABC):
|
||||
{"project_id": self.project_id, "entity_id": entity_id},
|
||||
)
|
||||
existing_by_key = {row.chunk_key: row for row in existing_rows_result.fetchall()}
|
||||
existing_chunks_count = len(existing_by_key)
|
||||
incoming_hashes = {
|
||||
record["chunk_key"]: record["source_hash"] for record in chunk_records
|
||||
}
|
||||
@@ -895,7 +625,6 @@ class SearchRepositoryBase(ABC):
|
||||
for chunk_key, row in existing_by_key.items()
|
||||
if chunk_key not in incoming_hashes
|
||||
]
|
||||
stale_chunks_count = len(stale_ids)
|
||||
|
||||
if stale_ids:
|
||||
await self._delete_stale_chunks(session, stale_ids, entity_id)
|
||||
@@ -909,8 +638,6 @@ class SearchRepositoryBase(ABC):
|
||||
{"project_id": self.project_id, "entity_id": entity_id},
|
||||
)
|
||||
orphan_rows = orphan_result.fetchall()
|
||||
orphan_ids = {int(row.id) for row in orphan_rows}
|
||||
orphan_chunks_count = len(orphan_ids)
|
||||
|
||||
# --- Upsert changed / new chunks, collect embedding jobs ---
|
||||
timestamp_expr = self._timestamp_now_expr()
|
||||
@@ -921,7 +648,7 @@ class SearchRepositoryBase(ABC):
|
||||
# Trigger: chunk exists and hash matches (no content change)
|
||||
# but chunk has no embedding (orphan from crash).
|
||||
# Outcome: schedule re-embedding without touching chunk metadata.
|
||||
is_orphan = current and int(current.id) in orphan_ids
|
||||
is_orphan = current and any(o.id == current.id for o in orphan_rows)
|
||||
if current and current.source_hash == record["source_hash"] and not is_orphan:
|
||||
continue
|
||||
|
||||
@@ -964,141 +691,20 @@ class SearchRepositoryBase(ABC):
|
||||
row_id = int(inserted.scalar_one())
|
||||
embedding_jobs.append((row_id, record["chunk_text"]))
|
||||
|
||||
logger.info(
|
||||
"Vector sync diff complete: project_id={project_id} entity_id={entity_id} "
|
||||
"existing_chunks_count={existing_chunks_count} "
|
||||
"stale_chunks_count={stale_chunks_count} "
|
||||
"orphan_chunks_count={orphan_chunks_count} "
|
||||
"embedding_jobs_count={embedding_jobs_count}",
|
||||
project_id=self.project_id,
|
||||
entity_id=entity_id,
|
||||
existing_chunks_count=existing_chunks_count,
|
||||
stale_chunks_count=stale_chunks_count,
|
||||
orphan_chunks_count=orphan_chunks_count,
|
||||
embedding_jobs_count=len(embedding_jobs),
|
||||
)
|
||||
await session.commit()
|
||||
|
||||
return _PreparedEntityVectorSync(
|
||||
entity_id=entity_id,
|
||||
sync_start=sync_start,
|
||||
source_rows_count=source_rows_count,
|
||||
embedding_jobs=embedding_jobs,
|
||||
)
|
||||
if not embedding_jobs:
|
||||
return
|
||||
|
||||
async def _flush_embedding_jobs(
|
||||
self,
|
||||
flush_jobs: list[_PendingEmbeddingJob],
|
||||
entity_runtime: dict[int, _EntitySyncRuntime],
|
||||
synced_entity_ids: set[int],
|
||||
) -> tuple[float, float]:
|
||||
"""Embed and persist one queued flush chunk."""
|
||||
if not flush_jobs:
|
||||
return 0.0, 0.0
|
||||
assert self._embedding_provider is not None
|
||||
|
||||
embed_start = time.perf_counter()
|
||||
texts = [job.chunk_text for job in flush_jobs]
|
||||
texts = [t for _, t in embedding_jobs]
|
||||
embeddings = await self._embedding_provider.embed_documents(texts)
|
||||
embed_seconds = time.perf_counter() - embed_start
|
||||
embed_rate = (len(flush_jobs) / embed_seconds) if embed_seconds > 0 else 0.0
|
||||
logger.info(
|
||||
"Vector batch embed flush: project_id={project_id} chunk_count={chunk_count} "
|
||||
"embed_seconds={embed_seconds:.3f} embed_rate_chunks_per_second={embed_rate:.2f}",
|
||||
project_id=self.project_id,
|
||||
chunk_count=len(flush_jobs),
|
||||
embed_seconds=embed_seconds,
|
||||
embed_rate=embed_rate,
|
||||
)
|
||||
if len(embeddings) != len(flush_jobs):
|
||||
if len(embeddings) != len(embedding_jobs):
|
||||
raise RuntimeError("Embedding provider returned an unexpected number of vectors.")
|
||||
|
||||
write_start = time.perf_counter()
|
||||
async with db.scoped_session(self.session_maker) as session:
|
||||
await self._prepare_vector_session(session)
|
||||
write_jobs = [(job.chunk_row_id, job.chunk_text) for job in flush_jobs]
|
||||
await self._write_embeddings(session, write_jobs, embeddings)
|
||||
await self._write_embeddings(session, embedding_jobs, embeddings)
|
||||
await session.commit()
|
||||
write_seconds = time.perf_counter() - write_start
|
||||
write_rate = (len(flush_jobs) / write_seconds) if write_seconds > 0 else 0.0
|
||||
logger.info(
|
||||
"Vector batch write flush: project_id={project_id} row_count={row_count} "
|
||||
"write_seconds={write_seconds:.3f} write_rate_rows_per_second={write_rate:.2f}",
|
||||
project_id=self.project_id,
|
||||
row_count=len(flush_jobs),
|
||||
write_seconds=write_seconds,
|
||||
write_rate=write_rate,
|
||||
)
|
||||
|
||||
flush_size = len(flush_jobs)
|
||||
entity_job_counts: dict[int, int] = {}
|
||||
for job in flush_jobs:
|
||||
entity_job_counts[job.entity_id] = entity_job_counts.get(job.entity_id, 0) + 1
|
||||
|
||||
for entity_id, entity_job_count in entity_job_counts.items():
|
||||
runtime = entity_runtime.get(entity_id)
|
||||
if runtime is None:
|
||||
continue
|
||||
runtime.remaining_jobs -= entity_job_count
|
||||
|
||||
# Attribute flush wall-clock to entities in proportion to rows written.
|
||||
flush_share = entity_job_count / flush_size
|
||||
runtime.embed_seconds += embed_seconds * flush_share
|
||||
runtime.write_seconds += write_seconds * flush_share
|
||||
|
||||
if runtime.remaining_jobs <= 0:
|
||||
synced_entity_ids.add(entity_id)
|
||||
total_seconds = time.perf_counter() - runtime.sync_start
|
||||
self._log_vector_sync_complete(
|
||||
entity_id=entity_id,
|
||||
total_seconds=total_seconds,
|
||||
embed_seconds=runtime.embed_seconds,
|
||||
write_seconds=runtime.write_seconds,
|
||||
source_rows_count=runtime.source_rows_count,
|
||||
embedding_jobs_count=runtime.embedding_jobs_count,
|
||||
)
|
||||
entity_runtime.pop(entity_id, None)
|
||||
|
||||
return embed_seconds, write_seconds
|
||||
|
||||
def _log_vector_sync_complete(
|
||||
self,
|
||||
*,
|
||||
entity_id: int,
|
||||
total_seconds: float,
|
||||
embed_seconds: float,
|
||||
write_seconds: float,
|
||||
source_rows_count: int,
|
||||
embedding_jobs_count: int,
|
||||
) -> None:
|
||||
"""Log completion and slow-entity warnings with a consistent format."""
|
||||
logger.info(
|
||||
"Vector sync complete: project_id={project_id} entity_id={entity_id} "
|
||||
"total_seconds={total_seconds:.3f} embed_seconds={embed_seconds:.3f} "
|
||||
"write_seconds={write_seconds:.3f} source_rows_count={source_rows_count} "
|
||||
"embedding_jobs_count={embedding_jobs_count}",
|
||||
project_id=self.project_id,
|
||||
entity_id=entity_id,
|
||||
total_seconds=total_seconds,
|
||||
embed_seconds=embed_seconds,
|
||||
write_seconds=write_seconds,
|
||||
source_rows_count=source_rows_count,
|
||||
embedding_jobs_count=embedding_jobs_count,
|
||||
)
|
||||
if total_seconds > 10:
|
||||
logger.warning(
|
||||
"Vector sync slow entity: project_id={project_id} entity_id={entity_id} "
|
||||
"total_seconds={total_seconds:.3f} embed_seconds={embed_seconds:.3f} "
|
||||
"write_seconds={write_seconds:.3f} source_rows_count={source_rows_count} "
|
||||
"embedding_jobs_count={embedding_jobs_count}",
|
||||
project_id=self.project_id,
|
||||
entity_id=entity_id,
|
||||
total_seconds=total_seconds,
|
||||
embed_seconds=embed_seconds,
|
||||
write_seconds=write_seconds,
|
||||
source_rows_count=source_rows_count,
|
||||
embedding_jobs_count=embedding_jobs_count,
|
||||
)
|
||||
|
||||
async def _prepare_vector_session(self, session: AsyncSession) -> None:
|
||||
"""Hook for per-session setup (e.g. loading sqlite-vec extension).
|
||||
@@ -1243,7 +849,6 @@ class SearchRepositoryBase(ABC):
|
||||
min_similarity: Optional[float] = None,
|
||||
limit: int,
|
||||
offset: int,
|
||||
_emit_observability_log: bool = True,
|
||||
) -> List[SearchIndexRow]:
|
||||
"""Run vector-only search returning chunk-level results.
|
||||
|
||||
@@ -1254,70 +859,21 @@ class SearchRepositoryBase(ABC):
|
||||
self._assert_semantic_available()
|
||||
await self._ensure_vector_tables()
|
||||
assert self._embedding_provider is not None
|
||||
query_text = search_text.strip()
|
||||
query_embedding = await self._embedding_provider.embed_query(search_text.strip())
|
||||
candidate_limit = max(self._semantic_vector_k, (limit + offset) * 10)
|
||||
query_start = time.perf_counter()
|
||||
embed_start = time.perf_counter()
|
||||
query_embedding = await self._embedding_provider.embed_query(query_text)
|
||||
embed_ms = (time.perf_counter() - embed_start) * 1000
|
||||
vector_query_start = time.perf_counter()
|
||||
|
||||
async with db.scoped_session(self.session_maker) as session:
|
||||
await self._prepare_vector_session(session)
|
||||
vector_rows = await self._run_vector_query(session, query_embedding, candidate_limit)
|
||||
vector_query_ms = (time.perf_counter() - vector_query_start) * 1000
|
||||
vector_row_count = len(vector_rows)
|
||||
hydrate_ms = 0.0
|
||||
|
||||
def _log_vector_summary() -> None:
|
||||
if not _emit_observability_log:
|
||||
return
|
||||
|
||||
total_ms = (time.perf_counter() - query_start) * 1000
|
||||
logger.info(
|
||||
"Semantic query timing: project_id={project_id} retrieval_mode={retrieval_mode} "
|
||||
"query_length={query_length} candidate_limit={candidate_limit} "
|
||||
"vector_row_count={vector_row_count} embed_ms={embed_ms:.2f} "
|
||||
"vector_query_ms={vector_query_ms:.2f} hydrate_ms={hydrate_ms:.2f} "
|
||||
"total_ms={total_ms:.2f}",
|
||||
project_id=self.project_id,
|
||||
retrieval_mode="vector",
|
||||
query_length=len(query_text),
|
||||
candidate_limit=candidate_limit,
|
||||
vector_row_count=vector_row_count,
|
||||
embed_ms=embed_ms,
|
||||
vector_query_ms=vector_query_ms,
|
||||
hydrate_ms=hydrate_ms,
|
||||
total_ms=total_ms,
|
||||
)
|
||||
if total_ms > 2000:
|
||||
logger.warning(
|
||||
"[SEMANTIC_SLOW_QUERY] Semantic query timing: project_id={project_id} "
|
||||
"retrieval_mode={retrieval_mode} query_length={query_length} "
|
||||
"candidate_limit={candidate_limit} vector_row_count={vector_row_count} "
|
||||
"embed_ms={embed_ms:.2f} vector_query_ms={vector_query_ms:.2f} "
|
||||
"hydrate_ms={hydrate_ms:.2f} total_ms={total_ms:.2f}",
|
||||
project_id=self.project_id,
|
||||
retrieval_mode="vector",
|
||||
query_length=len(query_text),
|
||||
candidate_limit=candidate_limit,
|
||||
vector_row_count=vector_row_count,
|
||||
embed_ms=embed_ms,
|
||||
vector_query_ms=vector_query_ms,
|
||||
hydrate_ms=hydrate_ms,
|
||||
total_ms=total_ms,
|
||||
)
|
||||
|
||||
if not vector_rows:
|
||||
_log_vector_summary()
|
||||
return []
|
||||
|
||||
hydrate_start = time.perf_counter()
|
||||
# Build per-search_index_row similarity scores from chunk-level results.
|
||||
# Each chunk_key encodes the search_index row type and id.
|
||||
# Track the best similarity per row (for ranking) and all chunks (for context).
|
||||
# Keep the best similarity (and its chunk text) per search_index row id.
|
||||
similarity_by_si_id: dict[int, float] = {}
|
||||
chunks_by_si_id: dict[int, list[tuple[float, str]]] = {}
|
||||
best_chunk_by_si_id: dict[int, str] = {}
|
||||
for row in vector_rows:
|
||||
chunk_key = row.get("chunk_key", "")
|
||||
distance = float(row["best_distance"])
|
||||
@@ -1331,11 +887,9 @@ class SearchRepositoryBase(ABC):
|
||||
current = similarity_by_si_id.get(si_id)
|
||||
if current is None or similarity > current:
|
||||
similarity_by_si_id[si_id] = similarity
|
||||
chunks_by_si_id.setdefault(si_id, []).append((similarity, chunk_text))
|
||||
best_chunk_by_si_id[si_id] = chunk_text
|
||||
|
||||
if not similarity_by_si_id:
|
||||
hydrate_ms = (time.perf_counter() - hydrate_start) * 1000
|
||||
_log_vector_summary()
|
||||
return []
|
||||
|
||||
# Filter out results below the minimum similarity threshold.
|
||||
@@ -1348,8 +902,6 @@ class SearchRepositoryBase(ABC):
|
||||
k: v for k, v in similarity_by_si_id.items() if v >= effective_min_similarity
|
||||
}
|
||||
if not similarity_by_si_id:
|
||||
hydrate_ms = (time.perf_counter() - hydrate_start) * 1000
|
||||
_log_vector_summary()
|
||||
return []
|
||||
|
||||
# Fetch the actual search_index rows
|
||||
@@ -1395,29 +947,15 @@ class SearchRepositoryBase(ABC):
|
||||
row = search_index_rows.get(si_id)
|
||||
if row is None:
|
||||
continue
|
||||
|
||||
# Small notes: return full content so the answer is always present.
|
||||
# Large notes: return top-N most relevant chunks for richer context.
|
||||
content_snippet = row.content_snippet or ""
|
||||
if content_snippet and len(content_snippet) <= SMALL_NOTE_CONTENT_LIMIT:
|
||||
matched_chunk_text = content_snippet
|
||||
else:
|
||||
si_chunks = chunks_by_si_id.get(si_id, [])
|
||||
si_chunks.sort(key=lambda c: c[0], reverse=True)
|
||||
top_texts = [text for _, text in si_chunks[:TOP_CHUNKS_PER_RESULT]]
|
||||
matched_chunk_text = "\n---\n".join(top_texts) if top_texts else None
|
||||
|
||||
ranked_rows.append(
|
||||
replace(
|
||||
row,
|
||||
score=similarity,
|
||||
matched_chunk_text=matched_chunk_text,
|
||||
matched_chunk_text=best_chunk_by_si_id.get(si_id),
|
||||
)
|
||||
)
|
||||
|
||||
ranked_rows.sort(key=lambda item: item.score or 0.0, reverse=True)
|
||||
hydrate_ms = (time.perf_counter() - hydrate_start) * 1000
|
||||
_log_vector_summary()
|
||||
return ranked_rows[offset : offset + limit]
|
||||
|
||||
async def _fetch_entity_rows_by_ids(self, entity_ids: list[int]) -> dict[int, SearchIndexRow]:
|
||||
@@ -1515,7 +1053,7 @@ class SearchRepositoryBase(ABC):
|
||||
return result
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Shared semantic search: hybrid score-based fusion
|
||||
# Shared semantic search: hybrid RRF fusion
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
async def _search_hybrid(
|
||||
@@ -1533,17 +1071,13 @@ class SearchRepositoryBase(ABC):
|
||||
limit: int,
|
||||
offset: int,
|
||||
) -> List[SearchIndexRow]:
|
||||
"""Fuse FTS and vector results using score-based fusion.
|
||||
"""Fuse FTS and vector rankings using reciprocal rank fusion (RRF).
|
||||
|
||||
Uses search_index row id as the fusion key. The formula
|
||||
``max(vec, fts) + FUSION_BONUS * min(vec, fts)`` preserves
|
||||
the dominant signal and rewards dual-source agreement.
|
||||
Uses entity_id as the fusion key (not permalink) to correctly handle
|
||||
entities with NULL permalinks.
|
||||
"""
|
||||
self._assert_semantic_available()
|
||||
query_text = search_text.strip()
|
||||
query_start = time.perf_counter()
|
||||
candidate_limit = max(self._semantic_vector_k, (limit + offset) * 10)
|
||||
fts_start = time.perf_counter()
|
||||
fts_results = await self.search(
|
||||
search_text=search_text,
|
||||
permalink=permalink,
|
||||
@@ -1557,8 +1091,6 @@ class SearchRepositoryBase(ABC):
|
||||
limit=candidate_limit,
|
||||
offset=0,
|
||||
)
|
||||
fts_ms = (time.perf_counter() - fts_start) * 1000
|
||||
vector_start = time.perf_counter()
|
||||
vector_results = await self._search_vector_only(
|
||||
search_text=search_text,
|
||||
permalink=permalink,
|
||||
@@ -1571,14 +1103,12 @@ class SearchRepositoryBase(ABC):
|
||||
min_similarity=min_similarity,
|
||||
limit=candidate_limit,
|
||||
offset=0,
|
||||
_emit_observability_log=False,
|
||||
)
|
||||
vector_ms = (time.perf_counter() - vector_start) * 1000
|
||||
fusion_start = time.perf_counter()
|
||||
|
||||
# --- Score-based fusion keyed on search_index row id ---
|
||||
# FTS scores are normalized to [0, 1] (BM25 is unbounded).
|
||||
# Vector scores are used raw — already calibrated [0, 1] by _distance_to_similarity().
|
||||
# Score-weighted RRF fusion keyed on search_index row id.
|
||||
# Multiplies the standard 1/(k+rank) score by the normalized original score
|
||||
# so that high-confidence matches contribute more than weak ones at the same rank.
|
||||
fused_scores: dict[int, float] = {}
|
||||
rows_by_id: dict[int, SearchIndexRow] = {}
|
||||
|
||||
# Normalize FTS scores to [0, 1] — handles both SQLite (negative bm25)
|
||||
@@ -1586,80 +1116,27 @@ class SearchRepositoryBase(ABC):
|
||||
fts_abs = [abs(row.score or 0.0) for row in fts_results]
|
||||
fts_max = max(fts_abs) if fts_abs else 1.0
|
||||
|
||||
fts_scores: dict[int, float] = {}
|
||||
for row in fts_results:
|
||||
for rank, row in enumerate(fts_results, start=1):
|
||||
if row.id is None:
|
||||
continue
|
||||
norm = abs(row.score or 0.0) / fts_max if fts_max > 0 else 0.0
|
||||
# Gate: FTS scores below threshold contribute zero
|
||||
if norm < FTS_GATE_THRESHOLD:
|
||||
norm = 0.0
|
||||
fts_scores[row.id] = norm
|
||||
weight = max(norm, 0.1) # floor preserves RRF stability
|
||||
fused_scores[row.id] = fused_scores.get(row.id, 0.0) + weight * (1.0 / (RRF_K + rank))
|
||||
rows_by_id[row.id] = row
|
||||
|
||||
vec_scores: dict[int, float] = {}
|
||||
for row in vector_results:
|
||||
# Vector scores already in [0, 1] from the similarity formula
|
||||
vec_max = max((row.score or 0.0) for row in vector_results) if vector_results else 1.0
|
||||
|
||||
for rank, row in enumerate(vector_results, start=1):
|
||||
if row.id is None:
|
||||
continue
|
||||
# Trigger: no re-normalization by vec_max
|
||||
# Why: vector similarity is already calibrated [0, 1]; re-normalizing
|
||||
# inflates weak matches when the entire result set is mediocre
|
||||
vec_scores[row.id] = row.score or 0.0
|
||||
norm = (row.score or 0.0) / vec_max if vec_max > 0 else 0.0
|
||||
weight = max(norm, 0.1) # floor preserves RRF stability
|
||||
fused_scores[row.id] = fused_scores.get(row.id, 0.0) + weight * (1.0 / (RRF_K + rank))
|
||||
rows_by_id[row.id] = row
|
||||
|
||||
# Fuse: max(v, f) + FUSION_BONUS * min(v, f)
|
||||
# Preserves the dominant signal; bonus rewards dual-source agreement.
|
||||
# Output range: [0, 1.3] for dual-source, [0, 1.0] for single-source.
|
||||
fused_scores: dict[int, float] = {}
|
||||
for row_id in fts_scores.keys() | vec_scores.keys():
|
||||
v = vec_scores.get(row_id, 0.0)
|
||||
f = fts_scores.get(row_id, 0.0)
|
||||
fused_scores[row_id] = max(v, f) + FUSION_BONUS * min(v, f)
|
||||
|
||||
ranked = sorted(fused_scores.items(), key=lambda item: item[1], reverse=True)
|
||||
output: list[SearchIndexRow] = []
|
||||
for row_id, fused_score in ranked[offset : offset + limit]:
|
||||
row = rows_by_id[row_id]
|
||||
# Trigger: FTS-only results have no matched_chunk_text from vector search.
|
||||
# Why: without chunk text, API falls back to truncated content, losing answer text.
|
||||
# Outcome: FTS-only results get full content_snippet as matched_chunk.
|
||||
if row.matched_chunk_text is None and row.content_snippet:
|
||||
row = replace(row, matched_chunk_text=row.content_snippet)
|
||||
output.append(replace(row, score=fused_score))
|
||||
fusion_ms = (time.perf_counter() - fusion_start) * 1000
|
||||
total_ms = (time.perf_counter() - query_start) * 1000
|
||||
logger.info(
|
||||
"Semantic query timing: project_id={project_id} retrieval_mode={retrieval_mode} "
|
||||
"query_length={query_length} candidate_limit={candidate_limit} "
|
||||
"fts_count={fts_count} vector_count={vector_count} fts_ms={fts_ms:.2f} "
|
||||
"vector_ms={vector_ms:.2f} fusion_ms={fusion_ms:.2f} total_ms={total_ms:.2f}",
|
||||
project_id=self.project_id,
|
||||
retrieval_mode="hybrid",
|
||||
query_length=len(query_text),
|
||||
candidate_limit=candidate_limit,
|
||||
fts_count=len(fts_results),
|
||||
vector_count=len(vector_results),
|
||||
fts_ms=fts_ms,
|
||||
vector_ms=vector_ms,
|
||||
fusion_ms=fusion_ms,
|
||||
total_ms=total_ms,
|
||||
)
|
||||
if total_ms > 2500:
|
||||
logger.warning(
|
||||
"[SEMANTIC_SLOW_QUERY] Semantic query timing: project_id={project_id} "
|
||||
"retrieval_mode={retrieval_mode} query_length={query_length} "
|
||||
"candidate_limit={candidate_limit} fts_count={fts_count} "
|
||||
"vector_count={vector_count} fts_ms={fts_ms:.2f} vector_ms={vector_ms:.2f} "
|
||||
"fusion_ms={fusion_ms:.2f} total_ms={total_ms:.2f}",
|
||||
project_id=self.project_id,
|
||||
retrieval_mode="hybrid",
|
||||
query_length=len(query_text),
|
||||
candidate_limit=candidate_limit,
|
||||
fts_count=len(fts_results),
|
||||
vector_count=len(vector_results),
|
||||
fts_ms=fts_ms,
|
||||
vector_ms=vector_ms,
|
||||
fusion_ms=fusion_ms,
|
||||
total_ms=total_ms,
|
||||
)
|
||||
output.append(replace(rows_by_id[row_id], score=fused_score))
|
||||
return output
|
||||
|
||||
@@ -52,9 +52,6 @@ class SQLiteSearchRepository(SearchRepositoryBase):
|
||||
self._semantic_enabled = self._app_config.semantic_search_enabled
|
||||
self._semantic_vector_k = self._app_config.semantic_vector_k
|
||||
self._semantic_min_similarity = self._app_config.semantic_min_similarity
|
||||
self._semantic_embedding_sync_batch_size = (
|
||||
self._app_config.semantic_embedding_sync_batch_size
|
||||
)
|
||||
self._embedding_provider = embedding_provider
|
||||
self._sqlite_vec_lock = asyncio.Lock()
|
||||
self._vector_tables_initialized = False
|
||||
@@ -82,7 +79,7 @@ class SQLiteSearchRepository(SearchRepositoryBase):
|
||||
across server restarts. Also creates vector tables when semantic search
|
||||
is enabled so missing dependencies are caught at startup, not first query.
|
||||
"""
|
||||
logger.debug("Initializing SQLite FTS5 search index")
|
||||
logger.info("Initializing SQLite FTS5 search index")
|
||||
try:
|
||||
async with db.scoped_session(self.session_maker) as session:
|
||||
# Create FTS5 virtual table if it doesn't exist
|
||||
@@ -381,7 +378,7 @@ class SQLiteSearchRepository(SearchRepositoryBase):
|
||||
if self._vector_tables_initialized:
|
||||
return
|
||||
|
||||
logger.debug("Ensuring SQLite vector tables exist for semantic search")
|
||||
logger.info("Ensuring SQLite vector tables exist for semantic search")
|
||||
|
||||
async with db.scoped_session(self.session_maker) as session:
|
||||
await self._ensure_sqlite_vec_loaded(session)
|
||||
@@ -434,7 +431,7 @@ class SQLiteSearchRepository(SearchRepositoryBase):
|
||||
await session.execute(create_sqlite_search_vector_embeddings(self._vector_dimensions))
|
||||
await session.commit()
|
||||
|
||||
logger.debug(f"SQLite vector tables ready (dimensions={self._vector_dimensions})")
|
||||
logger.info(f"SQLite vector tables ready (dimensions={self._vector_dimensions})")
|
||||
self._vector_tables_initialized = True
|
||||
|
||||
async def _prepare_vector_session(self, session: AsyncSession) -> None:
|
||||
|
||||
@@ -14,7 +14,6 @@ Syntax reference:
|
||||
EntityName as type (capitalized) # entity reference
|
||||
"""
|
||||
|
||||
import re
|
||||
from dataclasses import dataclass, field
|
||||
|
||||
|
||||
@@ -125,31 +124,6 @@ def _is_entity_ref_type(type_str: str) -> bool:
|
||||
return len(type_str) > 0 and type_str[0].isupper()
|
||||
|
||||
|
||||
# --- Enum String Parsing ---
|
||||
|
||||
|
||||
def _parse_enum_string(value: str) -> tuple[list[str], str | None]:
|
||||
"""Parse a string-typed enum value into enum values and optional description.
|
||||
|
||||
When picoschema enum values are quoted in YAML frontmatter (required when a
|
||||
description follows the list), YAML parses the whole thing as a string. This
|
||||
function extracts the enum values and description from that string.
|
||||
|
||||
Examples:
|
||||
"[active, blocked, done], current state" -> (['active', 'blocked', 'done'], 'current state')
|
||||
"[active, blocked]" -> (['active', 'blocked'], None)
|
||||
"active" -> (['active'], None)
|
||||
"""
|
||||
# Match bracketed list with optional trailing description
|
||||
m = re.match(r"\[([^\]]+)\](?:\s*,\s*(.+))?", value)
|
||||
if m:
|
||||
items = [item.strip() for item in m.group(1).split(",")]
|
||||
description = m.group(2).strip() if m.group(2) else None
|
||||
return items, description
|
||||
# Plain string — single enum value
|
||||
return [value.strip()], None
|
||||
|
||||
|
||||
# --- Main Parser ---
|
||||
|
||||
|
||||
@@ -173,25 +147,18 @@ def parse_picoschema(yaml_dict: dict) -> list[SchemaField]:
|
||||
name, required, is_array, is_enum, is_object = _parse_field_key(key)
|
||||
|
||||
# --- Enum fields ---
|
||||
# Trigger: value is a list or a string containing bracketed enum values
|
||||
# Why: enums declare allowed values directly as a YAML list, or as a quoted
|
||||
# string when a description follows (e.g., "[a, b], desc" must be quoted
|
||||
# in YAML to avoid parse errors)
|
||||
# Trigger: value is a list (e.g., [active, inactive])
|
||||
# Why: enums declare allowed values directly as a YAML list
|
||||
# Outcome: SchemaField with is_enum=True and enum_values populated
|
||||
if is_enum:
|
||||
description = None
|
||||
if isinstance(value, list):
|
||||
enum_values = [str(v) for v in value]
|
||||
else:
|
||||
enum_values, description = _parse_enum_string(str(value))
|
||||
enum_values = value if isinstance(value, list) else [str(value)]
|
||||
fields.append(
|
||||
SchemaField(
|
||||
name=name,
|
||||
type="enum",
|
||||
required=required,
|
||||
is_enum=True,
|
||||
enum_values=enum_values,
|
||||
description=description,
|
||||
enum_values=[str(v) for v in enum_values],
|
||||
)
|
||||
)
|
||||
continue
|
||||
|
||||
@@ -1,318 +0,0 @@
|
||||
"""Schemas for Local+ graph intelligence and FCM contracts."""
|
||||
|
||||
from datetime import datetime
|
||||
from typing import Any, Literal
|
||||
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
|
||||
# --- Graph contracts ---
|
||||
|
||||
|
||||
class GraphLineageRequest(BaseModel):
|
||||
"""Request contract for graph lineage queries."""
|
||||
|
||||
start: str
|
||||
goal: str | None = None
|
||||
max_hops: int = Field(default=4, ge=1, le=6)
|
||||
relation_filters: list[str] = Field(default_factory=list)
|
||||
|
||||
|
||||
class GraphNodeRef(BaseModel):
|
||||
"""Minimal graph node descriptor."""
|
||||
|
||||
id: str
|
||||
title: str
|
||||
permalink: str | None = None
|
||||
|
||||
|
||||
class GraphPathEdge(BaseModel):
|
||||
"""Edge descriptor for lineage paths."""
|
||||
|
||||
relation: str
|
||||
direction: Literal["outgoing", "incoming"]
|
||||
|
||||
|
||||
class GraphLineagePath(BaseModel):
|
||||
"""Single lineage path with scores and provenance."""
|
||||
|
||||
path_id: str
|
||||
nodes: list[GraphNodeRef] = Field(default_factory=list)
|
||||
edges: list[GraphPathEdge] = Field(default_factory=list)
|
||||
deterministic_path_score: float
|
||||
confidence: float
|
||||
evidence_refs: list[str] = Field(default_factory=list)
|
||||
|
||||
|
||||
class GraphLineageResponse(BaseModel):
|
||||
"""Response contract for graph lineage queries."""
|
||||
|
||||
root: GraphNodeRef
|
||||
paths: list[GraphLineagePath] = Field(default_factory=list)
|
||||
generated_at: datetime
|
||||
|
||||
|
||||
class GraphImpactRequest(BaseModel):
|
||||
"""Request contract for impact-radius queries."""
|
||||
|
||||
target: str
|
||||
horizon: int = Field(ge=1, le=4)
|
||||
relation_filters: list[str] = Field(default_factory=list)
|
||||
include_reasons: bool = True
|
||||
|
||||
|
||||
class GraphImpactTarget(BaseModel):
|
||||
"""Impact response target descriptor."""
|
||||
|
||||
id: str
|
||||
title: str
|
||||
|
||||
|
||||
class GraphImpactItem(BaseModel):
|
||||
"""Affected node entry for impact responses."""
|
||||
|
||||
id: str
|
||||
title: str
|
||||
distance: int
|
||||
impact_score: float
|
||||
confidence: float
|
||||
reasons: list[str] = Field(default_factory=list)
|
||||
evidence_refs: list[str] = Field(default_factory=list)
|
||||
|
||||
|
||||
class GraphImpactSummary(BaseModel):
|
||||
"""Summary counters for impact responses."""
|
||||
|
||||
total_considered: int
|
||||
total_returned: int
|
||||
|
||||
|
||||
class GraphImpactResponse(BaseModel):
|
||||
"""Response contract for impact-radius queries."""
|
||||
|
||||
target: GraphImpactTarget
|
||||
affected: list[GraphImpactItem] = Field(default_factory=list)
|
||||
summary: GraphImpactSummary
|
||||
|
||||
|
||||
class GraphHealthMetrics(BaseModel):
|
||||
"""Top-level graph health metrics."""
|
||||
|
||||
orphan_rate: float
|
||||
stale_central_nodes: int
|
||||
overloaded_hubs: int
|
||||
contradiction_candidates: int
|
||||
|
||||
|
||||
class GraphHealthIssue(BaseModel):
|
||||
"""Actionable graph-health issue entry."""
|
||||
|
||||
issue_type: Literal[
|
||||
"orphan",
|
||||
"stale_central",
|
||||
"overloaded_hub",
|
||||
"contradiction_candidate",
|
||||
]
|
||||
entity_id: str
|
||||
severity: Literal["low", "medium", "high"]
|
||||
reason: str
|
||||
suggested_action: str
|
||||
confidence: float | None = None
|
||||
|
||||
|
||||
class GraphHealthResponse(BaseModel):
|
||||
"""Response contract for health checks."""
|
||||
|
||||
metrics: GraphHealthMetrics
|
||||
issues: list[GraphHealthIssue] = Field(default_factory=list)
|
||||
computed_at: datetime
|
||||
|
||||
|
||||
class GraphReindexRequest(BaseModel):
|
||||
"""Request contract for graph reindex scheduling."""
|
||||
|
||||
mode: Literal["full", "incremental"] = "incremental"
|
||||
reason: str | None = None
|
||||
|
||||
|
||||
class GraphReindexResponse(BaseModel):
|
||||
"""Response contract for graph reindex scheduling."""
|
||||
|
||||
job_id: str
|
||||
status: Literal["queued", "running", "completed", "failed"]
|
||||
scheduled_at: datetime
|
||||
|
||||
|
||||
# --- FCM contracts ---
|
||||
|
||||
|
||||
class FCMAction(BaseModel):
|
||||
"""Action delta for simulation input."""
|
||||
|
||||
node_id: str
|
||||
delta: float
|
||||
|
||||
|
||||
class FCMScenario(BaseModel):
|
||||
"""Simulation runtime configuration."""
|
||||
|
||||
steps: int = 12
|
||||
activation: Literal["tanh", "sigmoid", "bounded_linear"] = "tanh"
|
||||
decay: float = 0.05
|
||||
|
||||
|
||||
class FCMClampRule(BaseModel):
|
||||
"""Clamp bounds for selected nodes."""
|
||||
|
||||
node_id: str
|
||||
min: float
|
||||
max: float
|
||||
|
||||
|
||||
class FCMSimulateRequest(BaseModel):
|
||||
"""Request contract for FCM simulation."""
|
||||
|
||||
actions: list[FCMAction]
|
||||
scenario: FCMScenario = Field(default_factory=FCMScenario)
|
||||
clamp_rules: list[FCMClampRule] = Field(default_factory=list)
|
||||
|
||||
|
||||
class FCMNodeState(BaseModel):
|
||||
"""Node state in baseline/projected vectors."""
|
||||
|
||||
node_id: str
|
||||
state: float
|
||||
|
||||
|
||||
class FCMNodeDelta(BaseModel):
|
||||
"""Node delta entry in simulation output."""
|
||||
|
||||
node_id: str
|
||||
delta: float
|
||||
|
||||
|
||||
class FCMStability(BaseModel):
|
||||
"""Simulation stability metadata."""
|
||||
|
||||
converged: bool
|
||||
iterations_used: int
|
||||
residual: float
|
||||
|
||||
|
||||
class FCMInfluencer(BaseModel):
|
||||
"""Top influencer entry for explanation payload."""
|
||||
|
||||
source: str
|
||||
weight: float
|
||||
|
||||
|
||||
class FCMExplanation(BaseModel):
|
||||
"""Per-node explanation payload."""
|
||||
|
||||
node_id: str
|
||||
top_influencers: list[FCMInfluencer] = Field(default_factory=list)
|
||||
|
||||
|
||||
class FCMSimulateResponse(BaseModel):
|
||||
"""Response contract for FCM simulation."""
|
||||
|
||||
baseline: list[FCMNodeState] = Field(default_factory=list)
|
||||
projected: list[FCMNodeState] = Field(default_factory=list)
|
||||
deltas: list[FCMNodeDelta] = Field(default_factory=list)
|
||||
stability: FCMStability
|
||||
confidence: float
|
||||
explanations: list[FCMExplanation] = Field(default_factory=list)
|
||||
evidence_refs: list[str] = Field(default_factory=list)
|
||||
|
||||
|
||||
class FCMRankConstraints(BaseModel):
|
||||
"""Constraint set for action ranking."""
|
||||
|
||||
max_negative_impact: float | None = None
|
||||
required_tags: list[str] = Field(default_factory=list)
|
||||
disallowed_nodes: list[str] = Field(default_factory=list)
|
||||
|
||||
|
||||
class FCMRankActionsRequest(BaseModel):
|
||||
"""Request contract for FCM action ranking."""
|
||||
|
||||
goal: str
|
||||
constraints: FCMRankConstraints = Field(default_factory=FCMRankConstraints)
|
||||
top_k: int = Field(default=10, ge=1, le=25)
|
||||
|
||||
|
||||
class FCMGoalRef(BaseModel):
|
||||
"""Goal descriptor for ranking output."""
|
||||
|
||||
node_id: str
|
||||
label: str
|
||||
|
||||
|
||||
class FCMRecommendation(BaseModel):
|
||||
"""Ranked intervention candidate."""
|
||||
|
||||
action_node_id: str
|
||||
expected_goal_delta: float
|
||||
risk_penalty: float
|
||||
net_score: float
|
||||
confidence: float
|
||||
rationale: list[str] = Field(default_factory=list)
|
||||
evidence_refs: list[str] = Field(default_factory=list)
|
||||
|
||||
|
||||
class FCMRankActionsResponse(BaseModel):
|
||||
"""Response contract for action ranking."""
|
||||
|
||||
goal: FCMGoalRef
|
||||
recommendations: list[FCMRecommendation] = Field(default_factory=list)
|
||||
|
||||
|
||||
class FCMImportRequest(BaseModel):
|
||||
"""Request contract for model import."""
|
||||
|
||||
source: str
|
||||
format: Literal["csv_bundle_v1"] = "csv_bundle_v1"
|
||||
merge_mode: Literal["replace", "upsert"] = "upsert"
|
||||
|
||||
|
||||
class FCMImportResponse(BaseModel):
|
||||
"""Response contract for model import."""
|
||||
|
||||
import_id: str
|
||||
nodes_loaded: int
|
||||
edges_loaded: int
|
||||
warnings: list[str] = Field(default_factory=list)
|
||||
errors: list[str] = Field(default_factory=list)
|
||||
|
||||
|
||||
class FCMExportSelection(BaseModel):
|
||||
"""Scope selection for model export."""
|
||||
|
||||
scope: Literal["all", "tag", "subgraph"] = "all"
|
||||
tag: str | None = None
|
||||
seed_nodes: list[str] = Field(default_factory=list)
|
||||
|
||||
|
||||
class FCMExportRequest(BaseModel):
|
||||
"""Request contract for model export."""
|
||||
|
||||
format: Literal["csv_bundle_v1"] = "csv_bundle_v1"
|
||||
selection: FCMExportSelection = Field(default_factory=FCMExportSelection)
|
||||
|
||||
|
||||
class FCMExportFile(BaseModel):
|
||||
"""Single file descriptor in an export response."""
|
||||
|
||||
name: str
|
||||
path: str
|
||||
|
||||
|
||||
class FCMExportResponse(BaseModel):
|
||||
"""Response contract for model export."""
|
||||
|
||||
export_id: str
|
||||
format: Literal["csv_bundle_v1"]
|
||||
files: list[FCMExportFile] = Field(default_factory=list)
|
||||
node_count: int
|
||||
edge_count: int
|
||||
metadata: dict[str, Any] | None = None
|
||||
@@ -125,8 +125,7 @@ class EntitySummary(BaseModel):
|
||||
|
||||
type: Literal["entity"] = "entity"
|
||||
external_id: str # UUID for v2 API routing
|
||||
# COMPAT(v0.18): old clients expect these fields in JSON
|
||||
entity_id: Optional[int] = None
|
||||
entity_id: int # Database ID for v2 API consistency
|
||||
permalink: Optional[str]
|
||||
title: str
|
||||
content: Optional[str] = None
|
||||
@@ -144,19 +143,18 @@ class RelationSummary(BaseModel):
|
||||
"""Simplified relation representation."""
|
||||
|
||||
type: Literal["relation"] = "relation"
|
||||
# COMPAT(v0.18): old clients expect these fields in JSON
|
||||
relation_id: Optional[int] = None
|
||||
entity_id: Optional[int] = None
|
||||
relation_id: int # Database ID for v2 API consistency
|
||||
entity_id: Optional[int] = None # ID of the entity this relation belongs to
|
||||
title: str
|
||||
file_path: str
|
||||
permalink: str
|
||||
relation_type: str
|
||||
from_entity: Optional[str] = None
|
||||
from_entity_id: Optional[int] = None
|
||||
from_entity_external_id: Optional[str] = None
|
||||
from_entity_id: Optional[int] = None # ID of source entity
|
||||
from_entity_external_id: Optional[str] = None # UUID of source entity for v2 API routing
|
||||
to_entity: Optional[str] = None
|
||||
to_entity_id: Optional[int] = None
|
||||
to_entity_external_id: Optional[str] = None
|
||||
to_entity_id: Optional[int] = None # ID of target entity
|
||||
to_entity_external_id: Optional[str] = None # UUID of target entity for v2 API routing
|
||||
created_at: Annotated[
|
||||
datetime, Field(json_schema_extra={"type": "string", "format": "date-time"})
|
||||
]
|
||||
@@ -170,11 +168,10 @@ class ObservationSummary(BaseModel):
|
||||
"""Simplified observation representation."""
|
||||
|
||||
type: Literal["observation"] = "observation"
|
||||
# COMPAT(v0.18): old clients expect these fields in JSON
|
||||
observation_id: Optional[int] = None
|
||||
entity_id: Optional[int] = None
|
||||
entity_external_id: Optional[str] = None
|
||||
title: Optional[str] = None
|
||||
observation_id: int # Database ID for v2 API consistency
|
||||
entity_id: Optional[int] = None # ID of the entity this observation belongs to
|
||||
entity_external_id: Optional[str] = None # UUID of parent entity for v2 API routing
|
||||
title: str
|
||||
file_path: str
|
||||
permalink: str
|
||||
category: str
|
||||
@@ -195,17 +192,18 @@ class MemoryMetadata(BaseModel):
|
||||
types: Optional[List[SearchItemType]] = None
|
||||
depth: int
|
||||
timeframe: Optional[str] = None
|
||||
# COMPAT(v0.18): old clients expect generated_at and total_results in JSON
|
||||
generated_at: Optional[datetime] = None
|
||||
primary_count: Optional[int] = None
|
||||
related_count: Optional[int] = None
|
||||
total_results: Optional[int] = None
|
||||
generated_at: Annotated[
|
||||
datetime, Field(json_schema_extra={"type": "string", "format": "date-time"})
|
||||
]
|
||||
primary_count: Optional[int] = None # Changed field name
|
||||
related_count: Optional[int] = None # Changed field name
|
||||
total_results: Optional[int] = None # For backward compatibility
|
||||
total_relations: Optional[int] = None
|
||||
total_observations: Optional[int] = None
|
||||
|
||||
@field_serializer("generated_at")
|
||||
def serialize_generated_at(self, dt: Optional[datetime]) -> Optional[str]:
|
||||
return dt.isoformat() if dt else None
|
||||
def serialize_generated_at(self, dt: datetime) -> str:
|
||||
return dt.isoformat()
|
||||
|
||||
|
||||
class ContextResult(BaseModel):
|
||||
|
||||
@@ -14,7 +14,7 @@ Key Features:
|
||||
from datetime import datetime
|
||||
from typing import List, Optional, Dict
|
||||
|
||||
from pydantic import BaseModel, ConfigDict, Field, computed_field, model_validator
|
||||
from pydantic import BaseModel, ConfigDict, Field, model_validator
|
||||
|
||||
from basic_memory.schemas.base import Relation, Permalink, NoteType, ContentType, Observation
|
||||
|
||||
@@ -192,13 +192,6 @@ class EntityResponse(SQLAlchemyModel):
|
||||
title: str
|
||||
file_path: str
|
||||
note_type: NoteType
|
||||
|
||||
# COMPAT(v0.18): old clients expect entity_type; remove when no longer needed
|
||||
@computed_field # type: ignore[prop-decorator]
|
||||
@property
|
||||
def entity_type(self) -> str:
|
||||
return self.note_type
|
||||
|
||||
entity_metadata: Optional[Dict] = None
|
||||
checksum: Optional[str] = None
|
||||
content_type: ContentType
|
||||
|
||||
@@ -19,11 +19,7 @@ from basic_memory.file_utils import (
|
||||
dump_frontmatter,
|
||||
)
|
||||
from basic_memory.markdown import EntityMarkdown
|
||||
from basic_memory.markdown.entity_parser import (
|
||||
EntityParser,
|
||||
_coerce_to_string,
|
||||
normalize_frontmatter_metadata,
|
||||
)
|
||||
from basic_memory.markdown.entity_parser import EntityParser, normalize_frontmatter_metadata
|
||||
from basic_memory.markdown.utils import entity_model_from_markdown, schema_to_markdown
|
||||
from basic_memory.models import Entity as EntityModel
|
||||
from basic_memory.models import Observation, Relation
|
||||
@@ -361,11 +357,8 @@ class EntityService(BaseService[EntityModel]):
|
||||
# in the existing file. Setting it unconditionally preserves the correct value.
|
||||
existing_markdown.frontmatter.metadata["permalink"] = new_permalink
|
||||
|
||||
# Create a new post with merged metadata.
|
||||
# Avoid **metadata unpacking — user frontmatter may contain reserved keys
|
||||
# like 'content' or 'handler' that conflict with Post.__init__ (cloud#375).
|
||||
merged_post = frontmatter.Post(post.content)
|
||||
merged_post.metadata.update(existing_markdown.frontmatter.metadata)
|
||||
# Create a new post with merged metadata
|
||||
merged_post = frontmatter.Post(post.content, **existing_markdown.frontmatter.metadata)
|
||||
|
||||
# write file
|
||||
final_content = dump_frontmatter(merged_post)
|
||||
@@ -508,11 +501,10 @@ class EntityService(BaseService[EntityModel]):
|
||||
if has_frontmatter(new_content):
|
||||
content_frontmatter = parse_frontmatter(new_content)
|
||||
|
||||
# Coerce to string — YAML may parse these as lists (cloud#376)
|
||||
if "title" in content_frontmatter:
|
||||
update_data["title"] = _coerce_to_string(content_frontmatter["title"])
|
||||
update_data["title"] = content_frontmatter["title"]
|
||||
if "type" in content_frontmatter:
|
||||
update_data["note_type"] = _coerce_to_string(content_frontmatter["type"])
|
||||
update_data["note_type"] = content_frontmatter["type"]
|
||||
|
||||
if "permalink" in content_frontmatter:
|
||||
content_markdown = self._build_frontmatter_markdown(
|
||||
|
||||
@@ -1,96 +0,0 @@
|
||||
"""Service layer for FCM contract endpoints."""
|
||||
|
||||
from uuid import uuid4
|
||||
|
||||
from basic_memory.schemas.graph_intelligence import (
|
||||
FCMExportFile,
|
||||
FCMExportRequest,
|
||||
FCMExportResponse,
|
||||
FCMGoalRef,
|
||||
FCMImportRequest,
|
||||
FCMImportResponse,
|
||||
FCMNodeDelta,
|
||||
FCMNodeState,
|
||||
FCMRankActionsRequest,
|
||||
FCMRankActionsResponse,
|
||||
FCMRecommendation,
|
||||
FCMSimulateRequest,
|
||||
FCMSimulateResponse,
|
||||
FCMStability,
|
||||
)
|
||||
|
||||
|
||||
class FCMService:
|
||||
"""FCM contract service.
|
||||
|
||||
Phase 1 keeps deterministic behavior so API and tool surfaces stabilize
|
||||
before introducing advanced simulation engines.
|
||||
"""
|
||||
|
||||
async def simulate(self, request: FCMSimulateRequest) -> FCMSimulateResponse:
|
||||
"""Return deterministic baseline/projected state vectors."""
|
||||
baseline = [FCMNodeState(node_id=action.node_id, state=0.0) for action in request.actions]
|
||||
projected = [
|
||||
FCMNodeState(node_id=action.node_id, state=action.delta) for action in request.actions
|
||||
]
|
||||
deltas = [
|
||||
FCMNodeDelta(node_id=action.node_id, delta=action.delta) for action in request.actions
|
||||
]
|
||||
return FCMSimulateResponse(
|
||||
baseline=baseline,
|
||||
projected=projected,
|
||||
deltas=deltas,
|
||||
stability=FCMStability(
|
||||
converged=True,
|
||||
iterations_used=min(request.scenario.steps, 5),
|
||||
residual=0.0,
|
||||
),
|
||||
confidence=0.5,
|
||||
explanations=[],
|
||||
evidence_refs=[],
|
||||
)
|
||||
|
||||
async def rank_actions(self, request: FCMRankActionsRequest) -> FCMRankActionsResponse:
|
||||
"""Return deterministic ranked actions for a target goal."""
|
||||
recommendations = [
|
||||
FCMRecommendation(
|
||||
action_node_id=f"{request.goal}:action:{idx + 1}",
|
||||
expected_goal_delta=0.25 - (idx * 0.01),
|
||||
risk_penalty=0.05 + (idx * 0.005),
|
||||
net_score=0.20 - (idx * 0.015),
|
||||
confidence=0.5,
|
||||
rationale=["Contract skeleton recommendation"],
|
||||
evidence_refs=[],
|
||||
)
|
||||
for idx in range(min(request.top_k, 3))
|
||||
]
|
||||
return FCMRankActionsResponse(
|
||||
goal=FCMGoalRef(node_id=request.goal, label=request.goal),
|
||||
recommendations=recommendations,
|
||||
)
|
||||
|
||||
async def import_model(self, request: FCMImportRequest) -> FCMImportResponse:
|
||||
"""Return deterministic import metadata."""
|
||||
_ = request
|
||||
return FCMImportResponse(
|
||||
import_id=str(uuid4()),
|
||||
nodes_loaded=0,
|
||||
edges_loaded=0,
|
||||
warnings=[],
|
||||
errors=[],
|
||||
)
|
||||
|
||||
async def export_model(self, request: FCMExportRequest) -> FCMExportResponse:
|
||||
"""Return deterministic export metadata and file descriptors."""
|
||||
scope = request.selection.scope
|
||||
return FCMExportResponse(
|
||||
export_id=str(uuid4()),
|
||||
format=request.format,
|
||||
files=[
|
||||
FCMExportFile(name="nodes.csv", path=f"/tmp/{scope}-nodes.csv"),
|
||||
FCMExportFile(name="edges.csv", path=f"/tmp/{scope}-edges.csv"),
|
||||
],
|
||||
node_count=0,
|
||||
edge_count=0,
|
||||
metadata={"scope": scope},
|
||||
)
|
||||
@@ -1,122 +0,0 @@
|
||||
"""Service layer for graph intelligence contract endpoints."""
|
||||
|
||||
from datetime import datetime, timezone
|
||||
from uuid import uuid4
|
||||
|
||||
from basic_memory.schemas.graph_intelligence import (
|
||||
GraphHealthMetrics,
|
||||
GraphHealthResponse,
|
||||
GraphImpactItem,
|
||||
GraphImpactRequest,
|
||||
GraphImpactResponse,
|
||||
GraphImpactSummary,
|
||||
GraphImpactTarget,
|
||||
GraphLineagePath,
|
||||
GraphLineageRequest,
|
||||
GraphLineageResponse,
|
||||
GraphNodeRef,
|
||||
GraphPathEdge,
|
||||
GraphReindexResponse,
|
||||
)
|
||||
|
||||
|
||||
def _normalize_memory_ref(value: str) -> str:
|
||||
"""Normalize user input into a memory:// reference string."""
|
||||
if value.startswith("memory://"):
|
||||
return value
|
||||
return f"memory://{value}"
|
||||
|
||||
|
||||
def _normalize_node_id(value: str) -> str:
|
||||
"""Return a stable node id for contract skeleton outputs."""
|
||||
return value.removeprefix("memory://")
|
||||
|
||||
|
||||
class GraphIntelligenceService:
|
||||
"""Graph intelligence contract service.
|
||||
|
||||
Phase 1 behavior is intentionally deterministic and lightweight so routing,
|
||||
clients, and contract tests can ship before deeper traversal engines.
|
||||
"""
|
||||
|
||||
async def lineage(self, request: GraphLineageRequest) -> GraphLineageResponse:
|
||||
"""Return a deterministic lineage payload for the requested root/goal."""
|
||||
root_ref = _normalize_memory_ref(request.start)
|
||||
root = GraphNodeRef(
|
||||
id=_normalize_node_id(root_ref),
|
||||
title=_normalize_node_id(root_ref),
|
||||
permalink=_normalize_node_id(root_ref),
|
||||
)
|
||||
|
||||
nodes = [root]
|
||||
edges: list[GraphPathEdge] = []
|
||||
if request.goal:
|
||||
goal_ref = _normalize_memory_ref(request.goal)
|
||||
nodes.append(
|
||||
GraphNodeRef(
|
||||
id=_normalize_node_id(goal_ref),
|
||||
title=_normalize_node_id(goal_ref),
|
||||
permalink=_normalize_node_id(goal_ref),
|
||||
)
|
||||
)
|
||||
edges.append(GraphPathEdge(relation="related_to", direction="outgoing"))
|
||||
|
||||
path = GraphLineagePath(
|
||||
path_id=f"path-{uuid4()}",
|
||||
nodes=nodes,
|
||||
edges=edges,
|
||||
deterministic_path_score=1.0 if request.goal else 0.5,
|
||||
confidence=0.5,
|
||||
evidence_refs=[root_ref],
|
||||
)
|
||||
|
||||
return GraphLineageResponse(
|
||||
root=root,
|
||||
paths=[path],
|
||||
generated_at=datetime.now(timezone.utc),
|
||||
)
|
||||
|
||||
async def impact(self, request: GraphImpactRequest) -> GraphImpactResponse:
|
||||
"""Return a deterministic impact preview payload."""
|
||||
target_id = _normalize_node_id(_normalize_memory_ref(request.target))
|
||||
affected = [
|
||||
GraphImpactItem(
|
||||
id=f"{target_id}:neighbor:1",
|
||||
title=f"{target_id} dependent",
|
||||
distance=min(request.horizon, 1),
|
||||
impact_score=0.55,
|
||||
confidence=0.5,
|
||||
reasons=["Connected via typed relation in contract skeleton"],
|
||||
evidence_refs=[_normalize_memory_ref(request.target)],
|
||||
)
|
||||
]
|
||||
if not request.include_reasons:
|
||||
affected[0].reasons = []
|
||||
|
||||
return GraphImpactResponse(
|
||||
target=GraphImpactTarget(id=target_id, title=target_id),
|
||||
affected=affected,
|
||||
summary=GraphImpactSummary(total_considered=1, total_returned=1),
|
||||
)
|
||||
|
||||
async def health(self, scope: str | None, timeframe: str | None) -> GraphHealthResponse:
|
||||
"""Return deterministic baseline health metrics."""
|
||||
_ = (scope, timeframe)
|
||||
return GraphHealthResponse(
|
||||
metrics=GraphHealthMetrics(
|
||||
orphan_rate=0.0,
|
||||
stale_central_nodes=0,
|
||||
overloaded_hubs=0,
|
||||
contradiction_candidates=0,
|
||||
),
|
||||
issues=[],
|
||||
computed_at=datetime.now(timezone.utc),
|
||||
)
|
||||
|
||||
async def start_reindex_job(self) -> GraphReindexResponse:
|
||||
"""Create reindex job metadata for queued responses."""
|
||||
return GraphReindexResponse(
|
||||
job_id=str(uuid4()),
|
||||
status="queued",
|
||||
scheduled_at=datetime.now(timezone.utc),
|
||||
)
|
||||
@@ -301,10 +301,8 @@ class LinkResolver:
|
||||
)
|
||||
|
||||
if results:
|
||||
# Both SQLite and Postgres return results sorted best-first in SQL
|
||||
# (SQLite: ORDER BY score ASC for negative BM25, Postgres: ORDER BY score DESC
|
||||
# for positive ts_rank). Using results[0] is backend-agnostic and correct.
|
||||
best_match = results[0]
|
||||
# Look for best match
|
||||
best_match = min(results, key=lambda x: x.score) # pyright: ignore
|
||||
logger.trace(
|
||||
f"Selected best match from {len(results)} results: {best_match.permalink}"
|
||||
)
|
||||
|
||||
@@ -82,21 +82,6 @@ class ProjectService:
|
||||
"""
|
||||
return self.config_manager.default_project
|
||||
|
||||
async def get_default_project_name(self) -> str:
|
||||
"""Get the default project name, falling back to the database.
|
||||
|
||||
ConfigManager reads from the local config file, which doesn't exist
|
||||
in cloud mode. When it returns None, fall back to the is_default
|
||||
flag stored in the database.
|
||||
"""
|
||||
default = self.config_manager.default_project
|
||||
if default is not None:
|
||||
return default
|
||||
db_default = await self.repository.get_default_project()
|
||||
if db_default is not None:
|
||||
return db_default.name
|
||||
raise ValueError("No default project configured")
|
||||
|
||||
@property
|
||||
def current_project(self) -> Optional[str]:
|
||||
"""Get the name of the currently active project.
|
||||
@@ -957,21 +942,19 @@ class ProjectService:
|
||||
is_postgres = config.database_backend == DatabaseBackend.POSTGRES
|
||||
|
||||
# --- Check vector table existence ---
|
||||
# Both search_vector_chunks and search_vector_embeddings must exist
|
||||
# for the detailed stats queries (JOINs between them) to work.
|
||||
if is_postgres:
|
||||
table_check_sql = text(
|
||||
"SELECT COUNT(*) FROM information_schema.tables "
|
||||
"WHERE table_name IN ('search_vector_chunks', 'search_vector_embeddings')"
|
||||
"WHERE table_name = 'search_vector_chunks'"
|
||||
)
|
||||
else:
|
||||
table_check_sql = text(
|
||||
"SELECT COUNT(*) FROM sqlite_master "
|
||||
"WHERE type = 'table' AND name IN ('search_vector_chunks', 'search_vector_embeddings')"
|
||||
"WHERE type = 'table' AND name = 'search_vector_chunks'"
|
||||
)
|
||||
|
||||
table_result = await self.repository.execute_query(table_check_sql, {})
|
||||
vector_tables_exist = (table_result.scalar() or 0) == 2
|
||||
vector_tables_exist = (table_result.scalar() or 0) > 0
|
||||
|
||||
if not vector_tables_exist:
|
||||
# Count distinct entities in search index for the recommendation message
|
||||
@@ -992,13 +975,16 @@ class ProjectService:
|
||||
total_indexed_entities=total_indexed_entities,
|
||||
vector_tables_exist=False,
|
||||
reindex_recommended=True,
|
||||
reindex_reason=("Vector tables not initialized — run: bm reindex --embeddings"),
|
||||
reindex_reason=(
|
||||
"Vector tables not initialized — run: bm reindex --embeddings"
|
||||
),
|
||||
)
|
||||
|
||||
# --- Count queries (tables exist) ---
|
||||
si_result = await self.repository.execute_query(
|
||||
text(
|
||||
"SELECT COUNT(DISTINCT entity_id) FROM search_index WHERE project_id = :project_id"
|
||||
"SELECT COUNT(DISTINCT entity_id) FROM search_index "
|
||||
"WHERE project_id = :project_id"
|
||||
),
|
||||
{"project_id": project_id},
|
||||
)
|
||||
@@ -1052,7 +1038,9 @@ class ProjectService:
|
||||
"WHERE c.project_id = :project_id AND e.rowid IS NULL"
|
||||
)
|
||||
|
||||
orphan_result = await self.repository.execute_query(orphan_sql, {"project_id": project_id})
|
||||
orphan_result = await self.repository.execute_query(
|
||||
orphan_sql, {"project_id": project_id}
|
||||
)
|
||||
orphaned_chunks = orphan_result.scalar() or 0
|
||||
|
||||
# --- Reindex recommendation logic (priority order) ---
|
||||
@@ -1061,7 +1049,9 @@ class ProjectService:
|
||||
|
||||
if total_indexed_entities > 0 and total_chunks == 0:
|
||||
reindex_recommended = True
|
||||
reindex_reason = "Embeddings have never been built — run: bm reindex --embeddings"
|
||||
reindex_reason = (
|
||||
"Embeddings have never been built — run: bm reindex --embeddings"
|
||||
)
|
||||
elif orphaned_chunks > 0:
|
||||
reindex_recommended = True
|
||||
reindex_reason = (
|
||||
@@ -1071,7 +1061,9 @@ class ProjectService:
|
||||
elif total_indexed_entities > total_entities_with_chunks:
|
||||
missing = total_indexed_entities - total_entities_with_chunks
|
||||
reindex_recommended = True
|
||||
reindex_reason = f"{missing} entities missing embeddings — run: bm reindex --embeddings"
|
||||
reindex_reason = (
|
||||
f"{missing} entities missing embeddings — run: bm reindex --embeddings"
|
||||
)
|
||||
|
||||
return EmbeddingStatus(
|
||||
semantic_search_enabled=True,
|
||||
|
||||
@@ -13,11 +13,7 @@ from sqlalchemy import text
|
||||
|
||||
from basic_memory.models import Entity
|
||||
from basic_memory.repository import EntityRepository
|
||||
from basic_memory.repository.search_repository import (
|
||||
SearchIndexRow,
|
||||
SearchRepository,
|
||||
VectorSyncBatchResult,
|
||||
)
|
||||
from basic_memory.repository.search_repository import SearchRepository, SearchIndexRow
|
||||
from basic_memory.schemas.search import SearchQuery, SearchItemType, SearchRetrievalMode
|
||||
from basic_memory.services import FileService
|
||||
|
||||
@@ -351,7 +347,7 @@ class SearchService:
|
||||
entity: Entity,
|
||||
content: str | None = None,
|
||||
) -> None:
|
||||
logger.debug(
|
||||
logger.info(
|
||||
f"[BackgroundTask] Starting search index for entity_id={entity.id} "
|
||||
f"permalink={entity.permalink} project_id={entity.project_id}"
|
||||
)
|
||||
@@ -364,7 +360,7 @@ class SearchService:
|
||||
entity, content
|
||||
) if entity.is_markdown else await self.index_entity_file(entity)
|
||||
|
||||
logger.debug(
|
||||
logger.info(
|
||||
f"[BackgroundTask] Completed search index for entity_id={entity.id} "
|
||||
f"permalink={entity.permalink}"
|
||||
)
|
||||
@@ -381,17 +377,6 @@ class SearchService:
|
||||
"""Refresh vector chunks for one entity in repositories that support semantic indexing."""
|
||||
await self.repository.sync_entity_vectors(entity_id)
|
||||
|
||||
async def sync_entity_vectors_batch(
|
||||
self,
|
||||
entity_ids: list[int],
|
||||
progress_callback=None,
|
||||
) -> VectorSyncBatchResult:
|
||||
"""Refresh vector chunks for a batch of entities."""
|
||||
return await self.repository.sync_entity_vectors_batch(
|
||||
entity_ids,
|
||||
progress_callback=progress_callback,
|
||||
)
|
||||
|
||||
async def reindex_vectors(self, progress_callback=None) -> dict:
|
||||
"""Rebuild vector embeddings for all entities.
|
||||
|
||||
@@ -402,20 +387,17 @@ class SearchService:
|
||||
dict with stats: total_entities, embedded, skipped, errors
|
||||
"""
|
||||
entities = await self.entity_repository.find_all()
|
||||
entity_ids = [entity.id for entity in entities]
|
||||
batch_result = await self.repository.sync_entity_vectors_batch(
|
||||
entity_ids,
|
||||
progress_callback=progress_callback,
|
||||
)
|
||||
stats = {
|
||||
"total_entities": batch_result.entities_total,
|
||||
"embedded": batch_result.entities_synced,
|
||||
"skipped": 0,
|
||||
"errors": batch_result.entities_failed,
|
||||
}
|
||||
stats = {"total_entities": len(entities), "embedded": 0, "skipped": 0, "errors": 0}
|
||||
|
||||
for failed_entity_id in batch_result.failed_entity_ids:
|
||||
logger.warning(f"Failed to embed entity {failed_entity_id}")
|
||||
for i, entity in enumerate(entities):
|
||||
if progress_callback:
|
||||
progress_callback(entity.id, i, len(entities))
|
||||
try:
|
||||
await self.repository.sync_entity_vectors(entity.id)
|
||||
stats["embedded"] += 1
|
||||
except Exception as e:
|
||||
logger.warning(f"Failed to embed entity {entity.id} ({entity.permalink}): {e}")
|
||||
stats["errors"] += 1
|
||||
|
||||
return stats
|
||||
|
||||
|
||||
@@ -437,13 +437,13 @@ class SyncService:
|
||||
elif project.last_scan_timestamp is not None:
|
||||
# Incremental scan: only files modified since last scan
|
||||
scan_type = "incremental"
|
||||
logger.debug(
|
||||
logger.info(
|
||||
f"Running incremental scan for files modified since {project.last_scan_timestamp}"
|
||||
)
|
||||
file_paths_to_scan = await self._scan_directory_modified_since(
|
||||
directory, project.last_scan_timestamp
|
||||
)
|
||||
logger.debug(
|
||||
logger.info(
|
||||
f"Incremental scan found {len(file_paths_to_scan)} potentially changed files"
|
||||
)
|
||||
|
||||
@@ -705,7 +705,7 @@ class SyncService:
|
||||
|
||||
# If permalink changed, update the file
|
||||
if permalink != entity_markdown.frontmatter.permalink:
|
||||
logger.debug(
|
||||
logger.info(
|
||||
f"Updating permalink for path: {path}, old_permalink: {entity_markdown.frontmatter.permalink}, new_permalink: {permalink}"
|
||||
)
|
||||
|
||||
|
||||
@@ -281,7 +281,7 @@ def setup_logging(
|
||||
str(log_path),
|
||||
level=log_level,
|
||||
rotation="10 MB",
|
||||
retention=5,
|
||||
retention="10 days",
|
||||
backtrace=True,
|
||||
diagnose=True,
|
||||
enqueue=False,
|
||||
|
||||
@@ -211,34 +211,6 @@ def test_edit_note_replace_section_fails_without_section(
|
||||
assert "section parameter is required for replace_section operation" in result.output
|
||||
|
||||
|
||||
def test_edit_note_append_creates_nonexistent_note_cli(
|
||||
app, app_config, test_project, config_manager
|
||||
):
|
||||
"""append to a non-existent note via CLI should auto-create and include fileCreated."""
|
||||
result = runner.invoke(
|
||||
cli_app,
|
||||
[
|
||||
"tool",
|
||||
"edit-note",
|
||||
"cli-tests/auto-created-note",
|
||||
"--operation",
|
||||
"append",
|
||||
"--content",
|
||||
"# Auto Created\n\nCreated via CLI append.",
|
||||
],
|
||||
)
|
||||
|
||||
assert result.exit_code == 0, result.output
|
||||
data = json.loads(result.stdout)
|
||||
assert data["fileCreated"] is True
|
||||
assert data["operation"] == "append"
|
||||
assert data["title"] is not None
|
||||
|
||||
# Verify the note is readable
|
||||
read_data = _read_note(data["permalink"])
|
||||
assert "Auto Created" in read_data["content"]
|
||||
|
||||
|
||||
def test_edit_note_json_format_contract(app, app_config, test_project, config_manager):
|
||||
"""JSON output returns metadata keys required by contract."""
|
||||
note = _write_note(
|
||||
@@ -262,16 +234,8 @@ def test_edit_note_json_format_contract(app, app_config, test_project, config_ma
|
||||
|
||||
assert result.exit_code == 0, result.output
|
||||
data = json.loads(result.stdout)
|
||||
assert set(data.keys()) == {
|
||||
"title",
|
||||
"permalink",
|
||||
"file_path",
|
||||
"operation",
|
||||
"checksum",
|
||||
"fileCreated",
|
||||
}
|
||||
assert set(data.keys()) == {"title", "permalink", "file_path", "operation", "checksum"}
|
||||
assert data["operation"] == "append"
|
||||
assert data["fileCreated"] is False
|
||||
assert data["title"] == "Edit JSON Note"
|
||||
|
||||
|
||||
|
||||
@@ -33,8 +33,9 @@ def test_project_info(app, app_config, test_project, config_manager):
|
||||
print(f"STDOUT: {result.stdout}")
|
||||
print(f"STDERR: {result.stderr}")
|
||||
assert result.exit_code == 0
|
||||
assert "Basic Memory Project Info" in result.stdout
|
||||
assert "test-project" in result.stdout
|
||||
assert "Knowledge Graph" in result.stdout
|
||||
assert "Statistics" in result.stdout
|
||||
|
||||
|
||||
def test_project_info_json(app, app_config, test_project, config_manager):
|
||||
|
||||
@@ -93,33 +93,32 @@ async def test_explicit_project_overrides_default(
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_no_config_default_falls_back_to_db(mcp_server, app, test_project):
|
||||
"""When ConfigManager has no default_project, tools fall back to the database is_default flag."""
|
||||
async def test_no_default_project_requires_project(mcp_server, app, test_project):
|
||||
"""Test that tools require project parameter when no default_project is configured."""
|
||||
|
||||
mock_config = BasicMemoryConfig(
|
||||
default_project=None, # No config default
|
||||
default_project=None, # No default
|
||||
projects={test_project.name: test_project.path},
|
||||
)
|
||||
|
||||
# test_project has is_default=True in the database, so write_note should
|
||||
# resolve to it via the API fallback in resolve_project_parameter.
|
||||
with patch.object(ConfigManager, "config", mock_config):
|
||||
async with Client(mcp_server) as client:
|
||||
result = await client.call_tool(
|
||||
"write_note",
|
||||
{
|
||||
"title": "DB Fallback Test",
|
||||
"directory": "test",
|
||||
"content": "# DB Fallback Test\n\nShould resolve to the database default project.",
|
||||
},
|
||||
with pytest.raises(Exception) as exc_info:
|
||||
await client.call_tool(
|
||||
"write_note",
|
||||
{
|
||||
"title": "Should Fail",
|
||||
"directory": "test",
|
||||
"content": "# Should Fail\n\nThis should fail because no project specified.",
|
||||
},
|
||||
)
|
||||
|
||||
error_message = str(exc_info.value)
|
||||
assert (
|
||||
"No project specified" in error_message
|
||||
or "project parameter" in error_message.lower()
|
||||
)
|
||||
|
||||
assert len(result.content) == 1
|
||||
response_text = result.content[0].text # pyright: ignore [reportAttributeAccessIssue]
|
||||
|
||||
assert f"project: {test_project.name}" in response_text
|
||||
assert "# Created note" in response_text
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_cli_constraint_overrides_default_project(
|
||||
|
||||
@@ -105,8 +105,11 @@ async def test_delete_note_by_permalink(mcp_server, app, test_project):
|
||||
},
|
||||
)
|
||||
|
||||
# Default text format returns "No results found" when empty
|
||||
assert "No results found" in search_result.content[0].text
|
||||
# Should have no results
|
||||
assert (
|
||||
'"results": []' in search_result.content[0].text
|
||||
or '"results":[]' in search_result.content[0].text
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@@ -384,8 +387,11 @@ async def test_delete_multiple_notes_sequentially(mcp_server, app, test_project)
|
||||
},
|
||||
)
|
||||
|
||||
# Default text format returns "No results found" when empty
|
||||
assert "No results found" in search_result.content[0].text
|
||||
# Should have no results
|
||||
assert (
|
||||
'"results": []' in search_result.content[0].text
|
||||
or '"results":[]' in search_result.content[0].text
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
|
||||
@@ -323,18 +323,17 @@ Current endpoints include user management."""
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_edit_note_error_handling_note_not_found(mcp_server, app, test_project):
|
||||
"""Test error handling when using find_replace on a non-existent note."""
|
||||
"""Test error handling when trying to edit a non-existent note."""
|
||||
|
||||
async with Client(mcp_server) as client:
|
||||
# find_replace on a non-existent note should still error
|
||||
# Try to edit a note that doesn't exist
|
||||
edit_result = await client.call_tool(
|
||||
"edit_note",
|
||||
{
|
||||
"project": test_project.name,
|
||||
"identifier": "Non-existent Note",
|
||||
"operation": "find_replace",
|
||||
"content": "replacement",
|
||||
"find_text": "old text",
|
||||
"operation": "append",
|
||||
"content": "Some content to add",
|
||||
},
|
||||
)
|
||||
|
||||
@@ -346,78 +345,6 @@ async def test_edit_note_error_handling_note_not_found(mcp_server, app, test_pro
|
||||
assert "search_notes(" in error_text
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_edit_note_append_creates_nonexistent_note(mcp_server, app, test_project):
|
||||
"""append to a non-existent note should auto-create it and make it readable."""
|
||||
|
||||
async with Client(mcp_server) as client:
|
||||
# Append to a note that doesn't exist yet
|
||||
edit_result = await client.call_tool(
|
||||
"edit_note",
|
||||
{
|
||||
"project": test_project.name,
|
||||
"identifier": "conversations/daily-log",
|
||||
"operation": "append",
|
||||
"content": "# Daily Log\n\nFirst entry for today.",
|
||||
},
|
||||
)
|
||||
|
||||
# Should return a "Created note" summary
|
||||
assert len(edit_result.content) == 1
|
||||
edit_text = edit_result.content[0].text
|
||||
assert "Created note (append)" in edit_text
|
||||
assert "fileCreated: true" in edit_text
|
||||
|
||||
# The note should now be readable
|
||||
read_result = await client.call_tool(
|
||||
"read_note",
|
||||
{
|
||||
"project": test_project.name,
|
||||
"identifier": "conversations/daily-log",
|
||||
},
|
||||
)
|
||||
|
||||
content = read_result.content[0].text
|
||||
assert "Daily Log" in content
|
||||
assert "First entry for today." in content
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_edit_note_prepend_creates_nonexistent_note(mcp_server, app, test_project):
|
||||
"""prepend to a non-existent note should auto-create it and make it readable."""
|
||||
|
||||
async with Client(mcp_server) as client:
|
||||
# Prepend to a note that doesn't exist yet
|
||||
edit_result = await client.call_tool(
|
||||
"edit_note",
|
||||
{
|
||||
"project": test_project.name,
|
||||
"identifier": "notes/quick-thought",
|
||||
"operation": "prepend",
|
||||
"content": "# Quick Thought\n\nSomething important.",
|
||||
},
|
||||
)
|
||||
|
||||
# Should return a "Created note" summary
|
||||
assert len(edit_result.content) == 1
|
||||
edit_text = edit_result.content[0].text
|
||||
assert "Created note (prepend)" in edit_text
|
||||
assert "fileCreated: true" in edit_text
|
||||
|
||||
# The note should now be readable
|
||||
read_result = await client.call_tool(
|
||||
"read_note",
|
||||
{
|
||||
"project": test_project.name,
|
||||
"identifier": "notes/quick-thought",
|
||||
},
|
||||
)
|
||||
|
||||
content = read_result.content[0].text
|
||||
assert "Quick Thought" in content
|
||||
assert "Something important." in content
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_edit_note_error_handling_text_not_found(mcp_server, app, test_project):
|
||||
"""Test error handling when find_text is not found in the note."""
|
||||
|
||||
@@ -362,9 +362,9 @@ async def test_search_pagination(mcp_server, app, test_project):
|
||||
)
|
||||
|
||||
result_text = search_result.content[0].text
|
||||
# Text format includes pagination info in footer
|
||||
assert "page 1" in result_text
|
||||
assert "page_size 5" in result_text
|
||||
# Should contain 5 results and pagination info
|
||||
assert '"current_page":1' in result_text
|
||||
assert '"page_size":5' in result_text
|
||||
|
||||
# Search page 2
|
||||
search_result = await client.call_tool(
|
||||
@@ -378,7 +378,7 @@ async def test_search_pagination(mcp_server, app, test_project):
|
||||
)
|
||||
|
||||
result_text = search_result.content[0].text
|
||||
assert "page 2" in result_text
|
||||
assert '"current_page":2' in result_text
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@@ -407,9 +407,8 @@ async def test_search_no_results(mcp_server, app, test_project):
|
||||
},
|
||||
)
|
||||
|
||||
# Default text format returns "No results found" when empty
|
||||
result_text = search_result.content[0].text
|
||||
assert "No results found" in result_text
|
||||
assert '"results": []' in result_text or '"results":[]' in result_text
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
|
||||
@@ -88,7 +88,7 @@ async def test_write_note_update_existing(mcp_server, app, test_project):
|
||||
|
||||
assert "# Created note" in result1.content[0].text # pyright: ignore [reportAttributeAccessIssue]
|
||||
|
||||
# Update the same note (explicit overwrite)
|
||||
# Update the same note
|
||||
result2 = await client.call_tool(
|
||||
"write_note",
|
||||
{
|
||||
@@ -97,7 +97,6 @@ async def test_write_note_update_existing(mcp_server, app, test_project):
|
||||
"directory": "test",
|
||||
"content": "# Update Test\n\nUpdated content with changes.",
|
||||
"tags": "updated,modified",
|
||||
"overwrite": True,
|
||||
},
|
||||
)
|
||||
|
||||
@@ -476,49 +475,3 @@ async def test_write_note_project_path_validation(mcp_server, app, test_project)
|
||||
# Should successfully create without path validation errors
|
||||
assert "# Created note" in response_text
|
||||
assert "not allowed" not in response_text
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_write_note_overwrite_guard_via_mcp_client(mcp_server, app, test_project):
|
||||
"""End-to-end test: overwrite guard works through the MCP Client protocol."""
|
||||
|
||||
async with Client(mcp_server) as client:
|
||||
# Create initial note
|
||||
result1 = await client.call_tool(
|
||||
"write_note",
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "MCP Guard Test",
|
||||
"directory": "guard",
|
||||
"content": "# MCP Guard Test\n\nOriginal content via MCP.",
|
||||
},
|
||||
)
|
||||
assert "# Created note" in result1.content[0].text # pyright: ignore [reportAttributeAccessIssue]
|
||||
|
||||
# Second write without overwrite should be blocked
|
||||
result2 = await client.call_tool(
|
||||
"write_note",
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "MCP Guard Test",
|
||||
"directory": "guard",
|
||||
"content": "# MCP Guard Test\n\nReplacement content via MCP.",
|
||||
},
|
||||
)
|
||||
response_text = result2.content[0].text # pyright: ignore [reportAttributeAccessIssue]
|
||||
assert "# Error: Note already exists" in response_text
|
||||
assert "edit_note" in response_text
|
||||
|
||||
# Overwrite with explicit flag should succeed
|
||||
result3 = await client.call_tool(
|
||||
"write_note",
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "MCP Guard Test",
|
||||
"directory": "guard",
|
||||
"content": "# MCP Guard Test\n\nReplacement content via MCP.",
|
||||
"overwrite": True,
|
||||
},
|
||||
)
|
||||
response_text3 = result3.content[0].text # pyright: ignore [reportAttributeAccessIssue]
|
||||
assert "# Updated note" in response_text3
|
||||
|
||||
@@ -3,7 +3,7 @@
|
||||
These tests isolate specific problems with the search pipeline:
|
||||
1. Similarity score compression — cosine distances map to a narrow similarity band
|
||||
2. Observation noise — context-free observations match too broadly
|
||||
3. Hybrid fusion behavior — how FTS and vector scores interact
|
||||
3. RRF fusion behavior — how FTS and vector scores interact
|
||||
4. Min-similarity threshold effectiveness
|
||||
"""
|
||||
|
||||
@@ -235,16 +235,17 @@ async def test_observation_noise_vs_entity(sqlite_engine_factory, tmp_path):
|
||||
print(f" {r.permalink}: {r.score:.4f}")
|
||||
|
||||
|
||||
# --- Test: Score-based fusion — vector vs hybrid comparison ---
|
||||
# --- Test: RRF fusion — vector vs hybrid comparison ---
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.semantic
|
||||
@pytest.mark.benchmark
|
||||
async def test_score_fusion_preserves_strong_vector_match(sqlite_engine_factory, tmp_path):
|
||||
async def test_rrf_fusion_preserves_strong_vector_match(sqlite_engine_factory, tmp_path):
|
||||
"""When vector gives a strong match and FTS doesn't, hybrid should still surface it.
|
||||
|
||||
Score-based fusion preserves dominant signals instead of compressing them.
|
||||
This is the core claim of issue #577 — that RRF dilutes strong vector scores.
|
||||
Let's verify with a controlled corpus.
|
||||
"""
|
||||
skip_if_needed(DIAG_COMBO)
|
||||
provider = _create_fastembed_provider()
|
||||
@@ -295,7 +296,7 @@ async def test_score_fusion_preserves_strong_vector_match(sqlite_engine_factory,
|
||||
# Auth should still be in the top 3 in hybrid mode
|
||||
assert hybrid_auth_rank <= 3, (
|
||||
f"Hybrid pushed auth from vector rank 1 to hybrid rank {hybrid_auth_rank}. "
|
||||
f"Fusion diluted strong vector match."
|
||||
f"RRF dilution confirmed."
|
||||
)
|
||||
else:
|
||||
print("\n WARNING: Auth found by vector but missing from hybrid results entirely!")
|
||||
|
||||
@@ -82,10 +82,10 @@ async def test_postgres_vector_table_setup_and_query(postgres_engine_factory, tm
|
||||
@pytest.mark.semantic
|
||||
@pytest.mark.benchmark
|
||||
async def test_postgres_hybrid_search(postgres_engine_factory, tmp_path):
|
||||
"""Exercise the hybrid (score-based fusion) code path on Postgres.
|
||||
"""Exercise the hybrid (RRF fusion) code path on Postgres.
|
||||
|
||||
This covers the full _search_hybrid path including both FTS and vector
|
||||
retrieval with score-based fusion.
|
||||
retrieval with reciprocal rank fusion.
|
||||
"""
|
||||
skip_if_needed(PG_FASTEMBED)
|
||||
if postgres_engine_factory is None:
|
||||
@@ -98,7 +98,7 @@ async def test_postgres_hybrid_search(postgres_engine_factory, tmp_path):
|
||||
|
||||
await seed_benchmark_notes(search_service, note_count=20)
|
||||
|
||||
# Hybrid search — exercises _search_hybrid score-based fusion
|
||||
# Hybrid search — exercises _search_hybrid RRF fusion
|
||||
results = await search_service.search(
|
||||
SearchQuery(
|
||||
text="database migration schema",
|
||||
|
||||
@@ -1,120 +0,0 @@
|
||||
"""Tests for v2 graph intelligence and FCM routers."""
|
||||
|
||||
import pytest
|
||||
from httpx import AsyncClient
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_graph_lineage_contract(client: AsyncClient, v2_project_url: str):
|
||||
response = await client.post(
|
||||
f"{v2_project_url}/graph/lineage",
|
||||
json={"start": "memory://specs/search"},
|
||||
)
|
||||
|
||||
assert response.status_code == 200
|
||||
data = response.json()
|
||||
assert set(["root", "paths", "generated_at"]).issubset(data.keys())
|
||||
assert data["root"]["id"] == "specs/search"
|
||||
assert isinstance(data["paths"], list)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_graph_impact_contract(client: AsyncClient, v2_project_url: str):
|
||||
response = await client.post(
|
||||
f"{v2_project_url}/graph/impact",
|
||||
json={"target": "memory://specs/search", "horizon": 2},
|
||||
)
|
||||
|
||||
assert response.status_code == 200
|
||||
data = response.json()
|
||||
assert set(["target", "affected", "summary"]).issubset(data.keys())
|
||||
assert data["summary"]["total_considered"] >= data["summary"]["total_returned"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_graph_health_contract(client: AsyncClient, v2_project_url: str):
|
||||
response = await client.get(
|
||||
f"{v2_project_url}/graph/health",
|
||||
params={"scope": "specs", "timeframe": "30d"},
|
||||
)
|
||||
|
||||
assert response.status_code == 200
|
||||
data = response.json()
|
||||
assert set(["metrics", "issues", "computed_at"]).issubset(data.keys())
|
||||
assert "orphan_rate" in data["metrics"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_graph_reindex_schedules_task(
|
||||
client: AsyncClient,
|
||||
v2_project_url: str,
|
||||
task_scheduler_spy: list[dict[str, object]],
|
||||
):
|
||||
response = await client.post(
|
||||
f"{v2_project_url}/graph/reindex",
|
||||
json={"mode": "full", "reason": "contract test"},
|
||||
)
|
||||
|
||||
assert response.status_code == 200
|
||||
data = response.json()
|
||||
assert data["status"] == "queued"
|
||||
assert data["job_id"]
|
||||
|
||||
assert task_scheduler_spy
|
||||
last = task_scheduler_spy[-1]
|
||||
assert last["task_name"] == "reindex_graph_project"
|
||||
assert last["payload"]["mode"] == "full"
|
||||
assert last["payload"]["reason"] == "contract test"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_fcm_simulate_contract(client: AsyncClient, v2_project_url: str):
|
||||
response = await client.post(
|
||||
f"{v2_project_url}/fcm/simulate",
|
||||
json={"actions": [{"node_id": "test-node", "delta": 0.2}]},
|
||||
)
|
||||
|
||||
assert response.status_code == 200
|
||||
data = response.json()
|
||||
assert set(["baseline", "projected", "deltas", "stability", "confidence"]).issubset(data.keys())
|
||||
assert data["stability"]["converged"] is True
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_fcm_rank_actions_contract(client: AsyncClient, v2_project_url: str):
|
||||
response = await client.post(
|
||||
f"{v2_project_url}/fcm/rank-actions",
|
||||
json={"goal": "reduce-regressions", "top_k": 2},
|
||||
)
|
||||
|
||||
assert response.status_code == 200
|
||||
data = response.json()
|
||||
assert set(["goal", "recommendations"]).issubset(data.keys())
|
||||
assert len(data["recommendations"]) <= 2
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_fcm_import_contract(client: AsyncClient, v2_project_url: str):
|
||||
response = await client.post(
|
||||
f"{v2_project_url}/fcm/import",
|
||||
json={"source": "/tmp/model.csv", "format": "csv_bundle_v1"},
|
||||
)
|
||||
|
||||
assert response.status_code == 200
|
||||
data = response.json()
|
||||
assert set(["import_id", "nodes_loaded", "edges_loaded", "warnings", "errors"]).issubset(
|
||||
data.keys()
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_fcm_export_contract(client: AsyncClient, v2_project_url: str):
|
||||
response = await client.post(
|
||||
f"{v2_project_url}/fcm/export",
|
||||
json={"format": "csv_bundle_v1", "selection": {"scope": "all"}},
|
||||
)
|
||||
|
||||
assert response.status_code == 200
|
||||
data = response.json()
|
||||
assert set(["export_id", "format", "files", "node_count", "edge_count"]).issubset(data.keys())
|
||||
assert len(data["files"]) == 2
|
||||
@@ -836,7 +836,9 @@ async def test_delete_directory_v2_nested_structure(client: AsyncClient, v2_proj
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_entity_response_includes_user_tracking_fields(client: AsyncClient, v2_project_url):
|
||||
async def test_entity_response_includes_user_tracking_fields(
|
||||
client: AsyncClient, v2_project_url
|
||||
):
|
||||
"""EntityResponseV2 includes created_by and last_updated_by fields (null for local)."""
|
||||
entity_data = {
|
||||
"title": "UserTrackingTest",
|
||||
|
||||
@@ -11,21 +11,6 @@ from basic_memory.schemas.project_info import ProjectItem, ProjectStatusResponse
|
||||
from basic_memory.schemas.v2 import ProjectResolveResponse
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_list_projects(client: AsyncClient, test_project: Project, v2_projects_url):
|
||||
"""Test listing projects returns default_project from the database."""
|
||||
response = await client.get(f"{v2_projects_url}/")
|
||||
|
||||
assert response.status_code == 200
|
||||
data = response.json()
|
||||
|
||||
# default_project must be populated from the is_default flag in the database
|
||||
assert data["default_project"] == test_project.name
|
||||
|
||||
project_names = [p["name"] for p in data["projects"]]
|
||||
assert test_project.name in project_names
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_get_project_by_id(client: AsyncClient, test_project: Project, v2_projects_url):
|
||||
"""Test getting a project by its external_id UUID."""
|
||||
@@ -376,10 +361,9 @@ async def test_legacy_v1_list_projects_endpoint(client: AsyncClient, test_projec
|
||||
assert response.status_code == 200
|
||||
data = response.json()
|
||||
assert "projects" in data
|
||||
assert "default_project" in data
|
||||
|
||||
# default_project must be populated, not null
|
||||
assert data["default_project"] == test_project.name
|
||||
|
||||
# Verify the test project is in the list
|
||||
project_names = [p["name"] for p in data["projects"]]
|
||||
assert test_project.name in project_names
|
||||
|
||||
|
||||
@@ -7,7 +7,6 @@ Note: EntityType uses BeforeValidator(to_snake_case) so "Person" becomes "person
|
||||
in the database. All query params must use the stored (snake_case) form.
|
||||
"""
|
||||
|
||||
from pathlib import Path
|
||||
from textwrap import dedent
|
||||
|
||||
import pytest
|
||||
@@ -15,7 +14,6 @@ from httpx import AsyncClient
|
||||
|
||||
from basic_memory.models import Project
|
||||
from basic_memory.schemas.base import Entity as EntitySchema
|
||||
from basic_memory.services.file_service import FileService
|
||||
|
||||
|
||||
# --- Helpers ---
|
||||
@@ -626,276 +624,3 @@ async def test_diff_with_schema_note(
|
||||
assert isinstance(data["new_fields"], list)
|
||||
assert isinstance(data["dropped_fields"], list)
|
||||
assert isinstance(data["cardinality_changes"], list)
|
||||
|
||||
|
||||
# --- File-based schema frontmatter tests ---
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_validate_reads_schema_from_file_not_database(
|
||||
client: AsyncClient,
|
||||
test_project: Project,
|
||||
v2_project_url: str,
|
||||
entity_service,
|
||||
search_service,
|
||||
file_service: FileService,
|
||||
):
|
||||
"""Validate uses schema frontmatter from the file, not stale database metadata.
|
||||
|
||||
Simulates the core bug from #634: user edits a schema file to change
|
||||
validation mode from 'warn' to 'strict', but the file watcher hasn't
|
||||
synced. The database still has 'warn', but validation should use 'strict'
|
||||
from the file.
|
||||
"""
|
||||
# Create schema entity — DB gets validation=warn
|
||||
schema_entity, _ = await entity_service.create_or_update_entity(
|
||||
EntitySchema(
|
||||
title="Editable Schema",
|
||||
directory="schemas",
|
||||
note_type="schema",
|
||||
entity_metadata={
|
||||
"entity": "editable_type",
|
||||
"schema": {"name": "string", "role": "string"},
|
||||
"settings": {"validation": "warn"},
|
||||
},
|
||||
content=dedent("""\
|
||||
## Observations
|
||||
- [note] Schema that will be edited on disk
|
||||
"""),
|
||||
)
|
||||
)
|
||||
await search_service.index_entity(schema_entity)
|
||||
|
||||
# Overwrite the file on disk with validation=strict
|
||||
file_path = Path(file_service.base_path) / schema_entity.file_path
|
||||
file_path.write_text(
|
||||
dedent("""\
|
||||
---
|
||||
title: Editable Schema
|
||||
permalink: schemas/editable-schema
|
||||
type: schema
|
||||
entity: editable_type
|
||||
schema:
|
||||
name: string
|
||||
role: string
|
||||
settings:
|
||||
validation: strict
|
||||
---
|
||||
|
||||
# Editable Schema
|
||||
|
||||
## Observations
|
||||
- [note] Schema that will be edited on disk
|
||||
""")
|
||||
)
|
||||
|
||||
# Create a note missing "role" — strict mode should produce errors, not warnings
|
||||
note_entity, _ = await entity_service.create_or_update_entity(
|
||||
EntitySchema(
|
||||
title="TestNote",
|
||||
directory="notes",
|
||||
note_type="editable_type",
|
||||
content=dedent("""\
|
||||
## Observations
|
||||
- [name] Test Person
|
||||
"""),
|
||||
)
|
||||
)
|
||||
await search_service.index_entity(note_entity)
|
||||
|
||||
response = await client.post(
|
||||
f"{v2_project_url}/schema/validate",
|
||||
params={"identifier": note_entity.permalink},
|
||||
)
|
||||
|
||||
assert response.status_code == 200
|
||||
data = response.json()
|
||||
assert data["total_notes"] == 1
|
||||
result = data["results"][0]
|
||||
# strict mode: missing required field is an error, not a warning
|
||||
assert result["passed"] is False
|
||||
assert any("role" in e for e in result["errors"])
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_validate_falls_back_to_db_on_incomplete_frontmatter(
|
||||
client: AsyncClient,
|
||||
test_project: Project,
|
||||
v2_project_url: str,
|
||||
entity_service,
|
||||
search_service,
|
||||
file_service: FileService,
|
||||
):
|
||||
"""Validate falls back to database metadata when file has incomplete frontmatter.
|
||||
|
||||
Simulates a mid-edit state where the user has removed the 'schema' key
|
||||
from the file. The validator should use the last-known-good metadata
|
||||
from the database rather than failing with a 500.
|
||||
"""
|
||||
schema_entity, _ = await entity_service.create_or_update_entity(
|
||||
EntitySchema(
|
||||
title="Incomplete Schema",
|
||||
directory="schemas",
|
||||
note_type="schema",
|
||||
entity_metadata={
|
||||
"entity": "incomplete_type",
|
||||
"schema": {"name": "string"},
|
||||
},
|
||||
content=dedent("""\
|
||||
## Observations
|
||||
- [note] Schema that will have incomplete frontmatter
|
||||
"""),
|
||||
)
|
||||
)
|
||||
await search_service.index_entity(schema_entity)
|
||||
|
||||
# Overwrite file with frontmatter missing the 'schema' key
|
||||
file_path = Path(file_service.base_path) / schema_entity.file_path
|
||||
file_path.write_text(
|
||||
dedent("""\
|
||||
---
|
||||
title: Incomplete Schema
|
||||
permalink: schemas/incomplete-schema
|
||||
type: schema
|
||||
entity: incomplete_type
|
||||
---
|
||||
|
||||
# Incomplete Schema
|
||||
|
||||
## Observations
|
||||
- [note] Mid-edit state
|
||||
""")
|
||||
)
|
||||
|
||||
# Create a note to validate against this schema
|
||||
note_entity, _ = await entity_service.create_or_update_entity(
|
||||
EntitySchema(
|
||||
title="FallbackNote",
|
||||
directory="notes",
|
||||
note_type="incomplete_type",
|
||||
content=dedent("""\
|
||||
## Observations
|
||||
- [name] Test Fallback
|
||||
"""),
|
||||
)
|
||||
)
|
||||
await search_service.index_entity(note_entity)
|
||||
|
||||
response = await client.post(
|
||||
f"{v2_project_url}/schema/validate",
|
||||
params={"note_type": "incomplete_type"},
|
||||
)
|
||||
|
||||
# Should not 500 — falls back to DB metadata and validates successfully
|
||||
assert response.status_code == 200
|
||||
data = response.json()
|
||||
assert data["total_notes"] == 1
|
||||
result = data["results"][0]
|
||||
assert result["passed"] is True
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_validate_falls_back_to_db_on_missing_file(
|
||||
client: AsyncClient,
|
||||
test_project: Project,
|
||||
v2_project_url: str,
|
||||
entity_service,
|
||||
search_service,
|
||||
file_service: FileService,
|
||||
):
|
||||
"""Validate falls back to database metadata when schema file is missing.
|
||||
|
||||
Simulates a race condition where the file has been deleted but the
|
||||
database still has the entity. The validator should use DB metadata
|
||||
rather than failing entirely.
|
||||
"""
|
||||
schema_entity, _ = await entity_service.create_or_update_entity(
|
||||
EntitySchema(
|
||||
title="Missing File Schema",
|
||||
directory="schemas",
|
||||
note_type="schema",
|
||||
entity_metadata={
|
||||
"entity": "missing_file_type",
|
||||
"schema": {"name": "string"},
|
||||
},
|
||||
content=dedent("""\
|
||||
## Observations
|
||||
- [note] Schema whose file will be deleted
|
||||
"""),
|
||||
)
|
||||
)
|
||||
await search_service.index_entity(schema_entity)
|
||||
|
||||
# Delete the schema file from disk
|
||||
file_path = Path(file_service.base_path) / schema_entity.file_path
|
||||
file_path.unlink()
|
||||
|
||||
# Create a note to validate
|
||||
note_entity, _ = await entity_service.create_or_update_entity(
|
||||
EntitySchema(
|
||||
title="OrphanNote",
|
||||
directory="notes",
|
||||
note_type="missing_file_type",
|
||||
content=dedent("""\
|
||||
## Observations
|
||||
- [name] Test Orphan
|
||||
"""),
|
||||
)
|
||||
)
|
||||
await search_service.index_entity(note_entity)
|
||||
|
||||
response = await client.post(
|
||||
f"{v2_project_url}/schema/validate",
|
||||
params={"note_type": "missing_file_type"},
|
||||
)
|
||||
|
||||
# Should not 500 — falls back to DB metadata and validates
|
||||
assert response.status_code == 200
|
||||
data = response.json()
|
||||
assert data["total_notes"] == 1
|
||||
result = data["results"][0]
|
||||
assert result["passed"] is True
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_diff_falls_back_to_db_on_missing_file(
|
||||
client: AsyncClient,
|
||||
test_project: Project,
|
||||
v2_project_url: str,
|
||||
entity_service,
|
||||
search_service,
|
||||
file_service: FileService,
|
||||
):
|
||||
"""Diff endpoint falls back to DB metadata when schema file is missing."""
|
||||
schema_entity, _ = await entity_service.create_or_update_entity(
|
||||
EntitySchema(
|
||||
title="Diff Missing Schema",
|
||||
directory="schemas",
|
||||
note_type="schema",
|
||||
entity_metadata={
|
||||
"entity": "diff_missing_type",
|
||||
"schema": {"name": "string", "role": "string"},
|
||||
},
|
||||
content=dedent("""\
|
||||
## Observations
|
||||
- [note] Schema for diff fallback test
|
||||
"""),
|
||||
)
|
||||
)
|
||||
await search_service.index_entity(schema_entity)
|
||||
|
||||
# Delete the schema file
|
||||
file_path = Path(file_service.base_path) / schema_entity.file_path
|
||||
file_path.unlink()
|
||||
|
||||
# Create person entities
|
||||
await create_person_entities(entity_service, search_service)
|
||||
|
||||
response = await client.get(
|
||||
f"{v2_project_url}/schema/diff/diff_missing_type",
|
||||
)
|
||||
|
||||
# Should not 500 — falls back to DB metadata for schema resolution
|
||||
assert response.status_code == 200
|
||||
data = response.json()
|
||||
assert data["note_type"] == "diff_missing_type"
|
||||
|
||||
@@ -76,7 +76,7 @@ class TestTrack:
|
||||
|
||||
captured_target = None
|
||||
|
||||
def fake_thread(target):
|
||||
def fake_thread(target, daemon):
|
||||
nonlocal captured_target
|
||||
captured_target = target
|
||||
mock = MagicMock()
|
||||
@@ -103,7 +103,7 @@ class TestTrack:
|
||||
with patch("basic_memory.cli.analytics.urllib.request.urlopen", fake_urlopen):
|
||||
with patch("basic_memory.cli.analytics.threading.Thread") as mock_thread:
|
||||
# Capture the target function and call it directly
|
||||
def run_target(target):
|
||||
def run_target(target, daemon):
|
||||
target() # Execute synchronously
|
||||
return MagicMock()
|
||||
|
||||
@@ -113,7 +113,6 @@ class TestTrack:
|
||||
assert captured_request is not None
|
||||
assert captured_request.full_url == "https://analytics.example.com/api/send"
|
||||
body = json.loads(captured_request.data)
|
||||
assert body["type"] == "event"
|
||||
assert body["payload"]["name"] == "cli-cloud-login-started"
|
||||
assert body["payload"]["website"] == "test-site-id"
|
||||
assert body["payload"]["hostname"] == "cli.basicmemory.com"
|
||||
@@ -130,7 +129,7 @@ class TestTrack:
|
||||
with patch("basic_memory.cli.analytics.urllib.request.urlopen", fake_urlopen):
|
||||
with patch("basic_memory.cli.analytics.threading.Thread") as mock_thread:
|
||||
|
||||
def run_target(target):
|
||||
def run_target(target, daemon):
|
||||
target() # Should not raise
|
||||
return MagicMock()
|
||||
|
||||
|
||||
@@ -1,184 +0,0 @@
|
||||
"""Tests for graph/FCM CLI tool JSON passthrough commands."""
|
||||
|
||||
import json
|
||||
from unittest.mock import AsyncMock, patch
|
||||
|
||||
from typer.testing import CliRunner
|
||||
|
||||
from basic_memory.cli.main import app as cli_app
|
||||
|
||||
runner = CliRunner()
|
||||
|
||||
|
||||
@patch(
|
||||
"basic_memory.cli.commands.tool.mcp_graph_lineage",
|
||||
new_callable=AsyncMock,
|
||||
return_value={
|
||||
"root": {"id": "specs/search"},
|
||||
"paths": [],
|
||||
"generated_at": "2026-03-05T00:00:00Z",
|
||||
},
|
||||
)
|
||||
def test_graph_lineage_json_output(mock_tool):
|
||||
result = runner.invoke(cli_app, ["tool", "graph-lineage", "memory://specs/search"])
|
||||
assert result.exit_code == 0, f"CLI failed: {result.output}"
|
||||
data = json.loads(result.output)
|
||||
assert data["root"]["id"] == "specs/search"
|
||||
assert mock_tool.call_args.kwargs["output_format"] == "json"
|
||||
|
||||
|
||||
@patch(
|
||||
"basic_memory.cli.commands.tool.mcp_graph_impact",
|
||||
new_callable=AsyncMock,
|
||||
return_value={
|
||||
"target": {"id": "specs/search", "title": "specs/search"},
|
||||
"affected": [],
|
||||
"summary": {"total_considered": 0, "total_returned": 0},
|
||||
},
|
||||
)
|
||||
def test_graph_impact_passthrough(mock_tool):
|
||||
result = runner.invoke(
|
||||
cli_app,
|
||||
[
|
||||
"tool",
|
||||
"graph-impact",
|
||||
"memory://specs/search",
|
||||
"--horizon",
|
||||
"3",
|
||||
"--relation-filter",
|
||||
"depends_on",
|
||||
],
|
||||
)
|
||||
assert result.exit_code == 0, f"CLI failed: {result.output}"
|
||||
assert mock_tool.call_args.kwargs["horizon"] == 3
|
||||
assert mock_tool.call_args.kwargs["relation_filters"] == ["depends_on"]
|
||||
assert mock_tool.call_args.kwargs["output_format"] == "json"
|
||||
|
||||
|
||||
@patch(
|
||||
"basic_memory.cli.commands.tool.mcp_graph_health",
|
||||
new_callable=AsyncMock,
|
||||
return_value={
|
||||
"metrics": {
|
||||
"orphan_rate": 0.0,
|
||||
"stale_central_nodes": 0,
|
||||
"overloaded_hubs": 0,
|
||||
"contradiction_candidates": 0,
|
||||
},
|
||||
"issues": [],
|
||||
"computed_at": "2026-03-05T00:00:00Z",
|
||||
},
|
||||
)
|
||||
def test_graph_health_json_output(mock_tool):
|
||||
result = runner.invoke(
|
||||
cli_app,
|
||||
["tool", "graph-health", "--scope", "specs", "--timeframe", "30d"],
|
||||
)
|
||||
assert result.exit_code == 0, f"CLI failed: {result.output}"
|
||||
data = json.loads(result.output)
|
||||
assert "metrics" in data
|
||||
assert mock_tool.call_args.kwargs["scope"] == "specs"
|
||||
assert mock_tool.call_args.kwargs["timeframe"] == "30d"
|
||||
|
||||
|
||||
@patch(
|
||||
"basic_memory.cli.commands.tool.mcp_fcm_simulate",
|
||||
new_callable=AsyncMock,
|
||||
return_value={
|
||||
"baseline": [],
|
||||
"projected": [],
|
||||
"deltas": [],
|
||||
"stability": {"converged": True, "iterations_used": 1, "residual": 0.0},
|
||||
"confidence": 0.5,
|
||||
},
|
||||
)
|
||||
def test_fcm_simulate_json_output(mock_tool):
|
||||
result = runner.invoke(
|
||||
cli_app,
|
||||
[
|
||||
"tool",
|
||||
"fcm-simulate",
|
||||
"--actions-json",
|
||||
'[{"node_id":"n1","delta":0.2}]',
|
||||
"--scenario-json",
|
||||
'{"steps":8}',
|
||||
],
|
||||
)
|
||||
assert result.exit_code == 0, f"CLI failed: {result.output}"
|
||||
assert mock_tool.call_args.kwargs["actions"] == [{"node_id": "n1", "delta": 0.2}]
|
||||
assert mock_tool.call_args.kwargs["scenario"] == {"steps": 8}
|
||||
|
||||
|
||||
def test_fcm_simulate_invalid_actions_json():
|
||||
result = runner.invoke(
|
||||
cli_app,
|
||||
["tool", "fcm-simulate", "--actions-json", '{"node_id":"n1","delta":0.2}'],
|
||||
)
|
||||
assert result.exit_code == 1
|
||||
assert "expected a JSON array" in result.output
|
||||
|
||||
|
||||
@patch(
|
||||
"basic_memory.cli.commands.tool.mcp_fcm_rank_actions",
|
||||
new_callable=AsyncMock,
|
||||
return_value={"goal": {"node_id": "g1", "label": "g1"}, "recommendations": []},
|
||||
)
|
||||
def test_fcm_rank_actions_passthrough(mock_tool):
|
||||
result = runner.invoke(
|
||||
cli_app,
|
||||
["tool", "fcm-rank-actions", "g1", "--constraints-json", '{"required_tags":["risk"]}'],
|
||||
)
|
||||
assert result.exit_code == 0, f"CLI failed: {result.output}"
|
||||
assert mock_tool.call_args.kwargs["constraints"] == {"required_tags": ["risk"]}
|
||||
assert mock_tool.call_args.kwargs["output_format"] == "json"
|
||||
|
||||
|
||||
@patch(
|
||||
"basic_memory.cli.commands.tool.mcp_fcm_import_model",
|
||||
new_callable=AsyncMock,
|
||||
return_value={
|
||||
"import_id": "imp-1",
|
||||
"nodes_loaded": 0,
|
||||
"edges_loaded": 0,
|
||||
"warnings": [],
|
||||
"errors": [],
|
||||
},
|
||||
)
|
||||
def test_fcm_import_model_json_output(mock_tool):
|
||||
result = runner.invoke(
|
||||
cli_app,
|
||||
["tool", "fcm-import-model", "/tmp/model.csv", "--format", "csv_bundle_v1"],
|
||||
)
|
||||
assert result.exit_code == 0, f"CLI failed: {result.output}"
|
||||
data = json.loads(result.output)
|
||||
assert data["import_id"] == "imp-1"
|
||||
assert mock_tool.call_args.kwargs["output_format"] == "json"
|
||||
|
||||
|
||||
@patch(
|
||||
"basic_memory.cli.commands.tool.mcp_fcm_export_model",
|
||||
new_callable=AsyncMock,
|
||||
return_value={
|
||||
"export_id": "exp-1",
|
||||
"format": "csv_bundle_v1",
|
||||
"files": [],
|
||||
"node_count": 0,
|
||||
"edge_count": 0,
|
||||
},
|
||||
)
|
||||
def test_fcm_export_model_json_output(mock_tool):
|
||||
result = runner.invoke(
|
||||
cli_app,
|
||||
[
|
||||
"tool",
|
||||
"fcm-export-model",
|
||||
"--format",
|
||||
"csv_bundle_v1",
|
||||
"--selection-json",
|
||||
'{"scope":"all"}',
|
||||
],
|
||||
)
|
||||
assert result.exit_code == 0, f"CLI failed: {result.output}"
|
||||
data = json.loads(result.output)
|
||||
assert data["export_id"] == "exp-1"
|
||||
assert mock_tool.call_args.kwargs["selection"] == {"scope": "all"}
|
||||
@@ -8,7 +8,6 @@ from typer.testing import CliRunner
|
||||
from basic_memory.cli.app import app
|
||||
import basic_memory
|
||||
from basic_memory.cli.promo import (
|
||||
_is_interactive_session,
|
||||
maybe_show_cloud_promo,
|
||||
maybe_show_init_line,
|
||||
)
|
||||
@@ -21,6 +20,43 @@ def _capture_console() -> tuple[Console, StringIO]:
|
||||
return Console(file=buf, force_terminal=True), buf
|
||||
|
||||
|
||||
# --- _is_interactive_session tests ---
|
||||
|
||||
|
||||
def test_is_interactive_session_returns_false_on_closed_stdin(monkeypatch):
|
||||
"""isatty() raises ValueError when stdio is closed (e.g., MCP shutdown)."""
|
||||
import sys
|
||||
|
||||
from basic_memory.cli.promo import _is_interactive_session
|
||||
|
||||
class _ClosedStdin:
|
||||
def isatty(self):
|
||||
raise ValueError("I/O operation on closed file")
|
||||
|
||||
monkeypatch.setattr(sys, "stdin", _ClosedStdin())
|
||||
assert _is_interactive_session() is False
|
||||
|
||||
|
||||
def test_is_interactive_session_returns_false_on_closed_stdout(monkeypatch):
|
||||
"""isatty() raises ValueError on closed stdout (e.g., MCP shutdown)."""
|
||||
import sys
|
||||
|
||||
from basic_memory.cli.promo import _is_interactive_session
|
||||
|
||||
class _ClosedStdout:
|
||||
def isatty(self):
|
||||
raise ValueError("I/O operation on closed file")
|
||||
|
||||
# stdin reports interactive, but stdout is closed
|
||||
class _InteractiveStdin:
|
||||
def isatty(self):
|
||||
return True
|
||||
|
||||
monkeypatch.setattr(sys, "stdin", _InteractiveStdin())
|
||||
monkeypatch.setattr(sys, "stdout", _ClosedStdout())
|
||||
assert _is_interactive_session() is False
|
||||
|
||||
|
||||
# --- maybe_show_init_line tests ---
|
||||
|
||||
|
||||
@@ -295,17 +331,3 @@ def test_cloud_promo_command_on_clears_opt_out(monkeypatch):
|
||||
assert "Cloud promo messages enabled" in result.stdout
|
||||
assert len(instances) == 1
|
||||
assert instances[0].saved_config.cloud_promo_opt_out is False
|
||||
|
||||
|
||||
# --- _is_interactive_session tests ---
|
||||
|
||||
|
||||
def test_is_interactive_session_returns_false_when_streams_closed(monkeypatch):
|
||||
"""isatty() raises ValueError on closed file descriptors (e.g., MCP shutdown)."""
|
||||
|
||||
class ClosedStream:
|
||||
def isatty(self):
|
||||
raise ValueError("I/O operation on closed file")
|
||||
|
||||
monkeypatch.setattr("sys.stdin", ClosedStream())
|
||||
assert _is_interactive_session() is False
|
||||
|
||||
@@ -1,146 +0,0 @@
|
||||
"""Tests for cloud status command."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import time
|
||||
|
||||
import httpx
|
||||
import pytest
|
||||
from typer.testing import CliRunner
|
||||
|
||||
from basic_memory.cli.app import app
|
||||
from basic_memory.cli.commands.cloud.api_client import CloudAPIError
|
||||
|
||||
|
||||
# --- status command integration tests ---
|
||||
|
||||
|
||||
class _FakeTokens:
|
||||
"""Provides canned token data for CLIAuth stubs."""
|
||||
|
||||
@classmethod
|
||||
def valid(cls) -> dict:
|
||||
return {
|
||||
"access_token": "fake-access-token",
|
||||
"refresh_token": "rt_test",
|
||||
"expires_at": int(time.time()) + 3600,
|
||||
}
|
||||
|
||||
@classmethod
|
||||
def expired(cls) -> dict:
|
||||
return {
|
||||
"access_token": "fake-access-token",
|
||||
"refresh_token": "rt_test",
|
||||
"expires_at": int(time.time()) - 3600,
|
||||
}
|
||||
|
||||
|
||||
def _patch_status_deps(monkeypatch, *, tokens=None, api_side_effect=None):
|
||||
"""Patch ConfigManager and CLIAuth for the status command."""
|
||||
|
||||
class FakeConfig:
|
||||
cloud_client_id = "cid"
|
||||
cloud_domain = "https://auth.example.com"
|
||||
cloud_host = "https://cloud.example.com"
|
||||
cloud_api_key = "bmc_test123"
|
||||
|
||||
class FakeConfigManager:
|
||||
config = FakeConfig()
|
||||
|
||||
def load_config(self):
|
||||
return self.config
|
||||
|
||||
class FakeAuth:
|
||||
def __init__(self, **_kwargs):
|
||||
pass
|
||||
|
||||
def load_tokens(self):
|
||||
return tokens
|
||||
|
||||
def is_token_valid(self, t):
|
||||
return t.get("expires_at", 0) > time.time()
|
||||
|
||||
monkeypatch.setattr(
|
||||
"basic_memory.cli.commands.cloud.core_commands.ConfigManager", FakeConfigManager
|
||||
)
|
||||
monkeypatch.setattr("basic_memory.cli.commands.cloud.core_commands.CLIAuth", FakeAuth)
|
||||
monkeypatch.setattr(
|
||||
"basic_memory.cli.commands.cloud.core_commands.get_cloud_config",
|
||||
lambda: ("cid", "domain", "https://cloud.example.com"),
|
||||
)
|
||||
|
||||
if api_side_effect is None:
|
||||
# Default: cloud is reachable
|
||||
async def _ok(*_a, **_kw):
|
||||
return httpx.Response(200, json={"status": "ok"})
|
||||
|
||||
api_side_effect = _ok
|
||||
|
||||
monkeypatch.setattr(
|
||||
"basic_memory.cli.commands.cloud.core_commands.make_api_request", api_side_effect
|
||||
)
|
||||
|
||||
|
||||
class TestStatusCommand:
|
||||
def test_status_connected(self, monkeypatch):
|
||||
_patch_status_deps(monkeypatch, tokens=_FakeTokens.valid())
|
||||
runner = CliRunner()
|
||||
result = runner.invoke(app, ["cloud", "status"])
|
||||
|
||||
assert result.exit_code == 0
|
||||
assert "Cloud Status" in result.stdout
|
||||
assert "cloud.example.com" in result.stdout
|
||||
assert "token valid" in result.stdout
|
||||
assert "Cloud connected" in result.stdout
|
||||
|
||||
def test_status_expired_token(self, monkeypatch):
|
||||
_patch_status_deps(monkeypatch, tokens=_FakeTokens.expired())
|
||||
runner = CliRunner()
|
||||
result = runner.invoke(app, ["cloud", "status"])
|
||||
|
||||
assert result.exit_code == 0
|
||||
assert "token expired" in result.stdout
|
||||
|
||||
def test_status_no_credentials(self, monkeypatch):
|
||||
_patch_status_deps(monkeypatch, tokens=None)
|
||||
|
||||
# Also clear the API key so there are no credentials at all
|
||||
class FakeConfig:
|
||||
cloud_client_id = "cid"
|
||||
cloud_domain = "https://auth.example.com"
|
||||
cloud_host = "https://cloud.example.com"
|
||||
cloud_api_key = ""
|
||||
|
||||
class FakeConfigManager:
|
||||
config = FakeConfig()
|
||||
|
||||
def load_config(self):
|
||||
return self.config
|
||||
|
||||
monkeypatch.setattr(
|
||||
"basic_memory.cli.commands.cloud.core_commands.ConfigManager", FakeConfigManager
|
||||
)
|
||||
|
||||
runner = CliRunner()
|
||||
result = runner.invoke(app, ["cloud", "status"])
|
||||
|
||||
assert result.exit_code == 0
|
||||
assert "No cloud credentials found" in result.stdout
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"exc",
|
||||
[
|
||||
CloudAPIError("connection refused"),
|
||||
Exception("network timeout"),
|
||||
],
|
||||
)
|
||||
def test_status_cloud_not_connected(self, monkeypatch, exc):
|
||||
async def _fail(*_a, **_kw):
|
||||
raise exc
|
||||
|
||||
_patch_status_deps(monkeypatch, tokens=_FakeTokens.valid(), api_side_effect=_fail)
|
||||
runner = CliRunner()
|
||||
result = runner.invoke(app, ["cloud", "status"])
|
||||
|
||||
assert result.exit_code == 0
|
||||
assert "Cloud not connected" in result.stdout
|
||||
@@ -1,418 +0,0 @@
|
||||
"""Tests for --json output across CLI commands.
|
||||
|
||||
Each test verifies:
|
||||
- Exit code 0 (or 1 for strict mode)
|
||||
- Output is valid json.loads()-able
|
||||
- Expected keys present in the parsed data
|
||||
"""
|
||||
|
||||
import json
|
||||
from contextlib import asynccontextmanager
|
||||
from datetime import datetime
|
||||
from unittest.mock import AsyncMock, MagicMock, patch
|
||||
|
||||
import pytest
|
||||
from typer.testing import CliRunner
|
||||
|
||||
from basic_memory.cli.main import app as cli_app
|
||||
from basic_memory.mcp.clients.project import ProjectClient
|
||||
from basic_memory.schemas.project_info import ProjectList
|
||||
from basic_memory.schemas.sync_report import SyncReportResponse
|
||||
|
||||
# Importing registers subcommands on the shared app instance.
|
||||
import basic_memory.cli.commands.project as project_cmd # noqa: F401
|
||||
|
||||
runner = CliRunner()
|
||||
|
||||
|
||||
def _parse_json_output(output: str) -> dict:
|
||||
"""Extract and parse the JSON object from CLI output.
|
||||
|
||||
The CliRunner may capture log lines before the JSON payload.
|
||||
We find the first '{' and parse from there.
|
||||
"""
|
||||
start = output.index("{")
|
||||
return json.loads(output[start:])
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Shared mock helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _mock_config_manager():
|
||||
"""Create a mock ConfigManager that avoids reading real config."""
|
||||
mock_cm = MagicMock()
|
||||
mock_cm.config = MagicMock()
|
||||
mock_cm.default_project = "test-project"
|
||||
mock_cm.get_project.return_value = ("test-project", "/tmp/test")
|
||||
return mock_cm
|
||||
|
||||
|
||||
SYNC_REPORT_WITH_CHANGES = SyncReportResponse(
|
||||
new={"notes/new-file.md"},
|
||||
modified={"notes/existing.md"},
|
||||
deleted={"notes/old.md"},
|
||||
moves={"notes/moved-from.md": "notes/moved-to.md"},
|
||||
checksums={"notes/new-file.md": "abc12345", "notes/existing.md": "def67890"},
|
||||
skipped_files=[],
|
||||
total=4,
|
||||
)
|
||||
|
||||
SYNC_REPORT_EMPTY = SyncReportResponse(
|
||||
new=set(),
|
||||
modified=set(),
|
||||
deleted=set(),
|
||||
moves={},
|
||||
checksums={},
|
||||
skipped_files=[],
|
||||
total=0,
|
||||
)
|
||||
|
||||
SYNC_REPORT_WITH_SKIPPED = SyncReportResponse(
|
||||
new=set(),
|
||||
modified=set(),
|
||||
deleted=set(),
|
||||
moves={},
|
||||
checksums={},
|
||||
skipped_files=[
|
||||
{
|
||||
"path": "bad/file.md",
|
||||
"reason": "parse error",
|
||||
"failure_count": 3,
|
||||
"first_failed": datetime(2025, 6, 15, 12, 0, 0),
|
||||
}
|
||||
],
|
||||
total=0,
|
||||
)
|
||||
|
||||
VALIDATE_REPORT = {
|
||||
"note_type": "person",
|
||||
"total_notes": 2,
|
||||
"total_entities": 2,
|
||||
"valid_count": 1,
|
||||
"warning_count": 1,
|
||||
"error_count": 1,
|
||||
"results": [
|
||||
{
|
||||
"note_identifier": "people/alice",
|
||||
"schema_entity": "person",
|
||||
"passed": True,
|
||||
"warnings": [],
|
||||
"errors": [],
|
||||
},
|
||||
{
|
||||
"note_identifier": "people/bob",
|
||||
"schema_entity": "person",
|
||||
"passed": False,
|
||||
"warnings": ["Missing optional field: role"],
|
||||
"errors": ["Missing required field: name"],
|
||||
},
|
||||
],
|
||||
}
|
||||
|
||||
INFER_REPORT = {
|
||||
"note_type": "person",
|
||||
"notes_analyzed": 5,
|
||||
"field_frequencies": [
|
||||
{"name": "name", "source": "observation", "count": 5, "total": 5, "percentage": 1.0},
|
||||
{"name": "role", "source": "observation", "count": 3, "total": 5, "percentage": 0.6},
|
||||
],
|
||||
"suggested_schema": {"name": "string, full name", "role?": "string, job title"},
|
||||
"suggested_required": ["name"],
|
||||
"suggested_optional": ["role"],
|
||||
"excluded": [],
|
||||
}
|
||||
|
||||
DIFF_REPORT_WITH_DRIFT = {
|
||||
"note_type": "person",
|
||||
"schema_found": True,
|
||||
"new_fields": [
|
||||
{"name": "email", "source": "observation", "count": 3, "total": 5, "percentage": 0.6}
|
||||
],
|
||||
"dropped_fields": [
|
||||
{"name": "phone", "source": "observation", "count": 0, "total": 5, "percentage": 0.0}
|
||||
],
|
||||
"cardinality_changes": ["role: single -> array"],
|
||||
}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Status --json
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
_MOCK_PROJECT_ITEM = MagicMock()
|
||||
_MOCK_PROJECT_ITEM.name = "test-project"
|
||||
_MOCK_PROJECT_ITEM.external_id = "11111111-1111-1111-1111-111111111111"
|
||||
|
||||
|
||||
@patch("basic_memory.cli.commands.status.ConfigManager")
|
||||
@patch("basic_memory.cli.commands.status.get_active_project", new_callable=AsyncMock)
|
||||
@patch("basic_memory.cli.commands.status.get_client")
|
||||
def test_status_json_outputs_sync_report(mock_get_client, mock_get_active, mock_config_cls):
|
||||
"""bm status --json outputs a valid JSON sync report with changes."""
|
||||
mock_config_cls.return_value = _mock_config_manager()
|
||||
mock_get_active.return_value = _MOCK_PROJECT_ITEM
|
||||
|
||||
mock_project_client = AsyncMock()
|
||||
mock_project_client.get_status.return_value = SYNC_REPORT_WITH_CHANGES
|
||||
|
||||
@asynccontextmanager
|
||||
async def fake_get_client(project_name=None):
|
||||
yield MagicMock()
|
||||
|
||||
mock_get_client.side_effect = fake_get_client
|
||||
|
||||
with patch.object(ProjectClient, "get_status", mock_project_client.get_status):
|
||||
result = runner.invoke(cli_app, ["status", "--json"])
|
||||
|
||||
assert result.exit_code == 0, f"CLI failed: {result.output}"
|
||||
data = _parse_json_output(result.output)
|
||||
assert data["total"] == 4
|
||||
assert "new" in data
|
||||
assert "modified" in data
|
||||
assert "deleted" in data
|
||||
assert "moves" in data
|
||||
|
||||
|
||||
@patch("basic_memory.cli.commands.status.ConfigManager")
|
||||
@patch("basic_memory.cli.commands.status.get_active_project", new_callable=AsyncMock)
|
||||
@patch("basic_memory.cli.commands.status.get_client")
|
||||
def test_status_json_no_changes(mock_get_client, mock_get_active, mock_config_cls):
|
||||
"""bm status --json with empty report outputs total: 0."""
|
||||
mock_config_cls.return_value = _mock_config_manager()
|
||||
mock_get_active.return_value = _MOCK_PROJECT_ITEM
|
||||
|
||||
mock_project_client = AsyncMock()
|
||||
mock_project_client.get_status.return_value = SYNC_REPORT_EMPTY
|
||||
|
||||
@asynccontextmanager
|
||||
async def fake_get_client(project_name=None):
|
||||
yield MagicMock()
|
||||
|
||||
mock_get_client.side_effect = fake_get_client
|
||||
|
||||
with patch.object(ProjectClient, "get_status", mock_project_client.get_status):
|
||||
result = runner.invoke(cli_app, ["status", "--json"])
|
||||
|
||||
assert result.exit_code == 0, f"CLI failed: {result.output}"
|
||||
data = _parse_json_output(result.output)
|
||||
assert data["total"] == 0
|
||||
assert data["new"] == []
|
||||
assert data["modified"] == []
|
||||
|
||||
|
||||
@patch("basic_memory.cli.commands.status.ConfigManager")
|
||||
@patch("basic_memory.cli.commands.status.get_active_project", new_callable=AsyncMock)
|
||||
@patch("basic_memory.cli.commands.status.get_client")
|
||||
def test_status_json_with_skipped_files(mock_get_client, mock_get_active, mock_config_cls):
|
||||
"""bm status --json serializes skipped_files with datetime fields."""
|
||||
mock_config_cls.return_value = _mock_config_manager()
|
||||
mock_get_active.return_value = _MOCK_PROJECT_ITEM
|
||||
|
||||
mock_project_client = AsyncMock()
|
||||
mock_project_client.get_status.return_value = SYNC_REPORT_WITH_SKIPPED
|
||||
|
||||
@asynccontextmanager
|
||||
async def fake_get_client(project_name=None):
|
||||
yield MagicMock()
|
||||
|
||||
mock_get_client.side_effect = fake_get_client
|
||||
|
||||
with patch.object(ProjectClient, "get_status", mock_project_client.get_status):
|
||||
result = runner.invoke(cli_app, ["status", "--json"])
|
||||
|
||||
assert result.exit_code == 0, f"CLI failed: {result.output}"
|
||||
data = _parse_json_output(result.output)
|
||||
assert len(data["skipped_files"]) == 1
|
||||
assert data["skipped_files"][0]["path"] == "bad/file.md"
|
||||
# datetime should be serialized as ISO string via mode="json"
|
||||
assert "2025-06-15" in data["skipped_files"][0]["first_failed"]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Schema validate --json
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@patch("basic_memory.cli.commands.schema.ConfigManager")
|
||||
@patch(
|
||||
"basic_memory.cli.commands.schema.mcp_schema_validate",
|
||||
new_callable=AsyncMock,
|
||||
return_value=VALIDATE_REPORT,
|
||||
)
|
||||
def test_schema_validate_json(mock_mcp, mock_config_cls):
|
||||
"""bm schema validate person --json outputs the validation report as JSON."""
|
||||
mock_config_cls.return_value = _mock_config_manager()
|
||||
|
||||
result = runner.invoke(cli_app, ["schema", "validate", "person", "--json"])
|
||||
|
||||
assert result.exit_code == 0, f"CLI failed: {result.output}"
|
||||
data = _parse_json_output(result.output)
|
||||
assert data["note_type"] == "person"
|
||||
assert data["total_notes"] == 2
|
||||
assert len(data["results"]) == 2
|
||||
|
||||
|
||||
@patch("basic_memory.cli.commands.schema.ConfigManager")
|
||||
@patch(
|
||||
"basic_memory.cli.commands.schema.mcp_schema_validate",
|
||||
new_callable=AsyncMock,
|
||||
return_value={"error": "No schema found for type 'person'"},
|
||||
)
|
||||
def test_schema_validate_json_error(mock_mcp, mock_config_cls):
|
||||
"""bm schema validate --json with error dict outputs the error as JSON."""
|
||||
mock_config_cls.return_value = _mock_config_manager()
|
||||
|
||||
result = runner.invoke(cli_app, ["schema", "validate", "person", "--json"])
|
||||
|
||||
assert result.exit_code == 0, f"CLI failed: {result.output}"
|
||||
data = _parse_json_output(result.output)
|
||||
assert "error" in data
|
||||
|
||||
|
||||
@patch("basic_memory.cli.commands.schema.ConfigManager")
|
||||
@patch(
|
||||
"basic_memory.cli.commands.schema.mcp_schema_validate",
|
||||
new_callable=AsyncMock,
|
||||
return_value=VALIDATE_REPORT,
|
||||
)
|
||||
def test_schema_validate_json_strict_exit(mock_mcp, mock_config_cls):
|
||||
"""bm schema validate --json --strict exits 1 when errors present."""
|
||||
mock_config_cls.return_value = _mock_config_manager()
|
||||
|
||||
result = runner.invoke(cli_app, ["schema", "validate", "person", "--json", "--strict"])
|
||||
|
||||
assert result.exit_code == 1
|
||||
# JSON should still be valid in stdout
|
||||
data = _parse_json_output(result.output)
|
||||
assert data["error_count"] == 1
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Schema infer --json
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@patch("basic_memory.cli.commands.schema.ConfigManager")
|
||||
@patch(
|
||||
"basic_memory.cli.commands.schema.mcp_schema_infer",
|
||||
new_callable=AsyncMock,
|
||||
return_value=INFER_REPORT,
|
||||
)
|
||||
def test_schema_infer_json(mock_mcp, mock_config_cls):
|
||||
"""bm schema infer person --json outputs the inference report as JSON."""
|
||||
mock_config_cls.return_value = _mock_config_manager()
|
||||
|
||||
result = runner.invoke(cli_app, ["schema", "infer", "person", "--json"])
|
||||
|
||||
assert result.exit_code == 0, f"CLI failed: {result.output}"
|
||||
data = _parse_json_output(result.output)
|
||||
assert data["note_type"] == "person"
|
||||
assert data["notes_analyzed"] == 5
|
||||
assert "suggested_schema" in data
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Schema diff --json
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@patch("basic_memory.cli.commands.schema.ConfigManager")
|
||||
@patch(
|
||||
"basic_memory.cli.commands.schema.mcp_schema_diff",
|
||||
new_callable=AsyncMock,
|
||||
return_value=DIFF_REPORT_WITH_DRIFT,
|
||||
)
|
||||
def test_schema_diff_json(mock_mcp, mock_config_cls):
|
||||
"""bm schema diff person --json outputs the drift report as JSON."""
|
||||
mock_config_cls.return_value = _mock_config_manager()
|
||||
|
||||
result = runner.invoke(cli_app, ["schema", "diff", "person", "--json"])
|
||||
|
||||
assert result.exit_code == 0, f"CLI failed: {result.output}"
|
||||
data = _parse_json_output(result.output)
|
||||
assert data["note_type"] == "person"
|
||||
assert len(data["new_fields"]) == 1
|
||||
assert len(data["dropped_fields"]) == 1
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Project list --json
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def write_config(tmp_path, monkeypatch):
|
||||
"""Write config.json under a temporary HOME and return the file path."""
|
||||
|
||||
def _write(config_data: dict):
|
||||
from basic_memory import config as config_module
|
||||
|
||||
config_module._CONFIG_CACHE = None
|
||||
|
||||
config_dir = tmp_path / ".basic-memory"
|
||||
config_dir.mkdir(parents=True, exist_ok=True)
|
||||
config_file = config_dir / "config.json"
|
||||
config_file.write_text(json.dumps(config_data, indent=2))
|
||||
monkeypatch.setenv("HOME", str(tmp_path))
|
||||
return config_file
|
||||
|
||||
return _write
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def mock_client(monkeypatch):
|
||||
"""Mock get_client with a no-op async context manager."""
|
||||
|
||||
@asynccontextmanager
|
||||
async def fake_get_client(workspace=None):
|
||||
yield object()
|
||||
|
||||
monkeypatch.setattr(project_cmd, "get_client", fake_get_client)
|
||||
|
||||
|
||||
def test_project_list_json_outputs_projects(write_config, mock_client, tmp_path, monkeypatch):
|
||||
"""project list --json --local outputs structured JSON with project data."""
|
||||
alpha_local = (tmp_path / "alpha-local").as_posix()
|
||||
|
||||
write_config(
|
||||
{
|
||||
"env": "dev",
|
||||
"projects": {
|
||||
"alpha": {"path": alpha_local, "mode": "local"},
|
||||
},
|
||||
"default_project": "alpha",
|
||||
}
|
||||
)
|
||||
|
||||
local_payload = {
|
||||
"projects": [
|
||||
{
|
||||
"id": 1,
|
||||
"external_id": "11111111-1111-1111-1111-111111111111",
|
||||
"name": "alpha",
|
||||
"path": alpha_local,
|
||||
"is_default": True,
|
||||
}
|
||||
],
|
||||
"default_project": "alpha",
|
||||
}
|
||||
|
||||
async def fake_list_projects(self):
|
||||
return ProjectList.model_validate(local_payload)
|
||||
|
||||
monkeypatch.setattr(ProjectClient, "list_projects", fake_list_projects)
|
||||
|
||||
result = runner.invoke(cli_app, ["project", "list", "--json", "--local"])
|
||||
|
||||
assert result.exit_code == 0, f"CLI failed: {result.output}"
|
||||
data = _parse_json_output(result.output)
|
||||
assert "projects" in data
|
||||
assert len(data["projects"]) == 1
|
||||
proj = data["projects"][0]
|
||||
assert proj["name"] == "alpha"
|
||||
assert proj["is_default"] is True
|
||||
assert "local_path" in proj
|
||||
assert "cli_route" in proj
|
||||
assert "mcp_stdio" in proj
|
||||
+1
-13
@@ -188,12 +188,7 @@ async def engine_factory(
|
||||
|
||||
Uses parameterized db_backend fixture to run tests against both backends.
|
||||
"""
|
||||
from basic_memory.models.search import (
|
||||
CREATE_SEARCH_INDEX,
|
||||
CREATE_SQLITE_SEARCH_VECTOR_CHUNKS,
|
||||
CREATE_SQLITE_SEARCH_VECTOR_CHUNKS_PROJECT_ENTITY,
|
||||
CREATE_SQLITE_SEARCH_VECTOR_CHUNKS_UNIQUE,
|
||||
)
|
||||
from basic_memory.models.search import CREATE_SEARCH_INDEX
|
||||
|
||||
if db_backend == "postgres":
|
||||
# Postgres mode using testcontainers
|
||||
@@ -226,8 +221,6 @@ async def engine_factory(
|
||||
CREATE_POSTGRES_SEARCH_INDEX_FTS,
|
||||
CREATE_POSTGRES_SEARCH_INDEX_METADATA,
|
||||
CREATE_POSTGRES_SEARCH_INDEX_PERMALINK,
|
||||
CREATE_POSTGRES_SEARCH_VECTOR_CHUNKS_TABLE,
|
||||
CREATE_POSTGRES_SEARCH_VECTOR_CHUNKS_INDEX,
|
||||
)
|
||||
|
||||
# Drop and recreate all tables for test isolation
|
||||
@@ -242,8 +235,6 @@ async def engine_factory(
|
||||
await conn.execute(CREATE_POSTGRES_SEARCH_INDEX_FTS)
|
||||
await conn.execute(CREATE_POSTGRES_SEARCH_INDEX_METADATA)
|
||||
await conn.execute(CREATE_POSTGRES_SEARCH_INDEX_PERMALINK)
|
||||
await conn.execute(CREATE_POSTGRES_SEARCH_VECTOR_CHUNKS_TABLE)
|
||||
await conn.execute(CREATE_POSTGRES_SEARCH_VECTOR_CHUNKS_INDEX)
|
||||
|
||||
# Mark migrations as already applied for this test-created schema.
|
||||
#
|
||||
@@ -278,9 +269,6 @@ async def engine_factory(
|
||||
async with engine.begin() as conn:
|
||||
await conn.run_sync(Base.metadata.create_all)
|
||||
await conn.execute(CREATE_SEARCH_INDEX)
|
||||
await conn.execute(CREATE_SQLITE_SEARCH_VECTOR_CHUNKS)
|
||||
await conn.execute(CREATE_SQLITE_SEARCH_VECTOR_CHUNKS_PROJECT_ENTITY)
|
||||
await conn.execute(CREATE_SQLITE_SEARCH_VECTOR_CHUNKS_UNIQUE)
|
||||
|
||||
# Yield after setup is complete
|
||||
yield engine, session_maker
|
||||
|
||||
@@ -5,10 +5,7 @@ in YAML frontmatter are automatically parsed as datetime.date objects by PyYAML,
|
||||
but later code expects strings and calls .strip() on them, causing AttributeError.
|
||||
"""
|
||||
|
||||
from textwrap import dedent
|
||||
|
||||
import pytest
|
||||
|
||||
from basic_memory.markdown.entity_parser import EntityParser
|
||||
|
||||
|
||||
@@ -16,22 +13,20 @@ from basic_memory.markdown.entity_parser import EntityParser
|
||||
def test_file_with_date(tmp_path):
|
||||
"""Create a test file with date fields in frontmatter."""
|
||||
test_file = tmp_path / "test_note.md"
|
||||
test_file.write_text(
|
||||
dedent("""\
|
||||
---
|
||||
title: Test Note
|
||||
date: 2025-10-24
|
||||
created: 2025-10-24
|
||||
tags:
|
||||
- python
|
||||
- testing
|
||||
---
|
||||
content = """---
|
||||
title: Test Note
|
||||
date: 2025-10-24
|
||||
created: 2025-10-24
|
||||
tags:
|
||||
- python
|
||||
- testing
|
||||
---
|
||||
|
||||
# Test Content
|
||||
# Test Content
|
||||
|
||||
This file has date fields in frontmatter that PyYAML will parse as datetime.date objects.
|
||||
""")
|
||||
)
|
||||
This file has date fields in frontmatter that PyYAML will parse as datetime.date objects.
|
||||
"""
|
||||
test_file.write_text(content)
|
||||
return test_file
|
||||
|
||||
|
||||
@@ -39,18 +34,16 @@ def test_file_with_date(tmp_path):
|
||||
def test_file_with_date_in_tags(tmp_path):
|
||||
"""Create a test file with a date value in tags (edge case)."""
|
||||
test_file = tmp_path / "test_note_date_tags.md"
|
||||
test_file.write_text(
|
||||
dedent("""\
|
||||
---
|
||||
title: Test Note with Date Tags
|
||||
tags: 2025-10-24
|
||||
---
|
||||
content = """---
|
||||
title: Test Note with Date Tags
|
||||
tags: 2025-10-24
|
||||
---
|
||||
|
||||
# Test Content
|
||||
# Test Content
|
||||
|
||||
This file has a date value as tags, which will be parsed as datetime.date.
|
||||
""")
|
||||
)
|
||||
This file has a date value as tags, which will be parsed as datetime.date.
|
||||
"""
|
||||
test_file.write_text(content)
|
||||
return test_file
|
||||
|
||||
|
||||
@@ -58,21 +51,19 @@ def test_file_with_date_in_tags(tmp_path):
|
||||
def test_file_with_dates_in_tag_list(tmp_path):
|
||||
"""Create a test file with dates in a tag list (edge case)."""
|
||||
test_file = tmp_path / "test_note_dates_in_list.md"
|
||||
test_file.write_text(
|
||||
dedent("""\
|
||||
---
|
||||
title: Test Note with Dates in Tags List
|
||||
tags:
|
||||
- valid-tag
|
||||
- 2025-10-24
|
||||
- another-tag
|
||||
---
|
||||
content = """---
|
||||
title: Test Note with Dates in Tags List
|
||||
tags:
|
||||
- valid-tag
|
||||
- 2025-10-24
|
||||
- another-tag
|
||||
---
|
||||
|
||||
# Test Content
|
||||
# Test Content
|
||||
|
||||
This file has date values mixed into tags list.
|
||||
""")
|
||||
)
|
||||
This file has date values mixed into tags list.
|
||||
"""
|
||||
test_file.write_text(content)
|
||||
return test_file
|
||||
|
||||
|
||||
@@ -138,87 +129,6 @@ async def test_parse_file_with_dates_in_tag_list(test_file_with_dates_in_tag_lis
|
||||
assert "2025-10-24" in tags
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_parse_file_with_list_frontmatter_fields(tmp_path):
|
||||
"""Test that list values in expected-string frontmatter fields are coerced to strings.
|
||||
|
||||
Reproduces basic-memory-cloud#376 where a markdown file has YAML list values
|
||||
in frontmatter fields like 'title' or 'type' that downstream code expects
|
||||
to be strings, causing 'list' object has no attribute 'strip'.
|
||||
"""
|
||||
test_file = tmp_path / "test_list_fields.md"
|
||||
test_file.write_text(
|
||||
dedent("""\
|
||||
---
|
||||
title:
|
||||
- Week 2 Discussion Post
|
||||
- Alternate Title
|
||||
tags:
|
||||
- coursework
|
||||
- sie-571
|
||||
type:
|
||||
- note
|
||||
- assignment
|
||||
some_field:
|
||||
- item1
|
||||
- item2
|
||||
---
|
||||
|
||||
# Content
|
||||
|
||||
Some body text.
|
||||
""")
|
||||
)
|
||||
|
||||
parser = EntityParser(tmp_path)
|
||||
entity_markdown = await parser.parse_file(test_file)
|
||||
|
||||
# title must always be a string, even when YAML parses it as a list
|
||||
title = entity_markdown.frontmatter.title
|
||||
assert isinstance(title, str), f"Expected str, got {type(title)}"
|
||||
assert "Week 2 Discussion Post" in title
|
||||
|
||||
# type must always be a string
|
||||
note_type = entity_markdown.frontmatter.type
|
||||
assert isinstance(note_type, str), f"Expected str, got {type(note_type)}"
|
||||
|
||||
# tags should still be a list (they're explicitly handled)
|
||||
tags = entity_markdown.frontmatter.tags
|
||||
assert isinstance(tags, list)
|
||||
assert "coursework" in tags
|
||||
|
||||
# arbitrary list fields in metadata are preserved as lists
|
||||
some_field = entity_markdown.frontmatter.metadata.get("some_field")
|
||||
assert isinstance(some_field, list)
|
||||
assert some_field == ["item1", "item2"]
|
||||
|
||||
# Verify title is safe for .strip() and .casefold() (the actual crash sites)
|
||||
assert title.strip().casefold()
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_parse_file_with_list_title_single_item(tmp_path):
|
||||
"""Test that a single-item list title is coerced to a plain string."""
|
||||
test_file = tmp_path / "test_single_list_title.md"
|
||||
test_file.write_text(
|
||||
dedent("""\
|
||||
---
|
||||
title:
|
||||
- My Single Title
|
||||
---
|
||||
|
||||
# Content
|
||||
""")
|
||||
)
|
||||
|
||||
parser = EntityParser(tmp_path)
|
||||
entity_markdown = await parser.parse_file(test_file)
|
||||
|
||||
title = entity_markdown.frontmatter.title
|
||||
assert isinstance(title, str)
|
||||
assert title == "My Single Title"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_parse_file_with_various_yaml_types(tmp_path):
|
||||
"""Test that various YAML types in frontmatter don't cause errors.
|
||||
@@ -228,26 +138,24 @@ async def test_parse_file_with_various_yaml_types(tmp_path):
|
||||
when code expects strings and calls .strip().
|
||||
"""
|
||||
test_file = tmp_path / "test_yaml_types.md"
|
||||
test_file.write_text(
|
||||
dedent("""\
|
||||
---
|
||||
title: Test YAML Types
|
||||
date: 2025-10-24
|
||||
priority: 1
|
||||
completed: true
|
||||
tags:
|
||||
- python
|
||||
- testing
|
||||
metadata:
|
||||
author: Test User
|
||||
version: 1.0
|
||||
---
|
||||
content = """---
|
||||
title: Test YAML Types
|
||||
date: 2025-10-24
|
||||
priority: 1
|
||||
completed: true
|
||||
tags:
|
||||
- python
|
||||
- testing
|
||||
metadata:
|
||||
author: Test User
|
||||
version: 1.0
|
||||
---
|
||||
|
||||
# Test Content
|
||||
# Test Content
|
||||
|
||||
This file has various YAML types that need to be normalized.
|
||||
""")
|
||||
)
|
||||
This file has various YAML types that need to be normalized.
|
||||
"""
|
||||
test_file.write_text(content)
|
||||
|
||||
parser = EntityParser(tmp_path)
|
||||
entity_markdown = await parser.parse_file(test_file)
|
||||
@@ -294,19 +202,20 @@ async def test_parse_file_with_datetime_objects(tmp_path):
|
||||
with time components (as parsed by PyYAML), ensuring they're converted to ISO format strings.
|
||||
"""
|
||||
test_file = tmp_path / "test_datetime.md"
|
||||
test_file.write_text(
|
||||
dedent("""\
|
||||
---
|
||||
title: Test Datetime
|
||||
created_at: 2025-10-24 14:30:00
|
||||
updated_at: 2025-10-24T00:00:00
|
||||
---
|
||||
|
||||
# Test Content
|
||||
# YAML datetime strings that PyYAML will parse as datetime objects
|
||||
# Format: YYYY-MM-DD HH:MM:SS or YYYY-MM-DDTHH:MM:SS
|
||||
content = """---
|
||||
title: Test Datetime
|
||||
created_at: 2025-10-24 14:30:00
|
||||
updated_at: 2025-10-24T00:00:00
|
||||
---
|
||||
|
||||
This file has datetime values in frontmatter that PyYAML will parse as datetime objects.
|
||||
""")
|
||||
)
|
||||
# Test Content
|
||||
|
||||
This file has datetime values in frontmatter that PyYAML will parse as datetime objects.
|
||||
"""
|
||||
test_file.write_text(content)
|
||||
|
||||
parser = EntityParser(tmp_path)
|
||||
entity_markdown = await parser.parse_file(test_file)
|
||||
@@ -325,37 +234,3 @@ async def test_parse_file_with_datetime_objects(tmp_path):
|
||||
assert "2025-10-24" in updated_at and "00:00:00" in updated_at, (
|
||||
f"Datetime at midnight should be normalized to ISO format, got: {updated_at}"
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_parse_file_with_reserved_frontmatter_field_content(tmp_path):
|
||||
"""Test that a 'content' field in frontmatter doesn't break parsing.
|
||||
|
||||
Reproduces basic-memory-cloud#375 where frontmatter containing a field named
|
||||
'content' causes frontmatter.Post.__init__() to receive multiple values for
|
||||
the 'content' positional argument.
|
||||
"""
|
||||
test_file = tmp_path / "topic-note-template.md"
|
||||
test_file.write_text(
|
||||
dedent("""\
|
||||
---
|
||||
title: Topic Note Template
|
||||
content: Template for topic notes
|
||||
handler: some-handler-value
|
||||
---
|
||||
|
||||
# Template Body
|
||||
|
||||
Actual body content here.
|
||||
""")
|
||||
)
|
||||
|
||||
parser = EntityParser(tmp_path)
|
||||
entity_markdown = await parser.parse_file(test_file)
|
||||
|
||||
assert entity_markdown.frontmatter.title == "Topic Note Template"
|
||||
# The 'content' and 'handler' fields should be preserved in metadata
|
||||
assert entity_markdown.frontmatter.metadata.get("content") == "Template for topic notes"
|
||||
assert entity_markdown.frontmatter.metadata.get("handler") == "some-handler-value"
|
||||
# The actual body content should be parsed correctly
|
||||
assert "Template Body" in entity_markdown.content
|
||||
|
||||
@@ -1,216 +0,0 @@
|
||||
"""Tests for graph and FCM typed clients."""
|
||||
|
||||
from unittest.mock import MagicMock
|
||||
|
||||
import pytest
|
||||
|
||||
from basic_memory.mcp.clients import FCMClient, GraphClient
|
||||
|
||||
|
||||
class TestGraphClient:
|
||||
def test_init(self):
|
||||
mock_http = MagicMock()
|
||||
client = GraphClient(mock_http, "project-123")
|
||||
assert client.http_client is mock_http
|
||||
assert client.project_id == "project-123"
|
||||
assert client._base_path == "/v2/projects/project-123/graph"
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_lineage(self, monkeypatch):
|
||||
from basic_memory.mcp.clients import graph as graph_mod
|
||||
from basic_memory.schemas.graph_intelligence import GraphLineageRequest
|
||||
|
||||
mock_response = MagicMock()
|
||||
mock_response.json.return_value = {
|
||||
"root": {"id": "specs/search", "title": "specs/search", "permalink": "specs/search"},
|
||||
"paths": [],
|
||||
"generated_at": "2026-03-05T00:00:00+00:00",
|
||||
}
|
||||
|
||||
async def mock_call_post(client, url, **kwargs):
|
||||
assert "/v2/projects/proj-123/graph/lineage" in url
|
||||
return mock_response
|
||||
|
||||
monkeypatch.setattr(graph_mod, "call_post", mock_call_post)
|
||||
|
||||
client = GraphClient(MagicMock(), "proj-123")
|
||||
result = await client.lineage(GraphLineageRequest(start="memory://specs/search"))
|
||||
assert result.root.id == "specs/search"
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_impact(self, monkeypatch):
|
||||
from basic_memory.mcp.clients import graph as graph_mod
|
||||
from basic_memory.schemas.graph_intelligence import GraphImpactRequest
|
||||
|
||||
mock_response = MagicMock()
|
||||
mock_response.json.return_value = {
|
||||
"target": {"id": "specs/search", "title": "specs/search"},
|
||||
"affected": [],
|
||||
"summary": {"total_considered": 0, "total_returned": 0},
|
||||
}
|
||||
|
||||
async def mock_call_post(client, url, **kwargs):
|
||||
assert "/v2/projects/proj-123/graph/impact" in url
|
||||
return mock_response
|
||||
|
||||
monkeypatch.setattr(graph_mod, "call_post", mock_call_post)
|
||||
|
||||
client = GraphClient(MagicMock(), "proj-123")
|
||||
result = await client.impact(GraphImpactRequest(target="memory://specs/search", horizon=2))
|
||||
assert result.summary.total_returned == 0
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_health(self, monkeypatch):
|
||||
from basic_memory.mcp.clients import graph as graph_mod
|
||||
|
||||
mock_response = MagicMock()
|
||||
mock_response.json.return_value = {
|
||||
"metrics": {
|
||||
"orphan_rate": 0.0,
|
||||
"stale_central_nodes": 0,
|
||||
"overloaded_hubs": 0,
|
||||
"contradiction_candidates": 0,
|
||||
},
|
||||
"issues": [],
|
||||
"computed_at": "2026-03-05T00:00:00+00:00",
|
||||
}
|
||||
|
||||
async def mock_call_get(client, url, **kwargs):
|
||||
assert "/v2/projects/proj-123/graph/health" in url
|
||||
assert kwargs["params"]["scope"] == "specs"
|
||||
return mock_response
|
||||
|
||||
monkeypatch.setattr(graph_mod, "call_get", mock_call_get)
|
||||
|
||||
client = GraphClient(MagicMock(), "proj-123")
|
||||
result = await client.health(scope="specs", timeframe="30d")
|
||||
assert result.metrics.orphan_rate == 0.0
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_reindex(self, monkeypatch):
|
||||
from basic_memory.mcp.clients import graph as graph_mod
|
||||
from basic_memory.schemas.graph_intelligence import GraphReindexRequest
|
||||
|
||||
mock_response = MagicMock()
|
||||
mock_response.json.return_value = {
|
||||
"job_id": "job-123",
|
||||
"status": "queued",
|
||||
"scheduled_at": "2026-03-05T00:00:00+00:00",
|
||||
}
|
||||
|
||||
async def mock_call_post(client, url, **kwargs):
|
||||
assert "/v2/projects/proj-123/graph/reindex" in url
|
||||
return mock_response
|
||||
|
||||
monkeypatch.setattr(graph_mod, "call_post", mock_call_post)
|
||||
|
||||
client = GraphClient(MagicMock(), "proj-123")
|
||||
result = await client.reindex(GraphReindexRequest(mode="full"))
|
||||
assert result.status == "queued"
|
||||
|
||||
|
||||
class TestFCMClient:
|
||||
def test_init(self):
|
||||
mock_http = MagicMock()
|
||||
client = FCMClient(mock_http, "project-123")
|
||||
assert client.http_client is mock_http
|
||||
assert client.project_id == "project-123"
|
||||
assert client._base_path == "/v2/projects/project-123/fcm"
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_simulate(self, monkeypatch):
|
||||
from basic_memory.mcp.clients import fcm as fcm_mod
|
||||
from basic_memory.schemas.graph_intelligence import FCMSimulateRequest
|
||||
|
||||
mock_response = MagicMock()
|
||||
mock_response.json.return_value = {
|
||||
"baseline": [{"node_id": "n1", "state": 0.0}],
|
||||
"projected": [{"node_id": "n1", "state": 0.2}],
|
||||
"deltas": [{"node_id": "n1", "delta": 0.2}],
|
||||
"stability": {"converged": True, "iterations_used": 3, "residual": 0.0},
|
||||
"confidence": 0.5,
|
||||
"explanations": [],
|
||||
"evidence_refs": [],
|
||||
}
|
||||
|
||||
async def mock_call_post(client, url, **kwargs):
|
||||
assert "/v2/projects/proj-123/fcm/simulate" in url
|
||||
return mock_response
|
||||
|
||||
monkeypatch.setattr(fcm_mod, "call_post", mock_call_post)
|
||||
|
||||
request = FCMSimulateRequest(actions=[{"node_id": "n1", "delta": 0.2}])
|
||||
result = await FCMClient(MagicMock(), "proj-123").simulate(request)
|
||||
assert result.stability.converged is True
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_rank_actions(self, monkeypatch):
|
||||
from basic_memory.mcp.clients import fcm as fcm_mod
|
||||
from basic_memory.schemas.graph_intelligence import FCMRankActionsRequest
|
||||
|
||||
mock_response = MagicMock()
|
||||
mock_response.json.return_value = {
|
||||
"goal": {"node_id": "g1", "label": "g1"},
|
||||
"recommendations": [],
|
||||
}
|
||||
|
||||
async def mock_call_post(client, url, **kwargs):
|
||||
assert "/v2/projects/proj-123/fcm/rank-actions" in url
|
||||
return mock_response
|
||||
|
||||
monkeypatch.setattr(fcm_mod, "call_post", mock_call_post)
|
||||
|
||||
request = FCMRankActionsRequest(goal="g1")
|
||||
result = await FCMClient(MagicMock(), "proj-123").rank_actions(request)
|
||||
assert result.goal.node_id == "g1"
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_import_model(self, monkeypatch):
|
||||
from basic_memory.mcp.clients import fcm as fcm_mod
|
||||
from basic_memory.schemas.graph_intelligence import FCMImportRequest
|
||||
|
||||
mock_response = MagicMock()
|
||||
mock_response.json.return_value = {
|
||||
"import_id": "imp-1",
|
||||
"nodes_loaded": 0,
|
||||
"edges_loaded": 0,
|
||||
"warnings": [],
|
||||
"errors": [],
|
||||
}
|
||||
|
||||
async def mock_call_post(client, url, **kwargs):
|
||||
assert "/v2/projects/proj-123/fcm/import" in url
|
||||
return mock_response
|
||||
|
||||
monkeypatch.setattr(fcm_mod, "call_post", mock_call_post)
|
||||
|
||||
request = FCMImportRequest(source="/tmp/model.csv")
|
||||
result = await FCMClient(MagicMock(), "proj-123").import_model(request)
|
||||
assert result.import_id == "imp-1"
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_export_model(self, monkeypatch):
|
||||
from basic_memory.mcp.clients import fcm as fcm_mod
|
||||
from basic_memory.schemas.graph_intelligence import FCMExportRequest
|
||||
|
||||
mock_response = MagicMock()
|
||||
mock_response.json.return_value = {
|
||||
"export_id": "exp-1",
|
||||
"format": "csv_bundle_v1",
|
||||
"files": [
|
||||
{"name": "nodes.csv", "path": "/tmp/nodes.csv"},
|
||||
{"name": "edges.csv", "path": "/tmp/edges.csv"},
|
||||
],
|
||||
"node_count": 0,
|
||||
"edge_count": 0,
|
||||
}
|
||||
|
||||
async def mock_call_post(client, url, **kwargs):
|
||||
assert "/v2/projects/proj-123/fcm/export" in url
|
||||
return mock_response
|
||||
|
||||
monkeypatch.setattr(fcm_mod, "call_post", mock_call_post)
|
||||
|
||||
request = FCMExportRequest()
|
||||
result = await FCMClient(MagicMock(), "proj-123").export_model(request)
|
||||
assert result.format == "csv_bundle_v1"
|
||||
@@ -143,7 +143,6 @@ async def test_write_note_update_preserves_yaml_format(app, project_config, test
|
||||
directory="test",
|
||||
content="Updated content",
|
||||
tags=["updated", "new-tag", "format"],
|
||||
overwrite=True,
|
||||
)
|
||||
|
||||
# Should be an update, not a new creation
|
||||
|
||||
@@ -168,7 +168,6 @@ async def test_notes_with_similar_titles_maintain_separate_files(app, test_proje
|
||||
title=title,
|
||||
directory=folder,
|
||||
content=f"# {title}\n\nUnique content for {title}",
|
||||
overwrite=True,
|
||||
)
|
||||
|
||||
permalink = None
|
||||
|
||||
@@ -31,34 +31,17 @@ async def test_returns_none_when_no_default_and_no_project(config_manager, monke
|
||||
config_manager.save_config(cfg)
|
||||
|
||||
monkeypatch.delenv("BASIC_MEMORY_MCP_PROJECT", raising=False)
|
||||
|
||||
# Prevent API fallback from returning a project via stale dependency overrides
|
||||
async def _no_api_fallback():
|
||||
return None
|
||||
|
||||
monkeypatch.setattr(
|
||||
"basic_memory.mcp.project_context._resolve_default_project_from_api",
|
||||
_no_api_fallback,
|
||||
)
|
||||
assert await resolve_project_parameter(project=None, allow_discovery=False) is None
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_allows_discovery_when_enabled(config_manager, monkeypatch):
|
||||
async def test_allows_discovery_when_enabled(config_manager):
|
||||
from basic_memory.mcp.project_context import resolve_project_parameter
|
||||
|
||||
cfg = config_manager.load_config()
|
||||
cfg.default_project = None
|
||||
config_manager.save_config(cfg)
|
||||
|
||||
# Prevent API fallback from returning a project via stale dependency overrides
|
||||
async def _no_api_fallback():
|
||||
return None
|
||||
|
||||
monkeypatch.setattr(
|
||||
"basic_memory.mcp.project_context._resolve_default_project_from_api",
|
||||
_no_api_fallback,
|
||||
)
|
||||
assert await resolve_project_parameter(project=None, allow_discovery=True) is None
|
||||
|
||||
|
||||
@@ -118,15 +101,6 @@ async def test_returns_none_when_no_default(config_manager, monkeypatch):
|
||||
config_manager.save_config(cfg)
|
||||
|
||||
monkeypatch.delenv("BASIC_MEMORY_MCP_PROJECT", raising=False)
|
||||
|
||||
# Prevent API fallback from returning a project via stale dependency overrides
|
||||
async def _no_api_fallback():
|
||||
return None
|
||||
|
||||
monkeypatch.setattr(
|
||||
"basic_memory.mcp.project_context._resolve_default_project_from_api",
|
||||
_no_api_fallback,
|
||||
)
|
||||
assert await resolve_project_parameter(project=None) is None
|
||||
|
||||
|
||||
@@ -501,75 +475,3 @@ class TestGetProjectClientRoutingOrder:
|
||||
assert "resolve_workspace_parameter should not be called" not in error_msg
|
||||
# Should not get a local ASGI routing error
|
||||
assert "no project found" not in error_msg
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_factory_mode_skips_workspace_resolution(self, config_manager, monkeypatch):
|
||||
"""When a client factory is set (in-process cloud server), skip workspace resolution.
|
||||
|
||||
The cloud MCP server calls set_client_factory() so that get_client() routes
|
||||
requests through TenantASGITransport. In this mode, workspace and tenant context
|
||||
are already resolved by the transport layer. Attempting cloud workspace resolution
|
||||
would call the production control-plane API and fail with 401.
|
||||
"""
|
||||
from contextlib import asynccontextmanager
|
||||
|
||||
from basic_memory.mcp import async_client
|
||||
from basic_memory.mcp.project_context import get_project_client
|
||||
from basic_memory.config import ProjectEntry, ProjectMode
|
||||
|
||||
config = config_manager.load_config()
|
||||
config.projects["cloud-proj"] = ProjectEntry(
|
||||
path=str(config_manager.config_dir.parent / "cloud-proj"),
|
||||
mode=ProjectMode.CLOUD,
|
||||
)
|
||||
config_manager.save_config(config)
|
||||
|
||||
# Set up a factory (simulates what cloud MCP server does)
|
||||
@asynccontextmanager
|
||||
async def fake_factory():
|
||||
from httpx import ASGITransport, AsyncClient
|
||||
from basic_memory.api.app import app as fastapi_app
|
||||
|
||||
async with AsyncClient(
|
||||
transport=ASGITransport(app=fastapi_app),
|
||||
base_url="http://test",
|
||||
) as client:
|
||||
yield client
|
||||
|
||||
original_factory = async_client._client_factory
|
||||
async_client.set_client_factory(fake_factory)
|
||||
|
||||
# Patch workspace resolution to fail if called — factory mode should skip it
|
||||
async def fail_if_called(**kwargs): # pragma: no cover
|
||||
raise AssertionError("resolve_workspace_parameter must not be called in factory mode")
|
||||
|
||||
monkeypatch.setattr(
|
||||
"basic_memory.mcp.project_context.resolve_workspace_parameter",
|
||||
fail_if_called,
|
||||
)
|
||||
|
||||
# Patch get_cloud_control_plane_client to fail if called
|
||||
@asynccontextmanager
|
||||
async def fail_control_plane(): # pragma: no cover
|
||||
raise AssertionError(
|
||||
"get_cloud_control_plane_client must not be called in factory mode"
|
||||
)
|
||||
|
||||
monkeypatch.setattr(
|
||||
"basic_memory.mcp.async_client.get_cloud_control_plane_client",
|
||||
fail_control_plane,
|
||||
)
|
||||
|
||||
try:
|
||||
# Will fail at project validation (no real project in DB), but proves
|
||||
# workspace resolution and control-plane calls were skipped
|
||||
with pytest.raises(Exception) as exc_info:
|
||||
async with get_project_client(project="cloud-proj"):
|
||||
pass
|
||||
|
||||
error_msg = str(exc_info.value).lower()
|
||||
assert "resolve_workspace_parameter must not be called" not in error_msg
|
||||
assert "get_cloud_control_plane_client must not be called" not in error_msg
|
||||
finally:
|
||||
# Restore original factory to avoid polluting other tests
|
||||
async_client._client_factory = original_factory
|
||||
|
||||
@@ -9,6 +9,7 @@ import pytest
|
||||
|
||||
from basic_memory.mcp.prompts.search import search_prompt
|
||||
from basic_memory.mcp.prompts.continue_conversation import continue_conversation
|
||||
from basic_memory.schemas.search import SearchResponse, SearchResult
|
||||
|
||||
|
||||
# --- search_prompt ---
|
||||
@@ -19,20 +20,19 @@ async def test_search_prompt_delegates_to_search_notes(monkeypatch):
|
||||
"""Search prompt should call search_notes tool and wrap output."""
|
||||
captured_kwargs = {}
|
||||
|
||||
# Prompts use output_format="json", so mock returns a dict
|
||||
fake_result = {
|
||||
"results": [
|
||||
{
|
||||
"type": "entity",
|
||||
"title": "Test Note",
|
||||
"permalink": "test-note",
|
||||
"file_path": "test-note.md",
|
||||
"score": 0.95,
|
||||
}
|
||||
fake_result = SearchResponse(
|
||||
results=[
|
||||
SearchResult(
|
||||
type="entity",
|
||||
title="Test Note",
|
||||
permalink="test-note",
|
||||
file_path="test-note.md",
|
||||
score=0.95,
|
||||
)
|
||||
],
|
||||
"current_page": 1,
|
||||
"page_size": 10,
|
||||
}
|
||||
current_page=1,
|
||||
page_size=10,
|
||||
)
|
||||
|
||||
async def fake_search_notes(**kwargs):
|
||||
captured_kwargs.update(kwargs)
|
||||
@@ -45,7 +45,6 @@ async def test_search_prompt_delegates_to_search_notes(monkeypatch):
|
||||
# Verify delegation
|
||||
assert captured_kwargs["query"] == "my query"
|
||||
assert captured_kwargs["after_date"] == "1w"
|
||||
assert captured_kwargs["output_format"] == "json"
|
||||
|
||||
# Verify output wrapping
|
||||
assert 'Search Results: "my query"' in out
|
||||
@@ -56,7 +55,7 @@ async def test_search_prompt_delegates_to_search_notes(monkeypatch):
|
||||
@pytest.mark.asyncio
|
||||
async def test_search_prompt_handles_no_results(monkeypatch):
|
||||
"""Search prompt should handle empty results gracefully."""
|
||||
fake_result = {"results": [], "current_page": 1, "page_size": 10}
|
||||
fake_result = SearchResponse(results=[], current_page=1, page_size=10)
|
||||
|
||||
async def fake_search_notes(**kwargs):
|
||||
return fake_result
|
||||
@@ -92,20 +91,19 @@ async def test_continue_conversation_delegates_to_search_notes(monkeypatch):
|
||||
"""Continue conversation with topic should call search_notes."""
|
||||
captured_kwargs = {}
|
||||
|
||||
# Prompts use output_format="json", so mock returns a dict
|
||||
fake_result = {
|
||||
"results": [
|
||||
{
|
||||
"type": "entity",
|
||||
"title": "Previous Discussion",
|
||||
"permalink": "discussions/previous",
|
||||
"file_path": "discussions/previous.md",
|
||||
"score": 0.9,
|
||||
}
|
||||
fake_result = SearchResponse(
|
||||
results=[
|
||||
SearchResult(
|
||||
type="entity",
|
||||
title="Previous Discussion",
|
||||
permalink="discussions/previous",
|
||||
file_path="discussions/previous.md",
|
||||
score=0.9,
|
||||
)
|
||||
],
|
||||
"current_page": 1,
|
||||
"page_size": 10,
|
||||
}
|
||||
current_page=1,
|
||||
page_size=10,
|
||||
)
|
||||
|
||||
async def fake_search_notes(**kwargs):
|
||||
captured_kwargs.update(kwargs)
|
||||
@@ -119,7 +117,6 @@ async def test_continue_conversation_delegates_to_search_notes(monkeypatch):
|
||||
|
||||
assert captured_kwargs["query"] == "my topic"
|
||||
assert captured_kwargs["after_date"] == "3d"
|
||||
assert captured_kwargs["output_format"] == "json"
|
||||
|
||||
assert "'my topic'" in out
|
||||
assert "Previous Discussion" in out
|
||||
@@ -167,7 +164,7 @@ async def test_continue_conversation_no_topic_default_timeframe(monkeypatch):
|
||||
@pytest.mark.asyncio
|
||||
async def test_continue_conversation_no_results_for_topic(monkeypatch):
|
||||
"""Continue conversation should show capture opportunity when no results found."""
|
||||
fake_result = {"results": [], "current_page": 1, "page_size": 10}
|
||||
fake_result = SearchResponse(results=[], current_page=1, page_size=10)
|
||||
|
||||
async def fake_search_notes(**kwargs):
|
||||
return fake_result
|
||||
|
||||
@@ -9,7 +9,7 @@ from basic_memory.mcp.tools import build_context
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_get_basic_discussion_context(client, test_graph, test_project):
|
||||
"""Test getting basic discussion context returns JSON dict with expected fields."""
|
||||
"""Test getting basic discussion context returns slimmed JSON dict."""
|
||||
result = await build_context(project=test_project.name, url="memory://test/root")
|
||||
|
||||
assert isinstance(result, dict)
|
||||
@@ -19,46 +19,25 @@ async def test_get_basic_discussion_context(client, test_graph, test_project):
|
||||
assert primary["permalink"] == f"{test_project.name}/test/root"
|
||||
assert len(result["results"][0]["related_results"]) > 0
|
||||
|
||||
# Verify metadata fields
|
||||
# Verify metadata — stripped fields should be absent
|
||||
meta = result["metadata"]
|
||||
assert meta["uri"] == f"{test_project.name}/test/root"
|
||||
assert meta["depth"] == 1 # default depth
|
||||
assert meta["timeframe"] is not None
|
||||
assert meta["primary_count"] == 1
|
||||
# COMPAT(v0.18): generated_at and total_results restored for old clients
|
||||
assert "generated_at" in meta
|
||||
assert "total_results" in meta
|
||||
assert "generated_at" not in meta
|
||||
assert "total_results" not in meta
|
||||
|
||||
# Entity fields present
|
||||
assert "entity_id" in primary
|
||||
assert "created_at" in primary
|
||||
# Verify entity-level stripped fields
|
||||
assert "entity_id" not in primary
|
||||
assert "created_at" not in primary
|
||||
|
||||
# Verify observation-level fields
|
||||
# Verify observation-level stripped fields
|
||||
if result["results"][0]["observations"]:
|
||||
obs = result["results"][0]["observations"][0]
|
||||
assert "observation_id" in obs
|
||||
assert "entity_id" in obs
|
||||
assert "file_path" in obs
|
||||
assert "created_at" in obs
|
||||
assert "permalink" in obs
|
||||
assert "category" in obs
|
||||
assert "content" in obs
|
||||
|
||||
# Verify related_results item structure — entities have identifying fields
|
||||
for related in result["results"][0]["related_results"]:
|
||||
item_type = related["type"]
|
||||
if item_type == "entity":
|
||||
assert "title" in related
|
||||
assert "file_path" in related
|
||||
assert "created_at" in related
|
||||
assert "entity_id" in related
|
||||
elif item_type == "relation":
|
||||
assert "relation_type" in related
|
||||
assert "title" in related
|
||||
assert "file_path" in related
|
||||
assert "created_at" in related
|
||||
assert "relation_id" in related
|
||||
assert "entity_id" in related
|
||||
assert "observation_id" not in obs
|
||||
assert "entity_id" not in obs
|
||||
assert "file_path" not in obs
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
|
||||
@@ -35,31 +35,7 @@ EXPECTED_TOOL_SIGNATURES: dict[str, list[str]] = {
|
||||
"expected_replacements",
|
||||
"output_format",
|
||||
],
|
||||
"fcm_export_model": ["format", "selection", "project", "workspace", "output_format"],
|
||||
"fcm_import_model": ["source", "format", "merge_mode", "project", "workspace", "output_format"],
|
||||
"fcm_rank_actions": ["goal", "constraints", "top_k", "project", "workspace", "output_format"],
|
||||
"fcm_simulate": ["actions", "scenario", "clamp_rules", "project", "workspace", "output_format"],
|
||||
"fetch": ["id"],
|
||||
"graph_health": ["scope", "timeframe", "project", "workspace", "output_format"],
|
||||
"graph_impact": [
|
||||
"target",
|
||||
"horizon",
|
||||
"relation_filters",
|
||||
"include_reasons",
|
||||
"project",
|
||||
"workspace",
|
||||
"output_format",
|
||||
],
|
||||
"graph_lineage": [
|
||||
"start",
|
||||
"goal",
|
||||
"max_hops",
|
||||
"relation_filters",
|
||||
"project",
|
||||
"workspace",
|
||||
"output_format",
|
||||
],
|
||||
"graph_reindex": ["mode", "reason", "project", "workspace", "output_format"],
|
||||
"list_directory": ["dir_name", "depth", "file_name_glob", "project", "workspace"],
|
||||
"list_memory_projects": ["output_format", "workspace"],
|
||||
"list_workspaces": ["output_format"],
|
||||
@@ -97,6 +73,7 @@ EXPECTED_TOOL_SIGNATURES: dict[str, list[str]] = {
|
||||
"schema_infer": ["note_type", "threshold", "project", "workspace", "output_format"],
|
||||
"schema_validate": ["note_type", "identifier", "project", "workspace", "output_format"],
|
||||
"search": ["query"],
|
||||
"search_by_metadata": ["filters", "project", "workspace", "limit", "offset"],
|
||||
"search_notes": [
|
||||
"query",
|
||||
"project",
|
||||
@@ -123,7 +100,6 @@ EXPECTED_TOOL_SIGNATURES: dict[str, list[str]] = {
|
||||
"tags",
|
||||
"note_type",
|
||||
"metadata",
|
||||
"overwrite",
|
||||
"output_format",
|
||||
],
|
||||
}
|
||||
@@ -137,15 +113,7 @@ TOOL_FUNCTIONS: dict[str, object] = {
|
||||
"delete_note": tools.delete_note,
|
||||
"delete_project": tools.delete_project,
|
||||
"edit_note": tools.edit_note,
|
||||
"fcm_export_model": tools.fcm_export_model,
|
||||
"fcm_import_model": tools.fcm_import_model,
|
||||
"fcm_rank_actions": tools.fcm_rank_actions,
|
||||
"fcm_simulate": tools.fcm_simulate,
|
||||
"fetch": tools.fetch,
|
||||
"graph_health": tools.graph_health,
|
||||
"graph_impact": tools.graph_impact,
|
||||
"graph_lineage": tools.graph_lineage,
|
||||
"graph_reindex": tools.graph_reindex,
|
||||
"list_directory": tools.list_directory,
|
||||
"list_memory_projects": tools.list_memory_projects,
|
||||
"list_workspaces": tools.list_workspaces,
|
||||
@@ -158,6 +126,7 @@ TOOL_FUNCTIONS: dict[str, object] = {
|
||||
"schema_infer": tools.schema_infer,
|
||||
"schema_validate": tools.schema_validate,
|
||||
"search": tools.search,
|
||||
"search_by_metadata": tools.search_by_metadata,
|
||||
"search_notes": tools.search_notes,
|
||||
"view_note": tools.view_note,
|
||||
"write_note": tools.write_note,
|
||||
|
||||
@@ -120,141 +120,19 @@ async def test_edit_note_replace_section_operation(client, test_project):
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_edit_note_nonexistent_note_find_replace(client, test_project):
|
||||
"""Test find_replace on a note that doesn't exist - should return helpful guidance."""
|
||||
async def test_edit_note_nonexistent_note(client, test_project):
|
||||
"""Test editing a note that doesn't exist - should return helpful guidance."""
|
||||
result = await edit_note(
|
||||
project=test_project.name,
|
||||
identifier="nonexistent/note",
|
||||
operation="find_replace",
|
||||
content="replacement",
|
||||
find_text="old text",
|
||||
operation="append",
|
||||
content="Some content",
|
||||
)
|
||||
|
||||
assert isinstance(result, str)
|
||||
assert "# Edit Failed" in result
|
||||
assert "search_notes" in result # Should suggest searching
|
||||
assert "append" in result # Should suggest using append/prepend instead
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_edit_note_nonexistent_note_replace_section(client, test_project):
|
||||
"""Test replace_section on a note that doesn't exist - should return helpful guidance."""
|
||||
result = await edit_note(
|
||||
project=test_project.name,
|
||||
identifier="nonexistent/note",
|
||||
operation="replace_section",
|
||||
content="new section content",
|
||||
section="## Missing Section",
|
||||
)
|
||||
|
||||
assert isinstance(result, str)
|
||||
assert "# Edit Failed" in result
|
||||
assert "search_notes" in result # Should suggest searching
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_edit_note_append_creates_note_if_not_found(client, test_project):
|
||||
"""append to a non-existent note should create it automatically."""
|
||||
result = await edit_note(
|
||||
project=test_project.name,
|
||||
identifier="auto-created-note",
|
||||
operation="append",
|
||||
content="# New Note\n\nCreated via append.",
|
||||
)
|
||||
|
||||
assert isinstance(result, str)
|
||||
assert "Created note (append)" in result
|
||||
assert "fileCreated: true" in result
|
||||
assert f"project: {test_project.name}" in result
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_edit_note_prepend_creates_note_if_not_found(client, test_project):
|
||||
"""prepend to a non-existent note should create it automatically."""
|
||||
result = await edit_note(
|
||||
project=test_project.name,
|
||||
identifier="auto-created-prepend",
|
||||
operation="prepend",
|
||||
content="# Prepended Note\n\nCreated via prepend.",
|
||||
)
|
||||
|
||||
assert isinstance(result, str)
|
||||
assert "Created note (prepend)" in result
|
||||
assert "fileCreated: true" in result
|
||||
assert f"project: {test_project.name}" in result
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_edit_note_append_creates_with_directory_from_identifier(client, test_project):
|
||||
"""Identifier 'conversations/my-note' should create in conversations/ directory."""
|
||||
result = await edit_note(
|
||||
project=test_project.name,
|
||||
identifier="conversations/my-note",
|
||||
operation="append",
|
||||
content="# My Note\n\nCreated in conversations directory.",
|
||||
)
|
||||
|
||||
assert isinstance(result, str)
|
||||
assert "Created note (append)" in result
|
||||
assert "fileCreated: true" in result
|
||||
assert "conversations/" in result
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_edit_note_append_creates_at_root_when_no_directory(client, test_project):
|
||||
"""Identifier 'my-note' (no slash) should create at project root."""
|
||||
result = await edit_note(
|
||||
project=test_project.name,
|
||||
identifier="root-level-note",
|
||||
operation="append",
|
||||
content="# Root Note\n\nCreated at root.",
|
||||
)
|
||||
|
||||
assert isinstance(result, str)
|
||||
assert "Created note (append)" in result
|
||||
assert "fileCreated: true" in result
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_edit_note_append_creates_json_format(client, test_project):
|
||||
"""JSON output should include fileCreated: true when note is auto-created."""
|
||||
result = await edit_note(
|
||||
project=test_project.name,
|
||||
identifier="json-auto-create",
|
||||
operation="append",
|
||||
content="# JSON Test\n\nAuto-created.",
|
||||
output_format="json",
|
||||
)
|
||||
|
||||
assert isinstance(result, dict)
|
||||
assert result["fileCreated"] is True
|
||||
assert result["title"] is not None
|
||||
assert result["operation"] == "append"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_edit_note_existing_note_json_includes_file_created_false(client, test_project):
|
||||
"""JSON output for editing an existing note should include fileCreated: false."""
|
||||
# Create the note first
|
||||
await write_note(
|
||||
project=test_project.name,
|
||||
title="Existing JSON Note",
|
||||
directory="test",
|
||||
content="# Existing Note\nOriginal content.",
|
||||
)
|
||||
|
||||
result = await edit_note(
|
||||
project=test_project.name,
|
||||
identifier="test/existing-json-note",
|
||||
operation="append",
|
||||
content="\nAppended content.",
|
||||
output_format="json",
|
||||
)
|
||||
|
||||
assert isinstance(result, dict)
|
||||
assert result["fileCreated"] is False
|
||||
assert result["title"] == "Existing JSON Note"
|
||||
assert result["operation"] == "append"
|
||||
assert "read_note" in result # Should suggest reading to verify
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@@ -536,39 +414,6 @@ async def test_edit_note_find_replace_empty_find_text(client, test_project):
|
||||
# Should contain helpful guidance about the error
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_edit_note_append_with_null_optional_fields(client, test_project):
|
||||
"""Regression test: MCP clients may send explicit null for unused optional fields.
|
||||
|
||||
When an MCP client sends find_text=None, section=None, expected_replacements=None
|
||||
for an append operation, the tool should accept them without validation errors.
|
||||
"""
|
||||
# Create initial note
|
||||
await write_note(
|
||||
project=test_project.name,
|
||||
title="Null Fields Test",
|
||||
directory="test",
|
||||
content="# Null Fields Test\nOriginal content.",
|
||||
)
|
||||
|
||||
# Call edit_note with explicit None for all optional fields (simulates MCP null)
|
||||
result = await edit_note(
|
||||
project=test_project.name,
|
||||
identifier="test/null-fields-test",
|
||||
operation="append",
|
||||
content="\nAppended content.",
|
||||
find_text=None,
|
||||
section=None,
|
||||
expected_replacements=None,
|
||||
)
|
||||
|
||||
assert isinstance(result, str)
|
||||
assert "Edited note (append)" in result
|
||||
assert f"project: {test_project.name}" in result
|
||||
assert "file_path: test/Null Fields Test.md" in result
|
||||
assert f"[Session: Using project '{test_project.name}']" in result
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_edit_note_preserves_permalink_when_frontmatter_missing(client, test_project):
|
||||
"""Test that editing a note preserves the permalink when frontmatter doesn't contain one.
|
||||
|
||||
@@ -1,114 +0,0 @@
|
||||
"""Tests for graph intelligence MCP tools."""
|
||||
|
||||
import pytest
|
||||
|
||||
from basic_memory.mcp.tools import (
|
||||
fcm_export_model,
|
||||
fcm_import_model,
|
||||
fcm_rank_actions,
|
||||
fcm_simulate,
|
||||
graph_health,
|
||||
graph_impact,
|
||||
graph_lineage,
|
||||
graph_reindex,
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_graph_lineage_json_and_text_modes(app, test_project):
|
||||
json_result = await graph_lineage(
|
||||
start="memory://specs/search",
|
||||
project=test_project.name,
|
||||
output_format="json",
|
||||
)
|
||||
assert isinstance(json_result, dict)
|
||||
assert set(["root", "paths", "generated_at"]).issubset(json_result.keys())
|
||||
|
||||
text_result = await graph_lineage(
|
||||
start="memory://specs/search",
|
||||
project=test_project.name,
|
||||
output_format="text",
|
||||
)
|
||||
assert isinstance(text_result, str)
|
||||
assert "Graph Lineage" in text_result
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_graph_impact_and_health(app, test_project):
|
||||
impact = await graph_impact(
|
||||
target="memory://specs/search",
|
||||
horizon=2,
|
||||
project=test_project.name,
|
||||
output_format="json",
|
||||
)
|
||||
assert isinstance(impact, dict)
|
||||
assert set(["target", "affected", "summary"]).issubset(impact.keys())
|
||||
|
||||
health = await graph_health(
|
||||
scope="specs",
|
||||
timeframe="30d",
|
||||
project=test_project.name,
|
||||
output_format="json",
|
||||
)
|
||||
assert isinstance(health, dict)
|
||||
assert set(["metrics", "issues", "computed_at"]).issubset(health.keys())
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_graph_reindex(app, test_project):
|
||||
result = await graph_reindex(project=test_project.name, output_format="json")
|
||||
assert isinstance(result, dict)
|
||||
assert result["status"] == "queued"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_fcm_simulate_and_rank_actions(app, test_project):
|
||||
simulation = await fcm_simulate(
|
||||
actions=[{"node_id": "n1", "delta": 0.2}],
|
||||
project=test_project.name,
|
||||
output_format="json",
|
||||
)
|
||||
assert isinstance(simulation, dict)
|
||||
assert set(["baseline", "projected", "deltas", "stability", "confidence"]).issubset(
|
||||
simulation.keys()
|
||||
)
|
||||
|
||||
ranking = await fcm_rank_actions(
|
||||
goal="reduce-regressions",
|
||||
top_k=2,
|
||||
project=test_project.name,
|
||||
output_format="json",
|
||||
)
|
||||
assert isinstance(ranking, dict)
|
||||
assert set(["goal", "recommendations"]).issubset(ranking.keys())
|
||||
assert len(ranking["recommendations"]) <= 2
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_fcm_import_export_json_and_text(app, test_project):
|
||||
imported = await fcm_import_model(
|
||||
source="/tmp/model.csv",
|
||||
format="csv_bundle_v1",
|
||||
project=test_project.name,
|
||||
output_format="json",
|
||||
)
|
||||
assert isinstance(imported, dict)
|
||||
assert "import_id" in imported
|
||||
|
||||
exported_json = await fcm_export_model(
|
||||
format="csv_bundle_v1",
|
||||
selection={"scope": "all"},
|
||||
project=test_project.name,
|
||||
output_format="json",
|
||||
)
|
||||
assert isinstance(exported_json, dict)
|
||||
assert set(["export_id", "files", "node_count", "edge_count"]).issubset(exported_json.keys())
|
||||
|
||||
exported_text = await fcm_export_model(
|
||||
format="csv_bundle_v1",
|
||||
selection={"scope": "all"},
|
||||
project=test_project.name,
|
||||
output_format="text",
|
||||
)
|
||||
assert isinstance(exported_text, str)
|
||||
assert "FCM Export" in exported_text
|
||||
@@ -38,7 +38,6 @@ async def test_write_note_text_and_json_modes(app, test_project):
|
||||
directory="mode-tests",
|
||||
content="# Mode Write Note\n\nupdated",
|
||||
output_format="json",
|
||||
overwrite=True,
|
||||
)
|
||||
assert isinstance(json_result, dict)
|
||||
assert json_result["title"] == "Mode Write Note"
|
||||
|
||||
@@ -128,16 +128,69 @@ async def test_recent_activity_type_invalid(client, test_project, test_graph):
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_recent_activity_uses_default_project(client, test_project, test_graph):
|
||||
"""When no project parameter is given, recent_activity uses the default project."""
|
||||
# Call without explicit project — should resolve to the default
|
||||
async def test_recent_activity_discovery_mode(client, test_project, test_graph, config_manager):
|
||||
"""Test that recent_activity discovery mode works without project parameter."""
|
||||
# Clear default_project to test discovery mode
|
||||
cfg = config_manager.load_config()
|
||||
cfg.default_project = None
|
||||
config_manager.save_config(cfg)
|
||||
|
||||
# Test discovery mode (no project parameter)
|
||||
result = await recent_activity()
|
||||
assert result is not None
|
||||
assert isinstance(result, str)
|
||||
|
||||
# Should return project-specific output for the default project
|
||||
assert "Recent Activity:" in result
|
||||
assert "Activity Summary:" in result
|
||||
# Check that we get a formatted summary
|
||||
assert "Recent Activity Summary" in result
|
||||
assert "Most Active Project:" in result or "Other Active Projects:" in result
|
||||
assert "Summary:" in result
|
||||
assert "active projects" in result
|
||||
|
||||
# Should contain project discovery guidance
|
||||
assert "Suggested project:" in result or "Multiple active projects" in result
|
||||
assert "Session reminder:" in result
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_recent_activity_discovery_mode_no_activity(client, test_project, config_manager):
|
||||
"""If there is no activity in any project, discovery mode should say so."""
|
||||
# Clear default_project to test discovery mode
|
||||
cfg = config_manager.load_config()
|
||||
cfg.default_project = None
|
||||
config_manager.save_config(cfg)
|
||||
|
||||
result = await recent_activity()
|
||||
assert "Recent Activity Summary" in result
|
||||
assert "No recent activity found in any project." in result
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_recent_activity_discovery_mode_multiple_active_projects(
|
||||
app, client, test_project, tmp_path_factory, config_manager
|
||||
):
|
||||
"""Discovery mode should use the multi-project guidance when multiple projects have activity."""
|
||||
# Clear default_project to test discovery mode
|
||||
cfg = config_manager.load_config()
|
||||
cfg.default_project = None
|
||||
config_manager.save_config(cfg)
|
||||
|
||||
from basic_memory.mcp.tools import create_memory_project, write_note
|
||||
|
||||
second_root = tmp_path_factory.mktemp("second-project-home")
|
||||
|
||||
result = await create_memory_project(
|
||||
project_name="second-project",
|
||||
project_path=str(second_root),
|
||||
set_default=False,
|
||||
)
|
||||
assert result.startswith("✓")
|
||||
|
||||
await write_note(project=test_project.name, title="One", directory="notes", content="one")
|
||||
await write_note(project="second-project", title="Two", directory="notes", content="two")
|
||||
|
||||
out = await recent_activity()
|
||||
assert "Recent Activity Summary" in out
|
||||
assert "or would you prefer a different project" in out
|
||||
|
||||
|
||||
def test_recent_activity_format_relative_time_and_truncate_helpers():
|
||||
|
||||
+15
-116
@@ -12,6 +12,7 @@ import pytest
|
||||
|
||||
from basic_memory.mcp.tools.schema import schema_validate, schema_infer, schema_diff
|
||||
from basic_memory.mcp.tools.write_note import write_note
|
||||
from basic_memory.schemas.schema import ValidationReport, InferenceReport, DriftReport
|
||||
|
||||
|
||||
# --- Helpers ---
|
||||
@@ -81,39 +82,8 @@ async def test_schema_validate_by_type(app, test_project, sync_service):
|
||||
project=test_project.name,
|
||||
)
|
||||
|
||||
assert isinstance(result, str)
|
||||
assert "Schema Validation: person" in result
|
||||
assert "Notes: 1" in result
|
||||
assert "**Alice**" in result
|
||||
assert "valid" in result
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_schema_validate_json_output(app, test_project, sync_service):
|
||||
"""JSON output returns a dict with full structured data."""
|
||||
project_path = Path(test_project.path)
|
||||
|
||||
_write_schema_file(project_path, "schemas/Person.md", PERSON_SCHEMA)
|
||||
_write_schema_file(
|
||||
project_path,
|
||||
"people/Alice.md",
|
||||
PERSON_NOTE.format(name="Alice", permalink="alice"),
|
||||
)
|
||||
|
||||
await sync_service.sync(project_path)
|
||||
|
||||
result = await schema_validate(
|
||||
note_type="person",
|
||||
project=test_project.name,
|
||||
output_format="json",
|
||||
)
|
||||
|
||||
assert isinstance(result, dict)
|
||||
assert result["total_notes"] == 1
|
||||
assert result["valid_count"] == 1
|
||||
assert len(result["results"]) == 1
|
||||
assert result["results"][0]["note_identifier"] == "Alice"
|
||||
assert result["results"][0]["passed"] is True
|
||||
assert isinstance(result, ValidationReport)
|
||||
assert result.total_notes >= 1
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@@ -135,70 +105,8 @@ async def test_schema_validate_by_identifier(app, test_project, sync_service):
|
||||
project=test_project.name,
|
||||
)
|
||||
|
||||
assert isinstance(result, str)
|
||||
assert "**Alice**" in result
|
||||
assert "valid" in result
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_schema_validate_by_title(app, test_project, sync_service):
|
||||
"""Validate a specific note by title (not permalink).
|
||||
|
||||
Regression test for issue #33: schema_validate(identifier="Note Title")
|
||||
returned 0 notes because the router only searched by permalink.
|
||||
"""
|
||||
project_path = Path(test_project.path)
|
||||
|
||||
_write_schema_file(project_path, "schemas/Person.md", PERSON_SCHEMA)
|
||||
_write_schema_file(
|
||||
project_path,
|
||||
"people/Alice.md",
|
||||
PERSON_NOTE.format(name="Alice", permalink="alice"),
|
||||
)
|
||||
|
||||
await sync_service.sync(project_path)
|
||||
|
||||
# Use the title "Alice" instead of the permalink "people/alice"
|
||||
result = await schema_validate(
|
||||
identifier="Alice",
|
||||
project=test_project.name,
|
||||
)
|
||||
|
||||
assert isinstance(result, str)
|
||||
assert "**Alice**" in result
|
||||
assert "Notes: 1" in result
|
||||
assert "valid" in result
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_schema_validate_identifier_no_schema_returns_guidance(
|
||||
app, test_project, sync_service
|
||||
):
|
||||
"""When a note exists but no schema is defined, return guidance.
|
||||
|
||||
Regression test for issue #33: when validating a single note by identifier
|
||||
and no schema exists, the tool should return guidance instead of an empty report.
|
||||
"""
|
||||
project_path = Path(test_project.path)
|
||||
|
||||
# Create a person note but no schema note
|
||||
_write_schema_file(
|
||||
project_path,
|
||||
"people/Alice.md",
|
||||
PERSON_NOTE.format(name="Alice", permalink="alice"),
|
||||
)
|
||||
|
||||
await sync_service.sync(project_path)
|
||||
|
||||
result = await schema_validate(
|
||||
identifier="Alice",
|
||||
project=test_project.name,
|
||||
)
|
||||
|
||||
# Should return guidance string about missing schema
|
||||
assert isinstance(result, str)
|
||||
assert "No Schema Found" in result
|
||||
assert "person" in result
|
||||
assert isinstance(result, ValidationReport)
|
||||
assert result.total_notes >= 1
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@@ -220,12 +128,9 @@ async def test_schema_infer(app, test_project, sync_service):
|
||||
project=test_project.name,
|
||||
)
|
||||
|
||||
assert isinstance(result, str)
|
||||
assert "Schema Inference: person" in result
|
||||
assert "Notes analyzed: 3" in result
|
||||
assert "Field Frequencies" in result
|
||||
assert "**name**" in result
|
||||
assert "**role**" in result
|
||||
assert isinstance(result, InferenceReport)
|
||||
assert result.note_type == "person"
|
||||
assert result.notes_analyzed >= 3
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@@ -262,10 +167,8 @@ permalink: people/dave
|
||||
project=test_project.name,
|
||||
)
|
||||
|
||||
assert isinstance(result, str)
|
||||
assert "Schema Drift: person" in result
|
||||
# Dave has a "hobby" field not in the schema, so drift should be detected
|
||||
assert "**hobby**" in result
|
||||
assert isinstance(result, DriftReport)
|
||||
assert result.note_type == "person"
|
||||
|
||||
|
||||
# --- write_note metadata → schema workflow ---
|
||||
@@ -311,9 +214,8 @@ async def test_write_note_metadata_creates_schema_note(app, test_project, sync_s
|
||||
project=test_project.name,
|
||||
)
|
||||
|
||||
assert isinstance(result, str)
|
||||
assert "Schema Validation: person" in result
|
||||
assert "valid" in result
|
||||
assert isinstance(result, ValidationReport)
|
||||
assert result.total_notes >= 2
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@@ -377,13 +279,10 @@ permalink: employees/{name.lower()}
|
||||
project=test_project.name,
|
||||
)
|
||||
|
||||
assert isinstance(result, str)
|
||||
assert "Schema Validation: employee" in result
|
||||
assert "Notes: 2" in result
|
||||
assert "Valid: 2" in result
|
||||
assert isinstance(result, ValidationReport)
|
||||
assert result.total_notes == 2
|
||||
# Both notes have name + department, schema requires name and optionally department
|
||||
assert "**Alice**" in result
|
||||
assert "**Bob**" in result
|
||||
assert result.valid_count == 2
|
||||
|
||||
|
||||
# --- Empty schema guard ---
|
||||
|
||||
+123
-572
@@ -5,12 +5,8 @@ from contextlib import asynccontextmanager
|
||||
from datetime import datetime, timedelta
|
||||
|
||||
from basic_memory.mcp.tools import write_note
|
||||
from basic_memory.mcp.tools.search import (
|
||||
search_notes,
|
||||
_format_search_error_response,
|
||||
_format_search_markdown,
|
||||
)
|
||||
from basic_memory.schemas.search import SearchResponse
|
||||
from basic_memory.mcp.tools.search import search_notes, _format_search_error_response
|
||||
from basic_memory.schemas.search import SearchItemType, SearchResponse
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@@ -26,18 +22,15 @@ async def test_search_text(client, test_project):
|
||||
)
|
||||
assert result
|
||||
|
||||
# Search for it (use json format to inspect structured results)
|
||||
response = await search_notes(
|
||||
project=test_project.name, query="searchable", output_format="json"
|
||||
)
|
||||
# Search for it
|
||||
response = await search_notes(project=test_project.name, query="searchable")
|
||||
|
||||
# Verify results - handle both success and error cases
|
||||
if isinstance(response, dict):
|
||||
# Success case - verify dict response
|
||||
assert len(response["results"]) > 0
|
||||
if isinstance(response, SearchResponse):
|
||||
# Success case - verify SearchResponse
|
||||
assert len(response.results) > 0
|
||||
assert any(
|
||||
r["permalink"] == f"{test_project.name}/test/test-search-note"
|
||||
for r in response["results"]
|
||||
r.permalink == f"{test_project.name}/test/test-search-note" for r in response.results
|
||||
)
|
||||
else:
|
||||
# If search failed and returned error message, test should fail with informative message
|
||||
@@ -57,22 +50,21 @@ async def test_search_title(client, test_project):
|
||||
)
|
||||
assert result
|
||||
|
||||
# Search for it (use json format to inspect structured results)
|
||||
# Search for it
|
||||
response = await search_notes(
|
||||
project=test_project.name, query="Search Note", search_type="title", output_format="json"
|
||||
project=test_project.name, query="Search Note", search_type="title"
|
||||
)
|
||||
|
||||
# Verify results - handle both success and error cases
|
||||
if isinstance(response, dict):
|
||||
# Success case - verify dict response
|
||||
assert len(response["results"]) > 0
|
||||
assert any(
|
||||
r["permalink"] == f"{test_project.name}/test/test-search-note"
|
||||
for r in response["results"]
|
||||
)
|
||||
else:
|
||||
if isinstance(response, str):
|
||||
# If search failed and returned error message, test should fail with informative message
|
||||
pytest.fail(f"Search failed with error: {response}")
|
||||
else:
|
||||
# Success case - verify SearchResponse
|
||||
assert len(response.results) > 0
|
||||
assert any(
|
||||
r.permalink == f"{test_project.name}/test/test-search-note" for r in response.results
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@@ -88,21 +80,19 @@ async def test_search_permalink(client, test_project):
|
||||
)
|
||||
assert result
|
||||
|
||||
# Search for it (use json format to inspect structured results)
|
||||
# Search for it
|
||||
response = await search_notes(
|
||||
project=test_project.name,
|
||||
query=f"{test_project.name}/test/test-search-note",
|
||||
search_type="permalink",
|
||||
output_format="json",
|
||||
)
|
||||
|
||||
# Verify results - handle both success and error cases
|
||||
if isinstance(response, dict):
|
||||
# Success case - verify dict response
|
||||
assert len(response["results"]) > 0
|
||||
if isinstance(response, SearchResponse):
|
||||
# Success case - verify SearchResponse
|
||||
assert len(response.results) > 0
|
||||
assert any(
|
||||
r["permalink"] == f"{test_project.name}/test/test-search-note"
|
||||
for r in response["results"]
|
||||
r.permalink == f"{test_project.name}/test/test-search-note" for r in response.results
|
||||
)
|
||||
else:
|
||||
# If search failed and returned error message, test should fail with informative message
|
||||
@@ -122,21 +112,19 @@ async def test_search_permalink_match(client, test_project):
|
||||
)
|
||||
assert result
|
||||
|
||||
# Search for it (use json format to inspect structured results)
|
||||
# Search for it
|
||||
response = await search_notes(
|
||||
project=test_project.name,
|
||||
query=f"{test_project.name}/test/test-search-*",
|
||||
search_type="permalink",
|
||||
output_format="json",
|
||||
)
|
||||
|
||||
# Verify results - handle both success and error cases
|
||||
if isinstance(response, dict):
|
||||
# Success case - verify dict response
|
||||
assert len(response["results"]) > 0
|
||||
if isinstance(response, SearchResponse):
|
||||
# Success case - verify SearchResponse
|
||||
assert len(response.results) > 0
|
||||
assert any(
|
||||
r["permalink"] == f"{test_project.name}/test/test-search-note"
|
||||
for r in response["results"]
|
||||
r.permalink == f"{test_project.name}/test/test-search-note" for r in response.results
|
||||
)
|
||||
else:
|
||||
# If search failed and returned error message, test should fail with informative message
|
||||
@@ -154,15 +142,13 @@ async def test_search_memory_url_with_project_prefix(client, test_project):
|
||||
)
|
||||
assert result
|
||||
|
||||
response = await search_notes(
|
||||
query=f"memory://{test_project.name}/test/memory-url-search-note", output_format="json"
|
||||
)
|
||||
response = await search_notes(query=f"memory://{test_project.name}/test/memory-url-search-note")
|
||||
|
||||
if isinstance(response, dict):
|
||||
assert len(response["results"]) > 0
|
||||
if isinstance(response, SearchResponse):
|
||||
assert len(response.results) > 0
|
||||
assert any(
|
||||
r["permalink"] == f"{test_project.name}/test/memory-url-search-note"
|
||||
for r in response["results"]
|
||||
r.permalink == f"{test_project.name}/test/memory-url-search-note"
|
||||
for r in response.results
|
||||
)
|
||||
else:
|
||||
pytest.fail(f"Search failed with error: {response}")
|
||||
@@ -181,18 +167,17 @@ async def test_search_pagination(client, test_project):
|
||||
)
|
||||
assert result
|
||||
|
||||
# Search for it (use json format to inspect structured results)
|
||||
# Search for it
|
||||
response = await search_notes(
|
||||
project=test_project.name, query="searchable", page=1, page_size=1, output_format="json"
|
||||
project=test_project.name, query="searchable", page=1, page_size=1
|
||||
)
|
||||
|
||||
# Verify results - handle both success and error cases
|
||||
if isinstance(response, dict):
|
||||
# Success case - verify dict response
|
||||
assert len(response["results"]) == 1
|
||||
if isinstance(response, SearchResponse):
|
||||
# Success case - verify SearchResponse
|
||||
assert len(response.results) == 1
|
||||
assert any(
|
||||
r["permalink"] == f"{test_project.name}/test/test-search-note"
|
||||
for r in response["results"]
|
||||
r.permalink == f"{test_project.name}/test/test-search-note" for r in response.results
|
||||
)
|
||||
else:
|
||||
# If search failed and returned error message, test should fail with informative message
|
||||
@@ -210,15 +195,13 @@ async def test_search_with_type_filter(client, test_project):
|
||||
content="# Test\nFiltered by type",
|
||||
)
|
||||
|
||||
# Search with note type filter (use json format to inspect structured results)
|
||||
response = await search_notes(
|
||||
project=test_project.name, query="type", note_types=["note"], output_format="json"
|
||||
)
|
||||
# Search with note type filter
|
||||
response = await search_notes(project=test_project.name, query="type", note_types=["note"])
|
||||
|
||||
# Verify results - handle both success and error cases
|
||||
if isinstance(response, dict):
|
||||
if isinstance(response, SearchResponse):
|
||||
# Success case - verify all results are entities
|
||||
assert all(r["type"] == "entity" for r in response["results"])
|
||||
assert all(r.type == "entity" for r in response.results)
|
||||
else:
|
||||
# If search failed and returned error message, test should fail with informative message
|
||||
pytest.fail(f"Search failed with error: {response}")
|
||||
@@ -235,15 +218,13 @@ async def test_search_with_entity_type_filter(client, test_project):
|
||||
content="# Test\nFiltered by type",
|
||||
)
|
||||
|
||||
# Search with entity_types (SearchItemType) filter (use json format)
|
||||
response = await search_notes(
|
||||
project=test_project.name, query="type", entity_types=["entity"], output_format="json"
|
||||
)
|
||||
# Search with entity_types (SearchItemType) filter
|
||||
response = await search_notes(project=test_project.name, query="type", entity_types=["entity"])
|
||||
|
||||
# Verify results - handle both success and error cases
|
||||
if isinstance(response, dict):
|
||||
if isinstance(response, SearchResponse):
|
||||
# Success case - verify all results are entities
|
||||
assert all(r["type"] == "entity" for r in response["results"])
|
||||
assert all(r.type == "entity" for r in response.results)
|
||||
else:
|
||||
# If search failed and returned error message, test should fail with informative message
|
||||
pytest.fail(f"Search failed with error: {response}")
|
||||
@@ -260,19 +241,16 @@ async def test_search_with_date_filter(client, test_project):
|
||||
content="# Test\nRecent content",
|
||||
)
|
||||
|
||||
# Search with date filter (use json format to inspect structured results)
|
||||
# Search with date filter
|
||||
one_hour_ago = datetime.now() - timedelta(hours=1)
|
||||
response = await search_notes(
|
||||
project=test_project.name,
|
||||
query="recent",
|
||||
after_date=one_hour_ago.isoformat(),
|
||||
output_format="json",
|
||||
project=test_project.name, query="recent", after_date=one_hour_ago.isoformat()
|
||||
)
|
||||
|
||||
# Verify results - handle both success and error cases
|
||||
if isinstance(response, dict):
|
||||
if isinstance(response, SearchResponse):
|
||||
# Success case - verify we get results within timeframe
|
||||
assert len(response["results"]) > 0
|
||||
assert len(response.results) > 0
|
||||
else:
|
||||
# If search failed and returned error message, test should fail with informative message
|
||||
pytest.fail(f"Search failed with error: {response}")
|
||||
@@ -490,8 +468,7 @@ async def test_search_notes_sets_retrieval_mode_for_semantic_types(monkeypatch,
|
||||
search_type=search_type,
|
||||
)
|
||||
|
||||
# Default text format returns a formatted string for empty results
|
||||
assert isinstance(result, str)
|
||||
assert isinstance(result, SearchResponse)
|
||||
assert captured_payload["text"] == "semantic lookup"
|
||||
# "semantic" is an alias for "vector" retrieval mode
|
||||
expected_mode = "vector" if search_type in ("vector", "semantic") else search_type
|
||||
@@ -549,12 +526,14 @@ async def test_search_notes_passes_metadata_filters(monkeypatch):
|
||||
assert captured_payload["status"] == "published"
|
||||
|
||||
|
||||
# --- Tests for filter-only search (query=None) --------------------------------
|
||||
# --- Tests for search_by_metadata tool (lines 505-556) ---------------------
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_search_notes_filter_only_metadata(monkeypatch):
|
||||
"""search_notes with metadata_filters only (no query) sends correct payload."""
|
||||
async def test_search_by_metadata_basic(monkeypatch):
|
||||
"""search_by_metadata calls SearchClient with correct structured query."""
|
||||
from basic_memory.mcp.tools.search import search_by_metadata
|
||||
|
||||
import importlib
|
||||
|
||||
search_mod = importlib.import_module("basic_memory.mcp.tools.search")
|
||||
@@ -581,23 +560,37 @@ async def test_search_notes_filter_only_metadata(monkeypatch):
|
||||
monkeypatch.setattr(search_mod, "get_project_client", fake_get_project_client)
|
||||
monkeypatch.setattr(clients_mod, "SearchClient", MockSearchClient)
|
||||
|
||||
result = await search_mod.search_notes(
|
||||
result = await search_by_metadata(
|
||||
filters={"status": "in-progress"},
|
||||
project="test-project",
|
||||
metadata_filters={"status": "in-progress"},
|
||||
limit=10,
|
||||
offset=0,
|
||||
)
|
||||
|
||||
# Default text format returns a formatted string for empty results
|
||||
assert isinstance(result, str)
|
||||
assert isinstance(result, SearchResponse)
|
||||
assert captured_payload["metadata_filters"] == {"status": "in-progress"}
|
||||
# No text/title/permalink should be set
|
||||
assert captured_payload.get("text") is None
|
||||
assert captured_payload.get("title") is None
|
||||
assert captured_payload.get("permalink") is None
|
||||
assert captured_payload["entity_types"] == ["entity"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_search_notes_filter_only_tags(monkeypatch):
|
||||
"""search_notes with tags only (no query) sends correct payload."""
|
||||
async def test_search_by_metadata_limit_zero():
|
||||
"""search_by_metadata rejects limit <= 0 with error string."""
|
||||
from basic_memory.mcp.tools.search import search_by_metadata
|
||||
|
||||
result = await search_by_metadata(
|
||||
filters={"status": "active"},
|
||||
limit=0,
|
||||
)
|
||||
|
||||
assert isinstance(result, str)
|
||||
assert "limit" in result.lower()
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_search_by_metadata_offset_within_page(monkeypatch):
|
||||
"""When offset doesn't align to page boundary, results are trimmed."""
|
||||
from basic_memory.mcp.tools.search import search_by_metadata
|
||||
|
||||
import importlib
|
||||
|
||||
search_mod = importlib.import_module("basic_memory.mcp.tools.search")
|
||||
@@ -611,36 +604,54 @@ async def test_search_notes_filter_only_tags(monkeypatch):
|
||||
async def fake_get_project_client(*args, **kwargs):
|
||||
yield (object(), StubProject())
|
||||
|
||||
captured_payload: dict = {}
|
||||
from basic_memory.schemas.search import SearchResult
|
||||
|
||||
fake_items = [
|
||||
SearchResult(
|
||||
title=f"Item {i}",
|
||||
permalink=f"item-{i}",
|
||||
file_path=f"item-{i}.md",
|
||||
type=SearchItemType.ENTITY,
|
||||
score=1.0 - i * 0.1,
|
||||
)
|
||||
for i in range(5)
|
||||
]
|
||||
|
||||
class MockSearchClient:
|
||||
def __init__(self, *args, **kwargs):
|
||||
pass
|
||||
self.call_count = 0
|
||||
|
||||
async def search(self, payload, page, page_size):
|
||||
captured_payload.update(payload)
|
||||
self.call_count += 1
|
||||
if page == 1:
|
||||
return SearchResponse(results=fake_items, current_page=1, page_size=page_size)
|
||||
return SearchResponse(results=[], current_page=page, page_size=page_size)
|
||||
|
||||
monkeypatch.setattr(search_mod, "get_project_client", fake_get_project_client)
|
||||
monkeypatch.setattr(clients_mod, "SearchClient", MockSearchClient)
|
||||
|
||||
result = await search_mod.search_notes(
|
||||
# offset=2, limit=5 → page=1, offset_within_page=2
|
||||
result = await search_by_metadata(
|
||||
filters={"status": "active"},
|
||||
project="test-project",
|
||||
tags=["security", "oauth"],
|
||||
limit=5,
|
||||
offset=2,
|
||||
)
|
||||
|
||||
# Default text format returns a formatted string for empty results
|
||||
assert isinstance(result, str)
|
||||
assert captured_payload["tags"] == ["security", "oauth"]
|
||||
assert captured_payload.get("text") is None
|
||||
assert isinstance(result, SearchResponse)
|
||||
# Should have sliced off the first 2 items
|
||||
assert result.results[0].title == "Item 2"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_search_notes_no_criteria_returns_error(monkeypatch):
|
||||
"""search_notes with no args at all returns a helpful error string."""
|
||||
async def test_search_by_metadata_error_handling(monkeypatch):
|
||||
"""search_by_metadata returns error string on exception."""
|
||||
from basic_memory.mcp.tools.search import search_by_metadata
|
||||
|
||||
import importlib
|
||||
|
||||
search_mod = importlib.import_module("basic_memory.mcp.tools.search")
|
||||
clients_mod = importlib.import_module("basic_memory.mcp.clients")
|
||||
|
||||
class StubProject:
|
||||
name = "test-project"
|
||||
@@ -650,12 +661,23 @@ async def test_search_notes_no_criteria_returns_error(monkeypatch):
|
||||
async def fake_get_project_client(*args, **kwargs):
|
||||
yield (object(), StubProject())
|
||||
|
||||
monkeypatch.setattr(search_mod, "get_project_client", fake_get_project_client)
|
||||
class MockSearchClient:
|
||||
def __init__(self, *args, **kwargs):
|
||||
pass
|
||||
|
||||
result = await search_mod.search_notes(project="test-project")
|
||||
async def search(self, *args, **kwargs):
|
||||
raise RuntimeError("database connection lost")
|
||||
|
||||
monkeypatch.setattr(search_mod, "get_project_client", fake_get_project_client)
|
||||
monkeypatch.setattr(clients_mod, "SearchClient", MockSearchClient)
|
||||
|
||||
result = await search_by_metadata(
|
||||
filters={"status": "active"},
|
||||
project="test-project",
|
||||
)
|
||||
|
||||
assert isinstance(result, str)
|
||||
assert "No Search Criteria" in result
|
||||
assert "Search Failed" in result
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@@ -1048,474 +1070,3 @@ async def test_search_notes_defaults_to_fts_when_container_not_initialized_and_s
|
||||
|
||||
assert captured_payload["retrieval_mode"] == "fts"
|
||||
assert captured_payload["text"] == "test query"
|
||||
|
||||
|
||||
# --- Tests for default entity_types (issue #31) --------------------------------
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_search_notes_defaults_entity_types_to_entity(monkeypatch):
|
||||
"""search_notes defaults entity_types to ['entity'] when not explicitly provided.
|
||||
|
||||
This prevents individual observations/relations from appearing as separate
|
||||
search results, since the entity row already indexes full file content.
|
||||
"""
|
||||
import importlib
|
||||
|
||||
search_mod = importlib.import_module("basic_memory.mcp.tools.search")
|
||||
clients_mod = importlib.import_module("basic_memory.mcp.clients")
|
||||
|
||||
class StubProject:
|
||||
name = "test-project"
|
||||
external_id = "test-external-id"
|
||||
|
||||
@asynccontextmanager
|
||||
async def fake_get_project_client(*args, **kwargs):
|
||||
yield (object(), StubProject())
|
||||
|
||||
async def fake_resolve_project_and_path(
|
||||
client, identifier, project=None, context=None, headers=None
|
||||
):
|
||||
return StubProject(), identifier, False
|
||||
|
||||
captured_payload: dict = {}
|
||||
|
||||
class MockSearchClient:
|
||||
def __init__(self, *args, **kwargs):
|
||||
pass
|
||||
|
||||
async def search(self, payload, page, page_size):
|
||||
captured_payload.update(payload)
|
||||
return SearchResponse(results=[], current_page=page, page_size=page_size)
|
||||
|
||||
monkeypatch.setattr(search_mod, "get_project_client", fake_get_project_client)
|
||||
monkeypatch.setattr(search_mod, "resolve_project_and_path", fake_resolve_project_and_path)
|
||||
monkeypatch.setattr(clients_mod, "SearchClient", MockSearchClient)
|
||||
|
||||
await search_mod.search_notes(
|
||||
project="test-project",
|
||||
query="test",
|
||||
)
|
||||
|
||||
# entity_types should default to ["entity"]
|
||||
assert captured_payload["entity_types"] == ["entity"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_search_notes_explicit_entity_types_overrides_default(monkeypatch):
|
||||
"""Explicit entity_types parameter overrides the default ['entity'] filter."""
|
||||
import importlib
|
||||
|
||||
search_mod = importlib.import_module("basic_memory.mcp.tools.search")
|
||||
clients_mod = importlib.import_module("basic_memory.mcp.clients")
|
||||
|
||||
class StubProject:
|
||||
name = "test-project"
|
||||
external_id = "test-external-id"
|
||||
|
||||
@asynccontextmanager
|
||||
async def fake_get_project_client(*args, **kwargs):
|
||||
yield (object(), StubProject())
|
||||
|
||||
async def fake_resolve_project_and_path(
|
||||
client, identifier, project=None, context=None, headers=None
|
||||
):
|
||||
return StubProject(), identifier, False
|
||||
|
||||
captured_payload: dict = {}
|
||||
|
||||
class MockSearchClient:
|
||||
def __init__(self, *args, **kwargs):
|
||||
pass
|
||||
|
||||
async def search(self, payload, page, page_size):
|
||||
captured_payload.update(payload)
|
||||
return SearchResponse(results=[], current_page=page, page_size=page_size)
|
||||
|
||||
monkeypatch.setattr(search_mod, "get_project_client", fake_get_project_client)
|
||||
monkeypatch.setattr(search_mod, "resolve_project_and_path", fake_resolve_project_and_path)
|
||||
monkeypatch.setattr(clients_mod, "SearchClient", MockSearchClient)
|
||||
|
||||
await search_mod.search_notes(
|
||||
project="test-project",
|
||||
query="test",
|
||||
entity_types=["observation"],
|
||||
)
|
||||
|
||||
# Explicit entity_types should be used, not the default
|
||||
assert captured_payload["entity_types"] == ["observation"]
|
||||
|
||||
|
||||
# --- Tests for tag: prefix parsing (issue #30) ---------------------------------
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_search_notes_tag_prefix_converts_to_tags_filter(monkeypatch):
|
||||
"""query='tag:security' should be converted to a tags filter with no text query."""
|
||||
import importlib
|
||||
|
||||
search_mod = importlib.import_module("basic_memory.mcp.tools.search")
|
||||
clients_mod = importlib.import_module("basic_memory.mcp.clients")
|
||||
|
||||
class StubProject:
|
||||
name = "test-project"
|
||||
external_id = "test-external-id"
|
||||
|
||||
@asynccontextmanager
|
||||
async def fake_get_project_client(*args, **kwargs):
|
||||
yield (object(), StubProject())
|
||||
|
||||
captured_payload: dict = {}
|
||||
|
||||
class MockSearchClient:
|
||||
def __init__(self, *args, **kwargs):
|
||||
pass
|
||||
|
||||
async def search(self, payload, page, page_size):
|
||||
captured_payload.update(payload)
|
||||
return SearchResponse(results=[], current_page=page, page_size=page_size)
|
||||
|
||||
monkeypatch.setattr(search_mod, "get_project_client", fake_get_project_client)
|
||||
monkeypatch.setattr(clients_mod, "SearchClient", MockSearchClient)
|
||||
|
||||
result = await search_mod.search_notes(
|
||||
project="test-project",
|
||||
query="tag:security",
|
||||
)
|
||||
|
||||
# Default text format returns a formatted string for empty results
|
||||
assert isinstance(result, str)
|
||||
assert captured_payload["tags"] == ["security"]
|
||||
# No text query should be set — tag: prefix was consumed
|
||||
assert captured_payload.get("text") is None
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_search_notes_tag_prefix_merges_with_explicit_tags(monkeypatch):
|
||||
"""query='tag:security' with tags=['oauth'] should merge both tag values."""
|
||||
import importlib
|
||||
|
||||
search_mod = importlib.import_module("basic_memory.mcp.tools.search")
|
||||
clients_mod = importlib.import_module("basic_memory.mcp.clients")
|
||||
|
||||
class StubProject:
|
||||
name = "test-project"
|
||||
external_id = "test-external-id"
|
||||
|
||||
@asynccontextmanager
|
||||
async def fake_get_project_client(*args, **kwargs):
|
||||
yield (object(), StubProject())
|
||||
|
||||
captured_payload: dict = {}
|
||||
|
||||
class MockSearchClient:
|
||||
def __init__(self, *args, **kwargs):
|
||||
pass
|
||||
|
||||
async def search(self, payload, page, page_size):
|
||||
captured_payload.update(payload)
|
||||
return SearchResponse(results=[], current_page=page, page_size=page_size)
|
||||
|
||||
monkeypatch.setattr(search_mod, "get_project_client", fake_get_project_client)
|
||||
monkeypatch.setattr(clients_mod, "SearchClient", MockSearchClient)
|
||||
|
||||
result = await search_mod.search_notes(
|
||||
project="test-project",
|
||||
query="tag:security",
|
||||
tags=["oauth"],
|
||||
)
|
||||
|
||||
# Default text format returns a formatted string for empty results
|
||||
assert isinstance(result, str)
|
||||
assert set(captured_payload["tags"]) == {"security", "oauth"}
|
||||
assert captured_payload.get("text") is None
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_search_notes_multiple_tag_prefixes(monkeypatch):
|
||||
"""query='tag:coffee AND tag:brewing' should extract both tags."""
|
||||
import importlib
|
||||
|
||||
search_mod = importlib.import_module("basic_memory.mcp.tools.search")
|
||||
clients_mod = importlib.import_module("basic_memory.mcp.clients")
|
||||
|
||||
class StubProject:
|
||||
name = "test-project"
|
||||
external_id = "test-external-id"
|
||||
|
||||
@asynccontextmanager
|
||||
async def fake_get_project_client(*args, **kwargs):
|
||||
yield (object(), StubProject())
|
||||
|
||||
captured_payload: dict = {}
|
||||
|
||||
class MockSearchClient:
|
||||
def __init__(self, *args, **kwargs):
|
||||
pass
|
||||
|
||||
async def search(self, payload, page, page_size):
|
||||
captured_payload.update(payload)
|
||||
return SearchResponse(results=[], current_page=page, page_size=page_size)
|
||||
|
||||
monkeypatch.setattr(search_mod, "get_project_client", fake_get_project_client)
|
||||
monkeypatch.setattr(clients_mod, "SearchClient", MockSearchClient)
|
||||
|
||||
result = await search_mod.search_notes(
|
||||
project="test-project",
|
||||
query="tag:coffee AND tag:brewing",
|
||||
)
|
||||
|
||||
# Default text format returns a formatted string for empty results
|
||||
assert isinstance(result, str)
|
||||
assert set(captured_payload["tags"]) == {"coffee", "brewing"}
|
||||
# Boolean connector AND should be stripped, leaving no text query
|
||||
assert captured_payload.get("text") is None
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_search_notes_tag_prefix_with_remaining_text(monkeypatch):
|
||||
"""query='authentication tag:security' should keep text and extract tag."""
|
||||
import importlib
|
||||
|
||||
search_mod = importlib.import_module("basic_memory.mcp.tools.search")
|
||||
clients_mod = importlib.import_module("basic_memory.mcp.clients")
|
||||
|
||||
class StubProject:
|
||||
name = "test-project"
|
||||
external_id = "test-external-id"
|
||||
|
||||
@asynccontextmanager
|
||||
async def fake_get_project_client(*args, **kwargs):
|
||||
yield (object(), StubProject())
|
||||
|
||||
captured_payload: dict = {}
|
||||
|
||||
class MockSearchClient:
|
||||
def __init__(self, *args, **kwargs):
|
||||
pass
|
||||
|
||||
async def search(self, payload, page, page_size):
|
||||
captured_payload.update(payload)
|
||||
return SearchResponse(results=[], current_page=page, page_size=page_size)
|
||||
|
||||
# Remaining text query triggers resolve_project_and_path, so stub it too
|
||||
async def fake_resolve(client, query, project, context):
|
||||
return project, query, False
|
||||
|
||||
monkeypatch.setattr(search_mod, "get_project_client", fake_get_project_client)
|
||||
monkeypatch.setattr(search_mod, "resolve_project_and_path", fake_resolve)
|
||||
monkeypatch.setattr(clients_mod, "SearchClient", MockSearchClient)
|
||||
|
||||
result = await search_mod.search_notes(
|
||||
project="test-project",
|
||||
query="authentication tag:security",
|
||||
)
|
||||
|
||||
# Default text format returns a formatted string for empty results
|
||||
assert isinstance(result, str)
|
||||
assert captured_payload["tags"] == ["security"]
|
||||
# Remaining text should be preserved as the query
|
||||
assert captured_payload["text"] == "authentication"
|
||||
|
||||
|
||||
# --- Tests for text output format (#641) -----------------------------------
|
||||
|
||||
|
||||
def test_format_search_markdown_with_results():
|
||||
"""_format_search_markdown returns readable markdown for non-empty results."""
|
||||
from basic_memory.schemas.search import SearchResult, SearchItemType
|
||||
|
||||
result = SearchResponse(
|
||||
results=[
|
||||
SearchResult(
|
||||
title="My Note",
|
||||
type=SearchItemType.ENTITY,
|
||||
score=0.85,
|
||||
permalink="docs/my-note",
|
||||
file_path="docs/My Note.md",
|
||||
matched_chunk="This is a matching snippet",
|
||||
),
|
||||
SearchResult(
|
||||
title="Other Note",
|
||||
type=SearchItemType.ENTITY,
|
||||
score=0.42,
|
||||
permalink="docs/other-note",
|
||||
file_path="docs/Other Note.md",
|
||||
),
|
||||
],
|
||||
current_page=1,
|
||||
page_size=10,
|
||||
)
|
||||
|
||||
text = _format_search_markdown(result, "test-project", "my query")
|
||||
assert isinstance(text, str)
|
||||
assert "# Search Results: my query" in text
|
||||
assert "test-project" in text
|
||||
assert "### My Note" in text
|
||||
assert "permalink: docs/my-note" in text
|
||||
assert "0.8500" in text
|
||||
assert "match: This is a matching snippet" in text
|
||||
assert "### Other Note" in text
|
||||
assert "2 results" in text
|
||||
assert "page 1" in text
|
||||
|
||||
|
||||
def test_format_search_markdown_empty_results():
|
||||
"""_format_search_markdown returns a no-results message when results are empty."""
|
||||
result = SearchResponse(results=[], current_page=1, page_size=10)
|
||||
text = _format_search_markdown(result, "test-project", "missing")
|
||||
assert isinstance(text, str)
|
||||
assert "No results found" in text
|
||||
assert "missing" in text
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_search_notes_text_format_returns_string(monkeypatch):
|
||||
"""search_notes with output_format='text' returns a formatted markdown string."""
|
||||
import importlib
|
||||
|
||||
from basic_memory.schemas.search import SearchResult, SearchItemType
|
||||
|
||||
search_mod = importlib.import_module("basic_memory.mcp.tools.search")
|
||||
clients_mod = importlib.import_module("basic_memory.mcp.clients")
|
||||
|
||||
class StubProject:
|
||||
name = "test-project"
|
||||
external_id = "test-external-id"
|
||||
|
||||
@asynccontextmanager
|
||||
async def fake_get_project_client(*args, **kwargs):
|
||||
yield (object(), StubProject())
|
||||
|
||||
async def fake_resolve_project_and_path(
|
||||
client, identifier, project=None, context=None, headers=None
|
||||
):
|
||||
return StubProject(), identifier, False
|
||||
|
||||
class MockSearchClient:
|
||||
def __init__(self, *args, **kwargs):
|
||||
pass
|
||||
|
||||
async def search(self, payload, page, page_size):
|
||||
return SearchResponse(
|
||||
results=[
|
||||
SearchResult(
|
||||
title="Found Note",
|
||||
type=SearchItemType.ENTITY,
|
||||
score=0.9,
|
||||
permalink="docs/found-note",
|
||||
file_path="docs/Found Note.md",
|
||||
matched_chunk="snippet",
|
||||
),
|
||||
],
|
||||
current_page=page,
|
||||
page_size=page_size,
|
||||
)
|
||||
|
||||
monkeypatch.setattr(search_mod, "get_project_client", fake_get_project_client)
|
||||
monkeypatch.setattr(search_mod, "resolve_project_and_path", fake_resolve_project_and_path)
|
||||
monkeypatch.setattr(clients_mod, "SearchClient", MockSearchClient)
|
||||
|
||||
result = await search_mod.search_notes(
|
||||
project="test-project",
|
||||
query="test",
|
||||
output_format="text",
|
||||
)
|
||||
|
||||
assert isinstance(result, str)
|
||||
assert "# Search Results: test" in result
|
||||
assert "### Found Note" in result
|
||||
assert "permalink: docs/found-note" in result
|
||||
|
||||
|
||||
# --- Tests for metadata_filters key aliasing (#642) ----------------------------
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_search_notes_metadata_filters_aliases_note_type(monkeypatch):
|
||||
"""metadata_filters={'note_type': 'note'} is aliased to {'type': 'note'}."""
|
||||
import importlib
|
||||
|
||||
search_mod = importlib.import_module("basic_memory.mcp.tools.search")
|
||||
clients_mod = importlib.import_module("basic_memory.mcp.clients")
|
||||
|
||||
class StubProject:
|
||||
name = "test-project"
|
||||
external_id = "test-external-id"
|
||||
|
||||
@asynccontextmanager
|
||||
async def fake_get_project_client(*args, **kwargs):
|
||||
yield (object(), StubProject())
|
||||
|
||||
async def fake_resolve_project_and_path(
|
||||
client, identifier, project=None, context=None, headers=None
|
||||
):
|
||||
return StubProject(), identifier, False
|
||||
|
||||
captured_payload: dict = {}
|
||||
|
||||
class MockSearchClient:
|
||||
def __init__(self, *args, **kwargs):
|
||||
pass
|
||||
|
||||
async def search(self, payload, page, page_size):
|
||||
captured_payload.update(payload)
|
||||
return SearchResponse(results=[], current_page=page, page_size=page_size)
|
||||
|
||||
monkeypatch.setattr(search_mod, "get_project_client", fake_get_project_client)
|
||||
monkeypatch.setattr(search_mod, "resolve_project_and_path", fake_resolve_project_and_path)
|
||||
monkeypatch.setattr(clients_mod, "SearchClient", MockSearchClient)
|
||||
|
||||
await search_mod.search_notes(
|
||||
project="test-project",
|
||||
query="test",
|
||||
metadata_filters={"note_type": "note"},
|
||||
)
|
||||
|
||||
# "note_type" should be aliased to "type" in the payload
|
||||
assert captured_payload["metadata_filters"] == {"type": "note"}
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_search_notes_metadata_filters_preserves_non_aliased_keys(monkeypatch):
|
||||
"""metadata_filters with non-aliased keys pass through unchanged."""
|
||||
import importlib
|
||||
|
||||
search_mod = importlib.import_module("basic_memory.mcp.tools.search")
|
||||
clients_mod = importlib.import_module("basic_memory.mcp.clients")
|
||||
|
||||
class StubProject:
|
||||
name = "test-project"
|
||||
external_id = "test-external-id"
|
||||
|
||||
@asynccontextmanager
|
||||
async def fake_get_project_client(*args, **kwargs):
|
||||
yield (object(), StubProject())
|
||||
|
||||
async def fake_resolve_project_and_path(
|
||||
client, identifier, project=None, context=None, headers=None
|
||||
):
|
||||
return StubProject(), identifier, False
|
||||
|
||||
captured_payload: dict = {}
|
||||
|
||||
class MockSearchClient:
|
||||
def __init__(self, *args, **kwargs):
|
||||
pass
|
||||
|
||||
async def search(self, payload, page, page_size):
|
||||
captured_payload.update(payload)
|
||||
return SearchResponse(results=[], current_page=page, page_size=page_size)
|
||||
|
||||
monkeypatch.setattr(search_mod, "get_project_client", fake_get_project_client)
|
||||
monkeypatch.setattr(search_mod, "resolve_project_and_path", fake_resolve_project_and_path)
|
||||
monkeypatch.setattr(clients_mod, "SearchClient", MockSearchClient)
|
||||
|
||||
await search_mod.search_notes(
|
||||
project="test-project",
|
||||
query="test",
|
||||
metadata_filters={"note_type": "spec", "priority": "high"},
|
||||
)
|
||||
|
||||
# "note_type" aliased to "type", "priority" passes through unchanged
|
||||
assert captured_payload["metadata_filters"] == {"type": "spec", "priority": "high"}
|
||||
|
||||
@@ -2,7 +2,6 @@
|
||||
|
||||
import pytest
|
||||
|
||||
from basic_memory.mcp.project_context import get_available_workspaces, set_workspace_provider
|
||||
from basic_memory.mcp.tools.workspaces import list_workspaces
|
||||
from basic_memory.schemas.cloud import WorkspaceInfo
|
||||
|
||||
@@ -106,89 +105,3 @@ async def test_list_workspaces_uses_context_cache_path(monkeypatch):
|
||||
assert "# Available Workspaces (1)" in first
|
||||
assert "# Available Workspaces (1)" in second
|
||||
assert call_count["fetches"] == 1
|
||||
|
||||
|
||||
# --- Workspace provider injection tests ---
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def _reset_workspace_provider(monkeypatch):
|
||||
"""Ensure _workspace_provider is reset after each test."""
|
||||
import basic_memory.mcp.project_context as _mod
|
||||
|
||||
monkeypatch.setattr(_mod, "_workspace_provider", None)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.usefixtures("_reset_workspace_provider")
|
||||
async def test_get_available_workspaces_uses_provider_when_set():
|
||||
"""When a workspace provider is injected, it is called instead of the control-plane client."""
|
||||
expected = [
|
||||
WorkspaceInfo(
|
||||
tenant_id="aaaa-bbbb",
|
||||
workspace_type="personal",
|
||||
name="Injected",
|
||||
role="owner",
|
||||
),
|
||||
]
|
||||
|
||||
async def fake_provider() -> list[WorkspaceInfo]:
|
||||
return expected
|
||||
|
||||
set_workspace_provider(fake_provider)
|
||||
|
||||
result = await get_available_workspaces()
|
||||
assert len(result) == 1
|
||||
assert result[0].tenant_id == "aaaa-bbbb"
|
||||
assert result[0].name == "Injected"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.usefixtures("_reset_workspace_provider")
|
||||
async def test_get_available_workspaces_falls_back_without_provider(monkeypatch):
|
||||
"""Without a provider, get_available_workspaces uses the control-plane client (existing path)."""
|
||||
called = {"control_plane": False}
|
||||
|
||||
async def fake_control_plane_path(context=None):
|
||||
called["control_plane"] = True
|
||||
return []
|
||||
|
||||
# Patch the entire function to avoid needing real credentials
|
||||
monkeypatch.setattr(
|
||||
"basic_memory.mcp.tools.workspaces.get_available_workspaces",
|
||||
fake_control_plane_path,
|
||||
)
|
||||
|
||||
result = await list_workspaces()
|
||||
assert called["control_plane"]
|
||||
assert "# No Workspaces Available" in result
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.usefixtures("_reset_workspace_provider")
|
||||
async def test_get_available_workspaces_provider_caches_in_context():
|
||||
"""Provider results are cached in the MCP context for subsequent calls."""
|
||||
call_count = {"provider": 0}
|
||||
workspace = WorkspaceInfo(
|
||||
tenant_id="cccc-dddd",
|
||||
workspace_type="organization",
|
||||
name="Cached Provider",
|
||||
role="editor",
|
||||
)
|
||||
|
||||
async def counting_provider() -> list[WorkspaceInfo]:
|
||||
call_count["provider"] += 1
|
||||
return [workspace]
|
||||
|
||||
set_workspace_provider(counting_provider)
|
||||
context = _ContextState()
|
||||
|
||||
# First call: provider is invoked, result cached
|
||||
first = await get_available_workspaces(context=context)
|
||||
assert len(first) == 1
|
||||
assert call_count["provider"] == 1
|
||||
|
||||
# Second call: served from context cache, provider not called again
|
||||
second = await get_available_workspaces(context=context)
|
||||
assert len(second) == 1
|
||||
assert call_count["provider"] == 1
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user