Compare commits

...

5 Commits

Author SHA1 Message Date
phernandez 0bce4be1a6 chore: update version to 0.19.0 for v0.19.0 release 2026-03-07 14:27:43 -06:00
phernandez a316424edf docs: add v0.19.0 changelog entry
Comprehensive changelog for 114 commits since v0.18.5 covering semantic
vector search, schema system, per-project cloud routing, FastMCP 3.0
upgrade, CLI overhaul, and numerous bug fixes.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: phernandez <paul@basicmachines.co>
2026-03-07 14:27:29 -06:00
phernandez af71cf4896 fix(test): clear search_vector_chunks before embedding backfill test
Test was polluted by other tests leaving rows in search_vector_chunks,
causing _needs_semantic_embedding_backfill to return False.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: phernandez <paul@basicmachines.co>
2026-03-07 13:54:58 -06:00
phernandez e846ae85d8 fix: semantic embeddings not generated on fresh DB or upgrade
The previous backfill trigger relied on Alembic revision tracking, but
alembic_version only stores the head revision — intermediate revisions
(like the backfill trigger) are invisible after a multi-step upgrade or
fresh DB creation.

Three changes fix this:

1. Replace Alembic revision check with a simple "entities exist but
   embeddings are empty" check that works regardless of migration path
2. Generate embeddings during sync — after FTS indexing, batch-embed all
   synced entities at the end of the sync operation
3. Add background backfill at MCP startup for the upgrade path (entities
   already exist, no embeddings) without blocking server readiness

Also adds clear startup logging for semantic embedding status so issues
are easy to spot in the logs.

📋 Covers: fresh DB, upgrade from pre-embedding version, db reset,
   interrupted backfill

Signed-off-by: Pedro Hernandez <pedro@basicmachines.co>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: phernandez <paul@basicmachines.co>
2026-03-07 13:23:39 -06:00
phernandez 63e4bcdf1d fix: clarify search_notes parameter naming and fix note_types case sensitivity
- Add Annotated descriptions to note_types and entity_types parameters so
  LLMs can distinguish frontmatter type filtering from knowledge graph item
  type filtering (search.py, ui_sdk.py)
- Lowercase note_types values at filter time so "Chapter" matches stored
  "chapter"
- Fix misleading entity_types references in schema.py guidance strings
  (should be note_types)
- Add permalink pattern documentation note about full path matching
- Add test for note_types case-insensitive lowercasing

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: phernandez <paul@basicmachines.co>
2026-03-05 15:08:52 -06:00
13 changed files with 418 additions and 98 deletions
+130 -4
View File
@@ -2,13 +2,139 @@
## Unreleased
## v0.19.0 (2026-03-07)
### Highlights
- **Semantic vector search** for SQLite and Postgres with FastEmbed embeddings
- **Schema system** for validating and inferring knowledge base structure
- **Per-project cloud routing** with API key authentication
- **Upgraded to FastMCP 3.0** with tool annotations
- **CLI overhaul** with JSON output, workspace awareness, and project dashboard
### Features
- **#550**: Add semantic vector search for SQLite and Postgres
- FastEmbed-based embeddings with automatic backfill
- Hybrid search combining full-text and vector similarity
- Score-based fusion replacing RRF for better ranking
- `min_similarity` override for tuning search precision
- Semantic dependencies are now default, with optional extras fallback
- **#549**: Schema system for Basic Memory
- `schema_infer` — infer schema from existing notes
- `schema_validate` — validate notes against a schema definition
- `schema_diff` — compare schemas across projects
- Frontmatter validation support (#597)
- Read schema definitions from file instead of stale DB metadata (#635)
- **#555**: Per-project local/cloud routing with API key auth
- Individual projects route through cloud while others stay local
- `basic-memory cloud set-key` and `basic-memory project set-cloud/set-local`
- Stdio MCP honors per-project cloud routing (#590)
- **#598**: Upgrade FastMCP 2.12.3 to 3.0.1 with tool annotations
- **#585**: Add JSON output mode for MCP tools (default text)
- `--json` output for CLI commands for scripting and CI
- **#576**: Add workspace selection flow for MCP and CLI
- Workspace-aware cloud project listing
- CLI refactoring for workspace support
- **#544**: Project-prefixed permalinks and memory URL routing
- **#632**: Add overwrite guard to `write_note` tool
- **#614**: `edit_note` append/prepend auto-creates note if not found
- **#609**: Richer content context in search results
- Return matched chunk text in search results (#601)
- Improved content hit rate
- **#602**: Add `created_by` and `last_updated_by` user tracking to Entity
- **#600**: Rename `entity_type` to `note_type` across codebase
- **#574**: Add `display_name` and `is_private` to ProjectItem
- **#569**: Expose `external_id` in EntityResponse and link resolver
- **#567**: Isolate default SQLite DB by config dir
- **#560**: Enable `default_project_mode` by default
- **#559**: Add `basic-memory watch` CLI command
- **#546**: Add cloud discovery touchpoints to CLI and MCP
- **#572**: CLI analytics via Umami event collector
- Replace project info with htop-inspired dashboard
- Merge `search_by_metadata` into `search_notes` with optional query
- Add `--strip-frontmatter` to `basic-memory tool read-note`
- Default behavior is unchanged: `content` still includes raw markdown with frontmatter.
- With `--strip-frontmatter`, both text and JSON modes return body-only markdown content.
- JSON output now includes an additive `frontmatter` field with parsed YAML metadata (or `null`
when no valid opening frontmatter block exists).
- Add `destination_folder` parameter to `move_note` tool
### Bug Fixes
- **#644**: Fix default project resolution in cloud mode
- ChatGPT search/fetch tools broken in cloud mode
- `resolve_project_parameter` falls back to projects API
- **#638**: Restore API backward compatibility for v0.18.x clients
- **#637**: Create backup before config migration overwrites old format
- **#636**: `list_workspaces` bypasses factory pattern on cloud MCP server
- **#631**: `build_context` related_results schema validation failure
- **#613**: Reduce excessive log volume by demoting per-request noise to DEBUG
- **#612**: Handle quoted picoschema enum strings in YAML frontmatter
- **#607**: Guard against closed streams in promo and missing vector tables
- **#606**: Accept null for `expected_replacements` in `edit_note`
- **#595**: `recent_activity` dedup and pagination across MCP tools
- **#593**: Backend-specific distance-to-similarity conversion
- **#582**: Use LinkResolver fallback in `build_context` for flexible identifier matching
- **#577**: Replace RRF with score-based fusion in hybrid search
- **#575**: Remove hardcoded "main" default from `default_project`
- **#534**: Speed up `bm --version` startup
- Fix semantic embeddings not generated on fresh DB or upgrade
- Clarify `search_notes` parameter naming and fix `note_types` case sensitivity
- Parse `tag:` prefix at MCP tool level to avoid hybrid search failure
- Cap sqlite-vec knn k parameter at 4096 limit
- Parameterize SQL queries in search repository type filters
- Coerce list frontmatter values to strings for title and type fields
- Avoid `Post(**metadata)` crash when frontmatter contains 'content' or 'handler' keys
- Upgrade cryptography and python-multipart for security advisories
### Internal
- **#594**: Add `ty` as supplemental type checker
- Batched vector sync orchestration across repositories
- FastEmbed parallel guardrails and provider caching
- Improved cloud CLI status and error messages
- CI coverage and Postgres test fixes
## v0.18.5 (2026-02-13)
+2 -2
View File
@@ -6,12 +6,12 @@
"url": "https://github.com/basicmachines-co/basic-memory.git",
"source": "github"
},
"version": "0.18.5",
"version": "0.19.0",
"packages": [
{
"registryType": "pypi",
"identifier": "basic-memory",
"version": "0.18.5",
"version": "0.19.0",
"runtimeHint": "uvx",
"runtimeArguments": [
{"type": "positional", "value": "basic-memory"},
+1 -1
View File
@@ -1,7 +1,7 @@
"""basic-memory - Local-first knowledge management combining Zettelkasten with knowledge graphs"""
# Package version - updated by release automation
__version__ = "0.18.5"
__version__ = "0.19.0"
# API version for FastAPI - independent of package version
__api_version__ = "v0"
+1 -1
View File
@@ -12,7 +12,7 @@ from basic_memory.config import ConfigManager
OSS_DISCOUNT_CODE = "BMFOSS"
CLOUD_LEARN_MORE_URL = (
"https://basicmemory.com?utm_source=bm-cli&utm_medium=promo&utm_campaign=cloud-upsell"
"https://basicmemory.com?utm_source=bm-foss&utm_medium=promo&utm_campaign=cloud-upsell"
)
+34 -52
View File
@@ -43,40 +43,37 @@ if sys.platform == "win32": # pragma: no cover
_engine: Optional[AsyncEngine] = None
_session_maker: Optional[async_sessionmaker[AsyncSession]] = None
# Alembic revision that enables one-time automatic embedding backfill.
SEMANTIC_EMBEDDING_BACKFILL_REVISION = "i2c3d4e5f6g7"
async def _load_applied_alembic_revisions(
async def _needs_semantic_embedding_backfill(
app_config: BasicMemoryConfig,
session_maker: async_sessionmaker[AsyncSession],
) -> set[str]:
"""Load applied Alembic revisions from alembic_version.
) -> bool:
"""Check if entities exist but vector embeddings are empty.
Returns an empty set when the version table does not exist yet
(fresh database before first migration).
This is the reliable way to detect that embeddings need to be generated,
regardless of how migrations were applied (fresh DB, upgrade, reset, etc.).
"""
if not app_config.semantic_search_enabled:
return False
try:
async with scoped_session(session_maker) as session:
result = await session.execute(text("SELECT version_num FROM alembic_version"))
return {str(row[0]) for row in result.fetchall() if row[0]}
entity_count = (
await session.execute(text("SELECT COUNT(*) FROM entity"))
).scalar() or 0
if entity_count == 0:
return False
# Check if vector chunks table exists and is empty
embedding_count = (
await session.execute(text("SELECT COUNT(*) FROM search_vector_chunks"))
).scalar() or 0
return embedding_count == 0
except Exception as exc:
error_message = str(exc).lower()
if "alembic_version" in error_message and (
"no such table" in error_message or "does not exist" in error_message
):
return set()
raise
def _should_run_semantic_embedding_backfill(
revisions_before_upgrade: set[str],
revisions_after_upgrade: set[str],
) -> bool:
"""Check if this migration run newly applied the backfill-trigger revision."""
return (
SEMANTIC_EMBEDDING_BACKFILL_REVISION in revisions_after_upgrade
and SEMANTIC_EMBEDDING_BACKFILL_REVISION not in revisions_before_upgrade
)
# Table might not exist yet (pre-migration)
logger.debug(f"Could not check embedding status: {exc}")
return False
async def _run_semantic_embedding_backfill(
@@ -480,26 +477,9 @@ async def run_migrations(
Note: Alembic tracks which migrations have been applied via the alembic_version table,
so it's safe to call this multiple times - it will only run pending migrations.
"""
logger.debug("Running database migrations...")
logger.info("Running database migrations...")
temp_engine: AsyncEngine | None = None
try:
revisions_before_upgrade: set[str] = set()
# Trigger: run_migrations() can be invoked before module-level session maker is set.
# Why: we still need reliable before/after revision detection for one-time backfill.
# Outcome: create a short-lived session maker when needed, then dispose it immediately.
if _session_maker is None:
precheck_engine, temp_session_maker = _create_engine_and_session(
app_config.database_path,
database_type,
app_config,
)
try:
revisions_before_upgrade = await _load_applied_alembic_revisions(temp_session_maker)
finally:
await precheck_engine.dispose()
else:
revisions_before_upgrade = await _load_applied_alembic_revisions(_session_maker)
# Get the absolute path to the alembic directory relative to this file
alembic_dir = Path(__file__).parent / "alembic"
config = Config()
@@ -519,7 +499,7 @@ async def run_migrations(
config.set_main_option("sqlalchemy.url", db_url)
command.upgrade(config, "head")
logger.debug("Migrations completed successfully")
logger.info("Migrations completed successfully")
# Get session maker - ensure we don't trigger recursive migration calls
if _session_maker is None:
@@ -541,12 +521,14 @@ async def run_migrations(
else:
await SQLiteSearchRepository(session_maker, 1).init_search_index()
revisions_after_upgrade = await _load_applied_alembic_revisions(session_maker)
if _should_run_semantic_embedding_backfill(
revisions_before_upgrade,
revisions_after_upgrade,
):
await _run_semantic_embedding_backfill(app_config, session_maker)
# Check if backfill is needed — actual backfill runs in background
# from the MCP server lifespan to avoid blocking startup.
if await _needs_semantic_embedding_backfill(app_config, session_maker):
logger.info(
"Semantic embeddings missing — backfill will run in background after startup"
)
else:
logger.info("Semantic embeddings: up to date")
except Exception as e: # pragma: no cover
logger.error(f"Error running migrations: {e}")
raise
+72
View File
@@ -2,18 +2,71 @@
Basic Memory FastMCP server.
"""
import asyncio
import time
from contextlib import asynccontextmanager
from fastmcp import FastMCP
from loguru import logger
from sqlalchemy import text
from sqlalchemy.ext.asyncio import async_sessionmaker, AsyncSession
from basic_memory import db
from basic_memory.cli.auth import CLIAuth
from basic_memory.config import BasicMemoryConfig
from basic_memory.db import (
scoped_session,
_needs_semantic_embedding_backfill,
_run_semantic_embedding_backfill,
)
from basic_memory.mcp.container import McpContainer, set_container
from basic_memory.services.initialization import initialize_app
async def _log_embedding_status(session_maker: async_sessionmaker[AsyncSession]) -> None:
"""Log a clear summary of semantic embedding status at startup."""
try:
async with scoped_session(session_maker) as session:
entity_count = (
await session.execute(text("SELECT COUNT(*) FROM entity"))
).scalar() or 0
chunk_count = (
await session.execute(text("SELECT COUNT(*) FROM search_vector_chunks"))
).scalar() or 0
embedding_count = (
await session.execute(text("SELECT COUNT(*) FROM search_vector_embeddings_rowids"))
).scalar() or 0
if entity_count == 0:
logger.info("Semantic embeddings: no entities yet")
elif embedding_count == 0:
logger.warning(
f"Semantic embeddings: EMPTY — {entity_count} entities have no embeddings. "
"Backfill running in background..."
)
else:
logger.info(
f"Semantic embeddings: {embedding_count} embeddings "
f"across {chunk_count} chunks for {entity_count} entities"
)
except Exception as exc:
logger.debug(f"Could not check embedding status at startup: {exc}")
async def _background_embedding_backfill(
config: BasicMemoryConfig,
session_maker: async_sessionmaker[AsyncSession],
) -> None:
"""Run semantic embedding backfill in the background without blocking startup."""
try:
if await _needs_semantic_embedding_backfill(config, session_maker):
logger.info("Background embedding backfill starting...")
await _run_semantic_embedding_backfill(config, session_maker)
await _log_embedding_status(session_maker)
except Exception as exc:
logger.error(f"Background embedding backfill failed: {exc}")
@asynccontextmanager
async def lifespan(app: FastMCP):
"""Lifecycle manager for the MCP server.
@@ -70,6 +123,16 @@ async def lifespan(app: FastMCP):
# Initialize app (runs migrations, reconciles projects)
await initialize_app(container.config)
# Log embedding status so it's easy to spot in the logs
backfill_task: asyncio.Task | None = None # type: ignore[type-arg]
if config.semantic_search_enabled and db._session_maker is not None:
await _log_embedding_status(db._session_maker)
# Launch backfill in background so MCP server is ready immediately
backfill_task = asyncio.create_task(
_background_embedding_backfill(config, db._session_maker),
name="embedding-backfill",
)
# Create and start sync coordinator (lifecycle centralized in coordinator)
sync_coordinator = container.create_sync_coordinator()
await sync_coordinator.start()
@@ -79,6 +142,15 @@ async def lifespan(app: FastMCP):
finally:
# Shutdown - coordinator handles clean task cancellation
logger.debug("Shutting down Basic Memory MCP server")
# Cancel embedding backfill if still running
if backfill_task is not None and not backfill_task.done():
backfill_task.cancel()
try:
await backfill_task
except asyncio.CancelledError:
logger.info("Background embedding backfill cancelled during shutdown")
await sync_coordinator.stop()
# Only shutdown DB if we created it (not if test fixture provided it)
+2 -2
View File
@@ -160,7 +160,7 @@ def _no_notes_guidance(note_type: str, tool_name: str) -> str:
f"## Next Steps\n\n"
f"1. **Create notes of this type** — use `write_note` with "
f'`note_type="{note_type}"` to create notes\n'
f"2. **Check existing types** — use `search_notes` with `entity_types` "
f"2. **Check existing types** — use `search_notes` with `note_types` "
f"filter to see what types exist\n"
f"3. **Browse content** — use `list_directory` or `recent_activity` to "
f"see what's in the project\n"
@@ -397,7 +397,7 @@ async def schema_infer(
f"share a consistent structure.\n\n"
f"## Suggestions\n"
f"1. **Use a more specific type** — try `search_notes` with "
f"`entity_types` filter to see what types exist\n"
f"`note_types` filter to see what types exist\n"
f"2. **Lower the threshold** — "
f'`schema_infer("{note_type}", threshold=0.1)` to include '
f"rarer fields\n"
+17 -6
View File
@@ -2,7 +2,7 @@
import re
from textwrap import dedent
from typing import List, Optional, Dict, Any, Literal
from typing import Annotated, List, Optional, Dict, Any, Literal
from loguru import logger
from fastmcp import Context
@@ -165,7 +165,7 @@ def _format_search_error_response(
- Remove restrictive terms: Focus on the most important keywords
5. **Use filtering to narrow scope**:
- By content type: `search_notes("{project}","{query}", note_types=["note"])`
- By note type in frontmatter: `search_notes("{project}","{query}", note_types=["note"])`
- By recent content: `search_notes("{project}","{query}", after_date="1 week")`
- By entity type: `search_notes("{project}","{query}", entity_types=["observation"])`
@@ -305,8 +305,17 @@ async def search_notes(
page_size: int = 10,
search_type: str | None = None,
output_format: Literal["text", "json"] = "text",
note_types: List[str] | None = None,
entity_types: List[str] | None = None,
note_types: Annotated[
List[str] | None,
"Filter by the 'type' field in note frontmatter (e.g. 'note', 'chapter', 'person'). "
"Case-insensitive.",
] = None,
entity_types: Annotated[
List[str] | None,
"Filter by knowledge graph item type: 'entity' (whole notes), 'observation', or "
"'relation'. Defaults to 'entity'. Do NOT pass schema/frontmatter types like "
"'Chapter' here — use note_types instead.",
] = None,
after_date: Optional[str] = None,
metadata_filters: Optional[Dict[str, Any]] = None,
tags: Optional[List[str]] = None,
@@ -350,6 +359,7 @@ async def search_notes(
### Search Type Examples
- `search_notes("my-project", "Meeting", search_type="title")` - Search only in titles
- `search_notes("work-docs", "docs/meeting-*", search_type="permalink")` - Pattern match permalinks
Note: Permalink patterns match the full path (e.g., "project/folder/chapter-13*", not just "chapter-13*").
- `search_notes("research", "keyword")` - Default search (hybrid when semantic is enabled,
text when disabled)
@@ -436,7 +446,7 @@ async def search_notes(
# Exact phrase search
results = await search_notes("\"weekly standup meeting\"")
# Search with note type filter
# Search with note type filter - type property in frontmatter
results = await search_notes(
"meeting notes",
note_types=["note"],
@@ -477,7 +487,8 @@ async def search_notes(
results = await search_notes("project planning", project="my-project")
"""
# Avoid mutable-default-argument footguns. Treat None as "no filter".
note_types = note_types or []
# Lowercase note_types so "Chapter" matches the stored "chapter".
note_types = [t.lower() for t in note_types] if note_types else []
entity_types = entity_types or []
# Parse tag:<value> shorthand at tool level so it works with all search modes.
+12 -3
View File
@@ -2,7 +2,7 @@
from __future__ import annotations
from typing import Any, Dict, List, Optional
from typing import Annotated, Any, Dict, List, Optional
from fastmcp import Context
from mcp.types import ContentBlock, TextContent
@@ -28,8 +28,17 @@ async def search_notes_ui(
page: int = 1,
page_size: int = 10,
search_type: Optional[str] = None,
note_types: List[str] | None = None,
entity_types: List[str] | None = None,
note_types: Annotated[
List[str] | None,
"Filter by the 'type' field in note frontmatter (e.g. 'note', 'chapter', 'person'). "
"Case-insensitive.",
] = None,
entity_types: Annotated[
List[str] | None,
"Filter by knowledge graph item type: 'entity' (whole notes), 'observation', or "
"'relation'. Defaults to 'entity'. Do NOT pass schema/frontmatter types like "
"'Chapter' here — use note_types instead.",
] = None,
after_date: Optional[str] = None,
metadata_filters: Optional[Dict[str, Any]] = None,
tags: Optional[List[str]] = None,
+29 -3
View File
@@ -293,12 +293,16 @@ class SyncService:
for path in report.deleted:
await self.handle_delete(path)
# then new and modified
# then new and modified — collect entity IDs for batch vector embedding
synced_entity_ids: list[int] = []
for path in report.new:
entity, _ = await self.sync_file(path, new=True)
if entity is not None:
synced_entity_ids.append(entity.id)
# Track if file was skipped
if entity is None and await self._should_skip_file(path):
elif await self._should_skip_file(path):
failure_info = self._file_failures[path]
report.skipped_files.append(
SkippedFile(
@@ -312,8 +316,10 @@ class SyncService:
for path in report.modified:
entity, _ = await self.sync_file(path, new=False)
if entity is not None:
synced_entity_ids.append(entity.id)
# Track if file was skipped
if entity is None and await self._should_skip_file(path):
elif await self._should_skip_file(path):
failure_info = self._file_failures[path]
report.skipped_files.append(
SkippedFile(
@@ -331,6 +337,26 @@ class SyncService:
else:
logger.info("Skipping relation resolution - no file changes detected")
# Batch-generate vector embeddings for all synced entities
if synced_entity_ids and self.app_config.semantic_search_enabled:
try:
logger.info(
f"Generating semantic embeddings for {len(synced_entity_ids)} entities..."
)
batch_result = await self.search_service.sync_entity_vectors_batch(
synced_entity_ids
)
logger.info(
f"Semantic embeddings complete: "
f"synced={batch_result.entities_synced}, "
f"failed={batch_result.entities_failed}"
)
except SemanticDependenciesMissingError:
logger.warning(
"Semantic search dependencies missing — vector embeddings skipped. "
"Run 'bm reindex --embeddings' after resolving the dependency issue."
)
# Update scan watermark after successful sync
# Use the timestamp from sync start (not end) to ensure we catch files
# created during the sync on the next iteration
+2
View File
@@ -31,6 +31,7 @@ async def test_returns_none_when_no_default_and_no_project(config_manager, monke
config_manager.save_config(cfg)
monkeypatch.delenv("BASIC_MEMORY_MCP_PROJECT", raising=False)
# Prevent API fallback from returning a project via stale dependency overrides
async def _no_api_fallback():
return None
@@ -117,6 +118,7 @@ async def test_returns_none_when_no_default(config_manager, monkeypatch):
config_manager.save_config(cfg)
monkeypatch.delenv("BASIC_MEMORY_MCP_PROJECT", raising=False)
# Prevent API fallback from returning a project via stale dependency overrides
async def _no_api_fallback():
return None
+48
View File
@@ -1146,6 +1146,54 @@ async def test_search_notes_explicit_entity_types_overrides_default(monkeypatch)
assert captured_payload["entity_types"] == ["observation"]
# --- Tests for note_types case-insensitivity ------------------------------------
@pytest.mark.asyncio
async def test_search_notes_note_types_lowercased(monkeypatch):
"""note_types values are lowercased so 'Chapter' matches stored 'chapter'."""
import importlib
search_mod = importlib.import_module("basic_memory.mcp.tools.search")
clients_mod = importlib.import_module("basic_memory.mcp.clients")
class StubProject:
name = "test-project"
external_id = "test-external-id"
@asynccontextmanager
async def fake_get_project_client(*args, **kwargs):
yield (object(), StubProject())
async def fake_resolve_project_and_path(
client, identifier, project=None, context=None, headers=None
):
return StubProject(), identifier, False
captured_payload: dict = {}
class MockSearchClient:
def __init__(self, *args, **kwargs):
pass
async def search(self, payload, page, page_size):
captured_payload.update(payload)
return SearchResponse(results=[], current_page=page, page_size=page_size)
monkeypatch.setattr(search_mod, "get_project_client", fake_get_project_client)
monkeypatch.setattr(search_mod, "resolve_project_and_path", fake_resolve_project_and_path)
monkeypatch.setattr(clients_mod, "SearchClient", MockSearchClient)
await search_mod.search_notes(
project="test-project",
query="test",
note_types=["Chapter", "Person"],
)
# note_types should be lowercased
assert captured_payload["note_types"] == ["chapter", "person"]
# --- Tests for tag: prefix parsing (issue #30) ---------------------------------
+68 -24
View File
@@ -200,10 +200,10 @@ async def test_initialize_app_no_precedence_warning_when_not_conflicting(
@pytest.mark.asyncio
async def test_run_migrations_triggers_embedding_backfill_on_new_revision(
async def test_run_migrations_triggers_embedding_backfill_when_entities_exist_but_no_embeddings(
monkeypatch, app_config: BasicMemoryConfig
):
"""When the trigger revision is newly applied, run automatic embedding backfill once."""
"""run_migrations checks for missing embeddings (actual backfill runs in background from MCP)."""
class StubSearchRepository:
def __init__(self, *args, **kwargs):
@@ -224,29 +224,24 @@ async def test_run_migrations_triggers_embedding_backfill_on_new_revision(
monkeypatch.setattr("basic_memory.db.SQLiteSearchRepository", StubSearchRepository)
monkeypatch.setattr("basic_memory.db.PostgresSearchRepository", StubSearchRepository)
load_revisions_mock = AsyncMock(
side_effect=[
set(),
{db.SEMANTIC_EMBEDDING_BACKFILL_REVISION},
]
needs_backfill_mock = AsyncMock(return_value=True)
monkeypatch.setattr(
"basic_memory.db._needs_semantic_embedding_backfill", needs_backfill_mock
)
backfill_mock = AsyncMock()
monkeypatch.setattr("basic_memory.db._load_applied_alembic_revisions", load_revisions_mock)
monkeypatch.setattr("basic_memory.db._run_semantic_embedding_backfill", backfill_mock)
await db.run_migrations(app_config)
assert load_revisions_mock.await_count == 2
backfill_mock.assert_awaited_once_with(app_config, session_marker)
# Verifies the check runs — backfill itself is launched by MCP lifespan
needs_backfill_mock.assert_awaited_once_with(app_config, session_marker)
finally:
db._session_maker = original_session_maker # pyright: ignore [reportPrivateUsage]
@pytest.mark.asyncio
async def test_run_migrations_skips_embedding_backfill_when_revision_already_applied(
async def test_run_migrations_skips_embedding_backfill_when_embeddings_already_exist(
monkeypatch, app_config: BasicMemoryConfig
):
"""If the trigger revision was already present before upgrade, skip backfill."""
"""When embeddings already exist, no backfill is needed."""
class StubSearchRepository:
def __init__(self, *args, **kwargs):
@@ -267,20 +262,14 @@ async def test_run_migrations_skips_embedding_backfill_when_revision_already_app
monkeypatch.setattr("basic_memory.db.SQLiteSearchRepository", StubSearchRepository)
monkeypatch.setattr("basic_memory.db.PostgresSearchRepository", StubSearchRepository)
load_revisions_mock = AsyncMock(
side_effect=[
{db.SEMANTIC_EMBEDDING_BACKFILL_REVISION},
{db.SEMANTIC_EMBEDDING_BACKFILL_REVISION},
]
needs_backfill_mock = AsyncMock(return_value=False)
monkeypatch.setattr(
"basic_memory.db._needs_semantic_embedding_backfill", needs_backfill_mock
)
backfill_mock = AsyncMock()
monkeypatch.setattr("basic_memory.db._load_applied_alembic_revisions", load_revisions_mock)
monkeypatch.setattr("basic_memory.db._run_semantic_embedding_backfill", backfill_mock)
await db.run_migrations(app_config)
assert load_revisions_mock.await_count == 2
assert backfill_mock.await_count == 0
needs_backfill_mock.assert_awaited_once_with(app_config, session_marker)
finally:
db._session_maker = original_session_maker # pyright: ignore [reportPrivateUsage]
@@ -378,3 +367,58 @@ async def test_semantic_embedding_backfill_skips_when_semantic_disabled(
app_config.semantic_search_enabled = False
await db._run_semantic_embedding_backfill(app_config, session_maker) # pyright: ignore [reportPrivateUsage]
assert called is False
@pytest.mark.asyncio
async def test_needs_semantic_embedding_backfill_true_when_entities_exist_no_embeddings(
app_config: BasicMemoryConfig,
session_maker,
test_project,
):
"""Should return True when entities exist but vector chunks table is empty."""
from basic_memory.repository.entity_repository import EntityRepository
entity_repository = EntityRepository(session_maker, project_id=test_project.id)
await entity_repository.create(
{
"title": "Test Entity",
"note_type": "note",
"entity_metadata": {},
"content_type": "text/markdown",
"file_path": "test/backfill-check.md",
"permalink": "test/backfill-check",
"project_id": test_project.id,
"created_at": datetime.now(),
"updated_at": datetime.now(),
}
)
# Clear any embeddings left by other tests in the shared DB
async with db.scoped_session(session_maker) as session:
await session.execute(db.text("DELETE FROM search_vector_chunks"))
app_config.semantic_search_enabled = True
result = await db._needs_semantic_embedding_backfill(app_config, session_maker) # pyright: ignore [reportPrivateUsage]
assert result is True
@pytest.mark.asyncio
async def test_needs_semantic_embedding_backfill_false_when_no_entities(
app_config: BasicMemoryConfig,
session_maker,
):
"""Should return False when no entities exist (nothing to backfill)."""
app_config.semantic_search_enabled = True
result = await db._needs_semantic_embedding_backfill(app_config, session_maker) # pyright: ignore [reportPrivateUsage]
assert result is False
@pytest.mark.asyncio
async def test_needs_semantic_embedding_backfill_false_when_semantic_disabled(
app_config: BasicMemoryConfig,
session_maker,
):
"""Should return False when semantic search is disabled."""
app_config.semantic_search_enabled = False
result = await db._needs_semantic_embedding_backfill(app_config, session_maker) # pyright: ignore [reportPrivateUsage]
assert result is False