Compare commits

...

3 Commits

Author SHA1 Message Date
phernandez 214a54d740 chore: update version to 0.21.0 for v0.21.0 release 2026-05-16 17:15:53 -05:00
phernandez 5a90c7cedf docs: add v0.21.0 changelog entry
Promotes the Unreleased breaking-change note and adds the full v0.21.0
section covering ~80 commits since v0.20.3: workspace-routing fixes
across MCP/CLI/API, recent_activity ordering and search opt-in changes,
sync hardening, sqlite-vec graceful degrade, perf wins on CLI startup
and sync, and the project-delete cleanup landed in #832.

Signed-off-by: phernandez <paul@basicmachines.co>
2026-05-16 17:15:43 -05:00
Paul Hernandez 9e3fe26a83 fix(core): purge SQLite search_index on project delete (#832)
Signed-off-by: phernandez <paul@basicmachines.co>
2026-05-16 16:47:24 -05:00
8 changed files with 689 additions and 9 deletions
+108 -1
View File
@@ -1,6 +1,10 @@
# CHANGELOG
## Unreleased
## v0.21.0 (2026-05-16)
Workspace-aware everywhere: every MCP tool and CLI command now routes through the
same workspace/project model, the search and sync paths are noticeably faster,
and a handful of long-standing parsing and routing footguns are gone.
### Breaking Changes
@@ -11,6 +15,109 @@
- Bare list wikilinks like `- [[Target]]` now index as `links_to`.
- Prose list items like `- some other thing [[Target]]` now index as `links_to`.
- To preserve existing multi-word relation types on re-sync, quote them before upgrading.
- See **#824**.
### Features
- **#816**: `bm orphan` CLI command surfaces entities whose underlying markdown
files are gone, with a flag to clean them out.
- **#789**: Create projects directly by cloud workspace slug from MCP
(`create_memory_project(workspace=...)`).
- **#757**: Discover projects across every accessible cloud workspace in MCP's
project list — no more per-workspace blind spots.
- **#766**: MCP tools accept training-data-friendly parameter aliases
(`q`/`search`/`text` for `query`, etc.) so models reach for them naturally.
- **#776**: `bm db reset` refuses to run while a `basic-memory mcp` process is
alive, so resets can no longer corrupt an open session.
- **#791**: Search responses include result totals so pagers can stop guessing.
- **#719**: Cloud `note_content` tenant schema primitive lands on the backend.
- **#715**: `bm project add` accepts a `--visibility` flag for cloud projects.
### Bug Fixes
#### Search and recent activity
- **#832**: SQLite project deletion now sweeps `search_index`,
`search_vector_chunks`, and `search_vector_embeddings`, so a project that
reuses a recycled auto-increment id can't inherit the previous tenant's content.
- **#812**: `recent_activity` orders and filters by `updated_at`, so edits bubble
to the top instead of staying pinned to creation time.
- **#807**: Multi-project `search_notes` is opt-in (`search_all_projects=True`);
default search stays scoped to the resolved project.
- **#785**: `recent_activity` caps responses and emits an explicit truncation
footer instead of silently dropping rows.
- **#713**: Eliminated an N+1 query in search result hydration.
#### Workspace / project routing
- **#822**: `bm project list` now includes projects from every workspace, not
just the current one.
- **#813**, **#808**, **#806**, **#803**, **#801**, **#795**, **#790**, **#778**,
**#777**, **#722**, **#712**, **#704**: Workspace-qualified permalink routing
is centralized and applied consistently across `edit_note`, `delete_project`,
`build_context`, `memory://` URLs, factory-mode project listing, cloud uploads,
and the API client.
#### Sync
- **#827**: `rclone bisync` filters from `.bmignore` are preserved across syncs.
- **#815**: Watch service ignores hidden paths relative to the watched project,
not just relative to `$HOME`.
- **#814**: `scan` subprocesses no longer go through the shell, avoiding quoting
issues with paths that contain special characters.
- **#759**: Watch service stays inside `--project` scope.
- **#746**: Canonical markdown is preserved during single-file sync.
#### Parsing
- **#796**: Picoschema modifier descriptions (`field?(modifier, description)`)
parse correctly.
- **#769**: Obsidian callout blocks are skipped by the observation parser
instead of being mis-extracted as observations.
#### CLI
- **#775**: `bm project set-cloud` / `set-local` cleans up local DB state for
the affected project.
- **#773**: `bm cloud logout` clears `default_workspace`.
- **#780**: `bm cloud setup` hint points at `bm cloud sync-setup`.
- **#734**: `bm project info` shows cloud index freshness.
- **#718**: Private cloud projects display under their `display_name` instead of
raw UUID.
- **#768**: `read_note` / `view_note` drop no-op pagination params from their
signatures.
#### Stability
- **#774**: `sqlite-vec` failures during init degrade gracefully to keyword-only
search instead of crashing startup.
- **#733**: Delete-vector and cloud-sync cleanup is now consistent.
- **#702**: Race conditions in concurrent `delete_entity` are resolved.
- **#724**: `external_id` is preserved when entities are re-upserted during a
re-index.
- **#728**: Vector init no longer issues runtime `ALTER TABLE`.
- **#744**: `BASIC_MEMORY_CONFIG_DIR` is honored across remaining call sites.
- **#743**: FastEmbed cache lives under the data dir instead of `/tmp`.
- **#752**: Cloud projects report `source=cloud` in factory mode.
- Stripped null bytes from markdown content before DB insert.
- Allowed long `relation_type` values in API responses.
#### Installer
- **#772**: Docker-compose config volume mounts under the `appuser` home.
- **#695**: Bumped `brew outdated` timeout from 15s to 60s.
### Performance
- **#828**: CLI startup no longer pulls in the local ASGI FastAPI app when it
isn't needed.
- **#751**, **#726**, **#717**, **#714**, single-file/batch indexing: marked
speedups on the sync hot path; unchanged markdown is skipped entirely.
- **#731**, **#723**: Vector sync is faster on both backends, with tuned
fastembed defaults.
### Maintenance
- **#825**: Dependency refresh + security hardening.
- Updated to `fastmcp` 3.3.1.
- **#754**: Removed in-house telemetry wrappers in favor of direct `logfire`
usage.
- **#736**: `ty` is now the default typechecker.
- **#771**, **#770**, **#716**: New regression guards for vector-row cleanup,
long relation types, and recent activity hydration.
## v0.20.3 (2026-03-26)
+2 -2
View File
@@ -6,12 +6,12 @@
"url": "https://github.com/basicmachines-co/basic-memory.git",
"source": "github"
},
"version": "0.20.3",
"version": "0.21.0",
"packages": [
{
"registryType": "pypi",
"identifier": "basic-memory",
"version": "0.20.3",
"version": "0.21.0",
"runtimeHint": "uvx",
"runtimeArguments": [
{"type": "positional", "value": "basic-memory"},
+1 -1
View File
@@ -1,7 +1,7 @@
"""basic-memory - Local-first knowledge management combining Zettelkasten with knowledge graphs"""
# Package version - updated by release automation
__version__ = "0.20.3"
__version__ = "0.21.0"
# API version for FastAPI - independent of package version
__api_version__ = "v0"
@@ -0,0 +1,86 @@
"""Remove orphaned search rows whose project was already deleted.
Revision ID: n7i8j9k0l1m2
Revises: m6h7i8j9k0l1
Create Date: 2026-05-15 18:30:00.000000
"""
from typing import Sequence, Union
from alembic import op
from sqlalchemy import inspect
revision: str = "n7i8j9k0l1m2"
down_revision: Union[str, None] = "m6h7i8j9k0l1"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def _table_exists(connection, table_name: str) -> bool:
"""Inspector-based table check, dialect agnostic.
Trigger: SQLite creates search_index as an FTS5 virtual table at runtime
via SearchRepository.init_search_index, not through Alembic, so fresh
installs hit this migration before the table exists.
Why: a blind DELETE against a missing table fails the whole upgrade.
Outcome: callers skip the sweep when the table isn't present yet — the
runtime-created table on a fresh DB has no orphans to clean.
"""
return table_name in inspect(connection).get_table_names()
def upgrade() -> None:
"""Purge orphaned search rows left over from prior project deletions.
Trigger: project deletion on SQLite never removed the derived FTS rows,
because the FTS5 virtual table can't carry a foreign key. The leak shows
up in two shapes:
1. project_id no longer exists in `project` (deleted project, id never
reused).
2. project_id still exists but `entity_id` no longer exists in `entity`
— auto-increment handed the id to a brand-new project and the FTS
rows from the deleted predecessor masquerade as the new tenant's data.
Why: search_index.project_id is the only scope predicate the search
repository applies, so leftover rows surface under the wrong project on
every search.
Outcome: a one-time sweep deletes both shapes, from the FTS index and
from search_vector_chunks. Postgres already cascaded on FK delete, so
these statements are no-ops there.
"""
connection = op.get_bind()
if _table_exists(connection, "search_index"):
op.execute(
"""
DELETE FROM search_index
WHERE project_id NOT IN (SELECT id FROM project)
"""
)
op.execute(
"""
DELETE FROM search_index
WHERE entity_id IS NOT NULL
AND entity_id NOT IN (SELECT id FROM entity)
"""
)
if _table_exists(connection, "search_vector_chunks"):
op.execute(
"""
DELETE FROM search_vector_chunks
WHERE project_id NOT IN (SELECT id FROM project)
"""
)
op.execute(
"""
DELETE FROM search_vector_chunks
WHERE entity_id NOT IN (SELECT id FROM entity)
"""
)
def downgrade() -> None:
"""No-op: orphan rows cannot be reconstructed."""
pass
+25 -2
View File
@@ -10,8 +10,13 @@ from loguru import logger
from fastmcp import Context
from pydantic import AliasChoices, BeforeValidator, Field
from basic_memory.config import ConfigManager
from basic_memory.config import ConfigManager, has_cloud_credentials
from basic_memory.utils import build_canonical_permalink, coerce_dict, coerce_list
from basic_memory.mcp.async_client import (
_explicit_routing,
_force_local_mode,
is_factory_mode,
)
from basic_memory.mcp.container import get_container
from basic_memory.mcp.project_context import (
detect_project_from_identifier_prefix,
@@ -479,12 +484,30 @@ async def _search_all_projects(
total = 0
any_project_has_more = False
# Trigger: caller asked for an account-wide search.
# Why: project_id (external UUID) routes through the cloud v2 API path,
# which 401s on local installs because there's no JWT to present.
# Project names route through the local-ASGI path and work for both
# backends — cloud disambiguates names via the workspace/project
# qualified_name already baked into project_ref["project"].
# Outcome: forward project_id only when the same signals get_project_client
# uses to pick a cloud route are present. Mirrors the cloud_available
# composite in project_context.get_project_client (single source of
# truth for "can we route to cloud?").
config = ConfigManager().config
use_cloud_routing = (
is_factory_mode()
or (_explicit_routing() and not _force_local_mode())
or has_cloud_credentials(config)
)
for project_ref in project_refs:
recursive_project_id = project_ref["project_id"] if use_cloud_routing else None
try:
results = await search_notes(
query=query,
project=project_ref["project"],
project_id=project_ref["project_id"],
project_id=recursive_project_id,
page=1,
page_size=per_project_page_size,
search_type=search_type,
@@ -4,7 +4,9 @@ from pathlib import Path
from typing import Optional, Sequence, Union
from sqlalchemy import text
from loguru import logger
from sqlalchemy import inspect as sa_inspect, select, text
from sqlalchemy.exc import NoResultFound, OperationalError
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
from basic_memory import db
@@ -12,6 +14,65 @@ from basic_memory.models.project import Project
from basic_memory.repository.repository import Repository
async def _load_sqlite_vec_on_session(session) -> bool:
"""Ensure the sqlite-vec extension is loaded on this session's connection.
Returns True when vec0 is available after the call. Returns False when the
extension can't be loaded on this Python build (e.g., python.org macOS or
Windows interpreters without `enable_load_extension`) — every connection in
the pool shares the same interpreter, so a False here also means no
embedding row could ever have been written, and skipping the embeddings
purge is safe.
Mirrors SQLiteSearchRepository._ensure_sqlite_vec_loaded but as a free
function: we don't have a SearchRepository instance during project delete,
and the per-connection nature of extension loading means a pooled connection
routed to this session might not have vec loaded even when other
connections wrote embeddings.
"""
try:
await session.execute(text("SELECT vec_version()"))
return True
except OperationalError:
pass
try:
import sqlite_vec # type: ignore[import-not-found]
except ImportError:
logger.debug("sqlite-vec package not installed; skipping vec purge")
return False
async_connection = await session.connection()
raw_connection = await async_connection.get_raw_connection()
driver_connection = raw_connection.driver_connection
if not hasattr(driver_connection, "enable_load_extension"):
# Trigger: CPython build without sqlite extension support (#711).
# Why: load_extension is unavailable, so no connection in this pool
# can host vec0. No embeddings exist anywhere.
# Outcome: skip the embeddings purge entirely.
logger.debug(
"Skipping search_vector_embeddings purge: this Python build does "
"not support SQLite extension loading"
)
return False
try:
await driver_connection.enable_load_extension(True)
await driver_connection.load_extension(sqlite_vec.loadable_path())
await driver_connection.enable_load_extension(False)
await session.execute(text("SELECT vec_version()"))
except Exception as exc:
logger.warning(
"Failed to load sqlite-vec for project delete cleanup; "
"skipping embeddings purge: {}",
exc,
)
return False
return True
class ProjectRepository(Repository[Project]):
"""Repository for Project model.
@@ -121,6 +182,83 @@ class ProjectRepository(Repository[Project]):
return target_project
return None # pragma: no cover
async def delete(self, entity_id: int) -> bool:
"""Delete a project and its derived search rows in one transaction.
The cascade picture differs by backend:
- search_index → project: Postgres has ON DELETE CASCADE FK; SQLite
stores search_index as an FTS5 virtual table and can't carry FKs,
so it needs explicit cleanup.
- search_vector_chunks → project: neither backend has an FK here, so
both need an explicit DELETE.
- search_vector_embeddings → search_vector_chunks: Postgres has an FK
(chunk_id REFERENCES … ON DELETE CASCADE); SQLite stores embeddings
in a vec0 virtual table keyed by rowid with no cascade. On SQLite
the embeddings must be purged before the chunk rows, otherwise
`_run_vector_query` keeps returning stale vectors that crowd out
live results.
Each derived table is created lazily (search_index by
SearchRepository.init_search_index, the vector tables once semantic
search initializes), so any of them may be absent on minimal test DBs.
Inspect the connection once and skip whichever is missing.
"""
logger.debug(f"Deleting Project and search rows for project_id: {entity_id}")
async with db.scoped_session(self.session_maker) as session:
try:
result = await session.execute(
select(self.Model).filter(self.primary_key == entity_id)
)
project = result.scalars().one()
except NoResultFound:
logger.debug(f"No Project found to delete: {entity_id}")
return False
dialect_name = session.bind.dialect.name if session.bind else "sqlite"
is_sqlite = dialect_name == "sqlite"
existing_tables = await session.run_sync(
lambda sync_session: set(sa_inspect(sync_session.connection()).get_table_names())
)
# search_index: SQLite has no FK on the FTS5 virtual table; Postgres
# cascades from the project FK, so the explicit DELETE is redundant.
if is_sqlite and "search_index" in existing_tables:
await session.execute(
text("DELETE FROM search_index WHERE project_id = :project_id"),
{"project_id": entity_id},
)
# search_vector_chunks: no FK to project on either backend, so both
# backends need this. SQLite must purge vec0 embeddings first
# (rowid pseudocolumn — Postgres uses chunk_id and would 500 here);
# Postgres' chunk_id FK CASCADE handles its embeddings cleanup when
# we delete the chunk rows below.
if "search_vector_chunks" in existing_tables:
if is_sqlite and "search_vector_embeddings" in existing_tables:
# Extension loading is per-connection. We must load vec0 on
# *this* session before the DELETE; otherwise a different
# pooled connection might have written embeddings that we'd
# silently leave behind.
if await _load_sqlite_vec_on_session(session):
await session.execute(
text(
"DELETE FROM search_vector_embeddings WHERE rowid IN ("
"SELECT id FROM search_vector_chunks "
"WHERE project_id = :project_id)"
),
{"project_id": entity_id},
)
await session.execute(
text("DELETE FROM search_vector_chunks WHERE project_id = :project_id"),
{"project_id": entity_id},
)
await session.delete(project)
logger.debug(f"Deleted Project and search rows for project_id: {entity_id}")
return True
async def update_path(self, project_id: int, new_path: str) -> Optional[Project]:
"""Update project path.
@@ -8,8 +8,38 @@ import pytest
from basic_memory.schemas.search import SearchItemType, SearchResponse, SearchResult
def _stub_routing_mode(monkeypatch, *, cloud: bool) -> None:
"""Pin the three cloud-route signals search.py reads.
`_search_all_projects` only forwards project_id (external UUID) when a
cloud route is available. The composite mirrors get_project_client:
factory mode OR explicit --cloud OR has_cloud_credentials. Tests stub
all three so a dev box with OAuth tokens on disk can't bleed into the
local-mode case.
"""
search_mod = importlib.import_module("basic_memory.mcp.tools.search")
monkeypatch.setattr(search_mod, "is_factory_mode", lambda: False)
monkeypatch.setattr(search_mod, "_explicit_routing", lambda: cloud)
monkeypatch.setattr(search_mod, "_force_local_mode", lambda: False)
monkeypatch.setattr(search_mod, "has_cloud_credentials", lambda config: cloud)
@pytest.fixture
def cloud_routing(monkeypatch):
"""Force the cloud-routing path for multi-project search tests."""
_stub_routing_mode(monkeypatch, cloud=True)
@pytest.fixture
def local_routing(monkeypatch):
"""Force the local-routing path for multi-project search tests."""
_stub_routing_mode(monkeypatch, cloud=False)
@pytest.mark.asyncio
async def test_search_notes_search_all_projects_qualifies_result_permalinks(monkeypatch):
async def test_search_notes_search_all_projects_qualifies_result_permalinks(
monkeypatch, cloud_routing
):
"""Multi-project search belongs to search_notes and keeps result ids routable."""
clients_mod = importlib.import_module("basic_memory.mcp.clients")
search_mod = importlib.import_module("basic_memory.mcp.tools.search")
@@ -169,7 +199,7 @@ async def test_search_notes_search_all_projects_with_no_refs_returns_empty_all_p
@pytest.mark.asyncio
async def test_search_notes_search_all_projects_continues_after_project_failure(
monkeypatch,
monkeypatch, cloud_routing
):
"""One failing project should not discard successful all-project search results."""
clients_mod = importlib.import_module("basic_memory.mcp.clients")
@@ -254,3 +284,85 @@ async def test_search_notes_search_all_projects_continues_after_project_failure(
assert result["total"] == 1
assert any("team-paul/main" in warning for warning in warnings)
assert any("team index unavailable" in warning for warning in warnings)
@pytest.mark.asyncio
async def test_search_notes_search_all_projects_local_omits_project_id(
monkeypatch, local_routing
):
"""Without a cloud route, fan-out must address each project by name only.
project_id (external UUID) routes through the cloud v2 API path, which
returns 401 on local installs because there's no JWT to present. Local
fan-out has to fall back to the name-routed path so each per-project
search actually returns results instead of silently failing.
"""
clients_mod = importlib.import_module("basic_memory.mcp.clients")
search_mod = importlib.import_module("basic_memory.mcp.tools.search")
project_refs = [
{
"project": "alpha",
"project_id": "11111111-1111-1111-1111-111111111111",
},
{
"project": "beta",
"project_id": "22222222-2222-2222-2222-222222222222",
},
]
searched_projects: list[tuple[str | None, str | None]] = []
async def fake_load_search_project_refs(context=None):
return project_refs
class StubProject:
def __init__(self, name: str | None, external_id: str | None):
self.name = name or "main"
self.external_id = external_id or "local-main"
@asynccontextmanager
async def fake_get_project_client(project=None, context=None, project_id=None):
searched_projects.append((project, project_id))
yield object(), StubProject(project, project_id)
async def fake_resolve_project_and_path(client, identifier, project=None, context=None):
return StubProject(project, None), identifier, False
class MockSearchClient:
def __init__(self, client, project_id):
self.project_id = project_id
async def search(self, payload, page, page_size):
return SearchResponse(
results=[
SearchResult(
title=f"Note in {self.project_id or 'local'}",
permalink="notes/example",
content="",
type=SearchItemType.ENTITY,
score=0.5,
file_path="/notes/example.md",
)
],
current_page=page,
page_size=page_size,
total=1,
)
monkeypatch.setattr(search_mod, "_load_search_project_refs", fake_load_search_project_refs)
monkeypatch.setattr(search_mod, "get_project_client", fake_get_project_client)
monkeypatch.setattr(search_mod, "resolve_project_and_path", fake_resolve_project_and_path)
monkeypatch.setattr(clients_mod, "SearchClient", MockSearchClient)
result = await search_mod.search_notes(
query="anything",
search_all_projects=True,
output_format="json",
)
assert isinstance(result, dict)
assert searched_projects == [("alpha", None), ("beta", None)], (
"Local fan-out must omit project_id so the recursive search_notes calls "
"take the name-routed path."
)
assert result["total"] == 2
+214
View File
@@ -6,7 +6,9 @@ from datetime import timezone, datetime
from pathlib import Path
import pytest
from sqlalchemy import text
from basic_memory import db
from basic_memory.services.project_service import ProjectService
@@ -136,3 +138,215 @@ async def test_remove_project_with_related_entities(project_service: ProjectServ
project = await project_service.get_project(test_project_name)
if project:
await project_service.repository.delete(project.id)
async def _table_exists(session_maker, table: str) -> bool:
"""Return True if the named table is present on the current connection."""
from sqlalchemy import inspect as sa_inspect
async with db.scoped_session(session_maker) as session:
return await session.run_sync(
lambda sync_session: table in sa_inspect(sync_session.connection()).get_table_names()
)
@pytest.mark.asyncio
async def test_remove_project_purges_search_rows(project_service: ProjectService):
"""Project deletion must sweep the derived search tables.
SQLite stores search_index as an FTS5 virtual table, which cannot carry a
foreign key, so without an explicit purge the FTS rows survive the project
and leak into the next project that reuses the same auto-increment id.
Postgres has the cascade FK, but we expect the same end-state on either
backend. This test fails on the pre-fix code: search_index still holds the
project's rows after remove_project completes.
"""
test_project_name = f"test-search-cleanup-{os.urandom(4).hex()}"
with tempfile.TemporaryDirectory() as temp_dir:
test_project_path = str(Path(temp_dir) / "test-search-cleanup")
os.makedirs(test_project_path, exist_ok=True)
await project_service.add_project(test_project_name, test_project_path)
project = await project_service.get_project(test_project_name)
assert project is not None
project_id = project.id
# Seed both derived tables directly. The bug is in the cleanup path,
# not the indexer, so a synthetic row is enough to prove the sweep.
async with db.scoped_session(project_service.repository.session_maker) as session:
await session.execute(
text(
"INSERT INTO search_index "
"(id, title, content_stems, content_snippet, permalink, "
" file_path, type, project_id) "
"VALUES (:id, :title, :stems, :snippet, :permalink, "
" :file_path, :type, :project_id)"
),
{
"id": 999_001,
"title": "leak canary",
"stems": "leak canary",
"snippet": "leak canary",
"permalink": f"leak-canary-{project_id}",
"file_path": "leak-canary.md",
"type": "entity",
"project_id": project_id,
},
)
await session.execute(
text(
"INSERT INTO search_vector_chunks "
"(entity_id, project_id, chunk_key, chunk_text, source_hash, "
" entity_fingerprint, embedding_model) "
"VALUES (:entity_id, :project_id, :chunk_key, :chunk_text, "
" :source_hash, :entity_fingerprint, :embedding_model)"
),
{
"entity_id": 999_001,
"project_id": project_id,
"chunk_key": "canary",
"chunk_text": "leak canary",
"source_hash": "abc",
"entity_fingerprint": "",
"embedding_model": "",
},
)
async with db.scoped_session(project_service.repository.session_maker) as session:
pre_index = (
await session.execute(
text("SELECT COUNT(*) FROM search_index WHERE project_id = :pid"),
{"pid": project_id},
)
).scalar_one()
pre_chunks = (
await session.execute(
text("SELECT COUNT(*) FROM search_vector_chunks WHERE project_id = :pid"),
{"pid": project_id},
)
).scalar_one()
assert pre_index >= 1, "seed row should exist before removal"
assert pre_chunks >= 1, "seed chunk should exist before removal"
await project_service.remove_project(test_project_name)
async with db.scoped_session(project_service.repository.session_maker) as session:
post_index = (
await session.execute(
text("SELECT COUNT(*) FROM search_index WHERE project_id = :pid"),
{"pid": project_id},
)
).scalar_one()
post_chunks = (
await session.execute(
text("SELECT COUNT(*) FROM search_vector_chunks WHERE project_id = :pid"),
{"pid": project_id},
)
).scalar_one()
assert post_index == 0, (
f"search_index still has {post_index} rows for deleted project_id={project_id} "
"— project deletion did not sweep the FTS table."
)
assert post_chunks == 0, (
f"search_vector_chunks still has {post_chunks} rows for deleted "
f"project_id={project_id}."
)
@pytest.mark.asyncio
async def test_delete_returns_false_for_missing_project_id(project_service: ProjectService):
"""ProjectRepository.delete must return False when the project id is gone.
The override loses the base Repository.delete contract if the NoResultFound
branch isn't covered — a silent True would mislead callers into thinking
a non-existent project was removed.
"""
result = await project_service.repository.delete(9_999_999)
assert result is False
@pytest.mark.asyncio
async def test_remove_project_purges_vector_embeddings(project_service: ProjectService):
"""Project deletion must also drop sqlite-vec embeddings keyed by chunk rowid.
sqlite-vec stores vectors in a vec0 virtual table that has no cascade
behavior. If embeddings linger after the chunks they reference are gone,
`_run_vector_query` pulls them as top-k candidates and crowds out live
results. The test only runs when the embeddings table is present, which
matches the install path that exercises semantic search.
"""
test_project_name = f"test-vec-cleanup-{os.urandom(4).hex()}"
session_maker = project_service.repository.session_maker
# The embeddings table only exists once semantic search has initialized.
# Skipping when it's absent keeps this test honest on minimal CI DBs.
if not await _table_exists(session_maker, "search_vector_embeddings"):
pytest.skip("search_vector_embeddings is not present on this connection")
with tempfile.TemporaryDirectory() as temp_dir:
test_project_path = str(Path(temp_dir) / "test-vec-cleanup")
os.makedirs(test_project_path, exist_ok=True)
await project_service.add_project(test_project_name, test_project_path)
project = await project_service.get_project(test_project_name)
assert project is not None
project_id = project.id
async with db.scoped_session(session_maker) as session:
await session.execute(
text(
"INSERT INTO search_vector_chunks "
"(id, entity_id, project_id, chunk_key, chunk_text, source_hash, "
" entity_fingerprint, embedding_model) "
"VALUES (:id, :entity_id, :project_id, :chunk_key, :chunk_text, "
" :source_hash, :entity_fingerprint, :embedding_model)"
),
{
"id": 999_201,
"entity_id": 999_201,
"project_id": project_id,
"chunk_key": "vec-canary",
"chunk_text": "vec canary",
"source_hash": "abc",
"entity_fingerprint": "",
"embedding_model": "",
},
)
# vec0 requires a vector matching the configured dimensions, but the
# delete path filters by rowid; a non-existing dimension would block
# this seed step. Skip the insert if the embeddings DDL hasn't run.
try:
await session.execute(
text(
"INSERT INTO search_vector_embeddings (rowid, embedding) "
"VALUES (:rowid, :embedding)"
),
{"rowid": 999_201, "embedding": "[" + ",".join(["0.0"] * 384) + "]"},
)
except Exception:
pytest.skip("search_vector_embeddings rejected the synthetic seed row")
async with db.scoped_session(session_maker) as session:
pre = (
await session.execute(
text("SELECT COUNT(*) FROM search_vector_embeddings WHERE rowid = :rowid"),
{"rowid": 999_201},
)
).scalar_one()
assert pre >= 1, "seed embedding should exist before removal"
await project_service.remove_project(test_project_name)
async with db.scoped_session(session_maker) as session:
post = (
await session.execute(
text("SELECT COUNT(*) FROM search_vector_embeddings WHERE rowid = :rowid"),
{"rowid": 999_201},
)
).scalar_one()
assert post == 0, (
f"search_vector_embeddings still has {post} rows for rowid 999_201 "
"— project deletion did not sweep the sqlite-vec embeddings table."
)