Compare commits

...

7 Commits

Author SHA1 Message Date
phernandez 21d49023e7 fix(mcp): route multi-project search by name only
The previous revision gated project_id forwarding on the
`cloud_available` composite (factory_mode OR explicit_cloud OR
has_cloud_credentials), mirroring get_project_client. But
`has_cloud_credentials` returns True any time OAuth tokens linger
from a past `bm cloud login`, even when the user is back to working
purely locally. So on a typical dev box the fan-out still forwarded
project_id, hit get_project_client's UUID branch (which treats unknown
identifiers as cloud since local config doesn't key by UUID), and 401d
silently — leaving merged results empty.

Route by name only: project_refs already carry the
workspace/project qualified_name, which is just as unambiguous as the
external_id for both backends. project_id stays in the call signature
purely as a fallback for refs that unexpectedly have no name.

Confirmed live in a restarted MCP — the previous fix did not actually
deliver multi-project results when the dev environment had any cloud
credentials at all.

Tests now drop the routing-mode stubs; the cloud-style tests just key
their MockSearchClient on the workspace-qualified name passed via the
project parameter.

Signed-off-by: phernandez <paul@basicmachines.co>
2026-05-16 16:55:21 -05:00
phernandez e8c11b843d fix(core): load sqlite-vec before embeddings purge; add coverage
Three follow-ups from PR review:

**Codex P2 — Load sqlite-vec on the delete session.** The previous
revision swallowed every `vec0` OperationalError as "no embeddings
exist", but sqlite-vec is loaded **per connection**: a pooled
connection that hosts ProjectRepository.delete may not have vec0
loaded even when another connection successfully wrote embeddings.
That would silently leave orphan vectors behind.

The new `_load_sqlite_vec_on_session` helper mirrors
SQLiteSearchRepository._ensure_sqlite_vec_loaded as a free function
and tries to load the extension on the current session. Only when the
load itself fails — because the Python build lacks
enable_load_extension, or the sqlite_vec package isn't installed — do
we skip the embeddings DELETE. Every connection in the pool shares the
same interpreter, so in that case no embeddings could have been
written from any connection and skipping is safe.

**Claude review — Missing logger.debug calls.** The override now logs
at entry, when the project id isn't found, and after the ORM delete,
matching the base Repository.delete contract.

**Claude review — NoResultFound branch uncovered.** New test
`test_delete_returns_false_for_missing_project_id` asserts the False
return for a nonexistent project id.

Verified locally: 32 passed / 1 skipped (SQLite), 27 passed / 1
skipped (Postgres via testcontainers).

Signed-off-by: phernandez <paul@basicmachines.co>
2026-05-16 16:07:19 -05:00
phernandez 43c50ea683 fix(core): tolerate missing vec0 module during project delete
On Windows SQLite CI the embeddings cleanup hits

  sqlite3.OperationalError: no such module: vec0
  [SQL: DELETE FROM search_vector_embeddings WHERE rowid IN (...)]

because the sqlite-vec extension isn't loaded into the connection
(some Windows Python builds don't expose enable_load_extension on
sqlite3.Connection — see #711). The vec0 virtual table is registered
in sqlite_master, so the table-existence check passes, but any access
to it fails until the module is loaded.

If vec0 isn't loadable, semantic search was never able to write
embeddings, so there's nothing to clean up. Wrap the embeddings DELETE
in a try/except that swallows OperationalError for vec0 and logs a
debug line — the chunk DELETE below still runs.

The KeyError from test_mcp_sse_forces_local on 3.14 SQLite is an
unrelated transient — same code passes on 3.12/3.13 SQLite, Postgres
3.14, and locally on Python 3.14. Will let CI re-run after this push.

Signed-off-by: phernandez <paul@basicmachines.co>
2026-05-16 15:53:49 -05:00
phernandez d3619f97f9 fix(core): dialect-gate sqlite-vec embeddings purge
The previous fix issued an unconditional
`DELETE FROM search_vector_embeddings WHERE rowid IN (...)` during project
deletion, which 500'd on Postgres:

  column "rowid" does not exist

Postgres uses `chunk_id` (a real column with an FK to
search_vector_chunks.id ON DELETE CASCADE), so it doesn't need or accept
the SQLite query at all. The cascade picture by backend:

  - search_index → project: Postgres has FK CASCADE; SQLite FTS5 virtual
    table can't carry FKs and needs explicit cleanup.
  - search_vector_chunks → project: neither backend has an FK, so both
    need an explicit DELETE.
  - search_vector_embeddings → search_vector_chunks: Postgres has FK
    CASCADE on chunk_id; SQLite vec0 virtual table is keyed by rowid
    with no cascade, so embeddings must be purged before the chunks.

Branch is now: chunks DELETE runs on both backends; search_index and
the vec0 embeddings DELETE only run on SQLite.

Verified locally against both backends (SQLite: 31 passed, Postgres
via testcontainers: 26 passed) for tests/services/test_project_removal_bug.py
and the CLI/MCP project-management integration suites.

Signed-off-by: phernandez <paul@basicmachines.co>
2026-05-16 15:28:41 -05:00
phernandez 2bd6552e84 fix(mcp): address PR review feedback (P2 + multi-project search)
Two follow-up fixes on the same branch:

1. **Purge sqlite-vec embeddings during project delete** (Codex P2)
   sqlite-vec stores vectors in a vec0 virtual table keyed by chunk rowid
   with no cascade. The previous purge removed search_vector_chunks but
   left the embeddings behind; `_run_vector_query` then keeps returning
   stale vectors that crowd live results.

   ProjectRepository.delete now deletes embeddings first (using the same
   rowid-IN-chunks pattern as SQLiteSearchRepository.delete_project_vector_rows),
   then the chunk rows. Both deletes are skipped if the underlying table
   is absent on a given install. New test test_remove_project_purges_vector_embeddings
   covers the happy path and skips cleanly when the embeddings table
   isn't initialized.

2. **Fix search_all_projects=True on local installs**
   `_search_all_projects` recurses into search_notes with both project=
   and project_id= set. project_id (external UUID) routes through the
   cloud v2 API path, which 401s on local installs because there's no
   JWT to present — so the inner calls silently failed and the merged
   result list stayed empty.

   The fan-out now mirrors get_project_client's cloud_available composite
   (factory mode OR explicit --cloud OR has_cloud_credentials). When that
   composite is false we forward project= only and take the name-routed
   local-ASGI path. Cloud disambiguation still works because the project
   name in project_ref is already the workspace/project qualified_name.

   The existing cloud-style fan-out tests now go through a cloud_routing
   fixture that pins the three signals; a new local_routing test confirms
   project_id is dropped when no cloud route is available.

Signed-off-by: phernandez <paul@basicmachines.co>
2026-05-16 15:11:39 -05:00
phernandez 1768de8571 fix(core): skip absent search tables in project delete
search_index is created lazily by SearchRepository.init_search_index and
search_vector_chunks only materializes once semantic search initializes,
so either table may be absent on minimal test DBs. The previous version
of ProjectRepository.delete unconditionally issued DELETEs against both
and crashed CLI integration tests on Postgres and SQLite where
search_vector_chunks hadn't been created yet:

  relation "search_vector_chunks" does not exist
  no such table: search_vector_chunks

Inspect the connection's tables once per call and only delete from
whichever derived tables are present. Idempotent on every backend.

Signed-off-by: phernandez <paul@basicmachines.co>
2026-05-16 14:25:07 -05:00
phernandez a85767ad92 fix(core): purge SQLite search_index on project delete
SQLite stores search_index as an FTS5 virtual table, which can't carry a
foreign key, so the ON DELETE CASCADE from search_index.project_id to
project.id only applies on Postgres. On SQLite, deleting a project left
its FTS rows behind — and when auto-increment handed the same id to a
new project, the leftover rows masqueraded as the new tenant's data and
leaked into searches scoped to that project.

- ProjectRepository.delete now explicitly purges search_index and
  search_vector_chunks for the project id in the same session before
  the ORM delete. Idempotent on Postgres (the cascade FK still runs).
- One-time cleanup migration sweeps two leftover shapes: rows whose
  project_id is gone, and rows whose entity_id is gone (the larger
  class from id reuse). Guarded by table-existence checks so fresh
  SQLite installs — where search_index is created at runtime by
  init_search_index, not by Alembic — don't fail the upgrade.
- Regression test seeds both derived tables, calls remove_project,
  and asserts both come out clean. Verified red on pre-fix code.

Signed-off-by: phernandez <paul@basicmachines.co>
2026-05-16 11:43:16 -05:00
5 changed files with 552 additions and 12 deletions
@@ -0,0 +1,86 @@
"""Remove orphaned search rows whose project was already deleted.
Revision ID: n7i8j9k0l1m2
Revises: m6h7i8j9k0l1
Create Date: 2026-05-15 18:30:00.000000
"""
from typing import Sequence, Union
from alembic import op
from sqlalchemy import inspect
revision: str = "n7i8j9k0l1m2"
down_revision: Union[str, None] = "m6h7i8j9k0l1"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def _table_exists(connection, table_name: str) -> bool:
"""Inspector-based table check, dialect agnostic.
Trigger: SQLite creates search_index as an FTS5 virtual table at runtime
via SearchRepository.init_search_index, not through Alembic, so fresh
installs hit this migration before the table exists.
Why: a blind DELETE against a missing table fails the whole upgrade.
Outcome: callers skip the sweep when the table isn't present yet — the
runtime-created table on a fresh DB has no orphans to clean.
"""
return table_name in inspect(connection).get_table_names()
def upgrade() -> None:
"""Purge orphaned search rows left over from prior project deletions.
Trigger: project deletion on SQLite never removed the derived FTS rows,
because the FTS5 virtual table can't carry a foreign key. The leak shows
up in two shapes:
1. project_id no longer exists in `project` (deleted project, id never
reused).
2. project_id still exists but `entity_id` no longer exists in `entity`
— auto-increment handed the id to a brand-new project and the FTS
rows from the deleted predecessor masquerade as the new tenant's data.
Why: search_index.project_id is the only scope predicate the search
repository applies, so leftover rows surface under the wrong project on
every search.
Outcome: a one-time sweep deletes both shapes, from the FTS index and
from search_vector_chunks. Postgres already cascaded on FK delete, so
these statements are no-ops there.
"""
connection = op.get_bind()
if _table_exists(connection, "search_index"):
op.execute(
"""
DELETE FROM search_index
WHERE project_id NOT IN (SELECT id FROM project)
"""
)
op.execute(
"""
DELETE FROM search_index
WHERE entity_id IS NOT NULL
AND entity_id NOT IN (SELECT id FROM entity)
"""
)
if _table_exists(connection, "search_vector_chunks"):
op.execute(
"""
DELETE FROM search_vector_chunks
WHERE project_id NOT IN (SELECT id FROM project)
"""
)
op.execute(
"""
DELETE FROM search_vector_chunks
WHERE entity_id NOT IN (SELECT id FROM entity)
"""
)
def downgrade() -> None:
"""No-op: orphan rows cannot be reconstructed."""
pass
+13 -2
View File
@@ -479,12 +479,23 @@ async def _search_all_projects(
total = 0
any_project_has_more = False
# Trigger: caller asked for an account-wide search.
# Why: forwarding project_id (external UUID) routes the per-project search
# through get_project_client's UUID branch, which treats unknown
# identifiers as cloud and 401s when local OAuth tokens linger from a
# past `bm cloud login` even though the project itself is local.
# Outcome: route by `project` name (qualified_name like "workspace/project"
# in cloud refs, plain name in local refs) — both backends resolve
# that without UUID lookup. project_id is only used as a fallback when
# a ref unexpectedly has no name to route by.
for project_ref in project_refs:
project_name = project_ref["project"]
recursive_project_id = None if project_name else project_ref["project_id"]
try:
results = await search_notes(
query=query,
project=project_ref["project"],
project_id=project_ref["project_id"],
project=project_name,
project_id=recursive_project_id,
page=1,
page_size=per_project_page_size,
search_type=search_type,
@@ -4,7 +4,9 @@ from pathlib import Path
from typing import Optional, Sequence, Union
from sqlalchemy import text
from loguru import logger
from sqlalchemy import inspect as sa_inspect, select, text
from sqlalchemy.exc import NoResultFound, OperationalError
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
from basic_memory import db
@@ -12,6 +14,65 @@ from basic_memory.models.project import Project
from basic_memory.repository.repository import Repository
async def _load_sqlite_vec_on_session(session) -> bool:
"""Ensure the sqlite-vec extension is loaded on this session's connection.
Returns True when vec0 is available after the call. Returns False when the
extension can't be loaded on this Python build (e.g., python.org macOS or
Windows interpreters without `enable_load_extension`) — every connection in
the pool shares the same interpreter, so a False here also means no
embedding row could ever have been written, and skipping the embeddings
purge is safe.
Mirrors SQLiteSearchRepository._ensure_sqlite_vec_loaded but as a free
function: we don't have a SearchRepository instance during project delete,
and the per-connection nature of extension loading means a pooled connection
routed to this session might not have vec loaded even when other
connections wrote embeddings.
"""
try:
await session.execute(text("SELECT vec_version()"))
return True
except OperationalError:
pass
try:
import sqlite_vec # type: ignore[import-not-found]
except ImportError:
logger.debug("sqlite-vec package not installed; skipping vec purge")
return False
async_connection = await session.connection()
raw_connection = await async_connection.get_raw_connection()
driver_connection = raw_connection.driver_connection
if not hasattr(driver_connection, "enable_load_extension"):
# Trigger: CPython build without sqlite extension support (#711).
# Why: load_extension is unavailable, so no connection in this pool
# can host vec0. No embeddings exist anywhere.
# Outcome: skip the embeddings purge entirely.
logger.debug(
"Skipping search_vector_embeddings purge: this Python build does "
"not support SQLite extension loading"
)
return False
try:
await driver_connection.enable_load_extension(True)
await driver_connection.load_extension(sqlite_vec.loadable_path())
await driver_connection.enable_load_extension(False)
await session.execute(text("SELECT vec_version()"))
except Exception as exc:
logger.warning(
"Failed to load sqlite-vec for project delete cleanup; "
"skipping embeddings purge: {}",
exc,
)
return False
return True
class ProjectRepository(Repository[Project]):
"""Repository for Project model.
@@ -121,6 +182,83 @@ class ProjectRepository(Repository[Project]):
return target_project
return None # pragma: no cover
async def delete(self, entity_id: int) -> bool:
"""Delete a project and its derived search rows in one transaction.
The cascade picture differs by backend:
- search_index → project: Postgres has ON DELETE CASCADE FK; SQLite
stores search_index as an FTS5 virtual table and can't carry FKs,
so it needs explicit cleanup.
- search_vector_chunks → project: neither backend has an FK here, so
both need an explicit DELETE.
- search_vector_embeddings → search_vector_chunks: Postgres has an FK
(chunk_id REFERENCES … ON DELETE CASCADE); SQLite stores embeddings
in a vec0 virtual table keyed by rowid with no cascade. On SQLite
the embeddings must be purged before the chunk rows, otherwise
`_run_vector_query` keeps returning stale vectors that crowd out
live results.
Each derived table is created lazily (search_index by
SearchRepository.init_search_index, the vector tables once semantic
search initializes), so any of them may be absent on minimal test DBs.
Inspect the connection once and skip whichever is missing.
"""
logger.debug(f"Deleting Project and search rows for project_id: {entity_id}")
async with db.scoped_session(self.session_maker) as session:
try:
result = await session.execute(
select(self.Model).filter(self.primary_key == entity_id)
)
project = result.scalars().one()
except NoResultFound:
logger.debug(f"No Project found to delete: {entity_id}")
return False
dialect_name = session.bind.dialect.name if session.bind else "sqlite"
is_sqlite = dialect_name == "sqlite"
existing_tables = await session.run_sync(
lambda sync_session: set(sa_inspect(sync_session.connection()).get_table_names())
)
# search_index: SQLite has no FK on the FTS5 virtual table; Postgres
# cascades from the project FK, so the explicit DELETE is redundant.
if is_sqlite and "search_index" in existing_tables:
await session.execute(
text("DELETE FROM search_index WHERE project_id = :project_id"),
{"project_id": entity_id},
)
# search_vector_chunks: no FK to project on either backend, so both
# backends need this. SQLite must purge vec0 embeddings first
# (rowid pseudocolumn — Postgres uses chunk_id and would 500 here);
# Postgres' chunk_id FK CASCADE handles its embeddings cleanup when
# we delete the chunk rows below.
if "search_vector_chunks" in existing_tables:
if is_sqlite and "search_vector_embeddings" in existing_tables:
# Extension loading is per-connection. We must load vec0 on
# *this* session before the DELETE; otherwise a different
# pooled connection might have written embeddings that we'd
# silently leave behind.
if await _load_sqlite_vec_on_session(session):
await session.execute(
text(
"DELETE FROM search_vector_embeddings WHERE rowid IN ("
"SELECT id FROM search_vector_chunks "
"WHERE project_id = :project_id)"
),
{"project_id": entity_id},
)
await session.execute(
text("DELETE FROM search_vector_chunks WHERE project_id = :project_id"),
{"project_id": entity_id},
)
await session.delete(project)
logger.debug(f"Deleted Project and search rows for project_id: {entity_id}")
return True
async def update_path(self, project_id: int, new_path: str) -> Optional[Project]:
"""Update project path.
@@ -32,7 +32,10 @@ async def test_search_notes_search_all_projects_qualifies_result_permalinks(monk
class StubProject:
def __init__(self, name: str | None, external_id: str | None):
self.name = name or "main"
self.external_id = external_id or "local-main"
# When the fan-out routes by name only, project_id is None — keep
# the test stable by falling back to the name so the downstream
# SearchClient still has a stable per-project identifier.
self.external_id = external_id or self.name
@asynccontextmanager
async def fake_get_project_client(project=None, context=None, project_id=None):
@@ -44,10 +47,12 @@ async def test_search_notes_search_all_projects_qualifies_result_permalinks(monk
class MockSearchClient:
def __init__(self, client, project_id):
# The fake project supplies the workspace-qualified name as the
# external_id, so each per-project search keys off that.
self.project_id = project_id
async def search(self, payload, page, page_size):
if self.project_id == "11111111-1111-1111-1111-111111111111":
if self.project_id == "personal/main":
title = "Personal MCP Test Note"
score = 0.5
else:
@@ -81,9 +86,13 @@ async def test_search_notes_search_all_projects_qualifies_result_permalinks(monk
)
assert isinstance(result, dict)
# Fan-out now routes by qualified_name only; project_id is omitted because
# the workspace-qualified name is already unambiguous and UUID routing
# hits get_project_client's cloud branch even for local projects when
# local OAuth credentials are present.
assert searched_projects == [
("personal/main", "11111111-1111-1111-1111-111111111111"),
("team-paul/main", "22222222-2222-2222-2222-222222222222"),
("personal/main", None),
("team-paul/main", None),
]
assert [item["permalink"] for item in result["results"]] == [
"team-paul/main/tests/mcp-test-note",
@@ -168,9 +177,7 @@ async def test_search_notes_search_all_projects_with_no_refs_returns_empty_all_p
@pytest.mark.asyncio
async def test_search_notes_search_all_projects_continues_after_project_failure(
monkeypatch,
):
async def test_search_notes_search_all_projects_continues_after_project_failure(monkeypatch):
"""One failing project should not discard successful all-project search results."""
clients_mod = importlib.import_module("basic_memory.mcp.clients")
search_mod = importlib.import_module("basic_memory.mcp.tools.search")
@@ -193,7 +200,7 @@ async def test_search_notes_search_all_projects_continues_after_project_failure(
class StubProject:
def __init__(self, name: str | None, external_id: str | None):
self.name = name or "main"
self.external_id = external_id or "local-main"
self.external_id = external_id or self.name
@asynccontextmanager
async def fake_get_project_client(project=None, context=None, project_id=None):
@@ -214,10 +221,12 @@ async def test_search_notes_search_all_projects_continues_after_project_failure(
class MockSearchClient:
def __init__(self, client, project_id):
# Fan-out routes by name now, so the stub project reflects the
# workspace-qualified name back as the SearchClient's project_id.
self.project_id = project_id
async def search(self, payload, page, page_size):
if self.project_id == "22222222-2222-2222-2222-222222222222":
if self.project_id == "team-paul/main":
raise RuntimeError("team index unavailable")
return SearchResponse(
results=[
@@ -254,3 +263,85 @@ async def test_search_notes_search_all_projects_continues_after_project_failure(
assert result["total"] == 1
assert any("team-paul/main" in warning for warning in warnings)
assert any("team index unavailable" in warning for warning in warnings)
@pytest.mark.asyncio
async def test_search_notes_search_all_projects_omits_project_id(monkeypatch):
"""Fan-out must address each project by name, never by external UUID.
project_id routes through get_project_client's UUID branch, which treats
any unknown identifier as cloud — so when a local install still has OAuth
tokens from a past `bm cloud login`, every per-project recursive call 401s
and the merged result list silently stays empty. Routing by name avoids
that path on both backends; cloud refs disambiguate via the
workspace/project qualified_name already baked into project_ref["project"].
"""
clients_mod = importlib.import_module("basic_memory.mcp.clients")
search_mod = importlib.import_module("basic_memory.mcp.tools.search")
project_refs = [
{
"project": "alpha",
"project_id": "11111111-1111-1111-1111-111111111111",
},
{
"project": "beta",
"project_id": "22222222-2222-2222-2222-222222222222",
},
]
searched_projects: list[tuple[str | None, str | None]] = []
async def fake_load_search_project_refs(context=None):
return project_refs
class StubProject:
def __init__(self, name: str | None, external_id: str | None):
self.name = name or "main"
self.external_id = external_id or "local-main"
@asynccontextmanager
async def fake_get_project_client(project=None, context=None, project_id=None):
searched_projects.append((project, project_id))
yield object(), StubProject(project, project_id)
async def fake_resolve_project_and_path(client, identifier, project=None, context=None):
return StubProject(project, None), identifier, False
class MockSearchClient:
def __init__(self, client, project_id):
self.project_id = project_id
async def search(self, payload, page, page_size):
return SearchResponse(
results=[
SearchResult(
title=f"Note in {self.project_id or 'local'}",
permalink="notes/example",
content="",
type=SearchItemType.ENTITY,
score=0.5,
file_path="/notes/example.md",
)
],
current_page=page,
page_size=page_size,
total=1,
)
monkeypatch.setattr(search_mod, "_load_search_project_refs", fake_load_search_project_refs)
monkeypatch.setattr(search_mod, "get_project_client", fake_get_project_client)
monkeypatch.setattr(search_mod, "resolve_project_and_path", fake_resolve_project_and_path)
monkeypatch.setattr(clients_mod, "SearchClient", MockSearchClient)
result = await search_mod.search_notes(
query="anything",
search_all_projects=True,
output_format="json",
)
assert isinstance(result, dict)
assert searched_projects == [("alpha", None), ("beta", None)], (
"Local fan-out must omit project_id so the recursive search_notes calls "
"take the name-routed path."
)
assert result["total"] == 2
+214
View File
@@ -6,7 +6,9 @@ from datetime import timezone, datetime
from pathlib import Path
import pytest
from sqlalchemy import text
from basic_memory import db
from basic_memory.services.project_service import ProjectService
@@ -136,3 +138,215 @@ async def test_remove_project_with_related_entities(project_service: ProjectServ
project = await project_service.get_project(test_project_name)
if project:
await project_service.repository.delete(project.id)
async def _table_exists(session_maker, table: str) -> bool:
"""Return True if the named table is present on the current connection."""
from sqlalchemy import inspect as sa_inspect
async with db.scoped_session(session_maker) as session:
return await session.run_sync(
lambda sync_session: table in sa_inspect(sync_session.connection()).get_table_names()
)
@pytest.mark.asyncio
async def test_remove_project_purges_search_rows(project_service: ProjectService):
"""Project deletion must sweep the derived search tables.
SQLite stores search_index as an FTS5 virtual table, which cannot carry a
foreign key, so without an explicit purge the FTS rows survive the project
and leak into the next project that reuses the same auto-increment id.
Postgres has the cascade FK, but we expect the same end-state on either
backend. This test fails on the pre-fix code: search_index still holds the
project's rows after remove_project completes.
"""
test_project_name = f"test-search-cleanup-{os.urandom(4).hex()}"
with tempfile.TemporaryDirectory() as temp_dir:
test_project_path = str(Path(temp_dir) / "test-search-cleanup")
os.makedirs(test_project_path, exist_ok=True)
await project_service.add_project(test_project_name, test_project_path)
project = await project_service.get_project(test_project_name)
assert project is not None
project_id = project.id
# Seed both derived tables directly. The bug is in the cleanup path,
# not the indexer, so a synthetic row is enough to prove the sweep.
async with db.scoped_session(project_service.repository.session_maker) as session:
await session.execute(
text(
"INSERT INTO search_index "
"(id, title, content_stems, content_snippet, permalink, "
" file_path, type, project_id) "
"VALUES (:id, :title, :stems, :snippet, :permalink, "
" :file_path, :type, :project_id)"
),
{
"id": 999_001,
"title": "leak canary",
"stems": "leak canary",
"snippet": "leak canary",
"permalink": f"leak-canary-{project_id}",
"file_path": "leak-canary.md",
"type": "entity",
"project_id": project_id,
},
)
await session.execute(
text(
"INSERT INTO search_vector_chunks "
"(entity_id, project_id, chunk_key, chunk_text, source_hash, "
" entity_fingerprint, embedding_model) "
"VALUES (:entity_id, :project_id, :chunk_key, :chunk_text, "
" :source_hash, :entity_fingerprint, :embedding_model)"
),
{
"entity_id": 999_001,
"project_id": project_id,
"chunk_key": "canary",
"chunk_text": "leak canary",
"source_hash": "abc",
"entity_fingerprint": "",
"embedding_model": "",
},
)
async with db.scoped_session(project_service.repository.session_maker) as session:
pre_index = (
await session.execute(
text("SELECT COUNT(*) FROM search_index WHERE project_id = :pid"),
{"pid": project_id},
)
).scalar_one()
pre_chunks = (
await session.execute(
text("SELECT COUNT(*) FROM search_vector_chunks WHERE project_id = :pid"),
{"pid": project_id},
)
).scalar_one()
assert pre_index >= 1, "seed row should exist before removal"
assert pre_chunks >= 1, "seed chunk should exist before removal"
await project_service.remove_project(test_project_name)
async with db.scoped_session(project_service.repository.session_maker) as session:
post_index = (
await session.execute(
text("SELECT COUNT(*) FROM search_index WHERE project_id = :pid"),
{"pid": project_id},
)
).scalar_one()
post_chunks = (
await session.execute(
text("SELECT COUNT(*) FROM search_vector_chunks WHERE project_id = :pid"),
{"pid": project_id},
)
).scalar_one()
assert post_index == 0, (
f"search_index still has {post_index} rows for deleted project_id={project_id} "
"— project deletion did not sweep the FTS table."
)
assert post_chunks == 0, (
f"search_vector_chunks still has {post_chunks} rows for deleted "
f"project_id={project_id}."
)
@pytest.mark.asyncio
async def test_delete_returns_false_for_missing_project_id(project_service: ProjectService):
"""ProjectRepository.delete must return False when the project id is gone.
The override loses the base Repository.delete contract if the NoResultFound
branch isn't covered — a silent True would mislead callers into thinking
a non-existent project was removed.
"""
result = await project_service.repository.delete(9_999_999)
assert result is False
@pytest.mark.asyncio
async def test_remove_project_purges_vector_embeddings(project_service: ProjectService):
"""Project deletion must also drop sqlite-vec embeddings keyed by chunk rowid.
sqlite-vec stores vectors in a vec0 virtual table that has no cascade
behavior. If embeddings linger after the chunks they reference are gone,
`_run_vector_query` pulls them as top-k candidates and crowds out live
results. The test only runs when the embeddings table is present, which
matches the install path that exercises semantic search.
"""
test_project_name = f"test-vec-cleanup-{os.urandom(4).hex()}"
session_maker = project_service.repository.session_maker
# The embeddings table only exists once semantic search has initialized.
# Skipping when it's absent keeps this test honest on minimal CI DBs.
if not await _table_exists(session_maker, "search_vector_embeddings"):
pytest.skip("search_vector_embeddings is not present on this connection")
with tempfile.TemporaryDirectory() as temp_dir:
test_project_path = str(Path(temp_dir) / "test-vec-cleanup")
os.makedirs(test_project_path, exist_ok=True)
await project_service.add_project(test_project_name, test_project_path)
project = await project_service.get_project(test_project_name)
assert project is not None
project_id = project.id
async with db.scoped_session(session_maker) as session:
await session.execute(
text(
"INSERT INTO search_vector_chunks "
"(id, entity_id, project_id, chunk_key, chunk_text, source_hash, "
" entity_fingerprint, embedding_model) "
"VALUES (:id, :entity_id, :project_id, :chunk_key, :chunk_text, "
" :source_hash, :entity_fingerprint, :embedding_model)"
),
{
"id": 999_201,
"entity_id": 999_201,
"project_id": project_id,
"chunk_key": "vec-canary",
"chunk_text": "vec canary",
"source_hash": "abc",
"entity_fingerprint": "",
"embedding_model": "",
},
)
# vec0 requires a vector matching the configured dimensions, but the
# delete path filters by rowid; a non-existing dimension would block
# this seed step. Skip the insert if the embeddings DDL hasn't run.
try:
await session.execute(
text(
"INSERT INTO search_vector_embeddings (rowid, embedding) "
"VALUES (:rowid, :embedding)"
),
{"rowid": 999_201, "embedding": "[" + ",".join(["0.0"] * 384) + "]"},
)
except Exception:
pytest.skip("search_vector_embeddings rejected the synthetic seed row")
async with db.scoped_session(session_maker) as session:
pre = (
await session.execute(
text("SELECT COUNT(*) FROM search_vector_embeddings WHERE rowid = :rowid"),
{"rowid": 999_201},
)
).scalar_one()
assert pre >= 1, "seed embedding should exist before removal"
await project_service.remove_project(test_project_name)
async with db.scoped_session(session_maker) as session:
post = (
await session.execute(
text("SELECT COUNT(*) FROM search_vector_embeddings WHERE rowid = :rowid"),
{"rowid": 999_201},
)
).scalar_one()
assert post == 0, (
f"search_vector_embeddings still has {post} rows for rowid 999_201 "
"— project deletion did not sweep the sqlite-vec embeddings table."
)