basicmachines-co-basic-memory/docs/releases/v0.19.0.md

v0.19.0 Release Notes

Overview

v0.19.0 is a major release that introduces semantic vector search, a schema validation system, project-prefixed permalinks, per-project cloud routing, and a significant upgrade to FastMCP 3.0. It includes 90+ commits since v0.18.0 spanning new features, architectural improvements, and stability fixes across both SQLite and Postgres backends.

---

Major Features

Semantic Vector Search

Full vector and hybrid search for SQLite (via sqlite-vec) and Postgres (via pgvector).

• Hybrid search mode combines full-text search (FTS) with vector similarity for best results
• Score-based fusion replaces RRF for hybrid ranking — max(vec, fts) + 0.3 * min(vec, fts) preserves dominant signals and rewards dual-source agreement (#577)
• Default search mode is now hybrid when semantic search is enabled, text when disabled
• Embedding providers: FastEmbed (local, default) or OpenAI API
• Configurable similarity threshold via semantic_min_similarity (default 0.55)
• Per-query min_similarity override on search_notes tool
• Auto-backfill: existing entities get embeddings generated on first startup
• Backend-specific distance-to-similarity conversion (cosine for SQLite, inner product for Postgres)
• FTS fallback: if semantic dependencies are missing, search gracefully degrades to text-only
• sqlite-vec knn k parameter capped at 4096 to prevent backend errors

Configuration:

{
  "semantic_search_enabled": true,
  "semantic_embedding_provider": "fastembed",
  "semantic_embedding_model": "bge-small-en-v1.5",
  "semantic_min_similarity": 0.55
}

Usage:

search_notes("machine learning concepts", search_type="hybrid")
search_notes("similar to my notes on coffee", search_type="vector")
search_notes("exact phrase match", search_type="text")
search_notes("broad search", min_similarity=0.3)  # lower threshold for more results

Schema System

Validate note structure against user-defined schemas with frontmatter-based rules.

• Define schemas as YAML in note frontmatter with field types, required fields, and constraints
• Frontmatter validation during sync — malformed notes get clear error messages
• Schema inference from existing notes to bootstrap schemas from your content
• Schema diff to compare two schemas and see changes
• Available via MCP tools and CLI

Project-Prefixed Permalinks

Permalinks now include the project name for unambiguous cross-project references.

• Memory URLs like memory://project-name/folder/note route to the correct project
• Existing non-prefixed permalinks continue to work (backwards compatible)
• Controlled by permalinks_include_project config (default: true)
• build_context and search_notes auto-detect project from URL prefix

Per-Project Cloud Routing

Individual projects can be routed through the cloud while others stay local.

• Set a project to cloud mode: bm project set-cloud research
• Revert to local: bm project set-local research
• Uses API key authentication: bm cloud set-key bmc_abc123...
• MCP tools automatically route based on each project's mode
• Local MCP server (bm mcp) still uses local routing for all projects by default
• --local and --cloud CLI flags override per-command

Workspace Selection

Cloud projects can target specific workspaces for multi-tenant environments.

• workspace parameter on MCP tools for explicit workspace targeting
• CLI workspace-aware project listing with bm project list
• Spinner feedback while fetching cloud projects

---

New Tools and Capabilities

Dashboard (bm project info)

bm project info now displays an htop-inspired compact dashboard with:

• Horizontal bar charts for note types (top 5)
• Embedding coverage bar with Unicode block characters
• Colored status dots for at-a-glance health
• EmbeddingStatus schema and get_embedding_status() service method for programmatic access

Unified Metadata Search

search_by_metadata has been merged into search_notes — one tool for all searches. query is now optional, so you can search purely by frontmatter metadata.

search_notes(metadata_filters={"status": "in-progress"})
search_notes(metadata_filters={"tags": ["security", "oauth"]})
search_notes(metadata_filters={"priority": {"$in": ["high", "critical"]}})
search_notes(metadata_filters={"schema.confidence": {"$gt": 0.7}})
search_notes(tags=["security"])  # convenience shorthand
search_notes(status="draft")     # convenience shorthand

JSON Output Mode

All MCP tools now support output_format="json" for machine-readable responses.

• Default remains "text" for human-readable output (no breaking changes)
• build_context defaults to "json" with slimmed payloads (redundant fields stripped)
• CLI tool commands support --format json flag

tag: Search Shorthand

Search by tag using convenient shorthand syntax.

search_notes("tag:security")
search_notes("tag:coffee AND tag:brewing")

Entity User Tracking

Entities now track created_by and last_updated_by fields for attribution.

Improved Search Result Content (#609)

Search results now surface more relevant context:

• matched_chunk_text populated for FTS-only hybrid results (no more fallback to truncated content)
• TOP_CHUNKS_PER_RESULT increased from 3 to 5, catching answers deeper in large notes (~2700 → ~4500 chars)
• CONTENT_DISPLAY_LIMIT doubled from 2000 to 4000 chars for results without matched chunks

write_note Overwrite Guard (#632)

write_note is now non-idempotent by default. If a note already exists, the tool returns an error instead of silently overwriting. Pass overwrite=True to replace, or use edit_note for incremental updates. Config option write_note_overwrite_default restores the old upsert behavior.

---

Architecture Changes

Score-Based Hybrid Fusion (#577)

RRF (Reciprocal Rank Fusion) compressed all fused scores to ~0.016, destroying ranking differentiation. The new formula max(vec, fts) + FUSION_BONUS * min(vec, fts) preserves dominant signals and rewards dual-source agreement. Zero-score results now produce zero fused score instead of receiving a 0.1 weight floor.

FastMCP 3.0 Upgrade

Upgraded from FastMCP 2.12.3 to 3.0.1.

• Tool annotations (readOnlyHint, openWorldHint) for better client integration
• Improved MCP protocol compliance
• Better error handling and context management

Prompts Call MCP Tools Directly

MCP prompts (search, continue_conversation) now call MCP tools directly instead of going through API endpoints. This fixes empty results in discovery mode and ensures prompts use the same resolution logic as tools (including LinkResolver fallback).

build_context LinkResolver Fallback

build_context now falls back to LinkResolver when an exact permalink lookup returns empty. This uses the same 7-strategy resolution pipeline as read_note, so callers no longer get empty results for valid note identifiers that don't match exact permalinks.

Sync Handles Semantic Dependency Errors Gracefully

When sqlite-vec or another embedding provider is unavailable, sync_file now catches SemanticDependenciesMissingError separately. The entity is created and FTS-indexed successfully — only vector embeddings are skipped, with a clear warning:

WARNING: Semantic search dependencies missing — vector embeddings skipped for path=note.md.
Run 'bm reindex --embeddings' after resolving the dependency issue.

Unified Project Path

Cloud projects with bisync now store the local filesystem path in path (not the Docker container path). Config migration automatically promotes local_sync_path → path for existing configs.

---

CLI Improvements

Status and Doctor Default to Local Routing

bm status and bm doctor now default to local routing since they scan the local filesystem. Previously, cloud-mode projects would route these commands to the cloud API, which returned Docker-internal paths that don't exist locally.

--format json for CLI Tool Commands

All bm tool subcommands support --format json for machine-readable output, enabling integration with scripts and plugins.

--json for Top-Level CLI Commands

Five additional CLI commands now support --json for machine-readable output:

• bm status --json — sync report with new/modified/deleted/moved files and skipped files
• bm project list --json — structured project list with name, paths, routing mode, and defaults
• bm schema validate --json — validation report with per-note pass/fail, warnings, and errors
• bm schema infer --json — field frequency analysis and suggested schema definition
• bm schema diff --json — drift report with new fields, dropped fields, and cardinality changes

This complements the existing bm project info --json and bm tool --format json support, making all major CLI commands scriptable for CI pipelines and automation.

Cloud Promo and Analytics

• Cloud promo panel shown on first run or version bump with OSS discount code
• Anonymous usage telemetry via Umami Cloud (promo/login funnel events only)
• Opt out with BASIC_MEMORY_NO_PROMOS=1
• No PII, no file contents, no per-command tracking
• See Telemetry in README

---

Bug Fixes

• #577: RRF fusion compressed all hybrid scores to ~0.016, destroying ranking differentiation
• #582: build_context returns empty results on valid note identifiers
• #575: Remove hardcoded "main" default from default_project
• #595: recent_activity dedup and pagination across MCP tools
• #593: Backend-specific distance-to-similarity conversion
• #592: Strip NUL bytes from content before PostgreSQL search indexing
• #562: Use VIRTUAL instead of STORED columns in SQLite migration
• #558: Add X-Tigris-Consistent headers to all rclone commands
• #541: Handle EntityCreationError as conflict
• #536: Stabilize metadata filters on Postgres
• #533: Fix recent_activity prompt defaults
• #530: Prevent spurious metadata: {} in frontmatter output
• #601: Return matched chunk text in search results
• #606: Accept null for expected_replacements in edit_note
• #579, #607: Guard against closed streams in promo panel and missing vector tables on shutdown
• #609: FTS-only hybrid results missing matched_chunk_text; content limits too conservative
• #631: build_context related_results schema validation failure — replaced fragile _slim_context() stripping with Pydantic exclude=True field config
• #630: Skip workspace resolution when client factory is active — prevents 401 errors in cloud MCP server mode
• #30: tag: prefix query fails with hybrid search — moved tag prefix parsing to MCP tool level so it works with all search modes
• #31: search_notes returns cluttered observation/relation-level results — now defaults to entity-level results
• #28: schema_infer and schema_diff return raw Pydantic models as "undefined" in LLM output — added markdown formatters
• Fix schema_validate identifier resolution (now uses LinkResolver) and text rendering (markdown formatter)
• #634: schema_validate and schema_diff use stale database metadata instead of reading schema definitions from file — now reads frontmatter directly from the file with fallback to database metadata
• Fix Post(**metadata) crash when frontmatter contains content or handler keys
• Fix list-valued frontmatter fields (title, type) crashing on .strip() — now coerced to strings
• Cap sqlite-vec knn k parameter at 4096 to prevent backend errors
• Parameterize SQL queries in search repository type filters
• Double-default display in project list
• ensure_frontmatter_on_sync default changed to True
• Status/doctor commands fail with cloud-mode projects (Docker path error)
• Prompts return "0 projects" in discovery mode

---

Security

• Upgrade cryptography for CVE advisory
• Upgrade python-multipart for security advisory

---

Internal / Developer

• #598: Upgrade FastMCP 2.12.3 → 3.0.1 with tool annotations
• #594: Add ty as supplemental type checker
• #538: Add fast feedback loop tooling (just fast-check, just doctor, just testmon)
• #600: Rename entity_type to note_type for consistency
• #596: Fix CLI runtime defects and audit regressions
• CLI refactoring and workspace-aware cloud project listing
• Split and speed up PR test matrix in CI
• Fix CI: collect coverage from test jobs instead of re-running all tests
• Create search_vector_chunks in test fixtures for Postgres compatibility

---

Configuration Changes

Setting	Old Default	New Default	Notes
semantic_search_enabled	false	true	Semantic search on by default
ensure_frontmatter_on_sync	false	true	Frontmatter added during sync
permalinks_include_project	false	true	Project prefix in permalinks

---

Upgrade Notes

• Semantic search dependencies are now included by default. If sqlite-vec fails to load, search gracefully falls back to FTS. Run bm reindex --embeddings to generate embeddings for existing content.
• Hybrid search scoring has changed from RRF to score-based fusion. Search result ordering may differ — results should be more accurate with better score differentiation.
• search_by_metadata is removed as a standalone tool. Use search_notes with metadata_filters instead (same parameters, same behavior).
• Project-prefixed permalinks are enabled by default. Existing notes keep their current permalinks until modified. Set permalinks_include_project: false to disable.
• Frontmatter on sync is now enabled by default. Files without frontmatter will have it added on next sync. Set ensure_frontmatter_on_sync: false to preserve old behavior.
• Config migration runs automatically for cloud projects with bisync — local_sync_path is promoted to path so filesystem operations work correctly.
• write_note is no longer idempotent — calls to write_note for existing notes now return an error unless overwrite=True is passed. Use edit_note for incremental changes, or set write_note_overwrite_default: true in config to restore the old behavior.