Compare commits

..

25 Commits

Author SHA1 Message Date
phernandez de7f15b7a2 chore: update version to 0.18.4 for v0.18.4 release
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: phernandez <paul@basicmachines.co>
2026-02-12 15:28:02 -06:00
phernandez 630eeb94ab docs: add CHANGELOG entry for v0.18.4
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: phernandez <paul@basicmachines.co>
2026-02-12 15:28:02 -06:00
phernandez 0eae0e1678 fix: use global --header flag for Tigris consistency on all rclone transactions
--header-download and --header-upload only apply to GET and PUT requests.
bisync starts by listing files via S3 ListObjectsV2 — neither a download
nor upload — so the consistency header was never sent on list requests.
Non-US users saw stale edge-cached metadata and bisync reported "0 changes."

--header is rclone's global flag that applies to ALL HTTP transactions
(list, download, upload), which is a superset of the previous two flags.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: phernandez <paul@basicmachines.co>
2026-02-12 15:27:32 -06:00
phernandez 88a5b07b89 chore: update version to 0.18.2 for v0.18.2 release
Signed-off-by: phernandez <paul@basicmachines.co>
2026-02-11 22:33:56 -06:00
phernandez 59e8a937ee fix: remove unused TIGRIS_CONSISTENCY_HEADERS import
Signed-off-by: phernandez <paul@basicmachines.co>
2026-02-11 22:33:36 -06:00
phernandez c07465d904 docs: add CHANGELOG entry for v0.18.2
Signed-off-by: phernandez <paul@basicmachines.co>
2026-02-11 22:33:10 -06:00
Paul Hernandez dfb89e841c fix: use VIRTUAL instead of STORED columns in SQLite migration (#562)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 22:32:50 -06:00
phernandez 00537272c6 chore: update version to 0.18.1 for v0.18.1 release 2026-02-11 14:28:52 -06:00
phernandez b057912452 docs: add CHANGELOG entry for v0.18.1
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: phernandez <paul@basicmachines.co>
2026-02-11 14:24:26 -06:00
Paul Hernandez 8489a3d37e fix: add X-Tigris-Consistent headers to all rclone commands (#558)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 14:22:49 -06:00
Paul Hernandez a47c9c021f feat: add --format json to CLI tool commands (#552)
Signed-off-by: phernandez <paul@basicmachines.co>
Signed-off-by: bm-clawd <clawd@basicmemory.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: bm-clawd <clawd@basicmemory.com>
2026-02-08 14:56:29 -06:00
phernandez c46d7a6833 fix: add POST legacy compat routes for v0.18.0 CLI
The v0.18.0 CLI (Homebrew) calls POST /projects/projects for project add
and POST /projects/config/sync for config sync. The previous legacy compat
fix (a0e754b) only added GET for list_projects but missed POST endpoints.

This caused 405 Method Not Allowed when running `bm project add` in cloud mode.

🔗 Logfire trace: trace_id=019c398cb257f040c1255821b6d5e385

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: phernandez <paul@basicmachines.co>
2026-02-07 13:41:41 -06:00
Paul Hernandez 343a6e118b fix: Handle EntityCreationError as conflict (#541)
Signed-off-by: phernandez <paul@basicmachines.co>
2026-02-03 22:28:58 -06:00
phernandez a0e754b7ae fix: restore legacy /projects/projects endpoint for older CLI versions
Older versions of basic-memory CLI (v0.17.4 and earlier) call
GET /projects/projects to list projects. This endpoint was removed
when we migrated to v2 routers.

Add explicit route at /projects/projects (without trailing slash) to
avoid 307 redirects that the cloud proxy doesn't follow.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: phernandez <paul@basicmachines.co>
2026-02-02 22:26:48 -06:00
Paul Hernandez 24ca5f6804 fix: recent_activity prompt defaults (#533)
Signed-off-by: phernandez <paul@basicmachines.co>
2026-02-02 19:22:33 -06:00
Paul Hernandez f1d50c2ba7 feat: Support tag: query shorthand in search (#535)
Signed-off-by: phernandez <paul@basicmachines.co>
2026-02-02 19:22:16 -06:00
Paul Hernandez 8072449a78 chore: Add fast feedback loop tooling (#538)
Signed-off-by: phernandez <paul@basicmachines.co>
2026-02-01 23:04:59 -06:00
phernandez 45d3f58e4d Merge branch 'main' of github.com:basicmachines-co/basic-memory 2026-02-01 21:17:39 -06:00
phernandez d9c8923148 fix ci runner for tests
Signed-off-by: phernandez <paul@basicmachines.co>
2026-02-01 21:16:10 -06:00
phernandez 15bd6b95ef fix ci runner i
Signed-off-by: phernandez <paul@basicmachines.co>
2026-02-01 21:10:26 -06:00
phernandez 0715dcff3d run ubuntu tests on depot
Signed-off-by: phernandez <paul@basicmachines.co>
2026-01-31 23:25:46 -06:00
Paul Hernandez 009e84926d fix: stabilize metadata filters on postgres (#536)
Signed-off-by: phernandez <paul@basicmachines.co>
2026-01-31 22:31:00 -06:00
phernandez 8838571509 Add metadata filter tests and fix fast write external_id
Signed-off-by: phernandez <paul@basicmachines.co>
2026-01-31 15:27:05 -06:00
Paul Hernandez 530cbac73f feat: fast edit entities, refactors for webui, enhance search (#532)
Signed-off-by: phernandez <paul@basicmachines.co>
2026-01-31 15:16:52 -06:00
Paul Hernandez e3ced49d9d fix: prevent spurious 'metadata: {}' in frontmatter output (#530)
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-31 12:17:10 -06:00
113 changed files with 4355 additions and 7539 deletions
+2
View File
@@ -1,6 +1,7 @@
*.py[cod]
__pycache__/
.pytest_cache/
.testmondata*
.coverage
htmlcov/
@@ -55,3 +56,4 @@ claude-output
**/.claude/settings.local.json
.mcp.json
.mcpregistry_*
/.testmondata
+408
View File
@@ -0,0 +1,408 @@
# AGENTS.md - Basic Memory Project Guide
## Project Overview
Basic Memory is a local-first knowledge management system built on the Model Context Protocol (MCP). It enables
bidirectional communication between LLMs (like Claude) and markdown files, creating a personal knowledge graph that can
be traversed using links between documents.
## CODEBASE DEVELOPMENT
### Project information
See the [README.md](README.md) file for a project overview.
### Build and Test Commands
- Install: `just install` or `pip install -e ".[dev]"`
- Run all tests (SQLite + Postgres): `just test`
- Run all tests against SQLite: `just test-sqlite`
- Run all tests against Postgres: `just test-postgres` (uses testcontainers)
- Run unit tests (SQLite): `just test-unit-sqlite`
- Run unit tests (Postgres): `just test-unit-postgres`
- Run integration tests (SQLite): `just test-int-sqlite`
- Run integration tests (Postgres): `just test-int-postgres`
- Run impacted tests: `just testmon` (pytest-testmon)
- Run MCP smoke test: `just test-smoke`
- Fast local loop: `just fast-check`
- Local consistency check: `just doctor`
- Generate HTML coverage: `just coverage`
- Single test: `pytest tests/path/to/test_file.py::test_function_name`
- Run benchmarks: `pytest test-int/test_sync_performance_benchmark.py -v -m "benchmark and not slow"`
- Lint: `just lint` or `ruff check . --fix`
- Type check: `just typecheck` or `uv run pyright`
- Format: `just format` or `uv run ruff format .`
- Run all code checks: `just check` (runs lint, format, typecheck, test)
- Create db migration: `just migration "Your migration message"`
- Run development MCP Inspector: `just run-inspector`
**Note:** Project requires Python 3.12+ (uses type parameter syntax and `type` aliases introduced in 3.12)
**Postgres Testing:** Uses [testcontainers](https://testcontainers-python.readthedocs.io/) which automatically spins up a Postgres instance in Docker. No manual database setup required - just have Docker running.
**Doctor Note:** `just doctor` runs with a temporary HOME/config so it won't touch your local Basic Memory settings. It leaves temp dirs in `/tmp` (safe to ignore or remove).
**Testmon Note:** When no files have changed, `just testmon` may collect 0 tests. That's expected and means no impacted tests were detected.
### Code/Test/Verify Loop (fast path)
1) **Code:** make changes.
2) **Test:** `just fast-check` (lint/format/typecheck + impacted tests + MCP smoke).
3) **Verify:** `just doctor` (end-to-end file ↔ DB loop in a temp project).
4) **Full gate (when needed):** `just test` or `just check` for SQLite + Postgres.
If testmon is “cold,” the first run may be long. Subsequent runs get much faster.
### Test Structure
- `tests/` - Unit tests for individual components (mocked, fast)
- `test-int/` - Integration tests for real-world scenarios (no mocks, realistic)
- Both directories are covered by unified coverage reporting
- Benchmark tests in `test-int/` are marked with `@pytest.mark.benchmark`
- Slow tests are marked with `@pytest.mark.slow`
- Smoke tests are marked with `@pytest.mark.smoke`
### Code Style Guidelines
- Line length: 100 characters max
- Python 3.12+ with full type annotations (uses type parameters and type aliases)
- Format with ruff (consistent styling)
- Import order: standard lib, third-party, local imports
- Naming: snake_case for functions/variables, PascalCase for classes
- Prefer async patterns with SQLAlchemy 2.0
- Use Pydantic v2 for data validation and schemas
- CLI uses Typer for command structure
- API uses FastAPI for endpoints
- Follow the repository pattern for data access
- Tools communicate to api routers via the httpx ASGI client (in process)
### Code Change Guidelines
- **Full file read before edits**: Before editing any file, read it in full first to ensure complete context; partial reads lead to corrupted edits
- **Minimize diffs**: Prefer the smallest change that satisfies the request. Avoid unrelated refactors or style rewrites unless necessary for correctness
- **No speculative getattr**: Never use `getattr(obj, "attr", default)` when unsure about attribute names. Check the class definition or source code first
- **Fail fast**: Write code with fail-fast logic by default. Do not swallow exceptions with errors or warnings
- **No fallback logic**: Do not add fallback logic unless explicitly told to and agreed with the user
- **No guessing**: Do not say "The issue is..." before you actually know what the issue is. Investigate first.
### Literate Programming Style
Code should tell a story. Comments must explain the "why" and narrative flow, not just the "what".
**Section Headers:**
For files with multiple phases of logic, add section headers so the control flow reads like chapters:
```python
# --- Authentication ---
# ... auth logic ...
# --- Data Validation ---
# ... validation logic ...
# --- Business Logic ---
# ... core logic ...
```
**Decision Point Comments:**
For conditionals that materially change behavior (gates, fallbacks, retries, feature flags), add comments with:
- **Trigger**: what condition causes this branch
- **Why**: the rationale (cost, correctness, UX, determinism)
- **Outcome**: what changes downstream
```python
# Trigger: project has no active sync watcher
# Why: avoid duplicate file system watchers consuming resources
# Outcome: starts new watcher, registers in active_watchers dict
if project_id not in active_watchers:
start_watcher(project_id)
```
**Constraint Comments:**
If code exists because of a constraint (async requirements, rate limits, schema compatibility), explain the constraint near the code:
```python
# SQLite requires WAL mode for concurrent read/write access
connection.execute("PRAGMA journal_mode=WAL")
```
**What NOT to Comment:**
Avoid comments that restate obvious code:
```python
# Bad - restates code
counter += 1 # increment counter
# Good - explains why
counter += 1 # track retries for backoff calculation
```
### Codebase Architecture
See [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) for detailed architecture documentation.
**Directory Structure:**
- `/alembic` - Alembic db migrations
- `/api` - FastAPI REST endpoints + `container.py` composition root
- `/cli` - Typer CLI + `container.py` composition root
- `/deps` - Feature-scoped FastAPI dependencies (config, db, projects, repositories, services, importers)
- `/importers` - Import functionality for Claude, ChatGPT, and other sources
- `/markdown` - Markdown parsing and processing
- `/mcp` - MCP server + `container.py` composition root + `clients/` typed API clients
- `/models` - SQLAlchemy ORM models
- `/repository` - Data access layer
- `/schemas` - Pydantic models for validation
- `/services` - Business logic layer
- `/sync` - File synchronization services + `coordinator.py` for lifecycle management
**Composition Roots:**
Each entrypoint (API, MCP, CLI) has a composition root that:
- Reads `ConfigManager` (the only place that reads global config)
- Resolves runtime mode via `RuntimeMode` enum (TEST > CLOUD > LOCAL)
- Provides dependencies to downstream code explicitly
**Typed API Clients (MCP):**
MCP tools use typed clients in `mcp/clients/` to communicate with the API:
- `KnowledgeClient` - Entity CRUD operations
- `SearchClient` - Search operations
- `MemoryClient` - Context building
- `DirectoryClient` - Directory listing
- `ResourceClient` - Resource reading
- `ProjectClient` - Project management
Flow: MCP Tool → Typed Client → HTTP API → Router → Service → Repository
### Development Notes
- MCP tools are defined in src/basic_memory/mcp/tools/
- MCP prompts are defined in src/basic_memory/mcp/prompts/
- MCP tools should be atomic, composable operations
- Use `textwrap.dedent()` for multi-line string formatting in prompts and tools
- MCP Prompts are used to invoke tools and format content with instructions for an LLM
- Schema changes require Alembic migrations
- SQLite is used for indexing and full text search, files are source of truth
- Testing uses pytest with asyncio support (strict mode)
- Unit tests (`tests/`) use mocks when necessary; integration tests (`test-int/`) use real implementations
- By default, tests run against SQLite (fast, no Docker needed)
- Set `BASIC_MEMORY_TEST_POSTGRES=1` to run against Postgres (uses testcontainers - Docker required)
- Each test runs in a standalone environment with isolated database and tmp_path directory
- CI runs SQLite and Postgres tests in parallel for faster feedback
- Performance benchmarks are in `test-int/test_sync_performance_benchmark.py`
- Use pytest markers: `@pytest.mark.benchmark` for benchmarks, `@pytest.mark.slow` for slow tests
- **Coverage must stay at 100%**: Write tests for new code. Only use `# pragma: no cover` when tests would require excessive mocking (e.g., TYPE_CHECKING blocks, error handlers that need failure injection, runtime-mode-dependent code paths)
### Async Client Pattern (Important!)
**All MCP tools and CLI commands use the context manager pattern for HTTP clients:**
```python
from basic_memory.mcp.async_client import get_client
async def my_mcp_tool():
async with get_client() as client:
# Use client for API calls
response = await call_get(client, "/path")
return response
```
**Do NOT use:**
-`from basic_memory.mcp.async_client import client` (deprecated module-level client)
- ❌ Manual auth header management
-`inject_auth_header()` (deleted)
**Key principles:**
- Auth happens at client creation, not per-request
- Proper resource management via context managers
- Supports three modes: Local (ASGI), CLI cloud (HTTP + auth), Cloud app (factory injection)
- Factory pattern enables dependency injection for cloud consolidation
**For cloud app integration:**
```python
from basic_memory.mcp import async_client
# Set custom factory before importing tools
async_client.set_client_factory(your_custom_factory)
```
See SPEC-16 for full context manager refactor details.
## BASIC MEMORY PRODUCT USAGE
### Knowledge Structure
- Entity: Any concept, document, or idea represented as a markdown file
- Observation: A categorized fact about an entity (`- [category] content`)
- Relation: A directional link between entities (`- relation_type [[Target]]`)
- Frontmatter: YAML metadata at the top of markdown files
- Knowledge representation follows precise markdown format:
- Observations with [category] prefixes
- Relations with WikiLinks [[Entity]]
- Frontmatter with metadata
### Basic Memory Commands
**Local Commands:**
- Check sync status: `basic-memory status`
- Doctor check (file <-> DB loop): `basic-memory doctor`
- Import from Claude: `basic-memory import claude conversations`
- Import from ChatGPT: `basic-memory import chatgpt`
- Import from Memory JSON: `basic-memory import memory-json`
- Tool access: `basic-memory tool` (provides CLI access to MCP tools)
- Continue: `basic-memory tool continue-conversation --topic="search"`
**Project Management:**
- List projects: `basic-memory project list`
- Add project: `basic-memory project add "name" ~/path`
- Project info: `basic-memory project info`
- One-way sync (local -> cloud): `basic-memory project sync`
- Bidirectional sync: `basic-memory project bisync`
- Integrity check: `basic-memory project check`
**Cloud Commands (requires subscription):**
- Authenticate: `basic-memory cloud login`
- Logout: `basic-memory cloud logout`
- Check cloud status: `basic-memory cloud status`
- Setup cloud sync: `basic-memory cloud setup`
- Manage snapshots: `basic-memory cloud snapshot [create|list|delete|show|browse]`
- Restore from snapshot: `basic-memory cloud restore <path> --snapshot <id>`
### MCP Capabilities
- Basic Memory exposes these MCP tools to LLMs:
**Content Management:**
- `write_note(title, content, directory, tags)` - Create/update markdown notes with semantic observations and relations
- `read_note(identifier, page, page_size)` - Read notes by title, permalink, or memory:// URL with knowledge graph awareness
- `read_content(path)` - Read raw file content (text, images, binaries) without knowledge graph processing
- `view_note(identifier, page, page_size)` - View notes as formatted artifacts for better readability
- `edit_note(identifier, operation, content)` - Edit notes incrementally (append, prepend, find/replace, replace_section)
- `move_note(identifier, destination_path, is_directory)` - Move notes or directories to new locations, updating database and maintaining links
- `delete_note(identifier, is_directory)` - Delete notes or directories from the knowledge base
**Knowledge Graph Navigation:**
- `build_context(url, depth, timeframe)` - Navigate the knowledge graph via memory:// URLs for conversation continuity
- `recent_activity(type, depth, timeframe)` - Get recently updated information with specified timeframe (e.g., "1d", "1 week")
- `list_directory(dir_name, depth, file_name_glob)` - Browse directory contents with filtering and depth control
**Search & Discovery:**
- `search_notes(query, page, page_size, search_type, types, entity_types, after_date)` - Full-text search across all content with advanced filtering options
**Project Management:**
- `list_memory_projects()` - List all available projects with their status
- `create_memory_project(project_name, project_path, set_default)` - Create new Basic Memory projects
- `delete_project(project_name)` - Delete a project from configuration
**Visualization:**
- `canvas(nodes, edges, title, directory)` - Generate Obsidian canvas files for knowledge graph visualization
**ChatGPT-Compatible Tools:**
- `search(query)` - Search across knowledge base (OpenAI actions compatible)
- `fetch(id)` - Fetch full content of a search result document
- MCP Prompts for better AI interaction:
- `ai_assistant_guide()` - Guidance on effectively using Basic Memory tools for AI assistants
- `continue_conversation(topic, timeframe)` - Continue previous conversations with relevant historical context
- `search(query, after_date)` - Search with detailed, formatted results for better context understanding
- `recent_activity(timeframe)` - View recently changed items with formatted output
### Cloud Features (v0.15.0+)
Basic Memory now supports cloud synchronization and storage (requires active subscription):
**Authentication:**
- JWT-based authentication with subscription validation
- Secure session management with token refresh
- Support for multiple cloud projects
**Bidirectional Sync:**
- rclone bisync integration for two-way synchronization
- Conflict resolution and integrity verification
- Real-time sync with change detection
- Mount/unmount cloud storage for direct file access
**Cloud Project Management:**
- Create and manage projects in the cloud
- Toggle between local and cloud modes
- Per-project sync configuration
- Subscription-based access control
**Security & Performance:**
- Removed .env file loading for improved security
- .gitignore integration (respects gitignored files)
- WAL mode for SQLite performance
- Background relation resolution (non-blocking startup)
- API performance optimizations (SPEC-11)
**CLI Routing Flags:**
When cloud mode is enabled, CLI commands route to the cloud API by default. Use `--local` and `--cloud` flags to override:
```bash
# Force local routing (ignore cloud mode)
basic-memory status --local
basic-memory project list --local
# Force cloud routing (when cloud mode is disabled)
basic-memory status --cloud
basic-memory project info my-project --cloud
```
Key behaviors:
- The local MCP server (`basic-memory mcp`) automatically uses local routing
- This allows simultaneous use of local Claude Desktop and cloud-based clients
- Some commands (like `project default`, `project sync-config`, `project move`) require `--local` in cloud mode since they modify local configuration
- Environment variable `BASIC_MEMORY_FORCE_LOCAL=true` forces local routing globally
## AI-Human Collaborative Development
Basic Memory emerged from and enables a new kind of development process that combines human and AI capabilities. Instead
of using AI just for code generation, we've developed a true collaborative workflow:
1. AI (LLM) writes initial implementation based on specifications and context
2. Human reviews, runs tests, and commits code with any necessary adjustments
3. Knowledge persists across conversations using Basic Memory's knowledge graph
4. Development continues seamlessly across different AI sessions with consistent context
5. Results improve through iterative collaboration and shared understanding
This approach has allowed us to tackle more complex challenges and build a more robust system than either humans or AI
could achieve independently.
**Problem-Solving Guidance:**
- If a solution isn't working after reasonable effort, suggest alternative approaches
- Don't persist with a problematic library or pattern when better alternatives exist
- Example: When py-pglite caused cascading test failures, switching to testcontainers-postgres was the right call
## GitHub Integration
Basic Memory has taken AI-Human collaboration to the next level by integrating Claude directly into the development workflow through GitHub:
### GitHub MCP Tools
Using the GitHub Model Context Protocol server, Claude can now:
- **Repository Management**:
- View repository files and structure
- Read file contents
- Create new branches
- Create and update files
- **Issue Management**:
- Create new issues
- Comment on existing issues
- Close and update issues
- Search across issues
- **Pull Request Workflow**:
- Create pull requests
- Review code changes
- Add comments to PRs
This integration enables Claude to participate as a full team member in the development process, not just as a code generation tool. Claude's GitHub account ([bm-claudeai](https://github.com/bm-claudeai)) is a member of the Basic Machines organization with direct contributor access to the codebase.
### Collaborative Development Process
With GitHub integration, the development workflow includes:
1. **Direct code review** - Claude can analyze PRs and provide detailed feedback
2. **Contribution tracking** - All of Claude's contributions are properly attributed in the Git history
3. **Branch management** - Claude can create feature branches for implementations
4. **Documentation maintenance** - Claude can keep documentation updated as the code evolves
5. **Code Commits**: ALWAYS sign off commits with `git commit -s`
This level of integration represents a new paradigm in AI-human collaboration, where the AI assistant becomes a full-fledged team member rather than just a tool for generating code snippets.
+64
View File
@@ -1,5 +1,69 @@
# CHANGELOG
## v0.18.4 (2026-02-12)
### Bug Fixes
- Use global `--header` flag for Tigris consistency on all rclone transactions
([`0eae0e1`](https://github.com/basicmachines-co/basic-memory/commit/0eae0e1))
- `--header-download` / `--header-upload` only apply to GET/PUT requests, missing S3
ListObjectsV2 calls that bisync issues first. Non-US users saw stale edge-cached metadata.
- `--header` applies to ALL HTTP transactions (list, download, upload), fixing bisync for
users outside the Tigris origin region.
## v0.18.2 (2026-02-11)
### Bug Fixes
- **#562**: Use VIRTUAL instead of STORED columns in SQLite migration
([`344e651`](https://github.com/basicmachines-co/basic-memory/commit/344e651))
- Fixes compatibility issue with SQLite STORED generated columns
## v0.18.1 (2026-02-11)
### Features
- **#552**: Add `--format json` to CLI tool commands
([`a47c9c0`](https://github.com/basicmachines-co/basic-memory/commit/a47c9c0))
- CLI tool commands now support `--format json` for machine-readable output
- **#535**: Support `tag:` query shorthand in search
([`f1d50c2`](https://github.com/basicmachines-co/basic-memory/commit/f1d50c2))
- Use `tag:mytag` as a convenient shorthand in search queries
- **#532**: Fast edit entities, refactors for webui, enhanced search
([`530cbac`](https://github.com/basicmachines-co/basic-memory/commit/530cbac))
- Performance improvements for entity editing and search operations
### Bug Fixes
- **#558**: Add X-Tigris-Consistent headers to all rclone commands
([`8489a3d`](https://github.com/basicmachines-co/basic-memory/commit/8489a3d))
- Ensures consistent reads from Tigris object storage during sync
- **#541**: Handle EntityCreationError as conflict
([`343a6e1`](https://github.com/basicmachines-co/basic-memory/commit/343a6e1))
- **#536**: Stabilize metadata filters on Postgres
([`009e849`](https://github.com/basicmachines-co/basic-memory/commit/009e849))
- **#533**: Fix recent_activity prompt defaults
([`24ca5f6`](https://github.com/basicmachines-co/basic-memory/commit/24ca5f6))
- **#530**: Prevent spurious `metadata: {}` in frontmatter output
([`e3ced49`](https://github.com/basicmachines-co/basic-memory/commit/e3ced49))
- Add POST legacy compat routes for v0.18.0 CLI
([`c46d7a6`](https://github.com/basicmachines-co/basic-memory/commit/c46d7a6))
- Restore legacy `/projects/projects` endpoint for older CLI versions
([`a0e754b`](https://github.com/basicmachines-co/basic-memory/commit/a0e754b))
### Internal
- **#538**: Add fast feedback loop tooling (`just fast-check`, `just doctor`, `just testmon`)
([`8072449`](https://github.com/basicmachines-co/basic-memory/commit/8072449))
## v0.18.0 (2026-01-28)
### Features
-389
View File
@@ -1,389 +0,0 @@
# CLAUDE.md - Basic Memory Project Guide
## Project Overview
Basic Memory is a local-first knowledge management system built on the Model Context Protocol (MCP). It enables
bidirectional communication between LLMs (like Claude) and markdown files, creating a personal knowledge graph that can
be traversed using links between documents.
## CODEBASE DEVELOPMENT
### Project information
See the [README.md](README.md) file for a project overview.
### Build and Test Commands
- Install: `just install` or `pip install -e ".[dev]"`
- Run all tests (SQLite + Postgres): `just test`
- Run all tests against SQLite: `just test-sqlite`
- Run all tests against Postgres: `just test-postgres` (uses testcontainers)
- Run unit tests (SQLite): `just test-unit-sqlite`
- Run unit tests (Postgres): `just test-unit-postgres`
- Run integration tests (SQLite): `just test-int-sqlite`
- Run integration tests (Postgres): `just test-int-postgres`
- Generate HTML coverage: `just coverage`
- Single test: `pytest tests/path/to/test_file.py::test_function_name`
- Run benchmarks: `pytest test-int/test_sync_performance_benchmark.py -v -m "benchmark and not slow"`
- Lint: `just lint` or `ruff check . --fix`
- Type check: `just typecheck` or `uv run pyright`
- Format: `just format` or `uv run ruff format .`
- Run all code checks: `just check` (runs lint, format, typecheck, test)
- Create db migration: `just migration "Your migration message"`
- Run development MCP Inspector: `just run-inspector`
**Note:** Project requires Python 3.12+ (uses type parameter syntax and `type` aliases introduced in 3.12)
**Postgres Testing:** Uses [testcontainers](https://testcontainers-python.readthedocs.io/) which automatically spins up a Postgres instance in Docker. No manual database setup required - just have Docker running.
### Test Structure
- `tests/` - Unit tests for individual components (mocked, fast)
- `test-int/` - Integration tests for real-world scenarios (no mocks, realistic)
- Both directories are covered by unified coverage reporting
- Benchmark tests in `test-int/` are marked with `@pytest.mark.benchmark`
- Slow tests are marked with `@pytest.mark.slow`
### Code Style Guidelines
- Line length: 100 characters max
- Python 3.12+ with full type annotations (uses type parameters and type aliases)
- Format with ruff (consistent styling)
- Import order: standard lib, third-party, local imports
- Naming: snake_case for functions/variables, PascalCase for classes
- Prefer async patterns with SQLAlchemy 2.0
- Use Pydantic v2 for data validation and schemas
- CLI uses Typer for command structure
- API uses FastAPI for endpoints
- Follow the repository pattern for data access
- Tools communicate to api routers via the httpx ASGI client (in process)
### Code Change Guidelines
- **Full file read before edits**: Before editing any file, read it in full first to ensure complete context; partial reads lead to corrupted edits
- **Minimize diffs**: Prefer the smallest change that satisfies the request. Avoid unrelated refactors or style rewrites unless necessary for correctness
- **No speculative getattr**: Never use `getattr(obj, "attr", default)` when unsure about attribute names. Check the class definition or source code first
- **Fail fast**: Write code with fail-fast logic by default. Do not swallow exceptions with errors or warnings
- **No fallback logic**: Do not add fallback logic unless explicitly told to and agreed with the user
- **No guessing**: Do not say "The issue is..." before you actually know what the issue is. Investigate first.
### Literate Programming Style
Code should tell a story. Comments must explain the "why" and narrative flow, not just the "what".
**Section Headers:**
For files with multiple phases of logic, add section headers so the control flow reads like chapters:
```python
# --- Authentication ---
# ... auth logic ...
# --- Data Validation ---
# ... validation logic ...
# --- Business Logic ---
# ... core logic ...
```
**Decision Point Comments:**
For conditionals that materially change behavior (gates, fallbacks, retries, feature flags), add comments with:
- **Trigger**: what condition causes this branch
- **Why**: the rationale (cost, correctness, UX, determinism)
- **Outcome**: what changes downstream
```python
# Trigger: project has no active sync watcher
# Why: avoid duplicate file system watchers consuming resources
# Outcome: starts new watcher, registers in active_watchers dict
if project_id not in active_watchers:
start_watcher(project_id)
```
**Constraint Comments:**
If code exists because of a constraint (async requirements, rate limits, schema compatibility), explain the constraint near the code:
```python
# SQLite requires WAL mode for concurrent read/write access
connection.execute("PRAGMA journal_mode=WAL")
```
**What NOT to Comment:**
Avoid comments that restate obvious code:
```python
# Bad - restates code
counter += 1 # increment counter
# Good - explains why
counter += 1 # track retries for backoff calculation
```
### Codebase Architecture
See [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) for detailed architecture documentation.
**Directory Structure:**
- `/alembic` - Alembic db migrations
- `/api` - FastAPI REST endpoints + `container.py` composition root
- `/cli` - Typer CLI + `container.py` composition root
- `/deps` - Feature-scoped FastAPI dependencies (config, db, projects, repositories, services, importers)
- `/importers` - Import functionality for Claude, ChatGPT, and other sources
- `/markdown` - Markdown parsing and processing
- `/mcp` - MCP server + `container.py` composition root + `clients/` typed API clients
- `/models` - SQLAlchemy ORM models
- `/repository` - Data access layer
- `/schemas` - Pydantic models for validation
- `/services` - Business logic layer
- `/sync` - File synchronization services + `coordinator.py` for lifecycle management
**Composition Roots:**
Each entrypoint (API, MCP, CLI) has a composition root that:
- Reads `ConfigManager` (the only place that reads global config)
- Resolves runtime mode via `RuntimeMode` enum (TEST > CLOUD > LOCAL)
- Provides dependencies to downstream code explicitly
**Typed API Clients (MCP):**
MCP tools use typed clients in `mcp/clients/` to communicate with the API:
- `KnowledgeClient` - Entity CRUD operations
- `SearchClient` - Search operations
- `MemoryClient` - Context building
- `DirectoryClient` - Directory listing
- `ResourceClient` - Resource reading
- `ProjectClient` - Project management
Flow: MCP Tool → Typed Client → HTTP API → Router → Service → Repository
### Development Notes
- MCP tools are defined in src/basic_memory/mcp/tools/
- MCP prompts are defined in src/basic_memory/mcp/prompts/
- MCP tools should be atomic, composable operations
- Use `textwrap.dedent()` for multi-line string formatting in prompts and tools
- MCP Prompts are used to invoke tools and format content with instructions for an LLM
- Schema changes require Alembic migrations
- SQLite is used for indexing and full text search, files are source of truth
- Testing uses pytest with asyncio support (strict mode)
- Unit tests (`tests/`) use mocks when necessary; integration tests (`test-int/`) use real implementations
- By default, tests run against SQLite (fast, no Docker needed)
- Set `BASIC_MEMORY_TEST_POSTGRES=1` to run against Postgres (uses testcontainers - Docker required)
- Each test runs in a standalone environment with isolated database and tmp_path directory
- CI runs SQLite and Postgres tests in parallel for faster feedback
- Performance benchmarks are in `test-int/test_sync_performance_benchmark.py`
- Use pytest markers: `@pytest.mark.benchmark` for benchmarks, `@pytest.mark.slow` for slow tests
- **Coverage must stay at 100%**: Write tests for new code. Only use `# pragma: no cover` when tests would require excessive mocking (e.g., TYPE_CHECKING blocks, error handlers that need failure injection, runtime-mode-dependent code paths)
### Async Client Pattern (Important!)
**All MCP tools and CLI commands use the context manager pattern for HTTP clients:**
```python
from basic_memory.mcp.async_client import get_client
async def my_mcp_tool():
async with get_client() as client:
# Use client for API calls
response = await call_get(client, "/path")
return response
```
**Do NOT use:**
-`from basic_memory.mcp.async_client import client` (deprecated module-level client)
- ❌ Manual auth header management
-`inject_auth_header()` (deleted)
**Key principles:**
- Auth happens at client creation, not per-request
- Proper resource management via context managers
- Supports three modes: Local (ASGI), CLI cloud (HTTP + auth), Cloud app (factory injection)
- Factory pattern enables dependency injection for cloud consolidation
**For cloud app integration:**
```python
from basic_memory.mcp import async_client
# Set custom factory before importing tools
async_client.set_client_factory(your_custom_factory)
```
See SPEC-16 for full context manager refactor details.
## BASIC MEMORY PRODUCT USAGE
### Knowledge Structure
- Entity: Any concept, document, or idea represented as a markdown file
- Observation: A categorized fact about an entity (`- [category] content`)
- Relation: A directional link between entities (`- relation_type [[Target]]`)
- Frontmatter: YAML metadata at the top of markdown files
- Knowledge representation follows precise markdown format:
- Observations with [category] prefixes
- Relations with WikiLinks [[Entity]]
- Frontmatter with metadata
### Basic Memory Commands
**Local Commands:**
- Check sync status: `basic-memory status`
- Import from Claude: `basic-memory import claude conversations`
- Import from ChatGPT: `basic-memory import chatgpt`
- Import from Memory JSON: `basic-memory import memory-json`
- Tool access: `basic-memory tool` (provides CLI access to MCP tools)
- Continue: `basic-memory tool continue-conversation --topic="search"`
**Project Management:**
- List projects: `basic-memory project list`
- Add project: `basic-memory project add "name" ~/path`
- Project info: `basic-memory project info`
- One-way sync (local -> cloud): `basic-memory project sync`
- Bidirectional sync: `basic-memory project bisync`
- Integrity check: `basic-memory project check`
**Cloud Commands (requires subscription):**
- Authenticate: `basic-memory cloud login`
- Logout: `basic-memory cloud logout`
- Check cloud status: `basic-memory cloud status`
- Setup cloud sync: `basic-memory cloud setup`
- Manage snapshots: `basic-memory cloud snapshot [create|list|delete|show|browse]`
- Restore from snapshot: `basic-memory cloud restore <path> --snapshot <id>`
### MCP Capabilities
- Basic Memory exposes these MCP tools to LLMs:
**Content Management:**
- `write_note(title, content, directory, tags)` - Create/update markdown notes with semantic observations and relations
- `read_note(identifier, page, page_size)` - Read notes by title, permalink, or memory:// URL with knowledge graph awareness
- `read_content(path)` - Read raw file content (text, images, binaries) without knowledge graph processing
- `view_note(identifier, page, page_size)` - View notes as formatted artifacts for better readability
- `edit_note(identifier, operation, content)` - Edit notes incrementally (append, prepend, find/replace, replace_section)
- `move_note(identifier, destination_path, is_directory)` - Move notes or directories to new locations, updating database and maintaining links
- `delete_note(identifier, is_directory)` - Delete notes or directories from the knowledge base
**Knowledge Graph Navigation:**
- `build_context(url, depth, timeframe)` - Navigate the knowledge graph via memory:// URLs for conversation continuity
- `recent_activity(type, depth, timeframe)` - Get recently updated information with specified timeframe (e.g., "1d", "1 week")
- `list_directory(dir_name, depth, file_name_glob)` - Browse directory contents with filtering and depth control
**Search & Discovery:**
- `search_notes(query, page, page_size, search_type, types, entity_types, after_date)` - Full-text search across all content with advanced filtering options
**Project Management:**
- `list_memory_projects()` - List all available projects with their status
- `create_memory_project(project_name, project_path, set_default)` - Create new Basic Memory projects
- `delete_project(project_name)` - Delete a project from configuration
**Visualization:**
- `canvas(nodes, edges, title, directory)` - Generate Obsidian canvas files for knowledge graph visualization
**ChatGPT-Compatible Tools:**
- `search(query)` - Search across knowledge base (OpenAI actions compatible)
- `fetch(id)` - Fetch full content of a search result document
- MCP Prompts for better AI interaction:
- `ai_assistant_guide()` - Guidance on effectively using Basic Memory tools for AI assistants
- `continue_conversation(topic, timeframe)` - Continue previous conversations with relevant historical context
- `search(query, after_date)` - Search with detailed, formatted results for better context understanding
- `recent_activity(timeframe)` - View recently changed items with formatted output
### Cloud Features (v0.15.0+)
Basic Memory now supports cloud synchronization and storage (requires active subscription):
**Authentication:**
- JWT-based authentication with subscription validation
- Secure session management with token refresh
- Support for multiple cloud projects
**Bidirectional Sync:**
- rclone bisync integration for two-way synchronization
- Conflict resolution and integrity verification
- Real-time sync with change detection
- Mount/unmount cloud storage for direct file access
**Cloud Project Management:**
- Create and manage projects in the cloud
- Toggle between local and cloud modes
- Per-project sync configuration
- Subscription-based access control
**Security & Performance:**
- Removed .env file loading for improved security
- .gitignore integration (respects gitignored files)
- WAL mode for SQLite performance
- Background relation resolution (non-blocking startup)
- API performance optimizations (SPEC-11)
**CLI Routing Flags:**
When cloud mode is enabled, CLI commands route to the cloud API by default. Use `--local` and `--cloud` flags to override:
```bash
# Force local routing (ignore cloud mode)
basic-memory status --local
basic-memory project list --local
# Force cloud routing (when cloud mode is disabled)
basic-memory status --cloud
basic-memory project info my-project --cloud
```
Key behaviors:
- The local MCP server (`basic-memory mcp`) automatically uses local routing
- This allows simultaneous use of local Claude Desktop and cloud-based clients
- Some commands (like `project default`, `project sync-config`, `project move`) require `--local` in cloud mode since they modify local configuration
- Environment variable `BASIC_MEMORY_FORCE_LOCAL=true` forces local routing globally
## AI-Human Collaborative Development
Basic Memory emerged from and enables a new kind of development process that combines human and AI capabilities. Instead
of using AI just for code generation, we've developed a true collaborative workflow:
1. AI (LLM) writes initial implementation based on specifications and context
2. Human reviews, runs tests, and commits code with any necessary adjustments
3. Knowledge persists across conversations using Basic Memory's knowledge graph
4. Development continues seamlessly across different AI sessions with consistent context
5. Results improve through iterative collaboration and shared understanding
This approach has allowed us to tackle more complex challenges and build a more robust system than either humans or AI
could achieve independently.
**Problem-Solving Guidance:**
- If a solution isn't working after reasonable effort, suggest alternative approaches
- Don't persist with a problematic library or pattern when better alternatives exist
- Example: When py-pglite caused cascading test failures, switching to testcontainers-postgres was the right call
## GitHub Integration
Basic Memory has taken AI-Human collaboration to the next level by integrating Claude directly into the development workflow through GitHub:
### GitHub MCP Tools
Using the GitHub Model Context Protocol server, Claude can now:
- **Repository Management**:
- View repository files and structure
- Read file contents
- Create new branches
- Create and update files
- **Issue Management**:
- Create new issues
- Comment on existing issues
- Close and update issues
- Search across issues
- **Pull Request Workflow**:
- Create pull requests
- Review code changes
- Add comments to PRs
This integration enables Claude to participate as a full team member in the development process, not just as a code generation tool. Claude's GitHub account ([bm-claudeai](https://github.com/bm-claudeai)) is a member of the Basic Machines organization with direct contributor access to the codebase.
### Collaborative Development Process
With GitHub integration, the development workflow includes:
1. **Direct code review** - Claude can analyze PRs and provide detailed feedback
2. **Contribution tracking** - All of Claude's contributions are properly attributed in the Git history
3. **Branch management** - Claude can create feature branches for implementations
4. **Documentation maintenance** - Claude can keep documentation updated as the code evolves
5. **Code Commits**: ALWAYS sign off commits with `git commit -s`
This level of integration represents a new paradigm in AI-human collaboration, where the AI assistant becomes a full-fledged team member rather than just a tool for generating code snippets.
Symlink
+1
View File
@@ -0,0 +1 @@
AGENTS.md
+16
View File
@@ -396,6 +396,8 @@ list_directory(dir_name, depth) - Browse directory contents with filtering
**Search & Discovery:**
```
search(query, page, page_size) - Search across your knowledge base
search_notes(query, page, page_size, search_type, types, entity_types, after_date, metadata_filters, tags, status, project) - Search with filters
search_by_metadata(filters, limit, offset, project) - Structured frontmatter search
```
**Project Management:**
@@ -494,16 +496,23 @@ just test
- `just test-int-postgres` - Run integration tests against Postgres
- `just test-windows` - Run Windows-specific tests (auto-skips on other platforms)
- `just test-benchmark` - Run performance benchmark tests
- `just testmon` - Run tests impacted by recent changes (pytest-testmon)
- `just test-smoke` - Run fast MCP end-to-end smoke test
- `just fast-check` - Run fix/format/typecheck + impacted tests + smoke test
- `just doctor` - Run local file <-> DB consistency checks with temp config
**Postgres Testing:**
Postgres tests use [testcontainers](https://testcontainers-python.readthedocs.io/) which automatically spins up a Postgres instance in Docker. No manual database setup required - just have Docker running.
**Testmon Note:** When no files have changed, `just testmon` may collect 0 tests. That's expected and means no impacted tests were detected.
**Test Markers:**
Tests use pytest markers for selective execution:
- `windows` - Windows-specific database optimizations
- `benchmark` - Performance tests (excluded from default runs)
- `smoke` - Fast MCP end-to-end smoke tests
**Other Development Commands:**
```bash
@@ -511,10 +520,17 @@ just install # Install with dev dependencies
just lint # Run linting checks
just typecheck # Run type checking
just format # Format code with ruff
just fast-check # Fast local loop (fix/format/typecheck + testmon + smoke)
just doctor # Local consistency check (temp config)
just check # Run all quality checks
just migration "msg" # Create database migration
```
**Local Consistency Check:**
```bash
basic-memory doctor # Verifies file <-> database sync in a temp project
```
See the [justfile](justfile) for the complete list of development commands.
## License
+15 -2
View File
@@ -214,15 +214,28 @@ Example tool using typed client:
```python
@mcp.tool()
async def search_notes(query: str, project: str | None = None) -> SearchResponse:
async def search_notes(
query: str,
project: str | None = None,
metadata_filters: dict | None = None,
tags: list[str] | None = None,
status: str | None = None,
) -> SearchResponse:
async with get_client() as client:
active_project = await get_active_project(client, project)
# Import client inside function to avoid circular imports
from basic_memory.mcp.clients import SearchClient
from basic_memory.schemas.search import SearchQuery
search_query = SearchQuery(
text=query,
metadata_filters=metadata_filters,
tags=tags,
status=status,
)
search_client = SearchClient(client, active_project.external_id)
return await search_client.search(query)
return await search_client.search(search_query.model_dump())
```
## Sync Coordination
+494
View File
@@ -0,0 +1,494 @@
# Note Format Reference
Every document in Basic Memory is a plain Markdown file. Files are the source of truth — changes to files automatically update the knowledge graph in the database. You maintain complete ownership, files work with git, and knowledge persists independently of any AI conversation.
## Document Structure
A note has three parts: YAML frontmatter, content (observations), and relations.
```markdown
---
title: Coffee Brewing Methods
type: note
tags: [coffee, brewing]
permalink: coffee-brewing-methods
---
# Coffee Brewing Methods
## Observations
- [method] Pour over provides more flavor clarity than French press
- [technique] Water temperature at 205°F extracts optimal compounds #brewing
- [preference] Ethiopian beans work well with lighter roasts (personal experience)
## Relations
- relates_to [[Coffee Bean Origins]]
- requires [[Proper Grinding Technique]]
- contrasts_with [[Tea Brewing Methods]]
```
The `## Observations` and `## Relations` headings are conventional but not required — the parser detects observations and relations by their syntax patterns anywhere in the document.
## Frontmatter
YAML metadata between `---` fences at the top of the file.
| Field | Required | Default | Description |
|-------|----------|---------|-------------|
| `title` | No | filename stem | Used for linking and references. Auto-set from filename if missing. |
| `type` | No | `note` | Entity type. Used for schema resolution and filtering. |
| `tags` | No | `[]` | List or comma-separated string. Used for organization and search. |
| `permalink` | No | generated from title | Stable identifier. Persists even if the file moves. |
| `schema` | No | none | Schema attachment — dict (inline), string (reference), or omitted (implicit). |
Custom fields are allowed. Any key not in the standard set is stored as `entity_metadata` and indexed for search and filtering.
```yaml
---
title: Paul Graham
type: Person
tags: [startups, essays, lisp]
permalink: paul-graham
status: active
source: wikipedia
---
```
Here `status` and `source` are custom fields stored in `entity_metadata`.
### Frontmatter Value Handling
YAML automatically converts some values to native types. Basic Memory normalizes them:
- Date strings (`2025-10-24`) → kept as ISO format strings
- Numbers (`1.0`) → converted to strings
- Booleans (`true`) → converted to strings (`"True"`)
- Lists and dicts → preserved, items normalized recursively
This prevents errors when downstream code expects string values.
## Observations
An observation is a categorized fact about the entity. Written as a Markdown list item.
**Syntax:**
```
- [category] content text #tag1 #tag2 (context)
```
| Part | Required | Description |
|------|----------|-------------|
| `[category]` | Yes | Classification in square brackets. Any text except `[]()` chars. |
| content | Yes | The fact or statement. |
| `#tags` | No | Inline tags. Space-separated, each starting with `#`. |
| `(context)` | No | Parenthesized text at end of line. Supporting details or source. |
### Examples
```markdown
- [tech] Uses SQLite for storage #database
- [design] Follows local-first architecture #architecture
- [decision] Selected bcrypt for passwords #security (based on OWASP audit)
- [name] Paul Graham
- [expertise] Startups
- [expertise] Lisp
- [expertise] Essay writing
```
Array-like fields use repeated categories — multiple `[expertise]` observations above.
### What Is Not an Observation
The parser excludes these list item patterns:
| Pattern | Example | Reason |
|---------|---------|--------|
| Checkboxes | `- [ ] Todo item`, `- [x] Done`, `- [-] Cancelled` | Task list syntax |
| Markdown links | `- [text](url)` | URL link syntax |
| Bare wiki links | `- [[Target]]` | Treated as a relation instead |
A list item with `#tags` but no `[category]` is still parsed — the tags are extracted and the category defaults to `Note`.
## Relations
Relations connect documents to form the knowledge graph. There are two kinds.
### Explicit Relations
Written as list items with a relation type and a `[[wiki link]]` target.
**Syntax:**
```
- relation_type [[Target Entity]] (context)
```
| Part | Required | Description |
|------|----------|-------------|
| `relation_type` | No | Text before `[[`. Defaults to `relates_to` if omitted. |
| `[[Target]]` | Yes | Wiki link to the target entity. Matched by title or permalink. |
| `(context)` | No | Parenthesized text after `]]`. Supporting details. |
### Examples
```markdown
- implements [[Search Design]]
- depends_on [[Database Schema]]
- works_at [[Y Combinator]] (co-founder)
- [[Some Entity]]
```
The last example — a bare `[[wiki link]]` in a list item — gets relation type `relates_to`.
Common relation types:
- `implements`, `depends_on`, `relates_to`, `inspired_by`
- `extends`, `part_of`, `contains`, `pairs_with`
- `works_at`, `authored`, `collaborated_with`
Any text works as a relation type. These are conventions, not a fixed set.
### Inline References
Wiki links appearing in regular prose (not as list items) create implicit `links_to` relations.
```markdown
This builds on [[Core Design]] and uses [[Utility Functions]].
```
This creates two relations: `links_to [[Core Design]]` and `links_to [[Utility Functions]]`.
### Forward References
Relations can link to entities that don't exist yet. Basic Memory resolves them when the target is created.
## Permalinks and memory:// URLs
Every document has a unique **permalink** — a stable identifier derived from its title. You can set one explicitly in frontmatter, or let the system generate it.
```yaml
permalink: auth-approaches-2024
```
Permalinks form the basis of `memory://` URLs:
```
memory://auth-approaches-2024 # By permalink
memory://Authentication Approaches # By title (auto-resolves)
memory://project/auth-approaches # By path
```
Pattern matching is supported:
```
memory://auth* # Starts with "auth"
memory://*/approaches # Ends with "approaches"
memory://project/*/requirements # Nested wildcard
```
## Schemas
Schemas declare the expected structure of a note — which observation categories and relation types a well-formed note should have. They use Picoschema, a compact notation from Google's Dotprompt that fits naturally in YAML frontmatter.
### Picoschema Syntax
```yaml
schema:
name: string, full name # required field with description
email?: string, contact email # ? = optional
role?: string, job title
works_at?: Organization, employer # capitalized type = entity reference
tags?(array): string, categories # array of type
status?(enum): [active, inactive] # enum with allowed values
metadata?(object): # nested object
updated_at?: string
source?: string
```
| Notation | Meaning | Example |
|----------|---------|---------|
| `field: type` | Required field | `name: string` |
| `field?: type` | Optional field | `role?: string` |
| `field(array): type` | Array of values | `expertise(array): string` |
| `field?(enum): [vals]` | Enum with allowed values | `status?(enum): [active, inactive]` |
| `field?(object):` | Nested object with sub-fields | `metadata?(object):` |
| `, description` | Description after comma | `name: string, full name` |
| `EntityName` | Capitalized type = entity reference | `works_at?: Organization` |
**Scalar types:** `string`, `integer`, `number`, `boolean`, `any`
Any type not in that set whose first letter is uppercase is treated as an entity reference (a relation target).
### Schema-to-Note Mapping
Schemas validate against existing observation/relation syntax. Note authors don't learn new syntax.
| Schema Declaration | Maps To | Example in Note |
|--------------------|---------|-----------------|
| `field: string` | Observation `[field] value` | `- [name] Paul Graham` |
| `field?(array): string` | Multiple `[field]` observations | `- [expertise] Lisp` (repeated) |
| `field?: EntityType` | Relation `field [[Target]]` | `- works_at [[Y Combinator]]` |
| `field?(array): EntityType` | Multiple `field` relations | `- authored [[Book]]` (repeated) |
| `tags` | Frontmatter `tags` array | `tags: [startups, essays]` |
| `field?(enum): [vals]` | Observation `[field] value` where value is in the set | `- [status] active` |
Observations and relations not covered by the schema are valid — schemas describe a subset, not a straitjacket.
### Schema Attachment
Three ways to attach a schema to a note, resolved in priority order:
**1. Inline schema**`schema` is a dict in frontmatter:
```yaml
---
title: Team Standup 2024-01-15
type: meeting
schema:
attendees(array): string, who was there
decisions(array): string, what was decided
action_items(array): string, follow-ups
blockers?(array): string, anything stuck
---
```
Good for one-off structured notes or prototyping a schema before extracting it.
**2. Explicit reference**`schema` is a string naming a schema note:
```yaml
---
title: Basic Memory
schema: SoftwareProject
---
```
or by permalink:
```yaml
---
title: LLM Memory Patterns
schema: schema/research-project
---
```
Use when the note's `type` differs from the schema it should validate against, or when multiple schema variants exist.
**3. Implicit by type** — no `schema` field, resolved by matching `type`:
```yaml
---
title: Paul Graham
type: Person
---
```
The system looks up a schema note where `entity: Person`. If found, it applies. If not, no validation occurs.
**4. No schema** — perfectly fine. Most notes don't need one.
### Schema Notes
A schema is itself a Basic Memory note with `type: schema`. It lives anywhere (though `schema/` is the conventional directory).
```yaml
# schema/Person.md
---
title: Person
type: schema
entity: Person
version: 1
schema:
name: string, full name
role?: string, job title or position
works_at?: Organization, employer
expertise?(array): string, areas of knowledge
email?: string, contact email
settings:
validation: warn
---
# Person
A human individual in the knowledge graph.
```
| Field | Required | Description |
|-------|----------|-------------|
| `type` | Yes | Must be `schema` |
| `entity` | Yes | The entity type this schema describes (e.g., `Person`) |
| `version` | No | Schema version number (default: `1`) |
| `schema` | Yes | Picoschema dict defining the fields |
| `settings.validation` | No | Validation mode (default: `warn`) |
Schema notes are regular notes — they show up in search, can have observations and relations, and participate in the knowledge graph.
### Validation Modes
| Mode | Behavior |
|------|----------|
| `warn` | Warnings in output, doesn't block (default) |
| `strict` | Errors that block sync, for CI/CD enforcement |
| `off` | No validation |
### Validation Output
```
$ bm schema validate people/ada-lovelace.md
⚠ Person schema validation:
- Missing required field: name (expected [name] observation)
- Missing optional field: role
- Missing optional field: works_at (no relation found)
Unmatched observations: [fact] ×2, [born] ×1
Unmatched relations: collaborated_with
```
"Unmatched" items are informational — observations and relations the schema doesn't cover.
### Schema Inference
Generate schemas from existing notes by analyzing observation and relation frequency:
```
$ bm schema infer Person
Analyzing 30 notes with type: Person...
Observations found:
[name] 30/30 100% → name: string
[role] 27/30 90% → role?: string
[expertise] 18/30 60% → expertise?(array): string
[email] 8/30 27% → email?: string
Relations found:
works_at 22/30 73% → works_at?: Organization
Suggested schema:
name: string, full name
role?: string, job title
expertise?(array): string, areas of knowledge
email?: string, contact email
works_at?: Organization, employer
Save to schema/Person.md? [y/n]
```
Frequency thresholds:
- **100% present** → required field
- **25%+ present** → optional field
- **Below 25%** → excluded from suggestion
### Schema Drift Detection
Track how usage patterns shift over time:
```
$ bm schema diff Person
Schema drift detected:
+ expertise: now in 81% of notes (was 12%)
- department: dropped to 3% of notes
~ works_at: cardinality changed (one → many)
Update schema? [y/n/review]
```
## Complete Examples
### Simple Note (No Schema)
```markdown
---
title: Project Ideas
type: note
tags: [ideas, brainstorm]
---
# Project Ideas
## Observations
- [idea] Build a CLI tool for markdown linting #tooling
- [idea] Create a recipe knowledge base #cooking
- [priority] Focus on developer tools first (Q1 goal)
## Relations
- inspired_by [[Developer Workflow Research]]
- part_of [[Q1 Planning]]
```
### Schema-Validated Note
Schema at `schema/Person.md`:
```yaml
---
title: Person
type: schema
entity: Person
version: 1
schema:
name: string, full name
role?: string, job title or position
works_at?: Organization, employer
expertise?(array): string, areas of knowledge
email?: string, contact email
settings:
validation: warn
---
# Person
A human individual in the knowledge graph.
```
Note at `people/paul-graham.md`:
```markdown
---
title: Paul Graham
type: Person
tags: [startups, essays, lisp]
---
# Paul Graham
## Observations
- [name] Paul Graham
- [role] Essayist and investor
- [expertise] Startups
- [expertise] Lisp
- [expertise] Essay writing
- [fact] Created Viaweb, the first web app
## Relations
- works_at [[Y Combinator]]
- authored [[Hackers and Painters]]
```
The `[fact]` observation and `authored` relation are not in the schema — they're valid, just unmatched. The schema only checks that `[name]` exists (required) and looks for optional fields like `[role]`, `[expertise]`, and `works_at`.
### Inline Schema Note
```markdown
---
title: Team Standup 2024-01-15
type: meeting
schema:
attendees(array): string, who was there
decisions(array): string, what was decided
action_items(array): string, follow-ups
blockers?(array): string, anything stuck
---
# Team Standup 2024-01-15
## Observations
- [attendees] Paul
- [attendees] Sarah
- [decisions] Ship v2 by Friday
- [action_items] Paul to review PR #42
- [blockers] Waiting on API credentials
```
+49 -1
View File
@@ -1038,6 +1038,35 @@ recent_decisions = await search_notes(
)
```
**Structured frontmatter filters**:
```python
# Filter by tags and status
results = await search_notes(
query="authentication",
tags=["security"],
status="in-progress",
project="main"
)
# Complex metadata filters (supports $in, $gt, $gte, $lt, $lte, $between)
results = await search_notes(
query="api design",
metadata_filters={
"type": "spec",
"priority": {"$in": ["high", "critical"]},
"tags": ["architecture"]
},
project="main"
)
# Metadata-only search
results = await search_by_metadata(
filters={"type": "spec", "status": "in-progress"},
project="main"
)
```
### Search Types
**Text search (default)**:
@@ -2861,7 +2890,7 @@ contents = await list_directory(
### Search & Discovery
**search_notes(query, page, page_size, search_type, types, entity_types, after_date, project)**
**search_notes(query, page, page_size, search_type, types, entity_types, after_date, metadata_filters, tags, status, project)**
- Search across knowledge base
- Parameters:
- `query` (required): Search query
@@ -2871,6 +2900,9 @@ contents = await list_directory(
- `types` (optional): Entity type filter
- `entity_types` (optional): Observation category filter
- `after_date` (optional): Date filter (ISO format)
- `metadata_filters` (optional): Structured frontmatter filters (dict)
- `tags` (optional): Frontmatter tags filter (list)
- `status` (optional): Frontmatter status filter (string)
- `project` (required unless default_project_mode): Target project
- Returns: Matching entities with scores
- Example:
@@ -2883,6 +2915,22 @@ results = await search_notes(
)
```
**search_by_metadata(filters, limit, offset, project)**
- Metadata-only search using structured frontmatter
- Parameters:
- `filters` (required): Dict of field -> value (supports $in, $gt/$gte/$lt/$lte, $between)
- `limit` (optional): Max results (default: 20)
- `offset` (optional): Pagination offset (default: 0)
- `project` (required unless default_project_mode): Target project
- Returns: Matching entities
- Example:
```python
results = await search_by_metadata(
filters={"type": "spec", "status": "in-progress"},
project="main"
)
```
### Project Management
**list_memory_projects()**
+28
View File
@@ -62,6 +62,22 @@ test-int-postgres:
BASIC_MEMORY_TEST_POSTGRES=1 uv run pytest -p pytest_mock -v --no-cov test-int
fi
# Run tests impacted by recent changes (requires pytest-testmon)
testmon *args:
BASIC_MEMORY_ENV=test uv run pytest -p pytest_mock -v --no-cov --testmon --testmon-forceselect {{args}}
# Run MCP smoke test (fast end-to-end loop)
test-smoke:
BASIC_MEMORY_ENV=test uv run pytest -p pytest_mock -v --no-cov -m smoke test-int/mcp/test_smoke_integration.py
# Fast local loop: lint, format, typecheck, impacted tests
fast-check:
just fix
just format
just typecheck
just testmon
just test-smoke
# Reset Postgres test database (drops and recreates schema)
# Useful when Alembic migration state gets out of sync during development
# Uses credentials from docker-compose-postgres.yml
@@ -149,6 +165,18 @@ format:
run-inspector:
npx @modelcontextprotocol/inspector
# Run doctor checks in an isolated temp home/config
doctor:
#!/usr/bin/env bash
set -euo pipefail
TMP_HOME=$(mktemp -d)
TMP_CONFIG=$(mktemp -d)
HOME="$TMP_HOME" \
BASIC_MEMORY_ENV=test \
BASIC_MEMORY_HOME="$TMP_HOME/basic-memory" \
BASIC_MEMORY_CONFIG_DIR="$TMP_CONFIG" \
./.venv/bin/python -m basic_memory.cli.main doctor --local
# Update all dependencies to latest versions
update-deps:
+2
View File
@@ -71,6 +71,7 @@ markers = [
"slow: Slow-running tests (deselect with '-m \"not slow\"')",
"postgres: Tests that run against Postgres backend (deselect with '-m \"not postgres\"')",
"windows: Windows-specific tests (deselect with '-m \"not windows\"')",
"smoke: Fast end-to-end smoke tests for MCP flows",
]
[tool.ruff]
@@ -91,6 +92,7 @@ dev = [
"testcontainers[postgres]>=4.0.0",
"psycopg>=3.2.0",
"pyright>=1.1.408",
"pytest-testmon>=2.2.0",
]
[tool.hatch.version]
+2 -2
View File
@@ -6,12 +6,12 @@
"url": "https://github.com/basicmachines-co/basic-memory.git",
"source": "github"
},
"version": "0.18.0",
"version": "0.18.4",
"packages": [
{
"registryType": "pypi",
"identifier": "basic-memory",
"version": "0.18.0",
"version": "0.18.4",
"runtimeHint": "uvx",
"runtimeArguments": [
{"type": "positional", "value": "basic-memory"},
+1 -1
View File
@@ -1,7 +1,7 @@
"""basic-memory - Local-first knowledge management combining Zettelkasten with knowledge graphs"""
# Package version - updated by release automation
__version__ = "0.18.0"
__version__ = "0.18.4"
# API version for FastAPI - independent of package version
__api_version__ = "v0"
@@ -0,0 +1,154 @@
"""Add structured metadata indexes for entity frontmatter
Revision ID: d7e8f9a0b1c2
Revises: g9a0b3c4d5e6
Create Date: 2026-01-31 12:00:00.000000
"""
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
from sqlalchemy import text
def column_exists(connection, table: str, column: str) -> bool:
"""Check if a column exists in a table (idempotent migration support)."""
if connection.dialect.name == "postgresql":
result = connection.execute(
text(
"SELECT 1 FROM information_schema.columns "
"WHERE table_name = :table AND column_name = :column"
),
{"table": table, "column": column},
)
return result.fetchone() is not None
# SQLite
result = connection.execute(text(f"PRAGMA table_info({table})"))
columns = [row[1] for row in result]
return column in columns
def index_exists(connection, index_name: str) -> bool:
"""Check if an index exists (idempotent migration support)."""
if connection.dialect.name == "postgresql":
result = connection.execute(
text("SELECT 1 FROM pg_indexes WHERE indexname = :index_name"),
{"index_name": index_name},
)
return result.fetchone() is not None
# SQLite
result = connection.execute(
text("SELECT 1 FROM sqlite_master WHERE type='index' AND name = :index_name"),
{"index_name": index_name},
)
return result.fetchone() is not None
# revision identifiers, used by Alembic.
revision: str = "d7e8f9a0b1c2"
down_revision: Union[str, None] = "6830751f5fb6"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
"""Add JSONB/GiN indexes for Postgres and generated columns for SQLite."""
connection = op.get_bind()
dialect = connection.dialect.name
if dialect == "postgresql":
# Ensure JSONB for efficient indexing
result = connection.execute(
text(
"SELECT data_type FROM information_schema.columns "
"WHERE table_name = 'entity' AND column_name = 'entity_metadata'"
)
).fetchone()
if result and result[0] != "jsonb":
op.execute(
"ALTER TABLE entity ALTER COLUMN entity_metadata "
"TYPE jsonb USING entity_metadata::jsonb"
)
# General JSONB GIN index
op.execute(
"CREATE INDEX IF NOT EXISTS idx_entity_metadata_gin "
"ON entity USING GIN (entity_metadata jsonb_path_ops)"
)
# Common field indexes
op.execute(
"CREATE INDEX IF NOT EXISTS idx_entity_tags_json "
"ON entity USING GIN ((entity_metadata -> 'tags'))"
)
op.execute(
"CREATE INDEX IF NOT EXISTS idx_entity_frontmatter_type "
"ON entity ((entity_metadata ->> 'type'))"
)
op.execute(
"CREATE INDEX IF NOT EXISTS idx_entity_frontmatter_status "
"ON entity ((entity_metadata ->> 'status'))"
)
return
# SQLite: add generated columns for common frontmatter fields
# Constraint: SQLite ALTER TABLE ADD COLUMN only supports VIRTUAL generated columns,
# not STORED. json_extract is deterministic so VIRTUAL columns can still be indexed.
if not column_exists(connection, "entity", "tags_json"):
op.add_column(
"entity",
sa.Column(
"tags_json",
sa.Text(),
sa.Computed("json_extract(entity_metadata, '$.tags')", persisted=False),
),
)
if not column_exists(connection, "entity", "frontmatter_status"):
op.add_column(
"entity",
sa.Column(
"frontmatter_status",
sa.Text(),
sa.Computed("json_extract(entity_metadata, '$.status')", persisted=False),
),
)
if not column_exists(connection, "entity", "frontmatter_type"):
op.add_column(
"entity",
sa.Column(
"frontmatter_type",
sa.Text(),
sa.Computed("json_extract(entity_metadata, '$.type')", persisted=False),
),
)
# Index generated columns
if not index_exists(connection, "idx_entity_tags_json"):
op.create_index("idx_entity_tags_json", "entity", ["tags_json"])
if not index_exists(connection, "idx_entity_frontmatter_status"):
op.create_index("idx_entity_frontmatter_status", "entity", ["frontmatter_status"])
if not index_exists(connection, "idx_entity_frontmatter_type"):
op.create_index("idx_entity_frontmatter_type", "entity", ["frontmatter_type"])
def downgrade() -> None:
"""Best-effort downgrade (drop indexes, revert JSONB on Postgres)."""
connection = op.get_bind()
dialect = connection.dialect.name
if dialect == "postgresql":
op.execute("DROP INDEX IF EXISTS idx_entity_frontmatter_status")
op.execute("DROP INDEX IF EXISTS idx_entity_frontmatter_type")
op.execute("DROP INDEX IF EXISTS idx_entity_tags_json")
op.execute("DROP INDEX IF EXISTS idx_entity_metadata_gin")
op.execute(
"ALTER TABLE entity ALTER COLUMN entity_metadata TYPE json USING entity_metadata::json"
)
return
# SQLite: drop indexes (dropping generated columns requires table rebuild)
op.execute("DROP INDEX IF EXISTS idx_entity_frontmatter_status")
op.execute("DROP INDEX IF EXISTS idx_entity_frontmatter_type")
op.execute("DROP INDEX IF EXISTS idx_entity_tags_json")
+42 -24
View File
@@ -2,23 +2,13 @@
from contextlib import asynccontextmanager
from fastapi import FastAPI, HTTPException
from fastapi import FastAPI, HTTPException, Request
from fastapi.exception_handlers import http_exception_handler
from fastapi.routing import APIRouter
from loguru import logger
from basic_memory import __version__ as version
from basic_memory.api.container import ApiContainer, set_container
from basic_memory.api.routers import (
directory_router,
importer_router,
knowledge,
management,
memory,
project,
resource,
search,
prompt_router,
)
from basic_memory.api.v2.routers import (
knowledge_router as v2_knowledge,
project_router as v2_project,
@@ -29,7 +19,13 @@ from basic_memory.api.v2.routers import (
prompt_router as v2_prompt,
importer_router as v2_importer,
)
from basic_memory.api.v2.routers.project_router import (
add_project,
list_projects,
synchronize_projects,
)
from basic_memory.config import init_api_logging
from basic_memory.services.exceptions import EntityAlreadyExistsError
from basic_memory.services.initialization import initialize_app
@@ -90,19 +86,41 @@ app.include_router(v2_prompt, prefix="/v2/projects/{project_id}")
app.include_router(v2_importer, prefix="/v2/projects/{project_id}")
app.include_router(v2_project, prefix="/v2")
# Include v1 routers (/{project} is a catch-all, must come after specific prefixes)
app.include_router(knowledge.router, prefix="/{project}")
app.include_router(memory.router, prefix="/{project}")
app.include_router(resource.router, prefix="/{project}")
app.include_router(search.router, prefix="/{project}")
app.include_router(project.project_router, prefix="/{project}")
app.include_router(directory_router.router, prefix="/{project}")
app.include_router(prompt_router.router, prefix="/{project}")
app.include_router(importer_router.router, prefix="/{project}")
# Legacy web app proxy paths (compat with /proxy/projects/projects)
app.include_router(v2_project, prefix="/proxy/projects")
# Project resource router works across projects
app.include_router(project.project_resource_router)
app.include_router(management.router)
# Legacy v1 compat: older CLI versions (v0.18.0 and earlier) call /projects/...
# Using router mount causes 307 redirect which proxy doesn't follow, so add explicit routes
legacy_router = APIRouter(tags=["legacy"])
legacy_router.add_api_route("/projects/projects", list_projects, methods=["GET"])
legacy_router.add_api_route("/projects/projects", add_project, methods=["POST"])
legacy_router.add_api_route("/projects/config/sync", synchronize_projects, methods=["POST"])
app.include_router(legacy_router)
# V2 routers are the only public API surface
@app.exception_handler(EntityAlreadyExistsError)
async def entity_already_exists_error_handler(request: Request, exc: EntityAlreadyExistsError):
"""Handle entity creation conflicts (e.g., file already exists).
This is expected behavior when users try to create notes that exist,
so log at INFO level instead of ERROR.
"""
logger.info(
"Entity already exists",
url=str(request.url),
method=request.method,
path=request.url.path,
error=str(exc),
)
return await http_exception_handler(
request,
HTTPException(
status_code=409,
detail="Note already exists. Use edit_note to modify it, or delete it first.",
),
)
@app.exception_handler(Exception)
-11
View File
@@ -1,11 +0,0 @@
"""API routers."""
from . import knowledge_router as knowledge
from . import management_router as management
from . import memory_router as memory
from . import project_router as project
from . import resource_router as resource
from . import search_router as search
from . import prompt_router as prompt
__all__ = ["knowledge", "management", "memory", "project", "resource", "search", "prompt"]
@@ -1,84 +0,0 @@
"""Router for directory tree operations."""
from typing import List, Optional
from fastapi import APIRouter, Query
from basic_memory.deps import DirectoryServiceDep, ProjectIdDep
from basic_memory.schemas.directory import DirectoryNode
router = APIRouter(prefix="/directory", tags=["directory"])
@router.get("/tree", response_model=DirectoryNode, response_model_exclude_none=True)
async def get_directory_tree(
directory_service: DirectoryServiceDep,
project_id: ProjectIdDep,
):
"""Get hierarchical directory structure from the knowledge base.
Args:
directory_service: Service for directory operations
project_id: ID of the current project
Returns:
DirectoryNode representing the root of the hierarchical tree structure
"""
# Get a hierarchical directory tree for the specific project
tree = await directory_service.get_directory_tree()
# Return the hierarchical tree
return tree
@router.get("/structure", response_model=DirectoryNode, response_model_exclude_none=True)
async def get_directory_structure(
directory_service: DirectoryServiceDep,
project_id: ProjectIdDep,
):
"""Get folder structure for navigation (no files).
Optimized endpoint for folder tree navigation. Returns only directory nodes
without file metadata. For full tree with files, use /directory/tree.
Args:
directory_service: Service for directory operations
project_id: ID of the current project
Returns:
DirectoryNode tree containing only folders (type="directory")
"""
structure = await directory_service.get_directory_structure()
return structure
@router.get("/list", response_model=List[DirectoryNode], response_model_exclude_none=True)
async def list_directory(
directory_service: DirectoryServiceDep,
project_id: ProjectIdDep,
dir_name: str = Query("/", description="Directory path to list"),
depth: int = Query(1, ge=1, le=10, description="Recursion depth (1-10)"),
file_name_glob: Optional[str] = Query(
None, description="Glob pattern for filtering file names"
),
):
"""List directory contents with filtering and depth control.
Args:
directory_service: Service for directory operations
project_id: ID of the current project
dir_name: Directory path to list (default: root "/")
depth: Recursion depth (1-10, default: 1 for immediate children only)
file_name_glob: Optional glob pattern for filtering file names (e.g., "*.md", "*meeting*")
Returns:
List of DirectoryNode objects matching the criteria
"""
# Get directory listing with filtering
nodes = await directory_service.list_directory(
dir_name=dir_name,
depth=depth,
file_name_glob=file_name_glob,
)
return nodes
@@ -1,152 +0,0 @@
"""Import router for Basic Memory API."""
import json
import logging
from fastapi import APIRouter, Form, HTTPException, UploadFile, status
from basic_memory.deps import (
ChatGPTImporterDep,
ClaudeConversationsImporterDep,
ClaudeProjectsImporterDep,
MemoryJsonImporterDep,
)
from basic_memory.importers import Importer
from basic_memory.schemas.importer import (
ChatImportResult,
EntityImportResult,
ProjectImportResult,
)
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/import", tags=["import"])
@router.post("/chatgpt", response_model=ChatImportResult)
async def import_chatgpt(
importer: ChatGPTImporterDep,
file: UploadFile,
directory: str = Form("conversations"),
) -> ChatImportResult:
"""Import conversations from ChatGPT JSON export.
Args:
file: The ChatGPT conversations.json file.
directory: The directory to place the files in.
markdown_processor: MarkdownProcessor instance.
Returns:
ChatImportResult with import statistics.
Raises:
HTTPException: If import fails.
"""
return await import_file(importer, file, directory)
@router.post("/claude/conversations", response_model=ChatImportResult)
async def import_claude_conversations(
importer: ClaudeConversationsImporterDep,
file: UploadFile,
directory: str = Form("conversations"),
) -> ChatImportResult:
"""Import conversations from Claude conversations.json export.
Args:
file: The Claude conversations.json file.
directory: The directory to place the files in.
markdown_processor: MarkdownProcessor instance.
Returns:
ChatImportResult with import statistics.
Raises:
HTTPException: If import fails.
"""
return await import_file(importer, file, directory)
@router.post("/claude/projects", response_model=ProjectImportResult)
async def import_claude_projects(
importer: ClaudeProjectsImporterDep,
file: UploadFile,
directory: str = Form("projects"),
) -> ProjectImportResult:
"""Import projects from Claude projects.json export.
Args:
file: The Claude projects.json file.
directory: The directory to place the files in.
markdown_processor: MarkdownProcessor instance.
Returns:
ProjectImportResult with import statistics.
Raises:
HTTPException: If import fails.
"""
return await import_file(importer, file, directory)
@router.post("/memory-json", response_model=EntityImportResult)
async def import_memory_json(
importer: MemoryJsonImporterDep,
file: UploadFile,
directory: str = Form("conversations"),
) -> EntityImportResult:
"""Import entities and relations from a memory.json file.
Args:
file: The memory.json file.
directory: Optional destination directory within the project.
markdown_processor: MarkdownProcessor instance.
Returns:
EntityImportResult with import statistics.
Raises:
HTTPException: If import fails.
"""
try:
file_data = []
file_bytes = await file.read()
file_str = file_bytes.decode("utf-8")
for line in file_str.splitlines():
json_data = json.loads(line)
file_data.append(json_data)
result = await importer.import_data(file_data, directory)
if not result.success: # pragma: no cover
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail=result.error_message or "Import failed",
)
except Exception as e:
logger.exception("Import failed")
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail=f"Import failed: {str(e)}",
)
return result
async def import_file(importer: Importer, file: UploadFile, destination_folder: str):
try:
# Process file
json_data = json.load(file.file)
result = await importer.import_data(json_data, destination_folder)
if not result.success: # pragma: no cover
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail=result.error_message or "Import failed",
)
return result
except Exception as e:
logger.exception("Import failed")
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail=f"Import failed: {str(e)}",
)
@@ -1,364 +0,0 @@
"""Router for knowledge graph operations.
⚠️ DEPRECATED: This v1 API is deprecated and will be removed on June 30, 2026.
Please migrate to /v2/{project}/knowledge endpoints which use entity IDs instead
of path-based identifiers for improved performance and stability.
Migration guide: See docs/migration/v1-to-v2.md
"""
from typing import Annotated
from fastapi import APIRouter, HTTPException, BackgroundTasks, Depends, Query, Response
from loguru import logger
from basic_memory.deps import (
EntityServiceDep,
get_search_service,
SearchServiceDep,
LinkResolverDep,
ProjectPathDep,
FileServiceDep,
ProjectConfigDep,
AppConfigDep,
SyncServiceDep,
)
from basic_memory.schemas import (
EntityListResponse,
EntityResponse,
DeleteEntitiesResponse,
DeleteEntitiesRequest,
)
from basic_memory.schemas.request import EditEntityRequest, MoveEntityRequest, MoveDirectoryRequest
from basic_memory.schemas.response import DirectoryMoveResult
from basic_memory.schemas.base import Permalink, Entity
router = APIRouter(
prefix="/knowledge",
tags=["knowledge"],
deprecated=True, # Marks entire router as deprecated in OpenAPI docs
)
async def resolve_relations_background(sync_service, entity_id: int, entity_permalink: str) -> None:
"""Background task to resolve relations for a specific entity.
This runs asynchronously after the API response is sent, preventing
long delays when creating entities with many relations.
"""
try:
# Only resolve relations for the newly created entity
await sync_service.resolve_relations(entity_id=entity_id)
logger.debug(
f"Background: Resolved relations for entity {entity_permalink} (id={entity_id})"
)
except Exception as e: # pragma: no cover
# Log but don't fail - this is a background task.
# Avoid forcing synthetic failures just for coverage.
logger.warning( # pragma: no cover
f"Background: Failed to resolve relations for entity {entity_permalink}: {e}"
)
## Create endpoints
@router.post("/entities", response_model=EntityResponse)
async def create_entity(
data: Entity,
background_tasks: BackgroundTasks,
entity_service: EntityServiceDep,
search_service: SearchServiceDep,
) -> EntityResponse:
"""Create an entity."""
logger.info(
"API request", endpoint="create_entity", entity_type=data.entity_type, title=data.title
)
entity = await entity_service.create_entity(data)
# reindex
await search_service.index_entity(entity, background_tasks=background_tasks)
result = EntityResponse.model_validate(entity)
logger.info(
f"API response: endpoint='create_entity' title={result.title}, permalink={result.permalink}, status_code=201"
)
return result
@router.put("/entities/{permalink:path}", response_model=EntityResponse)
async def create_or_update_entity(
project: ProjectPathDep,
permalink: Permalink,
data: Entity,
response: Response,
background_tasks: BackgroundTasks,
entity_service: EntityServiceDep,
search_service: SearchServiceDep,
file_service: FileServiceDep,
sync_service: SyncServiceDep,
) -> EntityResponse:
"""Create or update an entity. If entity exists, it will be updated, otherwise created."""
logger.info(
f"API request: create_or_update_entity for {project=}, {permalink=}, {data.entity_type=}, {data.title=}"
)
# Validate permalink matches
if data.permalink != permalink:
logger.warning(
f"API validation error: creating/updating entity with permalink mismatch - url={permalink}, data={data.permalink}",
)
raise HTTPException(
status_code=400,
detail=f"Entity permalink {data.permalink} must match URL path: '{permalink}'",
)
# Try create_or_update operation
entity, created = await entity_service.create_or_update_entity(data)
response.status_code = 201 if created else 200
# reindex
await search_service.index_entity(entity, background_tasks=background_tasks)
# Schedule relation resolution as a background task for new entities
# This prevents blocking the API response while resolving potentially many relations
if created:
background_tasks.add_task(
resolve_relations_background, sync_service, entity.id, entity.permalink or ""
)
result = EntityResponse.model_validate(entity)
logger.info(
f"API response: {result.title=}, {result.permalink=}, {created=}, status_code={response.status_code}"
)
return result
@router.patch("/entities/{identifier:path}", response_model=EntityResponse)
async def edit_entity(
identifier: str,
data: EditEntityRequest,
background_tasks: BackgroundTasks,
entity_service: EntityServiceDep,
search_service: SearchServiceDep,
) -> EntityResponse:
"""Edit an existing entity using various operations like append, prepend, find_replace, or replace_section.
This endpoint allows for targeted edits without requiring the full entity content.
"""
logger.info(
f"API request: endpoint='edit_entity', identifier='{identifier}', operation='{data.operation}'"
)
try:
# Edit the entity using the service
entity = await entity_service.edit_entity(
identifier=identifier,
operation=data.operation,
content=data.content,
section=data.section,
find_text=data.find_text,
expected_replacements=data.expected_replacements,
)
# Reindex the updated entity
await search_service.index_entity(entity, background_tasks=background_tasks)
# Return the updated entity response
result = EntityResponse.model_validate(entity)
logger.info(
"API response",
endpoint="edit_entity",
identifier=identifier,
operation=data.operation,
permalink=result.permalink,
status_code=200,
)
return result
except Exception as e:
logger.error(f"Error editing entity: {e}")
raise HTTPException(status_code=400, detail=str(e))
@router.post("/move")
async def move_entity(
data: MoveEntityRequest,
background_tasks: BackgroundTasks,
entity_service: EntityServiceDep,
project_config: ProjectConfigDep,
app_config: AppConfigDep,
search_service: SearchServiceDep,
) -> EntityResponse:
"""Move an entity to a new file location with project consistency.
This endpoint moves a note to a different path while maintaining project
consistency and optionally updating permalinks based on configuration.
"""
logger.info(
f"API request: endpoint='move_entity', identifier='{data.identifier}', destination='{data.destination_path}'"
)
try:
# Move the entity using the service
moved_entity = await entity_service.move_entity(
identifier=data.identifier,
destination_path=data.destination_path,
project_config=project_config,
app_config=app_config,
)
# Get the moved entity to reindex it
entity = await entity_service.link_resolver.resolve_link(data.destination_path)
if entity:
await search_service.index_entity(entity, background_tasks=background_tasks)
logger.info(
"API response",
endpoint="move_entity",
identifier=data.identifier,
destination=data.destination_path,
status_code=200,
)
result = EntityResponse.model_validate(moved_entity)
return result
except Exception as e:
logger.error(f"Error moving entity: {e}")
raise HTTPException(status_code=400, detail=str(e))
@router.post("/move-directory")
async def move_directory(
data: MoveDirectoryRequest,
background_tasks: BackgroundTasks,
entity_service: EntityServiceDep,
project_config: ProjectConfigDep,
app_config: AppConfigDep,
search_service: SearchServiceDep,
) -> DirectoryMoveResult:
"""Move all entities in a directory to a new location.
This endpoint moves all files within a source directory to a destination
directory, updating database records and optionally updating permalinks.
"""
logger.info(
f"API request: endpoint='move_directory', source='{data.source_directory}', destination='{data.destination_directory}'"
)
try:
# Move the directory using the service
result = await entity_service.move_directory(
source_directory=data.source_directory,
destination_directory=data.destination_directory,
project_config=project_config,
app_config=app_config,
)
# Reindex moved entities
for file_path in result.moved_files:
entity = await entity_service.link_resolver.resolve_link(file_path)
if entity:
await search_service.index_entity(entity, background_tasks=background_tasks)
logger.info(
f"API response: endpoint='move_directory', "
f"total={result.total_files}, success={result.successful_moves}, failed={result.failed_moves}"
)
return result
except Exception as e:
logger.error(f"Error moving directory: {e}")
raise HTTPException(status_code=400, detail=str(e))
## Read endpoints
@router.get("/entities/{identifier:path}", response_model=EntityResponse)
async def get_entity(
entity_service: EntityServiceDep,
link_resolver: LinkResolverDep,
identifier: str,
) -> EntityResponse:
"""Get a specific entity by file path or permalink..
Args:
identifier: Entity file path or permalink
:param entity_service: EntityService
:param link_resolver: LinkResolver
"""
logger.info(f"request: get_entity with identifier={identifier}")
entity = await link_resolver.resolve_link(identifier)
if not entity:
raise HTTPException(status_code=404, detail=f"Entity {identifier} not found")
result = EntityResponse.model_validate(entity)
return result
@router.get("/entities", response_model=EntityListResponse)
async def get_entities(
entity_service: EntityServiceDep,
permalink: Annotated[list[str] | None, Query()] = None,
) -> EntityListResponse:
"""Open specific entities"""
logger.info(f"request: get_entities with permalinks={permalink}")
entities = await entity_service.get_entities_by_permalinks(permalink) if permalink else []
result = EntityListResponse(
entities=[EntityResponse.model_validate(entity) for entity in entities]
)
return result
## Delete endpoints
@router.delete("/entities/{identifier:path}", response_model=DeleteEntitiesResponse)
async def delete_entity(
identifier: str,
background_tasks: BackgroundTasks,
entity_service: EntityServiceDep,
link_resolver: LinkResolverDep,
search_service=Depends(get_search_service),
) -> DeleteEntitiesResponse:
"""Delete a single entity and remove from search index."""
logger.info(f"request: delete_entity with identifier={identifier}")
entity = await link_resolver.resolve_link(identifier)
if entity is None:
return DeleteEntitiesResponse(deleted=False)
# Delete the entity
deleted = await entity_service.delete_entity(entity.permalink or entity.id)
# Remove from search index (entity, observations, and relations)
background_tasks.add_task(search_service.handle_delete, entity)
result = DeleteEntitiesResponse(deleted=deleted)
return result
@router.post("/entities/delete", response_model=DeleteEntitiesResponse)
async def delete_entities(
data: DeleteEntitiesRequest,
background_tasks: BackgroundTasks,
entity_service: EntityServiceDep,
search_service=Depends(get_search_service),
) -> DeleteEntitiesResponse:
"""Delete entities and remove from search index."""
logger.info(f"request: delete_entities with data={data}")
deleted = False
# Remove each deleted entity from search index
for permalink in data.permalinks:
deleted = await entity_service.delete_entity(permalink)
background_tasks.add_task(search_service.delete_by_permalink, permalink)
result = DeleteEntitiesResponse(deleted=deleted)
return result
@@ -1,80 +0,0 @@
"""Management router for basic-memory API."""
import asyncio
from fastapi import APIRouter, Request
from loguru import logger
from pydantic import BaseModel
from basic_memory.config import ConfigManager
from basic_memory.deps import SyncServiceDep, ProjectRepositoryDep
router = APIRouter(prefix="/management", tags=["management"])
class WatchStatusResponse(BaseModel):
"""Response model for watch status."""
running: bool
"""Whether the watch service is currently running."""
@router.get("/watch/status", response_model=WatchStatusResponse)
async def get_watch_status(request: Request) -> WatchStatusResponse:
"""Get the current status of the watch service."""
return WatchStatusResponse(
running=request.app.state.watch_task is not None and not request.app.state.watch_task.done()
)
@router.post("/watch/start", response_model=WatchStatusResponse)
async def start_watch_service(
request: Request, project_repository: ProjectRepositoryDep, sync_service: SyncServiceDep
) -> WatchStatusResponse:
"""Start the watch service if it's not already running."""
# needed because of circular imports from sync -> app
from basic_memory.sync import WatchService
from basic_memory.sync.background_sync import create_background_sync_task
if request.app.state.watch_task is not None and not request.app.state.watch_task.done():
# Watch service is already running
return WatchStatusResponse(running=True)
app_config = ConfigManager().config
# Create and start a new watch service
logger.info("Starting watch service via management API")
# Get services needed for the watch task
watch_service = WatchService(
app_config=app_config,
project_repository=project_repository,
)
# Create and store the task
watch_task = create_background_sync_task(sync_service, watch_service)
request.app.state.watch_task = watch_task
return WatchStatusResponse(running=True)
@router.post("/watch/stop", response_model=WatchStatusResponse)
async def stop_watch_service(request: Request) -> WatchStatusResponse: # pragma: no cover
"""Stop the watch service if it's running."""
if request.app.state.watch_task is None or request.app.state.watch_task.done():
# Watch service is not running
return WatchStatusResponse(running=False)
# Cancel the running task
logger.info("Stopping watch service via management API")
request.app.state.watch_task.cancel()
# Wait for it to be properly cancelled
try:
await request.app.state.watch_task
except asyncio.CancelledError:
pass
request.app.state.watch_task = None
return WatchStatusResponse(running=False)
@@ -1,90 +0,0 @@
"""Routes for memory:// URI operations."""
from typing import Annotated, Optional
from fastapi import APIRouter, Query
from loguru import logger
from basic_memory.deps import ContextServiceDep, EntityRepositoryDep
from basic_memory.schemas.base import TimeFrame, parse_timeframe
from basic_memory.schemas.memory import (
GraphContext,
normalize_memory_url,
)
from basic_memory.schemas.search import SearchItemType
from basic_memory.api.routers.utils import to_graph_context
router = APIRouter(prefix="/memory", tags=["memory"])
@router.get("/recent", response_model=GraphContext)
async def recent(
context_service: ContextServiceDep,
entity_repository: EntityRepositoryDep,
type: Annotated[list[SearchItemType] | None, Query()] = None,
depth: int = 1,
timeframe: TimeFrame = "7d",
page: int = 1,
page_size: int = 10,
max_related: int = 10,
) -> GraphContext:
# return all types by default
types = (
[SearchItemType.ENTITY, SearchItemType.RELATION, SearchItemType.OBSERVATION]
if not type
else type
)
logger.debug(
f"Getting recent context: `{types}` depth: `{depth}` timeframe: `{timeframe}` page: `{page}` page_size: `{page_size}` max_related: `{max_related}`"
)
# Parse timeframe
since = parse_timeframe(timeframe)
limit = page_size
offset = (page - 1) * page_size
# Build context
context = await context_service.build_context(
types=types, depth=depth, since=since, limit=limit, offset=offset, max_related=max_related
)
recent_context = await to_graph_context(
context, entity_repository=entity_repository, page=page, page_size=page_size
)
logger.debug(f"Recent context: {recent_context.model_dump_json()}")
return recent_context
# get_memory_context needs to be declared last so other paths can match
@router.get("/{uri:path}", response_model=GraphContext)
async def get_memory_context(
context_service: ContextServiceDep,
entity_repository: EntityRepositoryDep,
uri: str,
depth: int = 1,
timeframe: Optional[TimeFrame] = None,
page: int = 1,
page_size: int = 10,
max_related: int = 10,
) -> GraphContext:
"""Get rich context from memory:// URI."""
# add the project name from the config to the url as the "host
# Parse URI
logger.debug(
f"Getting context for URI: `{uri}` depth: `{depth}` timeframe: `{timeframe}` page: `{page}` page_size: `{page_size}` max_related: `{max_related}`"
)
memory_url = normalize_memory_url(uri)
# Parse timeframe
since = parse_timeframe(timeframe) if timeframe else None
limit = page_size
offset = (page - 1) * page_size
# Build context
context = await context_service.build_context(
memory_url, depth=depth, since=since, limit=limit, offset=offset, max_related=max_related
)
return await to_graph_context(
context, entity_repository=entity_repository, page=page, page_size=page_size
)
@@ -1,472 +0,0 @@
"""Router for project management."""
import os
from fastapi import APIRouter, HTTPException, Path, Body, BackgroundTasks, Response, Query
from typing import Optional
from loguru import logger
from basic_memory.deps import (
ProjectConfigDep,
ProjectServiceDep,
ProjectPathDep,
SyncServiceDep,
)
from basic_memory.schemas import ProjectInfoResponse, SyncReportResponse
from basic_memory.schemas.project_info import (
ProjectList,
ProjectItem,
ProjectInfoRequest,
ProjectStatusResponse,
)
from basic_memory.utils import normalize_project_path
# Router for resources in a specific project
# The ProjectPathDep is used in the path as a prefix, so the request path is like /{project}/project/info
project_router = APIRouter(prefix="/project", tags=["project"])
# Router for managing project resources
project_resource_router = APIRouter(prefix="/projects", tags=["project_management"])
@project_router.get("/info", response_model=ProjectInfoResponse)
async def get_project_info(
project_service: ProjectServiceDep,
project: ProjectPathDep,
) -> ProjectInfoResponse:
"""Get comprehensive information about the specified Basic Memory project."""
return await project_service.get_project_info(project)
@project_router.get("/item", response_model=ProjectItem)
async def get_project(
project_service: ProjectServiceDep,
project: ProjectPathDep,
) -> ProjectItem:
"""Get bassic info about the specified Basic Memory project."""
found_project = await project_service.get_project(project)
if not found_project:
raise HTTPException(
status_code=404, detail=f"Project: '{project}' does not exist"
) # pragma: no cover
return ProjectItem(
id=found_project.id,
external_id=found_project.external_id,
name=found_project.name,
path=normalize_project_path(found_project.path),
is_default=found_project.is_default or False,
)
# Update a project
@project_router.patch("/{name}", response_model=ProjectStatusResponse)
async def update_project(
project_service: ProjectServiceDep,
name: str = Path(..., description="Name of the project to update"),
path: Optional[str] = Body(None, description="New absolute path for the project"),
is_active: Optional[bool] = Body(None, description="Status of the project (active/inactive)"),
) -> ProjectStatusResponse:
"""Update a project's information in configuration and database.
Args:
name: The name of the project to update
path: Optional new absolute path for the project
is_active: Optional status update for the project
Returns:
Response confirming the project was updated
"""
try:
# Validate that path is absolute if provided
if path and not os.path.isabs(path):
raise HTTPException(status_code=400, detail="Path must be absolute")
# Get original project info for the response
old_project = await project_service.get_project(name)
if not old_project:
raise HTTPException(
status_code=400, detail=f"Project '{name}' not found in configuration"
)
old_project_info = ProjectItem(
id=old_project.id,
external_id=old_project.external_id,
name=old_project.name,
path=old_project.path,
is_default=old_project.is_default or False,
)
if path:
await project_service.move_project(name, path)
elif is_active is not None:
await project_service.update_project(name, is_active=is_active)
# Get updated project info
updated_project = await project_service.get_project(name)
if not updated_project:
raise HTTPException( # pragma: no cover
status_code=404, detail=f"Project '{name}' not found after update"
)
return ProjectStatusResponse(
message=f"Project '{name}' updated successfully",
status="success",
default=(name == project_service.default_project),
old_project=old_project_info,
new_project=ProjectItem(
id=updated_project.id,
external_id=updated_project.external_id,
name=updated_project.name,
path=updated_project.path,
is_default=updated_project.is_default or False,
),
)
except ValueError as e:
raise HTTPException(status_code=400, detail=str(e)) # pragma: no cover
# Sync project filesystem
@project_router.post("/sync")
async def sync_project(
background_tasks: BackgroundTasks,
sync_service: SyncServiceDep,
project_config: ProjectConfigDep,
force_full: bool = Query(
False, description="Force full scan, bypassing watermark optimization"
),
run_in_background: bool = Query(True, description="Run in background"),
):
"""Force project filesystem sync to database.
Scans the project directory and updates the database with any new or modified files.
Args:
background_tasks: FastAPI background tasks
sync_service: Sync service for this project
project_config: Project configuration
force_full: If True, force a full scan even if watermark exists
run_in_background: If True, run sync in background and return immediately
Returns:
Response confirming sync was initiated (background) or SyncReportResponse (foreground)
"""
if run_in_background:
background_tasks.add_task(
sync_service.sync, project_config.home, project_config.name, force_full=force_full
)
logger.info(
f"Filesystem sync initiated for project: {project_config.name} (force_full={force_full})"
)
return {
"status": "sync_started",
"message": f"Filesystem sync initiated for project '{project_config.name}'",
}
else:
report = await sync_service.sync(
project_config.home, project_config.name, force_full=force_full
)
logger.info(
f"Filesystem sync completed for project: {project_config.name} (force_full={force_full})"
)
return SyncReportResponse.from_sync_report(report)
@project_router.post("/status", response_model=SyncReportResponse)
async def project_sync_status(
sync_service: SyncServiceDep,
project_config: ProjectConfigDep,
) -> SyncReportResponse:
"""Scan directory for changes compared to database state.
Args:
sync_service: Sync service for this project
project_config: Project configuration
Returns:
Scan report with details on files that need syncing
"""
logger.info(f"Scanning filesystem for project: {project_config.name}") # pragma: no cover
sync_report = await sync_service.scan(project_config.home) # pragma: no cover
return SyncReportResponse.from_sync_report(sync_report) # pragma: no cover
# List all available projects
@project_resource_router.get("/projects", response_model=ProjectList)
async def list_projects(
project_service: ProjectServiceDep,
) -> ProjectList:
"""List all configured projects.
Returns:
A list of all projects with metadata
"""
projects = await project_service.list_projects()
default_project = project_service.default_project
project_items = [
ProjectItem(
id=project.id,
external_id=project.external_id,
name=project.name,
path=normalize_project_path(project.path),
is_default=project.is_default or False,
)
for project in projects
]
return ProjectList(
projects=project_items,
default_project=default_project,
)
# Add a new project
@project_resource_router.post("/projects", response_model=ProjectStatusResponse, status_code=201)
async def add_project(
response: Response,
project_data: ProjectInfoRequest,
project_service: ProjectServiceDep,
) -> ProjectStatusResponse:
"""Add a new project to configuration and database.
Args:
project_data: The project name and path, with option to set as default
Returns:
Response confirming the project was added
"""
# Check if project already exists before attempting to add
existing_project = await project_service.get_project(project_data.name)
if existing_project:
# Project exists - check if paths match for true idempotency
# Normalize paths for comparison (resolve symlinks, etc.)
from pathlib import Path
requested_path = Path(project_data.path).resolve()
existing_path = Path(existing_project.path).resolve()
if requested_path == existing_path:
# Same name, same path - return 200 OK (idempotent)
response.status_code = 200
return ProjectStatusResponse( # pyright: ignore [reportCallIssue]
message=f"Project '{project_data.name}' already exists",
status="success",
default=existing_project.is_default or False,
new_project=ProjectItem(
id=existing_project.id,
external_id=existing_project.external_id,
name=existing_project.name,
path=existing_project.path,
is_default=existing_project.is_default or False,
),
)
else:
# Same name, different path - this is an error
raise HTTPException(
status_code=400,
detail=f"Project '{project_data.name}' already exists with different path. Existing: {existing_project.path}, Requested: {project_data.path}",
)
try: # pragma: no cover
# The service layer now handles cloud mode validation and path sanitization
await project_service.add_project(
project_data.name, project_data.path, set_default=project_data.set_default
)
# Fetch the newly created project to get its ID
new_project = await project_service.get_project(project_data.name)
if not new_project:
raise HTTPException(status_code=500, detail="Failed to retrieve newly created project")
return ProjectStatusResponse( # pyright: ignore [reportCallIssue]
message=f"Project '{new_project.name}' added successfully",
status="success",
default=project_data.set_default,
new_project=ProjectItem(
id=new_project.id,
external_id=new_project.external_id,
name=new_project.name,
path=new_project.path,
is_default=new_project.is_default or False,
),
)
except ValueError as e: # pragma: no cover
raise HTTPException(status_code=400, detail=str(e))
# Remove a project
@project_resource_router.delete("/{name}", response_model=ProjectStatusResponse)
async def remove_project(
project_service: ProjectServiceDep,
name: str = Path(..., description="Name of the project to remove"),
delete_notes: bool = Query(
False, description="If True, delete project directory from filesystem"
),
) -> ProjectStatusResponse:
"""Remove a project from configuration and database.
Args:
name: The name of the project to remove
delete_notes: If True, delete the project directory from the filesystem
Returns:
Response confirming the project was removed
"""
try:
old_project = await project_service.get_project(name)
if not old_project: # pragma: no cover
raise HTTPException(
status_code=404, detail=f"Project: '{name}' does not exist"
) # pragma: no cover
# Check if trying to delete the default project
# In cloud mode, database is source of truth; in local mode, check config
config_default = project_service.default_project
db_default = await project_service.repository.get_default_project()
# Use database default if available, otherwise fall back to config default
default_project_name = db_default.name if db_default else config_default
if name == default_project_name:
available_projects = await project_service.list_projects()
other_projects = [p.name for p in available_projects if p.name != name]
detail = f"Cannot delete default project '{name}'. "
if other_projects:
detail += (
f"Set another project as default first. Available: {', '.join(other_projects)}"
)
else:
detail += "This is the only project in your configuration."
raise HTTPException(status_code=400, detail=detail)
await project_service.remove_project(name, delete_notes=delete_notes)
return ProjectStatusResponse(
message=f"Project '{old_project.name}' removed successfully",
status="success",
default=False,
old_project=ProjectItem(
id=old_project.id,
external_id=old_project.external_id,
name=old_project.name,
path=old_project.path,
is_default=old_project.is_default or False,
),
new_project=None,
)
except ValueError as e: # pragma: no cover
raise HTTPException(status_code=400, detail=str(e))
# Set a project as default
@project_resource_router.put("/{name}/default", response_model=ProjectStatusResponse)
async def set_default_project(
project_service: ProjectServiceDep,
name: str = Path(..., description="Name of the project to set as default"),
) -> ProjectStatusResponse:
"""Set a project as the default project.
Args:
name: The name of the project to set as default
Returns:
Response confirming the project was set as default
"""
try:
# Get the old default project
default_name = project_service.default_project
default_project = await project_service.get_project(default_name)
if not default_project: # pragma: no cover
raise HTTPException( # pragma: no cover
status_code=404, detail=f"Default Project: '{default_name}' does not exist"
)
# get the new project
new_default_project = await project_service.get_project(name)
if not new_default_project: # pragma: no cover
raise HTTPException(
status_code=404, detail=f"Project: '{name}' does not exist"
) # pragma: no cover
await project_service.set_default_project(name)
return ProjectStatusResponse(
message=f"Project '{name}' set as default successfully",
status="success",
default=True,
old_project=ProjectItem(
id=default_project.id,
external_id=default_project.external_id,
name=default_name,
path=default_project.path,
is_default=False,
),
new_project=ProjectItem(
id=new_default_project.id,
external_id=new_default_project.external_id,
name=name,
path=new_default_project.path,
is_default=True,
),
)
except ValueError as e: # pragma: no cover
raise HTTPException(status_code=400, detail=str(e))
# Get the default project
@project_resource_router.get("/default", response_model=ProjectItem)
async def get_default_project(
project_service: ProjectServiceDep,
) -> ProjectItem:
"""Get the default project.
Returns:
Response with project default information
"""
# Get the default project
# In cloud mode, database is source of truth; in local mode, check config
config_default = project_service.default_project
db_default = await project_service.repository.get_default_project()
# Use database default if available, otherwise fall back to config default
default_name = db_default.name if db_default else config_default
default_project = await project_service.get_project(default_name)
if not default_project: # pragma: no cover
raise HTTPException( # pragma: no cover
status_code=404, detail=f"Default Project: '{default_name}' does not exist"
)
return ProjectItem(
id=default_project.id,
external_id=default_project.external_id,
name=default_project.name,
path=default_project.path,
is_default=True,
)
# Synchronize projects between config and database
@project_resource_router.post("/config/sync", response_model=ProjectStatusResponse)
async def synchronize_projects(
project_service: ProjectServiceDep,
) -> ProjectStatusResponse:
"""Synchronize projects between configuration file and database.
Ensures that all projects in the configuration file exist in the database
and vice versa.
Returns:
Response confirming synchronization was completed
"""
try: # pragma: no cover
await project_service.synchronize_projects()
return ProjectStatusResponse( # pyright: ignore [reportCallIssue]
message="Projects synchronized successfully between configuration and database",
status="success",
default=False,
)
except ValueError as e: # pragma: no cover
raise HTTPException(status_code=400, detail=str(e))
@@ -1,260 +0,0 @@
"""Router for prompt-related operations.
This router is responsible for rendering various prompts using Handlebars templates.
It centralizes all prompt formatting logic that was previously in the MCP prompts.
"""
from datetime import datetime, timezone
from fastapi import APIRouter, HTTPException, status
from loguru import logger
from basic_memory.api.routers.utils import to_graph_context, to_search_results
from basic_memory.api.template_loader import template_loader
from basic_memory.schemas.base import parse_timeframe
from basic_memory.deps import (
ContextServiceDep,
EntityRepositoryDep,
SearchServiceDep,
EntityServiceDep,
)
from basic_memory.schemas.prompt import (
ContinueConversationRequest,
SearchPromptRequest,
PromptResponse,
PromptMetadata,
)
from basic_memory.schemas.search import SearchItemType, SearchQuery
router = APIRouter(prefix="/prompt", tags=["prompt"])
@router.post("/continue-conversation", response_model=PromptResponse)
async def continue_conversation(
search_service: SearchServiceDep,
entity_service: EntityServiceDep,
context_service: ContextServiceDep,
entity_repository: EntityRepositoryDep,
request: ContinueConversationRequest,
) -> PromptResponse:
"""Generate a prompt for continuing a conversation.
This endpoint takes a topic and/or timeframe and generates a prompt with
relevant context from the knowledge base.
Args:
request: The request parameters
Returns:
Formatted continuation prompt with context
"""
logger.info(
f"Generating continue conversation prompt, topic: {request.topic}, timeframe: {request.timeframe}"
)
since = parse_timeframe(request.timeframe) if request.timeframe else None
# Initialize search results
search_results = []
# Get data needed for template
if request.topic:
query = SearchQuery(text=request.topic, after_date=request.timeframe)
results = await search_service.search(query, limit=request.search_items_limit)
search_results = await to_search_results(entity_service, results)
# Build context from results
all_hierarchical_results = []
for result in search_results:
if hasattr(result, "permalink") and result.permalink:
# Get hierarchical context using the new dataclass-based approach
context_result = await context_service.build_context(
result.permalink,
depth=request.depth,
since=since,
max_related=request.related_items_limit,
include_observations=True, # Include observations for entities
)
# Process results into the schema format
graph_context = await to_graph_context(
context_result, entity_repository=entity_repository
)
# Add results to our collection (limit to top results for each permalink)
if graph_context.results:
all_hierarchical_results.extend(graph_context.results[:3])
# Limit to a reasonable number of total results
all_hierarchical_results = all_hierarchical_results[:10]
template_context = {
"topic": request.topic,
"timeframe": request.timeframe,
"hierarchical_results": all_hierarchical_results,
"has_results": len(all_hierarchical_results) > 0,
}
else:
# If no topic, get recent activity
context_result = await context_service.build_context(
types=[SearchItemType.ENTITY],
depth=request.depth,
since=since,
max_related=request.related_items_limit,
include_observations=True,
)
recent_context = await to_graph_context(context_result, entity_repository=entity_repository)
hierarchical_results = recent_context.results[:5] # Limit to top 5 recent items
template_context = {
"topic": f"Recent Activity from ({request.timeframe})",
"timeframe": request.timeframe,
"hierarchical_results": hierarchical_results,
"has_results": len(hierarchical_results) > 0,
}
try:
# Render template
rendered_prompt = await template_loader.render(
"prompts/continue_conversation.hbs", template_context
)
# Calculate metadata
# Count items of different types
observation_count = 0
relation_count = 0
entity_count = 0
# Get the hierarchical results from the template context
hierarchical_results_for_count = template_context.get("hierarchical_results", [])
# For topic-based search
if request.topic:
for item in hierarchical_results_for_count:
if hasattr(item, "observations"):
observation_count += len(item.observations) if item.observations else 0
if hasattr(item, "related_results"):
for related in item.related_results or []:
if hasattr(related, "type"):
if related.type == "relation":
relation_count += 1
elif related.type == "entity": # pragma: no cover
entity_count += 1 # pragma: no cover
# For recent activity
else:
for item in hierarchical_results_for_count:
if hasattr(item, "observations"):
observation_count += len(item.observations) if item.observations else 0
if hasattr(item, "related_results"):
for related in item.related_results or []:
if hasattr(related, "type"):
if related.type == "relation":
relation_count += 1
elif related.type == "entity": # pragma: no cover
entity_count += 1 # pragma: no cover
# Build metadata
metadata = {
"query": request.topic,
"timeframe": request.timeframe,
"search_count": len(search_results)
if request.topic
else 0, # Original search results count
"context_count": len(hierarchical_results_for_count),
"observation_count": observation_count,
"relation_count": relation_count,
"total_items": (
len(hierarchical_results_for_count)
+ observation_count
+ relation_count
+ entity_count
),
"search_limit": request.search_items_limit,
"context_depth": request.depth,
"related_limit": request.related_items_limit,
"generated_at": datetime.now(timezone.utc).isoformat(),
}
prompt_metadata = PromptMetadata(**metadata)
return PromptResponse(
prompt=rendered_prompt, context=template_context, metadata=prompt_metadata
)
except Exception as e:
logger.error(f"Error rendering continue conversation template: {e}")
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail=f"Error rendering prompt template: {str(e)}",
)
@router.post("/search", response_model=PromptResponse)
async def search_prompt(
search_service: SearchServiceDep,
entity_service: EntityServiceDep,
request: SearchPromptRequest,
page: int = 1,
page_size: int = 10,
) -> PromptResponse:
"""Generate a prompt for search results.
This endpoint takes a search query and formats the results into a helpful
prompt with context and suggestions.
Args:
request: The search parameters
page: The page number for pagination
page_size: The number of results per page, defaults to 10
Returns:
Formatted search results prompt with context
"""
logger.info(f"Generating search prompt, query: {request.query}, timeframe: {request.timeframe}")
limit = page_size
offset = (page - 1) * page_size
query = SearchQuery(text=request.query, after_date=request.timeframe)
results = await search_service.search(query, limit=limit, offset=offset)
search_results = await to_search_results(entity_service, results)
template_context = {
"query": request.query,
"timeframe": request.timeframe,
"results": search_results,
"has_results": len(search_results) > 0,
"result_count": len(search_results),
}
try:
# Render template
rendered_prompt = await template_loader.render("prompts/search.hbs", template_context)
# Build metadata
metadata = {
"query": request.query,
"timeframe": request.timeframe,
"search_count": len(search_results),
"context_count": len(search_results),
"observation_count": 0, # Search results don't include observations
"relation_count": 0, # Search results don't include relations
"total_items": len(search_results),
"search_limit": limit,
"context_depth": 0, # No context depth for basic search
"related_limit": 0, # No related items for basic search
"generated_at": datetime.now(timezone.utc).isoformat(),
}
prompt_metadata = PromptMetadata(**metadata)
return PromptResponse(
prompt=rendered_prompt, context=template_context, metadata=prompt_metadata
)
except Exception as e:
logger.error(f"Error rendering search template: {e}")
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail=f"Error rendering prompt template: {str(e)}",
)
@@ -1,252 +0,0 @@
"""Routes for getting entity content."""
import tempfile
import uuid
from pathlib import Path
from typing import Annotated, Union
from fastapi import APIRouter, HTTPException, BackgroundTasks, Body, Response
from fastapi.responses import FileResponse, JSONResponse
from loguru import logger
from basic_memory.deps import (
ProjectConfigDep,
LinkResolverDep,
SearchServiceDep,
EntityServiceDep,
FileServiceDep,
EntityRepositoryDep,
)
from basic_memory.repository.search_repository import SearchIndexRow
from basic_memory.schemas.memory import normalize_memory_url
from basic_memory.schemas.search import SearchQuery, SearchItemType
from basic_memory.models.knowledge import Entity as EntityModel
from datetime import datetime
router = APIRouter(prefix="/resource", tags=["resources"])
def _mtime_to_datetime(entity: EntityModel) -> datetime:
"""Convert entity mtime (file modification time) to datetime.
Returns the file's actual modification time, falling back to updated_at
if mtime is not available.
"""
if entity.mtime: # pragma: no cover
return datetime.fromtimestamp(entity.mtime).astimezone() # pragma: no cover
return entity.updated_at
def get_entity_ids(item: SearchIndexRow) -> set[int]:
match item.type:
case SearchItemType.ENTITY:
return {item.id}
case SearchItemType.OBSERVATION:
return {item.entity_id} # pyright: ignore [reportReturnType]
case SearchItemType.RELATION:
from_entity = item.from_id
to_entity = item.to_id # pyright: ignore [reportReturnType]
return {from_entity, to_entity} if to_entity else {from_entity} # pyright: ignore [reportReturnType]
case _: # pragma: no cover
raise ValueError(f"Unexpected type: {item.type}")
@router.get("/{identifier:path}", response_model=None)
async def get_resource_content(
config: ProjectConfigDep,
link_resolver: LinkResolverDep,
search_service: SearchServiceDep,
entity_service: EntityServiceDep,
file_service: FileServiceDep,
background_tasks: BackgroundTasks,
identifier: str,
page: int = 1,
page_size: int = 10,
) -> Union[Response, FileResponse]:
"""Get resource content by identifier: name or permalink."""
logger.debug(f"Getting content for: {identifier}")
# Find single entity by permalink
entity = await link_resolver.resolve_link(identifier)
results = [entity] if entity else []
# pagination for multiple results
limit = page_size
offset = (page - 1) * page_size
# search using the identifier as a permalink
if not results:
# if the identifier contains a wildcard, use GLOB search
query = (
SearchQuery(permalink_match=identifier)
if "*" in identifier
else SearchQuery(permalink=identifier)
)
search_results = await search_service.search(query, limit, offset)
if not search_results:
raise HTTPException(status_code=404, detail=f"Resource not found: {identifier}")
# get the deduplicated entities related to the search results
entity_ids = {id for result in search_results for id in get_entity_ids(result)}
results = await entity_service.get_entities_by_id(list(entity_ids))
# return single response
if len(results) == 1:
entity = results[0]
# Check file exists via file_service (for cloud compatibility)
if not await file_service.exists(entity.file_path):
raise HTTPException(
status_code=404,
detail=f"File not found: {entity.file_path}",
)
# Read content via file_service as bytes (works with both local and S3)
content = await file_service.read_file_bytes(entity.file_path)
content_type = file_service.content_type(entity.file_path)
return Response(content=content, media_type=content_type)
# for multiple files, initialize a temporary file for writing the results
with tempfile.NamedTemporaryFile(delete=False, mode="w", suffix=".md") as tmp_file:
temp_file_path = tmp_file.name
for result in results:
# Read content for each entity
content = await file_service.read_entity_content(result)
memory_url = normalize_memory_url(result.permalink)
modified_date = _mtime_to_datetime(result).isoformat()
checksum = result.checksum[:8] if result.checksum else ""
# Prepare the delimited content
response_content = f"--- {memory_url} {modified_date} {checksum}\n"
response_content += f"\n{content}\n"
response_content += "\n"
# Write content directly to the temporary file in append mode
tmp_file.write(response_content)
# Ensure all content is written to disk
tmp_file.flush()
# Schedule the temporary file to be deleted after the response
background_tasks.add_task(cleanup_temp_file, temp_file_path)
# Return the file response
return FileResponse(path=temp_file_path)
def cleanup_temp_file(file_path: str):
"""Delete the temporary file."""
try:
Path(file_path).unlink() # Deletes the file
logger.debug(f"Temporary file deleted: {file_path}")
except Exception as e: # pragma: no cover
logger.error(f"Error deleting temporary file {file_path}: {e}")
@router.put("/{file_path:path}")
async def write_resource(
config: ProjectConfigDep,
file_service: FileServiceDep,
entity_repository: EntityRepositoryDep,
search_service: SearchServiceDep,
file_path: str,
content: Annotated[str, Body()],
) -> JSONResponse:
"""Write content to a file in the project.
This endpoint allows writing content directly to a file in the project.
Also creates an entity record and indexes the file for search.
Args:
file_path: Path to write to, relative to project root
request: Contains the content to write
Returns:
JSON response with file information
"""
try:
# Get content from request body
# Defensive type checking: ensure content is a string
# FastAPI should validate this, but if a dict somehow gets through
# (e.g., via JSON body parsing), we need to catch it here
if isinstance(content, dict):
logger.error( # pragma: no cover
f"Error writing resource {file_path}: "
f"content is a dict, expected string. Keys: {list(content.keys())}"
)
raise HTTPException( # pragma: no cover
status_code=400,
detail="content must be a string, not a dict. "
"Ensure request body is sent as raw string content, not JSON object.",
)
# Ensure it's UTF-8 string content
if isinstance(content, bytes): # pragma: no cover
content_str = content.decode("utf-8")
else:
content_str = str(content)
# Cloud compatibility: do not assume a local filesystem path structure.
# Delegate directory creation + writes to the configured FileService (local or S3).
await file_service.ensure_directory(Path(file_path).parent)
checksum = await file_service.write_file(file_path, content_str)
# Get file info
file_metadata = await file_service.get_file_metadata(file_path)
# Determine file details
file_name = Path(file_path).name
content_type = file_service.content_type(file_path)
entity_type = "canvas" if file_path.endswith(".canvas") else "file"
# Check if entity already exists
existing_entity = await entity_repository.get_by_file_path(file_path)
if existing_entity:
# Update existing entity
entity = await entity_repository.update(
existing_entity.id,
{
"title": file_name,
"entity_type": entity_type,
"content_type": content_type,
"file_path": file_path,
"checksum": checksum,
"updated_at": file_metadata.modified_at,
},
)
status_code = 200
else:
# Create a new entity model
# Explicitly set external_id to ensure NOT NULL constraint is satisfied (fixes #512)
entity = EntityModel(
external_id=str(uuid.uuid4()),
title=file_name,
entity_type=entity_type,
content_type=content_type,
file_path=file_path,
checksum=checksum,
created_at=file_metadata.created_at,
updated_at=file_metadata.modified_at,
)
entity = await entity_repository.add(entity)
status_code = 201
# Index the file for search
await search_service.index_entity(entity) # pyright: ignore
# Return success response
return JSONResponse(
status_code=status_code,
content={
"file_path": file_path,
"checksum": checksum,
"size": file_metadata.size,
"created_at": file_metadata.created_at.timestamp(),
"modified_at": file_metadata.modified_at.timestamp(),
},
)
except Exception as e: # pragma: no cover
logger.error(f"Error writing resource {file_path}: {e}")
raise HTTPException(status_code=500, detail=f"Failed to write resource: {str(e)}")
@@ -1,36 +0,0 @@
"""Router for search operations."""
from fastapi import APIRouter, BackgroundTasks
from basic_memory.api.routers.utils import to_search_results
from basic_memory.schemas.search import SearchQuery, SearchResponse
from basic_memory.deps import SearchServiceDep, EntityServiceDep
router = APIRouter(prefix="/search", tags=["search"])
@router.post("/", response_model=SearchResponse)
async def search(
query: SearchQuery,
search_service: SearchServiceDep,
entity_service: EntityServiceDep,
page: int = 1,
page_size: int = 10,
):
"""Search across all knowledge and documents."""
limit = page_size
offset = (page - 1) * page_size
results = await search_service.search(query, limit=limit, offset=offset)
search_results = await to_search_results(entity_service, results)
return SearchResponse(
results=search_results,
current_page=page,
page_size=page_size,
)
@router.post("/reindex")
async def reindex(background_tasks: BackgroundTasks, search_service: SearchServiceDep):
"""Recreate and populate the search index."""
await search_service.reindex_all(background_tasks=background_tasks)
return {"status": "ok", "message": "Reindex initiated"}
@@ -10,7 +10,7 @@ Key improvements:
- Simplified caching strategies
"""
from fastapi import APIRouter, HTTPException, BackgroundTasks, Depends, Response, Path
from fastapi import APIRouter, HTTPException, BackgroundTasks, Depends, Response, Path, Query
from loguru import logger
from basic_memory.deps import (
@@ -19,9 +19,10 @@ from basic_memory.deps import (
LinkResolverV2ExternalDep,
ProjectConfigV2ExternalDep,
AppConfigDep,
SyncServiceV2ExternalDep,
EntityRepositoryV2ExternalDep,
ProjectExternalIdPathDep,
TaskSchedulerDep,
FileServiceV2ExternalDep,
)
from basic_memory.schemas import DeleteEntitiesResponse
from basic_memory.schemas.base import Entity
@@ -38,26 +39,6 @@ from basic_memory.schemas.response import DirectoryMoveResult, DirectoryDeleteRe
router = APIRouter(prefix="/knowledge", tags=["knowledge-v2"])
async def resolve_relations_background(sync_service, entity_id: int, entity_permalink: str) -> None:
"""Background task to resolve relations for a specific entity.
This runs asynchronously after the API response is sent, preventing
long delays when creating entities with many relations.
"""
try: # pragma: no cover
# Only resolve relations for the newly created entity
await sync_service.resolve_relations(entity_id=entity_id) # pragma: no cover
logger.debug( # pragma: no cover
f"Background: Resolved relations for entity {entity_permalink} (id={entity_id})"
)
except Exception as e: # pragma: no cover
# Log but don't fail - this is a background task
logger.warning( # pragma: no cover
f"Background: Failed to resolve relations for entity {entity_permalink}: {e}"
)
## Resolution endpoint
@@ -186,24 +167,43 @@ async def create_entity(
background_tasks: BackgroundTasks,
entity_service: EntityServiceV2ExternalDep,
search_service: SearchServiceV2ExternalDep,
task_scheduler: TaskSchedulerDep,
file_service: FileServiceV2ExternalDep,
fast: bool = Query(
True, description="If true, write quickly and defer indexing to background tasks."
),
) -> EntityResponseV2:
"""Create a new entity.
Args:
data: Entity data to create
fast: If True, defer indexing to background tasks
Returns:
Created entity with generated external_id (UUID)
Created entity with generated external_id (UUID) and file content
"""
logger.info(
"API v2 request", endpoint="create_entity", entity_type=data.entity_type, title=data.title
)
entity = await entity_service.create_entity(data)
if fast:
entity = await entity_service.fast_write_entity(data)
task_scheduler.schedule(
"reindex_entity",
entity_id=entity.id,
project_id=project_id,
)
else:
entity = await entity_service.create_entity(data)
await search_service.index_entity(entity, background_tasks=background_tasks)
# reindex
await search_service.index_entity(entity, background_tasks=background_tasks)
result = EntityResponseV2.model_validate(entity)
if fast:
result = result.model_copy(update={"observations": [], "relations": []})
# Always read and return file content
content = await file_service.read_file_content(entity.file_path)
result = result.model_copy(update={"content": content})
logger.info(
f"API v2 response: endpoint='create_entity' external_id={entity.external_id}, title={result.title}, permalink={result.permalink}, status_code=201"
@@ -222,9 +222,13 @@ async def update_entity_by_id(
project_id: ProjectExternalIdPathDep,
entity_service: EntityServiceV2ExternalDep,
search_service: SearchServiceV2ExternalDep,
sync_service: SyncServiceV2ExternalDep,
entity_repository: EntityRepositoryV2ExternalDep,
task_scheduler: TaskSchedulerDep,
file_service: FileServiceV2ExternalDep,
entity_id: str = Path(..., description="Entity external ID (UUID)"),
fast: bool = Query(
True, description="If true, write quickly and defer indexing to background tasks."
),
) -> EntityResponseV2:
"""Update an entity by external ID.
@@ -233,30 +237,55 @@ async def update_entity_by_id(
Args:
entity_id: External ID (UUID string)
data: Updated entity data
fast: If True, defer indexing to background tasks
Returns:
Updated entity
Updated entity with file content
"""
logger.info(f"API v2 request: update_entity_by_id entity_id={entity_id}")
# Check if entity exists
# Check if entity exists (external_id is the source of truth for v2)
existing = await entity_repository.get_by_external_id(entity_id)
created = existing is None
# Perform update or create
entity, _ = await entity_service.create_or_update_entity(data)
response.status_code = 201 if created else 200
# reindex
await search_service.index_entity(entity, background_tasks=background_tasks)
# Schedule relation resolution for new entities
if created:
background_tasks.add_task( # pragma: no cover
resolve_relations_background, sync_service, entity.id, entity.permalink or ""
if fast:
entity = await entity_service.fast_write_entity(data, external_id=entity_id)
response.status_code = 200 if existing else 201
task_scheduler.schedule(
"reindex_entity",
entity_id=entity.id,
project_id=project_id,
resolve_relations=created,
)
else:
if existing:
# Update the existing entity in-place to avoid path-based duplication
entity = await entity_service.update_entity(existing, data)
response.status_code = 200
else:
# Create new entity, then bind external_id to the requested UUID
entity = await entity_service.create_entity(data)
if entity.external_id != entity_id:
entity = await entity_repository.update(
entity.id,
{"external_id": entity_id},
)
if not entity:
raise HTTPException(
status_code=404,
detail=f"Entity with external_id '{entity_id}' not found",
)
response.status_code = 201
await search_service.index_entity(entity, background_tasks=background_tasks)
result = EntityResponseV2.model_validate(entity)
if fast:
result = result.model_copy(update={"observations": [], "relations": []})
# Always read and return file content
content = await file_service.read_file_content(entity.file_path)
result = result.model_copy(update={"content": content})
logger.info(
f"API v2 response: external_id={entity_id}, created={created}, status_code={response.status_code}"
@@ -272,16 +301,22 @@ async def edit_entity_by_id(
entity_service: EntityServiceV2ExternalDep,
search_service: SearchServiceV2ExternalDep,
entity_repository: EntityRepositoryV2ExternalDep,
task_scheduler: TaskSchedulerDep,
file_service: FileServiceV2ExternalDep,
entity_id: str = Path(..., description="Entity external ID (UUID)"),
fast: bool = Query(
True, description="If true, write quickly and defer indexing to background tasks."
),
) -> EntityResponseV2:
"""Edit an existing entity by external ID using operations like append, prepend, etc.
Args:
entity_id: External ID (UUID string)
data: Edit operation details
fast: If True, defer indexing to background tasks
Returns:
Updated entity
Updated entity with file content
Raises:
HTTPException: 404 if entity not found, 400 if edit fails
@@ -298,21 +333,41 @@ async def edit_entity_by_id(
)
try:
# Edit using the entity's permalink or path
identifier = entity.permalink or entity.file_path
updated_entity = await entity_service.edit_entity(
identifier=identifier,
operation=data.operation,
content=data.content,
section=data.section,
find_text=data.find_text,
expected_replacements=data.expected_replacements,
)
if fast:
updated_entity = await entity_service.fast_edit_entity(
entity=entity,
operation=data.operation,
content=data.content,
section=data.section,
find_text=data.find_text,
expected_replacements=data.expected_replacements,
)
task_scheduler.schedule(
"reindex_entity",
entity_id=updated_entity.id,
project_id=project_id,
)
else:
# Edit using the entity's permalink or path
identifier = entity.permalink or entity.file_path
updated_entity = await entity_service.edit_entity(
identifier=identifier,
operation=data.operation,
content=data.content,
section=data.section,
find_text=data.find_text,
expected_replacements=data.expected_replacements,
)
# Reindex
await search_service.index_entity(updated_entity, background_tasks=background_tasks)
await search_service.index_entity(updated_entity, background_tasks=background_tasks)
result = EntityResponseV2.model_validate(updated_entity)
if fast:
result = result.model_copy(update={"observations": [], "relations": []})
# Always read and return file content
content = await file_service.read_file_content(updated_entity.file_path)
result = result.model_copy(update={"content": content})
logger.info(
f"API v2 response: external_id={entity_id}, operation='{data.operation}', status_code=200"
@@ -16,7 +16,7 @@ from basic_memory.schemas.memory import (
normalize_memory_url,
)
from basic_memory.schemas.search import SearchItemType
from basic_memory.api.routers.utils import to_graph_context
from basic_memory.api.v2.utils import to_graph_context
# Note: No prefix here - it's added during registration as /v2/{project_id}/memory
router = APIRouter(tags=["memory"])
@@ -19,9 +19,17 @@ from loguru import logger
from basic_memory.deps import (
ProjectServiceDep,
ProjectRepositoryDep,
ProjectConfigV2ExternalDep,
SyncServiceV2ExternalDep,
TaskSchedulerDep,
ProjectExternalIdPathDep,
)
from basic_memory.schemas import SyncReportResponse
from basic_memory.schemas.project_info import (
ProjectItem,
ProjectList,
ProjectInfoRequest,
ProjectInfoResponse,
ProjectStatusResponse,
)
from basic_memory.schemas.v2 import ProjectResolveRequest, ProjectResolveResponse
@@ -30,6 +38,175 @@ from basic_memory.utils import normalize_project_path, generate_permalink
router = APIRouter(prefix="/projects", tags=["project_management-v2"])
@router.get("/", response_model=ProjectList)
async def list_projects(
project_service: ProjectServiceDep,
) -> ProjectList:
"""List all configured projects.
Returns:
A list of all projects with metadata
"""
projects = await project_service.list_projects()
default_project = project_service.default_project
project_items = [
ProjectItem(
id=project.id,
external_id=project.external_id,
name=project.name,
path=normalize_project_path(project.path),
is_default=project.is_default or False,
)
for project in projects
]
return ProjectList(
projects=project_items,
default_project=default_project,
)
@router.post("/", response_model=ProjectStatusResponse, status_code=201)
async def add_project(
project_data: ProjectInfoRequest,
project_service: ProjectServiceDep,
) -> ProjectStatusResponse:
"""Add a new project to configuration and database.
Args:
project_data: The project name and path, with option to set as default
Returns:
Response confirming the project was added
"""
# Check if project already exists before attempting to add
existing_project = await project_service.get_project(project_data.name)
if existing_project:
# Project exists - check if paths match for true idempotency
# Normalize paths for comparison (resolve symlinks, etc.)
requested_path = os.path.abspath(os.path.expanduser(project_data.path))
existing_path = os.path.abspath(os.path.expanduser(existing_project.path))
if requested_path == existing_path:
# Same name, same path - return 200 OK (idempotent)
return ProjectStatusResponse( # pyright: ignore [reportCallIssue]
message=f"Project '{project_data.name}' already exists",
status="success",
default=existing_project.is_default or False,
new_project=ProjectItem(
id=existing_project.id,
external_id=existing_project.external_id,
name=existing_project.name,
path=existing_project.path,
is_default=existing_project.is_default or False,
),
)
else:
# Same name, different path - this is an error
raise HTTPException(
status_code=400,
detail=(
f"Project '{project_data.name}' already exists with different path. "
f"Existing: {existing_project.path}, Requested: {project_data.path}"
),
)
try: # pragma: no cover
# The service layer handles cloud mode validation and path sanitization
await project_service.add_project(
project_data.name, project_data.path, set_default=project_data.set_default
)
# Fetch the newly created project to get its ID
new_project = await project_service.get_project(project_data.name)
if not new_project:
raise HTTPException(status_code=500, detail="Failed to retrieve newly created project")
return ProjectStatusResponse( # pyright: ignore [reportCallIssue]
message=f"Project '{new_project.name}' added successfully",
status="success",
default=project_data.set_default,
new_project=ProjectItem(
id=new_project.id,
external_id=new_project.external_id,
name=new_project.name,
path=new_project.path,
is_default=new_project.is_default or False,
),
)
except ValueError as e: # pragma: no cover
raise HTTPException(status_code=400, detail=str(e))
@router.post("/config/sync", response_model=ProjectStatusResponse)
async def synchronize_projects(
project_service: ProjectServiceDep,
) -> ProjectStatusResponse:
"""Synchronize projects between configuration file and database."""
try: # pragma: no cover
await project_service.synchronize_projects()
return ProjectStatusResponse( # pyright: ignore [reportCallIssue]
message="Projects synchronized successfully between configuration and database",
status="success",
default=False,
)
except ValueError as e: # pragma: no cover
raise HTTPException(status_code=400, detail=str(e))
@router.post("/{project_id}/sync")
async def sync_project(
sync_service: SyncServiceV2ExternalDep,
project_config: ProjectConfigV2ExternalDep,
task_scheduler: TaskSchedulerDep,
project_internal_id: ProjectExternalIdPathDep,
force_full: bool = Query(
False, description="Force full scan, bypassing watermark optimization"
),
run_in_background: bool = Query(True, description="Run in background"),
):
"""Force project filesystem sync to database."""
if run_in_background:
task_scheduler.schedule(
"sync_project",
project_id=project_internal_id,
force_full=force_full,
)
logger.info(
f"Filesystem sync initiated for project: {project_config.name} (force_full={force_full})"
)
return {
"status": "sync_started",
"message": f"Filesystem sync initiated for project '{project_config.name}'",
}
report = await sync_service.sync(
project_config.home, project_config.name, force_full=force_full
)
logger.info(
f"Filesystem sync completed for project: {project_config.name} (force_full={force_full})"
)
return SyncReportResponse.from_sync_report(report)
@router.post("/{project_id}/status", response_model=SyncReportResponse)
async def get_project_status(
sync_service: SyncServiceV2ExternalDep,
project_config: ProjectConfigV2ExternalDep,
project_id: str = Path(..., description="Project external ID (UUID)"),
force_full: bool = Query(
False, description="Force full scan, bypassing watermark optimization"
),
) -> SyncReportResponse:
"""Get sync status of files vs database for a project."""
logger.info(f"API v2 request: get_project_status for project_id={project_id}")
report = await sync_service.scan(project_config.home, force_full=force_full)
return SyncReportResponse.from_sync_report(report)
@router.post("/resolve", response_model=ProjectResolveResponse)
async def resolve_project_identifier(
data: ProjectResolveRequest,
@@ -147,6 +324,22 @@ async def get_project_by_id(
)
@router.get("/{project_id}/info", response_model=ProjectInfoResponse)
async def get_project_info_by_id(
project_service: ProjectServiceDep,
project_repository: ProjectRepositoryDep,
project_id: str = Path(..., description="Project external ID (UUID)"),
) -> ProjectInfoResponse:
"""Get detailed project information by external ID."""
logger.info(f"API v2 request: get_project_info_by_id for project_id={project_id}")
project = await project_repository.get_by_external_id(project_id)
if not project:
raise HTTPException(
status_code=404, detail=f"Project with external_id '{project_id}' not found"
)
return await project_service.get_project_info(project.name)
@router.patch("/{project_id}", response_model=ProjectStatusResponse)
async def update_project_by_id(
project_service: ProjectServiceDep,
@@ -9,7 +9,7 @@ from datetime import datetime, timezone
from fastapi import APIRouter, HTTPException, status, Path
from loguru import logger
from basic_memory.api.routers.utils import to_graph_context, to_search_results
from basic_memory.api.v2.utils import to_graph_context, to_search_results
from basic_memory.api.template_loader import template_loader
from basic_memory.schemas.base import parse_timeframe
from basic_memory.deps import (
@@ -4,11 +4,16 @@ This router uses external_id UUIDs for stable, API-friendly routing.
V1 uses string-based project names which are less efficient and less stable.
"""
from fastapi import APIRouter, BackgroundTasks, Path
from fastapi import APIRouter, Path
from basic_memory.api.routers.utils import to_search_results
from basic_memory.api.v2.utils import to_search_results
from basic_memory.schemas.search import SearchQuery, SearchResponse
from basic_memory.deps import SearchServiceV2ExternalDep, EntityServiceV2ExternalDep
from basic_memory.deps import (
SearchServiceV2ExternalDep,
EntityServiceV2ExternalDep,
TaskSchedulerDep,
ProjectExternalIdPathDep,
)
# Note: No prefix here - it's added during registration as /v2/{project_id}/search
router = APIRouter(tags=["search"])
@@ -51,9 +56,8 @@ async def search(
@router.post("/search/reindex")
async def reindex(
background_tasks: BackgroundTasks,
search_service: SearchServiceV2ExternalDep,
project_id: str = Path(..., description="Project external UUID"),
task_scheduler: TaskSchedulerDep,
project_id: ProjectExternalIdPathDep,
):
"""Recreate and populate the search index for a project.
@@ -63,11 +67,10 @@ async def reindex(
Args:
project_id: Project external UUID from URL path
background_tasks: FastAPI background tasks handler
search_service: Search service scoped to project
task_scheduler: Task scheduler for background work
Returns:
Status message indicating reindex has been initiated
"""
await search_service.reindex_all(background_tasks=background_tasks)
task_scheduler.schedule("reindex_project", project_id=project_id)
return {"status": "ok", "message": "Reindex initiated"}
@@ -24,29 +24,42 @@ async def to_graph_context(
page: Optional[int] = None,
page_size: Optional[int] = None,
):
# First pass: collect all entity IDs needed for relations
# First pass: collect all entity IDs needed for external_id lookup
# This includes: entity primary results, observation parent entities, relation from/to entities
entity_ids_needed: set[int] = set()
for context_item in context_result.results:
for item in (
[context_item.primary_result] + context_item.observations + context_item.related_results
):
if item.type == SearchItemType.RELATION:
if item.type == SearchItemType.ENTITY:
# Entity's own ID for its external_id
entity_ids_needed.add(item.id)
elif item.type == SearchItemType.OBSERVATION:
# Parent entity ID for entity_external_id
if item.entity_id: # pyright: ignore
entity_ids_needed.add(item.entity_id) # pyright: ignore
elif item.type == SearchItemType.RELATION:
# Source and target entity IDs for external_ids
if item.from_id: # pyright: ignore
entity_ids_needed.add(item.from_id) # pyright: ignore
if item.to_id:
entity_ids_needed.add(item.to_id)
# Batch fetch all entities at once
entity_lookup: dict[int, str] = {}
# Batch fetch all entities at once - get both title and external_id
entity_title_lookup: dict[int, str] = {}
entity_external_id_lookup: dict[int, str] = {}
if entity_ids_needed:
entities = await entity_repository.find_by_ids(list(entity_ids_needed))
entity_lookup = {e.id: e.title for e in entities}
for e in entities:
entity_title_lookup[e.id] = e.title
entity_external_id_lookup[e.id] = e.external_id
# Helper function to convert items to summaries
def to_summary(item: SearchIndexRow | ContextResultRow):
match item.type:
case SearchItemType.ENTITY:
return EntitySummary(
external_id=entity_external_id_lookup.get(item.id, ""),
entity_id=item.id,
title=item.title, # pyright: ignore
permalink=item.permalink,
@@ -55,10 +68,14 @@ async def to_graph_context(
created_at=item.created_at,
)
case SearchItemType.OBSERVATION:
entity_ext_id = None
if item.entity_id: # pyright: ignore
entity_ext_id = entity_external_id_lookup.get(item.entity_id) # pyright: ignore
return ObservationSummary(
observation_id=item.id,
entity_id=item.entity_id, # pyright: ignore
title=item.title, # pyright: ignore
entity_external_id=entity_ext_id,
title=entity_title_lookup.get(item.entity_id), # pyright: ignore
file_path=item.file_path,
category=item.category, # pyright: ignore
content=item.content, # pyright: ignore
@@ -66,8 +83,10 @@ async def to_graph_context(
created_at=item.created_at,
)
case SearchItemType.RELATION:
from_title = entity_lookup.get(item.from_id) if item.from_id else None # pyright: ignore
to_title = entity_lookup.get(item.to_id) if item.to_id else None
from_title = entity_title_lookup.get(item.from_id) if item.from_id else None # pyright: ignore
to_title = entity_title_lookup.get(item.to_id) if item.to_id else None
from_ext_id = entity_external_id_lookup.get(item.from_id) if item.from_id else None # pyright: ignore
to_ext_id = entity_external_id_lookup.get(item.to_id) if item.to_id else None
return RelationSummary(
relation_id=item.id,
entity_id=item.entity_id, # pyright: ignore
@@ -77,8 +96,10 @@ async def to_graph_context(
relation_type=item.relation_type, # pyright: ignore
from_entity=from_title,
from_entity_id=item.from_id, # pyright: ignore
from_entity_external_id=from_ext_id,
to_entity=to_title,
to_entity_id=item.to_id,
to_entity_external_id=to_ext_id,
created_at=item.created_at,
)
case _: # pragma: no cover
+1 -1
View File
@@ -50,7 +50,7 @@ def app_callback(
# Skip for 'mcp' command - it has its own lifespan that handles initialization
# Skip for API-using commands (status, sync, etc.) - they handle initialization via deps.py
# Skip for 'reset' command - it manages its own database lifecycle
skip_init_commands = {"mcp", "status", "sync", "project", "tool", "reset"}
skip_init_commands = {"doctor", "mcp", "status", "sync", "project", "tool", "reset"}
if (
not version
and ctx.invoked_subcommand is not None
+2 -1
View File
@@ -1,11 +1,12 @@
"""CLI commands for basic-memory."""
from . import status, db, import_memory_json, mcp, import_claude_conversations
from . import status, db, doctor, import_memory_json, mcp, import_claude_conversations
from . import import_claude_projects, import_chatgpt, tool, project, format
__all__ = [
"status",
"db",
"doctor",
"import_memory_json",
"mcp",
"import_claude_conversations",
@@ -30,7 +30,7 @@ async def fetch_cloud_projects(
config = config_manager.config
host_url = config.cloud_host.rstrip("/")
response = await api_request(method="GET", url=f"{host_url}/proxy/projects/projects")
response = await api_request(method="GET", url=f"{host_url}/proxy/v2/projects/")
return CloudProjectList.model_validate(response.json())
except Exception as e:
@@ -66,7 +66,7 @@ async def create_cloud_project(
response = await api_request(
method="POST",
url=f"{host_url}/proxy/projects/projects",
url=f"{host_url}/proxy/v2/projects/",
headers={"Content-Type": "application/json"},
json_data=project_data.model_dump(),
)
@@ -27,6 +27,16 @@ console = Console()
# Minimum rclone version for --create-empty-src-dirs support
MIN_RCLONE_VERSION_EMPTY_DIRS = (1, 64, 0)
# Tigris edge caching returns stale data for users outside the origin region (iad).
# --header is rclone's global flag that applies to ALL HTTP transactions (list, download,
# upload). This is critical because bisync starts with S3 ListObjectsV2, which is neither
# a download nor upload — so --header-download/--header-upload would miss list requests.
# See: https://www.tigrisdata.com/docs/objects/consistency/
TIGRIS_CONSISTENCY_HEADERS = [
"--header",
"X-Tigris-Consistent: true",
]
class RunResult(Protocol):
returncode: int
@@ -210,6 +220,7 @@ def project_sync(
"sync",
str(local_path),
remote_path,
*TIGRIS_CONSISTENCY_HEADERS,
"--filter-from",
str(filter_path),
]
@@ -279,6 +290,7 @@ def project_bisync(
"bisync",
str(local_path),
remote_path,
*TIGRIS_CONSISTENCY_HEADERS,
"--resilient",
"--conflict-resolve=newer",
"--max-delete=25",
@@ -354,6 +366,7 @@ def project_check(
"check",
str(local_path),
remote_path,
*TIGRIS_CONSISTENCY_HEADERS,
"--filter-from",
str(filter_path),
]
@@ -393,6 +406,6 @@ def project_ls(
if path:
remote_path = f"{remote_path}/{path}"
cmd = ["rclone", "ls", remote_path]
cmd = ["rclone", "ls", *TIGRIS_CONSISTENCY_HEADERS, remote_path]
result = run(cmd, capture_output=True, text=True, check=True)
return result.stdout.splitlines()
@@ -58,7 +58,7 @@ async def run_sync(
try:
async with get_client() as client:
project_item = await get_active_project(client, project, None)
url = f"{project_item.project_url}/project/sync"
url = f"/v2/projects/{project_item.external_id}/sync"
params = []
if force_full:
params.append("force_full=true")
@@ -92,7 +92,7 @@ async def get_project_info(project: str):
try:
async with get_client() as client:
project_item = await get_active_project(client, project, None)
response = await call_get(client, f"{project_item.project_url}/project/info")
response = await call_get(client, f"/v2/projects/{project_item.external_id}/info")
return ProjectInfoResponse.model_validate(response.json())
except (ToolError, ValueError) as e:
console.print(f"[red]Sync failed: {e}[/red]")
+153
View File
@@ -0,0 +1,153 @@
"""Doctor command for local consistency checks."""
from __future__ import annotations
import tempfile
import uuid
from pathlib import Path
from loguru import logger
from mcp.server.fastmcp.exceptions import ToolError
from rich.console import Console
import typer
from basic_memory.cli.app import app
from basic_memory.cli.commands.command_utils import run_with_cleanup
from basic_memory.cli.commands.routing import force_routing, validate_routing_flags
from basic_memory.markdown.entity_parser import EntityParser
from basic_memory.markdown.markdown_processor import MarkdownProcessor
from basic_memory.markdown.schemas import EntityFrontmatter, EntityMarkdown
from basic_memory.mcp.async_client import get_client
from basic_memory.mcp.clients import KnowledgeClient, ProjectClient, SearchClient
from basic_memory.mcp.tools.utils import call_post
from basic_memory.schemas.base import Entity
from basic_memory.schemas.project_info import ProjectInfoRequest
from basic_memory.schemas.search import SearchQuery
from basic_memory.schemas import SyncReportResponse
console = Console()
async def run_doctor() -> None:
"""Run local consistency checks for file <-> database flows."""
console.print("[blue]Running Basic Memory doctor checks...[/blue]")
project_name = f"doctor-{uuid.uuid4().hex[:8]}"
api_note_title = "Doctor API Note"
manual_note_title = "Doctor Manual Note"
manual_permalink = "doctor/manual-note"
with tempfile.TemporaryDirectory() as temp_dir:
temp_path = Path(temp_dir)
async with get_client() as client:
project_client = ProjectClient(client)
project_request = ProjectInfoRequest(
name=project_name,
path=str(temp_path),
set_default=False,
)
project_id: str | None = None
try:
status = await project_client.create_project(project_request.model_dump())
if not status.new_project:
raise ValueError("Failed to create doctor project")
project_id = status.new_project.external_id
console.print(f"[green]OK[/green] Created doctor project: {project_name}")
# --- DB -> File: create an entity via API ---
knowledge_client = KnowledgeClient(client, project_id)
api_note = Entity(
title=api_note_title,
directory="doctor",
entity_type="note",
content_type="text/markdown",
content=f"# {api_note_title}\n\n- [note] API to file check",
entity_metadata={"tags": ["doctor"]},
)
api_result = await knowledge_client.create_entity(api_note.model_dump(), fast=False)
api_file = temp_path / api_result.file_path
if not api_file.exists():
raise ValueError(f"API note file missing: {api_result.file_path}")
api_text = api_file.read_text(encoding="utf-8")
if api_note_title not in api_text:
raise ValueError("API note content missing from file")
console.print("[green]OK[/green] API write created file")
# --- File -> DB: write markdown file directly, then sync ---
parser = EntityParser(temp_path)
processor = MarkdownProcessor(parser)
manual_markdown = EntityMarkdown(
frontmatter=EntityFrontmatter(
metadata={
"title": manual_note_title,
"type": "note",
"permalink": manual_permalink,
"tags": ["doctor"],
}
),
content=f"# {manual_note_title}\n\n- [note] File to DB check",
)
manual_path = temp_path / "doctor" / "manual-note.md"
await processor.write_file(manual_path, manual_markdown)
console.print("[green]OK[/green] Manual file written")
sync_response = await call_post(
client,
f"/v2/projects/{project_id}/sync?force_full=true&run_in_background=false",
)
sync_report = SyncReportResponse.model_validate(sync_response.json())
if sync_report.total == 0:
raise ValueError("Sync did not detect any changes")
console.print("[green]OK[/green] Sync indexed manual file")
search_client = SearchClient(client, project_id)
search_query = SearchQuery(title=manual_note_title)
search_results = await search_client.search(
search_query.model_dump(), page=1, page_size=5
)
if not any(result.title == manual_note_title for result in search_results.results):
raise ValueError("Manual note not found in search index")
console.print("[green]OK[/green] Search confirmed manual file")
status_response = await call_post(client, f"/v2/projects/{project_id}/status")
status_report = SyncReportResponse.model_validate(status_response.json())
if status_report.total != 0:
raise ValueError("Project status not clean after sync")
console.print("[green]OK[/green] Status clean after sync")
finally:
if project_id:
await project_client.delete_project(project_id)
console.print("[green]Doctor checks passed.[/green]")
@app.command()
def doctor(
local: bool = typer.Option(
False, "--local", help="Force local API routing (ignore cloud mode)"
),
cloud: bool = typer.Option(False, "--cloud", help="Force cloud API routing"),
) -> None:
"""Run local consistency checks to verify file/database sync."""
try:
validate_routing_flags(local, cloud)
with force_routing(local=local, cloud=cloud):
run_with_cleanup(run_doctor())
except (ToolError, ValueError) as e:
console.print(f"[red]Doctor failed: {e}[/red]")
raise typer.Exit(code=1)
except Exception as e:
logger.error(f"Doctor failed: {e}")
typer.echo(f"Doctor failed: {e}", err=True)
raise typer.Exit(code=1) # pragma: no cover
+25 -17
View File
@@ -17,6 +17,7 @@ from basic_memory.config import ConfigManager
from basic_memory.mcp.async_client import get_client
from basic_memory.mcp.tools.utils import call_delete, call_get, call_patch, call_post, call_put
from basic_memory.schemas.project_info import ProjectList, ProjectStatusResponse
from basic_memory.schemas.v2 import ProjectResolveResponse
from basic_memory.utils import generate_permalink, normalize_project_path
# Import rclone commands for project sync
@@ -65,7 +66,7 @@ def list_projects(
async def _list_projects():
async with get_client() as client:
response = await call_get(client, "/projects/projects")
response = await call_get(client, "/v2/projects/")
return ProjectList.model_validate(response.json())
try:
@@ -167,7 +168,7 @@ def add_project(
"local_sync_path": local_sync_path,
"set_default": set_default,
}
response = await call_post(client, "/projects/projects", json=data)
response = await call_post(client, "/v2/projects/", json=data)
return ProjectStatusResponse.model_validate(response.json())
else:
# Local mode: path is required
@@ -181,7 +182,7 @@ def add_project(
async def _add_project():
async with get_client() as client:
data = {"name": name, "path": resolved_path, "set_default": set_default}
response = await call_post(client, "/projects/projects", json=data)
response = await call_post(client, "/v2/projects/", json=data)
return ProjectStatusResponse.model_validate(response.json())
try:
@@ -234,7 +235,7 @@ def setup_project_sync(
async def _verify_project_exists():
"""Verify the project exists on cloud by listing all projects."""
async with get_client() as client:
response = await call_get(client, "/projects/projects")
response = await call_get(client, "/v2/projects/")
project_list = response.json()
project_names = [p["name"] for p in project_list["projects"]]
if name not in project_names:
@@ -433,7 +434,7 @@ def synchronize_projects(
async def _sync_config():
async with get_client() as client:
response = await call_post(client, "/projects/config/sync")
response = await call_post(client, "/v2/projects/config/sync")
return ProjectStatusResponse.model_validate(response.json())
try:
@@ -475,10 +476,15 @@ def move_project(
async def _move_project():
async with get_client() as client:
data = {"path": resolved_path}
project_permalink = generate_permalink(name)
# TODO fix route to use ProjectPathDep
response = await call_patch(client, f"/{name}/project/{project_permalink}", json=data)
resolve_response = await call_post(
client,
"/v2/projects/resolve",
json={"identifier": name},
)
project_info = ProjectResolveResponse.model_validate(resolve_response.json())
response = await call_patch(
client, f"/v2/projects/{project_info.external_id}", json=data
)
return ProjectStatusResponse.model_validate(response.json())
try:
@@ -530,7 +536,7 @@ def sync_project_command(
# Get project info
async def _get_project():
async with get_client() as client:
response = await call_get(client, "/projects/projects")
response = await call_get(client, "/v2/projects/")
projects_list = ProjectList.model_validate(response.json())
for proj in projects_list.projects:
if generate_permalink(proj.name) == generate_permalink(name):
@@ -571,9 +577,10 @@ def sync_project_command(
async def _trigger_db_sync():
async with get_client() as client:
permalink = generate_permalink(name)
response = await call_post(
client, f"/{permalink}/project/sync?force_full=true", json={}
client,
f"/v2/projects/{project_data.external_id}/sync?force_full=true",
json={},
)
return response.json()
@@ -621,7 +628,7 @@ def bisync_project_command(
# Get project info
async def _get_project():
async with get_client() as client:
response = await call_get(client, "/projects/projects")
response = await call_get(client, "/v2/projects/")
projects_list = ProjectList.model_validate(response.json())
for proj in projects_list.projects:
if generate_permalink(proj.name) == generate_permalink(name):
@@ -669,9 +676,10 @@ def bisync_project_command(
async def _trigger_db_sync():
async with get_client() as client:
permalink = generate_permalink(name)
response = await call_post(
client, f"/{permalink}/project/sync?force_full=true", json={}
client,
f"/v2/projects/{project_data.external_id}/sync?force_full=true",
json={},
)
return response.json()
@@ -715,7 +723,7 @@ def check_project_command(
# Get project info
async def _get_project():
async with get_client() as client:
response = await call_get(client, "/projects/projects")
response = await call_get(client, "/v2/projects/")
projects_list = ProjectList.model_validate(response.json())
for proj in projects_list.projects:
if generate_permalink(proj.name) == generate_permalink(name):
@@ -816,7 +824,7 @@ def ls_project_command(
# Get project info
async def _get_project():
async with get_client() as client:
response = await call_get(client, "/projects/projects")
response = await call_get(client, "/v2/projects/")
projects_list = ProjectList.model_validate(response.json())
for proj in projects_list.projects:
if generate_permalink(proj.name) == generate_permalink(name):
+1 -1
View File
@@ -146,7 +146,7 @@ async def run_status(project: Optional[str] = None, verbose: bool = False): # p
try:
async with get_client() as client:
project_item = await get_active_project(client, project, None)
response = await call_post(client, f"{project_item.project_url}/project/status")
response = await call_post(client, f"/v2/projects/{project_item.external_id}/status")
sync_report = SyncReportResponse.model_validate(response.json())
display_changes(project_item.name, "Status", sync_report, verbose)
+248 -27
View File
@@ -1,5 +1,6 @@
"""CLI tool commands for Basic Memory."""
import json
import sys
from typing import Annotated, List, Optional
@@ -11,6 +12,13 @@ from basic_memory.cli.app import app
from basic_memory.cli.commands.command_utils import run_with_cleanup
from basic_memory.cli.commands.routing import force_routing, validate_routing_flags
from basic_memory.config import ConfigManager
from basic_memory.mcp.async_client import get_client
from basic_memory.mcp.clients import KnowledgeClient, ResourceClient
from basic_memory.mcp.project_context import get_active_project
from basic_memory.mcp.tools.utils import call_get
from basic_memory.schemas.base import Entity, TimeFrame
from basic_memory.schemas.memory import GraphContext, MemoryUrl, memory_url_path
from basic_memory.schemas.search import SearchItemType
# Import prompts
from basic_memory.mcp.prompts.continue_conversation import (
@@ -24,14 +32,128 @@ from basic_memory.mcp.tools import read_note as mcp_read_note
from basic_memory.mcp.tools import recent_activity as mcp_recent_activity
from basic_memory.mcp.tools import search_notes as mcp_search
from basic_memory.mcp.tools import write_note as mcp_write_note
from basic_memory.schemas.base import TimeFrame
from basic_memory.schemas.memory import MemoryUrl
from basic_memory.schemas.search import SearchItemType
tool_app = typer.Typer()
app.add_typer(tool_app, name="tool", help="Access to MCP tools via CLI")
# --- JSON output helpers ---
# These async functions bypass the MCP tool (which returns formatted strings)
# and use API clients directly to return structured data for --format json.
async def _write_note_json(
title: str, content: str, folder: str, project_name: Optional[str], tags: Optional[List[str]]
) -> dict:
"""Write a note and return structured JSON metadata."""
# Use the MCP tool to create/update the entity (handles create-or-update logic)
await mcp_write_note.fn(title, content, folder, project_name, tags)
# Resolve the entity to get metadata back
async with get_client() as client:
active_project = await get_active_project(client, project_name)
knowledge_client = KnowledgeClient(client, active_project.external_id)
entity = Entity(title=title, directory=folder)
if not entity.permalink:
raise ValueError(f"Could not generate permalink for title={title}, folder={folder}")
entity_id = await knowledge_client.resolve_entity(entity.permalink)
entity = await knowledge_client.get_entity(entity_id)
return {
"title": entity.title,
"permalink": entity.permalink,
"content": content,
"file_path": entity.file_path,
}
async def _read_note_json(
identifier: str, project_name: Optional[str], page: int, page_size: int
) -> dict:
"""Read a note and return structured JSON with content and metadata."""
async with get_client() as client:
active_project = await get_active_project(client, project_name)
knowledge_client = KnowledgeClient(client, active_project.external_id)
resource_client = ResourceClient(client, active_project.external_id)
# Try direct resolution first (works for permalinks and memory URLs)
entity_path = memory_url_path(identifier)
entity_id = None
try:
entity_id = await knowledge_client.resolve_entity(entity_path)
except Exception:
logger.info(f"Direct lookup failed for '{entity_path}', trying title search")
# Fallback: title search (handles plain titles like "My Note")
if entity_id is None:
from basic_memory.mcp.tools.search import search_notes as mcp_search_tool
title_results = await mcp_search_tool.fn(
query=identifier, search_type="title", project=project_name
)
if title_results and hasattr(title_results, "results") and title_results.results:
result = title_results.results[0]
if result.permalink:
entity_id = await knowledge_client.resolve_entity(result.permalink)
if entity_id is None:
raise ValueError(f"Could not find note matching: {identifier}")
entity = await knowledge_client.get_entity(entity_id)
response = await resource_client.read(entity_id, page=page, page_size=page_size)
return {
"title": entity.title,
"permalink": entity.permalink,
"content": response.text,
"file_path": entity.file_path,
}
async def _recent_activity_json(
type: Optional[List[SearchItemType]],
depth: Optional[int],
timeframe: Optional[TimeFrame],
project_name: Optional[str] = None,
page: int = 1,
page_size: int = 50,
) -> list:
"""Get recent activity and return structured JSON list."""
async with get_client() as client:
# Build query params matching the MCP tool's logic
params: dict = {"page": page, "page_size": page_size, "max_related": 10}
if depth:
params["depth"] = depth
if timeframe:
params["timeframe"] = timeframe
if type:
params["type"] = [t.value for t in type]
active_project = await get_active_project(client, project_name)
response = await call_get(
client,
f"/v2/projects/{active_project.external_id}/memory/recent",
params=params,
)
activity_data = GraphContext.model_validate(response.json())
# Extract entity results
results = []
for result in activity_data.results:
pr = result.primary_result
if pr.type == "entity":
results.append(
{
"title": pr.title,
"permalink": pr.permalink,
"file_path": pr.file_path,
"created_at": str(pr.created_at) if pr.created_at else None,
}
)
return results
@tool_app.command()
def write_note(
title: Annotated[str, typer.Option(help="The title of the note")],
@@ -51,6 +173,7 @@ def write_note(
tags: Annotated[
Optional[List[str]], typer.Option(help="A list of tags to apply to the note")
] = None,
format: str = typer.Option("text", "--format", help="Output format: text or json"),
local: bool = typer.Option(
False, "--local", help="Force local API routing (ignore cloud mode)"
),
@@ -122,9 +245,20 @@ def write_note(
# use the project name, or the default from the config
project_name = project_name or config_manager.default_project
# content is validated non-None above (stdin or --content)
assert content is not None
with force_routing(local=local, cloud=cloud):
note = run_with_cleanup(mcp_write_note.fn(title, content, folder, project_name, tags))
rprint(note)
if format == "json":
result = run_with_cleanup(
_write_note_json(title, content, folder, project_name, tags)
)
print(json.dumps(result, indent=2, ensure_ascii=True, default=str))
else:
note = run_with_cleanup(
mcp_write_note.fn(title, content, folder, project_name, tags)
)
rprint(note)
except ValueError as e:
typer.echo(f"Error: {e}", err=True)
raise typer.Exit(1)
@@ -146,6 +280,7 @@ def read_note(
] = None,
page: int = 1,
page_size: int = 10,
format: str = typer.Option("text", "--format", help="Output format: text or json"),
local: bool = typer.Option(
False, "--local", help="Force local API routing (ignore cloud mode)"
),
@@ -172,8 +307,14 @@ def read_note(
project_name = project_name or config_manager.default_project
with force_routing(local=local, cloud=cloud):
note = run_with_cleanup(mcp_read_note.fn(identifier, project_name, page, page_size))
rprint(note)
if format == "json":
result = run_with_cleanup(
_read_note_json(identifier, project_name, page, page_size)
)
print(json.dumps(result, indent=2, ensure_ascii=True, default=str))
else:
note = run_with_cleanup(mcp_read_note.fn(identifier, project_name, page, page_size))
rprint(note)
except ValueError as e:
typer.echo(f"Error: {e}", err=True)
raise typer.Exit(1)
@@ -196,6 +337,7 @@ def build_context(
page: int = 1,
page_size: int = 10,
max_related: int = 10,
format: str = typer.Option("json", "--format", help="Output format: text or json"),
local: bool = typer.Option(
False, "--local", help="Force local API routing (ignore cloud mode)"
),
@@ -233,9 +375,6 @@ def build_context(
max_related=max_related,
)
)
# Use json module for more controlled serialization
import json
context_dict = context.model_dump(exclude_none=True)
print(json.dumps(context_dict, indent=2, ensure_ascii=True, default=str))
except ValueError as e:
@@ -251,8 +390,17 @@ def build_context(
@tool_app.command()
def recent_activity(
type: Annotated[Optional[List[SearchItemType]], typer.Option()] = None,
project: Annotated[
Optional[str],
typer.Option(help="The project to use. If not provided, the default project will be used."),
] = None,
depth: Optional[int] = 1,
timeframe: Optional[TimeFrame] = "7d",
page: int = typer.Option(1, "--page", help="Page number for pagination (JSON format)"),
page_size: int = typer.Option(
50, "--page-size", help="Number of results per page (JSON format)"
),
format: str = typer.Option("text", "--format", help="Output format: text or json"),
local: bool = typer.Option(
False, "--local", help="Force local API routing (ignore cloud mode)"
),
@@ -266,16 +414,33 @@ def recent_activity(
try:
validate_routing_flags(local, cloud)
# Resolve project from config for JSON mode
config_manager = ConfigManager()
project_name = None
if project is not None:
project_name, _ = config_manager.get_project(project)
if not project_name:
typer.echo(f"No project found named: {project}", err=True)
raise typer.Exit(1)
project_name = project_name or config_manager.default_project
with force_routing(local=local, cloud=cloud):
result = run_with_cleanup(
mcp_recent_activity.fn(
type=type, # pyright: ignore [reportArgumentType]
depth=depth,
timeframe=timeframe,
if format == "json":
result = run_with_cleanup(
_recent_activity_json(type, depth, timeframe, project_name, page, page_size)
)
)
# The tool now returns a formatted string directly
print(result)
print(json.dumps(result, indent=2, ensure_ascii=True, default=str))
else:
result = run_with_cleanup(
mcp_recent_activity.fn(
type=type, # pyright: ignore [reportArgumentType]
depth=depth,
timeframe=timeframe,
project=project_name,
)
)
# The tool returns a formatted string directly
print(result)
except ValueError as e:
typer.echo(f"Error: {e}", err=True)
raise typer.Exit(1)
@@ -288,7 +453,10 @@ def recent_activity(
@tool_app.command("search-notes")
def search_notes(
query: str,
query: Annotated[
Optional[str],
typer.Argument(help="Search query string (optional when using metadata filters)"),
] = "",
permalink: Annotated[bool, typer.Option("--permalink", help="Search permalink values")] = False,
title: Annotated[bool, typer.Option("--title", help="Search title values")] = False,
project: Annotated[
@@ -301,6 +469,26 @@ def search_notes(
Optional[str],
typer.Option("--after_date", help="Search results after date, eg. '2d', '1 week'"),
] = None,
tags: Annotated[
Optional[List[str]],
typer.Option("--tag", help="Filter by frontmatter tag (repeatable)"),
] = None,
status: Annotated[
Optional[str],
typer.Option("--status", help="Filter by frontmatter status"),
] = None,
note_types: Annotated[
Optional[List[str]],
typer.Option("--type", help="Filter by frontmatter type (repeatable)"),
] = None,
meta: Annotated[
Optional[List[str]],
typer.Option("--meta", help="Filter by frontmatter key=value (repeatable)"),
] = None,
filter_json: Annotated[
Optional[str],
typer.Option("--filter", help="JSON metadata filter (advanced)"),
] = None,
page: int = 1,
page_size: int = 10,
local: bool = typer.Option(
@@ -335,26 +523,59 @@ def search_notes(
)
raise typer.Exit(1)
# Build metadata filters from --filter and --meta
metadata_filters = {}
if filter_json:
try:
metadata_filters = json.loads(filter_json)
if not isinstance(metadata_filters, dict):
raise ValueError("Metadata filter JSON must be an object")
except json.JSONDecodeError as e:
typer.echo(f"Invalid JSON for --filter: {e}", err=True)
raise typer.Exit(1)
if meta:
for item in meta:
if "=" not in item:
typer.echo(
f"Invalid --meta entry '{item}'. Use key=value format.",
err=True,
)
raise typer.Exit(1)
key, value = item.split("=", 1)
key = key.strip()
if not key:
typer.echo(f"Invalid --meta entry '{item}'.", err=True)
raise typer.Exit(1)
metadata_filters[key] = value
if not metadata_filters:
metadata_filters = None
# set search type
search_type = ("permalink" if permalink else None,)
search_type = ("permalink_match" if permalink and "*" in query else None,)
search_type = ("title" if title else None,)
search_type = "text" if search_type is None else search_type
search_type = "text"
if permalink:
search_type = "permalink"
if query and "*" in query:
search_type = "permalink"
if title:
search_type = "title"
with force_routing(local=local, cloud=cloud):
results = run_with_cleanup(
mcp_search.fn(
query,
query or "",
project_name,
search_type=search_type,
page=page,
after_date=after_date,
page_size=page_size,
types=note_types,
metadata_filters=metadata_filters,
tags=tags,
status=status,
)
)
# Use json module for more controlled serialization
import json
results_dict = results.model_dump(exclude_none=True)
print(json.dumps(results_dict, indent=2, ensure_ascii=True, default=str))
except ValueError as e:
+1
View File
@@ -6,6 +6,7 @@ from basic_memory.cli.app import app # pragma: no cover
from basic_memory.cli.commands import ( # noqa: F401 # pragma: no cover
cloud,
db,
doctor,
import_chatgpt,
import_claude_conversations,
import_claude_projects,
+4
View File
@@ -91,6 +91,8 @@ from basic_memory.deps.services import (
FileServiceV2Dep,
get_file_service_v2_external,
FileServiceV2ExternalDep,
get_task_scheduler,
TaskSchedulerDep,
get_search_service,
SearchServiceDep,
get_search_service_v2,
@@ -227,6 +229,8 @@ __all__ = [
"FileServiceV2Dep",
"get_file_service_v2_external",
"FileServiceV2ExternalDep",
"get_task_scheduler",
"TaskSchedulerDep",
"get_search_service",
"SearchServiceDep",
"get_search_service_v2",
+83 -2
View File
@@ -7,7 +7,8 @@ This module provides service-layer dependencies:
- SyncService, ProjectService, DirectoryService
"""
from typing import Annotated
import asyncio
from typing import Annotated, Any, Callable, Coroutine, Mapping, Protocol
from fastapi import Depends
from loguru import logger
@@ -43,7 +44,6 @@ from basic_memory.services.link_resolver import LinkResolver
from basic_memory.services.search_service import SearchService
from basic_memory.sync import SyncService
# --- Entity Parser ---
@@ -430,6 +430,87 @@ async def get_sync_service_v2_external(
SyncServiceV2ExternalDep = Annotated[SyncService, Depends(get_sync_service_v2_external)]
# --- Background Task Scheduler ---
class TaskScheduler(Protocol):
def schedule(self, task_name: str, **payload: Any) -> None:
"""Schedule a background task by name."""
def _log_task_failure(completed: asyncio.Task) -> None:
try:
completed.result()
except Exception as exc: # pragma: no cover
logger.exception("Background task failed", error=str(exc))
class LocalTaskScheduler:
"""Default scheduler that runs tasks in-process via asyncio.create_task."""
def __init__(
self,
handlers: Mapping[str, Callable[..., Coroutine[Any, Any, None]]],
) -> None:
self._handlers = handlers
def schedule(self, task_name: str, **payload: Any) -> None:
handler = self._handlers.get(task_name)
# Trigger: task name is not registered
# Why: avoid silently dropping background work
# Outcome: fail fast to surface misconfiguration
if not handler:
raise ValueError(f"Unknown task name: {task_name}")
task = asyncio.create_task(handler(**payload))
task.add_done_callback(_log_task_failure)
async def get_task_scheduler(
entity_service: EntityServiceV2ExternalDep,
sync_service: SyncServiceV2ExternalDep,
search_service: SearchServiceV2ExternalDep,
project_config: ProjectConfigV2ExternalDep,
) -> TaskScheduler:
"""Create a scheduler that maps task specs to coroutines."""
async def _reindex_entity(
entity_id: int,
resolve_relations: bool = False,
**_: Any,
) -> None:
await entity_service.reindex_entity(entity_id)
# Trigger: caller requests relation resolution
# Why: resolve forward references created before the entity existed
# Outcome: updates unresolved relations pointing to this entity
if resolve_relations:
await sync_service.resolve_relations(entity_id=entity_id)
async def _resolve_relations(entity_id: int, **_: Any) -> None:
await sync_service.resolve_relations(entity_id=entity_id)
async def _sync_project(force_full: bool = False, **_: Any) -> None:
await sync_service.sync(
project_config.home,
project_config.name,
force_full=force_full,
)
async def _reindex_project(**_: Any) -> None:
await search_service.reindex_all()
return LocalTaskScheduler(
{
"reindex_entity": _reindex_entity,
"resolve_relations": _resolve_relations,
"sync_project": _sync_project,
"reindex_project": _reindex_project,
}
)
TaskSchedulerDep = Annotated[TaskScheduler, Depends(get_task_scheduler)]
# --- Project Service ---
+3 -1
View File
@@ -241,7 +241,9 @@ class EntityParser:
f"Failed to parse YAML frontmatter in {file_path}: {e}. "
f"Treating file as plain markdown without frontmatter."
)
post = frontmatter.Post(content, metadata={})
# Use Post(content) not Post(content, metadata={})
# The latter creates {"metadata": {}} in the metadata dict (issue #528)
post = frontmatter.Post(content)
# Normalize frontmatter values
metadata = normalize_frontmatter_metadata(post.metadata)
+4 -3
View File
@@ -9,6 +9,7 @@ from frontmatter import Post
from basic_memory.file_utils import has_frontmatter, remove_frontmatter, parse_frontmatter
from basic_memory.markdown import EntityMarkdown
from basic_memory.markdown.entity_parser import normalize_frontmatter_metadata
from basic_memory.models import Entity
from basic_memory.models import Observation as ObservationModel
@@ -58,9 +59,9 @@ def entity_model_from_markdown(
model.created_at = markdown.created
model.updated_at = markdown.modified
# Handle metadata - ensure all values are strings and filter None
metadata = markdown.frontmatter.metadata or {}
model.entity_metadata = {k: str(v) for k, v in metadata.items() if v is not None}
# Handle metadata - normalize values and filter None (preserve structured data)
metadata = normalize_frontmatter_metadata(markdown.frontmatter.metadata or {})
model.entity_metadata = {k: v for k, v in metadata.items() if v is not None}
# Get project_id from entity if not provided
obs_project_id = project_id or (model.project_id if hasattr(model, "project_id") else None)
+23 -3
View File
@@ -43,7 +43,9 @@ class KnowledgeClient:
# --- Entity CRUD Operations ---
async def create_entity(self, entity_data: dict[str, Any]) -> EntityResponse:
async def create_entity(
self, entity_data: dict[str, Any], *, fast: bool | None = None
) -> EntityResponse:
"""Create a new entity.
Args:
@@ -55,14 +57,22 @@ class KnowledgeClient:
Raises:
ToolError: If the request fails
"""
params = {"fast": fast} if fast is not None else None
response = await call_post(
self.http_client,
f"{self._base_path}/entities",
json=entity_data,
params=params,
)
return EntityResponse.model_validate(response.json())
async def update_entity(self, entity_id: str, entity_data: dict[str, Any]) -> EntityResponse:
async def update_entity(
self,
entity_id: str,
entity_data: dict[str, Any],
*,
fast: bool | None = None,
) -> EntityResponse:
"""Update an existing entity (full replacement).
Args:
@@ -75,10 +85,12 @@ class KnowledgeClient:
Raises:
ToolError: If the request fails
"""
params = {"fast": fast} if fast is not None else None
response = await call_put(
self.http_client,
f"{self._base_path}/entities/{entity_id}",
json=entity_data,
params=params,
)
return EntityResponse.model_validate(response.json())
@@ -100,7 +112,13 @@ class KnowledgeClient:
)
return EntityResponse.model_validate(response.json())
async def patch_entity(self, entity_id: str, patch_data: dict[str, Any]) -> EntityResponse:
async def patch_entity(
self,
entity_id: str,
patch_data: dict[str, Any],
*,
fast: bool | None = None,
) -> EntityResponse:
"""Partially update an entity.
Args:
@@ -113,10 +131,12 @@ class KnowledgeClient:
Raises:
ToolError: If the request fails
"""
params = {"fast": fast} if fast is not None else None
response = await call_patch(
self.http_client,
f"{self._base_path}/entities/{entity_id}",
json=patch_data,
params=params,
)
return EntityResponse.model_validate(response.json())
+2 -2
View File
@@ -47,7 +47,7 @@ class ProjectClient:
"""
response = await call_get(
self.http_client,
"/projects/projects",
"/v2/projects/",
)
return ProjectList.model_validate(response.json())
@@ -65,7 +65,7 @@ class ProjectClient:
"""
response = await call_post(
self.http_client,
"/projects/projects",
"/v2/projects/",
json=project_data,
)
return ProjectStatusResponse.model_validate(response.json())
+17 -6
View File
@@ -19,7 +19,7 @@ from fastmcp import Context
from basic_memory.config import ConfigManager
from basic_memory.project_resolver import ProjectResolver
from basic_memory.schemas.project_info import ProjectItem, ProjectList
from basic_memory.utils import generate_permalink
from basic_memory.schemas.v2 import ProjectResolveResponse
async def resolve_project_parameter(
@@ -78,7 +78,7 @@ async def get_project_names(client: AsyncClient, headers: HeaderTypes | None = N
# Deferred import to avoid circular dependency with tools
from basic_memory.mcp.tools.utils import call_get
response = await call_get(client, "/projects/projects", headers=headers)
response = await call_get(client, "/v2/projects/", headers=headers)
project_list = ProjectList.model_validate(response.json())
return [project.name for project in project_list.projects]
@@ -104,7 +104,7 @@ async def get_active_project(
HTTPError: If project doesn't exist or is inaccessible
"""
# Deferred import to avoid circular dependency with tools
from basic_memory.mcp.tools.utils import call_get
from basic_memory.mcp.tools.utils import call_post
resolved_project = await resolve_project_parameter(project)
if not resolved_project:
@@ -126,9 +126,20 @@ async def get_active_project(
# Validate project exists by calling API
logger.debug(f"Validating project: {project}")
permalink = generate_permalink(project)
response = await call_get(client, f"/{permalink}/project/item", headers=headers)
active_project = ProjectItem.model_validate(response.json())
response = await call_post(
client,
"/v2/projects/resolve",
json={"identifier": project},
headers=headers,
)
resolved = ProjectResolveResponse.model_validate(response.json())
active_project = ProjectItem(
id=resolved.project_id,
external_id=resolved.external_id,
name=resolved.name,
path=resolved.path,
is_default=resolved.is_default,
)
# Cache in context if available
if context:
@@ -9,8 +9,9 @@ from typing import Annotated, Optional
from loguru import logger
from pydantic import Field
from basic_memory.config import get_project_config
from basic_memory.config import ConfigManager
from basic_memory.mcp.async_client import get_client
from basic_memory.mcp.project_context import get_active_project
from basic_memory.mcp.server import mcp
from basic_memory.mcp.tools.utils import call_post
from basic_memory.schemas.prompt import ContinueConversationRequest
@@ -42,17 +43,18 @@ async def continue_conversation(
logger.info(f"Continuing session, topic: {topic}, timeframe: {timeframe}")
async with get_client() as client:
config = ConfigManager().config
active_project = await get_active_project(client, project=config.default_project)
# Create request model
request = ContinueConversationRequest( # pyright: ignore [reportCallIssue]
topic=topic, timeframe=timeframe
)
project_url = get_project_config().project_url
# Call the prompt API endpoint
response = await call_post(
client,
f"{project_url}/prompt/continue-conversation",
f"/v2/projects/{active_project.external_id}/prompt/continue-conversation",
json=request.model_dump(exclude_none=True),
)
@@ -42,13 +42,11 @@ async def recent_activity_prompt(
Returns:
Formatted summary of recent activity
"""
timeframe = timeframe or "7d"
logger.info(f"Getting recent activity, timeframe: {timeframe}, project: {project}")
# Call the tool function - it returns a well-formatted string
# Pass type as string values (not enum) to match the tool's expected input
activity_summary = await recent_activity.fn(
project=project, timeframe=timeframe, type="entity"
)
activity_summary = await recent_activity.fn(project=project, timeframe=timeframe)
# Build the prompt response
# The tool already returns formatted markdown, so we use it directly
@@ -92,7 +90,7 @@ async def recent_activity_prompt(
- summarizes [[Recent Work]]
''',
folder="insights",
project="{project or 'default'}"
project="{project or "default"}"
)
```
""")
+8 -4
View File
@@ -8,8 +8,9 @@ from typing import Annotated, Optional
from loguru import logger
from pydantic import Field
from basic_memory.config import get_project_config
from basic_memory.config import ConfigManager
from basic_memory.mcp.async_client import get_client
from basic_memory.mcp.project_context import get_active_project
from basic_memory.mcp.server import mcp
from basic_memory.mcp.tools.utils import call_post
from basic_memory.schemas.prompt import SearchPromptRequest
@@ -41,14 +42,17 @@ async def search_prompt(
logger.info(f"Searching knowledge base, query: {query}, timeframe: {timeframe}")
async with get_client() as client:
config = ConfigManager().config
active_project = await get_active_project(client, project=config.default_project)
# Create request model
request = SearchPromptRequest(query=query, timeframe=timeframe)
project_url = get_project_config().project_url
# Call the prompt API endpoint
response = await call_post(
client, f"{project_url}/prompt/search", json=request.model_dump(exclude_none=True)
client,
f"/v2/projects/{active_project.external_id}/prompt/search",
json=request.model_dump(exclude_none=True),
)
# Extract the rendered prompt from the response
@@ -62,10 +62,9 @@ async def project_info(
async with get_client() as client:
project_config = await get_active_project(client, project, context)
project_url = project_config.permalink
# Call the API endpoint
response = await call_get(client, f"{project_url}/project/info")
response = await call_get(client, f"/v2/projects/{project_config.external_id}/info")
# Convert response to ProjectInfoResponse
return ProjectInfoResponse.model_validate(response.json())
+2 -1
View File
@@ -13,7 +13,7 @@ from basic_memory.mcp.tools.recent_activity import recent_activity
from basic_memory.mcp.tools.read_note import read_note
from basic_memory.mcp.tools.view_note import view_note
from basic_memory.mcp.tools.write_note import write_note
from basic_memory.mcp.tools.search import search_notes
from basic_memory.mcp.tools.search import search_notes, search_by_metadata
from basic_memory.mcp.tools.canvas import canvas
from basic_memory.mcp.tools.list_directory import list_directory
from basic_memory.mcp.tools.edit_note import edit_note
@@ -42,6 +42,7 @@ __all__ = [
"read_note",
"recent_activity",
"search",
"search_by_metadata",
"search_notes",
"view_note",
"write_note",
+1 -1
View File
@@ -256,7 +256,7 @@ async def edit_note(
edit_data["expected_replacements"] = str(expected_replacements)
# Call the PATCH endpoint
result = await knowledge_client.patch_entity(entity_id, edit_data)
result = await knowledge_client.patch_entity(entity_id, edit_data, fast=False)
# Format summary
summary = [
+3 -1
View File
@@ -488,7 +488,9 @@ move_note("{identifier}", "notes/{destination_path.split("/")[-1] if "/" in dest
return "\n".join(result_lines)
except Exception as e: # pragma: no cover
logger.error(f"Directory move failed for '{identifier}' to '{destination_path}': {e}")
logger.error(
f"Directory move failed for '{identifier}' to '{destination_path}': {e}"
)
return f"""# Directory Move Failed
Error moving directory '{identifier}' to '{destination_path}': {str(e)}
@@ -144,7 +144,7 @@ async def recent_activity(
)
# Get list of all projects
response = await call_get(client, "/projects/projects")
response = await call_get(client, "/v2/projects/")
project_list = ProjectList.model_validate(response.json())
projects_activity = {}
+113 -1
View File
@@ -1,7 +1,7 @@
"""Search tools for Basic Memory MCP server."""
from textwrap import dedent
from typing import List, Optional
from typing import List, Optional, Dict, Any
from loguru import logger
from fastmcp import Context
@@ -207,6 +207,9 @@ async def search_notes(
types: List[str] | None = None,
entity_types: List[str] | None = None,
after_date: Optional[str] = None,
metadata_filters: Optional[Dict[str, Any]] = None,
tags: Optional[List[str]] = None,
status: Optional[str] = None,
context: Context | None = None,
) -> SearchResponse | str:
"""Search across all content in the knowledge base with comprehensive syntax support.
@@ -248,6 +251,27 @@ async def search_notes(
- `search_notes("research", "query", entity_types=["observation"])` - Filter by entity type
- `search_notes("team-docs", "query", after_date="2024-01-01")` - Recent content only
- `search_notes("my-project", "query", after_date="1 week")` - Relative date filtering
- `search_notes("my-project", "query", tags=["security"])` - Filter by frontmatter tags
- `search_notes("my-project", "query", status="in-progress")` - Filter by frontmatter status
- `search_notes("my-project", "query", metadata_filters={"priority": {"$in": ["high"]}})`
### Structured Metadata Filters
Filters are exact matches on frontmatter metadata. Supported forms:
- Equality: `{"status": "in-progress"}`
- Array contains (all): `{"tags": ["security", "oauth"]}`
- Operators:
- `$in`: `{"priority": {"$in": ["high", "critical"]}}`
- `$gt`, `$gte`, `$lt`, `$lte`: `{"schema.confidence": {"$gt": 0.7}}`
- `$between`: `{"schema.confidence": {"$between": [0.3, 0.6]}}`
- Nested keys use dot notation (e.g., `"schema.confidence"`).
### Filter-only Searches
You can pass an empty query string when only using structured filters:
- `search_notes("my-project", "", metadata_filters={"type": "spec"})`
### Convenience Filters
`tags` and `status` are shorthand for metadata_filters. If the same key exists in
metadata_filters, that value wins.
### Advanced Pattern Examples
- `search_notes("work-project", "project AND (meeting OR discussion)")` - Complex boolean logic
@@ -265,6 +289,9 @@ async def search_notes(
types: Optional list of note types to search (e.g., ["note", "person"])
entity_types: Optional list of entity types to filter by (e.g., ["entity", "observation"])
after_date: Optional date filter for recent content (e.g., "1 week", "2d", "2024-01-01")
metadata_filters: Optional structured frontmatter filters (e.g., {"status": "in-progress"})
tags: Optional tag filter (frontmatter tags); shorthand for metadata_filters["tags"]
status: Optional status filter (frontmatter status); shorthand for metadata_filters["status"]
context: Optional FastMCP context for performance caching.
Returns:
@@ -355,6 +382,12 @@ async def search_notes(
search_query.types = types
if after_date:
search_query.after_date = after_date
if metadata_filters:
search_query.metadata_filters = metadata_filters
if tags:
search_query.tags = tags
if status:
search_query.status = status
async with get_client() as client:
active_project = await get_active_project(client, project, context)
@@ -387,3 +420,82 @@ async def search_notes(
logger.error(f"Search failed for query '{query}': {e}, project: {active_project.name}")
# Return formatted error message as string for better user experience
return _format_search_error_response(active_project.name, str(e), query, search_type)
@mcp.tool(
description="Search entities by structured frontmatter metadata.",
)
async def search_by_metadata(
filters: Dict[str, Any],
project: Optional[str] = None,
limit: int = 20,
offset: int = 0,
context: Context | None = None,
) -> SearchResponse | str:
"""Search entities by structured frontmatter metadata.
Args:
filters: Dictionary of metadata filters (e.g., {"status": "in-progress"})
project: Project name to search in. Optional - server will resolve using hierarchy.
limit: Maximum number of results to return
offset: Number of results to skip (for pagination)
context: Optional FastMCP context for performance caching.
Returns:
SearchResponse with results, or helpful error guidance if search fails
"""
if limit <= 0:
return "# Error\n\n`limit` must be greater than 0."
# Build a structured-only search query
search_query = SearchQuery()
search_query.metadata_filters = filters
search_query.entity_types = [SearchItemType.ENTITY]
# Convert offset/limit to page/page_size (API uses paging)
page_size = limit
page = (offset // limit) + 1
offset_within_page = offset % limit
async with get_client() as client:
active_project = await get_active_project(client, project, context)
logger.info(
f"Structured search in project {active_project.name} filters={filters} limit={limit} offset={offset}"
)
try:
from basic_memory.mcp.clients import SearchClient
search_client = SearchClient(client, active_project.external_id)
result = await search_client.search(
search_query.model_dump(),
page=page,
page_size=page_size,
)
# Apply offset within page, fetch next page if needed
if offset_within_page:
remaining = result.results[offset_within_page:]
if len(remaining) < limit:
next_page = page + 1
extra = await search_client.search(
search_query.model_dump(),
page=next_page,
page_size=page_size,
)
remaining.extend(extra.results[: max(0, limit - len(remaining))])
result = SearchResponse(
results=remaining[:limit],
current_page=page,
page_size=page_size,
)
return result
except Exception as e:
logger.error(
f"Metadata search failed for filters '{filters}': {e}, project: {active_project.name}"
)
return _format_search_error_response(
active_project.name, str(e), str(filters), "metadata"
)
+4 -2
View File
@@ -159,7 +159,7 @@ async def write_note(
logger.debug(f"Attempting to create entity permalink={entity.permalink}")
action = "Created" # Default to created
try:
result = await knowledge_client.create_entity(entity.model_dump())
result = await knowledge_client.create_entity(entity.model_dump(), fast=False)
action = "Created"
except Exception as e:
# If creation failed due to conflict (already exists), try to update
@@ -175,7 +175,9 @@ async def write_note(
"Entity permalink is required for updates"
) # pragma: no cover
entity_id = await knowledge_client.resolve_entity(entity.permalink)
result = await knowledge_client.update_entity(entity_id, entity.model_dump())
result = await knowledge_client.update_entity(
entity_id, entity.model_dump(), fast=False
)
action = "Updated"
except Exception as update_error: # pragma: no cover
# Re-raise the original error if update also fails
@@ -0,0 +1,139 @@
"""Helpers for parsing structured metadata filters for search."""
from __future__ import annotations
from dataclasses import dataclass
from datetime import date, datetime
import re
from typing import Any, Iterable, List
_KEY_RE = re.compile(r"^[A-Za-z0-9_-]+(\.[A-Za-z0-9_-]+)*$")
_NUMERIC_RE = re.compile(r"^-?\d+(\.\d+)?$")
@dataclass(frozen=True)
class ParsedMetadataFilter:
"""Normalized metadata filter for SQL generation."""
path_parts: List[str]
op: str
value: Any
comparison: str | None = None # "numeric" or "text" for comparisons
def _is_numeric_value(value: Any) -> bool:
if isinstance(value, bool):
return False
if isinstance(value, (int, float)):
return True
if isinstance(value, str):
return bool(_NUMERIC_RE.match(value.strip()))
return False
def _is_numeric_collection(values: Iterable[Any]) -> bool:
return all(_is_numeric_value(v) for v in values)
def _normalize_scalar(value: Any) -> Any:
if isinstance(value, datetime):
return value.isoformat()
if isinstance(value, date):
return value.isoformat()
if isinstance(value, bool):
return str(value)
if isinstance(value, (int, float)):
return str(value)
return value
def parse_metadata_filters(filters: dict[str, Any]) -> List[ParsedMetadataFilter]:
"""Parse metadata filters into normalized clauses.
Supported forms:
- {"status": "in-progress"}
- {"tags": ["security", "oauth"]} # array contains all
- {"priority": {"$in": ["high", "critical"]}}
- {"schema.confidence": {"$gt": 0.7}}
- {"schema.confidence": {"$between": [0.3, 0.6]}}
"""
parsed: List[ParsedMetadataFilter] = []
for raw_key, raw_value in (filters or {}).items():
if not isinstance(raw_key, str) or not raw_key.strip():
raise ValueError("metadata filter keys must be non-empty strings")
key = raw_key.strip()
if not _KEY_RE.match(key):
raise ValueError(f"Unsupported metadata filter key: {raw_key}")
path_parts = key.split(".")
# Operator form
if isinstance(raw_value, dict):
if len(raw_value) != 1:
raise ValueError(f"Invalid metadata filter for '{raw_key}': {raw_value}")
op, value = next(iter(raw_value.items()))
if op == "$in":
if not isinstance(value, list) or not value:
raise ValueError(f"$in requires a non-empty list for '{raw_key}'")
parsed.append(
ParsedMetadataFilter(path_parts, "in", [_normalize_scalar(v) for v in value])
)
continue
if op in {"$gt", "$gte", "$lt", "$lte"}:
if _is_numeric_value(value):
normalized = float(value)
comparison = "numeric"
else:
normalized = _normalize_scalar(value)
comparison = "text"
parsed.append(
ParsedMetadataFilter(path_parts, op.lstrip("$"), normalized, comparison)
)
continue
if op == "$between":
if not isinstance(value, list) or len(value) != 2:
raise ValueError(f"$between requires [min, max] for '{raw_key}'")
if _is_numeric_collection(value):
normalized = [float(v) for v in value]
comparison = "numeric"
else:
normalized = [_normalize_scalar(v) for v in value]
comparison = "text"
parsed.append(ParsedMetadataFilter(path_parts, "between", normalized, comparison))
continue
raise ValueError(f"Unsupported operator '{op}' in metadata filter for '{raw_key}'")
# Array contains (all)
if isinstance(raw_value, list):
if not raw_value:
raise ValueError(f"Empty list not allowed for metadata filter '{raw_key}'")
parsed.append(
ParsedMetadataFilter(
path_parts, "contains", [_normalize_scalar(v) for v in raw_value]
)
)
continue
# Simple equality
parsed.append(ParsedMetadataFilter(path_parts, "eq", _normalize_scalar(raw_value)))
return parsed
def build_sqlite_json_path(parts: List[str]) -> str:
"""Build a SQLite JSON path for json_extract/json_each."""
path = "$"
for part in parts:
path += f'."{part}"'
return path
def build_postgres_json_path(parts: List[str]) -> str:
"""Build a Postgres JSON path for #>>/#> operators."""
return "{" + ",".join(parts) + "}"
@@ -12,6 +12,10 @@ from sqlalchemy import text
from basic_memory import db
from basic_memory.repository.search_index_row import SearchIndexRow
from basic_memory.repository.search_repository_base import SearchRepositoryBase
from basic_memory.repository.metadata_filters import (
parse_metadata_filters,
build_postgres_json_path,
)
from basic_memory.schemas.search import SearchItemType
@@ -215,6 +219,7 @@ class PostgresSearchRepository(SearchRepositoryBase):
types: Optional[List[str]] = None,
after_date: Optional[datetime] = None,
search_item_types: Optional[List[SearchItemType]] = None,
metadata_filters: Optional[dict] = None,
limit: int = 10,
offset: int = 0,
) -> List[SearchIndexRow]:
@@ -222,6 +227,7 @@ class PostgresSearchRepository(SearchRepositoryBase):
conditions = []
params = {}
order_by_clause = ""
from_clause = "search_index"
# Handle text search for title and content using tsvector
if search_text:
@@ -233,18 +239,22 @@ class PostgresSearchRepository(SearchRepositoryBase):
processed_text = self._prepare_search_term(search_text.strip())
params["text"] = processed_text
# Use @@ operator for tsvector matching
conditions.append("textsearchable_index_col @@ to_tsquery('english', :text)")
conditions.append(
"search_index.textsearchable_index_col @@ to_tsquery('english', :text)"
)
# Handle title search
if title:
title_text = self._prepare_search_term(title.strip(), is_prefix=False)
params["title_text"] = title_text
conditions.append("to_tsvector('english', title) @@ to_tsquery('english', :title_text)")
conditions.append(
"to_tsvector('english', search_index.title) @@ to_tsquery('english', :title_text)"
)
# Handle permalink exact search
if permalink:
params["permalink"] = permalink
conditions.append("permalink = :permalink")
conditions.append("search_index.permalink = :permalink")
# Handle permalink pattern match
if permalink_match:
@@ -255,14 +265,14 @@ class PostgresSearchRepository(SearchRepositoryBase):
# Convert * to % for SQL LIKE
permalink_pattern = permalink_text.replace("*", "%")
params["permalink"] = permalink_pattern
conditions.append("permalink LIKE :permalink")
conditions.append("search_index.permalink LIKE :permalink")
else:
conditions.append("permalink = :permalink")
conditions.append("search_index.permalink = :permalink")
# Handle search item type filter
if search_item_types:
type_list = ", ".join(f"'{t.value}'" for t in search_item_types)
conditions.append(f"type IN ({type_list})")
conditions.append(f"search_index.type IN ({type_list})")
# Handle entity type filter using JSONB containment
if types:
@@ -270,19 +280,88 @@ class PostgresSearchRepository(SearchRepositoryBase):
type_conditions = []
for entity_type in types:
# Create JSONB containment condition for each type
type_conditions.append(f'metadata @> \'{{"entity_type": "{entity_type}"}}\'')
type_conditions.append(
f'search_index.metadata @> \'{{"entity_type": "{entity_type}"}}\''
)
conditions.append(f"({' OR '.join(type_conditions)})")
# Handle date filter
if after_date:
params["after_date"] = after_date
conditions.append("created_at > :after_date")
conditions.append("search_index.created_at > :after_date")
# order by most recent first
order_by_clause = ", updated_at DESC"
order_by_clause = ", search_index.updated_at DESC"
# Handle structured metadata filters (frontmatter)
if metadata_filters:
parsed_filters = parse_metadata_filters(metadata_filters)
from_clause = "search_index JOIN entity ON search_index.entity_id = entity.id"
metadata_expr = "entity.entity_metadata::jsonb"
for idx, filt in enumerate(parsed_filters):
path = build_postgres_json_path(filt.path_parts)
text_expr = f"({metadata_expr} #>> '{path}')"
json_expr = f"({metadata_expr} #> '{path}')"
if filt.op == "eq":
value_param = f"meta_val_{idx}"
params[value_param] = filt.value
conditions.append(f"{text_expr} = :{value_param}")
continue
if filt.op == "in":
placeholders = []
for j, val in enumerate(filt.value):
value_param = f"meta_val_{idx}_{j}"
params[value_param] = val
placeholders.append(f":{value_param}")
conditions.append(f"{text_expr} IN ({', '.join(placeholders)})")
continue
if filt.op == "contains":
import json as _json
base_param = f"meta_val_{idx}"
tag_conditions = []
# Require all values to be present
for j, val in enumerate(filt.value):
tag_param = f"{base_param}_{j}"
params[tag_param] = _json.dumps([val])
like_param = f"{base_param}_{j}_like"
params[like_param] = f'%"{val}"%'
like_param_single = f"{base_param}_{j}_like_single"
params[like_param_single] = f"%'{val}'%"
tag_conditions.append(
f"({json_expr} @> CAST(:{tag_param} AS jsonb) "
f"OR {text_expr} LIKE :{like_param} "
f"OR {text_expr} LIKE :{like_param_single})"
)
conditions.append(" AND ".join(tag_conditions))
continue
if filt.op in {"gt", "gte", "lt", "lte", "between"}:
compare_expr = (
f"({metadata_expr} #>> '{path}')::double precision"
if filt.comparison == "numeric"
else text_expr
)
if filt.op == "between":
min_param = f"meta_val_{idx}_min"
max_param = f"meta_val_{idx}_max"
params[min_param] = filt.value[0]
params[max_param] = filt.value[1]
conditions.append(f"{compare_expr} BETWEEN :{min_param} AND :{max_param}")
else:
value_param = f"meta_val_{idx}"
params[value_param] = filt.value
operator = {"gt": ">", "gte": ">=", "lt": "<", "lte": "<="}[filt.op]
conditions.append(f"{compare_expr} {operator} :{value_param}")
continue
# Always filter by project_id
params["project_id"] = self.project_id
conditions.append("project_id = :project_id")
conditions.append("search_index.project_id = :project_id")
# set limit and offset
params["limit"] = limit
@@ -294,31 +373,33 @@ class PostgresSearchRepository(SearchRepositoryBase):
# Build SQL with ts_rank() for scoring
# Note: If no text search, score will be NULL, so we use COALESCE to default to 0
if search_text and search_text.strip() and search_text.strip() != "*":
score_expr = "ts_rank(textsearchable_index_col, to_tsquery('english', :text))"
score_expr = (
"ts_rank(search_index.textsearchable_index_col, to_tsquery('english', :text))"
)
else:
score_expr = "0"
sql = f"""
SELECT
project_id,
id,
title,
permalink,
file_path,
type,
metadata,
from_id,
to_id,
relation_type,
entity_id,
content_snippet,
category,
created_at,
updated_at,
search_index.project_id,
search_index.id,
search_index.title,
search_index.permalink,
search_index.file_path,
search_index.type,
search_index.metadata,
search_index.from_id,
search_index.to_id,
search_index.relation_type,
search_index.entity_id,
search_index.content_snippet,
search_index.category,
search_index.created_at,
search_index.updated_at,
{score_expr} as score
FROM search_index
FROM {from_clause}
WHERE {where_clause}
ORDER BY score DESC, id ASC {order_by_clause}
ORDER BY score DESC, search_index.id ASC {order_by_clause}
LIMIT :limit
OFFSET :offset
"""
@@ -40,6 +40,7 @@ class SearchRepository(Protocol):
types: Optional[List[str]] = None,
after_date: Optional[datetime] = None,
search_item_types: Optional[List[SearchItemType]] = None,
metadata_filters: Optional[dict] = None,
limit: int = 10,
offset: int = 0,
) -> List[SearchIndexRow]:
@@ -78,6 +78,7 @@ class SearchRepositoryBase(ABC):
types: Optional[List[str]] = None,
after_date: Optional[datetime] = None,
search_item_types: Optional[List[SearchItemType]] = None,
metadata_filters: Optional[Dict[str, Any]] = None,
limit: int = 10,
offset: int = 0,
) -> List[SearchIndexRow]:
@@ -91,6 +92,7 @@ class SearchRepositoryBase(ABC):
types: Filter by entity types (from metadata.entity_type)
after_date: Filter by created_at > after_date
search_item_types: Filter by SearchItemType (ENTITY, OBSERVATION, RELATION)
metadata_filters: Structured frontmatter metadata filters
limit: Maximum results to return
offset: Number of results to skip
@@ -13,6 +13,7 @@ from basic_memory import db
from basic_memory.models.search import CREATE_SEARCH_INDEX
from basic_memory.repository.search_index_row import SearchIndexRow
from basic_memory.repository.search_repository_base import SearchRepositoryBase
from basic_memory.repository.metadata_filters import parse_metadata_filters, build_sqlite_json_path
from basic_memory.schemas.search import SearchItemType
@@ -26,6 +27,17 @@ class SQLiteSearchRepository(SearchRepositoryBase):
- Prefix wildcard matching with *
"""
def __init__(self, session_maker, project_id: int):
super().__init__(session_maker, project_id)
self._entity_columns: set[str] | None = None
async def _get_entity_columns(self) -> set[str]:
if self._entity_columns is None:
async with db.scoped_session(self.session_maker) as session:
result = await session.execute(text("PRAGMA table_info(entity)"))
self._entity_columns = {row[1] for row in result.fetchall()}
return self._entity_columns
async def init_search_index(self):
"""Create FTS5 virtual table for search if it doesn't exist.
@@ -287,6 +299,7 @@ class SQLiteSearchRepository(SearchRepositoryBase):
types: Optional[List[str]] = None,
after_date: Optional[datetime] = None,
search_item_types: Optional[List[SearchItemType]] = None,
metadata_filters: Optional[dict] = None,
limit: int = 10,
offset: int = 0,
) -> List[SearchIndexRow]:
@@ -294,6 +307,7 @@ class SQLiteSearchRepository(SearchRepositoryBase):
conditions = []
params = {}
order_by_clause = ""
from_clause = "search_index"
# Handle text search for title and content
if search_text:
@@ -305,18 +319,20 @@ class SQLiteSearchRepository(SearchRepositoryBase):
# Use _prepare_search_term to handle both Boolean and non-Boolean queries
processed_text = self._prepare_search_term(search_text.strip())
params["text"] = processed_text
conditions.append("(title MATCH :text OR content_stems MATCH :text)")
conditions.append(
"(search_index.title MATCH :text OR search_index.content_stems MATCH :text)"
)
# Handle title match search
if title:
title_text = self._prepare_search_term(title.strip(), is_prefix=False)
params["title_text"] = title_text
conditions.append("title MATCH :title_text")
conditions.append("search_index.title MATCH :title_text")
# Handle permalink exact search
if permalink:
params["permalink"] = permalink
conditions.append("permalink = :permalink")
conditions.append("search_index.permalink = :permalink")
# Handle permalink match search, supports *
if permalink_match:
@@ -325,38 +341,122 @@ class SQLiteSearchRepository(SearchRepositoryBase):
permalink_text = permalink_match.lower().strip()
params["permalink"] = permalink_text
if "*" in permalink_match:
conditions.append("permalink GLOB :permalink")
conditions.append("search_index.permalink GLOB :permalink")
else:
# For exact matches without *, we can use FTS5 MATCH
# but only prepare the term if it doesn't look like a path
if "/" in permalink_text:
conditions.append("permalink = :permalink")
conditions.append("search_index.permalink = :permalink")
else:
permalink_text = self._prepare_search_term(permalink_text, is_prefix=False)
params["permalink"] = permalink_text
conditions.append("permalink MATCH :permalink")
conditions.append("search_index.permalink MATCH :permalink")
# Handle entity type filter
if search_item_types:
type_list = ", ".join(f"'{t.value}'" for t in search_item_types)
conditions.append(f"type IN ({type_list})")
conditions.append(f"search_index.type IN ({type_list})")
# Handle type filter
if types:
type_list = ", ".join(f"'{t}'" for t in types)
conditions.append(f"json_extract(metadata, '$.entity_type') IN ({type_list})")
conditions.append(
f"json_extract(search_index.metadata, '$.entity_type') IN ({type_list})"
)
# Handle date filter using datetime() for proper comparison
if after_date:
params["after_date"] = after_date
conditions.append("datetime(created_at) > datetime(:after_date)")
conditions.append("datetime(search_index.created_at) > datetime(:after_date)")
# order by most recent first
order_by_clause = ", updated_at DESC"
order_by_clause = ", search_index.updated_at DESC"
# Handle structured metadata filters (frontmatter)
if metadata_filters:
parsed_filters = parse_metadata_filters(metadata_filters)
from_clause = "search_index JOIN entity ON search_index.entity_id = entity.id"
entity_columns = await self._get_entity_columns()
for idx, filt in enumerate(parsed_filters):
path_param = f"meta_path_{idx}"
extract_expr = None
use_tags_column = False
if filt.path_parts == ["status"] and "frontmatter_status" in entity_columns:
extract_expr = "entity.frontmatter_status"
elif filt.path_parts == ["type"] and "frontmatter_type" in entity_columns:
extract_expr = "entity.frontmatter_type"
elif filt.path_parts == ["tags"] and "tags_json" in entity_columns:
extract_expr = "entity.tags_json"
use_tags_column = True
if extract_expr is None:
params[path_param] = build_sqlite_json_path(filt.path_parts)
extract_expr = f"json_extract(entity.entity_metadata, :{path_param})"
if filt.op == "eq":
value_param = f"meta_val_{idx}"
params[value_param] = filt.value
conditions.append(f"{extract_expr} = :{value_param}")
continue
if filt.op == "in":
placeholders = []
for j, val in enumerate(filt.value):
value_param = f"meta_val_{idx}_{j}"
params[value_param] = val
placeholders.append(f":{value_param}")
conditions.append(f"{extract_expr} IN ({', '.join(placeholders)})")
continue
if filt.op == "contains":
tag_conditions = []
for j, val in enumerate(filt.value):
value_param = f"meta_val_{idx}_{j}"
params[value_param] = val
like_param = f"{value_param}_like"
params[like_param] = f'%"{val}"%'
like_param_single = f"{value_param}_like_single"
params[like_param_single] = f"%'{val}'%"
json_each_expr = (
"json_each(entity.tags_json)"
if use_tags_column
else f"json_each(entity.entity_metadata, :{path_param})"
)
tag_conditions.append(
"("
f"EXISTS (SELECT 1 FROM {json_each_expr} WHERE value = :{value_param}) "
f"OR {extract_expr} LIKE :{like_param} "
f"OR {extract_expr} LIKE :{like_param_single}"
")"
)
conditions.append(" AND ".join(tag_conditions))
continue
if filt.op in {"gt", "gte", "lt", "lte", "between"}:
compare_expr = (
f"CAST({extract_expr} AS REAL)"
if filt.comparison == "numeric"
else extract_expr
)
if filt.op == "between":
min_param = f"meta_val_{idx}_min"
max_param = f"meta_val_{idx}_max"
params[min_param] = filt.value[0]
params[max_param] = filt.value[1]
conditions.append(f"{compare_expr} BETWEEN :{min_param} AND :{max_param}")
else:
value_param = f"meta_val_{idx}"
params[value_param] = filt.value
operator = {"gt": ">", "gte": ">=", "lt": "<", "lte": "<="}[filt.op]
conditions.append(f"{compare_expr} {operator} :{value_param}")
continue
# Always filter by project_id
params["project_id"] = self.project_id
conditions.append("project_id = :project_id")
conditions.append("search_index.project_id = :project_id")
# set limit on search query
params["limit"] = limit
@@ -367,23 +467,23 @@ class SQLiteSearchRepository(SearchRepositoryBase):
sql = f"""
SELECT
project_id,
id,
title,
permalink,
file_path,
type,
metadata,
from_id,
to_id,
relation_type,
entity_id,
content_snippet,
category,
created_at,
updated_at,
search_index.project_id,
search_index.id,
search_index.title,
search_index.permalink,
search_index.file_path,
search_index.type,
search_index.metadata,
search_index.from_id,
search_index.to_id,
search_index.relation_type,
search_index.entity_id,
search_index.content_snippet,
search_index.category,
search_index.created_at,
search_index.updated_at,
bm25(search_index) as score
FROM search_index
FROM {from_clause}
WHERE {where_clause}
ORDER BY score ASC {order_by_clause}
LIMIT :limit
+1 -1
View File
@@ -25,7 +25,7 @@ class CloudProject(BaseModel):
class CloudProjectList(BaseModel):
"""Response from /proxy/projects/projects endpoint."""
"""Response from /proxy/v2/projects endpoint."""
projects: list[CloudProject] = Field(default_factory=list, description="List of cloud projects")
+4
View File
@@ -124,6 +124,7 @@ class EntitySummary(BaseModel):
"""Simplified entity representation."""
type: Literal["entity"] = "entity"
external_id: str # UUID for v2 API routing
entity_id: int # Database ID for v2 API consistency
permalink: Optional[str]
title: str
@@ -150,8 +151,10 @@ class RelationSummary(BaseModel):
relation_type: str
from_entity: Optional[str] = None
from_entity_id: Optional[int] = None # ID of source entity
from_entity_external_id: Optional[str] = None # UUID of source entity for v2 API routing
to_entity: Optional[str] = None
to_entity_id: Optional[int] = None # ID of target entity
to_entity_external_id: Optional[str] = None # UUID of target entity for v2 API routing
created_at: Annotated[
datetime, Field(json_schema_extra={"type": "string", "format": "date-time"})
]
@@ -167,6 +170,7 @@ class ObservationSummary(BaseModel):
type: Literal["observation"] = "observation"
observation_id: int # Database ID for v2 API consistency
entity_id: Optional[int] = None # ID of the entity this observation belongs to
entity_external_id: Optional[str] = None # UUID of parent entity for v2 API routing
title: str
file_path: str
permalink: str
+22 -6
View File
@@ -6,7 +6,7 @@ The search system supports three primary modes:
3. Full-text search across content
"""
from typing import Optional, List, Union
from typing import Optional, List, Union, Any
from datetime import datetime
from enum import Enum
from pydantic import BaseModel, field_validator
@@ -29,11 +29,15 @@ class SearchQuery(BaseModel):
- permalink: Exact permalink match
- permalink_match: Path pattern with *
- text: Full-text search of title/content (supports boolean operators: AND, OR, NOT)
- title: Title only search
Optionally filter results by:
- types: Limit to specific item types
- entity_types: Limit to specific entity types
- types: Limit to specific entity types (frontmatter "type")
- entity_types: Limit to search item types (entity/observation/relation)
- after_date: Only items after date
- metadata_filters: Structured frontmatter filters (field -> value)
- tags: Convenience frontmatter tag filter
- status: Convenience frontmatter status filter
Boolean search examples:
- "python AND flask" - Find items with both terms
@@ -52,6 +56,9 @@ class SearchQuery(BaseModel):
types: Optional[List[str]] = None # Filter by type
entity_types: Optional[List[SearchItemType]] = None # Filter by entity type
after_date: Optional[Union[datetime, str]] = None # Time-based filter
metadata_filters: Optional[dict[str, Any]] = None # Structured frontmatter filters
tags: Optional[List[str]] = None # Convenience tag filter
status: Optional[str] = None # Convenience status filter
@field_validator("after_date")
@classmethod
@@ -62,14 +69,23 @@ class SearchQuery(BaseModel):
return v
def no_criteria(self) -> bool:
text_is_empty = self.text is None or (isinstance(self.text, str) and not self.text.strip())
metadata_is_empty = not self.metadata_filters
tags_is_empty = not self.tags
status_is_empty = self.status is None or (isinstance(self.status, str) and not self.status)
types_is_empty = not self.types
entity_types_is_empty = not self.entity_types
return (
self.permalink is None
and self.permalink_match is None
and self.title is None
and self.text is None
and text_is_empty
and self.after_date is None
and self.types is None
and self.entity_types is None
and types_is_empty
and entity_types_is_empty
and metadata_is_empty
and tags_is_empty
and status_is_empty
)
def has_boolean_operators(self) -> bool:
+220 -45
View File
@@ -1,5 +1,6 @@
"""Service for managing entities in the database."""
from datetime import datetime
from pathlib import Path
from typing import List, Optional, Sequence, Tuple, Union
@@ -17,7 +18,7 @@ from basic_memory.file_utils import (
dump_frontmatter,
)
from basic_memory.markdown import EntityMarkdown
from basic_memory.markdown.entity_parser import EntityParser
from basic_memory.markdown.entity_parser import EntityParser, normalize_frontmatter_metadata
from basic_memory.markdown.utils import entity_model_from_markdown, schema_to_markdown
from basic_memory.models import Entity as EntityModel
from basic_memory.models import Observation, Relation
@@ -33,7 +34,11 @@ from basic_memory.schemas.response import (
DirectoryDeleteError,
)
from basic_memory.services import BaseService, FileService
from basic_memory.services.exceptions import EntityCreationError, EntityNotFoundError
from basic_memory.services.exceptions import (
EntityAlreadyExistsError,
EntityCreationError,
EntityNotFoundError,
)
from basic_memory.services.link_resolver import LinkResolver
from basic_memory.services.search_service import SearchService
from basic_memory.utils import generate_permalink
@@ -167,6 +172,25 @@ class EntityService(BaseService[EntityModel]):
return permalink
def _build_frontmatter_markdown(
self, title: str, entity_type: str, permalink: str
) -> EntityMarkdown:
"""Build a minimal EntityMarkdown object for permalink resolution."""
from basic_memory.markdown.schemas import EntityFrontmatter
frontmatter_metadata = {
"title": title,
"type": entity_type,
"permalink": permalink,
}
frontmatter_obj = EntityFrontmatter(metadata=frontmatter_metadata)
return EntityMarkdown(
frontmatter=frontmatter_obj,
content="",
observations=[],
relations=[],
)
async def create_or_update_entity(self, schema: EntitySchema) -> Tuple[EntityModel, bool]:
"""Create new entity or update existing one.
Returns: (entity, is_new) where is_new is True if a new entity was created
@@ -196,7 +220,7 @@ class EntityService(BaseService[EntityModel]):
file_path = Path(schema.file_path)
if await self.file_service.exists(file_path):
raise EntityCreationError(
raise EntityAlreadyExistsError(
f"file for entity {schema.directory}/{schema.title} already exists: {file_path}"
)
@@ -210,20 +234,8 @@ class EntityService(BaseService[EntityModel]):
schema.entity_type = content_frontmatter["type"]
if "permalink" in content_frontmatter:
# Create a minimal EntityMarkdown object for permalink resolution
from basic_memory.markdown.schemas import EntityFrontmatter
frontmatter_metadata = {
"title": schema.title,
"type": schema.entity_type,
"permalink": content_frontmatter["permalink"],
}
frontmatter_obj = EntityFrontmatter(metadata=frontmatter_metadata)
content_markdown = EntityMarkdown(
frontmatter=frontmatter_obj,
content="", # content not needed for permalink resolution
observations=[],
relations=[],
content_markdown = self._build_frontmatter_markdown(
schema.title, schema.entity_type, content_frontmatter["permalink"]
)
# Get unique permalink (prioritizing content frontmatter) unless disabled
@@ -248,11 +260,8 @@ class EntityService(BaseService[EntityModel]):
content=final_content,
)
# create entity
created = await self.create_entity_from_markdown(file_path, entity_markdown)
# add relations
entity = await self.update_entity_relations(created.file_path, entity_markdown)
# create entity and relations
entity = await self.upsert_entity_from_markdown(file_path, entity_markdown, is_new=True)
# Set final checksum to mark complete
return await self.repository.update(entity.id, {"checksum": checksum})
@@ -283,20 +292,8 @@ class EntityService(BaseService[EntityModel]):
schema.entity_type = content_frontmatter["type"]
if "permalink" in content_frontmatter:
# Create a minimal EntityMarkdown object for permalink resolution
from basic_memory.markdown.schemas import EntityFrontmatter
frontmatter_metadata = {
"title": schema.title,
"type": schema.entity_type,
"permalink": content_frontmatter["permalink"],
}
frontmatter_obj = EntityFrontmatter(metadata=frontmatter_metadata)
content_markdown = EntityMarkdown(
frontmatter=frontmatter_obj,
content="", # content not needed for permalink resolution
observations=[],
relations=[],
content_markdown = self._build_frontmatter_markdown(
schema.title, schema.entity_type, content_frontmatter["permalink"]
)
# Check if we need to update the permalink based on content frontmatter (unless disabled)
@@ -333,17 +330,178 @@ class EntityService(BaseService[EntityModel]):
content=final_content,
)
# update entity in db
entity = await self.update_entity_and_observations(file_path, entity_markdown)
# add relations
await self.update_entity_relations(file_path.as_posix(), entity_markdown)
# update entity and relations
entity = await self.upsert_entity_from_markdown(file_path, entity_markdown, is_new=False)
# Set final checksum to match file
entity = await self.repository.update(entity.id, {"checksum": checksum})
return entity
async def fast_write_entity(
self,
schema: EntitySchema,
external_id: Optional[str] = None,
) -> EntityModel:
"""Write file and upsert a minimal entity row for fast responses."""
logger.debug(
"Fast-writing entity",
title=schema.title,
external_id=external_id,
content_type=schema.content_type,
)
# --- Identity & File Path ---
existing = await self.repository.get_by_external_id(external_id) if external_id else None
# Trigger: external_id already exists
# Why: avoid duplicate entities when title-derived paths change
# Outcome: update in-place and keep the existing file path
file_path = Path(existing.file_path) if existing else Path(schema.file_path)
if not existing and await self.file_service.exists(file_path):
raise EntityAlreadyExistsError(
f"file for entity {schema.directory}/{schema.title} already exists: {file_path}"
)
# --- Frontmatter Overrides ---
content_markdown = None
if schema.content and has_frontmatter(schema.content):
content_frontmatter = parse_frontmatter(schema.content)
if "type" in content_frontmatter:
schema.entity_type = content_frontmatter["type"]
if "permalink" in content_frontmatter:
content_markdown = self._build_frontmatter_markdown(
schema.title, schema.entity_type, content_frontmatter["permalink"]
)
# --- Permalink Resolution ---
if self.app_config and self.app_config.disable_permalinks:
schema._permalink = ""
else:
if existing and not (content_markdown and content_markdown.frontmatter.permalink):
schema._permalink = existing.permalink or await self.resolve_permalink(
file_path, skip_conflict_check=True
)
else:
schema._permalink = await self.resolve_permalink(
file_path, content_markdown, skip_conflict_check=True
)
# --- File Write ---
post = await schema_to_markdown(schema)
final_content = dump_frontmatter(post)
checksum = await self.file_service.write_file(file_path, final_content)
# --- Minimal DB Upsert ---
metadata = normalize_frontmatter_metadata(post.metadata or {})
entity_metadata = {k: v for k, v in metadata.items() if v is not None}
update_data = {
"title": schema.title,
"entity_type": schema.entity_type,
"file_path": file_path.as_posix(),
"content_type": schema.content_type,
"entity_metadata": entity_metadata or None,
"permalink": schema.permalink,
"checksum": checksum,
"updated_at": datetime.now().astimezone(),
}
if existing:
updated = await self.repository.update(existing.id, update_data)
if not updated:
raise ValueError(f"Failed to update entity in database: {existing.id}")
return updated
create_data = dict(update_data)
if external_id is not None:
create_data["external_id"] = external_id
return await self.repository.create(create_data)
async def fast_edit_entity(
self,
entity: EntityModel,
operation: str,
content: str,
section: Optional[str] = None,
find_text: Optional[str] = None,
expected_replacements: int = 1,
) -> EntityModel:
"""Edit an entity quickly and defer full indexing to background."""
logger.debug(f"Fast editing entity: {entity.external_id}, operation: {operation}")
# --- File Edit ---
file_path = Path(entity.file_path)
current_content, _ = await self.file_service.read_file(file_path)
new_content = self.apply_edit_operation(
current_content, operation, content, section, find_text, expected_replacements
)
checksum = await self.file_service.write_file(file_path, new_content)
# --- Frontmatter Overrides ---
update_data = {
"checksum": checksum,
"updated_at": datetime.now().astimezone(),
}
content_markdown = None
if has_frontmatter(new_content):
content_frontmatter = parse_frontmatter(new_content)
if "title" in content_frontmatter:
update_data["title"] = content_frontmatter["title"]
if "type" in content_frontmatter:
update_data["entity_type"] = content_frontmatter["type"]
if "permalink" in content_frontmatter:
content_markdown = self._build_frontmatter_markdown(
update_data.get("title", entity.title),
update_data.get("entity_type", entity.entity_type),
content_frontmatter["permalink"],
)
metadata = normalize_frontmatter_metadata(content_frontmatter or {})
update_data["entity_metadata"] = {k: v for k, v in metadata.items() if v is not None}
# --- Permalink Resolution ---
if self.app_config and self.app_config.disable_permalinks:
update_data["permalink"] = None
elif content_markdown and content_markdown.frontmatter.permalink:
update_data["permalink"] = await self.resolve_permalink(
file_path, content_markdown, skip_conflict_check=True
)
updated = await self.repository.update(entity.id, update_data)
if not updated:
raise ValueError(f"Failed to update entity in database: {entity.id}")
return updated
async def reindex_entity(self, entity_id: int) -> None:
"""Parse file content and rebuild observations/relations/search for an entity."""
entity = await self.repository.find_by_id(entity_id)
if not entity:
raise EntityNotFoundError(f"Entity not found: {entity_id}")
# --- Full Parse ---
file_path = Path(entity.file_path)
content = await self.file_service.read_file_content(file_path)
entity_markdown = await self.entity_parser.parse_markdown_content(
file_path=file_path,
content=content,
)
# --- DB Reindex ---
updated = await self.upsert_entity_from_markdown(file_path, entity_markdown, is_new=False)
checksum = await self.file_service.compute_checksum(file_path)
updated = await self.repository.update(updated.id, {"checksum": checksum})
if not updated:
raise ValueError(f"Failed to update entity in database: {entity.id}")
# --- Search Reindex ---
if self.search_service:
await self.search_service.index_entity_data(updated, content=content)
async def delete_entity(self, permalink_or_id: str | int) -> bool:
"""Delete entity and its file."""
logger.debug(f"Deleting entity: {permalink_or_id}")
@@ -465,6 +623,20 @@ class EntityService(BaseService[EntityModel]):
db_entity,
)
async def upsert_entity_from_markdown(
self,
file_path: Path,
markdown: EntityMarkdown,
*,
is_new: bool,
) -> EntityModel:
"""Create/update entity and relations from parsed markdown."""
if is_new:
created = await self.create_entity_from_markdown(file_path, markdown)
else:
created = await self.update_entity_and_observations(file_path, markdown)
return await self.update_entity_relations(created.file_path, markdown)
async def update_entity_relations(
self,
path: str,
@@ -589,8 +761,7 @@ class EntityService(BaseService[EntityModel]):
)
# Update entity and its relationships
entity = await self.update_entity_and_observations(file_path, entity_markdown)
await self.update_entity_relations(file_path.as_posix(), entity_markdown)
entity = await self.upsert_entity_from_markdown(file_path, entity_markdown, is_new=False)
# Set final checksum to match file
entity = await self.repository.update(entity.id, {"checksum": checksum})
@@ -927,7 +1098,9 @@ class EntityService(BaseService[EntityModel]):
old_path = entity.file_path
# Replace only the first occurrence of the source directory prefix
if old_path.startswith(f"{source_directory}/"):
new_path = old_path.replace(f"{source_directory}/", f"{destination_directory}/", 1)
new_path = old_path.replace(
f"{source_directory}/", f"{destination_directory}/", 1
)
else: # pragma: no cover
# Entity is directly in the source directory (shouldn't happen with prefix match)
new_path = f"{destination_directory}/{old_path}"
@@ -1016,7 +1189,9 @@ class EntityService(BaseService[EntityModel]):
logger.debug(f"Deleted entity: {file_path}")
else: # pragma: no cover
failed_deletes += 1
errors.append(DirectoryDeleteError(path=file_path, error="Delete returned False"))
errors.append(
DirectoryDeleteError(path=file_path, error="Delete returned False")
)
logger.warning(f"Delete returned False for entity: {file_path}")
except Exception as e: # pragma: no cover
+6
View File
@@ -16,6 +16,12 @@ class EntityCreationError(Exception):
pass
class EntityAlreadyExistsError(EntityCreationError):
"""Raised when an entity file already exists"""
pass
class DirectoryOperationError(Exception):
"""Raised when directory operations fail"""
+22 -1
View File
@@ -1,8 +1,9 @@
"""Service for search operations."""
import ast
import re
from datetime import datetime
from typing import List, Optional, Set
from typing import List, Optional, Set, Dict, Any
from dateparser import parse
@@ -79,6 +80,16 @@ class SearchService:
2. Pattern match: handles * wildcards in paths
3. Text search: full-text search across title/content
"""
# Support tag:<tag> shorthand by mapping to tags filter
if query.text:
text = query.text.strip()
if text.lower().startswith("tag:"):
tag_values = re.split(r"[,\s]+", text[4:].strip())
tags = [t for t in tag_values if t]
if tags:
query.tags = tags
query.text = None
if query.no_criteria():
logger.debug("no criteria passed to query")
return []
@@ -95,6 +106,15 @@ class SearchService:
else None
)
# Merge structured metadata filters (explicit + convenience fields)
metadata_filters: Optional[Dict[str, Any]] = None
if query.metadata_filters or query.tags or query.status:
metadata_filters = dict(query.metadata_filters or {})
if query.tags:
metadata_filters.setdefault("tags", query.tags)
if query.status:
metadata_filters.setdefault("status", query.status)
# search
results = await self.repository.search(
search_text=query.text,
@@ -104,6 +124,7 @@ class SearchService:
types=query.types,
search_item_types=query.entity_types,
after_date=after_date,
metadata_filters=metadata_filters,
limit=limit,
offset=offset,
)
+5 -13
View File
@@ -685,19 +685,11 @@ class SyncService:
entity_markdown.frontmatter.metadata["permalink"] = permalink
await self.file_service.update_frontmatter(path, {"permalink": permalink})
# if the file is new, create an entity
if new:
# Create entity with final permalink
logger.debug(f"Creating new entity from markdown, path={path}")
await self.entity_service.create_entity_from_markdown(Path(path), entity_markdown)
# otherwise we need to update the entity and observations
else:
logger.debug(f"Updating entity from markdown, path={path}")
await self.entity_service.update_entity_and_observations(Path(path), entity_markdown)
# Update relations and search index
entity = await self.entity_service.update_entity_relations(path, entity_markdown)
# Create/update entity and relations in one path
logger.debug(f"{'Creating' if new else 'Updating'} entity from markdown, path={path}")
entity = await self.entity_service.upsert_entity_from_markdown(
Path(path), entity_markdown, is_new=new
)
# After updating relations, we need to compute the checksum again
# This is necessary for files with wikilinks to ensure consistent checksums
@@ -0,0 +1,126 @@
"""Integration tests for CLI tool --format json output."""
import json
from typer.testing import CliRunner
from basic_memory.cli.main import app as cli_app
runner = CliRunner()
def test_write_note_json_format(app, app_config, test_project, config_manager):
"""Test write-note --format json returns valid JSON with expected keys."""
result = runner.invoke(
cli_app,
[
"tool",
"write-note",
"--title",
"Integration Test Note",
"--folder",
"test-notes",
"--content",
"# Test\n\nThis is test content.",
"--format",
"json",
],
)
if result.exit_code != 0:
print(f"STDOUT: {result.stdout}")
print(f"STDERR: {result.stderr if hasattr(result, 'stderr') else 'N/A'}")
print(f"Exception: {result.exception}")
assert result.exit_code == 0
data = json.loads(result.stdout)
assert data["title"] == "Integration Test Note"
assert "permalink" in data
assert data["content"] == "# Test\n\nThis is test content."
assert "file_path" in data
def test_read_note_json_format(app, app_config, test_project, config_manager):
"""Test read-note --format json returns valid JSON with expected keys."""
# First, write a note
write_result = runner.invoke(
cli_app,
[
"tool",
"write-note",
"--title",
"Read Test Note",
"--folder",
"test-notes",
"--content",
"# Read Test\n\nContent to read back.",
"--format",
"json",
],
)
assert write_result.exit_code == 0
write_data = json.loads(write_result.stdout)
permalink = write_data["permalink"]
# Now read it back
result = runner.invoke(
cli_app,
["tool", "read-note", permalink, "--format", "json"],
)
if result.exit_code != 0:
print(f"STDOUT: {result.stdout}")
print(f"Exception: {result.exception}")
assert result.exit_code == 0
data = json.loads(result.stdout)
assert data["title"] == "Read Test Note"
assert data["permalink"] == permalink
assert "content" in data
assert "file_path" in data
def test_recent_activity_json_format(app, app_config, test_project, config_manager, monkeypatch):
"""Test recent-activity --format json returns valid JSON list."""
# _recent_activity_json uses resolve_project_parameter which requires either
# default_project_mode=True or BASIC_MEMORY_MCP_PROJECT to resolve a project
monkeypatch.setenv("BASIC_MEMORY_MCP_PROJECT", test_project.name)
# Write a note to ensure there's recent activity
write_result = runner.invoke(
cli_app,
[
"tool",
"write-note",
"--title",
"Activity Test Note",
"--folder",
"test-notes",
"--content",
"# Activity\n\nTest content for activity.",
"--format",
"json",
],
)
assert write_result.exit_code == 0
# Get recent activity
result = runner.invoke(
cli_app,
["tool", "recent-activity", "--format", "json"],
)
if result.exit_code != 0:
print(f"STDOUT: {result.stdout}")
print(f"Exception: {result.exception}")
assert result.exit_code == 0
data = json.loads(result.stdout)
assert isinstance(data, list)
# Should have at least one entity from the note we just wrote
assert len(data) > 0
item = data[0]
assert "title" in item
assert "permalink" in item
assert "file_path" in item
assert "created_at" in item
+55
View File
@@ -0,0 +1,55 @@
"""Smoke test for MCP end-to-end flow."""
import pytest
from fastmcp import Client
@pytest.mark.asyncio
@pytest.mark.smoke
async def test_mcp_smoke_flow(mcp_server, app, test_project):
"""Verify write -> read -> search -> build_context works end-to-end."""
async with Client(mcp_server) as client:
title = "Smoke Test Note"
content = "# Smoke Test Note\n\n- [note] MCP smoke flow"
await client.call_tool(
"write_note",
{
"project": test_project.name,
"title": title,
"directory": "smoke",
"content": content,
"tags": "smoke,test",
},
)
read_result = await client.call_tool(
"read_note",
{
"project": test_project.name,
"identifier": title,
},
)
assert len(read_result.content) == 1
assert title in read_result.content[0].text
search_result = await client.call_tool(
"search_notes",
{
"project": test_project.name,
"query": "Smoke Test Note",
},
)
assert len(search_result.content) == 1
assert title in search_result.content[0].text
context_result = await client.call_tool(
"build_context",
{
"project": test_project.name,
"url": "smoke/*",
},
)
assert len(context_result.content) == 1
assert title in context_result.content[0].text
-40
View File
@@ -1,40 +0,0 @@
"""Tests for knowledge graph API routes."""
from typing import AsyncGenerator
import pytest
import pytest_asyncio
from fastapi import FastAPI
from httpx import AsyncClient, ASGITransport
from basic_memory.deps import get_project_config, get_engine_factory, get_app_config
from basic_memory.models import Project
@pytest_asyncio.fixture
async def app(test_config, engine_factory, app_config) -> FastAPI:
"""Create FastAPI test application."""
from basic_memory.api.app import app
app.dependency_overrides[get_app_config] = lambda: app_config
app.dependency_overrides[get_project_config] = lambda: test_config.project_config
app.dependency_overrides[get_engine_factory] = lambda: engine_factory
return app
@pytest_asyncio.fixture
async def client(app: FastAPI) -> AsyncGenerator[AsyncClient, None]:
"""Create client using ASGI transport - same as CLI will use."""
async with AsyncClient(transport=ASGITransport(app=app), base_url="http://test") as client:
yield client
@pytest.fixture
def project_url(test_project: Project) -> str:
"""Create a URL prefix for the project routes.
This helps tests generate the correct URL for project-scoped routes.
"""
# Make sure this matches what's in tests/conftest.py for test_project creation
# The permalink should be generated from "Test Project Context"
return f"/{test_project.permalink}"
-62
View File
@@ -1,62 +0,0 @@
"""Tests for API container composition root."""
import pytest
from basic_memory.api.container import (
ApiContainer,
get_container,
set_container,
)
from basic_memory.runtime import RuntimeMode
class TestApiContainer:
"""Tests for ApiContainer."""
def test_create_from_config(self, app_config):
"""Container can be created from config manager."""
container = ApiContainer(config=app_config, mode=RuntimeMode.LOCAL)
assert container.config == app_config
assert container.mode == RuntimeMode.LOCAL
def test_should_sync_files_when_enabled_and_not_test(self, app_config):
"""Sync should be enabled when config says so and not in test mode."""
app_config.sync_changes = True
container = ApiContainer(config=app_config, mode=RuntimeMode.LOCAL)
assert container.should_sync_files is True
def test_should_not_sync_files_when_disabled(self, app_config):
"""Sync should be disabled when config says so."""
app_config.sync_changes = False
container = ApiContainer(config=app_config, mode=RuntimeMode.LOCAL)
assert container.should_sync_files is False
def test_should_not_sync_files_in_test_mode(self, app_config):
"""Sync should be disabled in test mode regardless of config."""
app_config.sync_changes = True
container = ApiContainer(config=app_config, mode=RuntimeMode.TEST)
assert container.should_sync_files is False
class TestContainerAccessors:
"""Tests for container get/set functions."""
def test_get_container_raises_when_not_set(self, monkeypatch):
"""get_container raises RuntimeError when container not initialized."""
# Clear any existing container
import basic_memory.api.container as container_module
monkeypatch.setattr(container_module, "_container", None)
with pytest.raises(RuntimeError, match="API container not initialized"):
get_container()
def test_set_and_get_container(self, app_config, monkeypatch):
"""set_container allows get_container to return the container."""
import basic_memory.api.container as container_module
container = ApiContainer(config=app_config, mode=RuntimeMode.LOCAL)
monkeypatch.setattr(container_module, "_container", None)
set_container(container)
assert get_container() is container
-53
View File
@@ -1,53 +0,0 @@
"""Tests for async_client configuration."""
from httpx import AsyncClient, ASGITransport, Timeout
from basic_memory.mcp.async_client import create_client
def test_create_client_uses_asgi_when_no_remote_env(config_manager, monkeypatch):
"""Test that create_client uses ASGI transport when cloud mode is disabled."""
monkeypatch.delenv("BASIC_MEMORY_USE_REMOTE_API", raising=False)
monkeypatch.delenv("BASIC_MEMORY_CLOUD_MODE", raising=False)
cfg = config_manager.load_config()
cfg.cloud_mode = False
config_manager.save_config(cfg)
client = create_client()
assert isinstance(client, AsyncClient)
assert isinstance(client._transport, ASGITransport)
assert str(client.base_url) == "http://test"
def test_create_client_uses_http_when_cloud_mode_env_set(config_manager, monkeypatch):
"""Test that create_client uses HTTP transport when BASIC_MEMORY_CLOUD_MODE is set."""
monkeypatch.setenv("BASIC_MEMORY_CLOUD_MODE", "True")
config = config_manager.load_config()
client = create_client()
assert isinstance(client, AsyncClient)
assert not isinstance(client._transport, ASGITransport)
# Cloud mode uses cloud_host/proxy as base_url
assert str(client.base_url) == f"{config.cloud_host}/proxy/"
def test_create_client_configures_extended_timeouts(config_manager, monkeypatch):
"""Test that create_client configures 30-second timeouts for long operations."""
monkeypatch.delenv("BASIC_MEMORY_USE_REMOTE_API", raising=False)
monkeypatch.delenv("BASIC_MEMORY_CLOUD_MODE", raising=False)
cfg = config_manager.load_config()
cfg.cloud_mode = False
config_manager.save_config(cfg)
client = create_client()
# Verify timeout configuration
assert isinstance(client.timeout, Timeout)
assert client.timeout.connect == 10.0 # 10 seconds for connection
assert client.timeout.read == 30.0 # 30 seconds for reading
assert client.timeout.write == 30.0 # 30 seconds for writing
assert client.timeout.pool == 30.0 # 30 seconds for pool
@@ -1,145 +0,0 @@
"""Tests for the continue_conversation template rendering."""
import datetime
import pytest
from basic_memory.api.template_loader import TemplateLoader
from basic_memory.schemas.memory import EntitySummary
from basic_memory.schemas.search import SearchItemType
@pytest.fixture
def template_loader():
"""Return a TemplateLoader instance for testing."""
return TemplateLoader()
@pytest.fixture
def entity_summary():
"""Create a sample EntitySummary for testing."""
return EntitySummary(
entity_id=1,
title="Test Entity",
permalink="test/entity",
type=SearchItemType.ENTITY,
content="This is a test entity with some content.",
file_path="/path/to/test/entity.md",
created_at=datetime.datetime(2023, 1, 1, 12, 0),
)
@pytest.fixture
def context_with_results(entity_summary):
"""Create a sample context with results for testing."""
from basic_memory.schemas.memory import ObservationSummary, ContextResult
# Create an observation for the entity
observation = ObservationSummary(
observation_id=1,
entity_id=1,
title="Test Observation",
permalink="test/entity/observations/1",
category="test",
content="This is a test observation.",
file_path="/path/to/test/entity.md",
created_at=datetime.datetime(2023, 1, 1, 12, 0),
)
# Create a context result with primary_result, observations, and related_results
context_item = ContextResult(
primary_result=entity_summary,
observations=[observation],
related_results=[entity_summary],
)
return {
"topic": "Test Topic",
"timeframe": "7d",
"has_results": True,
"hierarchical_results": [context_item],
}
@pytest.fixture
def context_without_results():
"""Create a sample context without results for testing."""
return {
"topic": "Empty Topic",
"timeframe": "1d",
"has_results": False,
"hierarchical_results": [],
}
@pytest.mark.asyncio
async def test_continue_conversation_with_results(template_loader, context_with_results):
"""Test rendering the continue_conversation template with results."""
result = await template_loader.render("prompts/continue_conversation.hbs", context_with_results)
# Check that key elements are present
assert "Continuing conversation on: Test Topic" in result
assert "memory://test/entity" in result
assert "Test Entity" in result
assert "This is a test entity with some content." in result
assert "Related Context" in result
assert "read_note" in result
assert "Next Steps" in result
assert "Knowledge Capture Recommendation" in result
@pytest.mark.asyncio
async def test_continue_conversation_without_results(template_loader, context_without_results):
"""Test rendering the continue_conversation template without results."""
result = await template_loader.render(
"prompts/continue_conversation.hbs", context_without_results
)
# Check that key elements are present
assert "Continuing conversation on: Empty Topic" in result
assert "The supplied query did not return any information" in result
assert "Opportunity to Capture New Knowledge!" in result
assert 'title="Empty Topic"' in result
assert "Next Steps" in result
assert "Knowledge Capture Recommendation" in result
@pytest.mark.asyncio
async def test_next_steps_section(template_loader, context_with_results):
"""Test that the next steps section is rendered correctly."""
result = await template_loader.render("prompts/continue_conversation.hbs", context_with_results)
assert "Next Steps" in result
assert 'Explore more with: `search_notes("Test Topic")`' in result
assert (
f'See what\'s changed: `recent_activity(timeframe="{context_with_results["timeframe"]}")`'
in result
)
assert "Record new learnings or decisions from this conversation" in result
@pytest.mark.asyncio
async def test_knowledge_capture_recommendation(template_loader, context_with_results):
"""Test that the knowledge capture recommendation is rendered."""
result = await template_loader.render("prompts/continue_conversation.hbs", context_with_results)
assert "Knowledge Capture Recommendation" in result
assert "actively look for opportunities to:" in result
assert "Record key information, decisions, or insights" in result
assert "Link new knowledge to existing topics" in result
assert "Suggest capturing important context" in result
assert "one of the most valuable aspects of Basic Memory" in result
@pytest.mark.asyncio
async def test_timeframe_default_value(template_loader, context_with_results):
"""Test that the timeframe uses the default value when not provided."""
# Remove the timeframe from the context
context_without_timeframe = context_with_results.copy()
context_without_timeframe["timeframe"] = None
result = await template_loader.render(
"prompts/continue_conversation.hbs", context_without_timeframe
)
# Check that the default value is used
assert 'recent_activity(timeframe="7d")' in result
-212
View File
@@ -1,212 +0,0 @@
"""Tests for the directory router API endpoints."""
import pytest
@pytest.mark.asyncio
async def test_get_directory_tree_endpoint(test_graph, client, project_url):
"""Test the get_directory_tree endpoint returns correctly structured data."""
# Call the endpoint
response = await client.get(f"{project_url}/directory/tree")
# Verify response
assert response.status_code == 200
data = response.json()
# Check that the response is a valid directory tree
assert "name" in data
assert "directory_path" in data
assert "children" in data
assert "type" in data
# The root node should have children
assert isinstance(data["children"], list)
# Root name should be the project name or similar
assert data["name"]
# Root directory_path should be a string
assert isinstance(data["directory_path"], str)
@pytest.mark.asyncio
async def test_get_directory_tree_structure(test_graph, client, project_url):
"""Test the structure of the directory tree returned by the endpoint."""
# Call the endpoint
response = await client.get(f"{project_url}/directory/tree")
# Verify response
assert response.status_code == 200
data = response.json()
# Function to recursively check each node in the tree
def check_node_structure(node):
assert "name" in node
assert "directory_path" in node
assert "children" in node
assert "type" in node
assert isinstance(node["children"], list)
# Check each child recursively
for child in node["children"]:
check_node_structure(child)
# Check the entire tree structure
check_node_structure(data)
@pytest.mark.asyncio
async def test_list_directory_endpoint_default(test_graph, client, project_url):
"""Test the list_directory endpoint with default parameters."""
# Call the endpoint with default parameters
response = await client.get(f"{project_url}/directory/list")
# Verify response
assert response.status_code == 200
data = response.json()
# Should return a list
assert isinstance(data, list)
# With test_graph, should return the "test" directory
assert len(data) == 1
assert data[0]["name"] == "test"
assert data[0]["type"] == "directory"
@pytest.mark.asyncio
async def test_list_directory_endpoint_specific_path(test_graph, client, project_url):
"""Test the list_directory endpoint with specific directory path."""
# Call the endpoint with /test directory
response = await client.get(f"{project_url}/directory/list?dir_name=/test")
# Verify response
assert response.status_code == 200
data = response.json()
# Should return list of files in test directory
assert isinstance(data, list)
assert len(data) == 5
# All should be files (no subdirectories in test_graph)
for item in data:
assert item["type"] == "file"
assert item["name"].endswith(".md")
@pytest.mark.asyncio
async def test_list_directory_endpoint_with_glob(test_graph, client, project_url):
"""Test the list_directory endpoint with glob filtering."""
# Call the endpoint with glob filter
response = await client.get(
f"{project_url}/directory/list?dir_name=/test&file_name_glob=*Connected*"
)
# Verify response
assert response.status_code == 200
data = response.json()
# Should return only Connected Entity files
assert isinstance(data, list)
assert len(data) == 2
file_names = {item["name"] for item in data}
assert file_names == {"Connected Entity 1.md", "Connected Entity 2.md"}
@pytest.mark.asyncio
async def test_list_directory_endpoint_with_depth(test_graph, client, project_url):
"""Test the list_directory endpoint with depth control."""
# Test depth=1 (default)
response_depth_1 = await client.get(f"{project_url}/directory/list?dir_name=/&depth=1")
assert response_depth_1.status_code == 200
data_depth_1 = response_depth_1.json()
assert len(data_depth_1) == 1 # Just the test directory
# Test depth=2 (should include files in test directory)
response_depth_2 = await client.get(f"{project_url}/directory/list?dir_name=/&depth=2")
assert response_depth_2.status_code == 200
data_depth_2 = response_depth_2.json()
assert len(data_depth_2) == 6 # test directory + 5 files
@pytest.mark.asyncio
async def test_list_directory_endpoint_nonexistent_path(test_graph, client, project_url):
"""Test the list_directory endpoint with nonexistent directory."""
# Call the endpoint with nonexistent directory
response = await client.get(f"{project_url}/directory/list?dir_name=/nonexistent")
# Verify response
assert response.status_code == 200
data = response.json()
# Should return empty list
assert isinstance(data, list)
assert len(data) == 0
@pytest.mark.asyncio
async def test_list_directory_endpoint_validation_errors(client, project_url):
"""Test the list_directory endpoint with invalid parameters."""
# Test depth too low
response = await client.get(f"{project_url}/directory/list?depth=0")
assert response.status_code == 422 # Validation error
# Test depth too high
response = await client.get(f"{project_url}/directory/list?depth=11")
assert response.status_code == 422 # Validation error
@pytest.mark.asyncio
async def test_get_directory_structure_endpoint(test_graph, client, project_url):
"""Test the get_directory_structure endpoint returns folders only."""
# Call the endpoint
response = await client.get(f"{project_url}/directory/structure")
# Verify response
assert response.status_code == 200
data = response.json()
# Check that the response is a valid directory tree
assert "name" in data
assert "directory_path" in data
assert "children" in data
assert "type" in data
assert data["type"] == "directory"
# Root should be present
assert data["name"] == "Root"
assert data["directory_path"] == "/"
# Should have the test directory
assert len(data["children"]) == 1
test_dir = data["children"][0]
assert test_dir["name"] == "test"
assert test_dir["type"] == "directory"
assert test_dir["directory_path"] == "/test"
# Should NOT have any files (test_graph has files but no subdirectories)
assert len(test_dir["children"]) == 0
# Verify no file metadata is present in directory nodes
assert test_dir.get("entity_id") is None
assert test_dir.get("content_type") is None
assert test_dir.get("title") is None
assert test_dir.get("permalink") is None
@pytest.mark.asyncio
async def test_get_directory_structure_empty(client, project_url):
"""Test the get_directory_structure endpoint with empty database."""
# Call the endpoint
response = await client.get(f"{project_url}/directory/structure")
# Verify response
assert response.status_code == 200
data = response.json()
# Should return root with no children
assert data["name"] == "Root"
assert data["directory_path"] == "/"
assert data["type"] == "directory"
assert len(data["children"]) == 0
-465
View File
@@ -1,465 +0,0 @@
"""Tests for importer API routes."""
import json
from pathlib import Path
import pytest
from httpx import AsyncClient
from basic_memory.schemas.importer import (
ChatImportResult,
EntityImportResult,
ProjectImportResult,
)
@pytest.fixture
def chatgpt_json_content():
"""Sample ChatGPT conversation data for testing."""
return [
{
"title": "Test Conversation",
"create_time": 1736616594.24054, # Example timestamp
"update_time": 1736616603.164995,
"mapping": {
"root": {"id": "root", "message": None, "parent": None, "children": ["msg1"]},
"msg1": {
"id": "msg1",
"message": {
"id": "msg1",
"author": {"role": "user", "name": None, "metadata": {}},
"create_time": 1736616594.24054,
"content": {
"content_type": "text",
"parts": ["Hello, this is a test message"],
},
"status": "finished_successfully",
"metadata": {},
},
"parent": "root",
"children": ["msg2"],
},
"msg2": {
"id": "msg2",
"message": {
"id": "msg2",
"author": {"role": "assistant", "name": None, "metadata": {}},
"create_time": 1736616603.164995,
"content": {"content_type": "text", "parts": ["This is a test response"]},
"status": "finished_successfully",
"metadata": {},
},
"parent": "msg1",
"children": [],
},
},
}
]
@pytest.fixture
def claude_conversations_json_content():
"""Sample Claude conversations data for testing."""
return [
{
"uuid": "test-uuid",
"name": "Test Conversation",
"created_at": "2025-01-05T20:55:32.499880+00:00",
"updated_at": "2025-01-05T20:56:39.477600+00:00",
"chat_messages": [
{
"uuid": "msg-1",
"text": "Hello, this is a test",
"sender": "human",
"created_at": "2025-01-05T20:55:32.499880+00:00",
"content": [{"type": "text", "text": "Hello, this is a test"}],
},
{
"uuid": "msg-2",
"text": "Response to test",
"sender": "assistant",
"created_at": "2025-01-05T20:55:40.123456+00:00",
"content": [{"type": "text", "text": "Response to test"}],
},
],
}
]
@pytest.fixture
def claude_projects_json_content():
"""Sample Claude projects data for testing."""
return [
{
"uuid": "test-uuid",
"name": "Test Project",
"created_at": "2025-01-05T20:55:32.499880+00:00",
"updated_at": "2025-01-05T20:56:39.477600+00:00",
"prompt_template": "# Test Prompt\n\nThis is a test prompt.",
"docs": [
{
"uuid": "doc-uuid-1",
"filename": "Test Document",
"content": "# Test Document\n\nThis is test content.",
"created_at": "2025-01-05T20:56:39.477600+00:00",
},
{
"uuid": "doc-uuid-2",
"filename": "Another Document",
"content": "# Another Document\n\nMore test content.",
"created_at": "2025-01-05T20:56:39.477600+00:00",
},
],
}
]
@pytest.fixture
def memory_json_content():
"""Sample memory.json data for testing."""
return [
{
"type": "entity",
"name": "test_entity",
"entityType": "test",
"observations": ["Test observation 1", "Test observation 2"],
},
{
"type": "relation",
"from": "test_entity",
"to": "related_entity",
"relationType": "test_relation",
},
]
async def create_test_upload_file(tmp_path, content):
"""Create a test file for upload."""
file_path = tmp_path / "test_import.json"
with open(file_path, "w", encoding="utf-8") as f:
json.dump(content, f)
return file_path
@pytest.mark.asyncio
async def test_import_chatgpt(
project_config, client: AsyncClient, tmp_path, chatgpt_json_content, file_service, project_url
):
"""Test importing ChatGPT conversations."""
# Create a test file
file_path = await create_test_upload_file(tmp_path, chatgpt_json_content)
# Create a multipart form with the file
with open(file_path, "rb") as f:
files = {"file": ("conversations.json", f, "application/json")}
data = {"directory": "test_chatgpt"}
# Send request
response = await client.post(f"{project_url}/import/chatgpt", files=files, data=data)
# Check response
assert response.status_code == 200
result = ChatImportResult.model_validate(response.json())
assert result.success is True
assert result.conversations == 1
assert result.messages == 2
# Verify files were created
conv_path = Path("test_chatgpt") / "20250111-Test_Conversation.md"
assert await file_service.exists(conv_path)
content, _ = await file_service.read_file(conv_path)
assert "# Test Conversation" in content
assert "Hello, this is a test message" in content
assert "This is a test response" in content
@pytest.mark.asyncio
async def test_import_chatgpt_invalid_file(client: AsyncClient, tmp_path, project_url):
"""Test importing invalid ChatGPT file."""
# Create invalid file
file_path = tmp_path / "invalid.json"
with open(file_path, "w") as f:
f.write("This is not JSON")
# Create multipart form with invalid file
with open(file_path, "rb") as f:
files = {"file": ("invalid.json", f, "application/json")}
data = {"directory": "test_chatgpt"}
# Send request - this should return an error
response = await client.post(f"{project_url}/import/chatgpt", files=files, data=data)
# Check response
assert response.status_code == 500
assert "Import failed" in response.json()["detail"]
@pytest.mark.asyncio
async def test_import_claude_conversations(
client: AsyncClient, tmp_path, claude_conversations_json_content, file_service, project_url
):
"""Test importing Claude conversations."""
# Create a test file
file_path = await create_test_upload_file(tmp_path, claude_conversations_json_content)
# Create a multipart form with the file
with open(file_path, "rb") as f:
files = {"file": ("conversations.json", f, "application/json")}
data = {"directory": "test_claude_conversations"}
# Send request
response = await client.post(
f"{project_url}/import/claude/conversations", files=files, data=data
)
# Check response
assert response.status_code == 200
result = ChatImportResult.model_validate(response.json())
assert result.success is True
assert result.conversations == 1
assert result.messages == 2
# Verify files were created
conv_path = Path("test_claude_conversations") / "20250105-Test_Conversation.md"
assert await file_service.exists(conv_path)
content, _ = await file_service.read_file(conv_path)
assert "# Test Conversation" in content
assert "Hello, this is a test" in content
assert "Response to test" in content
@pytest.mark.asyncio
async def test_import_claude_conversations_invalid_file(client: AsyncClient, tmp_path, project_url):
"""Test importing invalid Claude conversations file."""
# Create invalid file
file_path = tmp_path / "invalid.json"
with open(file_path, "w") as f:
f.write("This is not JSON")
# Create multipart form with invalid file
with open(file_path, "rb") as f:
files = {"file": ("invalid.json", f, "application/json")}
data = {"directory": "test_claude_conversations"}
# Send request - this should return an error
response = await client.post(
f"{project_url}/import/claude/conversations", files=files, data=data
)
# Check response
assert response.status_code == 500
assert "Import failed" in response.json()["detail"]
@pytest.mark.asyncio
async def test_import_claude_projects(
client: AsyncClient, tmp_path, claude_projects_json_content, file_service, project_url
):
"""Test importing Claude projects."""
# Create a test file
file_path = await create_test_upload_file(tmp_path, claude_projects_json_content)
# Create a multipart form with the file
with open(file_path, "rb") as f:
files = {"file": ("projects.json", f, "application/json")}
data = {"directory": "test_claude_projects"}
# Send request
response = await client.post(
f"{project_url}/import/claude/projects", files=files, data=data
)
# Check response
assert response.status_code == 200
result = ProjectImportResult.model_validate(response.json())
assert result.success is True
assert result.documents == 2
assert result.prompts == 1
# Verify files were created
project_dir = Path("test_claude_projects") / "Test_Project"
assert await file_service.exists(project_dir / "prompt-template.md")
assert await file_service.exists(project_dir / "docs" / "Test_Document.md")
assert await file_service.exists(project_dir / "docs" / "Another_Document.md")
# Check content
prompt_content, _ = await file_service.read_file(project_dir / "prompt-template.md")
assert "# Test Prompt" in prompt_content
doc_content, _ = await file_service.read_file(project_dir / "docs" / "Test_Document.md")
assert "# Test Document" in doc_content
assert "This is test content" in doc_content
@pytest.mark.asyncio
async def test_import_claude_projects_invalid_file(client: AsyncClient, tmp_path, project_url):
"""Test importing invalid Claude projects file."""
# Create invalid file
file_path = tmp_path / "invalid.json"
with open(file_path, "w") as f:
f.write("This is not JSON")
# Create multipart form with invalid file
with open(file_path, "rb") as f:
files = {"file": ("invalid.json", f, "application/json")}
data = {"directory": "test_claude_projects"}
# Send request - this should return an error
response = await client.post(
f"{project_url}/import/claude/projects", files=files, data=data
)
# Check response
assert response.status_code == 500
assert "Import failed" in response.json()["detail"]
@pytest.mark.asyncio
async def test_import_memory_json(
client: AsyncClient, tmp_path, memory_json_content, file_service, project_url
):
"""Test importing memory.json file."""
# Create a test file
json_file = tmp_path / "memory.json"
with open(json_file, "w", encoding="utf-8") as f:
for entity in memory_json_content:
f.write(json.dumps(entity) + "\n")
# Create a multipart form with the file
with open(json_file, "rb") as f:
files = {"file": ("memory.json", f, "application/json")}
data = {"directory": "test_memory_json"}
# Send request
response = await client.post(f"{project_url}/import/memory-json", files=files, data=data)
# Check response
assert response.status_code == 200
result = EntityImportResult.model_validate(response.json())
assert result.success is True
assert result.entities == 1
assert result.relations == 1
# Verify files were created
entity_path = Path("test_memory_json") / "test" / "test_entity.md"
assert await file_service.exists(entity_path)
# Check content
content, _ = await file_service.read_file(entity_path)
assert "Test observation 1" in content
assert "Test observation 2" in content
assert "test_relation [[related_entity]]" in content
@pytest.mark.asyncio
async def test_import_memory_json_without_folder(
client: AsyncClient, tmp_path, memory_json_content, file_service, project_url
):
"""Test importing memory.json file without specifying a destination folder."""
# Create a test file
json_file = tmp_path / "memory.json"
with open(json_file, "w", encoding="utf-8") as f:
for entity in memory_json_content:
f.write(json.dumps(entity) + "\n")
# Create a multipart form with the file
with open(json_file, "rb") as f:
files = {"file": ("memory.json", f, "application/json")}
# Send request without destination_folder
response = await client.post(f"{project_url}/import/memory-json", files=files)
# Check response
assert response.status_code == 200
result = EntityImportResult.model_validate(response.json())
assert result.success is True
assert result.entities == 1
assert result.relations == 1
# Verify files were created in the root directory
entity_path = Path("conversations") / "test" / "test_entity.md"
assert await file_service.exists(entity_path)
@pytest.mark.asyncio
async def test_import_memory_json_invalid_file(client: AsyncClient, tmp_path, project_url):
"""Test importing invalid memory.json file."""
# Create invalid file
file_path = tmp_path / "invalid.json"
with open(file_path, "w") as f:
f.write("This is not JSON")
# Create multipart form with invalid file
with open(file_path, "rb") as f:
files = {"file": ("invalid.json", f, "application/json")}
data = {"destination_folder": "test_memory_json"}
# Send request - this should return an error
response = await client.post(f"{project_url}/import/memory-json", files=files, data=data)
# Check response
assert response.status_code == 500
assert "Import failed" in response.json()["detail"]
@pytest.mark.asyncio
async def test_import_missing_file(client: AsyncClient, tmp_path, project_url):
"""Test importing with missing file."""
# Send a request without a file
response = await client.post(f"{project_url}/import/chatgpt", data={"directory": "test_folder"})
# Check that the request was rejected
assert response.status_code in [400, 422] # Either bad request or unprocessable entity
@pytest.mark.asyncio
async def test_import_empty_file(client: AsyncClient, tmp_path, project_url):
"""Test importing an empty file."""
# Create an empty file
file_path = tmp_path / "empty.json"
with open(file_path, "w") as f:
f.write("")
# Create multipart form with empty file
with open(file_path, "rb") as f:
files = {"file": ("empty.json", f, "application/json")}
data = {"directory": "test_chatgpt"}
# Send request
response = await client.post(f"{project_url}/import/chatgpt", files=files, data=data)
# Check response
assert response.status_code == 500
assert "Import failed" in response.json()["detail"]
@pytest.mark.asyncio
async def test_import_malformed_json(client: AsyncClient, tmp_path, project_url):
"""Test importing malformed JSON for all import endpoints."""
# Create malformed JSON file
file_path = tmp_path / "malformed.json"
with open(file_path, "w") as f:
f.write('{"incomplete": "json"') # Missing closing brace
# Test all import endpoints
endpoints = [
(f"{project_url}/import/chatgpt", {"directory": "test"}),
(f"{project_url}/import/claude/conversations", {"directory": "test"}),
(f"{project_url}/import/claude/projects", {"base_folder": "test"}),
(f"{project_url}/import/memory-json", {"destination_folder": "test"}),
]
for endpoint, data in endpoints:
# Create multipart form with malformed JSON
with open(file_path, "rb") as f:
files = {"file": ("malformed.json", f, "application/json")}
# Send request
response = await client.post(endpoint, files=files, data=data)
# Check response
assert response.status_code == 500
assert "Import failed" in response.json()["detail"]
File diff suppressed because it is too large Load Diff
-121
View File
@@ -1,121 +0,0 @@
"""Tests for management router API endpoints (minimal mocking).
These endpoints are mostly simple state checks and wiring; we use stub objects
and pytest monkeypatch instead of standard-library mocks.
"""
from __future__ import annotations
import pytest
from fastapi import FastAPI
from basic_memory.api.routers.management_router import (
WatchStatusResponse,
get_watch_status,
start_watch_service,
stop_watch_service,
)
class _Request:
def __init__(self, app: FastAPI):
self.app = app
class _Task:
def __init__(self, *, done: bool):
self._done = done
self.cancel_called = False
def done(self) -> bool:
return self._done
def cancel(self) -> None:
self.cancel_called = True
@pytest.fixture
def app_with_state() -> FastAPI:
app = FastAPI()
app.state.watch_task = None
return app
@pytest.mark.asyncio
async def test_get_watch_status_not_running(app_with_state: FastAPI):
app_with_state.state.watch_task = None
resp = await get_watch_status(_Request(app_with_state))
assert isinstance(resp, WatchStatusResponse)
assert resp.running is False
@pytest.mark.asyncio
async def test_get_watch_status_running(app_with_state: FastAPI):
app_with_state.state.watch_task = _Task(done=False)
resp = await get_watch_status(_Request(app_with_state))
assert resp.running is True
@pytest.mark.asyncio
async def test_start_watch_service_when_not_running(monkeypatch, app_with_state: FastAPI):
app_with_state.state.watch_task = None
created = {"watch_service": None, "task": None}
class _StubWatchService:
def __init__(self, *, app_config, project_repository):
self.app_config = app_config
self.project_repository = project_repository
created["watch_service"] = self
def _create_background_sync_task(sync_service, watch_service):
created["task"] = _Task(done=False)
return created["task"]
# start_watch_service imports these inside the function, so patch at the source modules.
monkeypatch.setattr("basic_memory.sync.WatchService", _StubWatchService)
monkeypatch.setattr(
"basic_memory.sync.background_sync.create_background_sync_task",
_create_background_sync_task,
)
project_repository = object()
sync_service = object()
resp = await start_watch_service(_Request(app_with_state), project_repository, sync_service)
assert resp.running is True
assert app_with_state.state.watch_task is created["task"]
assert created["watch_service"] is not None
assert created["watch_service"].project_repository is project_repository
@pytest.mark.asyncio
async def test_start_watch_service_already_running(monkeypatch, app_with_state: FastAPI):
existing = _Task(done=False)
app_with_state.state.watch_task = existing
def _should_not_be_called(*_args, **_kwargs):
raise AssertionError("create_background_sync_task should not be called if already running")
monkeypatch.setattr(
"basic_memory.sync.background_sync.create_background_sync_task",
_should_not_be_called,
)
resp = await start_watch_service(_Request(app_with_state), object(), object())
assert resp.running is True
assert app_with_state.state.watch_task is existing
@pytest.mark.asyncio
async def test_stop_watch_service_not_running(app_with_state: FastAPI):
app_with_state.state.watch_task = None
resp = await stop_watch_service(_Request(app_with_state))
assert resp.running is False
@pytest.mark.asyncio
async def test_stop_watch_service_already_done(app_with_state: FastAPI):
app_with_state.state.watch_task = _Task(done=True)
resp = await stop_watch_service(_Request(app_with_state))
assert resp.running is False
-146
View File
@@ -1,146 +0,0 @@
"""Tests for memory router endpoints."""
from datetime import datetime
import pytest
from basic_memory.schemas.memory import GraphContext
@pytest.mark.asyncio
async def test_get_memory_context(client, test_graph, project_url):
"""Test getting context from memory URL."""
response = await client.get(f"{project_url}/memory/test/root")
assert response.status_code == 200
context = GraphContext(**response.json())
assert len(context.results) == 1
assert context.results[0].primary_result.permalink == "test/root"
assert len(context.results[0].related_results) > 0
# Verify metadata
assert context.metadata.uri == "test/root"
assert context.metadata.depth == 1 # default depth
assert isinstance(context.metadata.generated_at, datetime)
assert context.metadata.primary_count + context.metadata.related_count > 0
assert context.metadata.total_results is not None # Backwards compatibility field
@pytest.mark.asyncio
async def test_get_memory_context_pagination(client, test_graph, project_url):
"""Test getting context from memory URL."""
response = await client.get(f"{project_url}/memory/test/root?page=1&page_size=1")
assert response.status_code == 200
context = GraphContext(**response.json())
assert len(context.results) == 1
assert context.results[0].primary_result.permalink == "test/root"
assert len(context.results[0].related_results) > 0
# Verify metadata
assert context.metadata.uri == "test/root"
assert context.metadata.depth == 1 # default depth
assert isinstance(context.metadata.generated_at, datetime)
assert context.metadata.primary_count > 0
@pytest.mark.asyncio
async def test_get_memory_context_pattern(client, test_graph, project_url):
"""Test getting context with pattern matching."""
response = await client.get(f"{project_url}/memory/test/*")
assert response.status_code == 200
context = GraphContext(**response.json())
assert len(context.results) > 1 # Should match multiple test/* paths
assert all("test/" in item.primary_result.permalink for item in context.results)
@pytest.mark.asyncio
async def test_get_memory_context_depth(client, test_graph, project_url):
"""Test depth parameter affects relation traversal."""
# With depth=1, should only get immediate connections
response = await client.get(f"{project_url}/memory/test/root?depth=1&max_results=20")
assert response.status_code == 200
context1 = GraphContext(**response.json())
# With depth=2, should get deeper connections
response = await client.get(f"{project_url}/memory/test/root?depth=3&max_results=20")
assert response.status_code == 200
context2 = GraphContext(**response.json())
# Calculate total related items in all result items
total_related1 = sum(len(item.related_results) for item in context1.results)
total_related2 = sum(len(item.related_results) for item in context2.results)
assert total_related2 > total_related1
@pytest.mark.asyncio
async def test_get_memory_context_timeframe(client, test_graph, project_url):
"""Test timeframe parameter filters by date."""
# Recent timeframe
response = await client.get(f"{project_url}/memory/test/root?timeframe=1d")
assert response.status_code == 200
recent = GraphContext(**response.json())
# Longer timeframe
response = await client.get(f"{project_url}/memory/test/root?timeframe=30d")
assert response.status_code == 200
older = GraphContext(**response.json())
# Calculate total related items
total_recent_related = (
sum(len(item.related_results) for item in recent.results) if recent.results else 0
)
total_older_related = (
sum(len(item.related_results) for item in older.results) if older.results else 0
)
assert total_older_related >= total_recent_related
@pytest.mark.asyncio
async def test_not_found(client, project_url):
"""Test handling of non-existent paths."""
response = await client.get(f"{project_url}/memory/test/does-not-exist")
assert response.status_code == 200
context = GraphContext(**response.json())
assert len(context.results) == 0
@pytest.mark.asyncio
async def test_recent_activity(client, test_graph, project_url):
"""Test handling of recent activity."""
response = await client.get(f"{project_url}/memory/recent")
assert response.status_code == 200
context = GraphContext(**response.json())
assert len(context.results) > 0
assert context.metadata.primary_count > 0
@pytest.mark.asyncio
async def test_recent_activity_pagination(client, test_graph, project_url):
"""Test pagination for recent activity."""
response = await client.get(f"{project_url}/memory/recent?page=1&page_size=1")
assert response.status_code == 200
context = GraphContext(**response.json())
assert len(context.results) == 1
assert context.page == 1
assert context.page_size == 1
@pytest.mark.asyncio
async def test_recent_activity_by_type(client, test_graph, project_url):
"""Test filtering recent activity by type."""
response = await client.get(f"{project_url}/memory/recent?type=relation&type=observation")
assert response.status_code == 200
context = GraphContext(**response.json())
assert len(context.results) > 0
# Check for relation and observation types in primary results
primary_types = [item.primary_result.type for item in context.results]
assert "relation" in primary_types or "observation" in primary_types
-843
View File
@@ -1,843 +0,0 @@
"""Tests for the project router API endpoints."""
import tempfile
from pathlib import Path
import pytest
from basic_memory.schemas.project_info import ProjectItem
@pytest.mark.asyncio
async def test_get_project_item(test_graph, client, project_config, test_project, project_url):
"""Test the project item endpoint returns correctly structured data."""
# Set up some test data in the database
# Call the endpoint
response = await client.get(f"{project_url}/project/item")
# Verify response
assert response.status_code == 200
project_info = ProjectItem.model_validate(response.json())
assert project_info.name == test_project.name
assert project_info.path == test_project.path
assert project_info.is_default == test_project.is_default
@pytest.mark.asyncio
async def test_get_project_item_not_found(
test_graph, client, project_config, test_project, project_url
):
"""Test the project item endpoint returns correctly structured data."""
# Set up some test data in the database
# Call the endpoint
response = await client.get("/not-found/project/item")
# Verify response
assert response.status_code == 404
@pytest.mark.asyncio
async def test_get_default_project(test_graph, client, project_config, test_project, project_url):
"""Test the default project item endpoint returns the default project."""
# Set up some test data in the database
# Call the endpoint
response = await client.get("/projects/default")
# Verify response
assert response.status_code == 200
project_info = ProjectItem.model_validate(response.json())
assert project_info.name == test_project.name
assert project_info.path == test_project.path
assert project_info.is_default == test_project.is_default
@pytest.mark.asyncio
async def test_get_project_info_endpoint(test_graph, client, project_config, project_url):
"""Test the project-info endpoint returns correctly structured data."""
# Set up some test data in the database
# Call the endpoint
response = await client.get(f"{project_url}/project/info")
# Verify response
assert response.status_code == 200
data = response.json()
# Check top-level keys
assert "project_name" in data
assert "project_path" in data
assert "available_projects" in data
assert "default_project" in data
assert "statistics" in data
assert "activity" in data
assert "system" in data
# Check statistics
stats = data["statistics"]
assert "total_entities" in stats
assert stats["total_entities"] >= 0
assert "total_observations" in stats
assert stats["total_observations"] >= 0
assert "total_relations" in stats
assert stats["total_relations"] >= 0
# Check activity
activity = data["activity"]
assert "recently_created" in activity
assert "recently_updated" in activity
assert "monthly_growth" in activity
# Check system
system = data["system"]
assert "version" in system
assert "database_path" in system
assert "database_size" in system
assert "timestamp" in system
@pytest.mark.asyncio
async def test_get_project_info_content(test_graph, client, project_config, project_url):
"""Test that project-info contains actual data from the test database."""
# Call the endpoint
response = await client.get(f"{project_url}/project/info")
# Verify response
assert response.status_code == 200
data = response.json()
# Check that test_graph content is reflected in statistics
stats = data["statistics"]
# Our test graph should have at least a few entities
assert stats["total_entities"] > 0
# It should also have some observations
assert stats["total_observations"] > 0
# And relations
assert stats["total_relations"] > 0
# Check that entity types include 'test'
assert "test" in stats["entity_types"] or "entity" in stats["entity_types"]
@pytest.mark.asyncio
async def test_list_projects_endpoint(test_config, test_graph, client, project_config, project_url):
"""Test the list projects endpoint returns correctly structured data."""
# Call the endpoint
response = await client.get("/projects/projects")
# Verify response
assert response.status_code == 200
data = response.json()
# Check that the response contains expected fields
assert "projects" in data
assert "default_project" in data
# Check that projects is a list
assert isinstance(data["projects"], list)
# There should be at least one project (the test project)
assert len(data["projects"]) > 0
# Verify project item structure
if data["projects"]:
project = data["projects"][0]
assert "name" in project
assert "path" in project
assert "is_default" in project
# Default project should be marked
default_project = next((p for p in data["projects"] if p["is_default"]), None)
assert default_project is not None
assert default_project["name"] == data["default_project"]
@pytest.mark.asyncio
async def test_remove_project_endpoint(test_config, client, project_service):
"""Test the remove project endpoint."""
# First create a test project to remove
test_project_name = "test-remove-project"
await project_service.add_project(test_project_name, "/tmp/test-remove-project")
# Verify it exists
project = await project_service.get_project(test_project_name)
assert project is not None
# Remove the project
response = await client.delete(f"/projects/{test_project_name}")
# Verify response
assert response.status_code == 200
data = response.json()
# Check response structure
assert "message" in data
assert "status" in data
assert data["status"] == "success"
assert "old_project" in data
assert data["old_project"]["name"] == test_project_name
# Verify project is actually removed
removed_project = await project_service.get_project(test_project_name)
assert removed_project is None
@pytest.mark.asyncio
async def test_set_default_project_endpoint(test_config, client, project_service):
"""Test the set default project endpoint."""
# Create a test project to set as default
test_project_name = "test-default-project"
await project_service.add_project(test_project_name, "/tmp/test-default-project")
# Set it as default
response = await client.put(f"/projects/{test_project_name}/default")
# Verify response
assert response.status_code == 200
data = response.json()
# Check response structure
assert "message" in data
assert "status" in data
assert data["status"] == "success"
assert "new_project" in data
assert data["new_project"]["name"] == test_project_name
# Verify it's actually set as default
assert project_service.default_project == test_project_name
@pytest.mark.asyncio
async def test_update_project_path_endpoint(test_config, client, project_service, project_url):
"""Test the update project endpoint for changing project path."""
# Create a test project to update
test_project_name = "test-update-project"
with tempfile.TemporaryDirectory() as temp_dir:
test_root = Path(temp_dir)
old_path = test_root / "old-location"
new_path = test_root / "new-location"
await project_service.add_project(test_project_name, str(old_path))
try:
# Verify initial state
project = await project_service.get_project(test_project_name)
assert project is not None
assert Path(project.path) == old_path
# Update the project path
response = await client.patch(
f"{project_url}/project/{test_project_name}", json={"path": str(new_path)}
)
# Verify response
assert response.status_code == 200
data = response.json()
# Check response structure
assert "message" in data
assert "status" in data
assert data["status"] == "success"
assert "old_project" in data
assert "new_project" in data
# Check old project data
assert data["old_project"]["name"] == test_project_name
assert Path(data["old_project"]["path"]) == old_path
# Check new project data
assert data["new_project"]["name"] == test_project_name
assert Path(data["new_project"]["path"]) == new_path
# Verify project was actually updated in database
updated_project = await project_service.get_project(test_project_name)
assert updated_project is not None
assert Path(updated_project.path) == new_path
finally:
# Clean up
try:
await project_service.remove_project(test_project_name)
except Exception:
pass
@pytest.mark.asyncio
async def test_update_project_is_active_endpoint(test_config, client, project_service, project_url):
"""Test the update project endpoint for changing is_active status."""
# Create a test project to update
test_project_name = "test-update-active-project"
test_path = "/tmp/test-update-active"
await project_service.add_project(test_project_name, test_path)
try:
# Update the project is_active status
response = await client.patch(
f"{project_url}/project/{test_project_name}", json={"is_active": False}
)
# Verify response
assert response.status_code == 200
data = response.json()
# Check response structure
assert "message" in data
assert "status" in data
assert data["status"] == "success"
assert f"Project '{test_project_name}' updated successfully" == data["message"]
finally:
# Clean up
try:
await project_service.remove_project(test_project_name)
except Exception:
pass
@pytest.mark.asyncio
async def test_update_project_both_params_endpoint(
test_config, client, project_service, project_url
):
"""Test the update project endpoint with both path and is_active parameters."""
# Create a test project to update
test_project_name = "test-update-both-project"
with tempfile.TemporaryDirectory() as temp_dir:
test_root = Path(temp_dir)
old_path = (test_root / "old-location").as_posix()
new_path = (test_root / "new-location").as_posix()
await project_service.add_project(test_project_name, old_path)
try:
# Update both path and is_active (path should take precedence)
response = await client.patch(
f"{project_url}/project/{test_project_name}",
json={"path": new_path, "is_active": False},
)
# Verify response
assert response.status_code == 200
data = response.json()
# Check that path update was performed (takes precedence)
assert data["new_project"]["path"] == new_path
# Verify project was actually updated in database
updated_project = await project_service.get_project(test_project_name)
assert updated_project is not None
assert updated_project.path == new_path
finally:
# Clean up
try:
await project_service.remove_project(test_project_name)
except Exception:
pass
@pytest.mark.asyncio
async def test_update_project_nonexistent_endpoint(client, project_url, tmp_path):
"""Test the update project endpoint with a nonexistent project."""
# Try to update a project that doesn't exist
# Use tmp_path for cross-platform absolute path compatibility
new_path = str(tmp_path / "new-path")
response = await client.patch(
f"{project_url}/project/nonexistent-project", json={"path": new_path}
)
# Should return 400 error
assert response.status_code == 400
data = response.json()
assert "detail" in data
assert "not found in configuration" in data["detail"]
@pytest.mark.asyncio
async def test_update_project_relative_path_error_endpoint(
test_config, client, project_service, project_url
):
"""Test the update project endpoint with relative path (should fail)."""
# Create a test project to update
test_project_name = "test-update-relative-project"
test_path = "/tmp/test-update-relative"
await project_service.add_project(test_project_name, test_path)
try:
# Try to update with relative path
response = await client.patch(
f"{project_url}/project/{test_project_name}", json={"path": "./relative-path"}
)
# Should return 400 error
assert response.status_code == 400
data = response.json()
assert "detail" in data
assert "Path must be absolute" in data["detail"]
finally:
# Clean up
try:
await project_service.remove_project(test_project_name)
except Exception:
pass
@pytest.mark.asyncio
async def test_update_project_no_params_endpoint(test_config, client, project_service, project_url):
"""Test the update project endpoint with no parameters (should fail)."""
# Create a test project to update
test_project_name = "test-update-no-params-project"
test_path = "/tmp/test-update-no-params"
await project_service.add_project(test_project_name, test_path)
proj_info = await project_service.get_project(test_project_name)
assert proj_info.name == test_project_name
# On Windows the path is prepended with a drive letter
assert test_path in proj_info.path
try:
# Try to update with no parameters
response = await client.patch(f"{project_url}/project/{test_project_name}", json={})
# Should return 200 (no-op)
assert response.status_code == 200
proj_info = await project_service.get_project(test_project_name)
assert proj_info.name == test_project_name
# On Windows the path is prepended with a drive letter
assert test_path in proj_info.path
finally:
# Clean up
try:
await project_service.remove_project(test_project_name)
except Exception:
pass
@pytest.mark.asyncio
async def test_update_project_empty_path_endpoint(
test_config, client, project_service, project_url
):
"""Test the update project endpoint with empty path parameter."""
# Create a test project to update
test_project_name = "test-update-empty-path-project"
test_path = "/tmp/test-update-empty-path"
await project_service.add_project(test_project_name, test_path)
try:
# Try to update with empty/null path - should be treated as no path update
response = await client.patch(
f"{project_url}/project/{test_project_name}", json={"path": None, "is_active": True}
)
# Should succeed and perform is_active update
assert response.status_code == 200
data = response.json()
assert data["status"] == "success"
finally:
# Clean up
try:
await project_service.remove_project(test_project_name)
except Exception:
pass
@pytest.mark.asyncio
async def test_sync_project_endpoint(test_graph, client, project_url):
"""Test the project sync endpoint initiates background sync."""
# Call the sync endpoint
response = await client.post(f"{project_url}/project/sync")
# Verify response
assert response.status_code == 200
data = response.json()
# Check response structure
assert "status" in data
assert "message" in data
assert data["status"] == "sync_started"
assert "Filesystem sync initiated" in data["message"]
@pytest.mark.asyncio
async def test_sync_project_endpoint_with_force_full(test_graph, client, project_url):
"""Test the project sync endpoint with force_full parameter."""
# Call the sync endpoint with force_full=true
response = await client.post(f"{project_url}/project/sync?force_full=true")
# Verify response
assert response.status_code == 200
data = response.json()
# Check response structure
assert "status" in data
assert "message" in data
assert data["status"] == "sync_started"
assert "Filesystem sync initiated" in data["message"]
@pytest.mark.asyncio
async def test_sync_project_endpoint_with_force_full_false(test_graph, client, project_url):
"""Test the project sync endpoint with force_full=false."""
# Call the sync endpoint with force_full=false
response = await client.post(f"{project_url}/project/sync?force_full=false")
# Verify response
assert response.status_code == 200
data = response.json()
# Check response structure
assert "status" in data
assert "message" in data
assert data["status"] == "sync_started"
assert "Filesystem sync initiated" in data["message"]
@pytest.mark.asyncio
async def test_sync_project_endpoint_not_found(client):
"""Test the project sync endpoint with nonexistent project."""
# Call the sync endpoint for a project that doesn't exist
response = await client.post("/nonexistent-project/project/sync")
# Should return 404
assert response.status_code == 404
@pytest.mark.asyncio
async def test_sync_project_endpoint_foreground(test_graph, client, project_url):
"""Test the project sync endpoint with run_in_background=false returns sync report."""
# Call the sync endpoint with run_in_background=false
response = await client.post(f"{project_url}/project/sync?run_in_background=false")
# Verify response
assert response.status_code == 200
data = response.json()
# Check that we get a sync report instead of status message
assert "new" in data
assert "modified" in data
assert "deleted" in data
assert "moves" in data
assert "checksums" in data
assert "skipped_files" in data
assert "total" in data
# Verify these are the right types
assert isinstance(data["new"], list)
assert isinstance(data["modified"], list)
assert isinstance(data["deleted"], list)
assert isinstance(data["moves"], dict)
assert isinstance(data["checksums"], dict)
assert isinstance(data["skipped_files"], list)
assert isinstance(data["total"], int)
@pytest.mark.asyncio
async def test_sync_project_endpoint_foreground_with_force_full(test_graph, client, project_url):
"""Test the project sync endpoint with run_in_background=false and force_full=true."""
# Call the sync endpoint with both parameters
response = await client.post(
f"{project_url}/project/sync?run_in_background=false&force_full=true"
)
# Verify response
assert response.status_code == 200
data = response.json()
# Check that we get a sync report with all expected fields
assert "new" in data
assert "modified" in data
assert "deleted" in data
assert "moves" in data
assert "checksums" in data
assert "skipped_files" in data
assert "total" in data
@pytest.mark.asyncio
async def test_sync_project_endpoint_foreground_with_changes(
test_graph, client, project_config, project_url, tmpdir
):
"""Test foreground sync detects actual file changes."""
# Create a new file in the project directory
import os
from pathlib import Path
test_file = Path(project_config.home) / "new_test_file.md"
test_file.write_text("# New Test File\n\nThis is a test file for sync detection.")
try:
# Call the sync endpoint with run_in_background=false
response = await client.post(f"{project_url}/project/sync?run_in_background=false")
# Verify response
assert response.status_code == 200
data = response.json()
# The sync report should show changes (the new file we created)
assert data["total"] >= 0 # Should have at least detected changes
assert "new" in data
assert "modified" in data
assert "deleted" in data
# At least one of these should have changes
has_changes = len(data["new"]) > 0 or len(data["modified"]) > 0 or len(data["deleted"]) > 0
assert has_changes or data["total"] >= 0 # Either changes detected or empty sync is valid
finally:
# Clean up the test file
if test_file.exists():
os.remove(test_file)
@pytest.mark.asyncio
async def test_remove_default_project_fails(test_config, client, project_service):
"""Test that removing the default project returns an error."""
# Get the current default project
default_project_name = project_service.default_project
# Try to remove the default project
response = await client.delete(f"/projects/{default_project_name}")
# Should return 400 with helpful error message
assert response.status_code == 400
data = response.json()
assert "detail" in data
assert "Cannot delete default project" in data["detail"]
assert default_project_name in data["detail"]
@pytest.mark.asyncio
async def test_remove_default_project_with_alternatives(test_config, client, project_service):
"""Test that error message includes alternative projects when trying to delete default."""
# Get the current default project
default_project_name = project_service.default_project
# Create another project so there are alternatives
test_project_name = "test-alternative-project"
await project_service.add_project(test_project_name, "/tmp/test-alternative")
try:
# Try to remove the default project
response = await client.delete(f"/projects/{default_project_name}")
# Should return 400 with helpful error message including alternatives
assert response.status_code == 400
data = response.json()
assert "detail" in data
assert "Cannot delete default project" in data["detail"]
assert "Set another project as default first" in data["detail"]
assert test_project_name in data["detail"]
finally:
# Clean up
try:
await project_service.remove_project(test_project_name)
except Exception:
pass
@pytest.mark.asyncio
async def test_remove_non_default_project_succeeds(test_config, client, project_service):
"""Test that removing a non-default project succeeds."""
# Create a test project to remove
test_project_name = "test-remove-non-default"
await project_service.add_project(test_project_name, "/tmp/test-remove-non-default")
# Verify it's not the default
assert project_service.default_project != test_project_name
# Remove the project
response = await client.delete(f"/projects/{test_project_name}")
# Should succeed
assert response.status_code == 200
data = response.json()
assert data["status"] == "success"
# Verify project is removed
removed_project = await project_service.get_project(test_project_name)
assert removed_project is None
@pytest.mark.asyncio
async def test_set_nonexistent_project_as_default_fails(test_config, client, project_service):
"""Test that setting a non-existent project as default returns 404."""
# Try to set a project that doesn't exist as default
response = await client.put("/projects/nonexistent-project/default")
# Should return 404
assert response.status_code == 404
data = response.json()
assert "detail" in data
assert "does not exist" in data["detail"]
@pytest.mark.asyncio
async def test_create_project_idempotent_same_path(test_config, client, project_service):
"""Test that creating a project with same name and same path is idempotent."""
# Create a project with platform-independent path
test_project_name = "test-idempotent"
with tempfile.TemporaryDirectory() as temp_dir:
test_project_path = (Path(temp_dir) / "test-idempotent").as_posix()
response1 = await client.post(
"/projects/projects",
json={"name": test_project_name, "path": test_project_path, "set_default": False},
)
# Should succeed with 201 Created
assert response1.status_code == 201
data1 = response1.json()
assert data1["status"] == "success"
assert data1["new_project"]["name"] == test_project_name
# Try to create the same project again with same name and path
response2 = await client.post(
"/projects/projects",
json={"name": test_project_name, "path": test_project_path, "set_default": False},
)
# Should also succeed (idempotent)
assert response2.status_code == 200
data2 = response2.json()
assert data2["status"] == "success"
assert "already exists" in data2["message"]
assert data2["new_project"]["name"] == test_project_name
# Normalize paths for cross-platform comparison
assert Path(data2["new_project"]["path"]).resolve() == Path(test_project_path).resolve()
# Clean up
try:
await project_service.remove_project(test_project_name)
except Exception:
pass
@pytest.mark.asyncio
async def test_create_project_fails_different_path(test_config, client, project_service):
"""Test that creating a project with same name but different path fails."""
# Create a project
test_project_name = "test-path-conflict"
test_project_path1 = "/tmp/test-path-conflict-1"
response1 = await client.post(
"/projects/projects",
json={"name": test_project_name, "path": test_project_path1, "set_default": False},
)
# Should succeed with 201 Created
assert response1.status_code == 201
# Try to create the same project with different path
test_project_path2 = "/tmp/test-path-conflict-2"
response2 = await client.post(
"/projects/projects",
json={"name": test_project_name, "path": test_project_path2, "set_default": False},
)
# Should fail with 400
assert response2.status_code == 400
data2 = response2.json()
assert "detail" in data2
assert "already exists with different path" in data2["detail"]
assert test_project_path1 in data2["detail"]
assert test_project_path2 in data2["detail"]
# Clean up
try:
await project_service.remove_project(test_project_name)
except Exception:
pass
@pytest.mark.asyncio
async def test_remove_project_with_delete_notes_false(test_config, client, project_service):
"""Test that removing a project with delete_notes=False leaves directory intact."""
# Create a test project with actual directory
test_project_name = "test-remove-keep-files"
with tempfile.TemporaryDirectory() as temp_dir:
test_path = Path(temp_dir) / "test-project"
test_path.mkdir()
test_file = test_path / "test.md"
test_file.write_text("# Test Note")
await project_service.add_project(test_project_name, str(test_path))
# Remove the project without deleting files (default)
response = await client.delete(f"/projects/{test_project_name}")
# Verify response
assert response.status_code == 200
data = response.json()
assert data["status"] == "success"
# Verify project is removed from config/db
removed_project = await project_service.get_project(test_project_name)
assert removed_project is None
# Verify directory still exists
assert test_path.exists()
assert test_file.exists()
@pytest.mark.asyncio
async def test_remove_project_with_delete_notes_true(test_config, client, project_service):
"""Test that removing a project with delete_notes=True deletes the directory."""
# Create a test project with actual directory
test_project_name = "test-remove-delete-files"
with tempfile.TemporaryDirectory() as temp_dir:
test_path = Path(temp_dir) / "test-project"
test_path.mkdir()
test_file = test_path / "test.md"
test_file.write_text("# Test Note")
await project_service.add_project(test_project_name, str(test_path))
# Remove the project with delete_notes=True
response = await client.delete(f"/projects/{test_project_name}?delete_notes=true")
# Verify response
assert response.status_code == 200
data = response.json()
assert data["status"] == "success"
# Verify project is removed from config/db
removed_project = await project_service.get_project(test_project_name)
assert removed_project is None
# Verify directory is deleted
assert not test_path.exists()
@pytest.mark.asyncio
async def test_remove_project_delete_notes_nonexistent_directory(
test_config, client, project_service
):
"""Test that removing a project with delete_notes=True handles missing directory gracefully."""
# Create a project pointing to a non-existent path
test_project_name = "test-remove-missing-dir"
test_path = "/tmp/this-directory-does-not-exist-12345"
await project_service.add_project(test_project_name, test_path)
# Remove the project with delete_notes=True (should not fail even if dir doesn't exist)
response = await client.delete(f"/projects/{test_project_name}?delete_notes=true")
# Should succeed
assert response.status_code == 200
data = response.json()
assert data["status"] == "success"
# Verify project is removed
removed_project = await project_service.get_project(test_project_name)
assert removed_project is None
@@ -1,55 +0,0 @@
"""Tests for project router operation endpoints."""
import pytest
@pytest.mark.asyncio
async def test_get_project_info_additional(client, test_graph, project_url):
"""Test additional fields in the project info endpoint."""
# Call the endpoint
response = await client.get(f"{project_url}/project/info")
# Verify response
assert response.status_code == 200
data = response.json()
# Check specific fields we're interested in
assert "available_projects" in data
assert isinstance(data["available_projects"], dict)
# Get a project from the list
for project_name, project_info in data["available_projects"].items():
# Verify project structure
assert "path" in project_info
assert "active" in project_info
assert "is_default" in project_info
break # Just check the first one for structure
@pytest.mark.asyncio
async def test_project_list_additional(client, project_url):
"""Test additional fields in the project list endpoint."""
# Call the endpoint
response = await client.get("/projects/projects")
# Verify response
assert response.status_code == 200
data = response.json()
# Verify projects list structure in more detail
assert "projects" in data
assert len(data["projects"]) > 0
# Verify the default project is identified
default_project = data["default_project"]
assert default_project
# Verify the default_project appears in the projects list and is marked as default
default_in_list = False
for project in data["projects"]:
if project["name"] == default_project:
assert project["is_default"] is True
default_in_list = True
break
assert default_in_list, "Default project should appear in the projects list"
-155
View File
@@ -1,155 +0,0 @@
"""Tests for the prompt router endpoints."""
import pytest
import pytest_asyncio
from httpx import AsyncClient
from basic_memory.services.context_service import ContextService
@pytest_asyncio.fixture
async def context_service(entity_repository, search_service, observation_repository):
"""Create a real context service for testing."""
return ContextService(entity_repository, search_service, observation_repository)
@pytest.mark.asyncio
async def test_continue_conversation_endpoint(
client: AsyncClient,
entity_service,
search_service,
context_service,
entity_repository,
test_graph,
project_url,
):
"""Test the continue_conversation endpoint with real services."""
# Create request data
request_data = {
"topic": "Root", # This should match our test entity in test_graph
"timeframe": "7d",
"depth": 1,
"related_items_limit": 2,
}
# Call the endpoint
response = await client.post(f"{project_url}/prompt/continue-conversation", json=request_data)
# Verify response
assert response.status_code == 200
result = response.json()
assert "prompt" in result
assert "context" in result
# Check content of context
context = result["context"]
assert context["topic"] == "Root"
assert context["timeframe"] == "7d"
assert context["has_results"] is True
assert len(context["hierarchical_results"]) > 0
# Check content of prompt
prompt = result["prompt"]
assert "Continuing conversation on: Root" in prompt
assert "memory retrieval session" in prompt
# Test without topic - should use recent activity
request_data = {"timeframe": "1d", "depth": 1, "related_items_limit": 2}
response = await client.post(f"{project_url}/prompt/continue-conversation", json=request_data)
assert response.status_code == 200
result = response.json()
assert "Recent Activity" in result["context"]["topic"]
@pytest.mark.asyncio
async def test_search_prompt_endpoint(
client: AsyncClient, entity_service, search_service, test_graph, project_url
):
"""Test the search_prompt endpoint with real services."""
# Create request data
request_data = {
"query": "Root", # This should match our test entity
"timeframe": "7d",
}
# Call the endpoint
response = await client.post(f"{project_url}/prompt/search", json=request_data)
# Verify response
assert response.status_code == 200
result = response.json()
assert "prompt" in result
assert "context" in result
# Check content of context
context = result["context"]
assert context["query"] == "Root"
assert context["timeframe"] == "7d"
assert context["has_results"] is True
assert len(context["results"]) > 0
# Check content of prompt
prompt = result["prompt"]
assert 'Search Results for: "Root"' in prompt
assert "This is a memory search session" in prompt
@pytest.mark.asyncio
async def test_search_prompt_no_results(
client: AsyncClient, entity_service, search_service, project_url
):
"""Test the search_prompt endpoint with a query that returns no results."""
# Create request data with a query that shouldn't match anything
request_data = {"query": "NonExistentQuery12345", "timeframe": "7d"}
# Call the endpoint
response = await client.post(f"{project_url}/prompt/search", json=request_data)
# Verify response
assert response.status_code == 200
result = response.json()
# Check content of context
context = result["context"]
assert context["query"] == "NonExistentQuery12345"
assert context["has_results"] is False
assert len(context["results"]) == 0
# Check content of prompt
prompt = result["prompt"]
assert 'Search Results for: "NonExistentQuery12345"' in prompt
assert "I couldn't find any results for this query" in prompt
assert "Opportunity to Capture Knowledge" in prompt
@pytest.mark.asyncio
async def test_error_handling(client: AsyncClient, monkeypatch, project_url):
"""Test error handling in the endpoints by breaking the template loader."""
# Patch the template loader to raise an exception
def mock_render(*args, **kwargs):
raise Exception("Template error")
# Apply the patch
monkeypatch.setattr("basic_memory.api.template_loader.TemplateLoader.render", mock_render)
# Test continue_conversation error handling
response = await client.post(
f"{project_url}/prompt/continue-conversation",
json={"topic": "test error", "timeframe": "7d"},
)
assert response.status_code == 500
assert "detail" in response.json()
assert "Template error" in response.json()["detail"]
# Test search_prompt error handling
response = await client.post(
f"{project_url}/prompt/search", json={"query": "test error", "timeframe": "7d"}
)
assert response.status_code == 500
assert "detail" in response.json()
assert "Template error" in response.json()["detail"]
@@ -1,52 +0,0 @@
"""Test that relation resolution happens in the background."""
import pytest
from basic_memory.api.routers.knowledge_router import resolve_relations_background
@pytest.mark.asyncio
async def test_resolve_relations_background_success():
"""Test that background relation resolution calls sync service correctly."""
class StubSyncService:
def __init__(self) -> None:
self.calls: list[int] = []
async def resolve_relations(self, *, entity_id: int) -> None:
self.calls.append(entity_id)
sync_service = StubSyncService()
entity_id = 123
entity_permalink = "test/entity"
# Call the background function
await resolve_relations_background(sync_service, entity_id, entity_permalink)
# Verify sync service was called with the entity_id
assert sync_service.calls == [entity_id]
@pytest.mark.asyncio
async def test_resolve_relations_background_handles_errors():
"""Test that background relation resolution handles errors gracefully."""
class StubSyncService:
def __init__(self) -> None:
self.calls: list[int] = []
async def resolve_relations(self, *, entity_id: int) -> None:
self.calls.append(entity_id)
raise Exception("Test error")
sync_service = StubSyncService()
entity_id = 123
entity_permalink = "test/entity"
# Call should not raise - errors are logged
await resolve_relations_background(sync_service, entity_id, entity_permalink)
# Verify sync service was called
assert sync_service.calls == [entity_id]
-454
View File
@@ -1,454 +0,0 @@
"""Tests for resource router endpoints."""
import json
from datetime import datetime, timezone
from pathlib import Path
import pytest
from basic_memory.schemas import EntityResponse
from basic_memory.utils import normalize_newlines
@pytest.mark.asyncio
async def test_get_resource_content(client, project_config, entity_repository, project_url):
"""Test getting content by permalink."""
# Create a test file
content = "# Test Content\n\nThis is a test file."
test_file = Path(project_config.home) / "test" / "test.md"
test_file.parent.mkdir(parents=True, exist_ok=True)
test_file.write_text(content)
# Create entity referencing the file
entity = await entity_repository.create(
{
"title": "Test Entity",
"entity_type": "test",
"permalink": "test/test",
"file_path": "test/test.md", # Relative to config.home
"content_type": "text/markdown",
"created_at": datetime.now(timezone.utc),
"updated_at": datetime.now(timezone.utc),
}
)
# Test getting the content
response = await client.get(f"{project_url}/resource/{entity.permalink}")
assert response.status_code == 200
assert response.headers["content-type"] == "text/markdown; charset=utf-8"
assert response.text == normalize_newlines(content)
@pytest.mark.asyncio
async def test_get_resource_pagination(client, project_config, entity_repository, project_url):
"""Test getting content by permalink with pagination."""
# Create a test file
content = "# Test Content\n\nThis is a test file."
test_file = Path(project_config.home) / "test" / "test.md"
test_file.parent.mkdir(parents=True, exist_ok=True)
test_file.write_text(content)
# Create entity referencing the file
entity = await entity_repository.create(
{
"title": "Test Entity",
"entity_type": "test",
"permalink": "test/test",
"file_path": "test/test.md", # Relative to config.home
"content_type": "text/markdown",
"created_at": datetime.now(timezone.utc),
"updated_at": datetime.now(timezone.utc),
}
)
# Test getting the content
response = await client.get(
f"{project_url}/resource/{entity.permalink}", params={"page": 1, "page_size": 1}
)
assert response.status_code == 200
assert response.headers["content-type"] == "text/markdown; charset=utf-8"
assert response.text == normalize_newlines(content)
@pytest.mark.asyncio
async def test_get_resource_by_title(client, project_config, entity_repository, project_url):
"""Test getting content by permalink."""
# Create a test file
content = "# Test Content\n\nThis is a test file."
test_file = Path(project_config.home) / "test" / "test.md"
test_file.parent.mkdir(parents=True, exist_ok=True)
test_file.write_text(content)
# Create entity referencing the file
entity = await entity_repository.create(
{
"title": "Test Entity",
"entity_type": "test",
"permalink": "test/test",
"file_path": "test/test.md", # Relative to config.home
"content_type": "text/markdown",
"created_at": datetime.now(timezone.utc),
"updated_at": datetime.now(timezone.utc),
}
)
# Test getting the content
response = await client.get(f"{project_url}/resource/{entity.title}")
assert response.status_code == 200
@pytest.mark.asyncio
async def test_get_resource_missing_entity(client, project_url):
"""Test 404 when entity doesn't exist."""
response = await client.get(f"{project_url}/resource/does/not/exist")
assert response.status_code == 404
assert "Resource not found" in response.json()["detail"]
@pytest.mark.asyncio
async def test_get_resource_missing_file(client, project_config, entity_repository, project_url):
"""Test 404 when file doesn't exist."""
# Create entity referencing non-existent file
entity = await entity_repository.create(
{
"title": "Missing File",
"entity_type": "test",
"permalink": "test/missing",
"file_path": "test/missing.md",
"content_type": "text/markdown",
"created_at": datetime.now(timezone.utc),
"updated_at": datetime.now(timezone.utc),
}
)
response = await client.get(f"{project_url}/resource/{entity.permalink}")
assert response.status_code == 404
assert "File not found" in response.json()["detail"]
@pytest.mark.asyncio
async def test_get_resource_observation(client, project_config, entity_repository, project_url):
"""Test getting content by observation permalink."""
# Create entity
content = "# Test Content\n\n- [note] an observation."
data = {
"title": "Test Entity",
"directory": "test",
"entity_type": "test",
"content": f"{content}",
}
response = await client.post(f"{project_url}/knowledge/entities", json=data)
entity_response = response.json()
entity = EntityResponse(**entity_response)
assert len(entity.observations) == 1
observation = entity.observations[0]
# Test getting the content via the observation
response = await client.get(f"{project_url}/resource/{observation.permalink}")
assert response.status_code == 200
assert response.headers["content-type"] == "text/markdown; charset=utf-8"
assert (
normalize_newlines(
"""
---
title: Test Entity
type: test
permalink: test/test-entity
---
# Test Content
- [note] an observation.
""".strip()
)
in response.text
)
@pytest.mark.asyncio
async def test_get_resource_entities(client, project_config, entity_repository, project_url):
"""Test getting content by permalink match."""
# Create entity
content1 = "# Test Content\n"
data = {
"title": "Test Entity",
"directory": "test",
"entity_type": "test",
"content": f"{content1}",
}
response = await client.post(f"{project_url}/knowledge/entities", json=data)
entity_response = response.json()
entity1 = EntityResponse(**entity_response)
content2 = "# Related Content\n- links to [[Test Entity]]"
data = {
"title": "Related Entity",
"directory": "test",
"entity_type": "test",
"content": f"{content2}",
}
response = await client.post(f"{project_url}/knowledge/entities", json=data)
entity_response = response.json()
entity2 = EntityResponse(**entity_response)
assert len(entity2.relations) == 1
# Test getting the content via the relation
response = await client.get(f"{project_url}/resource/test/*")
assert response.status_code == 200
assert response.headers["content-type"] == "text/markdown; charset=utf-8"
assert (
normalize_newlines(
f"""
--- memory://test/test-entity {entity1.updated_at.isoformat()} {entity1.checksum[:8]}
# Test Content
--- memory://test/related-entity {entity2.updated_at.isoformat()} {entity2.checksum[:8]}
# Related Content
- links to [[Test Entity]]
""".strip()
)
in response.text
)
@pytest.mark.asyncio
async def test_get_resource_entities_pagination(
client, project_config, entity_repository, project_url, db_backend
):
"""Test getting content by permalink match."""
if db_backend == "postgres":
pytest.skip(
"Pagination differs: relations expand to multiple entities, ordering is undefined"
)
# Create entity
content1 = "# Test Content\n"
data = {
"title": "Test Entity",
"directory": "test",
"entity_type": "test",
"content": f"{content1}",
}
response = await client.post(f"{project_url}/knowledge/entities", json=data)
entity_response = response.json()
entity1 = EntityResponse(**entity_response)
assert entity1
content2 = "# Related Content\n- links to [[Test Entity]]"
data = {
"title": "Related Entity",
"directory": "test",
"entity_type": "test",
"content": f"{content2}",
}
response = await client.post(f"{project_url}/knowledge/entities", json=data)
entity_response = response.json()
entity2 = EntityResponse(**entity_response)
assert len(entity2.relations) == 1
# Test getting second result
response = await client.get(
f"{project_url}/resource/test/*", params={"page": 2, "page_size": 1}
)
assert response.status_code == 200
assert response.headers["content-type"] == "text/markdown; charset=utf-8"
assert (
normalize_newlines(
"""
---
title: Related Entity
type: test
permalink: test/related-entity
---
# Related Content
- links to [[Test Entity]]
""".strip()
)
in response.text
)
@pytest.mark.asyncio
async def test_get_resource_relation(client, project_config, entity_repository, project_url):
"""Test getting content by relation permalink."""
# Create entity
content1 = "# Test Content\n"
data = {
"title": "Test Entity",
"directory": "test",
"entity_type": "test",
"content": f"{content1}",
}
response = await client.post(f"{project_url}/knowledge/entities", json=data)
entity_response = response.json()
entity1 = EntityResponse(**entity_response)
content2 = "# Related Content\n- links to [[Test Entity]]"
data = {
"title": "Related Entity",
"directory": "test",
"entity_type": "test",
"content": f"{content2}",
}
response = await client.post(f"{project_url}/knowledge/entities", json=data)
entity_response = response.json()
entity2 = EntityResponse(**entity_response)
assert len(entity2.relations) == 1
relation = entity2.relations[0]
# Test getting the content via the relation
response = await client.get(f"{project_url}/resource/{relation.permalink}")
assert response.status_code == 200
assert response.headers["content-type"] == "text/markdown; charset=utf-8"
assert (
normalize_newlines(
f"""
--- memory://test/test-entity {entity1.updated_at.isoformat()} {entity1.checksum[:8]}
# Test Content
--- memory://test/related-entity {entity2.updated_at.isoformat()} {entity2.checksum[:8]}
# Related Content
- links to [[Test Entity]]
""".strip()
)
in response.text
)
@pytest.mark.asyncio
async def test_put_resource_new_file(
client, project_config, entity_repository, search_repository, project_url
):
"""Test creating a new file via PUT."""
# Test data
file_path = "visualizations/test.canvas"
canvas_data = {
"nodes": [
{
"id": "node1",
"type": "text",
"text": "Test node content",
"x": 100,
"y": 200,
"width": 400,
"height": 300,
}
],
"edges": [],
}
# Make sure the file doesn't exist yet
full_path = Path(project_config.home) / file_path
if full_path.exists():
full_path.unlink()
# Execute PUT request
response = await client.put(
f"{project_url}/resource/{file_path}", json=json.dumps(canvas_data, indent=2)
)
# Verify response
assert response.status_code == 201
response_data = response.json()
assert response_data["file_path"] == file_path
assert "checksum" in response_data
assert "size" in response_data
# Verify file was created
full_path = Path(project_config.home) / file_path
assert full_path.exists()
# Verify file content
file_content = full_path.read_text(encoding="utf-8")
assert json.loads(file_content) == canvas_data
# Verify entity was created in DB
entity = await entity_repository.get_by_file_path(file_path)
assert entity is not None
assert entity.entity_type == "canvas"
assert entity.content_type == "application/json"
# Verify entity was indexed for search
search_results = await search_repository.search(title="test.canvas")
assert len(search_results) > 0
@pytest.mark.asyncio
async def test_put_resource_update_existing(client, project_config, entity_repository, project_url):
"""Test updating an existing file via PUT."""
# Create an initial file and entity
file_path = "visualizations/update-test.canvas"
full_path = Path(project_config.home) / file_path
full_path.parent.mkdir(parents=True, exist_ok=True)
initial_data = {
"nodes": [
{
"id": "initial",
"type": "text",
"text": "Initial content",
"x": 0,
"y": 0,
"width": 200,
"height": 100,
}
],
"edges": [],
}
full_path.write_text(json.dumps(initial_data))
# Create the initial entity
initial_entity = await entity_repository.create(
{
"title": "update-test.canvas",
"entity_type": "canvas",
"file_path": file_path,
"content_type": "application/json",
"checksum": "initial123",
"created_at": datetime.now(timezone.utc),
"updated_at": datetime.now(timezone.utc),
}
)
# New data for update
updated_data = {
"nodes": [
{
"id": "updated",
"type": "text",
"text": "Updated content",
"x": 100,
"y": 100,
"width": 300,
"height": 200,
}
],
"edges": [],
}
# Execute PUT request to update
response = await client.put(
f"{project_url}/resource/{file_path}", json=json.dumps(updated_data, indent=2)
)
# Verify response
assert response.status_code == 200
# Verify file was updated
updated_content = full_path.read_text(encoding="utf-8")
assert json.loads(updated_content) == updated_data
# Verify entity was updated
updated_entity = await entity_repository.get_by_file_path(file_path)
assert updated_entity.id == initial_entity.id # Same entity, updated
assert updated_entity.checksum != initial_entity.checksum # Checksum changed
-179
View File
@@ -1,179 +0,0 @@
"""Tests for search router."""
from datetime import datetime, timezone
import pytest
import pytest_asyncio
from sqlalchemy import text
from basic_memory import db
from basic_memory.schemas import Entity as EntitySchema
from basic_memory.schemas.search import SearchItemType, SearchResponse
@pytest_asyncio.fixture
async def indexed_entity(full_entity, search_service):
"""Create an entity and index it."""
await search_service.index_entity(full_entity)
return full_entity
@pytest.mark.asyncio
async def test_search_basic(client, indexed_entity, project_url):
"""Test basic text search."""
response = await client.post(f"{project_url}/search/", json={"text": "search"})
assert response.status_code == 200
search_results = SearchResponse.model_validate(response.json())
assert len(search_results.results) == 3
found = False
for r in search_results.results:
if r.type == SearchItemType.ENTITY.value:
assert r.permalink == indexed_entity.permalink
found = True
assert found, "Expected to find indexed entity in results"
@pytest.mark.asyncio
async def test_search_basic_pagination(client, indexed_entity, project_url):
"""Test basic text search."""
response = await client.post(
f"{project_url}/search/?page=3&page_size=1", json={"text": "search"}
)
assert response.status_code == 200
search_results = SearchResponse.model_validate(response.json())
assert len(search_results.results) == 1
assert search_results.current_page == 3
assert search_results.page_size == 1
@pytest.mark.asyncio
async def test_search_with_entity_type_filter(client, indexed_entity, project_url):
"""Test search with type filter."""
# Should find with correct type
response = await client.post(
f"{project_url}/search/",
json={"text": "test", "entity_types": [SearchItemType.ENTITY.value]},
)
assert response.status_code == 200
search_results = SearchResponse.model_validate(response.json())
assert len(search_results.results) > 0
# Should find with relation type
response = await client.post(
f"{project_url}/search/",
json={"text": "test", "entity_types": [SearchItemType.RELATION.value]},
)
assert response.status_code == 200
search_results = SearchResponse.model_validate(response.json())
assert len(search_results.results) == 2
@pytest.mark.asyncio
async def test_search_with_type_filter(client, indexed_entity, project_url):
"""Test search with entity type filter."""
# Should find with correct entity type
response = await client.post(f"{project_url}/search/", json={"text": "test", "types": ["test"]})
assert response.status_code == 200
search_results = SearchResponse.model_validate(response.json())
assert len(search_results.results) == 1
# Should not find with wrong entity type
response = await client.post(f"{project_url}/search/", json={"text": "test", "types": ["note"]})
assert response.status_code == 200
search_results = SearchResponse.model_validate(response.json())
assert len(search_results.results) == 0
@pytest.mark.asyncio
async def test_search_with_date_filter(client, indexed_entity, project_url):
"""Test search with date filter."""
# Should find with past date
past_date = datetime(2020, 1, 1, tzinfo=timezone.utc)
response = await client.post(
f"{project_url}/search/", json={"text": "test", "after_date": past_date.isoformat()}
)
assert response.status_code == 200
search_results = SearchResponse.model_validate(response.json())
# Should not find with future date
future_date = datetime(2030, 1, 1, tzinfo=timezone.utc)
response = await client.post(
f"{project_url}/search/", json={"text": "test", "after_date": future_date.isoformat()}
)
assert response.status_code == 200
search_results = SearchResponse.model_validate(response.json())
assert len(search_results.results) == 0
@pytest.mark.asyncio
async def test_search_empty(search_service, client, project_url):
"""Test search with no matches."""
response = await client.post(f"{project_url}/search/", json={"text": "nonexistent"})
assert response.status_code == 200
search_result = SearchResponse.model_validate(response.json())
assert len(search_result.results) == 0
@pytest.mark.asyncio
async def test_reindex(
client, search_service, entity_service, session_maker, project_url, app_config
):
"""Test reindex endpoint."""
# Skip for Postgres - needs investigation of database connection isolation
from basic_memory.config import DatabaseBackend
if app_config.database_backend == DatabaseBackend.POSTGRES:
pytest.skip("Not yet supported for Postgres - database connection isolation issue")
# Create test entity and document
await entity_service.create_entity(
EntitySchema(
title="TestEntity1",
directory="test",
entity_type="test",
),
)
# Clear search index
async with db.scoped_session(session_maker) as session:
await session.execute(text("DELETE FROM search_index"))
await session.commit()
# Verify nothing is searchable
response = await client.post(f"{project_url}/search/", json={"text": "test"})
search_results = SearchResponse.model_validate(response.json())
assert len(search_results.results) == 0
# Trigger reindex
reindex_response = await client.post(f"{project_url}/search/reindex")
assert reindex_response.status_code == 200
assert reindex_response.json()["status"] == "ok"
# Verify content is searchable again
search_response = await client.post(f"{project_url}/search/", json={"text": "test"})
search_results = SearchResponse.model_validate(search_response.json())
assert len(search_results.results) == 1
@pytest.mark.asyncio
async def test_multiple_filters(client, indexed_entity, project_url):
"""Test search with multiple filters combined."""
response = await client.post(
f"{project_url}/search/",
json={
"text": "test",
"entity_types": [SearchItemType.ENTITY.value],
"types": ["test"],
"after_date": datetime(2020, 1, 1, tzinfo=timezone.utc).isoformat(),
},
)
assert response.status_code == 200
search_result = SearchResponse.model_validate(response.json())
assert len(search_result.results) == 1
result = search_result.results[0]
assert result.permalink == indexed_entity.permalink
assert result.type == SearchItemType.ENTITY.value
assert result.metadata["entity_type"] == "test"
-158
View File
@@ -1,158 +0,0 @@
"""Tests for the search template rendering."""
import datetime
import pytest
from basic_memory.api.template_loader import TemplateLoader
from basic_memory.schemas.search import SearchItemType, SearchResult
@pytest.fixture
def template_loader():
"""Return a TemplateLoader instance for testing."""
return TemplateLoader()
@pytest.fixture
def search_result():
"""Create a sample SearchResult for testing."""
return SearchResult(
title="Test Search Result",
type=SearchItemType.ENTITY,
permalink="test/search-result",
score=0.95,
content="This is a test search result with some content.",
file_path="/path/to/test/search-result.md",
metadata={"created_at": datetime.datetime(2023, 2, 1, 12, 0)},
)
@pytest.fixture
def context_with_results(search_result):
"""Create a sample context with search results."""
return {
"query": "test query",
"timeframe": "30d",
"has_results": True,
"result_count": 1,
"results": [search_result],
}
@pytest.fixture
def context_without_results():
"""Create a sample context without search results."""
return {
"query": "empty query",
"timeframe": None,
"has_results": False,
"result_count": 0,
"results": [],
}
@pytest.mark.asyncio
async def test_search_with_results(template_loader, context_with_results):
"""Test rendering the search template with results."""
result = await template_loader.render("prompts/search.hbs", context_with_results)
# Check that key elements are present
assert 'Search Results for: "test query" (after 30d)' in result
assert "1.0. Test Search Result" in result
assert "Type**: entity" in result
assert "Relevance Score**: 0.95" in result
assert "This is a test search result with some content." in result
assert 'read_note("test/search-result")' in result
assert "Next Steps" in result
assert "Synthesize and Capture Knowledge" in result
@pytest.mark.asyncio
async def test_search_without_results(template_loader, context_without_results):
"""Test rendering the search template without results."""
result = await template_loader.render("prompts/search.hbs", context_without_results)
# Check that key elements are present
assert 'Search Results for: "empty query"' in result
assert "I couldn't find any results for this query." in result
assert "Opportunity to Capture Knowledge!" in result
assert "write_note(" in result
assert 'title="Empty query"' in result
assert "Other Suggestions" in result
@pytest.mark.asyncio
async def test_multiple_search_results(template_loader):
"""Test rendering the search template with multiple results."""
# Create multiple search results
results = []
for i in range(1, 6): # Create 5 results
results.append(
SearchResult(
title=f"Search Result {i}",
type=SearchItemType.ENTITY,
permalink=f"test/result-{i}",
score=1.0 - (i * 0.1), # Decreasing scores
content=f"Content for result {i}",
file_path=f"/path/to/result-{i}.md",
metadata={},
)
)
context = {
"query": "multiple results",
"timeframe": None,
"has_results": True,
"result_count": len(results),
"results": results,
}
result = await template_loader.render("prompts/search.hbs", context)
# Check that all results are rendered
for i in range(1, 6):
assert f"{i}.0. Search Result {i}" in result
assert f"Content for result {i}" in result
assert f'read_note("test/result-{i}")' in result
@pytest.mark.asyncio
async def test_capitalization_in_write_note_template(template_loader, context_with_results):
"""Test that the query is capitalized in the write_note template."""
result = await template_loader.render("prompts/search.hbs", context_with_results)
# The query should be capitalized in the suggested write_note call
assert "Synthesis of Test query Information" in result
@pytest.mark.asyncio
async def test_timeframe_display(template_loader):
"""Test that the timeframe is displayed correctly when present, and not when absent."""
# Context with timeframe
context_with_timeframe = {
"query": "with timeframe",
"timeframe": "7d",
"has_results": True,
"result_count": 0,
"results": [],
}
result_with_timeframe = await template_loader.render(
"prompts/search.hbs", context_with_timeframe
)
assert 'Search Results for: "with timeframe" (after 7d)' in result_with_timeframe
# Context without timeframe
context_without_timeframe = {
"query": "without timeframe",
"timeframe": None,
"has_results": True,
"result_count": 0,
"results": [],
}
result_without_timeframe = await template_loader.render(
"prompts/search.hbs", context_without_timeframe
)
assert 'Search Results for: "without timeframe"' in result_without_timeframe
assert 'Search Results for: "without timeframe" (after' not in result_without_timeframe
-219
View File
@@ -1,219 +0,0 @@
"""Tests for the template loader functionality."""
import datetime
import pytest
from pathlib import Path
from basic_memory.api.template_loader import TemplateLoader
@pytest.fixture
def temp_template_dir(tmpdir):
"""Create a temporary directory for test templates."""
template_dir = tmpdir.mkdir("templates").mkdir("prompts")
return template_dir
@pytest.fixture
def custom_template_loader(temp_template_dir):
"""Return a TemplateLoader instance with a custom template directory."""
return TemplateLoader(str(temp_template_dir))
@pytest.fixture
def simple_template(temp_template_dir):
"""Create a simple test template."""
template_path = temp_template_dir / "simple.hbs"
template_path.write_text("Hello, {{name}}!", encoding="utf-8")
return "simple.hbs"
@pytest.mark.asyncio
async def test_render_simple_template(custom_template_loader, simple_template):
"""Test rendering a simple template."""
context = {"name": "World"}
result = await custom_template_loader.render(simple_template, context)
assert result == "Hello, World!"
@pytest.mark.asyncio
async def test_template_cache(custom_template_loader, simple_template):
"""Test that templates are cached."""
context = {"name": "World"}
# First render, should load template
await custom_template_loader.render(simple_template, context)
# Check that template is in cache
assert simple_template in custom_template_loader.template_cache
# Modify the template file - shouldn't affect the cached version
template_path = Path(custom_template_loader.template_dir) / simple_template
template_path.write_text("Goodbye, {{name}}!", encoding="utf-8")
# Second render, should use cached template
result = await custom_template_loader.render(simple_template, context)
assert result == "Hello, World!"
# Clear cache and render again - should use updated template
custom_template_loader.clear_cache()
assert simple_template not in custom_template_loader.template_cache
result = await custom_template_loader.render(simple_template, context)
assert result == "Goodbye, World!"
@pytest.mark.asyncio
async def test_date_helper(custom_template_loader, temp_template_dir):
# Test date helper
date_path = temp_template_dir / "date.hbs"
date_path.write_text("{{date timestamp}}", encoding="utf-8")
date_result = await custom_template_loader.render(
"date.hbs", {"timestamp": datetime.datetime(2023, 1, 1, 12, 30)}
)
assert "2023-01-01" in date_result
@pytest.mark.asyncio
async def test_default_helper(custom_template_loader, temp_template_dir):
# Test default helper
default_path = temp_template_dir / "default.hbs"
default_path.write_text("{{default null 'default-value'}}", encoding="utf-8")
default_result = await custom_template_loader.render("default.hbs", {"null": None})
assert default_result == "default-value"
@pytest.mark.asyncio
async def test_capitalize_helper(custom_template_loader, temp_template_dir):
# Test capitalize helper
capitalize_path = temp_template_dir / "capitalize.hbs"
capitalize_path.write_text("{{capitalize 'test'}}", encoding="utf-8")
capitalize_result = await custom_template_loader.render("capitalize.hbs", {})
assert capitalize_result == "Test"
@pytest.mark.asyncio
async def test_size_helper(custom_template_loader, temp_template_dir):
# Test size helper
size_path = temp_template_dir / "size.hbs"
size_path.write_text("{{size collection}}", encoding="utf-8")
size_result = await custom_template_loader.render("size.hbs", {"collection": [1, 2, 3]})
assert size_result == "3"
@pytest.mark.asyncio
async def test_json_helper(custom_template_loader, temp_template_dir):
# Test json helper
json_path = temp_template_dir / "json.hbs"
json_path.write_text("{{json data}}", encoding="utf-8")
json_result = await custom_template_loader.render("json.hbs", {"data": {"key": "value"}})
assert json_result == '{"key": "value"}'
@pytest.mark.asyncio
async def test_less_than_helper(custom_template_loader, temp_template_dir):
# Test lt (less than) helper
lt_path = temp_template_dir / "lt.hbs"
lt_path.write_text("{{#if_cond (lt 2 3)}}true{{else}}false{{/if_cond}}", encoding="utf-8")
lt_result = await custom_template_loader.render("lt.hbs", {})
assert lt_result == "true"
@pytest.mark.asyncio
async def test_file_not_found(custom_template_loader):
"""Test that FileNotFoundError is raised when a template doesn't exist."""
with pytest.raises(FileNotFoundError):
await custom_template_loader.render("non_existent_template.hbs", {})
@pytest.mark.asyncio
async def test_extension_handling(custom_template_loader, temp_template_dir):
"""Test that template extensions are handled correctly."""
# Create template with .hbs extension
template_path = temp_template_dir / "test_extension.hbs"
template_path.write_text("Template with extension: {{value}}", encoding="utf-8")
# Test accessing with full extension
result = await custom_template_loader.render("test_extension.hbs", {"value": "works"})
assert result == "Template with extension: works"
# Test accessing without extension
result = await custom_template_loader.render("test_extension", {"value": "also works"})
assert result == "Template with extension: also works"
# Test accessing with wrong extension gets converted
template_path = temp_template_dir / "liquid_template.hbs"
template_path.write_text("Liquid template: {{value}}", encoding="utf-8")
result = await custom_template_loader.render("liquid_template.liquid", {"value": "converted"})
assert result == "Liquid template: converted"
@pytest.mark.asyncio
async def test_dedent_helper(custom_template_loader, temp_template_dir):
"""Test the dedent helper for text blocks."""
dedent_path = temp_template_dir / "dedent.hbs"
# Create a template with indented text blocks
template_content = """Before
{{#dedent}}
This is indented text
with nested indentation
that should be dedented
while preserving relative indentation
{{/dedent}}
After"""
dedent_path.write_text(template_content, encoding="utf-8")
# Render the template
result = await custom_template_loader.render("dedent.hbs", {})
# Print the actual output for debugging
print(f"Dedent helper result: {repr(result)}")
# Check that the indentation is properly removed
assert "This is indented text" in result
assert "with nested indentation" in result
assert "that should be dedented" in result
assert "while preserving relative indentation" in result
assert "Before" in result
assert "After" in result
# Check that relative indentation is preserved
assert result.find("with nested indentation") > result.find("This is indented text")
@pytest.mark.asyncio
async def test_nested_dedent_helper(custom_template_loader, temp_template_dir):
"""Test the dedent helper with nested content."""
dedent_path = temp_template_dir / "nested_dedent.hbs"
# Create a template with nested indented blocks
template_content = """
{{#each items}}
{{#dedent}}
--- Item {{this}}
Details for item {{this}}
- Indented detail 1
- Indented detail 2
{{/dedent}}
{{/each}}"""
dedent_path.write_text(template_content, encoding="utf-8")
# Render the template
result = await custom_template_loader.render("nested_dedent.hbs", {"items": [1, 2]})
# Print the actual output for debugging
print(f"Actual result: {repr(result)}")
# Use a more flexible assertion that checks individual components
# instead of exact string matching
assert "--- Item 1" in result
assert "Details for item 1" in result
assert "- Indented detail 1" in result
assert "--- Item 2" in result
assert "Details for item 2" in result
assert "- Indented detail 2" in result
-203
View File
@@ -1,203 +0,0 @@
"""Tests for additional template loader helpers."""
import pytest
from datetime import datetime
from basic_memory.api.template_loader import TemplateLoader
@pytest.fixture
def temp_template_dir(tmpdir):
"""Create a temporary directory for test templates."""
template_dir = tmpdir.mkdir("templates").mkdir("prompts")
return template_dir
@pytest.fixture
def custom_template_loader(temp_template_dir):
"""Return a TemplateLoader instance with a custom template directory."""
return TemplateLoader(str(temp_template_dir))
@pytest.mark.asyncio
async def test_round_helper(custom_template_loader, temp_template_dir):
"""Test the round helper for number formatting."""
# Create template file
round_path = temp_template_dir / "round.hbs"
round_path.write_text(
"{{round number}} {{round number 0}} {{round number 3}}",
encoding="utf-8",
)
# Test with various values
result = await custom_template_loader.render("round.hbs", {"number": 3.14159})
assert result == "3.14 3.0 3.142" or result == "3.14 3 3.142"
# Test with non-numeric value
result = await custom_template_loader.render("round.hbs", {"number": "not-a-number"})
assert "not-a-number" in result
# Test with insufficient args
empty_path = temp_template_dir / "round_empty.hbs"
empty_path.write_text("{{round}}", encoding="utf-8")
result = await custom_template_loader.render("round_empty.hbs", {})
assert result == ""
@pytest.mark.asyncio
async def test_date_helper_edge_cases(custom_template_loader, temp_template_dir):
"""Test edge cases for the date helper."""
# Create template file
date_path = temp_template_dir / "date_edge.hbs"
date_path.write_text(
"{{date timestamp}} {{date timestamp '%Y'}} {{date string_date}} {{date invalid_date}} {{date}}",
encoding="utf-8",
)
# Test with various values
result = await custom_template_loader.render(
"date_edge.hbs",
{
"timestamp": datetime(2023, 1, 1, 12, 30),
"string_date": "2023-01-01T12:30:00",
"invalid_date": "not-a-date",
},
)
assert "2023-01-01" in result
assert "2023" in result # Custom format
assert "not-a-date" in result # Invalid date passed through
assert result.strip() != "" # Empty date case
@pytest.mark.asyncio
async def test_size_helper_edge_cases(custom_template_loader, temp_template_dir):
"""Test edge cases for the size helper."""
# Create template file
size_path = temp_template_dir / "size_edge.hbs"
size_path.write_text(
"{{size list}} {{size string}} {{size dict}} {{size null}} {{size}}",
encoding="utf-8",
)
# Test with various values
result = await custom_template_loader.render(
"size_edge.hbs",
{
"list": [1, 2, 3, 4, 5],
"string": "hello",
"dict": {"a": 1, "b": 2, "c": 3},
"null": None,
},
)
assert "5" in result # List size
assert "hello".find("5") == -1 # String size should be 5
assert "3" in result # Dict size
assert "0" in result # Null size
assert result.count("0") >= 2 # At least two zeros (null and empty args)
@pytest.mark.asyncio
async def test_math_helper(custom_template_loader, temp_template_dir):
"""Test the math helper for basic arithmetic."""
# Create template file
math_path = temp_template_dir / "math.hbs"
math_path.write_text(
"{{math 5 '+' 3}} {{math 10 '-' 4}} {{math 6 '*' 7}} {{math 20 '/' 5}}",
encoding="utf-8",
)
# Test basic operations
result = await custom_template_loader.render("math.hbs", {})
assert "8" in result # Addition
assert "6" in result # Subtraction
assert "42" in result # Multiplication
assert "4" in result # Division
# Test with invalid operator
invalid_op_path = temp_template_dir / "math_invalid_op.hbs"
invalid_op_path.write_text("{{math 5 'invalid' 3}}", encoding="utf-8")
result = await custom_template_loader.render("math_invalid_op.hbs", {})
assert "Unsupported operator" in result
# Test with invalid numeric values
invalid_num_path = temp_template_dir / "math_invalid_num.hbs"
invalid_num_path.write_text("{{math 'not-a-number' '+' 3}}", encoding="utf-8")
result = await custom_template_loader.render("math_invalid_num.hbs", {})
assert "Math error" in result
# Test with insufficient arguments
insufficient_path = temp_template_dir / "math_insufficient.hbs"
insufficient_path.write_text("{{math 5 '+'}}", encoding="utf-8")
result = await custom_template_loader.render("math_insufficient.hbs", {})
assert "Insufficient arguments" in result
@pytest.mark.asyncio
async def test_if_cond_helper(custom_template_loader, temp_template_dir):
"""Test the if_cond helper for conditionals."""
# Create template file with true condition
if_true_path = temp_template_dir / "if_true.hbs"
if_true_path.write_text(
"{{#if_cond (lt 5 10)}}True condition{{else}}False condition{{/if_cond}}",
encoding="utf-8",
)
# Create template file with false condition
if_false_path = temp_template_dir / "if_false.hbs"
if_false_path.write_text(
"{{#if_cond (lt 15 10)}}True condition{{else}}False condition{{/if_cond}}",
encoding="utf-8",
)
# Test true condition
result = await custom_template_loader.render("if_true.hbs", {})
assert result == "True condition"
# Test false condition
result = await custom_template_loader.render("if_false.hbs", {})
assert result == "False condition"
@pytest.mark.asyncio
async def test_lt_helper_edge_cases(custom_template_loader, temp_template_dir):
"""Test edge cases for the lt (less than) helper."""
# Create template file
lt_path = temp_template_dir / "lt_edge.hbs"
lt_path.write_text(
"{{#if_cond (lt 'a' 'b')}}String LT True{{else}}String LT False{{/if_cond}} "
"{{#if_cond (lt 'z' 'a')}}String LT2 True{{else}}String LT2 False{{/if_cond}} "
"{{#if_cond (lt)}}Missing args True{{else}}Missing args False{{/if_cond}}",
encoding="utf-8",
)
# Test with string values and missing args
result = await custom_template_loader.render("lt_edge.hbs", {})
assert "String LT True" in result # 'a' < 'b' is true
assert "String LT2 False" in result # 'z' < 'a' is false
assert "Missing args False" in result # Missing args should return false
@pytest.mark.asyncio
async def test_dedent_helper_edge_case(custom_template_loader, temp_template_dir):
"""Test an edge case for the dedent helper."""
# Create template with empty dedent block
empty_dedent_path = temp_template_dir / "empty_dedent.hbs"
empty_dedent_path.write_text("{{#dedent}}{{/dedent}}", encoding="utf-8")
# Test empty block
result = await custom_template_loader.render("empty_dedent.hbs", {})
assert result == ""
# Test with complex content including lists
complex_dedent_path = temp_template_dir / "complex_dedent.hbs"
complex_dedent_path.write_text(
"{{#dedent}}\n {{#each items}}\n - {{this}}\n {{/each}}\n{{/dedent}}",
encoding="utf-8",
)
result = await custom_template_loader.render("complex_dedent.hbs", {"items": [1, 2, 3]})
assert "- 1" in result
assert "- 2" in result
assert "- 3" in result
+39 -1
View File
@@ -1,10 +1,48 @@
"""Fixtures for V2 API tests."""
import pytest
from typing import Any, AsyncGenerator
import pytest
import pytest_asyncio
from fastapi import FastAPI
from httpx import AsyncClient, ASGITransport
from basic_memory.deps import get_app_config, get_engine_factory
from basic_memory.deps.services import get_task_scheduler
from basic_memory.models import Project
@pytest_asyncio.fixture
async def app(test_config, engine_factory, app_config) -> FastAPI:
"""Create FastAPI test application."""
from basic_memory.api.app import app
app.dependency_overrides[get_app_config] = lambda: app_config
app.dependency_overrides[get_engine_factory] = lambda: engine_factory
return app
@pytest_asyncio.fixture
async def client(app: FastAPI) -> AsyncGenerator[AsyncClient, None]:
"""Create client using ASGI transport - same as CLI will use."""
async with AsyncClient(transport=ASGITransport(app=app), base_url="http://test") as client:
yield client
@pytest.fixture(autouse=True)
def task_scheduler_spy(app: FastAPI) -> list[dict[str, Any]]:
"""Capture scheduled task specs without executing them."""
scheduled: list[dict[str, Any]] = []
class SchedulerSpy:
def schedule(self, task_name: str, **payload: Any) -> None:
scheduled.append({"task_name": task_name, "payload": payload})
app.dependency_overrides[get_task_scheduler] = lambda: SchedulerSpy()
yield scheduled
app.dependency_overrides.pop(get_task_scheduler, None)
@pytest.fixture
def v2_project_url(test_project: Project) -> str:
"""Create a URL prefix for v2 project-scoped routes using project external_id.
+3 -1
View File
@@ -474,7 +474,9 @@ async def test_import_invalid_project_id(client: AsyncClient, tmp_path, chatgpt_
async def test_import_missing_file(client: AsyncClient, v2_project_url: str):
"""Test importing with missing file via v2 endpoint."""
# Send a request without a file
response = await client.post(f"{v2_project_url}/import/chatgpt", data={"directory": "test_folder"})
response = await client.post(
f"{v2_project_url}/import/chatgpt", data={"directory": "test_folder"}
)
# Check that the request was rejected
assert response.status_code in [400, 422] # Either bad request or unprocessable entity
+154 -3
View File
@@ -1,5 +1,7 @@
"""Tests for V2 knowledge graph API routes (ID-based endpoints)."""
import uuid
import pytest
from httpx import AsyncClient
@@ -78,7 +80,9 @@ async def test_resolve_identifier_no_fuzzy_match(client: AsyncClient, v2_project
@pytest.mark.asyncio
async def test_resolve_identifier_with_source_path_no_fuzzy_match(client: AsyncClient, v2_project_url):
async def test_resolve_identifier_with_source_path_no_fuzzy_match(
client: AsyncClient, v2_project_url
):
"""Test that context-aware resolution also uses strict mode.
Even with source_path for context-aware resolution, nonexistent
@@ -155,7 +159,9 @@ async def test_create_entity(client: AsyncClient, file_service, v2_project_url):
"content": "TestContent for V2",
}
response = await client.post(f"{v2_project_url}/knowledge/entities", json=data)
response = await client.post(
f"{v2_project_url}/knowledge/entities", json=data, params={"fast": False}
)
assert response.status_code == 200
entity = EntityResponseV2.model_validate(response.json())
@@ -175,6 +181,62 @@ async def test_create_entity(client: AsyncClient, file_service, v2_project_url):
assert data["content"] in file_content
@pytest.mark.asyncio
async def test_create_entity_conflict_returns_409(client: AsyncClient, v2_project_url):
"""Test creating a duplicate entity returns 409 Conflict."""
data = {
"title": "TestV2EntityConflict",
"directory": "conflict",
"entity_type": "note",
"content_type": "text/markdown",
"content": "Original content for conflict",
}
response = await client.post(
f"{v2_project_url}/knowledge/entities",
json=data,
params={"fast": False},
)
assert response.status_code == 200
response = await client.post(
f"{v2_project_url}/knowledge/entities",
json=data,
params={"fast": False},
)
assert response.status_code == 409
expected_detail = "Note already exists. Use edit_note to modify it, or delete it first."
assert response.json()["detail"] == expected_detail
@pytest.mark.asyncio
async def test_create_entity_returns_content(client: AsyncClient, file_service, v2_project_url):
"""Test creating an entity always returns file content with frontmatter."""
data = {
"title": "TestContentReturn",
"directory": "test",
"entity_type": "note",
"content_type": "text/markdown",
"content": "Body content for return test",
}
response = await client.post(
f"{v2_project_url}/knowledge/entities",
json=data,
params={"fast": False},
)
assert response.status_code == 200
entity = EntityResponseV2.model_validate(response.json())
# Content should always be populated with frontmatter
assert entity.content is not None
assert "---" in entity.content # frontmatter markers
assert "title: TestContentReturn" in entity.content
assert "type: note" in entity.content
assert "permalink:" in entity.content
assert data["content"] in entity.content
@pytest.mark.asyncio
async def test_create_entity_with_observations_and_relations(
client: AsyncClient, file_service, v2_project_url
@@ -192,7 +254,9 @@ async def test_create_entity_with_observations_and_relations(
""",
}
response = await client.post(f"{v2_project_url}/knowledge/entities", json=data)
response = await client.post(
f"{v2_project_url}/knowledge/entities", json=data, params={"fast": False}
)
assert response.status_code == 200
entity = EntityResponseV2.model_validate(response.json())
@@ -255,6 +319,93 @@ async def test_update_entity_by_id(
assert "Original content" not in file_content
@pytest.mark.asyncio
async def test_update_entity_by_id_fast_does_not_duplicate(
client: AsyncClient, v2_project_url, entity_repository
):
"""Fast PUT updates the existing external_id without creating duplicates."""
create_data = {
"title": "07 - Get Started",
"directory": "docs",
"content": "Original content",
}
response = await client.post(f"{v2_project_url}/knowledge/entities", json=create_data)
assert response.status_code == 200
created_entity = EntityResponseV2.model_validate(response.json())
update_data = {
"title": "07 Get Started",
"directory": "docs",
"content": "Updated content",
}
response = await client.put(
f"{v2_project_url}/knowledge/entities/{created_entity.external_id}",
json=update_data,
)
assert response.status_code == 200
entities = await entity_repository.find_all()
assert len(entities) == 1
assert entities[0].external_id == created_entity.external_id
@pytest.mark.asyncio
async def test_put_entity_fast_returns_minimal_row(
client: AsyncClient, v2_project_url, entity_repository
):
"""Fast PUT returns a minimal row and persists the external_id immediately."""
external_id = str(uuid.uuid4())
update_data = {
"title": "FastPutEntity",
"directory": "test",
"content": """
# FastPutEntity
## Observations
- [note] This should be deferred
- related_to [[AnotherEntity]]
""",
}
response = await client.put(
f"{v2_project_url}/knowledge/entities/{external_id}",
json=update_data,
params={"fast": True},
)
assert response.status_code == 201
created_entity = EntityResponseV2.model_validate(response.json())
assert created_entity.external_id == external_id
assert created_entity.observations == []
assert created_entity.relations == []
db_entity = await entity_repository.get_by_external_id(external_id)
assert db_entity is not None
@pytest.mark.asyncio
async def test_fast_create_schedules_reindex_task(
client: AsyncClient, v2_project_url, task_scheduler_spy
):
"""Fast create should enqueue a background reindex task."""
start_count = len(task_scheduler_spy)
response = await client.post(
f"{v2_project_url}/knowledge/entities",
json={
"title": "TaskScheduledEntity",
"directory": "test",
"content": "Content for task scheduling",
},
params={"fast": True},
)
assert response.status_code == 200
assert len(task_scheduler_spy) == start_count + 1
created_entity = EntityResponseV2.model_validate(response.json())
scheduled = task_scheduler_spy[-1]
assert scheduled["task_name"] == "reindex_entity"
assert scheduled["payload"]["entity_id"] == created_entity.id
@pytest.mark.asyncio
async def test_edit_entity_by_id_append(
client: AsyncClient, file_service, v2_project_url, entity_repository
+26
View File
@@ -340,3 +340,29 @@ async def test_resolve_project_empty_identifier(client: AsyncClient, v2_projects
response = await client.post(f"{v2_projects_url}/resolve", json=resolve_data)
assert response.status_code == 422 # Validation error
# --- Legacy v1 compatibility tests ---
@pytest.mark.asyncio
async def test_legacy_v1_list_projects_endpoint(client: AsyncClient, test_project: Project):
"""Test that the legacy /projects/projects endpoint still works for older CLI versions.
This endpoint was removed when we migrated to v2 but older versions of
basic-memory-cloud CLI still call it for `bm project list`.
Note: The route must be without trailing slash to avoid 307 redirects
that the cloud proxy doesn't follow.
"""
# The legacy v1 endpoint was at /projects/projects (no trailing slash)
response = await client.get("/projects/projects")
assert response.status_code == 200
data = response.json()
assert "projects" in data
assert "default_project" in data
# Verify the test project is in the list
project_names = [p["name"] for p in data["projects"]]
assert test_project.name in project_names
@@ -110,7 +110,7 @@ async def test_cloud_utils_fetch_and_exists_and_create_project(
seen = {"create_payload": None}
async def handler(request: httpx.Request) -> httpx.Response:
if request.method == "GET" and request.url.path == "/proxy/projects/projects":
if request.method == "GET" and request.url.path == "/proxy/v2/projects/":
return httpx.Response(
200,
json={
@@ -121,7 +121,7 @@ async def test_cloud_utils_fetch_and_exists_and_create_project(
},
)
if request.method == "POST" and request.url.path == "/proxy/projects/projects":
if request.method == "POST" and request.url.path == "/proxy/v2/projects/":
# httpx.Request doesn't have .json(); parse bytes payload.
seen["create_payload"] = json.loads(request.content.decode("utf-8"))
return httpx.Response(
+333
View File
@@ -0,0 +1,333 @@
"""Tests for --format json output in CLI tool commands.
Verifies that write-note, read-note, and recent-activity commands
produce valid JSON output when invoked with --format json, and that
the default text format still works via the MCP tool path.
"""
import json
from unittest.mock import AsyncMock, MagicMock, patch
from typer.testing import CliRunner
from basic_memory.cli.main import app as cli_app
runner = CliRunner()
# --- Shared mock data ---
WRITE_NOTE_RESULT = {
"title": "Test Note",
"permalink": "notes/test-note",
"content": "hello world",
"file_path": "notes/Test Note.md",
}
READ_NOTE_RESULT = {
"title": "Test Note",
"permalink": "notes/test-note",
"content": "# Test Note\n\nhello world",
"file_path": "notes/Test Note.md",
}
RECENT_ACTIVITY_RESULT = [
{
"title": "Note A",
"permalink": "notes/note-a",
"file_path": "notes/Note A.md",
"created_at": "2025-01-01 00:00:00",
},
{
"title": "Note B",
"permalink": "notes/note-b",
"file_path": "notes/Note B.md",
"created_at": "2025-01-02 00:00:00",
},
]
def _mock_config_manager():
"""Create a mock ConfigManager that avoids reading real config."""
mock_cm = MagicMock()
mock_cm.config = MagicMock()
mock_cm.default_project = "test-project"
mock_cm.get_project.return_value = ("test-project", "/tmp/test")
return mock_cm
# --- write-note --format json ---
@patch("basic_memory.cli.commands.tool.ConfigManager")
@patch(
"basic_memory.cli.commands.tool._write_note_json",
new_callable=AsyncMock,
return_value=WRITE_NOTE_RESULT,
)
def test_write_note_json_output(mock_write_json, mock_config_cls):
"""write-note --format json outputs valid JSON with expected keys."""
mock_config_cls.return_value = _mock_config_manager()
result = runner.invoke(
cli_app,
[
"tool",
"write-note",
"--title",
"Test Note",
"--folder",
"notes",
"--content",
"hello world",
"--format",
"json",
],
)
assert result.exit_code == 0, f"CLI failed: {result.output}"
data = json.loads(result.output)
assert data["title"] == "Test Note"
assert data["permalink"] == "notes/test-note"
assert data["content"] == "hello world"
assert data["file_path"] == "notes/Test Note.md"
mock_write_json.assert_called_once()
@patch("basic_memory.cli.commands.tool.ConfigManager")
@patch(
"basic_memory.cli.commands.tool.mcp_write_note",
)
def test_write_note_text_output(mock_mcp_write, mock_config_cls):
"""write-note with default text format uses the MCP tool path."""
mock_config_cls.return_value = _mock_config_manager()
# MCP tool .fn returns a formatted string
mock_mcp_write.fn = AsyncMock(return_value="Created note: Test Note")
result = runner.invoke(
cli_app,
[
"tool",
"write-note",
"--title",
"Test Note",
"--folder",
"notes",
"--content",
"hello world",
],
)
assert result.exit_code == 0, f"CLI failed: {result.output}"
assert "Created note: Test Note" in result.output
mock_mcp_write.fn.assert_called_once()
# --- read-note --format json ---
@patch("basic_memory.cli.commands.tool.ConfigManager")
@patch(
"basic_memory.cli.commands.tool._read_note_json",
new_callable=AsyncMock,
return_value=READ_NOTE_RESULT,
)
def test_read_note_json_output(mock_read_json, mock_config_cls):
"""read-note --format json outputs valid JSON with expected keys."""
mock_config_cls.return_value = _mock_config_manager()
result = runner.invoke(
cli_app,
["tool", "read-note", "test-note", "--format", "json"],
)
assert result.exit_code == 0, f"CLI failed: {result.output}"
data = json.loads(result.output)
assert data["title"] == "Test Note"
assert data["permalink"] == "notes/test-note"
assert data["content"] == "# Test Note\n\nhello world"
assert data["file_path"] == "notes/Test Note.md"
mock_read_json.assert_called_once()
@patch("basic_memory.cli.commands.tool.ConfigManager")
@patch(
"basic_memory.cli.commands.tool.mcp_read_note",
)
def test_read_note_text_output(mock_mcp_read, mock_config_cls):
"""read-note with default text format uses the MCP tool path."""
mock_config_cls.return_value = _mock_config_manager()
mock_mcp_read.fn = AsyncMock(return_value="# Test Note\n\nhello world")
result = runner.invoke(
cli_app,
["tool", "read-note", "test-note"],
)
assert result.exit_code == 0, f"CLI failed: {result.output}"
assert "Test Note" in result.output
mock_mcp_read.fn.assert_called_once()
# --- recent-activity --format json ---
@patch(
"basic_memory.cli.commands.tool._recent_activity_json",
new_callable=AsyncMock,
return_value=RECENT_ACTIVITY_RESULT,
)
def test_recent_activity_json_output(mock_recent_json):
"""recent-activity --format json outputs valid JSON list."""
result = runner.invoke(
cli_app,
["tool", "recent-activity", "--format", "json"],
)
assert result.exit_code == 0, f"CLI failed: {result.output}"
data = json.loads(result.output)
assert isinstance(data, list)
assert len(data) == 2
assert data[0]["title"] == "Note A"
assert data[0]["permalink"] == "notes/note-a"
assert data[0]["file_path"] == "notes/Note A.md"
assert data[0]["created_at"] == "2025-01-01 00:00:00"
assert data[1]["title"] == "Note B"
mock_recent_json.assert_called_once()
@patch(
"basic_memory.cli.commands.tool.mcp_recent_activity",
)
def test_recent_activity_text_output(mock_mcp_recent):
"""recent-activity with default text format uses the MCP tool path."""
mock_mcp_recent.fn = AsyncMock(return_value="Recent activity:\n- Note A\n- Note B")
result = runner.invoke(
cli_app,
["tool", "recent-activity"],
)
assert result.exit_code == 0, f"CLI failed: {result.output}"
assert "Recent activity:" in result.output
mock_mcp_recent.fn.assert_called_once()
# --- read-note title fallback ---
@patch("basic_memory.cli.commands.tool.ConfigManager")
@patch(
"basic_memory.cli.commands.tool._read_note_json",
new_callable=AsyncMock,
return_value=READ_NOTE_RESULT,
)
def test_read_note_json_with_plain_title(mock_read_json, mock_config_cls):
"""read-note --format json works with plain titles (not just permalinks)."""
mock_config_cls.return_value = _mock_config_manager()
result = runner.invoke(
cli_app,
["tool", "read-note", "My Note Title", "--format", "json"],
)
assert result.exit_code == 0, f"CLI failed: {result.output}"
data = json.loads(result.output)
assert data["title"] == "Test Note"
# Verify the identifier was passed through
call_args = mock_read_json.call_args
assert call_args[0][0] == "My Note Title" or call_args[1].get("identifier") == "My Note Title"
# --- recent-activity pagination ---
@patch(
"basic_memory.cli.commands.tool._recent_activity_json",
new_callable=AsyncMock,
return_value=RECENT_ACTIVITY_RESULT,
)
def test_recent_activity_json_pagination(mock_recent_json):
"""recent-activity --format json passes --page and --page-size to helper."""
result = runner.invoke(
cli_app,
["tool", "recent-activity", "--format", "json", "--page", "2", "--page-size", "10"],
)
assert result.exit_code == 0, f"CLI failed: {result.output}"
data = json.loads(result.output)
assert isinstance(data, list)
# Verify pagination params were passed through
mock_recent_json.assert_called_once()
call_kwargs = mock_recent_json.call_args
# positional args: type, depth, timeframe, project_name, page, page_size
assert call_kwargs[0][4] == 2 # page
assert call_kwargs[0][5] == 10 # page_size
# --- build-context --format json ---
@patch("basic_memory.cli.commands.tool.ConfigManager")
@patch("basic_memory.cli.commands.tool.mcp_build_context")
def test_build_context_format_json(mock_build_ctx, mock_config_cls):
"""build-context --format json outputs valid JSON."""
mock_config_cls.return_value = _mock_config_manager()
mock_context = MagicMock()
mock_context.model_dump.return_value = {
"primary_results": [],
"related_results": [],
}
mock_build_ctx.fn = AsyncMock(return_value=mock_context)
result = runner.invoke(
cli_app,
["tool", "build-context", "memory://test/topic", "--format", "json"],
)
assert result.exit_code == 0, f"CLI failed: {result.output}"
data = json.loads(result.output)
assert "primary_results" in data
mock_build_ctx.fn.assert_called_once()
@patch("basic_memory.cli.commands.tool.ConfigManager")
@patch("basic_memory.cli.commands.tool.mcp_build_context")
def test_build_context_default_format_is_json(mock_build_ctx, mock_config_cls):
"""build-context defaults to JSON output (backward compatible)."""
mock_config_cls.return_value = _mock_config_manager()
mock_context = MagicMock()
mock_context.model_dump.return_value = {"results": []}
mock_build_ctx.fn = AsyncMock(return_value=mock_context)
result = runner.invoke(
cli_app,
["tool", "build-context", "memory://test/topic"],
)
assert result.exit_code == 0, f"CLI failed: {result.output}"
data = json.loads(result.output)
assert isinstance(data, dict)
# --- Edge cases ---
@patch(
"basic_memory.cli.commands.tool._recent_activity_json",
new_callable=AsyncMock,
return_value=[],
)
def test_recent_activity_json_empty(mock_recent_json):
"""recent-activity --format json handles empty results."""
result = runner.invoke(
cli_app,
["tool", "recent-activity", "--format", "json"],
)
assert result.exit_code == 0, f"CLI failed: {result.output}"
data = json.loads(result.output)
assert data == []
@@ -291,3 +291,97 @@ async def test_parse_valid_file_still_works(tmp_path):
assert result.frontmatter.title == "Valid File"
assert result.frontmatter.type == "knowledge"
assert result.frontmatter.tags == ["test", "valid"]
@pytest.mark.asyncio
async def test_invalid_yaml_does_not_add_metadata_key(tmp_path):
"""Test that invalid YAML doesn't create spurious 'metadata' key in frontmatter (issue #528).
This tests a bug where `frontmatter.Post(content, metadata={})` was used incorrectly.
The `metadata={}` kwarg creates a KEY called "metadata" in the metadata dict,
rather than setting the metadata to an empty dict.
This caused files with invalid YAML to get `metadata: {}` in their frontmatter output,
which is incorrect and confusing.
"""
# Create a file with completely broken YAML that will trigger the fallback path
test_file = tmp_path / "broken_yaml.md"
content = dedent(
"""
---
title: Invalid YAML
this is: [not, valid, yaml
missing: closing bracket
---
# Content
This file has broken YAML frontmatter.
"""
).strip()
test_file.write_text(content)
# Parse the file
parser = EntityParser(tmp_path)
result = await parser.parse_file(test_file)
# The metadata dict should NOT contain a "metadata" key
# This was the bug: frontmatter.Post(content, metadata={}) creates {"metadata": {}}
assert "metadata" not in result.frontmatter.metadata, (
"Frontmatter metadata should not contain a 'metadata' key. "
"This indicates the bug where Post(content, metadata={}) was used incorrectly."
)
# Should still have the expected defaults
assert result.frontmatter.title == "broken_yaml"
assert result.frontmatter.type == "note"
@pytest.mark.asyncio
async def test_frontmatter_roundtrip_preserves_user_metadata(tmp_path):
"""Test that parsing and re-serializing frontmatter preserves user fields (issue #528).
Users reported that after cloud sync, their custom frontmatter fields like 'citekey'
were being lost and replaced with defaults. This test ensures user metadata is preserved.
"""
from basic_memory.file_utils import dump_frontmatter
import frontmatter
# Create a file with user's custom frontmatter (like the bug report)
test_file = tmp_path / "litnote.md"
content = dedent(
"""
---
title: "My Document Title"
type: litnote
tags:
- research
- methodology
citekey: authorTitleYear2024
---
# Content here...
"""
).strip()
test_file.write_text(content)
# Parse the file
parser = EntityParser(tmp_path)
result = await parser.parse_file(test_file)
# User's custom fields should be preserved
assert result.frontmatter.title == "My Document Title"
assert result.frontmatter.type == "litnote" # NOT overwritten to "note"
assert "citekey" in result.frontmatter.metadata
assert result.frontmatter.metadata["citekey"] == "authorTitleYear2024"
# Simulate what write_frontmatter does
post = frontmatter.Post(result.content, **result.frontmatter.metadata)
output = dump_frontmatter(post)
# The output should NOT have duplicate frontmatter or metadata: {} key
assert output.count("---") == 2, (
"Should have exactly one frontmatter block (two --- delimiters)"
)
assert "metadata:" not in output, "Should not have 'metadata:' key in output"
assert "citekey: authorTitleYear2024" in output, "User's citekey should be preserved"
assert "type: litnote" in output, "User's type should be preserved"
+1 -1
View File
@@ -298,7 +298,7 @@ class TestProjectClient:
}
async def mock_call_get(client, url, **kwargs):
assert "/projects/projects" in url
assert "/v2/projects" in url
return mock_response
monkeypatch.setattr(project_mod, "call_get", mock_call_get)
+1
View File
@@ -116,6 +116,7 @@ def test_prompt_context_with_file_path_no_permalink():
# Create a mock context with a file that has no permalink (like a binary file)
test_entity = EntitySummary(
external_id="550e8400-e29b-41d4-a716-446655440000",
entity_id=1,
type="entity",
title="Test File",
+18 -1
View File
@@ -88,4 +88,21 @@ async def test_recent_activity_prompt_passes_correct_params(monkeypatch):
assert captured_kwargs["timeframe"] == "2d"
assert captured_kwargs["project"] == "test-proj"
assert captured_kwargs["type"] == "entity"
assert "type" not in captured_kwargs
@pytest.mark.asyncio
async def test_recent_activity_prompt_defaults_timeframe(monkeypatch):
"""Prompt should fall back to 7d when timeframe omitted or falsy."""
captured_kwargs = {}
async def fake_fn(**kwargs):
captured_kwargs.update(kwargs)
return "## Recent Activity"
monkeypatch.setattr("basic_memory.mcp.prompts.recent_activity.recent_activity.fn", fake_fn)
await recent_activity_prompt.fn(timeframe=None, project=None) # pyright: ignore[reportGeneralTypeIssues]
assert captured_kwargs["timeframe"] == "7d"
assert captured_kwargs["project"] is None

Some files were not shown because too many files have changed in this diff Show More