mirror of
https://github.com/basicmachines-co/basic-memory
synced 2026-06-21 13:47:35 +00:00
Compare commits
36 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 449645b452 | |||
| ec9b2c4046 | |||
| de7f15b7a2 | |||
| 630eeb94ab | |||
| 0eae0e1678 | |||
| 88a5b07b89 | |||
| 59e8a937ee | |||
| c07465d904 | |||
| dfb89e841c | |||
| 00537272c6 | |||
| b057912452 | |||
| 8489a3d37e | |||
| a47c9c021f | |||
| c46d7a6833 | |||
| 343a6e118b | |||
| a0e754b7ae | |||
| 24ca5f6804 | |||
| f1d50c2ba7 | |||
| 8072449a78 | |||
| 45d3f58e4d | |||
| d9c8923148 | |||
| 15bd6b95ef | |||
| 0715dcff3d | |||
| 009e84926d | |||
| 8838571509 | |||
| 530cbac73f | |||
| e3ced49d9d | |||
| 8f962fdd87 | |||
| fbb497f6dc | |||
| 0023e736ab | |||
| 0b2080114b | |||
| 8730067f3a | |||
| e14ba92631 | |||
| 9d98892570 | |||
| 3be4495723 | |||
| 17c0e0a29b |
@@ -1,6 +1,7 @@
|
||||
*.py[cod]
|
||||
__pycache__/
|
||||
.pytest_cache/
|
||||
.testmondata*
|
||||
.coverage
|
||||
htmlcov/
|
||||
|
||||
@@ -55,3 +56,4 @@ claude-output
|
||||
**/.claude/settings.local.json
|
||||
.mcp.json
|
||||
.mcpregistry_*
|
||||
/.testmondata
|
||||
|
||||
@@ -0,0 +1,408 @@
|
||||
# AGENTS.md - Basic Memory Project Guide
|
||||
|
||||
## Project Overview
|
||||
|
||||
Basic Memory is a local-first knowledge management system built on the Model Context Protocol (MCP). It enables
|
||||
bidirectional communication between LLMs (like Claude) and markdown files, creating a personal knowledge graph that can
|
||||
be traversed using links between documents.
|
||||
|
||||
## CODEBASE DEVELOPMENT
|
||||
|
||||
### Project information
|
||||
|
||||
See the [README.md](README.md) file for a project overview.
|
||||
|
||||
### Build and Test Commands
|
||||
|
||||
- Install: `just install` or `pip install -e ".[dev]"`
|
||||
- Run all tests (SQLite + Postgres): `just test`
|
||||
- Run all tests against SQLite: `just test-sqlite`
|
||||
- Run all tests against Postgres: `just test-postgres` (uses testcontainers)
|
||||
- Run unit tests (SQLite): `just test-unit-sqlite`
|
||||
- Run unit tests (Postgres): `just test-unit-postgres`
|
||||
- Run integration tests (SQLite): `just test-int-sqlite`
|
||||
- Run integration tests (Postgres): `just test-int-postgres`
|
||||
- Run impacted tests: `just testmon` (pytest-testmon)
|
||||
- Run MCP smoke test: `just test-smoke`
|
||||
- Fast local loop: `just fast-check`
|
||||
- Local consistency check: `just doctor`
|
||||
- Generate HTML coverage: `just coverage`
|
||||
- Single test: `pytest tests/path/to/test_file.py::test_function_name`
|
||||
- Run benchmarks: `pytest test-int/test_sync_performance_benchmark.py -v -m "benchmark and not slow"`
|
||||
- Lint: `just lint` or `ruff check . --fix`
|
||||
- Type check: `just typecheck` or `uv run pyright`
|
||||
- Format: `just format` or `uv run ruff format .`
|
||||
- Run all code checks: `just check` (runs lint, format, typecheck, test)
|
||||
- Create db migration: `just migration "Your migration message"`
|
||||
- Run development MCP Inspector: `just run-inspector`
|
||||
|
||||
**Note:** Project requires Python 3.12+ (uses type parameter syntax and `type` aliases introduced in 3.12)
|
||||
|
||||
**Postgres Testing:** Uses [testcontainers](https://testcontainers-python.readthedocs.io/) which automatically spins up a Postgres instance in Docker. No manual database setup required - just have Docker running.
|
||||
|
||||
**Doctor Note:** `just doctor` runs with a temporary HOME/config so it won't touch your local Basic Memory settings. It leaves temp dirs in `/tmp` (safe to ignore or remove).
|
||||
|
||||
**Testmon Note:** When no files have changed, `just testmon` may collect 0 tests. That's expected and means no impacted tests were detected.
|
||||
|
||||
### Code/Test/Verify Loop (fast path)
|
||||
|
||||
1) **Code:** make changes.
|
||||
2) **Test:** `just fast-check` (lint/format/typecheck + impacted tests + MCP smoke).
|
||||
3) **Verify:** `just doctor` (end-to-end file ↔ DB loop in a temp project).
|
||||
4) **Full gate (when needed):** `just test` or `just check` for SQLite + Postgres.
|
||||
|
||||
If testmon is “cold,” the first run may be long. Subsequent runs get much faster.
|
||||
|
||||
### Test Structure
|
||||
|
||||
- `tests/` - Unit tests for individual components (mocked, fast)
|
||||
- `test-int/` - Integration tests for real-world scenarios (no mocks, realistic)
|
||||
- Both directories are covered by unified coverage reporting
|
||||
- Benchmark tests in `test-int/` are marked with `@pytest.mark.benchmark`
|
||||
- Slow tests are marked with `@pytest.mark.slow`
|
||||
- Smoke tests are marked with `@pytest.mark.smoke`
|
||||
|
||||
### Code Style Guidelines
|
||||
|
||||
- Line length: 100 characters max
|
||||
- Python 3.12+ with full type annotations (uses type parameters and type aliases)
|
||||
- Format with ruff (consistent styling)
|
||||
- Import order: standard lib, third-party, local imports
|
||||
- Naming: snake_case for functions/variables, PascalCase for classes
|
||||
- Prefer async patterns with SQLAlchemy 2.0
|
||||
- Use Pydantic v2 for data validation and schemas
|
||||
- CLI uses Typer for command structure
|
||||
- API uses FastAPI for endpoints
|
||||
- Follow the repository pattern for data access
|
||||
- Tools communicate to api routers via the httpx ASGI client (in process)
|
||||
|
||||
### Code Change Guidelines
|
||||
|
||||
- **Full file read before edits**: Before editing any file, read it in full first to ensure complete context; partial reads lead to corrupted edits
|
||||
- **Minimize diffs**: Prefer the smallest change that satisfies the request. Avoid unrelated refactors or style rewrites unless necessary for correctness
|
||||
- **No speculative getattr**: Never use `getattr(obj, "attr", default)` when unsure about attribute names. Check the class definition or source code first
|
||||
- **Fail fast**: Write code with fail-fast logic by default. Do not swallow exceptions with errors or warnings
|
||||
- **No fallback logic**: Do not add fallback logic unless explicitly told to and agreed with the user
|
||||
- **No guessing**: Do not say "The issue is..." before you actually know what the issue is. Investigate first.
|
||||
|
||||
### Literate Programming Style
|
||||
|
||||
Code should tell a story. Comments must explain the "why" and narrative flow, not just the "what".
|
||||
|
||||
**Section Headers:**
|
||||
For files with multiple phases of logic, add section headers so the control flow reads like chapters:
|
||||
```python
|
||||
# --- Authentication ---
|
||||
# ... auth logic ...
|
||||
|
||||
# --- Data Validation ---
|
||||
# ... validation logic ...
|
||||
|
||||
# --- Business Logic ---
|
||||
# ... core logic ...
|
||||
```
|
||||
|
||||
**Decision Point Comments:**
|
||||
For conditionals that materially change behavior (gates, fallbacks, retries, feature flags), add comments with:
|
||||
- **Trigger**: what condition causes this branch
|
||||
- **Why**: the rationale (cost, correctness, UX, determinism)
|
||||
- **Outcome**: what changes downstream
|
||||
|
||||
```python
|
||||
# Trigger: project has no active sync watcher
|
||||
# Why: avoid duplicate file system watchers consuming resources
|
||||
# Outcome: starts new watcher, registers in active_watchers dict
|
||||
if project_id not in active_watchers:
|
||||
start_watcher(project_id)
|
||||
```
|
||||
|
||||
**Constraint Comments:**
|
||||
If code exists because of a constraint (async requirements, rate limits, schema compatibility), explain the constraint near the code:
|
||||
```python
|
||||
# SQLite requires WAL mode for concurrent read/write access
|
||||
connection.execute("PRAGMA journal_mode=WAL")
|
||||
```
|
||||
|
||||
**What NOT to Comment:**
|
||||
Avoid comments that restate obvious code:
|
||||
```python
|
||||
# Bad - restates code
|
||||
counter += 1 # increment counter
|
||||
|
||||
# Good - explains why
|
||||
counter += 1 # track retries for backoff calculation
|
||||
```
|
||||
|
||||
### Codebase Architecture
|
||||
|
||||
See [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) for detailed architecture documentation.
|
||||
|
||||
**Directory Structure:**
|
||||
- `/alembic` - Alembic db migrations
|
||||
- `/api` - FastAPI REST endpoints + `container.py` composition root
|
||||
- `/cli` - Typer CLI + `container.py` composition root
|
||||
- `/deps` - Feature-scoped FastAPI dependencies (config, db, projects, repositories, services, importers)
|
||||
- `/importers` - Import functionality for Claude, ChatGPT, and other sources
|
||||
- `/markdown` - Markdown parsing and processing
|
||||
- `/mcp` - MCP server + `container.py` composition root + `clients/` typed API clients
|
||||
- `/models` - SQLAlchemy ORM models
|
||||
- `/repository` - Data access layer
|
||||
- `/schemas` - Pydantic models for validation
|
||||
- `/services` - Business logic layer
|
||||
- `/sync` - File synchronization services + `coordinator.py` for lifecycle management
|
||||
|
||||
**Composition Roots:**
|
||||
Each entrypoint (API, MCP, CLI) has a composition root that:
|
||||
- Reads `ConfigManager` (the only place that reads global config)
|
||||
- Resolves runtime mode via `RuntimeMode` enum (TEST > CLOUD > LOCAL)
|
||||
- Provides dependencies to downstream code explicitly
|
||||
|
||||
**Typed API Clients (MCP):**
|
||||
MCP tools use typed clients in `mcp/clients/` to communicate with the API:
|
||||
- `KnowledgeClient` - Entity CRUD operations
|
||||
- `SearchClient` - Search operations
|
||||
- `MemoryClient` - Context building
|
||||
- `DirectoryClient` - Directory listing
|
||||
- `ResourceClient` - Resource reading
|
||||
- `ProjectClient` - Project management
|
||||
|
||||
Flow: MCP Tool → Typed Client → HTTP API → Router → Service → Repository
|
||||
|
||||
### Development Notes
|
||||
|
||||
- MCP tools are defined in src/basic_memory/mcp/tools/
|
||||
- MCP prompts are defined in src/basic_memory/mcp/prompts/
|
||||
- MCP tools should be atomic, composable operations
|
||||
- Use `textwrap.dedent()` for multi-line string formatting in prompts and tools
|
||||
- MCP Prompts are used to invoke tools and format content with instructions for an LLM
|
||||
- Schema changes require Alembic migrations
|
||||
- SQLite is used for indexing and full text search, files are source of truth
|
||||
- Testing uses pytest with asyncio support (strict mode)
|
||||
- Unit tests (`tests/`) use mocks when necessary; integration tests (`test-int/`) use real implementations
|
||||
- By default, tests run against SQLite (fast, no Docker needed)
|
||||
- Set `BASIC_MEMORY_TEST_POSTGRES=1` to run against Postgres (uses testcontainers - Docker required)
|
||||
- Each test runs in a standalone environment with isolated database and tmp_path directory
|
||||
- CI runs SQLite and Postgres tests in parallel for faster feedback
|
||||
- Performance benchmarks are in `test-int/test_sync_performance_benchmark.py`
|
||||
- Use pytest markers: `@pytest.mark.benchmark` for benchmarks, `@pytest.mark.slow` for slow tests
|
||||
- **Coverage must stay at 100%**: Write tests for new code. Only use `# pragma: no cover` when tests would require excessive mocking (e.g., TYPE_CHECKING blocks, error handlers that need failure injection, runtime-mode-dependent code paths)
|
||||
|
||||
### Async Client Pattern (Important!)
|
||||
|
||||
**All MCP tools and CLI commands use the context manager pattern for HTTP clients:**
|
||||
|
||||
```python
|
||||
from basic_memory.mcp.async_client import get_client
|
||||
|
||||
async def my_mcp_tool():
|
||||
async with get_client() as client:
|
||||
# Use client for API calls
|
||||
response = await call_get(client, "/path")
|
||||
return response
|
||||
```
|
||||
|
||||
**Do NOT use:**
|
||||
- ❌ `from basic_memory.mcp.async_client import client` (deprecated module-level client)
|
||||
- ❌ Manual auth header management
|
||||
- ❌ `inject_auth_header()` (deleted)
|
||||
|
||||
**Key principles:**
|
||||
- Auth happens at client creation, not per-request
|
||||
- Proper resource management via context managers
|
||||
- Supports three modes: Local (ASGI), CLI cloud (HTTP + auth), Cloud app (factory injection)
|
||||
- Factory pattern enables dependency injection for cloud consolidation
|
||||
|
||||
**For cloud app integration:**
|
||||
```python
|
||||
from basic_memory.mcp import async_client
|
||||
|
||||
# Set custom factory before importing tools
|
||||
async_client.set_client_factory(your_custom_factory)
|
||||
```
|
||||
|
||||
See SPEC-16 for full context manager refactor details.
|
||||
|
||||
## BASIC MEMORY PRODUCT USAGE
|
||||
|
||||
### Knowledge Structure
|
||||
|
||||
- Entity: Any concept, document, or idea represented as a markdown file
|
||||
- Observation: A categorized fact about an entity (`- [category] content`)
|
||||
- Relation: A directional link between entities (`- relation_type [[Target]]`)
|
||||
- Frontmatter: YAML metadata at the top of markdown files
|
||||
- Knowledge representation follows precise markdown format:
|
||||
- Observations with [category] prefixes
|
||||
- Relations with WikiLinks [[Entity]]
|
||||
- Frontmatter with metadata
|
||||
|
||||
### Basic Memory Commands
|
||||
|
||||
**Local Commands:**
|
||||
- Check sync status: `basic-memory status`
|
||||
- Doctor check (file <-> DB loop): `basic-memory doctor`
|
||||
- Import from Claude: `basic-memory import claude conversations`
|
||||
- Import from ChatGPT: `basic-memory import chatgpt`
|
||||
- Import from Memory JSON: `basic-memory import memory-json`
|
||||
- Tool access: `basic-memory tool` (provides CLI access to MCP tools)
|
||||
- Continue: `basic-memory tool continue-conversation --topic="search"`
|
||||
|
||||
**Project Management:**
|
||||
- List projects: `basic-memory project list`
|
||||
- Add project: `basic-memory project add "name" ~/path`
|
||||
- Project info: `basic-memory project info`
|
||||
- One-way sync (local -> cloud): `basic-memory project sync`
|
||||
- Bidirectional sync: `basic-memory project bisync`
|
||||
- Integrity check: `basic-memory project check`
|
||||
|
||||
**Cloud Commands (requires subscription):**
|
||||
- Authenticate: `basic-memory cloud login`
|
||||
- Logout: `basic-memory cloud logout`
|
||||
- Check cloud status: `basic-memory cloud status`
|
||||
- Setup cloud sync: `basic-memory cloud setup`
|
||||
- Manage snapshots: `basic-memory cloud snapshot [create|list|delete|show|browse]`
|
||||
- Restore from snapshot: `basic-memory cloud restore <path> --snapshot <id>`
|
||||
|
||||
### MCP Capabilities
|
||||
|
||||
- Basic Memory exposes these MCP tools to LLMs:
|
||||
|
||||
**Content Management:**
|
||||
- `write_note(title, content, directory, tags)` - Create/update markdown notes with semantic observations and relations
|
||||
- `read_note(identifier, page, page_size)` - Read notes by title, permalink, or memory:// URL with knowledge graph awareness
|
||||
- `read_content(path)` - Read raw file content (text, images, binaries) without knowledge graph processing
|
||||
- `view_note(identifier, page, page_size)` - View notes as formatted artifacts for better readability
|
||||
- `edit_note(identifier, operation, content)` - Edit notes incrementally (append, prepend, find/replace, replace_section)
|
||||
- `move_note(identifier, destination_path, is_directory)` - Move notes or directories to new locations, updating database and maintaining links
|
||||
- `delete_note(identifier, is_directory)` - Delete notes or directories from the knowledge base
|
||||
|
||||
**Knowledge Graph Navigation:**
|
||||
- `build_context(url, depth, timeframe)` - Navigate the knowledge graph via memory:// URLs for conversation continuity
|
||||
- `recent_activity(type, depth, timeframe)` - Get recently updated information with specified timeframe (e.g., "1d", "1 week")
|
||||
- `list_directory(dir_name, depth, file_name_glob)` - Browse directory contents with filtering and depth control
|
||||
|
||||
**Search & Discovery:**
|
||||
- `search_notes(query, page, page_size, search_type, types, entity_types, after_date)` - Full-text search across all content with advanced filtering options
|
||||
|
||||
**Project Management:**
|
||||
- `list_memory_projects()` - List all available projects with their status
|
||||
- `create_memory_project(project_name, project_path, set_default)` - Create new Basic Memory projects
|
||||
- `delete_project(project_name)` - Delete a project from configuration
|
||||
|
||||
**Visualization:**
|
||||
- `canvas(nodes, edges, title, directory)` - Generate Obsidian canvas files for knowledge graph visualization
|
||||
|
||||
**ChatGPT-Compatible Tools:**
|
||||
- `search(query)` - Search across knowledge base (OpenAI actions compatible)
|
||||
- `fetch(id)` - Fetch full content of a search result document
|
||||
|
||||
- MCP Prompts for better AI interaction:
|
||||
- `ai_assistant_guide()` - Guidance on effectively using Basic Memory tools for AI assistants
|
||||
- `continue_conversation(topic, timeframe)` - Continue previous conversations with relevant historical context
|
||||
- `search(query, after_date)` - Search with detailed, formatted results for better context understanding
|
||||
- `recent_activity(timeframe)` - View recently changed items with formatted output
|
||||
|
||||
### Cloud Features (v0.15.0+)
|
||||
|
||||
Basic Memory now supports cloud synchronization and storage (requires active subscription):
|
||||
|
||||
**Authentication:**
|
||||
- JWT-based authentication with subscription validation
|
||||
- Secure session management with token refresh
|
||||
- Support for multiple cloud projects
|
||||
|
||||
**Bidirectional Sync:**
|
||||
- rclone bisync integration for two-way synchronization
|
||||
- Conflict resolution and integrity verification
|
||||
- Real-time sync with change detection
|
||||
- Mount/unmount cloud storage for direct file access
|
||||
|
||||
**Cloud Project Management:**
|
||||
- Create and manage projects in the cloud
|
||||
- Toggle between local and cloud modes
|
||||
- Per-project sync configuration
|
||||
- Subscription-based access control
|
||||
|
||||
**Security & Performance:**
|
||||
- Removed .env file loading for improved security
|
||||
- .gitignore integration (respects gitignored files)
|
||||
- WAL mode for SQLite performance
|
||||
- Background relation resolution (non-blocking startup)
|
||||
- API performance optimizations (SPEC-11)
|
||||
|
||||
**CLI Routing Flags:**
|
||||
|
||||
When cloud mode is enabled, CLI commands route to the cloud API by default. Use `--local` and `--cloud` flags to override:
|
||||
|
||||
```bash
|
||||
# Force local routing (ignore cloud mode)
|
||||
basic-memory status --local
|
||||
basic-memory project list --local
|
||||
|
||||
# Force cloud routing (when cloud mode is disabled)
|
||||
basic-memory status --cloud
|
||||
basic-memory project info my-project --cloud
|
||||
```
|
||||
|
||||
Key behaviors:
|
||||
- The local MCP server (`basic-memory mcp`) automatically uses local routing
|
||||
- This allows simultaneous use of local Claude Desktop and cloud-based clients
|
||||
- Some commands (like `project default`, `project sync-config`, `project move`) require `--local` in cloud mode since they modify local configuration
|
||||
- Environment variable `BASIC_MEMORY_FORCE_LOCAL=true` forces local routing globally
|
||||
|
||||
## AI-Human Collaborative Development
|
||||
|
||||
Basic Memory emerged from and enables a new kind of development process that combines human and AI capabilities. Instead
|
||||
of using AI just for code generation, we've developed a true collaborative workflow:
|
||||
|
||||
1. AI (LLM) writes initial implementation based on specifications and context
|
||||
2. Human reviews, runs tests, and commits code with any necessary adjustments
|
||||
3. Knowledge persists across conversations using Basic Memory's knowledge graph
|
||||
4. Development continues seamlessly across different AI sessions with consistent context
|
||||
5. Results improve through iterative collaboration and shared understanding
|
||||
|
||||
This approach has allowed us to tackle more complex challenges and build a more robust system than either humans or AI
|
||||
could achieve independently.
|
||||
|
||||
**Problem-Solving Guidance:**
|
||||
- If a solution isn't working after reasonable effort, suggest alternative approaches
|
||||
- Don't persist with a problematic library or pattern when better alternatives exist
|
||||
- Example: When py-pglite caused cascading test failures, switching to testcontainers-postgres was the right call
|
||||
|
||||
## GitHub Integration
|
||||
|
||||
Basic Memory has taken AI-Human collaboration to the next level by integrating Claude directly into the development workflow through GitHub:
|
||||
|
||||
### GitHub MCP Tools
|
||||
|
||||
Using the GitHub Model Context Protocol server, Claude can now:
|
||||
|
||||
- **Repository Management**:
|
||||
- View repository files and structure
|
||||
- Read file contents
|
||||
- Create new branches
|
||||
- Create and update files
|
||||
|
||||
- **Issue Management**:
|
||||
- Create new issues
|
||||
- Comment on existing issues
|
||||
- Close and update issues
|
||||
- Search across issues
|
||||
|
||||
- **Pull Request Workflow**:
|
||||
- Create pull requests
|
||||
- Review code changes
|
||||
- Add comments to PRs
|
||||
|
||||
This integration enables Claude to participate as a full team member in the development process, not just as a code generation tool. Claude's GitHub account ([bm-claudeai](https://github.com/bm-claudeai)) is a member of the Basic Machines organization with direct contributor access to the codebase.
|
||||
|
||||
### Collaborative Development Process
|
||||
|
||||
With GitHub integration, the development workflow includes:
|
||||
|
||||
1. **Direct code review** - Claude can analyze PRs and provide detailed feedback
|
||||
2. **Contribution tracking** - All of Claude's contributions are properly attributed in the Git history
|
||||
3. **Branch management** - Claude can create feature branches for implementations
|
||||
4. **Documentation maintenance** - Claude can keep documentation updated as the code evolves
|
||||
5. **Code Commits**: ALWAYS sign off commits with `git commit -s`
|
||||
|
||||
This level of integration represents a new paradigm in AI-human collaboration, where the AI assistant becomes a full-fledged team member rather than just a tool for generating code snippets.
|
||||
+104
@@ -1,5 +1,109 @@
|
||||
# CHANGELOG
|
||||
|
||||
## v0.18.4 (2026-02-12)
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
- Use global `--header` flag for Tigris consistency on all rclone transactions
|
||||
([`0eae0e1`](https://github.com/basicmachines-co/basic-memory/commit/0eae0e1))
|
||||
- `--header-download` / `--header-upload` only apply to GET/PUT requests, missing S3
|
||||
ListObjectsV2 calls that bisync issues first. Non-US users saw stale edge-cached metadata.
|
||||
- `--header` applies to ALL HTTP transactions (list, download, upload), fixing bisync for
|
||||
users outside the Tigris origin region.
|
||||
|
||||
## v0.18.2 (2026-02-11)
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
- **#562**: Use VIRTUAL instead of STORED columns in SQLite migration
|
||||
([`344e651`](https://github.com/basicmachines-co/basic-memory/commit/344e651))
|
||||
- Fixes compatibility issue with SQLite STORED generated columns
|
||||
|
||||
## v0.18.1 (2026-02-11)
|
||||
|
||||
### Features
|
||||
|
||||
- **#552**: Add `--format json` to CLI tool commands
|
||||
([`a47c9c0`](https://github.com/basicmachines-co/basic-memory/commit/a47c9c0))
|
||||
- CLI tool commands now support `--format json` for machine-readable output
|
||||
|
||||
- **#535**: Support `tag:` query shorthand in search
|
||||
([`f1d50c2`](https://github.com/basicmachines-co/basic-memory/commit/f1d50c2))
|
||||
- Use `tag:mytag` as a convenient shorthand in search queries
|
||||
|
||||
- **#532**: Fast edit entities, refactors for webui, enhanced search
|
||||
([`530cbac`](https://github.com/basicmachines-co/basic-memory/commit/530cbac))
|
||||
- Performance improvements for entity editing and search operations
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
- **#558**: Add X-Tigris-Consistent headers to all rclone commands
|
||||
([`8489a3d`](https://github.com/basicmachines-co/basic-memory/commit/8489a3d))
|
||||
- Ensures consistent reads from Tigris object storage during sync
|
||||
|
||||
- **#541**: Handle EntityCreationError as conflict
|
||||
([`343a6e1`](https://github.com/basicmachines-co/basic-memory/commit/343a6e1))
|
||||
|
||||
- **#536**: Stabilize metadata filters on Postgres
|
||||
([`009e849`](https://github.com/basicmachines-co/basic-memory/commit/009e849))
|
||||
|
||||
- **#533**: Fix recent_activity prompt defaults
|
||||
([`24ca5f6`](https://github.com/basicmachines-co/basic-memory/commit/24ca5f6))
|
||||
|
||||
- **#530**: Prevent spurious `metadata: {}` in frontmatter output
|
||||
([`e3ced49`](https://github.com/basicmachines-co/basic-memory/commit/e3ced49))
|
||||
|
||||
- Add POST legacy compat routes for v0.18.0 CLI
|
||||
([`c46d7a6`](https://github.com/basicmachines-co/basic-memory/commit/c46d7a6))
|
||||
|
||||
- Restore legacy `/projects/projects` endpoint for older CLI versions
|
||||
([`a0e754b`](https://github.com/basicmachines-co/basic-memory/commit/a0e754b))
|
||||
|
||||
### Internal
|
||||
|
||||
- **#538**: Add fast feedback loop tooling (`just fast-check`, `just doctor`, `just testmon`)
|
||||
([`8072449`](https://github.com/basicmachines-co/basic-memory/commit/8072449))
|
||||
|
||||
## v0.18.0 (2026-01-28)
|
||||
|
||||
### Features
|
||||
|
||||
- **#527**: Add context-aware wiki link resolution with source_path support
|
||||
([`0023e73`](https://github.com/basicmachines-co/basic-memory/commit/0023e73))
|
||||
- Add `source_path` parameter to `resolve_link()` for context-aware resolution
|
||||
- Relative path resolution: `[[nested/note]]` from `folder/file.md` → `folder/nested/note.md`
|
||||
- Proximity-based resolution for duplicate titles (prefers notes in same folder)
|
||||
- Strict mode to disable fuzzy search fallback for wiki links
|
||||
|
||||
- **#518**: Add directory support to move_note and delete_note tools
|
||||
([`0b20801`](https://github.com/basicmachines-co/basic-memory/commit/0b20801))
|
||||
- Add `is_directory` parameter to `move_note` and `delete_note` MCP tools
|
||||
- New `POST /move-directory` and delete directory API endpoints
|
||||
- Rename `folder` → `directory` parameter across codebase for consistency
|
||||
|
||||
- **#522**: Local MCP cloud mode routing
|
||||
([`8730067`](https://github.com/basicmachines-co/basic-memory/commit/8730067))
|
||||
- Add `--local` and `--cloud` CLI routing flags
|
||||
- Local MCP server (`basic-memory mcp`) automatically uses local routing
|
||||
- Enables simultaneous use of local Claude Desktop and cloud-based clients
|
||||
- Environment variable `BASIC_MEMORY_FORCE_LOCAL=true` for global override
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
- **#524**: Fix MCP prompt rendering errors
|
||||
([`e14ba92`](https://github.com/basicmachines-co/basic-memory/commit/e14ba92))
|
||||
- Fix "Error rendering prompt recent_activity" error
|
||||
- Change `TimeFrame` to `str` in prompt type annotations for FastMCP compatibility
|
||||
|
||||
## v0.17.9 (2026-01-24)
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
- **#523**: Fix `remove_project()` checking stale config in cloud mode
|
||||
([`17c0e0a`](https://github.com/basicmachines-co/basic-memory/commit/17c0e0a))
|
||||
- In cloud mode, only check database `is_default` field (source of truth)
|
||||
- Config file can become stale when users set default project via v2 API
|
||||
|
||||
## v0.17.8 (2026-01-24)
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
@@ -1,369 +0,0 @@
|
||||
# CLAUDE.md - Basic Memory Project Guide
|
||||
|
||||
## Project Overview
|
||||
|
||||
Basic Memory is a local-first knowledge management system built on the Model Context Protocol (MCP). It enables
|
||||
bidirectional communication between LLMs (like Claude) and markdown files, creating a personal knowledge graph that can
|
||||
be traversed using links between documents.
|
||||
|
||||
## CODEBASE DEVELOPMENT
|
||||
|
||||
### Project information
|
||||
|
||||
See the [README.md](README.md) file for a project overview.
|
||||
|
||||
### Build and Test Commands
|
||||
|
||||
- Install: `just install` or `pip install -e ".[dev]"`
|
||||
- Run all tests (SQLite + Postgres): `just test`
|
||||
- Run all tests against SQLite: `just test-sqlite`
|
||||
- Run all tests against Postgres: `just test-postgres` (uses testcontainers)
|
||||
- Run unit tests (SQLite): `just test-unit-sqlite`
|
||||
- Run unit tests (Postgres): `just test-unit-postgres`
|
||||
- Run integration tests (SQLite): `just test-int-sqlite`
|
||||
- Run integration tests (Postgres): `just test-int-postgres`
|
||||
- Generate HTML coverage: `just coverage`
|
||||
- Single test: `pytest tests/path/to/test_file.py::test_function_name`
|
||||
- Run benchmarks: `pytest test-int/test_sync_performance_benchmark.py -v -m "benchmark and not slow"`
|
||||
- Lint: `just lint` or `ruff check . --fix`
|
||||
- Type check: `just typecheck` or `uv run pyright`
|
||||
- Format: `just format` or `uv run ruff format .`
|
||||
- Run all code checks: `just check` (runs lint, format, typecheck, test)
|
||||
- Create db migration: `just migration "Your migration message"`
|
||||
- Run development MCP Inspector: `just run-inspector`
|
||||
|
||||
**Note:** Project requires Python 3.12+ (uses type parameter syntax and `type` aliases introduced in 3.12)
|
||||
|
||||
**Postgres Testing:** Uses [testcontainers](https://testcontainers-python.readthedocs.io/) which automatically spins up a Postgres instance in Docker. No manual database setup required - just have Docker running.
|
||||
|
||||
### Test Structure
|
||||
|
||||
- `tests/` - Unit tests for individual components (mocked, fast)
|
||||
- `test-int/` - Integration tests for real-world scenarios (no mocks, realistic)
|
||||
- Both directories are covered by unified coverage reporting
|
||||
- Benchmark tests in `test-int/` are marked with `@pytest.mark.benchmark`
|
||||
- Slow tests are marked with `@pytest.mark.slow`
|
||||
|
||||
### Code Style Guidelines
|
||||
|
||||
- Line length: 100 characters max
|
||||
- Python 3.12+ with full type annotations (uses type parameters and type aliases)
|
||||
- Format with ruff (consistent styling)
|
||||
- Import order: standard lib, third-party, local imports
|
||||
- Naming: snake_case for functions/variables, PascalCase for classes
|
||||
- Prefer async patterns with SQLAlchemy 2.0
|
||||
- Use Pydantic v2 for data validation and schemas
|
||||
- CLI uses Typer for command structure
|
||||
- API uses FastAPI for endpoints
|
||||
- Follow the repository pattern for data access
|
||||
- Tools communicate to api routers via the httpx ASGI client (in process)
|
||||
|
||||
### Code Change Guidelines
|
||||
|
||||
- **Full file read before edits**: Before editing any file, read it in full first to ensure complete context; partial reads lead to corrupted edits
|
||||
- **Minimize diffs**: Prefer the smallest change that satisfies the request. Avoid unrelated refactors or style rewrites unless necessary for correctness
|
||||
- **No speculative getattr**: Never use `getattr(obj, "attr", default)` when unsure about attribute names. Check the class definition or source code first
|
||||
- **Fail fast**: Write code with fail-fast logic by default. Do not swallow exceptions with errors or warnings
|
||||
- **No fallback logic**: Do not add fallback logic unless explicitly told to and agreed with the user
|
||||
- **No guessing**: Do not say "The issue is..." before you actually know what the issue is. Investigate first.
|
||||
|
||||
### Literate Programming Style
|
||||
|
||||
Code should tell a story. Comments must explain the "why" and narrative flow, not just the "what".
|
||||
|
||||
**Section Headers:**
|
||||
For files with multiple phases of logic, add section headers so the control flow reads like chapters:
|
||||
```python
|
||||
# --- Authentication ---
|
||||
# ... auth logic ...
|
||||
|
||||
# --- Data Validation ---
|
||||
# ... validation logic ...
|
||||
|
||||
# --- Business Logic ---
|
||||
# ... core logic ...
|
||||
```
|
||||
|
||||
**Decision Point Comments:**
|
||||
For conditionals that materially change behavior (gates, fallbacks, retries, feature flags), add comments with:
|
||||
- **Trigger**: what condition causes this branch
|
||||
- **Why**: the rationale (cost, correctness, UX, determinism)
|
||||
- **Outcome**: what changes downstream
|
||||
|
||||
```python
|
||||
# Trigger: project has no active sync watcher
|
||||
# Why: avoid duplicate file system watchers consuming resources
|
||||
# Outcome: starts new watcher, registers in active_watchers dict
|
||||
if project_id not in active_watchers:
|
||||
start_watcher(project_id)
|
||||
```
|
||||
|
||||
**Constraint Comments:**
|
||||
If code exists because of a constraint (async requirements, rate limits, schema compatibility), explain the constraint near the code:
|
||||
```python
|
||||
# SQLite requires WAL mode for concurrent read/write access
|
||||
connection.execute("PRAGMA journal_mode=WAL")
|
||||
```
|
||||
|
||||
**What NOT to Comment:**
|
||||
Avoid comments that restate obvious code:
|
||||
```python
|
||||
# Bad - restates code
|
||||
counter += 1 # increment counter
|
||||
|
||||
# Good - explains why
|
||||
counter += 1 # track retries for backoff calculation
|
||||
```
|
||||
|
||||
### Codebase Architecture
|
||||
|
||||
See [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) for detailed architecture documentation.
|
||||
|
||||
**Directory Structure:**
|
||||
- `/alembic` - Alembic db migrations
|
||||
- `/api` - FastAPI REST endpoints + `container.py` composition root
|
||||
- `/cli` - Typer CLI + `container.py` composition root
|
||||
- `/deps` - Feature-scoped FastAPI dependencies (config, db, projects, repositories, services, importers)
|
||||
- `/importers` - Import functionality for Claude, ChatGPT, and other sources
|
||||
- `/markdown` - Markdown parsing and processing
|
||||
- `/mcp` - MCP server + `container.py` composition root + `clients/` typed API clients
|
||||
- `/models` - SQLAlchemy ORM models
|
||||
- `/repository` - Data access layer
|
||||
- `/schemas` - Pydantic models for validation
|
||||
- `/services` - Business logic layer
|
||||
- `/sync` - File synchronization services + `coordinator.py` for lifecycle management
|
||||
|
||||
**Composition Roots:**
|
||||
Each entrypoint (API, MCP, CLI) has a composition root that:
|
||||
- Reads `ConfigManager` (the only place that reads global config)
|
||||
- Resolves runtime mode via `RuntimeMode` enum (TEST > CLOUD > LOCAL)
|
||||
- Provides dependencies to downstream code explicitly
|
||||
|
||||
**Typed API Clients (MCP):**
|
||||
MCP tools use typed clients in `mcp/clients/` to communicate with the API:
|
||||
- `KnowledgeClient` - Entity CRUD operations
|
||||
- `SearchClient` - Search operations
|
||||
- `MemoryClient` - Context building
|
||||
- `DirectoryClient` - Directory listing
|
||||
- `ResourceClient` - Resource reading
|
||||
- `ProjectClient` - Project management
|
||||
|
||||
Flow: MCP Tool → Typed Client → HTTP API → Router → Service → Repository
|
||||
|
||||
### Development Notes
|
||||
|
||||
- MCP tools are defined in src/basic_memory/mcp/tools/
|
||||
- MCP prompts are defined in src/basic_memory/mcp/prompts/
|
||||
- MCP tools should be atomic, composable operations
|
||||
- Use `textwrap.dedent()` for multi-line string formatting in prompts and tools
|
||||
- MCP Prompts are used to invoke tools and format content with instructions for an LLM
|
||||
- Schema changes require Alembic migrations
|
||||
- SQLite is used for indexing and full text search, files are source of truth
|
||||
- Testing uses pytest with asyncio support (strict mode)
|
||||
- Unit tests (`tests/`) use mocks when necessary; integration tests (`test-int/`) use real implementations
|
||||
- By default, tests run against SQLite (fast, no Docker needed)
|
||||
- Set `BASIC_MEMORY_TEST_POSTGRES=1` to run against Postgres (uses testcontainers - Docker required)
|
||||
- Each test runs in a standalone environment with isolated database and tmp_path directory
|
||||
- CI runs SQLite and Postgres tests in parallel for faster feedback
|
||||
- Performance benchmarks are in `test-int/test_sync_performance_benchmark.py`
|
||||
- Use pytest markers: `@pytest.mark.benchmark` for benchmarks, `@pytest.mark.slow` for slow tests
|
||||
- **Coverage must stay at 100%**: Write tests for new code. Only use `# pragma: no cover` when tests would require excessive mocking (e.g., TYPE_CHECKING blocks, error handlers that need failure injection, runtime-mode-dependent code paths)
|
||||
|
||||
### Async Client Pattern (Important!)
|
||||
|
||||
**All MCP tools and CLI commands use the context manager pattern for HTTP clients:**
|
||||
|
||||
```python
|
||||
from basic_memory.mcp.async_client import get_client
|
||||
|
||||
async def my_mcp_tool():
|
||||
async with get_client() as client:
|
||||
# Use client for API calls
|
||||
response = await call_get(client, "/path")
|
||||
return response
|
||||
```
|
||||
|
||||
**Do NOT use:**
|
||||
- ❌ `from basic_memory.mcp.async_client import client` (deprecated module-level client)
|
||||
- ❌ Manual auth header management
|
||||
- ❌ `inject_auth_header()` (deleted)
|
||||
|
||||
**Key principles:**
|
||||
- Auth happens at client creation, not per-request
|
||||
- Proper resource management via context managers
|
||||
- Supports three modes: Local (ASGI), CLI cloud (HTTP + auth), Cloud app (factory injection)
|
||||
- Factory pattern enables dependency injection for cloud consolidation
|
||||
|
||||
**For cloud app integration:**
|
||||
```python
|
||||
from basic_memory.mcp import async_client
|
||||
|
||||
# Set custom factory before importing tools
|
||||
async_client.set_client_factory(your_custom_factory)
|
||||
```
|
||||
|
||||
See SPEC-16 for full context manager refactor details.
|
||||
|
||||
## BASIC MEMORY PRODUCT USAGE
|
||||
|
||||
### Knowledge Structure
|
||||
|
||||
- Entity: Any concept, document, or idea represented as a markdown file
|
||||
- Observation: A categorized fact about an entity (`- [category] content`)
|
||||
- Relation: A directional link between entities (`- relation_type [[Target]]`)
|
||||
- Frontmatter: YAML metadata at the top of markdown files
|
||||
- Knowledge representation follows precise markdown format:
|
||||
- Observations with [category] prefixes
|
||||
- Relations with WikiLinks [[Entity]]
|
||||
- Frontmatter with metadata
|
||||
|
||||
### Basic Memory Commands
|
||||
|
||||
**Local Commands:**
|
||||
- Check sync status: `basic-memory status`
|
||||
- Import from Claude: `basic-memory import claude conversations`
|
||||
- Import from ChatGPT: `basic-memory import chatgpt`
|
||||
- Import from Memory JSON: `basic-memory import memory-json`
|
||||
- Tool access: `basic-memory tool` (provides CLI access to MCP tools)
|
||||
- Continue: `basic-memory tool continue-conversation --topic="search"`
|
||||
|
||||
**Project Management:**
|
||||
- List projects: `basic-memory project list`
|
||||
- Add project: `basic-memory project add "name" ~/path`
|
||||
- Project info: `basic-memory project info`
|
||||
- One-way sync (local -> cloud): `basic-memory project sync`
|
||||
- Bidirectional sync: `basic-memory project bisync`
|
||||
- Integrity check: `basic-memory project check`
|
||||
|
||||
**Cloud Commands (requires subscription):**
|
||||
- Authenticate: `basic-memory cloud login`
|
||||
- Logout: `basic-memory cloud logout`
|
||||
- Check cloud status: `basic-memory cloud status`
|
||||
- Setup cloud sync: `basic-memory cloud setup`
|
||||
- Manage snapshots: `basic-memory cloud snapshot [create|list|delete|show|browse]`
|
||||
- Restore from snapshot: `basic-memory cloud restore <path> --snapshot <id>`
|
||||
|
||||
### MCP Capabilities
|
||||
|
||||
- Basic Memory exposes these MCP tools to LLMs:
|
||||
|
||||
**Content Management:**
|
||||
- `write_note(title, content, folder, tags)` - Create/update markdown notes with semantic observations and relations
|
||||
- `read_note(identifier, page, page_size)` - Read notes by title, permalink, or memory:// URL with knowledge graph awareness
|
||||
- `read_content(path)` - Read raw file content (text, images, binaries) without knowledge graph processing
|
||||
- `view_note(identifier, page, page_size)` - View notes as formatted artifacts for better readability
|
||||
- `edit_note(identifier, operation, content)` - Edit notes incrementally (append, prepend, find/replace, replace_section)
|
||||
- `move_note(identifier, destination_path)` - Move notes to new locations, updating database and maintaining links
|
||||
- `delete_note(identifier)` - Delete notes from the knowledge base
|
||||
|
||||
**Knowledge Graph Navigation:**
|
||||
- `build_context(url, depth, timeframe)` - Navigate the knowledge graph via memory:// URLs for conversation continuity
|
||||
- `recent_activity(type, depth, timeframe)` - Get recently updated information with specified timeframe (e.g., "1d", "1 week")
|
||||
- `list_directory(dir_name, depth, file_name_glob)` - Browse directory contents with filtering and depth control
|
||||
|
||||
**Search & Discovery:**
|
||||
- `search_notes(query, page, page_size, search_type, types, entity_types, after_date)` - Full-text search across all content with advanced filtering options
|
||||
|
||||
**Project Management:**
|
||||
- `list_memory_projects()` - List all available projects with their status
|
||||
- `create_memory_project(project_name, project_path, set_default)` - Create new Basic Memory projects
|
||||
- `delete_project(project_name)` - Delete a project from configuration
|
||||
|
||||
**Visualization:**
|
||||
- `canvas(nodes, edges, title, folder)` - Generate Obsidian canvas files for knowledge graph visualization
|
||||
|
||||
**ChatGPT-Compatible Tools:**
|
||||
- `search(query)` - Search across knowledge base (OpenAI actions compatible)
|
||||
- `fetch(id)` - Fetch full content of a search result document
|
||||
|
||||
- MCP Prompts for better AI interaction:
|
||||
- `ai_assistant_guide()` - Guidance on effectively using Basic Memory tools for AI assistants
|
||||
- `continue_conversation(topic, timeframe)` - Continue previous conversations with relevant historical context
|
||||
- `search(query, after_date)` - Search with detailed, formatted results for better context understanding
|
||||
- `recent_activity(timeframe)` - View recently changed items with formatted output
|
||||
|
||||
### Cloud Features (v0.15.0+)
|
||||
|
||||
Basic Memory now supports cloud synchronization and storage (requires active subscription):
|
||||
|
||||
**Authentication:**
|
||||
- JWT-based authentication with subscription validation
|
||||
- Secure session management with token refresh
|
||||
- Support for multiple cloud projects
|
||||
|
||||
**Bidirectional Sync:**
|
||||
- rclone bisync integration for two-way synchronization
|
||||
- Conflict resolution and integrity verification
|
||||
- Real-time sync with change detection
|
||||
- Mount/unmount cloud storage for direct file access
|
||||
|
||||
**Cloud Project Management:**
|
||||
- Create and manage projects in the cloud
|
||||
- Toggle between local and cloud modes
|
||||
- Per-project sync configuration
|
||||
- Subscription-based access control
|
||||
|
||||
**Security & Performance:**
|
||||
- Removed .env file loading for improved security
|
||||
- .gitignore integration (respects gitignored files)
|
||||
- WAL mode for SQLite performance
|
||||
- Background relation resolution (non-blocking startup)
|
||||
- API performance optimizations (SPEC-11)
|
||||
|
||||
## AI-Human Collaborative Development
|
||||
|
||||
Basic Memory emerged from and enables a new kind of development process that combines human and AI capabilities. Instead
|
||||
of using AI just for code generation, we've developed a true collaborative workflow:
|
||||
|
||||
1. AI (LLM) writes initial implementation based on specifications and context
|
||||
2. Human reviews, runs tests, and commits code with any necessary adjustments
|
||||
3. Knowledge persists across conversations using Basic Memory's knowledge graph
|
||||
4. Development continues seamlessly across different AI sessions with consistent context
|
||||
5. Results improve through iterative collaboration and shared understanding
|
||||
|
||||
This approach has allowed us to tackle more complex challenges and build a more robust system than either humans or AI
|
||||
could achieve independently.
|
||||
|
||||
**Problem-Solving Guidance:**
|
||||
- If a solution isn't working after reasonable effort, suggest alternative approaches
|
||||
- Don't persist with a problematic library or pattern when better alternatives exist
|
||||
- Example: When py-pglite caused cascading test failures, switching to testcontainers-postgres was the right call
|
||||
|
||||
## GitHub Integration
|
||||
|
||||
Basic Memory has taken AI-Human collaboration to the next level by integrating Claude directly into the development workflow through GitHub:
|
||||
|
||||
### GitHub MCP Tools
|
||||
|
||||
Using the GitHub Model Context Protocol server, Claude can now:
|
||||
|
||||
- **Repository Management**:
|
||||
- View repository files and structure
|
||||
- Read file contents
|
||||
- Create new branches
|
||||
- Create and update files
|
||||
|
||||
- **Issue Management**:
|
||||
- Create new issues
|
||||
- Comment on existing issues
|
||||
- Close and update issues
|
||||
- Search across issues
|
||||
|
||||
- **Pull Request Workflow**:
|
||||
- Create pull requests
|
||||
- Review code changes
|
||||
- Add comments to PRs
|
||||
|
||||
This integration enables Claude to participate as a full team member in the development process, not just as a code generation tool. Claude's GitHub account ([bm-claudeai](https://github.com/bm-claudeai)) is a member of the Basic Machines organization with direct contributor access to the codebase.
|
||||
|
||||
### Collaborative Development Process
|
||||
|
||||
With GitHub integration, the development workflow includes:
|
||||
|
||||
1. **Direct code review** - Claude can analyze PRs and provide detailed feedback
|
||||
2. **Contribution tracking** - All of Claude's contributions are properly attributed in the Git history
|
||||
3. **Branch management** - Claude can create feature branches for implementations
|
||||
4. **Documentation maintenance** - Claude can keep documentation updated as the code evolves
|
||||
5. **Code Commits**: ALWAYS sign off commits with `git commit -s`
|
||||
|
||||
This level of integration represents a new paradigm in AI-human collaboration, where the AI assistant becomes a full-fledged team member rather than just a tool for generating code snippets.
|
||||
@@ -357,6 +357,22 @@ basic-memory cloud check
|
||||
basic-memory cloud mount
|
||||
```
|
||||
|
||||
**Routing Flags** (for users with cloud subscriptions):
|
||||
|
||||
When cloud mode is enabled, CLI commands communicate with the cloud API by default. Use routing flags to override this:
|
||||
|
||||
```bash
|
||||
# Force local routing (useful for local MCP server while cloud mode is enabled)
|
||||
basic-memory status --local
|
||||
basic-memory project list --local
|
||||
|
||||
# Force cloud routing (when cloud mode is disabled but you want cloud access)
|
||||
basic-memory status --cloud
|
||||
basic-memory project info my-project --cloud
|
||||
```
|
||||
|
||||
The local MCP server (`basic-memory mcp`) automatically uses local routing, so you can use both local Claude Desktop and cloud-based clients simultaneously.
|
||||
|
||||
4. In Claude Desktop, the LLM can now use these tools:
|
||||
|
||||
**Content Management:**
|
||||
@@ -380,6 +396,8 @@ list_directory(dir_name, depth) - Browse directory contents with filtering
|
||||
**Search & Discovery:**
|
||||
```
|
||||
search(query, page, page_size) - Search across your knowledge base
|
||||
search_notes(query, page, page_size, search_type, types, entity_types, after_date, metadata_filters, tags, status, project) - Search with filters
|
||||
search_by_metadata(filters, limit, offset, project) - Structured frontmatter search
|
||||
```
|
||||
|
||||
**Project Management:**
|
||||
@@ -433,6 +451,7 @@ Basic Memory uses [Loguru](https://github.com/Delgan/loguru) for logging. The lo
|
||||
|----------|---------|-------------|
|
||||
| `BASIC_MEMORY_LOG_LEVEL` | `INFO` | Log level: DEBUG, INFO, WARNING, ERROR |
|
||||
| `BASIC_MEMORY_CLOUD_MODE` | `false` | When `true`, API logs to stdout with structured context |
|
||||
| `BASIC_MEMORY_FORCE_LOCAL` | `false` | When `true`, forces local API routing (ignores cloud mode) |
|
||||
| `BASIC_MEMORY_ENV` | `dev` | Set to `test` for test mode (stderr only) |
|
||||
|
||||
### Examples
|
||||
@@ -477,16 +496,23 @@ just test
|
||||
- `just test-int-postgres` - Run integration tests against Postgres
|
||||
- `just test-windows` - Run Windows-specific tests (auto-skips on other platforms)
|
||||
- `just test-benchmark` - Run performance benchmark tests
|
||||
- `just testmon` - Run tests impacted by recent changes (pytest-testmon)
|
||||
- `just test-smoke` - Run fast MCP end-to-end smoke test
|
||||
- `just fast-check` - Run fix/format/typecheck + impacted tests + smoke test
|
||||
- `just doctor` - Run local file <-> DB consistency checks with temp config
|
||||
|
||||
**Postgres Testing:**
|
||||
|
||||
Postgres tests use [testcontainers](https://testcontainers-python.readthedocs.io/) which automatically spins up a Postgres instance in Docker. No manual database setup required - just have Docker running.
|
||||
|
||||
**Testmon Note:** When no files have changed, `just testmon` may collect 0 tests. That's expected and means no impacted tests were detected.
|
||||
|
||||
**Test Markers:**
|
||||
|
||||
Tests use pytest markers for selective execution:
|
||||
- `windows` - Windows-specific database optimizations
|
||||
- `benchmark` - Performance tests (excluded from default runs)
|
||||
- `smoke` - Fast MCP end-to-end smoke tests
|
||||
|
||||
**Other Development Commands:**
|
||||
```bash
|
||||
@@ -494,10 +520,17 @@ just install # Install with dev dependencies
|
||||
just lint # Run linting checks
|
||||
just typecheck # Run type checking
|
||||
just format # Format code with ruff
|
||||
just fast-check # Fast local loop (fix/format/typecheck + testmon + smoke)
|
||||
just doctor # Local consistency check (temp config)
|
||||
just check # Run all quality checks
|
||||
just migration "msg" # Create database migration
|
||||
```
|
||||
|
||||
**Local Consistency Check:**
|
||||
```bash
|
||||
basic-memory doctor # Verifies file <-> database sync in a temp project
|
||||
```
|
||||
|
||||
See the [justfile](justfile) for the complete list of development commands.
|
||||
|
||||
## License
|
||||
|
||||
+15
-2
@@ -214,15 +214,28 @@ Example tool using typed client:
|
||||
|
||||
```python
|
||||
@mcp.tool()
|
||||
async def search_notes(query: str, project: str | None = None) -> SearchResponse:
|
||||
async def search_notes(
|
||||
query: str,
|
||||
project: str | None = None,
|
||||
metadata_filters: dict | None = None,
|
||||
tags: list[str] | None = None,
|
||||
status: str | None = None,
|
||||
) -> SearchResponse:
|
||||
async with get_client() as client:
|
||||
active_project = await get_active_project(client, project)
|
||||
|
||||
# Import client inside function to avoid circular imports
|
||||
from basic_memory.mcp.clients import SearchClient
|
||||
from basic_memory.schemas.search import SearchQuery
|
||||
|
||||
search_query = SearchQuery(
|
||||
text=query,
|
||||
metadata_filters=metadata_filters,
|
||||
tags=tags,
|
||||
status=status,
|
||||
)
|
||||
search_client = SearchClient(client, active_project.external_id)
|
||||
return await search_client.search(query)
|
||||
return await search_client.search(search_query.model_dump())
|
||||
```
|
||||
|
||||
## Sync Coordination
|
||||
|
||||
@@ -0,0 +1,494 @@
|
||||
# Note Format Reference
|
||||
|
||||
Every document in Basic Memory is a plain Markdown file. Files are the source of truth — changes to files automatically update the knowledge graph in the database. You maintain complete ownership, files work with git, and knowledge persists independently of any AI conversation.
|
||||
|
||||
## Document Structure
|
||||
|
||||
A note has three parts: YAML frontmatter, content (observations), and relations.
|
||||
|
||||
```markdown
|
||||
---
|
||||
title: Coffee Brewing Methods
|
||||
type: note
|
||||
tags: [coffee, brewing]
|
||||
permalink: coffee-brewing-methods
|
||||
---
|
||||
|
||||
# Coffee Brewing Methods
|
||||
|
||||
## Observations
|
||||
- [method] Pour over provides more flavor clarity than French press
|
||||
- [technique] Water temperature at 205°F extracts optimal compounds #brewing
|
||||
- [preference] Ethiopian beans work well with lighter roasts (personal experience)
|
||||
|
||||
## Relations
|
||||
- relates_to [[Coffee Bean Origins]]
|
||||
- requires [[Proper Grinding Technique]]
|
||||
- contrasts_with [[Tea Brewing Methods]]
|
||||
```
|
||||
|
||||
The `## Observations` and `## Relations` headings are conventional but not required — the parser detects observations and relations by their syntax patterns anywhere in the document.
|
||||
|
||||
## Frontmatter
|
||||
|
||||
YAML metadata between `---` fences at the top of the file.
|
||||
|
||||
| Field | Required | Default | Description |
|
||||
|-------|----------|---------|-------------|
|
||||
| `title` | No | filename stem | Used for linking and references. Auto-set from filename if missing. |
|
||||
| `type` | No | `note` | Entity type. Used for schema resolution and filtering. |
|
||||
| `tags` | No | `[]` | List or comma-separated string. Used for organization and search. |
|
||||
| `permalink` | No | generated from title | Stable identifier. Persists even if the file moves. |
|
||||
| `schema` | No | none | Schema attachment — dict (inline), string (reference), or omitted (implicit). |
|
||||
|
||||
Custom fields are allowed. Any key not in the standard set is stored as `entity_metadata` and indexed for search and filtering.
|
||||
|
||||
```yaml
|
||||
---
|
||||
title: Paul Graham
|
||||
type: Person
|
||||
tags: [startups, essays, lisp]
|
||||
permalink: paul-graham
|
||||
status: active
|
||||
source: wikipedia
|
||||
---
|
||||
```
|
||||
|
||||
Here `status` and `source` are custom fields stored in `entity_metadata`.
|
||||
|
||||
### Frontmatter Value Handling
|
||||
|
||||
YAML automatically converts some values to native types. Basic Memory normalizes them:
|
||||
|
||||
- Date strings (`2025-10-24`) → kept as ISO format strings
|
||||
- Numbers (`1.0`) → converted to strings
|
||||
- Booleans (`true`) → converted to strings (`"True"`)
|
||||
- Lists and dicts → preserved, items normalized recursively
|
||||
|
||||
This prevents errors when downstream code expects string values.
|
||||
|
||||
## Observations
|
||||
|
||||
An observation is a categorized fact about the entity. Written as a Markdown list item.
|
||||
|
||||
**Syntax:**
|
||||
|
||||
```
|
||||
- [category] content text #tag1 #tag2 (context)
|
||||
```
|
||||
|
||||
| Part | Required | Description |
|
||||
|------|----------|-------------|
|
||||
| `[category]` | Yes | Classification in square brackets. Any text except `[]()` chars. |
|
||||
| content | Yes | The fact or statement. |
|
||||
| `#tags` | No | Inline tags. Space-separated, each starting with `#`. |
|
||||
| `(context)` | No | Parenthesized text at end of line. Supporting details or source. |
|
||||
|
||||
### Examples
|
||||
|
||||
```markdown
|
||||
- [tech] Uses SQLite for storage #database
|
||||
- [design] Follows local-first architecture #architecture
|
||||
- [decision] Selected bcrypt for passwords #security (based on OWASP audit)
|
||||
- [name] Paul Graham
|
||||
- [expertise] Startups
|
||||
- [expertise] Lisp
|
||||
- [expertise] Essay writing
|
||||
```
|
||||
|
||||
Array-like fields use repeated categories — multiple `[expertise]` observations above.
|
||||
|
||||
### What Is Not an Observation
|
||||
|
||||
The parser excludes these list item patterns:
|
||||
|
||||
| Pattern | Example | Reason |
|
||||
|---------|---------|--------|
|
||||
| Checkboxes | `- [ ] Todo item`, `- [x] Done`, `- [-] Cancelled` | Task list syntax |
|
||||
| Markdown links | `- [text](url)` | URL link syntax |
|
||||
| Bare wiki links | `- [[Target]]` | Treated as a relation instead |
|
||||
|
||||
A list item with `#tags` but no `[category]` is still parsed — the tags are extracted and the category defaults to `Note`.
|
||||
|
||||
## Relations
|
||||
|
||||
Relations connect documents to form the knowledge graph. There are two kinds.
|
||||
|
||||
### Explicit Relations
|
||||
|
||||
Written as list items with a relation type and a `[[wiki link]]` target.
|
||||
|
||||
**Syntax:**
|
||||
|
||||
```
|
||||
- relation_type [[Target Entity]] (context)
|
||||
```
|
||||
|
||||
| Part | Required | Description |
|
||||
|------|----------|-------------|
|
||||
| `relation_type` | No | Text before `[[`. Defaults to `relates_to` if omitted. |
|
||||
| `[[Target]]` | Yes | Wiki link to the target entity. Matched by title or permalink. |
|
||||
| `(context)` | No | Parenthesized text after `]]`. Supporting details. |
|
||||
|
||||
### Examples
|
||||
|
||||
```markdown
|
||||
- implements [[Search Design]]
|
||||
- depends_on [[Database Schema]]
|
||||
- works_at [[Y Combinator]] (co-founder)
|
||||
- [[Some Entity]]
|
||||
```
|
||||
|
||||
The last example — a bare `[[wiki link]]` in a list item — gets relation type `relates_to`.
|
||||
|
||||
Common relation types:
|
||||
- `implements`, `depends_on`, `relates_to`, `inspired_by`
|
||||
- `extends`, `part_of`, `contains`, `pairs_with`
|
||||
- `works_at`, `authored`, `collaborated_with`
|
||||
|
||||
Any text works as a relation type. These are conventions, not a fixed set.
|
||||
|
||||
### Inline References
|
||||
|
||||
Wiki links appearing in regular prose (not as list items) create implicit `links_to` relations.
|
||||
|
||||
```markdown
|
||||
This builds on [[Core Design]] and uses [[Utility Functions]].
|
||||
```
|
||||
|
||||
This creates two relations: `links_to [[Core Design]]` and `links_to [[Utility Functions]]`.
|
||||
|
||||
### Forward References
|
||||
|
||||
Relations can link to entities that don't exist yet. Basic Memory resolves them when the target is created.
|
||||
|
||||
## Permalinks and memory:// URLs
|
||||
|
||||
Every document has a unique **permalink** — a stable identifier derived from its title. You can set one explicitly in frontmatter, or let the system generate it.
|
||||
|
||||
```yaml
|
||||
permalink: auth-approaches-2024
|
||||
```
|
||||
|
||||
Permalinks form the basis of `memory://` URLs:
|
||||
|
||||
```
|
||||
memory://auth-approaches-2024 # By permalink
|
||||
memory://Authentication Approaches # By title (auto-resolves)
|
||||
memory://project/auth-approaches # By path
|
||||
```
|
||||
|
||||
Pattern matching is supported:
|
||||
|
||||
```
|
||||
memory://auth* # Starts with "auth"
|
||||
memory://*/approaches # Ends with "approaches"
|
||||
memory://project/*/requirements # Nested wildcard
|
||||
```
|
||||
|
||||
## Schemas
|
||||
|
||||
Schemas declare the expected structure of a note — which observation categories and relation types a well-formed note should have. They use Picoschema, a compact notation from Google's Dotprompt that fits naturally in YAML frontmatter.
|
||||
|
||||
### Picoschema Syntax
|
||||
|
||||
```yaml
|
||||
schema:
|
||||
name: string, full name # required field with description
|
||||
email?: string, contact email # ? = optional
|
||||
role?: string, job title
|
||||
works_at?: Organization, employer # capitalized type = entity reference
|
||||
tags?(array): string, categories # array of type
|
||||
status?(enum): [active, inactive] # enum with allowed values
|
||||
metadata?(object): # nested object
|
||||
updated_at?: string
|
||||
source?: string
|
||||
```
|
||||
|
||||
| Notation | Meaning | Example |
|
||||
|----------|---------|---------|
|
||||
| `field: type` | Required field | `name: string` |
|
||||
| `field?: type` | Optional field | `role?: string` |
|
||||
| `field(array): type` | Array of values | `expertise(array): string` |
|
||||
| `field?(enum): [vals]` | Enum with allowed values | `status?(enum): [active, inactive]` |
|
||||
| `field?(object):` | Nested object with sub-fields | `metadata?(object):` |
|
||||
| `, description` | Description after comma | `name: string, full name` |
|
||||
| `EntityName` | Capitalized type = entity reference | `works_at?: Organization` |
|
||||
|
||||
**Scalar types:** `string`, `integer`, `number`, `boolean`, `any`
|
||||
|
||||
Any type not in that set whose first letter is uppercase is treated as an entity reference (a relation target).
|
||||
|
||||
### Schema-to-Note Mapping
|
||||
|
||||
Schemas validate against existing observation/relation syntax. Note authors don't learn new syntax.
|
||||
|
||||
| Schema Declaration | Maps To | Example in Note |
|
||||
|--------------------|---------|-----------------|
|
||||
| `field: string` | Observation `[field] value` | `- [name] Paul Graham` |
|
||||
| `field?(array): string` | Multiple `[field]` observations | `- [expertise] Lisp` (repeated) |
|
||||
| `field?: EntityType` | Relation `field [[Target]]` | `- works_at [[Y Combinator]]` |
|
||||
| `field?(array): EntityType` | Multiple `field` relations | `- authored [[Book]]` (repeated) |
|
||||
| `tags` | Frontmatter `tags` array | `tags: [startups, essays]` |
|
||||
| `field?(enum): [vals]` | Observation `[field] value` where value is in the set | `- [status] active` |
|
||||
|
||||
Observations and relations not covered by the schema are valid — schemas describe a subset, not a straitjacket.
|
||||
|
||||
### Schema Attachment
|
||||
|
||||
Three ways to attach a schema to a note, resolved in priority order:
|
||||
|
||||
**1. Inline schema** — `schema` is a dict in frontmatter:
|
||||
|
||||
```yaml
|
||||
---
|
||||
title: Team Standup 2024-01-15
|
||||
type: meeting
|
||||
schema:
|
||||
attendees(array): string, who was there
|
||||
decisions(array): string, what was decided
|
||||
action_items(array): string, follow-ups
|
||||
blockers?(array): string, anything stuck
|
||||
---
|
||||
```
|
||||
|
||||
Good for one-off structured notes or prototyping a schema before extracting it.
|
||||
|
||||
**2. Explicit reference** — `schema` is a string naming a schema note:
|
||||
|
||||
```yaml
|
||||
---
|
||||
title: Basic Memory
|
||||
schema: SoftwareProject
|
||||
---
|
||||
```
|
||||
|
||||
or by permalink:
|
||||
|
||||
```yaml
|
||||
---
|
||||
title: LLM Memory Patterns
|
||||
schema: schema/research-project
|
||||
---
|
||||
```
|
||||
|
||||
Use when the note's `type` differs from the schema it should validate against, or when multiple schema variants exist.
|
||||
|
||||
**3. Implicit by type** — no `schema` field, resolved by matching `type`:
|
||||
|
||||
```yaml
|
||||
---
|
||||
title: Paul Graham
|
||||
type: Person
|
||||
---
|
||||
```
|
||||
|
||||
The system looks up a schema note where `entity: Person`. If found, it applies. If not, no validation occurs.
|
||||
|
||||
**4. No schema** — perfectly fine. Most notes don't need one.
|
||||
|
||||
### Schema Notes
|
||||
|
||||
A schema is itself a Basic Memory note with `type: schema`. It lives anywhere (though `schema/` is the conventional directory).
|
||||
|
||||
```yaml
|
||||
# schema/Person.md
|
||||
---
|
||||
title: Person
|
||||
type: schema
|
||||
entity: Person
|
||||
version: 1
|
||||
schema:
|
||||
name: string, full name
|
||||
role?: string, job title or position
|
||||
works_at?: Organization, employer
|
||||
expertise?(array): string, areas of knowledge
|
||||
email?: string, contact email
|
||||
settings:
|
||||
validation: warn
|
||||
---
|
||||
|
||||
# Person
|
||||
|
||||
A human individual in the knowledge graph.
|
||||
```
|
||||
|
||||
| Field | Required | Description |
|
||||
|-------|----------|-------------|
|
||||
| `type` | Yes | Must be `schema` |
|
||||
| `entity` | Yes | The entity type this schema describes (e.g., `Person`) |
|
||||
| `version` | No | Schema version number (default: `1`) |
|
||||
| `schema` | Yes | Picoschema dict defining the fields |
|
||||
| `settings.validation` | No | Validation mode (default: `warn`) |
|
||||
|
||||
Schema notes are regular notes — they show up in search, can have observations and relations, and participate in the knowledge graph.
|
||||
|
||||
### Validation Modes
|
||||
|
||||
| Mode | Behavior |
|
||||
|------|----------|
|
||||
| `warn` | Warnings in output, doesn't block (default) |
|
||||
| `strict` | Errors that block sync, for CI/CD enforcement |
|
||||
| `off` | No validation |
|
||||
|
||||
### Validation Output
|
||||
|
||||
```
|
||||
$ bm schema validate people/ada-lovelace.md
|
||||
|
||||
⚠ Person schema validation:
|
||||
- Missing required field: name (expected [name] observation)
|
||||
- Missing optional field: role
|
||||
- Missing optional field: works_at (no relation found)
|
||||
|
||||
ℹ Unmatched observations: [fact] ×2, [born] ×1
|
||||
ℹ Unmatched relations: collaborated_with
|
||||
```
|
||||
|
||||
"Unmatched" items are informational — observations and relations the schema doesn't cover.
|
||||
|
||||
### Schema Inference
|
||||
|
||||
Generate schemas from existing notes by analyzing observation and relation frequency:
|
||||
|
||||
```
|
||||
$ bm schema infer Person
|
||||
|
||||
Analyzing 30 notes with type: Person...
|
||||
|
||||
Observations found:
|
||||
[name] 30/30 100% → name: string
|
||||
[role] 27/30 90% → role?: string
|
||||
[expertise] 18/30 60% → expertise?(array): string
|
||||
[email] 8/30 27% → email?: string
|
||||
|
||||
Relations found:
|
||||
works_at 22/30 73% → works_at?: Organization
|
||||
|
||||
Suggested schema:
|
||||
name: string, full name
|
||||
role?: string, job title
|
||||
expertise?(array): string, areas of knowledge
|
||||
email?: string, contact email
|
||||
works_at?: Organization, employer
|
||||
|
||||
Save to schema/Person.md? [y/n]
|
||||
```
|
||||
|
||||
Frequency thresholds:
|
||||
- **100% present** → required field
|
||||
- **25%+ present** → optional field
|
||||
- **Below 25%** → excluded from suggestion
|
||||
|
||||
### Schema Drift Detection
|
||||
|
||||
Track how usage patterns shift over time:
|
||||
|
||||
```
|
||||
$ bm schema diff Person
|
||||
|
||||
Schema drift detected:
|
||||
|
||||
+ expertise: now in 81% of notes (was 12%)
|
||||
- department: dropped to 3% of notes
|
||||
~ works_at: cardinality changed (one → many)
|
||||
|
||||
Update schema? [y/n/review]
|
||||
```
|
||||
|
||||
## Complete Examples
|
||||
|
||||
### Simple Note (No Schema)
|
||||
|
||||
```markdown
|
||||
---
|
||||
title: Project Ideas
|
||||
type: note
|
||||
tags: [ideas, brainstorm]
|
||||
---
|
||||
|
||||
# Project Ideas
|
||||
|
||||
## Observations
|
||||
- [idea] Build a CLI tool for markdown linting #tooling
|
||||
- [idea] Create a recipe knowledge base #cooking
|
||||
- [priority] Focus on developer tools first (Q1 goal)
|
||||
|
||||
## Relations
|
||||
- inspired_by [[Developer Workflow Research]]
|
||||
- part_of [[Q1 Planning]]
|
||||
```
|
||||
|
||||
### Schema-Validated Note
|
||||
|
||||
Schema at `schema/Person.md`:
|
||||
|
||||
```yaml
|
||||
---
|
||||
title: Person
|
||||
type: schema
|
||||
entity: Person
|
||||
version: 1
|
||||
schema:
|
||||
name: string, full name
|
||||
role?: string, job title or position
|
||||
works_at?: Organization, employer
|
||||
expertise?(array): string, areas of knowledge
|
||||
email?: string, contact email
|
||||
settings:
|
||||
validation: warn
|
||||
---
|
||||
|
||||
# Person
|
||||
|
||||
A human individual in the knowledge graph.
|
||||
```
|
||||
|
||||
Note at `people/paul-graham.md`:
|
||||
|
||||
```markdown
|
||||
---
|
||||
title: Paul Graham
|
||||
type: Person
|
||||
tags: [startups, essays, lisp]
|
||||
---
|
||||
|
||||
# Paul Graham
|
||||
|
||||
## Observations
|
||||
- [name] Paul Graham
|
||||
- [role] Essayist and investor
|
||||
- [expertise] Startups
|
||||
- [expertise] Lisp
|
||||
- [expertise] Essay writing
|
||||
- [fact] Created Viaweb, the first web app
|
||||
|
||||
## Relations
|
||||
- works_at [[Y Combinator]]
|
||||
- authored [[Hackers and Painters]]
|
||||
```
|
||||
|
||||
The `[fact]` observation and `authored` relation are not in the schema — they're valid, just unmatched. The schema only checks that `[name]` exists (required) and looks for optional fields like `[role]`, `[expertise]`, and `works_at`.
|
||||
|
||||
### Inline Schema Note
|
||||
|
||||
```markdown
|
||||
---
|
||||
title: Team Standup 2024-01-15
|
||||
type: meeting
|
||||
schema:
|
||||
attendees(array): string, who was there
|
||||
decisions(array): string, what was decided
|
||||
action_items(array): string, follow-ups
|
||||
blockers?(array): string, anything stuck
|
||||
---
|
||||
|
||||
# Team Standup 2024-01-15
|
||||
|
||||
## Observations
|
||||
- [attendees] Paul
|
||||
- [attendees] Sarah
|
||||
- [decisions] Ship v2 by Friday
|
||||
- [action_items] Paul to review PR #42
|
||||
- [blockers] Waiting on API credentials
|
||||
```
|
||||
@@ -1038,6 +1038,35 @@ recent_decisions = await search_notes(
|
||||
)
|
||||
```
|
||||
|
||||
**Structured frontmatter filters**:
|
||||
|
||||
```python
|
||||
# Filter by tags and status
|
||||
results = await search_notes(
|
||||
query="authentication",
|
||||
tags=["security"],
|
||||
status="in-progress",
|
||||
project="main"
|
||||
)
|
||||
|
||||
# Complex metadata filters (supports $in, $gt, $gte, $lt, $lte, $between)
|
||||
results = await search_notes(
|
||||
query="api design",
|
||||
metadata_filters={
|
||||
"type": "spec",
|
||||
"priority": {"$in": ["high", "critical"]},
|
||||
"tags": ["architecture"]
|
||||
},
|
||||
project="main"
|
||||
)
|
||||
|
||||
# Metadata-only search
|
||||
results = await search_by_metadata(
|
||||
filters={"type": "spec", "status": "in-progress"},
|
||||
project="main"
|
||||
)
|
||||
```
|
||||
|
||||
### Search Types
|
||||
|
||||
**Text search (default)**:
|
||||
@@ -2861,7 +2890,7 @@ contents = await list_directory(
|
||||
|
||||
### Search & Discovery
|
||||
|
||||
**search_notes(query, page, page_size, search_type, types, entity_types, after_date, project)**
|
||||
**search_notes(query, page, page_size, search_type, types, entity_types, after_date, metadata_filters, tags, status, project)**
|
||||
- Search across knowledge base
|
||||
- Parameters:
|
||||
- `query` (required): Search query
|
||||
@@ -2871,6 +2900,9 @@ contents = await list_directory(
|
||||
- `types` (optional): Entity type filter
|
||||
- `entity_types` (optional): Observation category filter
|
||||
- `after_date` (optional): Date filter (ISO format)
|
||||
- `metadata_filters` (optional): Structured frontmatter filters (dict)
|
||||
- `tags` (optional): Frontmatter tags filter (list)
|
||||
- `status` (optional): Frontmatter status filter (string)
|
||||
- `project` (required unless default_project_mode): Target project
|
||||
- Returns: Matching entities with scores
|
||||
- Example:
|
||||
@@ -2883,6 +2915,22 @@ results = await search_notes(
|
||||
)
|
||||
```
|
||||
|
||||
**search_by_metadata(filters, limit, offset, project)**
|
||||
- Metadata-only search using structured frontmatter
|
||||
- Parameters:
|
||||
- `filters` (required): Dict of field -> value (supports $in, $gt/$gte/$lt/$lte, $between)
|
||||
- `limit` (optional): Max results (default: 20)
|
||||
- `offset` (optional): Pagination offset (default: 0)
|
||||
- `project` (required unless default_project_mode): Target project
|
||||
- Returns: Matching entities
|
||||
- Example:
|
||||
```python
|
||||
results = await search_by_metadata(
|
||||
filters={"type": "spec", "status": "in-progress"},
|
||||
project="main"
|
||||
)
|
||||
```
|
||||
|
||||
### Project Management
|
||||
|
||||
**list_memory_projects()**
|
||||
|
||||
@@ -62,6 +62,22 @@ test-int-postgres:
|
||||
BASIC_MEMORY_TEST_POSTGRES=1 uv run pytest -p pytest_mock -v --no-cov test-int
|
||||
fi
|
||||
|
||||
# Run tests impacted by recent changes (requires pytest-testmon)
|
||||
testmon *args:
|
||||
BASIC_MEMORY_ENV=test uv run pytest -p pytest_mock -v --no-cov --testmon --testmon-forceselect {{args}}
|
||||
|
||||
# Run MCP smoke test (fast end-to-end loop)
|
||||
test-smoke:
|
||||
BASIC_MEMORY_ENV=test uv run pytest -p pytest_mock -v --no-cov -m smoke test-int/mcp/test_smoke_integration.py
|
||||
|
||||
# Fast local loop: lint, format, typecheck, impacted tests
|
||||
fast-check:
|
||||
just fix
|
||||
just format
|
||||
just typecheck
|
||||
just testmon
|
||||
just test-smoke
|
||||
|
||||
# Reset Postgres test database (drops and recreates schema)
|
||||
# Useful when Alembic migration state gets out of sync during development
|
||||
# Uses credentials from docker-compose-postgres.yml
|
||||
@@ -149,6 +165,18 @@ format:
|
||||
run-inspector:
|
||||
npx @modelcontextprotocol/inspector
|
||||
|
||||
# Run doctor checks in an isolated temp home/config
|
||||
doctor:
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
TMP_HOME=$(mktemp -d)
|
||||
TMP_CONFIG=$(mktemp -d)
|
||||
HOME="$TMP_HOME" \
|
||||
BASIC_MEMORY_ENV=test \
|
||||
BASIC_MEMORY_HOME="$TMP_HOME/basic-memory" \
|
||||
BASIC_MEMORY_CONFIG_DIR="$TMP_CONFIG" \
|
||||
./.venv/bin/python -m basic_memory.cli.main doctor --local
|
||||
|
||||
|
||||
# Update all dependencies to latest versions
|
||||
update-deps:
|
||||
|
||||
@@ -71,6 +71,7 @@ markers = [
|
||||
"slow: Slow-running tests (deselect with '-m \"not slow\"')",
|
||||
"postgres: Tests that run against Postgres backend (deselect with '-m \"not postgres\"')",
|
||||
"windows: Windows-specific tests (deselect with '-m \"not windows\"')",
|
||||
"smoke: Fast end-to-end smoke tests for MCP flows",
|
||||
]
|
||||
|
||||
[tool.ruff]
|
||||
@@ -91,6 +92,7 @@ dev = [
|
||||
"testcontainers[postgres]>=4.0.0",
|
||||
"psycopg>=3.2.0",
|
||||
"pyright>=1.1.408",
|
||||
"pytest-testmon>=2.2.0",
|
||||
]
|
||||
|
||||
[tool.hatch.version]
|
||||
|
||||
+2
-2
@@ -6,12 +6,12 @@
|
||||
"url": "https://github.com/basicmachines-co/basic-memory.git",
|
||||
"source": "github"
|
||||
},
|
||||
"version": "0.17.8",
|
||||
"version": "0.18.5",
|
||||
"packages": [
|
||||
{
|
||||
"registryType": "pypi",
|
||||
"identifier": "basic-memory",
|
||||
"version": "0.17.8",
|
||||
"version": "0.18.5",
|
||||
"runtimeHint": "uvx",
|
||||
"runtimeArguments": [
|
||||
{"type": "positional", "value": "basic-memory"},
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
"""basic-memory - Local-first knowledge management combining Zettelkasten with knowledge graphs"""
|
||||
|
||||
# Package version - updated by release automation
|
||||
__version__ = "0.17.8"
|
||||
__version__ = "0.18.5"
|
||||
|
||||
# API version for FastAPI - independent of package version
|
||||
__api_version__ = "v0"
|
||||
|
||||
@@ -0,0 +1,154 @@
|
||||
"""Add structured metadata indexes for entity frontmatter
|
||||
|
||||
Revision ID: d7e8f9a0b1c2
|
||||
Revises: g9a0b3c4d5e6
|
||||
Create Date: 2026-01-31 12:00:00.000000
|
||||
|
||||
"""
|
||||
|
||||
from typing import Sequence, Union
|
||||
|
||||
import sqlalchemy as sa
|
||||
from alembic import op
|
||||
from sqlalchemy import text
|
||||
|
||||
|
||||
def column_exists(connection, table: str, column: str) -> bool:
|
||||
"""Check if a column exists in a table (idempotent migration support)."""
|
||||
if connection.dialect.name == "postgresql":
|
||||
result = connection.execute(
|
||||
text(
|
||||
"SELECT 1 FROM information_schema.columns "
|
||||
"WHERE table_name = :table AND column_name = :column"
|
||||
),
|
||||
{"table": table, "column": column},
|
||||
)
|
||||
return result.fetchone() is not None
|
||||
# SQLite
|
||||
result = connection.execute(text(f"PRAGMA table_info({table})"))
|
||||
columns = [row[1] for row in result]
|
||||
return column in columns
|
||||
|
||||
|
||||
def index_exists(connection, index_name: str) -> bool:
|
||||
"""Check if an index exists (idempotent migration support)."""
|
||||
if connection.dialect.name == "postgresql":
|
||||
result = connection.execute(
|
||||
text("SELECT 1 FROM pg_indexes WHERE indexname = :index_name"),
|
||||
{"index_name": index_name},
|
||||
)
|
||||
return result.fetchone() is not None
|
||||
# SQLite
|
||||
result = connection.execute(
|
||||
text("SELECT 1 FROM sqlite_master WHERE type='index' AND name = :index_name"),
|
||||
{"index_name": index_name},
|
||||
)
|
||||
return result.fetchone() is not None
|
||||
|
||||
|
||||
# revision identifiers, used by Alembic.
|
||||
revision: str = "d7e8f9a0b1c2"
|
||||
down_revision: Union[str, None] = "6830751f5fb6"
|
||||
branch_labels: Union[str, Sequence[str], None] = None
|
||||
depends_on: Union[str, Sequence[str], None] = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
"""Add JSONB/GiN indexes for Postgres and generated columns for SQLite."""
|
||||
connection = op.get_bind()
|
||||
dialect = connection.dialect.name
|
||||
|
||||
if dialect == "postgresql":
|
||||
# Ensure JSONB for efficient indexing
|
||||
result = connection.execute(
|
||||
text(
|
||||
"SELECT data_type FROM information_schema.columns "
|
||||
"WHERE table_name = 'entity' AND column_name = 'entity_metadata'"
|
||||
)
|
||||
).fetchone()
|
||||
if result and result[0] != "jsonb":
|
||||
op.execute(
|
||||
"ALTER TABLE entity ALTER COLUMN entity_metadata "
|
||||
"TYPE jsonb USING entity_metadata::jsonb"
|
||||
)
|
||||
|
||||
# General JSONB GIN index
|
||||
op.execute(
|
||||
"CREATE INDEX IF NOT EXISTS idx_entity_metadata_gin "
|
||||
"ON entity USING GIN (entity_metadata jsonb_path_ops)"
|
||||
)
|
||||
|
||||
# Common field indexes
|
||||
op.execute(
|
||||
"CREATE INDEX IF NOT EXISTS idx_entity_tags_json "
|
||||
"ON entity USING GIN ((entity_metadata -> 'tags'))"
|
||||
)
|
||||
op.execute(
|
||||
"CREATE INDEX IF NOT EXISTS idx_entity_frontmatter_type "
|
||||
"ON entity ((entity_metadata ->> 'type'))"
|
||||
)
|
||||
op.execute(
|
||||
"CREATE INDEX IF NOT EXISTS idx_entity_frontmatter_status "
|
||||
"ON entity ((entity_metadata ->> 'status'))"
|
||||
)
|
||||
return
|
||||
|
||||
# SQLite: add generated columns for common frontmatter fields
|
||||
# Constraint: SQLite ALTER TABLE ADD COLUMN only supports VIRTUAL generated columns,
|
||||
# not STORED. json_extract is deterministic so VIRTUAL columns can still be indexed.
|
||||
if not column_exists(connection, "entity", "tags_json"):
|
||||
op.add_column(
|
||||
"entity",
|
||||
sa.Column(
|
||||
"tags_json",
|
||||
sa.Text(),
|
||||
sa.Computed("json_extract(entity_metadata, '$.tags')", persisted=False),
|
||||
),
|
||||
)
|
||||
if not column_exists(connection, "entity", "frontmatter_status"):
|
||||
op.add_column(
|
||||
"entity",
|
||||
sa.Column(
|
||||
"frontmatter_status",
|
||||
sa.Text(),
|
||||
sa.Computed("json_extract(entity_metadata, '$.status')", persisted=False),
|
||||
),
|
||||
)
|
||||
if not column_exists(connection, "entity", "frontmatter_type"):
|
||||
op.add_column(
|
||||
"entity",
|
||||
sa.Column(
|
||||
"frontmatter_type",
|
||||
sa.Text(),
|
||||
sa.Computed("json_extract(entity_metadata, '$.type')", persisted=False),
|
||||
),
|
||||
)
|
||||
|
||||
# Index generated columns
|
||||
if not index_exists(connection, "idx_entity_tags_json"):
|
||||
op.create_index("idx_entity_tags_json", "entity", ["tags_json"])
|
||||
if not index_exists(connection, "idx_entity_frontmatter_status"):
|
||||
op.create_index("idx_entity_frontmatter_status", "entity", ["frontmatter_status"])
|
||||
if not index_exists(connection, "idx_entity_frontmatter_type"):
|
||||
op.create_index("idx_entity_frontmatter_type", "entity", ["frontmatter_type"])
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
"""Best-effort downgrade (drop indexes, revert JSONB on Postgres)."""
|
||||
connection = op.get_bind()
|
||||
dialect = connection.dialect.name
|
||||
|
||||
if dialect == "postgresql":
|
||||
op.execute("DROP INDEX IF EXISTS idx_entity_frontmatter_status")
|
||||
op.execute("DROP INDEX IF EXISTS idx_entity_frontmatter_type")
|
||||
op.execute("DROP INDEX IF EXISTS idx_entity_tags_json")
|
||||
op.execute("DROP INDEX IF EXISTS idx_entity_metadata_gin")
|
||||
op.execute(
|
||||
"ALTER TABLE entity ALTER COLUMN entity_metadata TYPE json USING entity_metadata::json"
|
||||
)
|
||||
return
|
||||
|
||||
# SQLite: drop indexes (dropping generated columns requires table rebuild)
|
||||
op.execute("DROP INDEX IF EXISTS idx_entity_frontmatter_status")
|
||||
op.execute("DROP INDEX IF EXISTS idx_entity_frontmatter_type")
|
||||
op.execute("DROP INDEX IF EXISTS idx_entity_tags_json")
|
||||
+42
-24
@@ -2,23 +2,13 @@
|
||||
|
||||
from contextlib import asynccontextmanager
|
||||
|
||||
from fastapi import FastAPI, HTTPException
|
||||
from fastapi import FastAPI, HTTPException, Request
|
||||
from fastapi.exception_handlers import http_exception_handler
|
||||
from fastapi.routing import APIRouter
|
||||
from loguru import logger
|
||||
|
||||
from basic_memory import __version__ as version
|
||||
from basic_memory.api.container import ApiContainer, set_container
|
||||
from basic_memory.api.routers import (
|
||||
directory_router,
|
||||
importer_router,
|
||||
knowledge,
|
||||
management,
|
||||
memory,
|
||||
project,
|
||||
resource,
|
||||
search,
|
||||
prompt_router,
|
||||
)
|
||||
from basic_memory.api.v2.routers import (
|
||||
knowledge_router as v2_knowledge,
|
||||
project_router as v2_project,
|
||||
@@ -29,7 +19,13 @@ from basic_memory.api.v2.routers import (
|
||||
prompt_router as v2_prompt,
|
||||
importer_router as v2_importer,
|
||||
)
|
||||
from basic_memory.api.v2.routers.project_router import (
|
||||
add_project,
|
||||
list_projects,
|
||||
synchronize_projects,
|
||||
)
|
||||
from basic_memory.config import init_api_logging
|
||||
from basic_memory.services.exceptions import EntityAlreadyExistsError
|
||||
from basic_memory.services.initialization import initialize_app
|
||||
|
||||
|
||||
@@ -90,19 +86,41 @@ app.include_router(v2_prompt, prefix="/v2/projects/{project_id}")
|
||||
app.include_router(v2_importer, prefix="/v2/projects/{project_id}")
|
||||
app.include_router(v2_project, prefix="/v2")
|
||||
|
||||
# Include v1 routers (/{project} is a catch-all, must come after specific prefixes)
|
||||
app.include_router(knowledge.router, prefix="/{project}")
|
||||
app.include_router(memory.router, prefix="/{project}")
|
||||
app.include_router(resource.router, prefix="/{project}")
|
||||
app.include_router(search.router, prefix="/{project}")
|
||||
app.include_router(project.project_router, prefix="/{project}")
|
||||
app.include_router(directory_router.router, prefix="/{project}")
|
||||
app.include_router(prompt_router.router, prefix="/{project}")
|
||||
app.include_router(importer_router.router, prefix="/{project}")
|
||||
# Legacy web app proxy paths (compat with /proxy/projects/projects)
|
||||
app.include_router(v2_project, prefix="/proxy/projects")
|
||||
|
||||
# Project resource router works across projects
|
||||
app.include_router(project.project_resource_router)
|
||||
app.include_router(management.router)
|
||||
# Legacy v1 compat: older CLI versions (v0.18.0 and earlier) call /projects/...
|
||||
# Using router mount causes 307 redirect which proxy doesn't follow, so add explicit routes
|
||||
legacy_router = APIRouter(tags=["legacy"])
|
||||
legacy_router.add_api_route("/projects/projects", list_projects, methods=["GET"])
|
||||
legacy_router.add_api_route("/projects/projects", add_project, methods=["POST"])
|
||||
legacy_router.add_api_route("/projects/config/sync", synchronize_projects, methods=["POST"])
|
||||
app.include_router(legacy_router)
|
||||
|
||||
# V2 routers are the only public API surface
|
||||
|
||||
|
||||
@app.exception_handler(EntityAlreadyExistsError)
|
||||
async def entity_already_exists_error_handler(request: Request, exc: EntityAlreadyExistsError):
|
||||
"""Handle entity creation conflicts (e.g., file already exists).
|
||||
|
||||
This is expected behavior when users try to create notes that exist,
|
||||
so log at INFO level instead of ERROR.
|
||||
"""
|
||||
logger.info(
|
||||
"Entity already exists",
|
||||
url=str(request.url),
|
||||
method=request.method,
|
||||
path=request.url.path,
|
||||
error=str(exc),
|
||||
)
|
||||
return await http_exception_handler(
|
||||
request,
|
||||
HTTPException(
|
||||
status_code=409,
|
||||
detail="Note already exists. Use edit_note to modify it, or delete it first.",
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
@app.exception_handler(Exception)
|
||||
|
||||
@@ -1,11 +0,0 @@
|
||||
"""API routers."""
|
||||
|
||||
from . import knowledge_router as knowledge
|
||||
from . import management_router as management
|
||||
from . import memory_router as memory
|
||||
from . import project_router as project
|
||||
from . import resource_router as resource
|
||||
from . import search_router as search
|
||||
from . import prompt_router as prompt
|
||||
|
||||
__all__ = ["knowledge", "management", "memory", "project", "resource", "search", "prompt"]
|
||||
@@ -1,84 +0,0 @@
|
||||
"""Router for directory tree operations."""
|
||||
|
||||
from typing import List, Optional
|
||||
|
||||
from fastapi import APIRouter, Query
|
||||
|
||||
from basic_memory.deps import DirectoryServiceDep, ProjectIdDep
|
||||
from basic_memory.schemas.directory import DirectoryNode
|
||||
|
||||
router = APIRouter(prefix="/directory", tags=["directory"])
|
||||
|
||||
|
||||
@router.get("/tree", response_model=DirectoryNode, response_model_exclude_none=True)
|
||||
async def get_directory_tree(
|
||||
directory_service: DirectoryServiceDep,
|
||||
project_id: ProjectIdDep,
|
||||
):
|
||||
"""Get hierarchical directory structure from the knowledge base.
|
||||
|
||||
Args:
|
||||
directory_service: Service for directory operations
|
||||
project_id: ID of the current project
|
||||
|
||||
Returns:
|
||||
DirectoryNode representing the root of the hierarchical tree structure
|
||||
"""
|
||||
# Get a hierarchical directory tree for the specific project
|
||||
tree = await directory_service.get_directory_tree()
|
||||
|
||||
# Return the hierarchical tree
|
||||
return tree
|
||||
|
||||
|
||||
@router.get("/structure", response_model=DirectoryNode, response_model_exclude_none=True)
|
||||
async def get_directory_structure(
|
||||
directory_service: DirectoryServiceDep,
|
||||
project_id: ProjectIdDep,
|
||||
):
|
||||
"""Get folder structure for navigation (no files).
|
||||
|
||||
Optimized endpoint for folder tree navigation. Returns only directory nodes
|
||||
without file metadata. For full tree with files, use /directory/tree.
|
||||
|
||||
Args:
|
||||
directory_service: Service for directory operations
|
||||
project_id: ID of the current project
|
||||
|
||||
Returns:
|
||||
DirectoryNode tree containing only folders (type="directory")
|
||||
"""
|
||||
structure = await directory_service.get_directory_structure()
|
||||
return structure
|
||||
|
||||
|
||||
@router.get("/list", response_model=List[DirectoryNode], response_model_exclude_none=True)
|
||||
async def list_directory(
|
||||
directory_service: DirectoryServiceDep,
|
||||
project_id: ProjectIdDep,
|
||||
dir_name: str = Query("/", description="Directory path to list"),
|
||||
depth: int = Query(1, ge=1, le=10, description="Recursion depth (1-10)"),
|
||||
file_name_glob: Optional[str] = Query(
|
||||
None, description="Glob pattern for filtering file names"
|
||||
),
|
||||
):
|
||||
"""List directory contents with filtering and depth control.
|
||||
|
||||
Args:
|
||||
directory_service: Service for directory operations
|
||||
project_id: ID of the current project
|
||||
dir_name: Directory path to list (default: root "/")
|
||||
depth: Recursion depth (1-10, default: 1 for immediate children only)
|
||||
file_name_glob: Optional glob pattern for filtering file names (e.g., "*.md", "*meeting*")
|
||||
|
||||
Returns:
|
||||
List of DirectoryNode objects matching the criteria
|
||||
"""
|
||||
# Get directory listing with filtering
|
||||
nodes = await directory_service.list_directory(
|
||||
dir_name=dir_name,
|
||||
depth=depth,
|
||||
file_name_glob=file_name_glob,
|
||||
)
|
||||
|
||||
return nodes
|
||||
@@ -1,152 +0,0 @@
|
||||
"""Import router for Basic Memory API."""
|
||||
|
||||
import json
|
||||
import logging
|
||||
|
||||
from fastapi import APIRouter, Form, HTTPException, UploadFile, status
|
||||
|
||||
from basic_memory.deps import (
|
||||
ChatGPTImporterDep,
|
||||
ClaudeConversationsImporterDep,
|
||||
ClaudeProjectsImporterDep,
|
||||
MemoryJsonImporterDep,
|
||||
)
|
||||
from basic_memory.importers import Importer
|
||||
from basic_memory.schemas.importer import (
|
||||
ChatImportResult,
|
||||
EntityImportResult,
|
||||
ProjectImportResult,
|
||||
)
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
router = APIRouter(prefix="/import", tags=["import"])
|
||||
|
||||
|
||||
@router.post("/chatgpt", response_model=ChatImportResult)
|
||||
async def import_chatgpt(
|
||||
importer: ChatGPTImporterDep,
|
||||
file: UploadFile,
|
||||
folder: str = Form("conversations"),
|
||||
) -> ChatImportResult:
|
||||
"""Import conversations from ChatGPT JSON export.
|
||||
|
||||
Args:
|
||||
file: The ChatGPT conversations.json file.
|
||||
folder: The folder to place the files in.
|
||||
markdown_processor: MarkdownProcessor instance.
|
||||
|
||||
Returns:
|
||||
ChatImportResult with import statistics.
|
||||
|
||||
Raises:
|
||||
HTTPException: If import fails.
|
||||
"""
|
||||
return await import_file(importer, file, folder)
|
||||
|
||||
|
||||
@router.post("/claude/conversations", response_model=ChatImportResult)
|
||||
async def import_claude_conversations(
|
||||
importer: ClaudeConversationsImporterDep,
|
||||
file: UploadFile,
|
||||
folder: str = Form("conversations"),
|
||||
) -> ChatImportResult:
|
||||
"""Import conversations from Claude conversations.json export.
|
||||
|
||||
Args:
|
||||
file: The Claude conversations.json file.
|
||||
folder: The folder to place the files in.
|
||||
markdown_processor: MarkdownProcessor instance.
|
||||
|
||||
Returns:
|
||||
ChatImportResult with import statistics.
|
||||
|
||||
Raises:
|
||||
HTTPException: If import fails.
|
||||
"""
|
||||
return await import_file(importer, file, folder)
|
||||
|
||||
|
||||
@router.post("/claude/projects", response_model=ProjectImportResult)
|
||||
async def import_claude_projects(
|
||||
importer: ClaudeProjectsImporterDep,
|
||||
file: UploadFile,
|
||||
folder: str = Form("projects"),
|
||||
) -> ProjectImportResult:
|
||||
"""Import projects from Claude projects.json export.
|
||||
|
||||
Args:
|
||||
file: The Claude projects.json file.
|
||||
base_folder: The base folder to place the files in.
|
||||
markdown_processor: MarkdownProcessor instance.
|
||||
|
||||
Returns:
|
||||
ProjectImportResult with import statistics.
|
||||
|
||||
Raises:
|
||||
HTTPException: If import fails.
|
||||
"""
|
||||
return await import_file(importer, file, folder)
|
||||
|
||||
|
||||
@router.post("/memory-json", response_model=EntityImportResult)
|
||||
async def import_memory_json(
|
||||
importer: MemoryJsonImporterDep,
|
||||
file: UploadFile,
|
||||
folder: str = Form("conversations"),
|
||||
) -> EntityImportResult:
|
||||
"""Import entities and relations from a memory.json file.
|
||||
|
||||
Args:
|
||||
file: The memory.json file.
|
||||
destination_folder: Optional destination folder within the project.
|
||||
markdown_processor: MarkdownProcessor instance.
|
||||
|
||||
Returns:
|
||||
EntityImportResult with import statistics.
|
||||
|
||||
Raises:
|
||||
HTTPException: If import fails.
|
||||
"""
|
||||
try:
|
||||
file_data = []
|
||||
file_bytes = await file.read()
|
||||
file_str = file_bytes.decode("utf-8")
|
||||
for line in file_str.splitlines():
|
||||
json_data = json.loads(line)
|
||||
file_data.append(json_data)
|
||||
|
||||
result = await importer.import_data(file_data, folder)
|
||||
if not result.success: # pragma: no cover
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
|
||||
detail=result.error_message or "Import failed",
|
||||
)
|
||||
except Exception as e:
|
||||
logger.exception("Import failed")
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
|
||||
detail=f"Import failed: {str(e)}",
|
||||
)
|
||||
return result
|
||||
|
||||
|
||||
async def import_file(importer: Importer, file: UploadFile, destination_folder: str):
|
||||
try:
|
||||
# Process file
|
||||
json_data = json.load(file.file)
|
||||
result = await importer.import_data(json_data, destination_folder)
|
||||
if not result.success: # pragma: no cover
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
|
||||
detail=result.error_message or "Import failed",
|
||||
)
|
||||
|
||||
return result
|
||||
|
||||
except Exception as e:
|
||||
logger.exception("Import failed")
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
|
||||
detail=f"Import failed: {str(e)}",
|
||||
)
|
||||
@@ -1,319 +0,0 @@
|
||||
"""Router for knowledge graph operations.
|
||||
|
||||
⚠️ DEPRECATED: This v1 API is deprecated and will be removed on June 30, 2026.
|
||||
Please migrate to /v2/{project}/knowledge endpoints which use entity IDs instead
|
||||
of path-based identifiers for improved performance and stability.
|
||||
|
||||
Migration guide: See docs/migration/v1-to-v2.md
|
||||
"""
|
||||
|
||||
from typing import Annotated
|
||||
|
||||
from fastapi import APIRouter, HTTPException, BackgroundTasks, Depends, Query, Response
|
||||
from loguru import logger
|
||||
|
||||
from basic_memory.deps import (
|
||||
EntityServiceDep,
|
||||
get_search_service,
|
||||
SearchServiceDep,
|
||||
LinkResolverDep,
|
||||
ProjectPathDep,
|
||||
FileServiceDep,
|
||||
ProjectConfigDep,
|
||||
AppConfigDep,
|
||||
SyncServiceDep,
|
||||
)
|
||||
from basic_memory.schemas import (
|
||||
EntityListResponse,
|
||||
EntityResponse,
|
||||
DeleteEntitiesResponse,
|
||||
DeleteEntitiesRequest,
|
||||
)
|
||||
from basic_memory.schemas.request import EditEntityRequest, MoveEntityRequest
|
||||
from basic_memory.schemas.base import Permalink, Entity
|
||||
|
||||
router = APIRouter(
|
||||
prefix="/knowledge",
|
||||
tags=["knowledge"],
|
||||
deprecated=True, # Marks entire router as deprecated in OpenAPI docs
|
||||
)
|
||||
|
||||
|
||||
async def resolve_relations_background(sync_service, entity_id: int, entity_permalink: str) -> None:
|
||||
"""Background task to resolve relations for a specific entity.
|
||||
|
||||
This runs asynchronously after the API response is sent, preventing
|
||||
long delays when creating entities with many relations.
|
||||
"""
|
||||
try:
|
||||
# Only resolve relations for the newly created entity
|
||||
await sync_service.resolve_relations(entity_id=entity_id)
|
||||
logger.debug(
|
||||
f"Background: Resolved relations for entity {entity_permalink} (id={entity_id})"
|
||||
)
|
||||
except Exception as e: # pragma: no cover
|
||||
# Log but don't fail - this is a background task.
|
||||
# Avoid forcing synthetic failures just for coverage.
|
||||
logger.warning( # pragma: no cover
|
||||
f"Background: Failed to resolve relations for entity {entity_permalink}: {e}"
|
||||
)
|
||||
|
||||
|
||||
## Create endpoints
|
||||
|
||||
|
||||
@router.post("/entities", response_model=EntityResponse)
|
||||
async def create_entity(
|
||||
data: Entity,
|
||||
background_tasks: BackgroundTasks,
|
||||
entity_service: EntityServiceDep,
|
||||
search_service: SearchServiceDep,
|
||||
) -> EntityResponse:
|
||||
"""Create an entity."""
|
||||
logger.info(
|
||||
"API request", endpoint="create_entity", entity_type=data.entity_type, title=data.title
|
||||
)
|
||||
|
||||
entity = await entity_service.create_entity(data)
|
||||
|
||||
# reindex
|
||||
await search_service.index_entity(entity, background_tasks=background_tasks)
|
||||
result = EntityResponse.model_validate(entity)
|
||||
|
||||
logger.info(
|
||||
f"API response: endpoint='create_entity' title={result.title}, permalink={result.permalink}, status_code=201"
|
||||
)
|
||||
return result
|
||||
|
||||
|
||||
@router.put("/entities/{permalink:path}", response_model=EntityResponse)
|
||||
async def create_or_update_entity(
|
||||
project: ProjectPathDep,
|
||||
permalink: Permalink,
|
||||
data: Entity,
|
||||
response: Response,
|
||||
background_tasks: BackgroundTasks,
|
||||
entity_service: EntityServiceDep,
|
||||
search_service: SearchServiceDep,
|
||||
file_service: FileServiceDep,
|
||||
sync_service: SyncServiceDep,
|
||||
) -> EntityResponse:
|
||||
"""Create or update an entity. If entity exists, it will be updated, otherwise created."""
|
||||
logger.info(
|
||||
f"API request: create_or_update_entity for {project=}, {permalink=}, {data.entity_type=}, {data.title=}"
|
||||
)
|
||||
|
||||
# Validate permalink matches
|
||||
if data.permalink != permalink:
|
||||
logger.warning(
|
||||
f"API validation error: creating/updating entity with permalink mismatch - url={permalink}, data={data.permalink}",
|
||||
)
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
detail=f"Entity permalink {data.permalink} must match URL path: '{permalink}'",
|
||||
)
|
||||
|
||||
# Try create_or_update operation
|
||||
entity, created = await entity_service.create_or_update_entity(data)
|
||||
response.status_code = 201 if created else 200
|
||||
|
||||
# reindex
|
||||
await search_service.index_entity(entity, background_tasks=background_tasks)
|
||||
|
||||
# Schedule relation resolution as a background task for new entities
|
||||
# This prevents blocking the API response while resolving potentially many relations
|
||||
if created:
|
||||
background_tasks.add_task(
|
||||
resolve_relations_background, sync_service, entity.id, entity.permalink or ""
|
||||
)
|
||||
|
||||
result = EntityResponse.model_validate(entity)
|
||||
|
||||
logger.info(
|
||||
f"API response: {result.title=}, {result.permalink=}, {created=}, status_code={response.status_code}"
|
||||
)
|
||||
return result
|
||||
|
||||
|
||||
@router.patch("/entities/{identifier:path}", response_model=EntityResponse)
|
||||
async def edit_entity(
|
||||
identifier: str,
|
||||
data: EditEntityRequest,
|
||||
background_tasks: BackgroundTasks,
|
||||
entity_service: EntityServiceDep,
|
||||
search_service: SearchServiceDep,
|
||||
) -> EntityResponse:
|
||||
"""Edit an existing entity using various operations like append, prepend, find_replace, or replace_section.
|
||||
|
||||
This endpoint allows for targeted edits without requiring the full entity content.
|
||||
"""
|
||||
logger.info(
|
||||
f"API request: endpoint='edit_entity', identifier='{identifier}', operation='{data.operation}'"
|
||||
)
|
||||
|
||||
try:
|
||||
# Edit the entity using the service
|
||||
entity = await entity_service.edit_entity(
|
||||
identifier=identifier,
|
||||
operation=data.operation,
|
||||
content=data.content,
|
||||
section=data.section,
|
||||
find_text=data.find_text,
|
||||
expected_replacements=data.expected_replacements,
|
||||
)
|
||||
|
||||
# Reindex the updated entity
|
||||
await search_service.index_entity(entity, background_tasks=background_tasks)
|
||||
|
||||
# Return the updated entity response
|
||||
result = EntityResponse.model_validate(entity)
|
||||
|
||||
logger.info(
|
||||
"API response",
|
||||
endpoint="edit_entity",
|
||||
identifier=identifier,
|
||||
operation=data.operation,
|
||||
permalink=result.permalink,
|
||||
status_code=200,
|
||||
)
|
||||
|
||||
return result
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Error editing entity: {e}")
|
||||
raise HTTPException(status_code=400, detail=str(e))
|
||||
|
||||
|
||||
@router.post("/move")
|
||||
async def move_entity(
|
||||
data: MoveEntityRequest,
|
||||
background_tasks: BackgroundTasks,
|
||||
entity_service: EntityServiceDep,
|
||||
project_config: ProjectConfigDep,
|
||||
app_config: AppConfigDep,
|
||||
search_service: SearchServiceDep,
|
||||
) -> EntityResponse:
|
||||
"""Move an entity to a new file location with project consistency.
|
||||
|
||||
This endpoint moves a note to a different path while maintaining project
|
||||
consistency and optionally updating permalinks based on configuration.
|
||||
"""
|
||||
logger.info(
|
||||
f"API request: endpoint='move_entity', identifier='{data.identifier}', destination='{data.destination_path}'"
|
||||
)
|
||||
|
||||
try:
|
||||
# Move the entity using the service
|
||||
moved_entity = await entity_service.move_entity(
|
||||
identifier=data.identifier,
|
||||
destination_path=data.destination_path,
|
||||
project_config=project_config,
|
||||
app_config=app_config,
|
||||
)
|
||||
|
||||
# Get the moved entity to reindex it
|
||||
entity = await entity_service.link_resolver.resolve_link(data.destination_path)
|
||||
if entity:
|
||||
await search_service.index_entity(entity, background_tasks=background_tasks)
|
||||
|
||||
logger.info(
|
||||
"API response",
|
||||
endpoint="move_entity",
|
||||
identifier=data.identifier,
|
||||
destination=data.destination_path,
|
||||
status_code=200,
|
||||
)
|
||||
result = EntityResponse.model_validate(moved_entity)
|
||||
return result
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Error moving entity: {e}")
|
||||
raise HTTPException(status_code=400, detail=str(e))
|
||||
|
||||
|
||||
## Read endpoints
|
||||
|
||||
|
||||
@router.get("/entities/{identifier:path}", response_model=EntityResponse)
|
||||
async def get_entity(
|
||||
entity_service: EntityServiceDep,
|
||||
link_resolver: LinkResolverDep,
|
||||
identifier: str,
|
||||
) -> EntityResponse:
|
||||
"""Get a specific entity by file path or permalink..
|
||||
|
||||
Args:
|
||||
identifier: Entity file path or permalink
|
||||
:param entity_service: EntityService
|
||||
:param link_resolver: LinkResolver
|
||||
"""
|
||||
logger.info(f"request: get_entity with identifier={identifier}")
|
||||
entity = await link_resolver.resolve_link(identifier)
|
||||
if not entity:
|
||||
raise HTTPException(status_code=404, detail=f"Entity {identifier} not found")
|
||||
|
||||
result = EntityResponse.model_validate(entity)
|
||||
return result
|
||||
|
||||
|
||||
@router.get("/entities", response_model=EntityListResponse)
|
||||
async def get_entities(
|
||||
entity_service: EntityServiceDep,
|
||||
permalink: Annotated[list[str] | None, Query()] = None,
|
||||
) -> EntityListResponse:
|
||||
"""Open specific entities"""
|
||||
logger.info(f"request: get_entities with permalinks={permalink}")
|
||||
|
||||
entities = await entity_service.get_entities_by_permalinks(permalink) if permalink else []
|
||||
result = EntityListResponse(
|
||||
entities=[EntityResponse.model_validate(entity) for entity in entities]
|
||||
)
|
||||
return result
|
||||
|
||||
|
||||
## Delete endpoints
|
||||
|
||||
|
||||
@router.delete("/entities/{identifier:path}", response_model=DeleteEntitiesResponse)
|
||||
async def delete_entity(
|
||||
identifier: str,
|
||||
background_tasks: BackgroundTasks,
|
||||
entity_service: EntityServiceDep,
|
||||
link_resolver: LinkResolverDep,
|
||||
search_service=Depends(get_search_service),
|
||||
) -> DeleteEntitiesResponse:
|
||||
"""Delete a single entity and remove from search index."""
|
||||
logger.info(f"request: delete_entity with identifier={identifier}")
|
||||
|
||||
entity = await link_resolver.resolve_link(identifier)
|
||||
if entity is None:
|
||||
return DeleteEntitiesResponse(deleted=False)
|
||||
|
||||
# Delete the entity
|
||||
deleted = await entity_service.delete_entity(entity.permalink or entity.id)
|
||||
|
||||
# Remove from search index (entity, observations, and relations)
|
||||
background_tasks.add_task(search_service.handle_delete, entity)
|
||||
|
||||
result = DeleteEntitiesResponse(deleted=deleted)
|
||||
return result
|
||||
|
||||
|
||||
@router.post("/entities/delete", response_model=DeleteEntitiesResponse)
|
||||
async def delete_entities(
|
||||
data: DeleteEntitiesRequest,
|
||||
background_tasks: BackgroundTasks,
|
||||
entity_service: EntityServiceDep,
|
||||
search_service=Depends(get_search_service),
|
||||
) -> DeleteEntitiesResponse:
|
||||
"""Delete entities and remove from search index."""
|
||||
logger.info(f"request: delete_entities with data={data}")
|
||||
deleted = False
|
||||
|
||||
# Remove each deleted entity from search index
|
||||
for permalink in data.permalinks:
|
||||
deleted = await entity_service.delete_entity(permalink)
|
||||
background_tasks.add_task(search_service.delete_by_permalink, permalink)
|
||||
|
||||
result = DeleteEntitiesResponse(deleted=deleted)
|
||||
return result
|
||||
@@ -1,80 +0,0 @@
|
||||
"""Management router for basic-memory API."""
|
||||
|
||||
import asyncio
|
||||
|
||||
from fastapi import APIRouter, Request
|
||||
from loguru import logger
|
||||
from pydantic import BaseModel
|
||||
|
||||
from basic_memory.config import ConfigManager
|
||||
from basic_memory.deps import SyncServiceDep, ProjectRepositoryDep
|
||||
|
||||
router = APIRouter(prefix="/management", tags=["management"])
|
||||
|
||||
|
||||
class WatchStatusResponse(BaseModel):
|
||||
"""Response model for watch status."""
|
||||
|
||||
running: bool
|
||||
"""Whether the watch service is currently running."""
|
||||
|
||||
|
||||
@router.get("/watch/status", response_model=WatchStatusResponse)
|
||||
async def get_watch_status(request: Request) -> WatchStatusResponse:
|
||||
"""Get the current status of the watch service."""
|
||||
return WatchStatusResponse(
|
||||
running=request.app.state.watch_task is not None and not request.app.state.watch_task.done()
|
||||
)
|
||||
|
||||
|
||||
@router.post("/watch/start", response_model=WatchStatusResponse)
|
||||
async def start_watch_service(
|
||||
request: Request, project_repository: ProjectRepositoryDep, sync_service: SyncServiceDep
|
||||
) -> WatchStatusResponse:
|
||||
"""Start the watch service if it's not already running."""
|
||||
|
||||
# needed because of circular imports from sync -> app
|
||||
from basic_memory.sync import WatchService
|
||||
from basic_memory.sync.background_sync import create_background_sync_task
|
||||
|
||||
if request.app.state.watch_task is not None and not request.app.state.watch_task.done():
|
||||
# Watch service is already running
|
||||
return WatchStatusResponse(running=True)
|
||||
|
||||
app_config = ConfigManager().config
|
||||
|
||||
# Create and start a new watch service
|
||||
logger.info("Starting watch service via management API")
|
||||
|
||||
# Get services needed for the watch task
|
||||
watch_service = WatchService(
|
||||
app_config=app_config,
|
||||
project_repository=project_repository,
|
||||
)
|
||||
|
||||
# Create and store the task
|
||||
watch_task = create_background_sync_task(sync_service, watch_service)
|
||||
request.app.state.watch_task = watch_task
|
||||
|
||||
return WatchStatusResponse(running=True)
|
||||
|
||||
|
||||
@router.post("/watch/stop", response_model=WatchStatusResponse)
|
||||
async def stop_watch_service(request: Request) -> WatchStatusResponse: # pragma: no cover
|
||||
"""Stop the watch service if it's running."""
|
||||
if request.app.state.watch_task is None or request.app.state.watch_task.done():
|
||||
# Watch service is not running
|
||||
return WatchStatusResponse(running=False)
|
||||
|
||||
# Cancel the running task
|
||||
logger.info("Stopping watch service via management API")
|
||||
request.app.state.watch_task.cancel()
|
||||
|
||||
# Wait for it to be properly cancelled
|
||||
try:
|
||||
await request.app.state.watch_task
|
||||
except asyncio.CancelledError:
|
||||
pass
|
||||
|
||||
request.app.state.watch_task = None
|
||||
return WatchStatusResponse(running=False)
|
||||
@@ -1,90 +0,0 @@
|
||||
"""Routes for memory:// URI operations."""
|
||||
|
||||
from typing import Annotated, Optional
|
||||
|
||||
from fastapi import APIRouter, Query
|
||||
from loguru import logger
|
||||
|
||||
from basic_memory.deps import ContextServiceDep, EntityRepositoryDep
|
||||
from basic_memory.schemas.base import TimeFrame, parse_timeframe
|
||||
from basic_memory.schemas.memory import (
|
||||
GraphContext,
|
||||
normalize_memory_url,
|
||||
)
|
||||
from basic_memory.schemas.search import SearchItemType
|
||||
from basic_memory.api.routers.utils import to_graph_context
|
||||
|
||||
router = APIRouter(prefix="/memory", tags=["memory"])
|
||||
|
||||
|
||||
@router.get("/recent", response_model=GraphContext)
|
||||
async def recent(
|
||||
context_service: ContextServiceDep,
|
||||
entity_repository: EntityRepositoryDep,
|
||||
type: Annotated[list[SearchItemType] | None, Query()] = None,
|
||||
depth: int = 1,
|
||||
timeframe: TimeFrame = "7d",
|
||||
page: int = 1,
|
||||
page_size: int = 10,
|
||||
max_related: int = 10,
|
||||
) -> GraphContext:
|
||||
# return all types by default
|
||||
types = (
|
||||
[SearchItemType.ENTITY, SearchItemType.RELATION, SearchItemType.OBSERVATION]
|
||||
if not type
|
||||
else type
|
||||
)
|
||||
|
||||
logger.debug(
|
||||
f"Getting recent context: `{types}` depth: `{depth}` timeframe: `{timeframe}` page: `{page}` page_size: `{page_size}` max_related: `{max_related}`"
|
||||
)
|
||||
# Parse timeframe
|
||||
since = parse_timeframe(timeframe)
|
||||
limit = page_size
|
||||
offset = (page - 1) * page_size
|
||||
|
||||
# Build context
|
||||
context = await context_service.build_context(
|
||||
types=types, depth=depth, since=since, limit=limit, offset=offset, max_related=max_related
|
||||
)
|
||||
recent_context = await to_graph_context(
|
||||
context, entity_repository=entity_repository, page=page, page_size=page_size
|
||||
)
|
||||
logger.debug(f"Recent context: {recent_context.model_dump_json()}")
|
||||
return recent_context
|
||||
|
||||
|
||||
# get_memory_context needs to be declared last so other paths can match
|
||||
|
||||
|
||||
@router.get("/{uri:path}", response_model=GraphContext)
|
||||
async def get_memory_context(
|
||||
context_service: ContextServiceDep,
|
||||
entity_repository: EntityRepositoryDep,
|
||||
uri: str,
|
||||
depth: int = 1,
|
||||
timeframe: Optional[TimeFrame] = None,
|
||||
page: int = 1,
|
||||
page_size: int = 10,
|
||||
max_related: int = 10,
|
||||
) -> GraphContext:
|
||||
"""Get rich context from memory:// URI."""
|
||||
# add the project name from the config to the url as the "host
|
||||
# Parse URI
|
||||
logger.debug(
|
||||
f"Getting context for URI: `{uri}` depth: `{depth}` timeframe: `{timeframe}` page: `{page}` page_size: `{page_size}` max_related: `{max_related}`"
|
||||
)
|
||||
memory_url = normalize_memory_url(uri)
|
||||
|
||||
# Parse timeframe
|
||||
since = parse_timeframe(timeframe) if timeframe else None
|
||||
limit = page_size
|
||||
offset = (page - 1) * page_size
|
||||
|
||||
# Build context
|
||||
context = await context_service.build_context(
|
||||
memory_url, depth=depth, since=since, limit=limit, offset=offset, max_related=max_related
|
||||
)
|
||||
return await to_graph_context(
|
||||
context, entity_repository=entity_repository, page=page, page_size=page_size
|
||||
)
|
||||
@@ -1,472 +0,0 @@
|
||||
"""Router for project management."""
|
||||
|
||||
import os
|
||||
from fastapi import APIRouter, HTTPException, Path, Body, BackgroundTasks, Response, Query
|
||||
from typing import Optional
|
||||
from loguru import logger
|
||||
|
||||
from basic_memory.deps import (
|
||||
ProjectConfigDep,
|
||||
ProjectServiceDep,
|
||||
ProjectPathDep,
|
||||
SyncServiceDep,
|
||||
)
|
||||
from basic_memory.schemas import ProjectInfoResponse, SyncReportResponse
|
||||
from basic_memory.schemas.project_info import (
|
||||
ProjectList,
|
||||
ProjectItem,
|
||||
ProjectInfoRequest,
|
||||
ProjectStatusResponse,
|
||||
)
|
||||
from basic_memory.utils import normalize_project_path
|
||||
|
||||
# Router for resources in a specific project
|
||||
# The ProjectPathDep is used in the path as a prefix, so the request path is like /{project}/project/info
|
||||
project_router = APIRouter(prefix="/project", tags=["project"])
|
||||
|
||||
# Router for managing project resources
|
||||
project_resource_router = APIRouter(prefix="/projects", tags=["project_management"])
|
||||
|
||||
|
||||
@project_router.get("/info", response_model=ProjectInfoResponse)
|
||||
async def get_project_info(
|
||||
project_service: ProjectServiceDep,
|
||||
project: ProjectPathDep,
|
||||
) -> ProjectInfoResponse:
|
||||
"""Get comprehensive information about the specified Basic Memory project."""
|
||||
return await project_service.get_project_info(project)
|
||||
|
||||
|
||||
@project_router.get("/item", response_model=ProjectItem)
|
||||
async def get_project(
|
||||
project_service: ProjectServiceDep,
|
||||
project: ProjectPathDep,
|
||||
) -> ProjectItem:
|
||||
"""Get bassic info about the specified Basic Memory project."""
|
||||
found_project = await project_service.get_project(project)
|
||||
if not found_project:
|
||||
raise HTTPException(
|
||||
status_code=404, detail=f"Project: '{project}' does not exist"
|
||||
) # pragma: no cover
|
||||
|
||||
return ProjectItem(
|
||||
id=found_project.id,
|
||||
external_id=found_project.external_id,
|
||||
name=found_project.name,
|
||||
path=normalize_project_path(found_project.path),
|
||||
is_default=found_project.is_default or False,
|
||||
)
|
||||
|
||||
|
||||
# Update a project
|
||||
@project_router.patch("/{name}", response_model=ProjectStatusResponse)
|
||||
async def update_project(
|
||||
project_service: ProjectServiceDep,
|
||||
name: str = Path(..., description="Name of the project to update"),
|
||||
path: Optional[str] = Body(None, description="New absolute path for the project"),
|
||||
is_active: Optional[bool] = Body(None, description="Status of the project (active/inactive)"),
|
||||
) -> ProjectStatusResponse:
|
||||
"""Update a project's information in configuration and database.
|
||||
|
||||
Args:
|
||||
name: The name of the project to update
|
||||
path: Optional new absolute path for the project
|
||||
is_active: Optional status update for the project
|
||||
|
||||
Returns:
|
||||
Response confirming the project was updated
|
||||
"""
|
||||
try:
|
||||
# Validate that path is absolute if provided
|
||||
if path and not os.path.isabs(path):
|
||||
raise HTTPException(status_code=400, detail="Path must be absolute")
|
||||
|
||||
# Get original project info for the response
|
||||
old_project = await project_service.get_project(name)
|
||||
if not old_project:
|
||||
raise HTTPException(
|
||||
status_code=400, detail=f"Project '{name}' not found in configuration"
|
||||
)
|
||||
|
||||
old_project_info = ProjectItem(
|
||||
id=old_project.id,
|
||||
external_id=old_project.external_id,
|
||||
name=old_project.name,
|
||||
path=old_project.path,
|
||||
is_default=old_project.is_default or False,
|
||||
)
|
||||
|
||||
if path:
|
||||
await project_service.move_project(name, path)
|
||||
elif is_active is not None:
|
||||
await project_service.update_project(name, is_active=is_active)
|
||||
|
||||
# Get updated project info
|
||||
updated_project = await project_service.get_project(name)
|
||||
if not updated_project:
|
||||
raise HTTPException( # pragma: no cover
|
||||
status_code=404, detail=f"Project '{name}' not found after update"
|
||||
)
|
||||
|
||||
return ProjectStatusResponse(
|
||||
message=f"Project '{name}' updated successfully",
|
||||
status="success",
|
||||
default=(name == project_service.default_project),
|
||||
old_project=old_project_info,
|
||||
new_project=ProjectItem(
|
||||
id=updated_project.id,
|
||||
external_id=updated_project.external_id,
|
||||
name=updated_project.name,
|
||||
path=updated_project.path,
|
||||
is_default=updated_project.is_default or False,
|
||||
),
|
||||
)
|
||||
except ValueError as e:
|
||||
raise HTTPException(status_code=400, detail=str(e)) # pragma: no cover
|
||||
|
||||
|
||||
# Sync project filesystem
|
||||
@project_router.post("/sync")
|
||||
async def sync_project(
|
||||
background_tasks: BackgroundTasks,
|
||||
sync_service: SyncServiceDep,
|
||||
project_config: ProjectConfigDep,
|
||||
force_full: bool = Query(
|
||||
False, description="Force full scan, bypassing watermark optimization"
|
||||
),
|
||||
run_in_background: bool = Query(True, description="Run in background"),
|
||||
):
|
||||
"""Force project filesystem sync to database.
|
||||
|
||||
Scans the project directory and updates the database with any new or modified files.
|
||||
|
||||
Args:
|
||||
background_tasks: FastAPI background tasks
|
||||
sync_service: Sync service for this project
|
||||
project_config: Project configuration
|
||||
force_full: If True, force a full scan even if watermark exists
|
||||
run_in_background: If True, run sync in background and return immediately
|
||||
|
||||
Returns:
|
||||
Response confirming sync was initiated (background) or SyncReportResponse (foreground)
|
||||
"""
|
||||
if run_in_background:
|
||||
background_tasks.add_task(
|
||||
sync_service.sync, project_config.home, project_config.name, force_full=force_full
|
||||
)
|
||||
logger.info(
|
||||
f"Filesystem sync initiated for project: {project_config.name} (force_full={force_full})"
|
||||
)
|
||||
|
||||
return {
|
||||
"status": "sync_started",
|
||||
"message": f"Filesystem sync initiated for project '{project_config.name}'",
|
||||
}
|
||||
else:
|
||||
report = await sync_service.sync(
|
||||
project_config.home, project_config.name, force_full=force_full
|
||||
)
|
||||
logger.info(
|
||||
f"Filesystem sync completed for project: {project_config.name} (force_full={force_full})"
|
||||
)
|
||||
return SyncReportResponse.from_sync_report(report)
|
||||
|
||||
|
||||
@project_router.post("/status", response_model=SyncReportResponse)
|
||||
async def project_sync_status(
|
||||
sync_service: SyncServiceDep,
|
||||
project_config: ProjectConfigDep,
|
||||
) -> SyncReportResponse:
|
||||
"""Scan directory for changes compared to database state.
|
||||
|
||||
Args:
|
||||
sync_service: Sync service for this project
|
||||
project_config: Project configuration
|
||||
|
||||
Returns:
|
||||
Scan report with details on files that need syncing
|
||||
"""
|
||||
logger.info(f"Scanning filesystem for project: {project_config.name}") # pragma: no cover
|
||||
sync_report = await sync_service.scan(project_config.home) # pragma: no cover
|
||||
|
||||
return SyncReportResponse.from_sync_report(sync_report) # pragma: no cover
|
||||
|
||||
|
||||
# List all available projects
|
||||
@project_resource_router.get("/projects", response_model=ProjectList)
|
||||
async def list_projects(
|
||||
project_service: ProjectServiceDep,
|
||||
) -> ProjectList:
|
||||
"""List all configured projects.
|
||||
|
||||
Returns:
|
||||
A list of all projects with metadata
|
||||
"""
|
||||
projects = await project_service.list_projects()
|
||||
default_project = project_service.default_project
|
||||
|
||||
project_items = [
|
||||
ProjectItem(
|
||||
id=project.id,
|
||||
external_id=project.external_id,
|
||||
name=project.name,
|
||||
path=normalize_project_path(project.path),
|
||||
is_default=project.is_default or False,
|
||||
)
|
||||
for project in projects
|
||||
]
|
||||
|
||||
return ProjectList(
|
||||
projects=project_items,
|
||||
default_project=default_project,
|
||||
)
|
||||
|
||||
|
||||
# Add a new project
|
||||
@project_resource_router.post("/projects", response_model=ProjectStatusResponse, status_code=201)
|
||||
async def add_project(
|
||||
response: Response,
|
||||
project_data: ProjectInfoRequest,
|
||||
project_service: ProjectServiceDep,
|
||||
) -> ProjectStatusResponse:
|
||||
"""Add a new project to configuration and database.
|
||||
|
||||
Args:
|
||||
project_data: The project name and path, with option to set as default
|
||||
|
||||
Returns:
|
||||
Response confirming the project was added
|
||||
"""
|
||||
# Check if project already exists before attempting to add
|
||||
existing_project = await project_service.get_project(project_data.name)
|
||||
if existing_project:
|
||||
# Project exists - check if paths match for true idempotency
|
||||
# Normalize paths for comparison (resolve symlinks, etc.)
|
||||
from pathlib import Path
|
||||
|
||||
requested_path = Path(project_data.path).resolve()
|
||||
existing_path = Path(existing_project.path).resolve()
|
||||
|
||||
if requested_path == existing_path:
|
||||
# Same name, same path - return 200 OK (idempotent)
|
||||
response.status_code = 200
|
||||
return ProjectStatusResponse( # pyright: ignore [reportCallIssue]
|
||||
message=f"Project '{project_data.name}' already exists",
|
||||
status="success",
|
||||
default=existing_project.is_default or False,
|
||||
new_project=ProjectItem(
|
||||
id=existing_project.id,
|
||||
external_id=existing_project.external_id,
|
||||
name=existing_project.name,
|
||||
path=existing_project.path,
|
||||
is_default=existing_project.is_default or False,
|
||||
),
|
||||
)
|
||||
else:
|
||||
# Same name, different path - this is an error
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
detail=f"Project '{project_data.name}' already exists with different path. Existing: {existing_project.path}, Requested: {project_data.path}",
|
||||
)
|
||||
|
||||
try: # pragma: no cover
|
||||
# The service layer now handles cloud mode validation and path sanitization
|
||||
await project_service.add_project(
|
||||
project_data.name, project_data.path, set_default=project_data.set_default
|
||||
)
|
||||
|
||||
# Fetch the newly created project to get its ID
|
||||
new_project = await project_service.get_project(project_data.name)
|
||||
if not new_project:
|
||||
raise HTTPException(status_code=500, detail="Failed to retrieve newly created project")
|
||||
|
||||
return ProjectStatusResponse( # pyright: ignore [reportCallIssue]
|
||||
message=f"Project '{new_project.name}' added successfully",
|
||||
status="success",
|
||||
default=project_data.set_default,
|
||||
new_project=ProjectItem(
|
||||
id=new_project.id,
|
||||
external_id=new_project.external_id,
|
||||
name=new_project.name,
|
||||
path=new_project.path,
|
||||
is_default=new_project.is_default or False,
|
||||
),
|
||||
)
|
||||
except ValueError as e: # pragma: no cover
|
||||
raise HTTPException(status_code=400, detail=str(e))
|
||||
|
||||
|
||||
# Remove a project
|
||||
@project_resource_router.delete("/{name}", response_model=ProjectStatusResponse)
|
||||
async def remove_project(
|
||||
project_service: ProjectServiceDep,
|
||||
name: str = Path(..., description="Name of the project to remove"),
|
||||
delete_notes: bool = Query(
|
||||
False, description="If True, delete project directory from filesystem"
|
||||
),
|
||||
) -> ProjectStatusResponse:
|
||||
"""Remove a project from configuration and database.
|
||||
|
||||
Args:
|
||||
name: The name of the project to remove
|
||||
delete_notes: If True, delete the project directory from the filesystem
|
||||
|
||||
Returns:
|
||||
Response confirming the project was removed
|
||||
"""
|
||||
try:
|
||||
old_project = await project_service.get_project(name)
|
||||
if not old_project: # pragma: no cover
|
||||
raise HTTPException(
|
||||
status_code=404, detail=f"Project: '{name}' does not exist"
|
||||
) # pragma: no cover
|
||||
|
||||
# Check if trying to delete the default project
|
||||
# In cloud mode, database is source of truth; in local mode, check config
|
||||
config_default = project_service.default_project
|
||||
db_default = await project_service.repository.get_default_project()
|
||||
|
||||
# Use database default if available, otherwise fall back to config default
|
||||
default_project_name = db_default.name if db_default else config_default
|
||||
|
||||
if name == default_project_name:
|
||||
available_projects = await project_service.list_projects()
|
||||
other_projects = [p.name for p in available_projects if p.name != name]
|
||||
detail = f"Cannot delete default project '{name}'. "
|
||||
if other_projects:
|
||||
detail += (
|
||||
f"Set another project as default first. Available: {', '.join(other_projects)}"
|
||||
)
|
||||
else:
|
||||
detail += "This is the only project in your configuration."
|
||||
raise HTTPException(status_code=400, detail=detail)
|
||||
|
||||
await project_service.remove_project(name, delete_notes=delete_notes)
|
||||
|
||||
return ProjectStatusResponse(
|
||||
message=f"Project '{old_project.name}' removed successfully",
|
||||
status="success",
|
||||
default=False,
|
||||
old_project=ProjectItem(
|
||||
id=old_project.id,
|
||||
external_id=old_project.external_id,
|
||||
name=old_project.name,
|
||||
path=old_project.path,
|
||||
is_default=old_project.is_default or False,
|
||||
),
|
||||
new_project=None,
|
||||
)
|
||||
except ValueError as e: # pragma: no cover
|
||||
raise HTTPException(status_code=400, detail=str(e))
|
||||
|
||||
|
||||
# Set a project as default
|
||||
@project_resource_router.put("/{name}/default", response_model=ProjectStatusResponse)
|
||||
async def set_default_project(
|
||||
project_service: ProjectServiceDep,
|
||||
name: str = Path(..., description="Name of the project to set as default"),
|
||||
) -> ProjectStatusResponse:
|
||||
"""Set a project as the default project.
|
||||
|
||||
Args:
|
||||
name: The name of the project to set as default
|
||||
|
||||
Returns:
|
||||
Response confirming the project was set as default
|
||||
"""
|
||||
try:
|
||||
# Get the old default project
|
||||
default_name = project_service.default_project
|
||||
default_project = await project_service.get_project(default_name)
|
||||
if not default_project: # pragma: no cover
|
||||
raise HTTPException( # pragma: no cover
|
||||
status_code=404, detail=f"Default Project: '{default_name}' does not exist"
|
||||
)
|
||||
|
||||
# get the new project
|
||||
new_default_project = await project_service.get_project(name)
|
||||
if not new_default_project: # pragma: no cover
|
||||
raise HTTPException(
|
||||
status_code=404, detail=f"Project: '{name}' does not exist"
|
||||
) # pragma: no cover
|
||||
|
||||
await project_service.set_default_project(name)
|
||||
|
||||
return ProjectStatusResponse(
|
||||
message=f"Project '{name}' set as default successfully",
|
||||
status="success",
|
||||
default=True,
|
||||
old_project=ProjectItem(
|
||||
id=default_project.id,
|
||||
external_id=default_project.external_id,
|
||||
name=default_name,
|
||||
path=default_project.path,
|
||||
is_default=False,
|
||||
),
|
||||
new_project=ProjectItem(
|
||||
id=new_default_project.id,
|
||||
external_id=new_default_project.external_id,
|
||||
name=name,
|
||||
path=new_default_project.path,
|
||||
is_default=True,
|
||||
),
|
||||
)
|
||||
except ValueError as e: # pragma: no cover
|
||||
raise HTTPException(status_code=400, detail=str(e))
|
||||
|
||||
|
||||
# Get the default project
|
||||
@project_resource_router.get("/default", response_model=ProjectItem)
|
||||
async def get_default_project(
|
||||
project_service: ProjectServiceDep,
|
||||
) -> ProjectItem:
|
||||
"""Get the default project.
|
||||
|
||||
Returns:
|
||||
Response with project default information
|
||||
"""
|
||||
# Get the default project
|
||||
# In cloud mode, database is source of truth; in local mode, check config
|
||||
config_default = project_service.default_project
|
||||
db_default = await project_service.repository.get_default_project()
|
||||
|
||||
# Use database default if available, otherwise fall back to config default
|
||||
default_name = db_default.name if db_default else config_default
|
||||
default_project = await project_service.get_project(default_name)
|
||||
if not default_project: # pragma: no cover
|
||||
raise HTTPException( # pragma: no cover
|
||||
status_code=404, detail=f"Default Project: '{default_name}' does not exist"
|
||||
)
|
||||
|
||||
return ProjectItem(
|
||||
id=default_project.id,
|
||||
external_id=default_project.external_id,
|
||||
name=default_project.name,
|
||||
path=default_project.path,
|
||||
is_default=True,
|
||||
)
|
||||
|
||||
|
||||
# Synchronize projects between config and database
|
||||
@project_resource_router.post("/config/sync", response_model=ProjectStatusResponse)
|
||||
async def synchronize_projects(
|
||||
project_service: ProjectServiceDep,
|
||||
) -> ProjectStatusResponse:
|
||||
"""Synchronize projects between configuration file and database.
|
||||
|
||||
Ensures that all projects in the configuration file exist in the database
|
||||
and vice versa.
|
||||
|
||||
Returns:
|
||||
Response confirming synchronization was completed
|
||||
"""
|
||||
try: # pragma: no cover
|
||||
await project_service.synchronize_projects()
|
||||
|
||||
return ProjectStatusResponse( # pyright: ignore [reportCallIssue]
|
||||
message="Projects synchronized successfully between configuration and database",
|
||||
status="success",
|
||||
default=False,
|
||||
)
|
||||
except ValueError as e: # pragma: no cover
|
||||
raise HTTPException(status_code=400, detail=str(e))
|
||||
@@ -1,260 +0,0 @@
|
||||
"""Router for prompt-related operations.
|
||||
|
||||
This router is responsible for rendering various prompts using Handlebars templates.
|
||||
It centralizes all prompt formatting logic that was previously in the MCP prompts.
|
||||
"""
|
||||
|
||||
from datetime import datetime, timezone
|
||||
from fastapi import APIRouter, HTTPException, status
|
||||
from loguru import logger
|
||||
|
||||
from basic_memory.api.routers.utils import to_graph_context, to_search_results
|
||||
from basic_memory.api.template_loader import template_loader
|
||||
from basic_memory.schemas.base import parse_timeframe
|
||||
from basic_memory.deps import (
|
||||
ContextServiceDep,
|
||||
EntityRepositoryDep,
|
||||
SearchServiceDep,
|
||||
EntityServiceDep,
|
||||
)
|
||||
from basic_memory.schemas.prompt import (
|
||||
ContinueConversationRequest,
|
||||
SearchPromptRequest,
|
||||
PromptResponse,
|
||||
PromptMetadata,
|
||||
)
|
||||
from basic_memory.schemas.search import SearchItemType, SearchQuery
|
||||
|
||||
router = APIRouter(prefix="/prompt", tags=["prompt"])
|
||||
|
||||
|
||||
@router.post("/continue-conversation", response_model=PromptResponse)
|
||||
async def continue_conversation(
|
||||
search_service: SearchServiceDep,
|
||||
entity_service: EntityServiceDep,
|
||||
context_service: ContextServiceDep,
|
||||
entity_repository: EntityRepositoryDep,
|
||||
request: ContinueConversationRequest,
|
||||
) -> PromptResponse:
|
||||
"""Generate a prompt for continuing a conversation.
|
||||
|
||||
This endpoint takes a topic and/or timeframe and generates a prompt with
|
||||
relevant context from the knowledge base.
|
||||
|
||||
Args:
|
||||
request: The request parameters
|
||||
|
||||
Returns:
|
||||
Formatted continuation prompt with context
|
||||
"""
|
||||
logger.info(
|
||||
f"Generating continue conversation prompt, topic: {request.topic}, timeframe: {request.timeframe}"
|
||||
)
|
||||
|
||||
since = parse_timeframe(request.timeframe) if request.timeframe else None
|
||||
|
||||
# Initialize search results
|
||||
search_results = []
|
||||
|
||||
# Get data needed for template
|
||||
if request.topic:
|
||||
query = SearchQuery(text=request.topic, after_date=request.timeframe)
|
||||
results = await search_service.search(query, limit=request.search_items_limit)
|
||||
search_results = await to_search_results(entity_service, results)
|
||||
|
||||
# Build context from results
|
||||
all_hierarchical_results = []
|
||||
for result in search_results:
|
||||
if hasattr(result, "permalink") and result.permalink:
|
||||
# Get hierarchical context using the new dataclass-based approach
|
||||
context_result = await context_service.build_context(
|
||||
result.permalink,
|
||||
depth=request.depth,
|
||||
since=since,
|
||||
max_related=request.related_items_limit,
|
||||
include_observations=True, # Include observations for entities
|
||||
)
|
||||
|
||||
# Process results into the schema format
|
||||
graph_context = await to_graph_context(
|
||||
context_result, entity_repository=entity_repository
|
||||
)
|
||||
|
||||
# Add results to our collection (limit to top results for each permalink)
|
||||
if graph_context.results:
|
||||
all_hierarchical_results.extend(graph_context.results[:3])
|
||||
|
||||
# Limit to a reasonable number of total results
|
||||
all_hierarchical_results = all_hierarchical_results[:10]
|
||||
|
||||
template_context = {
|
||||
"topic": request.topic,
|
||||
"timeframe": request.timeframe,
|
||||
"hierarchical_results": all_hierarchical_results,
|
||||
"has_results": len(all_hierarchical_results) > 0,
|
||||
}
|
||||
else:
|
||||
# If no topic, get recent activity
|
||||
context_result = await context_service.build_context(
|
||||
types=[SearchItemType.ENTITY],
|
||||
depth=request.depth,
|
||||
since=since,
|
||||
max_related=request.related_items_limit,
|
||||
include_observations=True,
|
||||
)
|
||||
recent_context = await to_graph_context(context_result, entity_repository=entity_repository)
|
||||
|
||||
hierarchical_results = recent_context.results[:5] # Limit to top 5 recent items
|
||||
|
||||
template_context = {
|
||||
"topic": f"Recent Activity from ({request.timeframe})",
|
||||
"timeframe": request.timeframe,
|
||||
"hierarchical_results": hierarchical_results,
|
||||
"has_results": len(hierarchical_results) > 0,
|
||||
}
|
||||
|
||||
try:
|
||||
# Render template
|
||||
rendered_prompt = await template_loader.render(
|
||||
"prompts/continue_conversation.hbs", template_context
|
||||
)
|
||||
|
||||
# Calculate metadata
|
||||
# Count items of different types
|
||||
observation_count = 0
|
||||
relation_count = 0
|
||||
entity_count = 0
|
||||
|
||||
# Get the hierarchical results from the template context
|
||||
hierarchical_results_for_count = template_context.get("hierarchical_results", [])
|
||||
|
||||
# For topic-based search
|
||||
if request.topic:
|
||||
for item in hierarchical_results_for_count:
|
||||
if hasattr(item, "observations"):
|
||||
observation_count += len(item.observations) if item.observations else 0
|
||||
|
||||
if hasattr(item, "related_results"):
|
||||
for related in item.related_results or []:
|
||||
if hasattr(related, "type"):
|
||||
if related.type == "relation":
|
||||
relation_count += 1
|
||||
elif related.type == "entity": # pragma: no cover
|
||||
entity_count += 1 # pragma: no cover
|
||||
# For recent activity
|
||||
else:
|
||||
for item in hierarchical_results_for_count:
|
||||
if hasattr(item, "observations"):
|
||||
observation_count += len(item.observations) if item.observations else 0
|
||||
|
||||
if hasattr(item, "related_results"):
|
||||
for related in item.related_results or []:
|
||||
if hasattr(related, "type"):
|
||||
if related.type == "relation":
|
||||
relation_count += 1
|
||||
elif related.type == "entity": # pragma: no cover
|
||||
entity_count += 1 # pragma: no cover
|
||||
|
||||
# Build metadata
|
||||
metadata = {
|
||||
"query": request.topic,
|
||||
"timeframe": request.timeframe,
|
||||
"search_count": len(search_results)
|
||||
if request.topic
|
||||
else 0, # Original search results count
|
||||
"context_count": len(hierarchical_results_for_count),
|
||||
"observation_count": observation_count,
|
||||
"relation_count": relation_count,
|
||||
"total_items": (
|
||||
len(hierarchical_results_for_count)
|
||||
+ observation_count
|
||||
+ relation_count
|
||||
+ entity_count
|
||||
),
|
||||
"search_limit": request.search_items_limit,
|
||||
"context_depth": request.depth,
|
||||
"related_limit": request.related_items_limit,
|
||||
"generated_at": datetime.now(timezone.utc).isoformat(),
|
||||
}
|
||||
|
||||
prompt_metadata = PromptMetadata(**metadata)
|
||||
|
||||
return PromptResponse(
|
||||
prompt=rendered_prompt, context=template_context, metadata=prompt_metadata
|
||||
)
|
||||
except Exception as e:
|
||||
logger.error(f"Error rendering continue conversation template: {e}")
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
|
||||
detail=f"Error rendering prompt template: {str(e)}",
|
||||
)
|
||||
|
||||
|
||||
@router.post("/search", response_model=PromptResponse)
|
||||
async def search_prompt(
|
||||
search_service: SearchServiceDep,
|
||||
entity_service: EntityServiceDep,
|
||||
request: SearchPromptRequest,
|
||||
page: int = 1,
|
||||
page_size: int = 10,
|
||||
) -> PromptResponse:
|
||||
"""Generate a prompt for search results.
|
||||
|
||||
This endpoint takes a search query and formats the results into a helpful
|
||||
prompt with context and suggestions.
|
||||
|
||||
Args:
|
||||
request: The search parameters
|
||||
page: The page number for pagination
|
||||
page_size: The number of results per page, defaults to 10
|
||||
|
||||
Returns:
|
||||
Formatted search results prompt with context
|
||||
"""
|
||||
logger.info(f"Generating search prompt, query: {request.query}, timeframe: {request.timeframe}")
|
||||
|
||||
limit = page_size
|
||||
offset = (page - 1) * page_size
|
||||
|
||||
query = SearchQuery(text=request.query, after_date=request.timeframe)
|
||||
results = await search_service.search(query, limit=limit, offset=offset)
|
||||
search_results = await to_search_results(entity_service, results)
|
||||
|
||||
template_context = {
|
||||
"query": request.query,
|
||||
"timeframe": request.timeframe,
|
||||
"results": search_results,
|
||||
"has_results": len(search_results) > 0,
|
||||
"result_count": len(search_results),
|
||||
}
|
||||
|
||||
try:
|
||||
# Render template
|
||||
rendered_prompt = await template_loader.render("prompts/search.hbs", template_context)
|
||||
|
||||
# Build metadata
|
||||
metadata = {
|
||||
"query": request.query,
|
||||
"timeframe": request.timeframe,
|
||||
"search_count": len(search_results),
|
||||
"context_count": len(search_results),
|
||||
"observation_count": 0, # Search results don't include observations
|
||||
"relation_count": 0, # Search results don't include relations
|
||||
"total_items": len(search_results),
|
||||
"search_limit": limit,
|
||||
"context_depth": 0, # No context depth for basic search
|
||||
"related_limit": 0, # No related items for basic search
|
||||
"generated_at": datetime.now(timezone.utc).isoformat(),
|
||||
}
|
||||
|
||||
prompt_metadata = PromptMetadata(**metadata)
|
||||
|
||||
return PromptResponse(
|
||||
prompt=rendered_prompt, context=template_context, metadata=prompt_metadata
|
||||
)
|
||||
except Exception as e:
|
||||
logger.error(f"Error rendering search template: {e}")
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
|
||||
detail=f"Error rendering prompt template: {str(e)}",
|
||||
)
|
||||
@@ -1,252 +0,0 @@
|
||||
"""Routes for getting entity content."""
|
||||
|
||||
import tempfile
|
||||
import uuid
|
||||
from pathlib import Path
|
||||
from typing import Annotated, Union
|
||||
|
||||
from fastapi import APIRouter, HTTPException, BackgroundTasks, Body, Response
|
||||
from fastapi.responses import FileResponse, JSONResponse
|
||||
from loguru import logger
|
||||
|
||||
from basic_memory.deps import (
|
||||
ProjectConfigDep,
|
||||
LinkResolverDep,
|
||||
SearchServiceDep,
|
||||
EntityServiceDep,
|
||||
FileServiceDep,
|
||||
EntityRepositoryDep,
|
||||
)
|
||||
from basic_memory.repository.search_repository import SearchIndexRow
|
||||
from basic_memory.schemas.memory import normalize_memory_url
|
||||
from basic_memory.schemas.search import SearchQuery, SearchItemType
|
||||
from basic_memory.models.knowledge import Entity as EntityModel
|
||||
from datetime import datetime
|
||||
|
||||
router = APIRouter(prefix="/resource", tags=["resources"])
|
||||
|
||||
|
||||
def _mtime_to_datetime(entity: EntityModel) -> datetime:
|
||||
"""Convert entity mtime (file modification time) to datetime.
|
||||
|
||||
Returns the file's actual modification time, falling back to updated_at
|
||||
if mtime is not available.
|
||||
"""
|
||||
if entity.mtime: # pragma: no cover
|
||||
return datetime.fromtimestamp(entity.mtime).astimezone() # pragma: no cover
|
||||
return entity.updated_at
|
||||
|
||||
|
||||
def get_entity_ids(item: SearchIndexRow) -> set[int]:
|
||||
match item.type:
|
||||
case SearchItemType.ENTITY:
|
||||
return {item.id}
|
||||
case SearchItemType.OBSERVATION:
|
||||
return {item.entity_id} # pyright: ignore [reportReturnType]
|
||||
case SearchItemType.RELATION:
|
||||
from_entity = item.from_id
|
||||
to_entity = item.to_id # pyright: ignore [reportReturnType]
|
||||
return {from_entity, to_entity} if to_entity else {from_entity} # pyright: ignore [reportReturnType]
|
||||
case _: # pragma: no cover
|
||||
raise ValueError(f"Unexpected type: {item.type}")
|
||||
|
||||
|
||||
@router.get("/{identifier:path}", response_model=None)
|
||||
async def get_resource_content(
|
||||
config: ProjectConfigDep,
|
||||
link_resolver: LinkResolverDep,
|
||||
search_service: SearchServiceDep,
|
||||
entity_service: EntityServiceDep,
|
||||
file_service: FileServiceDep,
|
||||
background_tasks: BackgroundTasks,
|
||||
identifier: str,
|
||||
page: int = 1,
|
||||
page_size: int = 10,
|
||||
) -> Union[Response, FileResponse]:
|
||||
"""Get resource content by identifier: name or permalink."""
|
||||
logger.debug(f"Getting content for: {identifier}")
|
||||
|
||||
# Find single entity by permalink
|
||||
entity = await link_resolver.resolve_link(identifier)
|
||||
results = [entity] if entity else []
|
||||
|
||||
# pagination for multiple results
|
||||
limit = page_size
|
||||
offset = (page - 1) * page_size
|
||||
|
||||
# search using the identifier as a permalink
|
||||
if not results:
|
||||
# if the identifier contains a wildcard, use GLOB search
|
||||
query = (
|
||||
SearchQuery(permalink_match=identifier)
|
||||
if "*" in identifier
|
||||
else SearchQuery(permalink=identifier)
|
||||
)
|
||||
search_results = await search_service.search(query, limit, offset)
|
||||
if not search_results:
|
||||
raise HTTPException(status_code=404, detail=f"Resource not found: {identifier}")
|
||||
|
||||
# get the deduplicated entities related to the search results
|
||||
entity_ids = {id for result in search_results for id in get_entity_ids(result)}
|
||||
results = await entity_service.get_entities_by_id(list(entity_ids))
|
||||
|
||||
# return single response
|
||||
if len(results) == 1:
|
||||
entity = results[0]
|
||||
# Check file exists via file_service (for cloud compatibility)
|
||||
if not await file_service.exists(entity.file_path):
|
||||
raise HTTPException(
|
||||
status_code=404,
|
||||
detail=f"File not found: {entity.file_path}",
|
||||
)
|
||||
# Read content via file_service as bytes (works with both local and S3)
|
||||
content = await file_service.read_file_bytes(entity.file_path)
|
||||
content_type = file_service.content_type(entity.file_path)
|
||||
return Response(content=content, media_type=content_type)
|
||||
|
||||
# for multiple files, initialize a temporary file for writing the results
|
||||
with tempfile.NamedTemporaryFile(delete=False, mode="w", suffix=".md") as tmp_file:
|
||||
temp_file_path = tmp_file.name
|
||||
|
||||
for result in results:
|
||||
# Read content for each entity
|
||||
content = await file_service.read_entity_content(result)
|
||||
memory_url = normalize_memory_url(result.permalink)
|
||||
modified_date = _mtime_to_datetime(result).isoformat()
|
||||
checksum = result.checksum[:8] if result.checksum else ""
|
||||
|
||||
# Prepare the delimited content
|
||||
response_content = f"--- {memory_url} {modified_date} {checksum}\n"
|
||||
response_content += f"\n{content}\n"
|
||||
response_content += "\n"
|
||||
|
||||
# Write content directly to the temporary file in append mode
|
||||
tmp_file.write(response_content)
|
||||
|
||||
# Ensure all content is written to disk
|
||||
tmp_file.flush()
|
||||
|
||||
# Schedule the temporary file to be deleted after the response
|
||||
background_tasks.add_task(cleanup_temp_file, temp_file_path)
|
||||
|
||||
# Return the file response
|
||||
return FileResponse(path=temp_file_path)
|
||||
|
||||
|
||||
def cleanup_temp_file(file_path: str):
|
||||
"""Delete the temporary file."""
|
||||
try:
|
||||
Path(file_path).unlink() # Deletes the file
|
||||
logger.debug(f"Temporary file deleted: {file_path}")
|
||||
except Exception as e: # pragma: no cover
|
||||
logger.error(f"Error deleting temporary file {file_path}: {e}")
|
||||
|
||||
|
||||
@router.put("/{file_path:path}")
|
||||
async def write_resource(
|
||||
config: ProjectConfigDep,
|
||||
file_service: FileServiceDep,
|
||||
entity_repository: EntityRepositoryDep,
|
||||
search_service: SearchServiceDep,
|
||||
file_path: str,
|
||||
content: Annotated[str, Body()],
|
||||
) -> JSONResponse:
|
||||
"""Write content to a file in the project.
|
||||
|
||||
This endpoint allows writing content directly to a file in the project.
|
||||
Also creates an entity record and indexes the file for search.
|
||||
|
||||
Args:
|
||||
file_path: Path to write to, relative to project root
|
||||
request: Contains the content to write
|
||||
|
||||
Returns:
|
||||
JSON response with file information
|
||||
"""
|
||||
try:
|
||||
# Get content from request body
|
||||
|
||||
# Defensive type checking: ensure content is a string
|
||||
# FastAPI should validate this, but if a dict somehow gets through
|
||||
# (e.g., via JSON body parsing), we need to catch it here
|
||||
if isinstance(content, dict):
|
||||
logger.error( # pragma: no cover
|
||||
f"Error writing resource {file_path}: "
|
||||
f"content is a dict, expected string. Keys: {list(content.keys())}"
|
||||
)
|
||||
raise HTTPException( # pragma: no cover
|
||||
status_code=400,
|
||||
detail="content must be a string, not a dict. "
|
||||
"Ensure request body is sent as raw string content, not JSON object.",
|
||||
)
|
||||
|
||||
# Ensure it's UTF-8 string content
|
||||
if isinstance(content, bytes): # pragma: no cover
|
||||
content_str = content.decode("utf-8")
|
||||
else:
|
||||
content_str = str(content)
|
||||
|
||||
# Cloud compatibility: do not assume a local filesystem path structure.
|
||||
# Delegate directory creation + writes to the configured FileService (local or S3).
|
||||
await file_service.ensure_directory(Path(file_path).parent)
|
||||
checksum = await file_service.write_file(file_path, content_str)
|
||||
|
||||
# Get file info
|
||||
file_metadata = await file_service.get_file_metadata(file_path)
|
||||
|
||||
# Determine file details
|
||||
file_name = Path(file_path).name
|
||||
content_type = file_service.content_type(file_path)
|
||||
|
||||
entity_type = "canvas" if file_path.endswith(".canvas") else "file"
|
||||
|
||||
# Check if entity already exists
|
||||
existing_entity = await entity_repository.get_by_file_path(file_path)
|
||||
|
||||
if existing_entity:
|
||||
# Update existing entity
|
||||
entity = await entity_repository.update(
|
||||
existing_entity.id,
|
||||
{
|
||||
"title": file_name,
|
||||
"entity_type": entity_type,
|
||||
"content_type": content_type,
|
||||
"file_path": file_path,
|
||||
"checksum": checksum,
|
||||
"updated_at": file_metadata.modified_at,
|
||||
},
|
||||
)
|
||||
status_code = 200
|
||||
else:
|
||||
# Create a new entity model
|
||||
# Explicitly set external_id to ensure NOT NULL constraint is satisfied (fixes #512)
|
||||
entity = EntityModel(
|
||||
external_id=str(uuid.uuid4()),
|
||||
title=file_name,
|
||||
entity_type=entity_type,
|
||||
content_type=content_type,
|
||||
file_path=file_path,
|
||||
checksum=checksum,
|
||||
created_at=file_metadata.created_at,
|
||||
updated_at=file_metadata.modified_at,
|
||||
)
|
||||
entity = await entity_repository.add(entity)
|
||||
status_code = 201
|
||||
|
||||
# Index the file for search
|
||||
await search_service.index_entity(entity) # pyright: ignore
|
||||
|
||||
# Return success response
|
||||
return JSONResponse(
|
||||
status_code=status_code,
|
||||
content={
|
||||
"file_path": file_path,
|
||||
"checksum": checksum,
|
||||
"size": file_metadata.size,
|
||||
"created_at": file_metadata.created_at.timestamp(),
|
||||
"modified_at": file_metadata.modified_at.timestamp(),
|
||||
},
|
||||
)
|
||||
except Exception as e: # pragma: no cover
|
||||
logger.error(f"Error writing resource {file_path}: {e}")
|
||||
raise HTTPException(status_code=500, detail=f"Failed to write resource: {str(e)}")
|
||||
@@ -1,36 +0,0 @@
|
||||
"""Router for search operations."""
|
||||
|
||||
from fastapi import APIRouter, BackgroundTasks
|
||||
|
||||
from basic_memory.api.routers.utils import to_search_results
|
||||
from basic_memory.schemas.search import SearchQuery, SearchResponse
|
||||
from basic_memory.deps import SearchServiceDep, EntityServiceDep
|
||||
|
||||
router = APIRouter(prefix="/search", tags=["search"])
|
||||
|
||||
|
||||
@router.post("/", response_model=SearchResponse)
|
||||
async def search(
|
||||
query: SearchQuery,
|
||||
search_service: SearchServiceDep,
|
||||
entity_service: EntityServiceDep,
|
||||
page: int = 1,
|
||||
page_size: int = 10,
|
||||
):
|
||||
"""Search across all knowledge and documents."""
|
||||
limit = page_size
|
||||
offset = (page - 1) * page_size
|
||||
results = await search_service.search(query, limit=limit, offset=offset)
|
||||
search_results = await to_search_results(entity_service, results)
|
||||
return SearchResponse(
|
||||
results=search_results,
|
||||
current_page=page,
|
||||
page_size=page_size,
|
||||
)
|
||||
|
||||
|
||||
@router.post("/reindex")
|
||||
async def reindex(background_tasks: BackgroundTasks, search_service: SearchServiceDep):
|
||||
"""Recreate and populate the search index."""
|
||||
await search_service.reindex_all(background_tasks=background_tasks)
|
||||
return {"status": "ok", "message": "Reindex initiated"}
|
||||
@@ -32,14 +32,14 @@ async def import_chatgpt(
|
||||
importer: ChatGPTImporterV2ExternalDep,
|
||||
file: UploadFile,
|
||||
project_id: str = Path(..., description="Project external UUID"),
|
||||
folder: str = Form("conversations"),
|
||||
directory: str = Form("conversations"),
|
||||
) -> ChatImportResult:
|
||||
"""Import conversations from ChatGPT JSON export.
|
||||
|
||||
Args:
|
||||
project_id: Project external UUID from URL path
|
||||
file: The ChatGPT conversations.json file.
|
||||
folder: The folder to place the files in.
|
||||
directory: The directory to place the files in.
|
||||
importer: ChatGPT importer instance.
|
||||
|
||||
Returns:
|
||||
@@ -49,7 +49,7 @@ async def import_chatgpt(
|
||||
HTTPException: If import fails.
|
||||
"""
|
||||
logger.info(f"V2 Importing ChatGPT conversations for project {project_id}")
|
||||
return await import_file(importer, file, folder)
|
||||
return await import_file(importer, file, directory)
|
||||
|
||||
|
||||
@router.post("/claude/conversations", response_model=ChatImportResult)
|
||||
@@ -57,14 +57,14 @@ async def import_claude_conversations(
|
||||
importer: ClaudeConversationsImporterV2ExternalDep,
|
||||
file: UploadFile,
|
||||
project_id: str = Path(..., description="Project external UUID"),
|
||||
folder: str = Form("conversations"),
|
||||
directory: str = Form("conversations"),
|
||||
) -> ChatImportResult:
|
||||
"""Import conversations from Claude conversations.json export.
|
||||
|
||||
Args:
|
||||
project_id: Project external UUID from URL path
|
||||
file: The Claude conversations.json file.
|
||||
folder: The folder to place the files in.
|
||||
directory: The directory to place the files in.
|
||||
importer: Claude conversations importer instance.
|
||||
|
||||
Returns:
|
||||
@@ -74,7 +74,7 @@ async def import_claude_conversations(
|
||||
HTTPException: If import fails.
|
||||
"""
|
||||
logger.info(f"V2 Importing Claude conversations for project {project_id}")
|
||||
return await import_file(importer, file, folder)
|
||||
return await import_file(importer, file, directory)
|
||||
|
||||
|
||||
@router.post("/claude/projects", response_model=ProjectImportResult)
|
||||
@@ -82,14 +82,14 @@ async def import_claude_projects(
|
||||
importer: ClaudeProjectsImporterV2ExternalDep,
|
||||
file: UploadFile,
|
||||
project_id: str = Path(..., description="Project external UUID"),
|
||||
folder: str = Form("projects"),
|
||||
directory: str = Form("projects"),
|
||||
) -> ProjectImportResult:
|
||||
"""Import projects from Claude projects.json export.
|
||||
|
||||
Args:
|
||||
project_id: Project external UUID from URL path
|
||||
file: The Claude projects.json file.
|
||||
folder: The base folder to place the files in.
|
||||
directory: The base directory to place the files in.
|
||||
importer: Claude projects importer instance.
|
||||
|
||||
Returns:
|
||||
@@ -99,7 +99,7 @@ async def import_claude_projects(
|
||||
HTTPException: If import fails.
|
||||
"""
|
||||
logger.info(f"V2 Importing Claude projects for project {project_id}")
|
||||
return await import_file(importer, file, folder)
|
||||
return await import_file(importer, file, directory)
|
||||
|
||||
|
||||
@router.post("/memory-json", response_model=EntityImportResult)
|
||||
@@ -107,14 +107,14 @@ async def import_memory_json(
|
||||
importer: MemoryJsonImporterV2ExternalDep,
|
||||
file: UploadFile,
|
||||
project_id: str = Path(..., description="Project external UUID"),
|
||||
folder: str = Form("conversations"),
|
||||
directory: str = Form("conversations"),
|
||||
) -> EntityImportResult:
|
||||
"""Import entities and relations from a memory.json file.
|
||||
|
||||
Args:
|
||||
project_id: Project external UUID from URL path
|
||||
file: The memory.json file.
|
||||
folder: Optional destination folder within the project.
|
||||
directory: Optional destination directory within the project.
|
||||
importer: Memory JSON importer instance.
|
||||
|
||||
Returns:
|
||||
@@ -132,7 +132,7 @@ async def import_memory_json(
|
||||
json_data = json.loads(line)
|
||||
file_data.append(json_data)
|
||||
|
||||
result = await importer.import_data(file_data, folder)
|
||||
result = await importer.import_data(file_data, directory)
|
||||
if not result.success: # pragma: no cover
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
|
||||
@@ -147,13 +147,13 @@ async def import_memory_json(
|
||||
return result
|
||||
|
||||
|
||||
async def import_file(importer: Importer, file: UploadFile, destination_folder: str):
|
||||
async def import_file(importer: Importer, file: UploadFile, destination_directory: str):
|
||||
"""Helper function to import a file using an importer instance.
|
||||
|
||||
Args:
|
||||
importer: The importer instance to use
|
||||
file: The file to import
|
||||
destination_folder: Destination folder for imported content
|
||||
destination_directory: Destination directory for imported content
|
||||
|
||||
Returns:
|
||||
Import result from the importer
|
||||
@@ -164,7 +164,7 @@ async def import_file(importer: Importer, file: UploadFile, destination_folder:
|
||||
try:
|
||||
# Process file
|
||||
json_data = json.load(file.file)
|
||||
result = await importer.import_data(json_data, destination_folder)
|
||||
result = await importer.import_data(json_data, destination_directory)
|
||||
if not result.success: # pragma: no cover
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
|
||||
|
||||
@@ -10,7 +10,7 @@ Key improvements:
|
||||
- Simplified caching strategies
|
||||
"""
|
||||
|
||||
from fastapi import APIRouter, HTTPException, BackgroundTasks, Depends, Response, Path
|
||||
from fastapi import APIRouter, HTTPException, BackgroundTasks, Depends, Response, Path, Query
|
||||
from loguru import logger
|
||||
|
||||
from basic_memory.deps import (
|
||||
@@ -19,9 +19,10 @@ from basic_memory.deps import (
|
||||
LinkResolverV2ExternalDep,
|
||||
ProjectConfigV2ExternalDep,
|
||||
AppConfigDep,
|
||||
SyncServiceV2ExternalDep,
|
||||
EntityRepositoryV2ExternalDep,
|
||||
ProjectExternalIdPathDep,
|
||||
TaskSchedulerDep,
|
||||
FileServiceV2ExternalDep,
|
||||
)
|
||||
from basic_memory.schemas import DeleteEntitiesResponse
|
||||
from basic_memory.schemas.base import Entity
|
||||
@@ -31,30 +32,13 @@ from basic_memory.schemas.v2 import (
|
||||
EntityResolveResponse,
|
||||
EntityResponseV2,
|
||||
MoveEntityRequestV2,
|
||||
MoveDirectoryRequestV2,
|
||||
DeleteDirectoryRequestV2,
|
||||
)
|
||||
from basic_memory.schemas.response import DirectoryMoveResult, DirectoryDeleteResult
|
||||
|
||||
router = APIRouter(prefix="/knowledge", tags=["knowledge-v2"])
|
||||
|
||||
|
||||
async def resolve_relations_background(sync_service, entity_id: int, entity_permalink: str) -> None:
|
||||
"""Background task to resolve relations for a specific entity.
|
||||
|
||||
This runs asynchronously after the API response is sent, preventing
|
||||
long delays when creating entities with many relations.
|
||||
"""
|
||||
try: # pragma: no cover
|
||||
# Only resolve relations for the newly created entity
|
||||
await sync_service.resolve_relations(entity_id=entity_id) # pragma: no cover
|
||||
logger.debug( # pragma: no cover
|
||||
f"Background: Resolved relations for entity {entity_permalink} (id={entity_id})"
|
||||
)
|
||||
except Exception as e: # pragma: no cover
|
||||
# Log but don't fail - this is a background task
|
||||
logger.warning( # pragma: no cover
|
||||
f"Background: Failed to resolve relations for entity {entity_permalink}: {e}"
|
||||
)
|
||||
|
||||
|
||||
## Resolution endpoint
|
||||
|
||||
|
||||
@@ -100,8 +84,12 @@ async def resolve_identifier(
|
||||
resolution_method = "external_id" if entity else "search"
|
||||
|
||||
# If not found by external_id, try other resolution methods
|
||||
# Pass source_path for context-aware resolution (prefers notes closer to source)
|
||||
# Pass strict to control fuzzy search fallback (default False allows fuzzy matching)
|
||||
if not entity:
|
||||
entity = await link_resolver.resolve_link(data.identifier)
|
||||
entity = await link_resolver.resolve_link(
|
||||
data.identifier, source_path=data.source_path, strict=data.strict
|
||||
)
|
||||
if entity:
|
||||
# Determine resolution method
|
||||
if entity.permalink == data.identifier:
|
||||
@@ -179,24 +167,43 @@ async def create_entity(
|
||||
background_tasks: BackgroundTasks,
|
||||
entity_service: EntityServiceV2ExternalDep,
|
||||
search_service: SearchServiceV2ExternalDep,
|
||||
task_scheduler: TaskSchedulerDep,
|
||||
file_service: FileServiceV2ExternalDep,
|
||||
fast: bool = Query(
|
||||
True, description="If true, write quickly and defer indexing to background tasks."
|
||||
),
|
||||
) -> EntityResponseV2:
|
||||
"""Create a new entity.
|
||||
|
||||
Args:
|
||||
data: Entity data to create
|
||||
fast: If True, defer indexing to background tasks
|
||||
|
||||
Returns:
|
||||
Created entity with generated external_id (UUID)
|
||||
Created entity with generated external_id (UUID) and file content
|
||||
"""
|
||||
logger.info(
|
||||
"API v2 request", endpoint="create_entity", entity_type=data.entity_type, title=data.title
|
||||
)
|
||||
|
||||
entity = await entity_service.create_entity(data)
|
||||
if fast:
|
||||
entity = await entity_service.fast_write_entity(data)
|
||||
task_scheduler.schedule(
|
||||
"reindex_entity",
|
||||
entity_id=entity.id,
|
||||
project_id=project_id,
|
||||
)
|
||||
else:
|
||||
entity = await entity_service.create_entity(data)
|
||||
await search_service.index_entity(entity, background_tasks=background_tasks)
|
||||
|
||||
# reindex
|
||||
await search_service.index_entity(entity, background_tasks=background_tasks)
|
||||
result = EntityResponseV2.model_validate(entity)
|
||||
if fast:
|
||||
result = result.model_copy(update={"observations": [], "relations": []})
|
||||
|
||||
# Always read and return file content
|
||||
content = await file_service.read_file_content(entity.file_path)
|
||||
result = result.model_copy(update={"content": content})
|
||||
|
||||
logger.info(
|
||||
f"API v2 response: endpoint='create_entity' external_id={entity.external_id}, title={result.title}, permalink={result.permalink}, status_code=201"
|
||||
@@ -215,9 +222,13 @@ async def update_entity_by_id(
|
||||
project_id: ProjectExternalIdPathDep,
|
||||
entity_service: EntityServiceV2ExternalDep,
|
||||
search_service: SearchServiceV2ExternalDep,
|
||||
sync_service: SyncServiceV2ExternalDep,
|
||||
entity_repository: EntityRepositoryV2ExternalDep,
|
||||
task_scheduler: TaskSchedulerDep,
|
||||
file_service: FileServiceV2ExternalDep,
|
||||
entity_id: str = Path(..., description="Entity external ID (UUID)"),
|
||||
fast: bool = Query(
|
||||
True, description="If true, write quickly and defer indexing to background tasks."
|
||||
),
|
||||
) -> EntityResponseV2:
|
||||
"""Update an entity by external ID.
|
||||
|
||||
@@ -226,30 +237,55 @@ async def update_entity_by_id(
|
||||
Args:
|
||||
entity_id: External ID (UUID string)
|
||||
data: Updated entity data
|
||||
fast: If True, defer indexing to background tasks
|
||||
|
||||
Returns:
|
||||
Updated entity
|
||||
Updated entity with file content
|
||||
"""
|
||||
logger.info(f"API v2 request: update_entity_by_id entity_id={entity_id}")
|
||||
|
||||
# Check if entity exists
|
||||
# Check if entity exists (external_id is the source of truth for v2)
|
||||
existing = await entity_repository.get_by_external_id(entity_id)
|
||||
created = existing is None
|
||||
|
||||
# Perform update or create
|
||||
entity, _ = await entity_service.create_or_update_entity(data)
|
||||
response.status_code = 201 if created else 200
|
||||
|
||||
# reindex
|
||||
await search_service.index_entity(entity, background_tasks=background_tasks)
|
||||
|
||||
# Schedule relation resolution for new entities
|
||||
if created:
|
||||
background_tasks.add_task( # pragma: no cover
|
||||
resolve_relations_background, sync_service, entity.id, entity.permalink or ""
|
||||
if fast:
|
||||
entity = await entity_service.fast_write_entity(data, external_id=entity_id)
|
||||
response.status_code = 200 if existing else 201
|
||||
task_scheduler.schedule(
|
||||
"reindex_entity",
|
||||
entity_id=entity.id,
|
||||
project_id=project_id,
|
||||
resolve_relations=created,
|
||||
)
|
||||
else:
|
||||
if existing:
|
||||
# Update the existing entity in-place to avoid path-based duplication
|
||||
entity = await entity_service.update_entity(existing, data)
|
||||
response.status_code = 200
|
||||
else:
|
||||
# Create new entity, then bind external_id to the requested UUID
|
||||
entity = await entity_service.create_entity(data)
|
||||
if entity.external_id != entity_id:
|
||||
entity = await entity_repository.update(
|
||||
entity.id,
|
||||
{"external_id": entity_id},
|
||||
)
|
||||
if not entity:
|
||||
raise HTTPException(
|
||||
status_code=404,
|
||||
detail=f"Entity with external_id '{entity_id}' not found",
|
||||
)
|
||||
response.status_code = 201
|
||||
|
||||
await search_service.index_entity(entity, background_tasks=background_tasks)
|
||||
|
||||
result = EntityResponseV2.model_validate(entity)
|
||||
if fast:
|
||||
result = result.model_copy(update={"observations": [], "relations": []})
|
||||
|
||||
# Always read and return file content
|
||||
content = await file_service.read_file_content(entity.file_path)
|
||||
result = result.model_copy(update={"content": content})
|
||||
|
||||
logger.info(
|
||||
f"API v2 response: external_id={entity_id}, created={created}, status_code={response.status_code}"
|
||||
@@ -265,16 +301,22 @@ async def edit_entity_by_id(
|
||||
entity_service: EntityServiceV2ExternalDep,
|
||||
search_service: SearchServiceV2ExternalDep,
|
||||
entity_repository: EntityRepositoryV2ExternalDep,
|
||||
task_scheduler: TaskSchedulerDep,
|
||||
file_service: FileServiceV2ExternalDep,
|
||||
entity_id: str = Path(..., description="Entity external ID (UUID)"),
|
||||
fast: bool = Query(
|
||||
True, description="If true, write quickly and defer indexing to background tasks."
|
||||
),
|
||||
) -> EntityResponseV2:
|
||||
"""Edit an existing entity by external ID using operations like append, prepend, etc.
|
||||
|
||||
Args:
|
||||
entity_id: External ID (UUID string)
|
||||
data: Edit operation details
|
||||
fast: If True, defer indexing to background tasks
|
||||
|
||||
Returns:
|
||||
Updated entity
|
||||
Updated entity with file content
|
||||
|
||||
Raises:
|
||||
HTTPException: 404 if entity not found, 400 if edit fails
|
||||
@@ -291,21 +333,41 @@ async def edit_entity_by_id(
|
||||
)
|
||||
|
||||
try:
|
||||
# Edit using the entity's permalink or path
|
||||
identifier = entity.permalink or entity.file_path
|
||||
updated_entity = await entity_service.edit_entity(
|
||||
identifier=identifier,
|
||||
operation=data.operation,
|
||||
content=data.content,
|
||||
section=data.section,
|
||||
find_text=data.find_text,
|
||||
expected_replacements=data.expected_replacements,
|
||||
)
|
||||
if fast:
|
||||
updated_entity = await entity_service.fast_edit_entity(
|
||||
entity=entity,
|
||||
operation=data.operation,
|
||||
content=data.content,
|
||||
section=data.section,
|
||||
find_text=data.find_text,
|
||||
expected_replacements=data.expected_replacements,
|
||||
)
|
||||
task_scheduler.schedule(
|
||||
"reindex_entity",
|
||||
entity_id=updated_entity.id,
|
||||
project_id=project_id,
|
||||
)
|
||||
else:
|
||||
# Edit using the entity's permalink or path
|
||||
identifier = entity.permalink or entity.file_path
|
||||
updated_entity = await entity_service.edit_entity(
|
||||
identifier=identifier,
|
||||
operation=data.operation,
|
||||
content=data.content,
|
||||
section=data.section,
|
||||
find_text=data.find_text,
|
||||
expected_replacements=data.expected_replacements,
|
||||
)
|
||||
|
||||
# Reindex
|
||||
await search_service.index_entity(updated_entity, background_tasks=background_tasks)
|
||||
await search_service.index_entity(updated_entity, background_tasks=background_tasks)
|
||||
|
||||
result = EntityResponseV2.model_validate(updated_entity)
|
||||
if fast:
|
||||
result = result.model_copy(update={"observations": [], "relations": []})
|
||||
|
||||
# Always read and return file content
|
||||
content = await file_service.read_file_content(updated_entity.file_path)
|
||||
result = result.model_copy(update={"content": content})
|
||||
|
||||
logger.info(
|
||||
f"API v2 response: external_id={entity_id}, operation='{data.operation}', status_code=200"
|
||||
@@ -423,3 +485,100 @@ async def move_entity(
|
||||
except Exception as e:
|
||||
logger.error(f"Error moving entity: {e}")
|
||||
raise HTTPException(status_code=400, detail=str(e))
|
||||
|
||||
|
||||
## Move directory endpoint
|
||||
|
||||
|
||||
@router.post("/move-directory", response_model=DirectoryMoveResult)
|
||||
async def move_directory(
|
||||
data: MoveDirectoryRequestV2,
|
||||
background_tasks: BackgroundTasks,
|
||||
project_id: ProjectExternalIdPathDep,
|
||||
entity_service: EntityServiceV2ExternalDep,
|
||||
project_config: ProjectConfigV2ExternalDep,
|
||||
app_config: AppConfigDep,
|
||||
search_service: SearchServiceV2ExternalDep,
|
||||
) -> DirectoryMoveResult:
|
||||
"""Move all entities in a directory to a new location.
|
||||
|
||||
V2 API uses project external_id in the URL path for stable references.
|
||||
Moves all files within a source directory to a destination directory,
|
||||
updating database records and optionally updating permalinks.
|
||||
|
||||
Args:
|
||||
project_id: Project external ID from URL path
|
||||
data: Move request with source and destination directories
|
||||
|
||||
Returns:
|
||||
DirectoryMoveResult with counts and details of moved files
|
||||
"""
|
||||
logger.info(
|
||||
f"API v2 request: move_directory source='{data.source_directory}', destination='{data.destination_directory}'"
|
||||
)
|
||||
|
||||
try:
|
||||
# Move the directory using the service
|
||||
result = await entity_service.move_directory(
|
||||
source_directory=data.source_directory,
|
||||
destination_directory=data.destination_directory,
|
||||
project_config=project_config,
|
||||
app_config=app_config,
|
||||
)
|
||||
|
||||
# Reindex moved entities
|
||||
for file_path in result.moved_files:
|
||||
entity = await entity_service.link_resolver.resolve_link(file_path)
|
||||
if entity:
|
||||
await search_service.index_entity(entity, background_tasks=background_tasks)
|
||||
|
||||
logger.info(
|
||||
f"API v2 response: move_directory "
|
||||
f"total={result.total_files}, success={result.successful_moves}, failed={result.failed_moves}"
|
||||
)
|
||||
return result
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Error moving directory: {e}")
|
||||
raise HTTPException(status_code=400, detail=str(e))
|
||||
|
||||
|
||||
## Delete directory endpoint
|
||||
|
||||
|
||||
@router.post("/delete-directory", response_model=DirectoryDeleteResult)
|
||||
async def delete_directory(
|
||||
data: DeleteDirectoryRequestV2,
|
||||
project_id: ProjectExternalIdPathDep,
|
||||
entity_service: EntityServiceV2ExternalDep,
|
||||
) -> DirectoryDeleteResult:
|
||||
"""Delete all entities in a directory.
|
||||
|
||||
V2 API uses project external_id in the URL path for stable references.
|
||||
Deletes all files within a directory, updating database records and
|
||||
removing files from the filesystem.
|
||||
|
||||
Args:
|
||||
project_id: Project external ID from URL path
|
||||
data: Delete request with directory path
|
||||
|
||||
Returns:
|
||||
DirectoryDeleteResult with counts and details of deleted files
|
||||
"""
|
||||
logger.info(f"API v2 request: delete_directory directory='{data.directory}'")
|
||||
|
||||
try:
|
||||
# Delete the directory using the service
|
||||
result = await entity_service.delete_directory(
|
||||
directory=data.directory,
|
||||
)
|
||||
|
||||
logger.info(
|
||||
f"API v2 response: delete_directory "
|
||||
f"total={result.total_files}, success={result.successful_deletes}, failed={result.failed_deletes}"
|
||||
)
|
||||
return result
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Error deleting directory: {e}")
|
||||
raise HTTPException(status_code=400, detail=str(e))
|
||||
|
||||
@@ -16,7 +16,7 @@ from basic_memory.schemas.memory import (
|
||||
normalize_memory_url,
|
||||
)
|
||||
from basic_memory.schemas.search import SearchItemType
|
||||
from basic_memory.api.routers.utils import to_graph_context
|
||||
from basic_memory.api.v2.utils import to_graph_context
|
||||
|
||||
# Note: No prefix here - it's added during registration as /v2/{project_id}/memory
|
||||
router = APIRouter(tags=["memory"])
|
||||
|
||||
@@ -19,9 +19,17 @@ from loguru import logger
|
||||
from basic_memory.deps import (
|
||||
ProjectServiceDep,
|
||||
ProjectRepositoryDep,
|
||||
ProjectConfigV2ExternalDep,
|
||||
SyncServiceV2ExternalDep,
|
||||
TaskSchedulerDep,
|
||||
ProjectExternalIdPathDep,
|
||||
)
|
||||
from basic_memory.schemas import SyncReportResponse
|
||||
from basic_memory.schemas.project_info import (
|
||||
ProjectItem,
|
||||
ProjectList,
|
||||
ProjectInfoRequest,
|
||||
ProjectInfoResponse,
|
||||
ProjectStatusResponse,
|
||||
)
|
||||
from basic_memory.schemas.v2 import ProjectResolveRequest, ProjectResolveResponse
|
||||
@@ -30,6 +38,175 @@ from basic_memory.utils import normalize_project_path, generate_permalink
|
||||
router = APIRouter(prefix="/projects", tags=["project_management-v2"])
|
||||
|
||||
|
||||
@router.get("/", response_model=ProjectList)
|
||||
async def list_projects(
|
||||
project_service: ProjectServiceDep,
|
||||
) -> ProjectList:
|
||||
"""List all configured projects.
|
||||
|
||||
Returns:
|
||||
A list of all projects with metadata
|
||||
"""
|
||||
projects = await project_service.list_projects()
|
||||
default_project = project_service.default_project
|
||||
|
||||
project_items = [
|
||||
ProjectItem(
|
||||
id=project.id,
|
||||
external_id=project.external_id,
|
||||
name=project.name,
|
||||
path=normalize_project_path(project.path),
|
||||
is_default=project.is_default or False,
|
||||
)
|
||||
for project in projects
|
||||
]
|
||||
|
||||
return ProjectList(
|
||||
projects=project_items,
|
||||
default_project=default_project,
|
||||
)
|
||||
|
||||
|
||||
@router.post("/", response_model=ProjectStatusResponse, status_code=201)
|
||||
async def add_project(
|
||||
project_data: ProjectInfoRequest,
|
||||
project_service: ProjectServiceDep,
|
||||
) -> ProjectStatusResponse:
|
||||
"""Add a new project to configuration and database.
|
||||
|
||||
Args:
|
||||
project_data: The project name and path, with option to set as default
|
||||
|
||||
Returns:
|
||||
Response confirming the project was added
|
||||
"""
|
||||
# Check if project already exists before attempting to add
|
||||
existing_project = await project_service.get_project(project_data.name)
|
||||
if existing_project:
|
||||
# Project exists - check if paths match for true idempotency
|
||||
# Normalize paths for comparison (resolve symlinks, etc.)
|
||||
requested_path = os.path.abspath(os.path.expanduser(project_data.path))
|
||||
existing_path = os.path.abspath(os.path.expanduser(existing_project.path))
|
||||
|
||||
if requested_path == existing_path:
|
||||
# Same name, same path - return 200 OK (idempotent)
|
||||
return ProjectStatusResponse( # pyright: ignore [reportCallIssue]
|
||||
message=f"Project '{project_data.name}' already exists",
|
||||
status="success",
|
||||
default=existing_project.is_default or False,
|
||||
new_project=ProjectItem(
|
||||
id=existing_project.id,
|
||||
external_id=existing_project.external_id,
|
||||
name=existing_project.name,
|
||||
path=existing_project.path,
|
||||
is_default=existing_project.is_default or False,
|
||||
),
|
||||
)
|
||||
else:
|
||||
# Same name, different path - this is an error
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
detail=(
|
||||
f"Project '{project_data.name}' already exists with different path. "
|
||||
f"Existing: {existing_project.path}, Requested: {project_data.path}"
|
||||
),
|
||||
)
|
||||
|
||||
try: # pragma: no cover
|
||||
# The service layer handles cloud mode validation and path sanitization
|
||||
await project_service.add_project(
|
||||
project_data.name, project_data.path, set_default=project_data.set_default
|
||||
)
|
||||
|
||||
# Fetch the newly created project to get its ID
|
||||
new_project = await project_service.get_project(project_data.name)
|
||||
if not new_project:
|
||||
raise HTTPException(status_code=500, detail="Failed to retrieve newly created project")
|
||||
|
||||
return ProjectStatusResponse( # pyright: ignore [reportCallIssue]
|
||||
message=f"Project '{new_project.name}' added successfully",
|
||||
status="success",
|
||||
default=project_data.set_default,
|
||||
new_project=ProjectItem(
|
||||
id=new_project.id,
|
||||
external_id=new_project.external_id,
|
||||
name=new_project.name,
|
||||
path=new_project.path,
|
||||
is_default=new_project.is_default or False,
|
||||
),
|
||||
)
|
||||
except ValueError as e: # pragma: no cover
|
||||
raise HTTPException(status_code=400, detail=str(e))
|
||||
|
||||
|
||||
@router.post("/config/sync", response_model=ProjectStatusResponse)
|
||||
async def synchronize_projects(
|
||||
project_service: ProjectServiceDep,
|
||||
) -> ProjectStatusResponse:
|
||||
"""Synchronize projects between configuration file and database."""
|
||||
try: # pragma: no cover
|
||||
await project_service.synchronize_projects()
|
||||
|
||||
return ProjectStatusResponse( # pyright: ignore [reportCallIssue]
|
||||
message="Projects synchronized successfully between configuration and database",
|
||||
status="success",
|
||||
default=False,
|
||||
)
|
||||
except ValueError as e: # pragma: no cover
|
||||
raise HTTPException(status_code=400, detail=str(e))
|
||||
|
||||
|
||||
@router.post("/{project_id}/sync")
|
||||
async def sync_project(
|
||||
sync_service: SyncServiceV2ExternalDep,
|
||||
project_config: ProjectConfigV2ExternalDep,
|
||||
task_scheduler: TaskSchedulerDep,
|
||||
project_internal_id: ProjectExternalIdPathDep,
|
||||
force_full: bool = Query(
|
||||
False, description="Force full scan, bypassing watermark optimization"
|
||||
),
|
||||
run_in_background: bool = Query(True, description="Run in background"),
|
||||
):
|
||||
"""Force project filesystem sync to database."""
|
||||
if run_in_background:
|
||||
task_scheduler.schedule(
|
||||
"sync_project",
|
||||
project_id=project_internal_id,
|
||||
force_full=force_full,
|
||||
)
|
||||
logger.info(
|
||||
f"Filesystem sync initiated for project: {project_config.name} (force_full={force_full})"
|
||||
)
|
||||
|
||||
return {
|
||||
"status": "sync_started",
|
||||
"message": f"Filesystem sync initiated for project '{project_config.name}'",
|
||||
}
|
||||
|
||||
report = await sync_service.sync(
|
||||
project_config.home, project_config.name, force_full=force_full
|
||||
)
|
||||
logger.info(
|
||||
f"Filesystem sync completed for project: {project_config.name} (force_full={force_full})"
|
||||
)
|
||||
return SyncReportResponse.from_sync_report(report)
|
||||
|
||||
|
||||
@router.post("/{project_id}/status", response_model=SyncReportResponse)
|
||||
async def get_project_status(
|
||||
sync_service: SyncServiceV2ExternalDep,
|
||||
project_config: ProjectConfigV2ExternalDep,
|
||||
project_id: str = Path(..., description="Project external ID (UUID)"),
|
||||
force_full: bool = Query(
|
||||
False, description="Force full scan, bypassing watermark optimization"
|
||||
),
|
||||
) -> SyncReportResponse:
|
||||
"""Get sync status of files vs database for a project."""
|
||||
logger.info(f"API v2 request: get_project_status for project_id={project_id}")
|
||||
report = await sync_service.scan(project_config.home, force_full=force_full)
|
||||
return SyncReportResponse.from_sync_report(report)
|
||||
|
||||
|
||||
@router.post("/resolve", response_model=ProjectResolveResponse)
|
||||
async def resolve_project_identifier(
|
||||
data: ProjectResolveRequest,
|
||||
@@ -147,6 +324,22 @@ async def get_project_by_id(
|
||||
)
|
||||
|
||||
|
||||
@router.get("/{project_id}/info", response_model=ProjectInfoResponse)
|
||||
async def get_project_info_by_id(
|
||||
project_service: ProjectServiceDep,
|
||||
project_repository: ProjectRepositoryDep,
|
||||
project_id: str = Path(..., description="Project external ID (UUID)"),
|
||||
) -> ProjectInfoResponse:
|
||||
"""Get detailed project information by external ID."""
|
||||
logger.info(f"API v2 request: get_project_info_by_id for project_id={project_id}")
|
||||
project = await project_repository.get_by_external_id(project_id)
|
||||
if not project:
|
||||
raise HTTPException(
|
||||
status_code=404, detail=f"Project with external_id '{project_id}' not found"
|
||||
)
|
||||
return await project_service.get_project_info(project.name)
|
||||
|
||||
|
||||
@router.patch("/{project_id}", response_model=ProjectStatusResponse)
|
||||
async def update_project_by_id(
|
||||
project_service: ProjectServiceDep,
|
||||
|
||||
@@ -9,7 +9,7 @@ from datetime import datetime, timezone
|
||||
from fastapi import APIRouter, HTTPException, status, Path
|
||||
from loguru import logger
|
||||
|
||||
from basic_memory.api.routers.utils import to_graph_context, to_search_results
|
||||
from basic_memory.api.v2.utils import to_graph_context, to_search_results
|
||||
from basic_memory.api.template_loader import template_loader
|
||||
from basic_memory.schemas.base import parse_timeframe
|
||||
from basic_memory.deps import (
|
||||
|
||||
@@ -4,11 +4,16 @@ This router uses external_id UUIDs for stable, API-friendly routing.
|
||||
V1 uses string-based project names which are less efficient and less stable.
|
||||
"""
|
||||
|
||||
from fastapi import APIRouter, BackgroundTasks, Path
|
||||
from fastapi import APIRouter, Path
|
||||
|
||||
from basic_memory.api.routers.utils import to_search_results
|
||||
from basic_memory.api.v2.utils import to_search_results
|
||||
from basic_memory.schemas.search import SearchQuery, SearchResponse
|
||||
from basic_memory.deps import SearchServiceV2ExternalDep, EntityServiceV2ExternalDep
|
||||
from basic_memory.deps import (
|
||||
SearchServiceV2ExternalDep,
|
||||
EntityServiceV2ExternalDep,
|
||||
TaskSchedulerDep,
|
||||
ProjectExternalIdPathDep,
|
||||
)
|
||||
|
||||
# Note: No prefix here - it's added during registration as /v2/{project_id}/search
|
||||
router = APIRouter(tags=["search"])
|
||||
@@ -51,9 +56,8 @@ async def search(
|
||||
|
||||
@router.post("/search/reindex")
|
||||
async def reindex(
|
||||
background_tasks: BackgroundTasks,
|
||||
search_service: SearchServiceV2ExternalDep,
|
||||
project_id: str = Path(..., description="Project external UUID"),
|
||||
task_scheduler: TaskSchedulerDep,
|
||||
project_id: ProjectExternalIdPathDep,
|
||||
):
|
||||
"""Recreate and populate the search index for a project.
|
||||
|
||||
@@ -63,11 +67,10 @@ async def reindex(
|
||||
|
||||
Args:
|
||||
project_id: Project external UUID from URL path
|
||||
background_tasks: FastAPI background tasks handler
|
||||
search_service: Search service scoped to project
|
||||
task_scheduler: Task scheduler for background work
|
||||
|
||||
Returns:
|
||||
Status message indicating reindex has been initiated
|
||||
"""
|
||||
await search_service.reindex_all(background_tasks=background_tasks)
|
||||
task_scheduler.schedule("reindex_project", project_id=project_id)
|
||||
return {"status": "ok", "message": "Reindex initiated"}
|
||||
|
||||
@@ -24,29 +24,42 @@ async def to_graph_context(
|
||||
page: Optional[int] = None,
|
||||
page_size: Optional[int] = None,
|
||||
):
|
||||
# First pass: collect all entity IDs needed for relations
|
||||
# First pass: collect all entity IDs needed for external_id lookup
|
||||
# This includes: entity primary results, observation parent entities, relation from/to entities
|
||||
entity_ids_needed: set[int] = set()
|
||||
for context_item in context_result.results:
|
||||
for item in (
|
||||
[context_item.primary_result] + context_item.observations + context_item.related_results
|
||||
):
|
||||
if item.type == SearchItemType.RELATION:
|
||||
if item.type == SearchItemType.ENTITY:
|
||||
# Entity's own ID for its external_id
|
||||
entity_ids_needed.add(item.id)
|
||||
elif item.type == SearchItemType.OBSERVATION:
|
||||
# Parent entity ID for entity_external_id
|
||||
if item.entity_id: # pyright: ignore
|
||||
entity_ids_needed.add(item.entity_id) # pyright: ignore
|
||||
elif item.type == SearchItemType.RELATION:
|
||||
# Source and target entity IDs for external_ids
|
||||
if item.from_id: # pyright: ignore
|
||||
entity_ids_needed.add(item.from_id) # pyright: ignore
|
||||
if item.to_id:
|
||||
entity_ids_needed.add(item.to_id)
|
||||
|
||||
# Batch fetch all entities at once
|
||||
entity_lookup: dict[int, str] = {}
|
||||
# Batch fetch all entities at once - get both title and external_id
|
||||
entity_title_lookup: dict[int, str] = {}
|
||||
entity_external_id_lookup: dict[int, str] = {}
|
||||
if entity_ids_needed:
|
||||
entities = await entity_repository.find_by_ids(list(entity_ids_needed))
|
||||
entity_lookup = {e.id: e.title for e in entities}
|
||||
for e in entities:
|
||||
entity_title_lookup[e.id] = e.title
|
||||
entity_external_id_lookup[e.id] = e.external_id
|
||||
|
||||
# Helper function to convert items to summaries
|
||||
def to_summary(item: SearchIndexRow | ContextResultRow):
|
||||
match item.type:
|
||||
case SearchItemType.ENTITY:
|
||||
return EntitySummary(
|
||||
external_id=entity_external_id_lookup.get(item.id, ""),
|
||||
entity_id=item.id,
|
||||
title=item.title, # pyright: ignore
|
||||
permalink=item.permalink,
|
||||
@@ -55,10 +68,14 @@ async def to_graph_context(
|
||||
created_at=item.created_at,
|
||||
)
|
||||
case SearchItemType.OBSERVATION:
|
||||
entity_ext_id = None
|
||||
if item.entity_id: # pyright: ignore
|
||||
entity_ext_id = entity_external_id_lookup.get(item.entity_id) # pyright: ignore
|
||||
return ObservationSummary(
|
||||
observation_id=item.id,
|
||||
entity_id=item.entity_id, # pyright: ignore
|
||||
title=item.title, # pyright: ignore
|
||||
entity_external_id=entity_ext_id,
|
||||
title=entity_title_lookup.get(item.entity_id), # pyright: ignore
|
||||
file_path=item.file_path,
|
||||
category=item.category, # pyright: ignore
|
||||
content=item.content, # pyright: ignore
|
||||
@@ -66,8 +83,10 @@ async def to_graph_context(
|
||||
created_at=item.created_at,
|
||||
)
|
||||
case SearchItemType.RELATION:
|
||||
from_title = entity_lookup.get(item.from_id) if item.from_id else None # pyright: ignore
|
||||
to_title = entity_lookup.get(item.to_id) if item.to_id else None
|
||||
from_title = entity_title_lookup.get(item.from_id) if item.from_id else None # pyright: ignore
|
||||
to_title = entity_title_lookup.get(item.to_id) if item.to_id else None
|
||||
from_ext_id = entity_external_id_lookup.get(item.from_id) if item.from_id else None # pyright: ignore
|
||||
to_ext_id = entity_external_id_lookup.get(item.to_id) if item.to_id else None
|
||||
return RelationSummary(
|
||||
relation_id=item.id,
|
||||
entity_id=item.entity_id, # pyright: ignore
|
||||
@@ -77,8 +96,10 @@ async def to_graph_context(
|
||||
relation_type=item.relation_type, # pyright: ignore
|
||||
from_entity=from_title,
|
||||
from_entity_id=item.from_id, # pyright: ignore
|
||||
from_entity_external_id=from_ext_id,
|
||||
to_entity=to_title,
|
||||
to_entity_id=item.to_id,
|
||||
to_entity_external_id=to_ext_id,
|
||||
created_at=item.created_at,
|
||||
)
|
||||
case _: # pragma: no cover
|
||||
@@ -50,7 +50,7 @@ def app_callback(
|
||||
# Skip for 'mcp' command - it has its own lifespan that handles initialization
|
||||
# Skip for API-using commands (status, sync, etc.) - they handle initialization via deps.py
|
||||
# Skip for 'reset' command - it manages its own database lifecycle
|
||||
skip_init_commands = {"mcp", "status", "sync", "project", "tool", "reset"}
|
||||
skip_init_commands = {"doctor", "mcp", "status", "sync", "project", "tool", "reset"}
|
||||
if (
|
||||
not version
|
||||
and ctx.invoked_subcommand is not None
|
||||
|
||||
@@ -1,11 +1,12 @@
|
||||
"""CLI commands for basic-memory."""
|
||||
|
||||
from . import status, db, import_memory_json, mcp, import_claude_conversations
|
||||
from . import status, db, doctor, import_memory_json, mcp, import_claude_conversations
|
||||
from . import import_claude_projects, import_chatgpt, tool, project, format
|
||||
|
||||
__all__ = [
|
||||
"status",
|
||||
"db",
|
||||
"doctor",
|
||||
"import_memory_json",
|
||||
"mcp",
|
||||
"import_claude_conversations",
|
||||
|
||||
@@ -30,7 +30,7 @@ async def fetch_cloud_projects(
|
||||
config = config_manager.config
|
||||
host_url = config.cloud_host.rstrip("/")
|
||||
|
||||
response = await api_request(method="GET", url=f"{host_url}/proxy/projects/projects")
|
||||
response = await api_request(method="GET", url=f"{host_url}/proxy/v2/projects/")
|
||||
|
||||
return CloudProjectList.model_validate(response.json())
|
||||
except Exception as e:
|
||||
@@ -66,7 +66,7 @@ async def create_cloud_project(
|
||||
|
||||
response = await api_request(
|
||||
method="POST",
|
||||
url=f"{host_url}/proxy/projects/projects",
|
||||
url=f"{host_url}/proxy/v2/projects/",
|
||||
headers={"Content-Type": "application/json"},
|
||||
json_data=project_data.model_dump(),
|
||||
)
|
||||
|
||||
@@ -27,6 +27,16 @@ console = Console()
|
||||
# Minimum rclone version for --create-empty-src-dirs support
|
||||
MIN_RCLONE_VERSION_EMPTY_DIRS = (1, 64, 0)
|
||||
|
||||
# Tigris edge caching returns stale data for users outside the origin region (iad).
|
||||
# --header is rclone's global flag that applies to ALL HTTP transactions (list, download,
|
||||
# upload). This is critical because bisync starts with S3 ListObjectsV2, which is neither
|
||||
# a download nor upload — so --header-download/--header-upload would miss list requests.
|
||||
# See: https://www.tigrisdata.com/docs/objects/consistency/
|
||||
TIGRIS_CONSISTENCY_HEADERS = [
|
||||
"--header",
|
||||
"X-Tigris-Consistent: true",
|
||||
]
|
||||
|
||||
|
||||
class RunResult(Protocol):
|
||||
returncode: int
|
||||
@@ -210,8 +220,12 @@ def project_sync(
|
||||
"sync",
|
||||
str(local_path),
|
||||
remote_path,
|
||||
*TIGRIS_CONSISTENCY_HEADERS,
|
||||
"--filter-from",
|
||||
str(filter_path),
|
||||
# Prevent NUL byte padding on virtual filesystems (e.g. Google Drive File Stream)
|
||||
# See: rclone/rclone#6801
|
||||
"--local-no-preallocate",
|
||||
]
|
||||
|
||||
if verbose:
|
||||
@@ -279,6 +293,7 @@ def project_bisync(
|
||||
"bisync",
|
||||
str(local_path),
|
||||
remote_path,
|
||||
*TIGRIS_CONSISTENCY_HEADERS,
|
||||
"--resilient",
|
||||
"--conflict-resolve=newer",
|
||||
"--max-delete=25",
|
||||
@@ -287,6 +302,9 @@ def project_bisync(
|
||||
str(filter_path),
|
||||
"--workdir",
|
||||
str(state_path),
|
||||
# Prevent NUL byte padding on virtual filesystems (e.g. Google Drive File Stream)
|
||||
# See: rclone/rclone#6801
|
||||
"--local-no-preallocate",
|
||||
]
|
||||
|
||||
# Add --create-empty-src-dirs if rclone version supports it (v1.64+)
|
||||
@@ -354,6 +372,7 @@ def project_check(
|
||||
"check",
|
||||
str(local_path),
|
||||
remote_path,
|
||||
*TIGRIS_CONSISTENCY_HEADERS,
|
||||
"--filter-from",
|
||||
str(filter_path),
|
||||
]
|
||||
@@ -393,6 +412,6 @@ def project_ls(
|
||||
if path:
|
||||
remote_path = f"{remote_path}/{path}"
|
||||
|
||||
cmd = ["rclone", "ls", remote_path]
|
||||
cmd = ["rclone", "ls", *TIGRIS_CONSISTENCY_HEADERS, remote_path]
|
||||
result = run(cmd, capture_output=True, text=True, check=True)
|
||||
return result.stdout.splitlines()
|
||||
|
||||
@@ -58,7 +58,7 @@ async def run_sync(
|
||||
try:
|
||||
async with get_client() as client:
|
||||
project_item = await get_active_project(client, project, None)
|
||||
url = f"{project_item.project_url}/project/sync"
|
||||
url = f"/v2/projects/{project_item.external_id}/sync"
|
||||
params = []
|
||||
if force_full:
|
||||
params.append("force_full=true")
|
||||
@@ -92,7 +92,7 @@ async def get_project_info(project: str):
|
||||
try:
|
||||
async with get_client() as client:
|
||||
project_item = await get_active_project(client, project, None)
|
||||
response = await call_get(client, f"{project_item.project_url}/project/info")
|
||||
response = await call_get(client, f"/v2/projects/{project_item.external_id}/info")
|
||||
return ProjectInfoResponse.model_validate(response.json())
|
||||
except (ToolError, ValueError) as e:
|
||||
console.print(f"[red]Sync failed: {e}[/red]")
|
||||
|
||||
@@ -0,0 +1,153 @@
|
||||
"""Doctor command for local consistency checks."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import tempfile
|
||||
import uuid
|
||||
from pathlib import Path
|
||||
|
||||
from loguru import logger
|
||||
from mcp.server.fastmcp.exceptions import ToolError
|
||||
from rich.console import Console
|
||||
import typer
|
||||
|
||||
from basic_memory.cli.app import app
|
||||
from basic_memory.cli.commands.command_utils import run_with_cleanup
|
||||
from basic_memory.cli.commands.routing import force_routing, validate_routing_flags
|
||||
from basic_memory.markdown.entity_parser import EntityParser
|
||||
from basic_memory.markdown.markdown_processor import MarkdownProcessor
|
||||
from basic_memory.markdown.schemas import EntityFrontmatter, EntityMarkdown
|
||||
from basic_memory.mcp.async_client import get_client
|
||||
from basic_memory.mcp.clients import KnowledgeClient, ProjectClient, SearchClient
|
||||
from basic_memory.mcp.tools.utils import call_post
|
||||
from basic_memory.schemas.base import Entity
|
||||
from basic_memory.schemas.project_info import ProjectInfoRequest
|
||||
from basic_memory.schemas.search import SearchQuery
|
||||
from basic_memory.schemas import SyncReportResponse
|
||||
|
||||
console = Console()
|
||||
|
||||
|
||||
async def run_doctor() -> None:
|
||||
"""Run local consistency checks for file <-> database flows."""
|
||||
console.print("[blue]Running Basic Memory doctor checks...[/blue]")
|
||||
|
||||
project_name = f"doctor-{uuid.uuid4().hex[:8]}"
|
||||
api_note_title = "Doctor API Note"
|
||||
manual_note_title = "Doctor Manual Note"
|
||||
manual_permalink = "doctor/manual-note"
|
||||
|
||||
with tempfile.TemporaryDirectory() as temp_dir:
|
||||
temp_path = Path(temp_dir)
|
||||
|
||||
async with get_client() as client:
|
||||
project_client = ProjectClient(client)
|
||||
project_request = ProjectInfoRequest(
|
||||
name=project_name,
|
||||
path=str(temp_path),
|
||||
set_default=False,
|
||||
)
|
||||
|
||||
project_id: str | None = None
|
||||
|
||||
try:
|
||||
status = await project_client.create_project(project_request.model_dump())
|
||||
if not status.new_project:
|
||||
raise ValueError("Failed to create doctor project")
|
||||
project_id = status.new_project.external_id
|
||||
console.print(f"[green]OK[/green] Created doctor project: {project_name}")
|
||||
|
||||
# --- DB -> File: create an entity via API ---
|
||||
knowledge_client = KnowledgeClient(client, project_id)
|
||||
api_note = Entity(
|
||||
title=api_note_title,
|
||||
directory="doctor",
|
||||
entity_type="note",
|
||||
content_type="text/markdown",
|
||||
content=f"# {api_note_title}\n\n- [note] API to file check",
|
||||
entity_metadata={"tags": ["doctor"]},
|
||||
)
|
||||
api_result = await knowledge_client.create_entity(api_note.model_dump(), fast=False)
|
||||
|
||||
api_file = temp_path / api_result.file_path
|
||||
if not api_file.exists():
|
||||
raise ValueError(f"API note file missing: {api_result.file_path}")
|
||||
|
||||
api_text = api_file.read_text(encoding="utf-8")
|
||||
if api_note_title not in api_text:
|
||||
raise ValueError("API note content missing from file")
|
||||
|
||||
console.print("[green]OK[/green] API write created file")
|
||||
|
||||
# --- File -> DB: write markdown file directly, then sync ---
|
||||
parser = EntityParser(temp_path)
|
||||
processor = MarkdownProcessor(parser)
|
||||
manual_markdown = EntityMarkdown(
|
||||
frontmatter=EntityFrontmatter(
|
||||
metadata={
|
||||
"title": manual_note_title,
|
||||
"type": "note",
|
||||
"permalink": manual_permalink,
|
||||
"tags": ["doctor"],
|
||||
}
|
||||
),
|
||||
content=f"# {manual_note_title}\n\n- [note] File to DB check",
|
||||
)
|
||||
|
||||
manual_path = temp_path / "doctor" / "manual-note.md"
|
||||
await processor.write_file(manual_path, manual_markdown)
|
||||
console.print("[green]OK[/green] Manual file written")
|
||||
|
||||
sync_response = await call_post(
|
||||
client,
|
||||
f"/v2/projects/{project_id}/sync?force_full=true&run_in_background=false",
|
||||
)
|
||||
sync_report = SyncReportResponse.model_validate(sync_response.json())
|
||||
if sync_report.total == 0:
|
||||
raise ValueError("Sync did not detect any changes")
|
||||
|
||||
console.print("[green]OK[/green] Sync indexed manual file")
|
||||
|
||||
search_client = SearchClient(client, project_id)
|
||||
search_query = SearchQuery(title=manual_note_title)
|
||||
search_results = await search_client.search(
|
||||
search_query.model_dump(), page=1, page_size=5
|
||||
)
|
||||
if not any(result.title == manual_note_title for result in search_results.results):
|
||||
raise ValueError("Manual note not found in search index")
|
||||
|
||||
console.print("[green]OK[/green] Search confirmed manual file")
|
||||
|
||||
status_response = await call_post(client, f"/v2/projects/{project_id}/status")
|
||||
status_report = SyncReportResponse.model_validate(status_response.json())
|
||||
if status_report.total != 0:
|
||||
raise ValueError("Project status not clean after sync")
|
||||
|
||||
console.print("[green]OK[/green] Status clean after sync")
|
||||
|
||||
finally:
|
||||
if project_id:
|
||||
await project_client.delete_project(project_id)
|
||||
|
||||
console.print("[green]Doctor checks passed.[/green]")
|
||||
|
||||
|
||||
@app.command()
|
||||
def doctor(
|
||||
local: bool = typer.Option(
|
||||
False, "--local", help="Force local API routing (ignore cloud mode)"
|
||||
),
|
||||
cloud: bool = typer.Option(False, "--cloud", help="Force cloud API routing"),
|
||||
) -> None:
|
||||
"""Run local consistency checks to verify file/database sync."""
|
||||
try:
|
||||
validate_routing_flags(local, cloud)
|
||||
with force_routing(local=local, cloud=cloud):
|
||||
run_with_cleanup(run_doctor())
|
||||
except (ToolError, ValueError) as e:
|
||||
console.print(f"[red]Doctor failed: {e}[/red]")
|
||||
raise typer.Exit(code=1)
|
||||
except Exception as e:
|
||||
logger.error(f"Doctor failed: {e}")
|
||||
typer.echo(f"Doctor failed: {e}", err=True)
|
||||
raise typer.Exit(code=1) # pragma: no cover
|
||||
@@ -17,60 +17,64 @@ import basic_memory.mcp.tools # noqa: F401 # pragma: no cover
|
||||
import basic_memory.mcp.prompts # noqa: F401 # pragma: no cover
|
||||
from loguru import logger
|
||||
|
||||
config = ConfigManager().config
|
||||
|
||||
if not config.cloud_mode_enabled:
|
||||
@app.command()
|
||||
def mcp(
|
||||
transport: str = typer.Option("stdio", help="Transport type: stdio, streamable-http, or sse"),
|
||||
host: str = typer.Option(
|
||||
"0.0.0.0", help="Host for HTTP transports (use 0.0.0.0 to allow external connections)"
|
||||
),
|
||||
port: int = typer.Option(8000, help="Port for HTTP transports"),
|
||||
path: str = typer.Option("/mcp", help="Path prefix for streamable-http transport"),
|
||||
project: Optional[str] = typer.Option(None, help="Restrict MCP server to single project"),
|
||||
): # pragma: no cover
|
||||
"""Run the MCP server with configurable transport options.
|
||||
|
||||
@app.command()
|
||||
def mcp(
|
||||
transport: str = typer.Option(
|
||||
"stdio", help="Transport type: stdio, streamable-http, or sse"
|
||||
),
|
||||
host: str = typer.Option(
|
||||
"0.0.0.0", help="Host for HTTP transports (use 0.0.0.0 to allow external connections)"
|
||||
),
|
||||
port: int = typer.Option(8000, help="Port for HTTP transports"),
|
||||
path: str = typer.Option("/mcp", help="Path prefix for streamable-http transport"),
|
||||
project: Optional[str] = typer.Option(None, help="Restrict MCP server to single project"),
|
||||
): # pragma: no cover
|
||||
"""Run the MCP server with configurable transport options.
|
||||
This command starts an MCP server using one of three transport options:
|
||||
|
||||
This command starts an MCP server using one of three transport options:
|
||||
- stdio: Standard I/O (good for local usage)
|
||||
- streamable-http: Recommended for web deployments (default)
|
||||
- sse: Server-Sent Events (for compatibility with existing clients)
|
||||
|
||||
- stdio: Standard I/O (good for local usage)
|
||||
- streamable-http: Recommended for web deployments (default)
|
||||
- sse: Server-Sent Events (for compatibility with existing clients)
|
||||
Initialization, file sync, and cleanup are handled by the MCP server's lifespan.
|
||||
|
||||
Initialization, file sync, and cleanup are handled by the MCP server's lifespan.
|
||||
"""
|
||||
# Initialize logging for MCP (file only, stdout breaks protocol)
|
||||
init_mcp_logging()
|
||||
Note: This command is available regardless of cloud mode setting.
|
||||
Users who have cloud mode enabled can still use local MCP for Claude Code
|
||||
and Claude Desktop while using cloud MCP for web and mobile access.
|
||||
"""
|
||||
# Force local routing for local MCP server
|
||||
# Why: The local MCP server should always talk to the local API, not the cloud proxy.
|
||||
# Even when cloud_mode_enabled is True, stdio MCP runs locally and needs local API access.
|
||||
os.environ["BASIC_MEMORY_FORCE_LOCAL"] = "true"
|
||||
|
||||
# Validate and set project constraint if specified
|
||||
if project:
|
||||
config_manager = ConfigManager()
|
||||
project_name, _ = config_manager.get_project(project)
|
||||
if not project_name:
|
||||
typer.echo(f"No project found named: {project}", err=True)
|
||||
raise typer.Exit(1)
|
||||
# Initialize logging for MCP (file only, stdout breaks protocol)
|
||||
init_mcp_logging()
|
||||
|
||||
# Set env var with validated project name
|
||||
os.environ["BASIC_MEMORY_MCP_PROJECT"] = project_name
|
||||
logger.info(f"MCP server constrained to project: {project_name}")
|
||||
# Validate and set project constraint if specified
|
||||
if project:
|
||||
config_manager = ConfigManager()
|
||||
project_name, _ = config_manager.get_project(project)
|
||||
if not project_name:
|
||||
typer.echo(f"No project found named: {project}", err=True)
|
||||
raise typer.Exit(1)
|
||||
|
||||
# Run the MCP server (blocks)
|
||||
# Lifespan handles: initialization, migrations, file sync, cleanup
|
||||
logger.info(f"Starting MCP server with {transport.upper()} transport")
|
||||
# Set env var with validated project name
|
||||
os.environ["BASIC_MEMORY_MCP_PROJECT"] = project_name
|
||||
logger.info(f"MCP server constrained to project: {project_name}")
|
||||
|
||||
if transport == "stdio":
|
||||
mcp_server.run(
|
||||
transport=transport,
|
||||
)
|
||||
elif transport == "streamable-http" or transport == "sse":
|
||||
mcp_server.run(
|
||||
transport=transport,
|
||||
host=host,
|
||||
port=port,
|
||||
path=path,
|
||||
log_level="INFO",
|
||||
)
|
||||
# Run the MCP server (blocks)
|
||||
# Lifespan handles: initialization, migrations, file sync, cleanup
|
||||
logger.info(f"Starting MCP server with {transport.upper()} transport")
|
||||
|
||||
if transport == "stdio":
|
||||
mcp_server.run(
|
||||
transport=transport,
|
||||
)
|
||||
elif transport == "streamable-http" or transport == "sse":
|
||||
mcp_server.run(
|
||||
transport=transport,
|
||||
host=host,
|
||||
port=port,
|
||||
path=path,
|
||||
log_level="INFO",
|
||||
)
|
||||
|
||||
@@ -1,32 +1,33 @@
|
||||
"""Command module for basic-memory project management."""
|
||||
|
||||
import json
|
||||
import os
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
|
||||
import typer
|
||||
from rich.console import Console
|
||||
from rich.panel import Panel
|
||||
from rich.table import Table
|
||||
|
||||
from basic_memory.cli.app import app
|
||||
from basic_memory.cli.commands.command_utils import get_project_info, run_with_cleanup
|
||||
from basic_memory.cli.commands.routing import force_routing, validate_routing_flags
|
||||
from basic_memory.config import ConfigManager
|
||||
import json
|
||||
from datetime import datetime
|
||||
|
||||
from rich.panel import Panel
|
||||
from basic_memory.mcp.async_client import get_client
|
||||
from basic_memory.mcp.tools.utils import call_get, call_post, call_delete, call_put, call_patch
|
||||
from basic_memory.mcp.tools.utils import call_delete, call_get, call_patch, call_post, call_put
|
||||
from basic_memory.schemas.project_info import ProjectList, ProjectStatusResponse
|
||||
from basic_memory.schemas.v2 import ProjectResolveResponse
|
||||
from basic_memory.utils import generate_permalink, normalize_project_path
|
||||
|
||||
# Import rclone commands for project sync
|
||||
from basic_memory.cli.commands.cloud.rclone_commands import (
|
||||
SyncProject,
|
||||
RcloneError,
|
||||
project_sync,
|
||||
SyncProject,
|
||||
project_bisync,
|
||||
project_check,
|
||||
project_ls,
|
||||
project_sync,
|
||||
)
|
||||
from basic_memory.cli.commands.cloud.bisync_commands import get_mount_info
|
||||
|
||||
@@ -46,28 +47,43 @@ def format_path(path: str) -> str:
|
||||
|
||||
|
||||
@project_app.command("list")
|
||||
def list_projects() -> None:
|
||||
"""List all Basic Memory projects."""
|
||||
def list_projects(
|
||||
local: bool = typer.Option(
|
||||
False, "--local", help="Force local API routing (ignore cloud mode)"
|
||||
),
|
||||
cloud: bool = typer.Option(False, "--cloud", help="Force cloud API routing"),
|
||||
) -> None:
|
||||
"""List all Basic Memory projects.
|
||||
|
||||
Use --local to force local routing when cloud mode is enabled.
|
||||
Use --cloud to force cloud routing when cloud mode is disabled.
|
||||
"""
|
||||
try:
|
||||
validate_routing_flags(local, cloud)
|
||||
except ValueError as e:
|
||||
console.print(f"[red]Error: {e}[/red]")
|
||||
raise typer.Exit(1)
|
||||
|
||||
async def _list_projects():
|
||||
async with get_client() as client:
|
||||
response = await call_get(client, "/projects/projects")
|
||||
response = await call_get(client, "/v2/projects/")
|
||||
return ProjectList.model_validate(response.json())
|
||||
|
||||
try:
|
||||
result = run_with_cleanup(_list_projects())
|
||||
with force_routing(local=local, cloud=cloud):
|
||||
result = run_with_cleanup(_list_projects())
|
||||
config = ConfigManager().config
|
||||
|
||||
table = Table(title="Basic Memory Projects")
|
||||
table.add_column("Name", style="cyan")
|
||||
table.add_column("Path", style="green")
|
||||
|
||||
# Add Local Path column if in cloud mode
|
||||
if config.cloud_mode_enabled:
|
||||
# Add Local Path column if in cloud mode and not forcing local
|
||||
if config.cloud_mode_enabled and not local:
|
||||
table.add_column("Local Path", style="yellow", no_wrap=True, overflow="fold")
|
||||
|
||||
# Show Default column in local mode or if default_project_mode is enabled in cloud mode
|
||||
show_default_column = not config.cloud_mode_enabled or config.default_project_mode
|
||||
show_default_column = local or not config.cloud_mode_enabled or config.default_project_mode
|
||||
if show_default_column:
|
||||
table.add_column("Default", style="magenta")
|
||||
|
||||
@@ -78,8 +94,8 @@ def list_projects() -> None:
|
||||
# Build row based on mode
|
||||
row = [project.name, format_path(normalized_path)]
|
||||
|
||||
# Add local path if in cloud mode
|
||||
if config.cloud_mode_enabled:
|
||||
# Add local path if in cloud mode and not forcing local
|
||||
if config.cloud_mode_enabled and not local:
|
||||
local_path = ""
|
||||
if project.name in config.cloud_projects:
|
||||
local_path = config.cloud_projects[project.name].local_path or ""
|
||||
@@ -108,9 +124,16 @@ def add_project(
|
||||
None, "--local-path", help="Local sync path for cloud mode (optional)"
|
||||
),
|
||||
set_default: bool = typer.Option(False, "--default", help="Set as default project"),
|
||||
local: bool = typer.Option(
|
||||
False, "--local", help="Force local API routing (ignore cloud mode)"
|
||||
),
|
||||
cloud: bool = typer.Option(False, "--cloud", help="Force cloud API routing"),
|
||||
) -> None:
|
||||
"""Add a new project.
|
||||
|
||||
Use --local to force local routing when cloud mode is enabled.
|
||||
Use --cloud to force cloud routing when cloud mode is disabled.
|
||||
|
||||
Cloud mode examples:\n
|
||||
bm project add research # No local sync\n
|
||||
bm project add research --local-path ~/docs # With local sync\n
|
||||
@@ -118,14 +141,23 @@ def add_project(
|
||||
Local mode example:\n
|
||||
bm project add research ~/Documents/research
|
||||
"""
|
||||
try:
|
||||
validate_routing_flags(local, cloud)
|
||||
except ValueError as e:
|
||||
console.print(f"[red]Error: {e}[/red]")
|
||||
raise typer.Exit(1)
|
||||
|
||||
config = ConfigManager().config
|
||||
|
||||
# Determine effective mode: local flag forces local mode behavior
|
||||
effective_cloud_mode = config.cloud_mode_enabled and not local
|
||||
|
||||
# Resolve local sync path early (needed for both cloud and local mode)
|
||||
local_sync_path: str | None = None
|
||||
if local_path:
|
||||
local_sync_path = Path(os.path.abspath(os.path.expanduser(local_path))).as_posix()
|
||||
|
||||
if config.cloud_mode_enabled:
|
||||
if effective_cloud_mode:
|
||||
# Cloud mode: path auto-generated from name, local sync is optional
|
||||
|
||||
async def _add_project():
|
||||
@@ -136,7 +168,7 @@ def add_project(
|
||||
"local_sync_path": local_sync_path,
|
||||
"set_default": set_default,
|
||||
}
|
||||
response = await call_post(client, "/projects/projects", json=data)
|
||||
response = await call_post(client, "/v2/projects/", json=data)
|
||||
return ProjectStatusResponse.model_validate(response.json())
|
||||
else:
|
||||
# Local mode: path is required
|
||||
@@ -150,15 +182,16 @@ def add_project(
|
||||
async def _add_project():
|
||||
async with get_client() as client:
|
||||
data = {"name": name, "path": resolved_path, "set_default": set_default}
|
||||
response = await call_post(client, "/projects/projects", json=data)
|
||||
response = await call_post(client, "/v2/projects/", json=data)
|
||||
return ProjectStatusResponse.model_validate(response.json())
|
||||
|
||||
try:
|
||||
result = run_with_cleanup(_add_project())
|
||||
with force_routing(local=local, cloud=cloud):
|
||||
result = run_with_cleanup(_add_project())
|
||||
console.print(f"[green]{result.message}[/green]")
|
||||
|
||||
# Save local sync path to config if in cloud mode
|
||||
if config.cloud_mode_enabled and local_sync_path:
|
||||
if effective_cloud_mode and local_sync_path:
|
||||
from basic_memory.config import CloudProjectConfig
|
||||
|
||||
# Create local directory if it doesn't exist
|
||||
@@ -202,7 +235,7 @@ def setup_project_sync(
|
||||
async def _verify_project_exists():
|
||||
"""Verify the project exists on cloud by listing all projects."""
|
||||
async with get_client() as client:
|
||||
response = await call_get(client, "/projects/projects")
|
||||
response = await call_get(client, "/v2/projects/")
|
||||
project_list = response.json()
|
||||
project_names = [p["name"] for p in project_list["projects"]]
|
||||
if name not in project_names:
|
||||
@@ -243,8 +276,21 @@ def remove_project(
|
||||
delete_notes: bool = typer.Option(
|
||||
False, "--delete-notes", help="Delete project files from disk"
|
||||
),
|
||||
local: bool = typer.Option(
|
||||
False, "--local", help="Force local API routing (ignore cloud mode)"
|
||||
),
|
||||
cloud: bool = typer.Option(False, "--cloud", help="Force cloud API routing"),
|
||||
) -> None:
|
||||
"""Remove a project."""
|
||||
"""Remove a project.
|
||||
|
||||
Use --local to force local routing when cloud mode is enabled.
|
||||
Use --cloud to force cloud routing when cloud mode is disabled.
|
||||
"""
|
||||
try:
|
||||
validate_routing_flags(local, cloud)
|
||||
except ValueError as e:
|
||||
console.print(f"[red]Error: {e}[/red]")
|
||||
raise typer.Exit(1)
|
||||
|
||||
async def _remove_project():
|
||||
async with get_client() as client:
|
||||
@@ -265,11 +311,11 @@ def remove_project(
|
||||
try:
|
||||
# Get config to check for local sync path and bisync state
|
||||
config = ConfigManager().config
|
||||
local_path = None
|
||||
local_path_config = None
|
||||
has_bisync_state = False
|
||||
|
||||
if config.cloud_mode_enabled and name in config.cloud_projects:
|
||||
local_path = config.cloud_projects[name].local_path
|
||||
if config.cloud_mode_enabled and not local and name in config.cloud_projects:
|
||||
local_path_config = config.cloud_projects[name].local_path
|
||||
|
||||
# Check for bisync state
|
||||
from basic_memory.cli.commands.cloud.rclone_commands import get_project_bisync_state
|
||||
@@ -278,17 +324,18 @@ def remove_project(
|
||||
has_bisync_state = bisync_state_path.exists()
|
||||
|
||||
# Remove project from cloud/API
|
||||
result = run_with_cleanup(_remove_project())
|
||||
with force_routing(local=local, cloud=cloud):
|
||||
result = run_with_cleanup(_remove_project())
|
||||
console.print(f"[green]{result.message}[/green]")
|
||||
|
||||
# Clean up local sync directory if it exists and delete_notes is True
|
||||
if delete_notes and local_path:
|
||||
local_dir = Path(local_path)
|
||||
if delete_notes and local_path_config:
|
||||
local_dir = Path(local_path_config)
|
||||
if local_dir.exists():
|
||||
import shutil
|
||||
|
||||
shutil.rmtree(local_dir)
|
||||
console.print(f"[green]Removed local sync directory: {local_path}[/green]")
|
||||
console.print(f"[green]Removed local sync directory: {local_path_config}[/green]")
|
||||
|
||||
# Clean up bisync state if it exists
|
||||
if has_bisync_state:
|
||||
@@ -301,14 +348,14 @@ def remove_project(
|
||||
console.print("[green]Removed bisync state[/green]")
|
||||
|
||||
# Clean up cloud_projects config entry
|
||||
if config.cloud_mode_enabled and name in config.cloud_projects:
|
||||
if config.cloud_mode_enabled and not local and name in config.cloud_projects:
|
||||
del config.cloud_projects[name]
|
||||
ConfigManager().save_config(config)
|
||||
|
||||
# Show informative message if files were not deleted
|
||||
if not delete_notes:
|
||||
if local_path:
|
||||
console.print(f"[yellow]Note: Local files remain at {local_path}[/yellow]")
|
||||
if local_path_config:
|
||||
console.print(f"[yellow]Note: Local files remain at {local_path_config}[/yellow]")
|
||||
|
||||
except Exception as e:
|
||||
console.print(f"[red]Error removing project: {str(e)}[/red]")
|
||||
@@ -318,15 +365,24 @@ def remove_project(
|
||||
@project_app.command("default")
|
||||
def set_default_project(
|
||||
name: str = typer.Argument(..., help="Name of the project to set as CLI default"),
|
||||
local: bool = typer.Option(
|
||||
False, "--local", help="Force local API routing (required in cloud mode)"
|
||||
),
|
||||
) -> None:
|
||||
"""Set the default project when 'config.default_project_mode' is set.
|
||||
|
||||
Note: This command is only available in local mode.
|
||||
In cloud mode, use --local to modify the local configuration.
|
||||
"""
|
||||
config = ConfigManager().config
|
||||
|
||||
if config.cloud_mode_enabled:
|
||||
console.print("[red]Error: 'default' command is not available in cloud mode[/red]")
|
||||
# Trigger: cloud mode enabled without --local flag
|
||||
# Why: default project is a local configuration concept
|
||||
# Outcome: require explicit --local flag to modify local config in cloud mode
|
||||
if config.cloud_mode_enabled and not local:
|
||||
console.print(
|
||||
"[red]Error: 'default' command requires --local flag in cloud mode[/red]\n"
|
||||
"[yellow]Hint: Use 'bm project default <name> --local' to set local default[/yellow]"
|
||||
)
|
||||
raise typer.Exit(1)
|
||||
|
||||
async def _set_default():
|
||||
@@ -346,7 +402,8 @@ def set_default_project(
|
||||
return ProjectStatusResponse.model_validate(response.json())
|
||||
|
||||
try:
|
||||
result = run_with_cleanup(_set_default())
|
||||
with force_routing(local=local):
|
||||
result = run_with_cleanup(_set_default())
|
||||
console.print(f"[green]{result.message}[/green]")
|
||||
except Exception as e:
|
||||
console.print(f"[red]Error setting default project: {str(e)}[/red]")
|
||||
@@ -354,24 +411,35 @@ def set_default_project(
|
||||
|
||||
|
||||
@project_app.command("sync-config")
|
||||
def synchronize_projects() -> None:
|
||||
def synchronize_projects(
|
||||
local: bool = typer.Option(
|
||||
False, "--local", help="Force local API routing (required in cloud mode)"
|
||||
),
|
||||
) -> None:
|
||||
"""Synchronize project config between configuration file and database.
|
||||
|
||||
Note: This command is only available in local mode.
|
||||
In cloud mode, use --local to sync local configuration.
|
||||
"""
|
||||
config = ConfigManager().config
|
||||
|
||||
if config.cloud_mode_enabled:
|
||||
console.print("[red]Error: 'sync-config' command is not available in cloud mode[/red]")
|
||||
# Trigger: cloud mode enabled without --local flag
|
||||
# Why: sync-config syncs local config file with local database
|
||||
# Outcome: require explicit --local flag to clarify intent in cloud mode
|
||||
if config.cloud_mode_enabled and not local:
|
||||
console.print(
|
||||
"[red]Error: 'sync-config' command requires --local flag in cloud mode[/red]\n"
|
||||
"[yellow]Hint: Use 'bm project sync-config --local' to sync local config[/yellow]"
|
||||
)
|
||||
raise typer.Exit(1)
|
||||
|
||||
async def _sync_config():
|
||||
async with get_client() as client:
|
||||
response = await call_post(client, "/projects/config/sync")
|
||||
response = await call_post(client, "/v2/projects/config/sync")
|
||||
return ProjectStatusResponse.model_validate(response.json())
|
||||
|
||||
try:
|
||||
result = run_with_cleanup(_sync_config())
|
||||
with force_routing(local=local):
|
||||
result = run_with_cleanup(_sync_config())
|
||||
console.print(f"[green]{result.message}[/green]")
|
||||
except Exception as e: # pragma: no cover
|
||||
console.print(f"[red]Error synchronizing projects: {str(e)}[/red]")
|
||||
@@ -382,15 +450,24 @@ def synchronize_projects() -> None:
|
||||
def move_project(
|
||||
name: str = typer.Argument(..., help="Name of the project to move"),
|
||||
new_path: str = typer.Argument(..., help="New absolute path for the project"),
|
||||
local: bool = typer.Option(
|
||||
False, "--local", help="Force local API routing (required in cloud mode)"
|
||||
),
|
||||
) -> None:
|
||||
"""Move a project to a new location.
|
||||
|
||||
Note: This command is only available in local mode.
|
||||
In cloud mode, use --local to modify local project paths.
|
||||
"""
|
||||
config = ConfigManager().config
|
||||
|
||||
if config.cloud_mode_enabled:
|
||||
console.print("[red]Error: 'move' command is not available in cloud mode[/red]")
|
||||
# Trigger: cloud mode enabled without --local flag
|
||||
# Why: moving a project is a local file system operation
|
||||
# Outcome: require explicit --local flag to clarify intent in cloud mode
|
||||
if config.cloud_mode_enabled and not local:
|
||||
console.print(
|
||||
"[red]Error: 'move' command requires --local flag in cloud mode[/red]\n"
|
||||
"[yellow]Hint: Use 'bm project move <name> <path> --local' to move local project[/yellow]"
|
||||
)
|
||||
raise typer.Exit(1)
|
||||
|
||||
# Resolve to absolute path
|
||||
@@ -399,14 +476,20 @@ def move_project(
|
||||
async def _move_project():
|
||||
async with get_client() as client:
|
||||
data = {"path": resolved_path}
|
||||
project_permalink = generate_permalink(name)
|
||||
|
||||
# TODO fix route to use ProjectPathDep
|
||||
response = await call_patch(client, f"/{name}/project/{project_permalink}", json=data)
|
||||
resolve_response = await call_post(
|
||||
client,
|
||||
"/v2/projects/resolve",
|
||||
json={"identifier": name},
|
||||
)
|
||||
project_info = ProjectResolveResponse.model_validate(resolve_response.json())
|
||||
response = await call_patch(
|
||||
client, f"/v2/projects/{project_info.external_id}", json=data
|
||||
)
|
||||
return ProjectStatusResponse.model_validate(response.json())
|
||||
|
||||
try:
|
||||
result = run_with_cleanup(_move_project())
|
||||
with force_routing(local=local):
|
||||
result = run_with_cleanup(_move_project())
|
||||
console.print(f"[green]{result.message}[/green]")
|
||||
|
||||
# Show important file movement reminder
|
||||
@@ -453,7 +536,7 @@ def sync_project_command(
|
||||
# Get project info
|
||||
async def _get_project():
|
||||
async with get_client() as client:
|
||||
response = await call_get(client, "/projects/projects")
|
||||
response = await call_get(client, "/v2/projects/")
|
||||
projects_list = ProjectList.model_validate(response.json())
|
||||
for proj in projects_list.projects:
|
||||
if generate_permalink(proj.name) == generate_permalink(name):
|
||||
@@ -494,9 +577,10 @@ def sync_project_command(
|
||||
|
||||
async def _trigger_db_sync():
|
||||
async with get_client() as client:
|
||||
permalink = generate_permalink(name)
|
||||
response = await call_post(
|
||||
client, f"/{permalink}/project/sync?force_full=true", json={}
|
||||
client,
|
||||
f"/v2/projects/{project_data.external_id}/sync?force_full=true",
|
||||
json={},
|
||||
)
|
||||
return response.json()
|
||||
|
||||
@@ -544,7 +628,7 @@ def bisync_project_command(
|
||||
# Get project info
|
||||
async def _get_project():
|
||||
async with get_client() as client:
|
||||
response = await call_get(client, "/projects/projects")
|
||||
response = await call_get(client, "/v2/projects/")
|
||||
projects_list = ProjectList.model_validate(response.json())
|
||||
for proj in projects_list.projects:
|
||||
if generate_permalink(proj.name) == generate_permalink(name):
|
||||
@@ -592,9 +676,10 @@ def bisync_project_command(
|
||||
|
||||
async def _trigger_db_sync():
|
||||
async with get_client() as client:
|
||||
permalink = generate_permalink(name)
|
||||
response = await call_post(
|
||||
client, f"/{permalink}/project/sync?force_full=true", json={}
|
||||
client,
|
||||
f"/v2/projects/{project_data.external_id}/sync?force_full=true",
|
||||
json={},
|
||||
)
|
||||
return response.json()
|
||||
|
||||
@@ -638,7 +723,7 @@ def check_project_command(
|
||||
# Get project info
|
||||
async def _get_project():
|
||||
async with get_client() as client:
|
||||
response = await call_get(client, "/projects/projects")
|
||||
response = await call_get(client, "/v2/projects/")
|
||||
projects_list = ProjectList.model_validate(response.json())
|
||||
for proj in projects_list.projects:
|
||||
if generate_permalink(proj.name) == generate_permalink(name):
|
||||
@@ -739,7 +824,7 @@ def ls_project_command(
|
||||
# Get project info
|
||||
async def _get_project():
|
||||
async with get_client() as client:
|
||||
response = await call_get(client, "/projects/projects")
|
||||
response = await call_get(client, "/v2/projects/")
|
||||
projects_list = ProjectList.model_validate(response.json())
|
||||
for proj in projects_list.projects:
|
||||
if generate_permalink(proj.name) == generate_permalink(name):
|
||||
@@ -779,11 +864,26 @@ def ls_project_command(
|
||||
def display_project_info(
|
||||
name: str = typer.Argument(..., help="Name of the project"),
|
||||
json_output: bool = typer.Option(False, "--json", help="Output in JSON format"),
|
||||
local: bool = typer.Option(
|
||||
False, "--local", help="Force local API routing (ignore cloud mode)"
|
||||
),
|
||||
cloud: bool = typer.Option(False, "--cloud", help="Force cloud API routing"),
|
||||
):
|
||||
"""Display detailed information and statistics about the current project."""
|
||||
"""Display detailed information and statistics about the current project.
|
||||
|
||||
Use --local to force local routing when cloud mode is enabled.
|
||||
Use --cloud to force cloud routing when cloud mode is disabled.
|
||||
"""
|
||||
try:
|
||||
validate_routing_flags(local, cloud)
|
||||
except ValueError as e:
|
||||
console.print(f"[red]Error: {e}[/red]")
|
||||
raise typer.Exit(1)
|
||||
|
||||
try:
|
||||
# Get project info
|
||||
info = run_with_cleanup(get_project_info(name))
|
||||
with force_routing(local=local, cloud=cloud):
|
||||
info = run_with_cleanup(get_project_info(name))
|
||||
|
||||
if json_output:
|
||||
# Convert to JSON and print
|
||||
|
||||
@@ -0,0 +1,73 @@
|
||||
"""CLI routing utilities for --local/--cloud flag handling.
|
||||
|
||||
This module provides utilities for CLI commands to override the default routing
|
||||
behavior (determined by cloud_mode_enabled in config). This allows users to:
|
||||
|
||||
1. Use local MCP server even when cloud mode is enabled
|
||||
2. Force local routing for specific CLI commands with --local flag
|
||||
3. Force cloud routing with --cloud flag (requires authentication)
|
||||
|
||||
The routing is controlled via environment variables:
|
||||
- BASIC_MEMORY_FORCE_LOCAL: When "true", forces local ASGI transport
|
||||
- These are checked in basic_memory.mcp.async_client.get_client()
|
||||
"""
|
||||
|
||||
import os
|
||||
from contextlib import contextmanager
|
||||
from typing import Generator
|
||||
|
||||
|
||||
@contextmanager
|
||||
def force_routing(local: bool = False, cloud: bool = False) -> Generator[None, None, None]:
|
||||
"""Context manager to temporarily override routing mode.
|
||||
|
||||
Sets environment variables that are checked by get_client() to determine
|
||||
whether to use local ASGI transport or cloud proxy transport.
|
||||
|
||||
Args:
|
||||
local: If True, force local ASGI transport (ignores cloud_mode_enabled)
|
||||
cloud: If True, clear force_local to allow cloud routing
|
||||
|
||||
Usage:
|
||||
with force_routing(local=True):
|
||||
# All API calls will use local ASGI transport
|
||||
await some_api_call()
|
||||
|
||||
Raises:
|
||||
ValueError: If both local and cloud are True
|
||||
"""
|
||||
if local and cloud:
|
||||
raise ValueError("Cannot specify both --local and --cloud")
|
||||
|
||||
# Save original values
|
||||
original_force_local = os.environ.get("BASIC_MEMORY_FORCE_LOCAL")
|
||||
|
||||
try:
|
||||
if local:
|
||||
# Force local routing by setting the env var
|
||||
os.environ["BASIC_MEMORY_FORCE_LOCAL"] = "true"
|
||||
elif cloud:
|
||||
# Ensure force_local is NOT set, let cloud_mode_enabled take effect
|
||||
os.environ.pop("BASIC_MEMORY_FORCE_LOCAL", None)
|
||||
# If neither is set, don't change anything (use default behavior)
|
||||
yield
|
||||
finally:
|
||||
# Restore original value
|
||||
if original_force_local is None:
|
||||
os.environ.pop("BASIC_MEMORY_FORCE_LOCAL", None)
|
||||
else:
|
||||
os.environ["BASIC_MEMORY_FORCE_LOCAL"] = original_force_local
|
||||
|
||||
|
||||
def validate_routing_flags(local: bool, cloud: bool) -> None:
|
||||
"""Validate that --local and --cloud flags are not both specified.
|
||||
|
||||
Args:
|
||||
local: Value of --local flag
|
||||
cloud: Value of --cloud flag
|
||||
|
||||
Raises:
|
||||
ValueError: If both flags are True
|
||||
"""
|
||||
if local and cloud:
|
||||
raise ValueError("Cannot specify both --local and --cloud flags")
|
||||
@@ -11,6 +11,7 @@ from rich.panel import Panel
|
||||
from rich.tree import Tree
|
||||
|
||||
from basic_memory.cli.app import app
|
||||
from basic_memory.cli.commands.routing import force_routing, validate_routing_flags
|
||||
from basic_memory.mcp.async_client import get_client
|
||||
from basic_memory.mcp.tools.utils import call_post
|
||||
from basic_memory.schemas import SyncReportResponse
|
||||
@@ -145,7 +146,7 @@ async def run_status(project: Optional[str] = None, verbose: bool = False): # p
|
||||
try:
|
||||
async with get_client() as client:
|
||||
project_item = await get_active_project(client, project, None)
|
||||
response = await call_post(client, f"{project_item.project_url}/project/status")
|
||||
response = await call_post(client, f"/v2/projects/{project_item.external_id}/status")
|
||||
sync_report = SyncReportResponse.model_validate(response.json())
|
||||
|
||||
display_changes(project_item.name, "Status", sync_report, verbose)
|
||||
@@ -162,12 +163,25 @@ def status(
|
||||
typer.Option(help="The project name."),
|
||||
] = None,
|
||||
verbose: bool = typer.Option(False, "--verbose", "-v", help="Show detailed file information"),
|
||||
local: bool = typer.Option(
|
||||
False, "--local", help="Force local API routing (ignore cloud mode)"
|
||||
),
|
||||
cloud: bool = typer.Option(False, "--cloud", help="Force cloud API routing"),
|
||||
):
|
||||
"""Show sync status between files and database."""
|
||||
"""Show sync status between files and database.
|
||||
|
||||
Use --local to force local routing when cloud mode is enabled.
|
||||
Use --cloud to force cloud routing when cloud mode is disabled.
|
||||
"""
|
||||
from basic_memory.cli.commands.command_utils import run_with_cleanup
|
||||
|
||||
try:
|
||||
run_with_cleanup(run_status(project, verbose)) # pragma: no cover
|
||||
validate_routing_flags(local, cloud)
|
||||
with force_routing(local=local, cloud=cloud):
|
||||
run_with_cleanup(run_status(project, verbose)) # pragma: no cover
|
||||
except ValueError as e:
|
||||
console.print(f"[red]Error: {e}[/red]")
|
||||
raise typer.Exit(code=1)
|
||||
except Exception as e:
|
||||
logger.error(f"Error checking status: {e}")
|
||||
typer.echo(f"Error checking status: {e}", err=True)
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
"""CLI tool commands for Basic Memory."""
|
||||
|
||||
import json
|
||||
import sys
|
||||
from typing import Annotated, List, Optional
|
||||
|
||||
@@ -9,7 +10,15 @@ from rich import print as rprint
|
||||
|
||||
from basic_memory.cli.app import app
|
||||
from basic_memory.cli.commands.command_utils import run_with_cleanup
|
||||
from basic_memory.cli.commands.routing import force_routing, validate_routing_flags
|
||||
from basic_memory.config import ConfigManager
|
||||
from basic_memory.mcp.async_client import get_client
|
||||
from basic_memory.mcp.clients import KnowledgeClient, ResourceClient
|
||||
from basic_memory.mcp.project_context import get_active_project
|
||||
from basic_memory.mcp.tools.utils import call_get
|
||||
from basic_memory.schemas.base import Entity, TimeFrame
|
||||
from basic_memory.schemas.memory import GraphContext, MemoryUrl, memory_url_path
|
||||
from basic_memory.schemas.search import SearchItemType
|
||||
|
||||
# Import prompts
|
||||
from basic_memory.mcp.prompts.continue_conversation import (
|
||||
@@ -23,14 +32,128 @@ from basic_memory.mcp.tools import read_note as mcp_read_note
|
||||
from basic_memory.mcp.tools import recent_activity as mcp_recent_activity
|
||||
from basic_memory.mcp.tools import search_notes as mcp_search
|
||||
from basic_memory.mcp.tools import write_note as mcp_write_note
|
||||
from basic_memory.schemas.base import TimeFrame
|
||||
from basic_memory.schemas.memory import MemoryUrl
|
||||
from basic_memory.schemas.search import SearchItemType
|
||||
|
||||
tool_app = typer.Typer()
|
||||
app.add_typer(tool_app, name="tool", help="Access to MCP tools via CLI")
|
||||
|
||||
|
||||
# --- JSON output helpers ---
|
||||
# These async functions bypass the MCP tool (which returns formatted strings)
|
||||
# and use API clients directly to return structured data for --format json.
|
||||
|
||||
|
||||
async def _write_note_json(
|
||||
title: str, content: str, folder: str, project_name: Optional[str], tags: Optional[List[str]]
|
||||
) -> dict:
|
||||
"""Write a note and return structured JSON metadata."""
|
||||
# Use the MCP tool to create/update the entity (handles create-or-update logic)
|
||||
await mcp_write_note.fn(title, content, folder, project_name, tags)
|
||||
|
||||
# Resolve the entity to get metadata back
|
||||
async with get_client() as client:
|
||||
active_project = await get_active_project(client, project_name)
|
||||
knowledge_client = KnowledgeClient(client, active_project.external_id)
|
||||
|
||||
entity = Entity(title=title, directory=folder)
|
||||
if not entity.permalink:
|
||||
raise ValueError(f"Could not generate permalink for title={title}, folder={folder}")
|
||||
entity_id = await knowledge_client.resolve_entity(entity.permalink)
|
||||
entity = await knowledge_client.get_entity(entity_id)
|
||||
|
||||
return {
|
||||
"title": entity.title,
|
||||
"permalink": entity.permalink,
|
||||
"content": content,
|
||||
"file_path": entity.file_path,
|
||||
}
|
||||
|
||||
|
||||
async def _read_note_json(
|
||||
identifier: str, project_name: Optional[str], page: int, page_size: int
|
||||
) -> dict:
|
||||
"""Read a note and return structured JSON with content and metadata."""
|
||||
async with get_client() as client:
|
||||
active_project = await get_active_project(client, project_name)
|
||||
knowledge_client = KnowledgeClient(client, active_project.external_id)
|
||||
resource_client = ResourceClient(client, active_project.external_id)
|
||||
|
||||
# Try direct resolution first (works for permalinks and memory URLs)
|
||||
entity_path = memory_url_path(identifier)
|
||||
entity_id = None
|
||||
try:
|
||||
entity_id = await knowledge_client.resolve_entity(entity_path)
|
||||
except Exception:
|
||||
logger.info(f"Direct lookup failed for '{entity_path}', trying title search")
|
||||
|
||||
# Fallback: title search (handles plain titles like "My Note")
|
||||
if entity_id is None:
|
||||
from basic_memory.mcp.tools.search import search_notes as mcp_search_tool
|
||||
|
||||
title_results = await mcp_search_tool.fn(
|
||||
query=identifier, search_type="title", project=project_name
|
||||
)
|
||||
if title_results and hasattr(title_results, "results") and title_results.results:
|
||||
result = title_results.results[0]
|
||||
if result.permalink:
|
||||
entity_id = await knowledge_client.resolve_entity(result.permalink)
|
||||
|
||||
if entity_id is None:
|
||||
raise ValueError(f"Could not find note matching: {identifier}")
|
||||
|
||||
entity = await knowledge_client.get_entity(entity_id)
|
||||
response = await resource_client.read(entity_id, page=page, page_size=page_size)
|
||||
|
||||
return {
|
||||
"title": entity.title,
|
||||
"permalink": entity.permalink,
|
||||
"content": response.text,
|
||||
"file_path": entity.file_path,
|
||||
}
|
||||
|
||||
|
||||
async def _recent_activity_json(
|
||||
type: Optional[List[SearchItemType]],
|
||||
depth: Optional[int],
|
||||
timeframe: Optional[TimeFrame],
|
||||
project_name: Optional[str] = None,
|
||||
page: int = 1,
|
||||
page_size: int = 50,
|
||||
) -> list:
|
||||
"""Get recent activity and return structured JSON list."""
|
||||
async with get_client() as client:
|
||||
# Build query params matching the MCP tool's logic
|
||||
params: dict = {"page": page, "page_size": page_size, "max_related": 10}
|
||||
if depth:
|
||||
params["depth"] = depth
|
||||
if timeframe:
|
||||
params["timeframe"] = timeframe
|
||||
if type:
|
||||
params["type"] = [t.value for t in type]
|
||||
|
||||
active_project = await get_active_project(client, project_name)
|
||||
response = await call_get(
|
||||
client,
|
||||
f"/v2/projects/{active_project.external_id}/memory/recent",
|
||||
params=params,
|
||||
)
|
||||
activity_data = GraphContext.model_validate(response.json())
|
||||
|
||||
# Extract entity results
|
||||
results = []
|
||||
for result in activity_data.results:
|
||||
pr = result.primary_result
|
||||
if pr.type == "entity":
|
||||
results.append(
|
||||
{
|
||||
"title": pr.title,
|
||||
"permalink": pr.permalink,
|
||||
"file_path": pr.file_path,
|
||||
"created_at": str(pr.created_at) if pr.created_at else None,
|
||||
}
|
||||
)
|
||||
return results
|
||||
|
||||
|
||||
@tool_app.command()
|
||||
def write_note(
|
||||
title: Annotated[str, typer.Option(help="The title of the note")],
|
||||
@@ -50,6 +173,11 @@ def write_note(
|
||||
tags: Annotated[
|
||||
Optional[List[str]], typer.Option(help="A list of tags to apply to the note")
|
||||
] = None,
|
||||
format: str = typer.Option("text", "--format", help="Output format: text or json"),
|
||||
local: bool = typer.Option(
|
||||
False, "--local", help="Force local API routing (ignore cloud mode)"
|
||||
),
|
||||
cloud: bool = typer.Option(False, "--cloud", help="Force cloud API routing"),
|
||||
):
|
||||
"""Create or update a markdown note. Content can be provided as an argument or read from stdin.
|
||||
|
||||
@@ -57,6 +185,9 @@ def write_note(
|
||||
1. Using the --content parameter
|
||||
2. Piping content through stdin (if --content is not provided)
|
||||
|
||||
Use --local to force local routing when cloud mode is enabled.
|
||||
Use --cloud to force cloud routing when cloud mode is disabled.
|
||||
|
||||
Examples:
|
||||
|
||||
# Using content parameter
|
||||
@@ -77,8 +208,13 @@ def write_note(
|
||||
|
||||
# Reading from a file
|
||||
cat document.md | basic-memory tools write-note --title "Document" --folder "docs"
|
||||
|
||||
# Force local routing in cloud mode
|
||||
basic-memory tools write-note --title "My Note" --folder "notes" --content "..." --local
|
||||
"""
|
||||
try:
|
||||
validate_routing_flags(local, cloud)
|
||||
|
||||
# If content is not provided, read from stdin
|
||||
if content is None:
|
||||
# Check if we're getting data from a pipe or redirect
|
||||
@@ -109,8 +245,23 @@ def write_note(
|
||||
# use the project name, or the default from the config
|
||||
project_name = project_name or config_manager.default_project
|
||||
|
||||
note = run_with_cleanup(mcp_write_note.fn(title, content, folder, project_name, tags))
|
||||
rprint(note)
|
||||
# content is validated non-None above (stdin or --content)
|
||||
assert content is not None
|
||||
|
||||
with force_routing(local=local, cloud=cloud):
|
||||
if format == "json":
|
||||
result = run_with_cleanup(
|
||||
_write_note_json(title, content, folder, project_name, tags)
|
||||
)
|
||||
print(json.dumps(result, indent=2, ensure_ascii=True, default=str))
|
||||
else:
|
||||
note = run_with_cleanup(
|
||||
mcp_write_note.fn(title, content, folder, project_name, tags)
|
||||
)
|
||||
rprint(note)
|
||||
except ValueError as e:
|
||||
typer.echo(f"Error: {e}", err=True)
|
||||
raise typer.Exit(1)
|
||||
except Exception as e: # pragma: no cover
|
||||
if not isinstance(e, typer.Exit):
|
||||
typer.echo(f"Error during write_note: {e}", err=True)
|
||||
@@ -129,24 +280,44 @@ def read_note(
|
||||
] = None,
|
||||
page: int = 1,
|
||||
page_size: int = 10,
|
||||
format: str = typer.Option("text", "--format", help="Output format: text or json"),
|
||||
local: bool = typer.Option(
|
||||
False, "--local", help="Force local API routing (ignore cloud mode)"
|
||||
),
|
||||
cloud: bool = typer.Option(False, "--cloud", help="Force cloud API routing"),
|
||||
):
|
||||
"""Read a markdown note from the knowledge base."""
|
||||
|
||||
# look for the project in the config
|
||||
config_manager = ConfigManager()
|
||||
project_name = None
|
||||
if project is not None:
|
||||
project_name, _ = config_manager.get_project(project)
|
||||
if not project_name:
|
||||
typer.echo(f"No project found named: {project}", err=True)
|
||||
raise typer.Exit(1)
|
||||
|
||||
# use the project name, or the default from the config
|
||||
project_name = project_name or config_manager.default_project
|
||||
"""Read a markdown note from the knowledge base.
|
||||
|
||||
Use --local to force local routing when cloud mode is enabled.
|
||||
Use --cloud to force cloud routing when cloud mode is disabled.
|
||||
"""
|
||||
try:
|
||||
note = run_with_cleanup(mcp_read_note.fn(identifier, project_name, page, page_size))
|
||||
rprint(note)
|
||||
validate_routing_flags(local, cloud)
|
||||
|
||||
# look for the project in the config
|
||||
config_manager = ConfigManager()
|
||||
project_name = None
|
||||
if project is not None:
|
||||
project_name, _ = config_manager.get_project(project)
|
||||
if not project_name:
|
||||
typer.echo(f"No project found named: {project}", err=True)
|
||||
raise typer.Exit(1)
|
||||
|
||||
# use the project name, or the default from the config
|
||||
project_name = project_name or config_manager.default_project
|
||||
|
||||
with force_routing(local=local, cloud=cloud):
|
||||
if format == "json":
|
||||
result = run_with_cleanup(
|
||||
_read_note_json(identifier, project_name, page, page_size)
|
||||
)
|
||||
print(json.dumps(result, indent=2, ensure_ascii=True, default=str))
|
||||
else:
|
||||
note = run_with_cleanup(mcp_read_note.fn(identifier, project_name, page, page_size))
|
||||
rprint(note)
|
||||
except ValueError as e:
|
||||
typer.echo(f"Error: {e}", err=True)
|
||||
raise typer.Exit(1)
|
||||
except Exception as e: # pragma: no cover
|
||||
if not isinstance(e, typer.Exit):
|
||||
typer.echo(f"Error during read_note: {e}", err=True)
|
||||
@@ -166,38 +337,49 @@ def build_context(
|
||||
page: int = 1,
|
||||
page_size: int = 10,
|
||||
max_related: int = 10,
|
||||
format: str = typer.Option("json", "--format", help="Output format: text or json"),
|
||||
local: bool = typer.Option(
|
||||
False, "--local", help="Force local API routing (ignore cloud mode)"
|
||||
),
|
||||
cloud: bool = typer.Option(False, "--cloud", help="Force cloud API routing"),
|
||||
):
|
||||
"""Get context needed to continue a discussion."""
|
||||
|
||||
# look for the project in the config
|
||||
config_manager = ConfigManager()
|
||||
project_name = None
|
||||
if project is not None:
|
||||
project_name, _ = config_manager.get_project(project)
|
||||
if not project_name:
|
||||
typer.echo(f"No project found named: {project}", err=True)
|
||||
raise typer.Exit(1)
|
||||
|
||||
# use the project name, or the default from the config
|
||||
project_name = project_name or config_manager.default_project
|
||||
"""Get context needed to continue a discussion.
|
||||
|
||||
Use --local to force local routing when cloud mode is enabled.
|
||||
Use --cloud to force cloud routing when cloud mode is disabled.
|
||||
"""
|
||||
try:
|
||||
context = run_with_cleanup(
|
||||
mcp_build_context.fn(
|
||||
project=project_name,
|
||||
url=url,
|
||||
depth=depth,
|
||||
timeframe=timeframe,
|
||||
page=page,
|
||||
page_size=page_size,
|
||||
max_related=max_related,
|
||||
)
|
||||
)
|
||||
# Use json module for more controlled serialization
|
||||
import json
|
||||
validate_routing_flags(local, cloud)
|
||||
|
||||
# look for the project in the config
|
||||
config_manager = ConfigManager()
|
||||
project_name = None
|
||||
if project is not None:
|
||||
project_name, _ = config_manager.get_project(project)
|
||||
if not project_name:
|
||||
typer.echo(f"No project found named: {project}", err=True)
|
||||
raise typer.Exit(1)
|
||||
|
||||
# use the project name, or the default from the config
|
||||
project_name = project_name or config_manager.default_project
|
||||
|
||||
with force_routing(local=local, cloud=cloud):
|
||||
context = run_with_cleanup(
|
||||
mcp_build_context.fn(
|
||||
project=project_name,
|
||||
url=url,
|
||||
depth=depth,
|
||||
timeframe=timeframe,
|
||||
page=page,
|
||||
page_size=page_size,
|
||||
max_related=max_related,
|
||||
)
|
||||
)
|
||||
context_dict = context.model_dump(exclude_none=True)
|
||||
print(json.dumps(context_dict, indent=2, ensure_ascii=True, default=str))
|
||||
except ValueError as e:
|
||||
typer.echo(f"Error: {e}", err=True)
|
||||
raise typer.Exit(1)
|
||||
except Exception as e: # pragma: no cover
|
||||
if not isinstance(e, typer.Exit):
|
||||
typer.echo(f"Error during build_context: {e}", err=True)
|
||||
@@ -208,20 +390,60 @@ def build_context(
|
||||
@tool_app.command()
|
||||
def recent_activity(
|
||||
type: Annotated[Optional[List[SearchItemType]], typer.Option()] = None,
|
||||
project: Annotated[
|
||||
Optional[str],
|
||||
typer.Option(help="The project to use. If not provided, the default project will be used."),
|
||||
] = None,
|
||||
depth: Optional[int] = 1,
|
||||
timeframe: Optional[TimeFrame] = "7d",
|
||||
page: int = typer.Option(1, "--page", help="Page number for pagination (JSON format)"),
|
||||
page_size: int = typer.Option(
|
||||
50, "--page-size", help="Number of results per page (JSON format)"
|
||||
),
|
||||
format: str = typer.Option("text", "--format", help="Output format: text or json"),
|
||||
local: bool = typer.Option(
|
||||
False, "--local", help="Force local API routing (ignore cloud mode)"
|
||||
),
|
||||
cloud: bool = typer.Option(False, "--cloud", help="Force cloud API routing"),
|
||||
):
|
||||
"""Get recent activity across the knowledge base."""
|
||||
"""Get recent activity across the knowledge base.
|
||||
|
||||
Use --local to force local routing when cloud mode is enabled.
|
||||
Use --cloud to force cloud routing when cloud mode is disabled.
|
||||
"""
|
||||
try:
|
||||
result = run_with_cleanup(
|
||||
mcp_recent_activity.fn(
|
||||
type=type, # pyright: ignore [reportArgumentType]
|
||||
depth=depth,
|
||||
timeframe=timeframe,
|
||||
)
|
||||
)
|
||||
# The tool now returns a formatted string directly
|
||||
print(result)
|
||||
validate_routing_flags(local, cloud)
|
||||
|
||||
# Resolve project from config for JSON mode
|
||||
config_manager = ConfigManager()
|
||||
project_name = None
|
||||
if project is not None:
|
||||
project_name, _ = config_manager.get_project(project)
|
||||
if not project_name:
|
||||
typer.echo(f"No project found named: {project}", err=True)
|
||||
raise typer.Exit(1)
|
||||
project_name = project_name or config_manager.default_project
|
||||
|
||||
with force_routing(local=local, cloud=cloud):
|
||||
if format == "json":
|
||||
result = run_with_cleanup(
|
||||
_recent_activity_json(type, depth, timeframe, project_name, page, page_size)
|
||||
)
|
||||
print(json.dumps(result, indent=2, ensure_ascii=True, default=str))
|
||||
else:
|
||||
result = run_with_cleanup(
|
||||
mcp_recent_activity.fn(
|
||||
type=type, # pyright: ignore [reportArgumentType]
|
||||
depth=depth,
|
||||
timeframe=timeframe,
|
||||
project=project_name,
|
||||
)
|
||||
)
|
||||
# The tool returns a formatted string directly
|
||||
print(result)
|
||||
except ValueError as e:
|
||||
typer.echo(f"Error: {e}", err=True)
|
||||
raise typer.Exit(1)
|
||||
except Exception as e: # pragma: no cover
|
||||
if not isinstance(e, typer.Exit):
|
||||
typer.echo(f"Error during recent_activity: {e}", err=True)
|
||||
@@ -231,7 +453,10 @@ def recent_activity(
|
||||
|
||||
@tool_app.command("search-notes")
|
||||
def search_notes(
|
||||
query: str,
|
||||
query: Annotated[
|
||||
Optional[str],
|
||||
typer.Argument(help="Search query string (optional when using metadata filters)"),
|
||||
] = "",
|
||||
permalink: Annotated[bool, typer.Option("--permalink", help="Search permalink values")] = False,
|
||||
title: Annotated[bool, typer.Option("--title", help="Search title values")] = False,
|
||||
project: Annotated[
|
||||
@@ -244,28 +469,53 @@ def search_notes(
|
||||
Optional[str],
|
||||
typer.Option("--after_date", help="Search results after date, eg. '2d', '1 week'"),
|
||||
] = None,
|
||||
tags: Annotated[
|
||||
Optional[List[str]],
|
||||
typer.Option("--tag", help="Filter by frontmatter tag (repeatable)"),
|
||||
] = None,
|
||||
status: Annotated[
|
||||
Optional[str],
|
||||
typer.Option("--status", help="Filter by frontmatter status"),
|
||||
] = None,
|
||||
note_types: Annotated[
|
||||
Optional[List[str]],
|
||||
typer.Option("--type", help="Filter by frontmatter type (repeatable)"),
|
||||
] = None,
|
||||
meta: Annotated[
|
||||
Optional[List[str]],
|
||||
typer.Option("--meta", help="Filter by frontmatter key=value (repeatable)"),
|
||||
] = None,
|
||||
filter_json: Annotated[
|
||||
Optional[str],
|
||||
typer.Option("--filter", help="JSON metadata filter (advanced)"),
|
||||
] = None,
|
||||
page: int = 1,
|
||||
page_size: int = 10,
|
||||
local: bool = typer.Option(
|
||||
False, "--local", help="Force local API routing (ignore cloud mode)"
|
||||
),
|
||||
cloud: bool = typer.Option(False, "--cloud", help="Force cloud API routing"),
|
||||
):
|
||||
"""Search across all content in the knowledge base."""
|
||||
|
||||
# look for the project in the config
|
||||
config_manager = ConfigManager()
|
||||
project_name = None
|
||||
if project is not None:
|
||||
project_name, _ = config_manager.get_project(project)
|
||||
if not project_name:
|
||||
typer.echo(f"No project found named: {project}", err=True)
|
||||
raise typer.Exit(1)
|
||||
|
||||
# use the project name, or the default from the config
|
||||
project_name = project_name or config_manager.default_project
|
||||
|
||||
if permalink and title: # pragma: no cover
|
||||
print("Cannot search both permalink and title")
|
||||
raise typer.Abort()
|
||||
"""Search across all content in the knowledge base.
|
||||
|
||||
Use --local to force local routing when cloud mode is enabled.
|
||||
Use --cloud to force cloud routing when cloud mode is disabled.
|
||||
"""
|
||||
try:
|
||||
validate_routing_flags(local, cloud)
|
||||
|
||||
# look for the project in the config
|
||||
config_manager = ConfigManager()
|
||||
project_name = None
|
||||
if project is not None:
|
||||
project_name, _ = config_manager.get_project(project)
|
||||
if not project_name:
|
||||
typer.echo(f"No project found named: {project}", err=True)
|
||||
raise typer.Exit(1)
|
||||
|
||||
# use the project name, or the default from the config
|
||||
project_name = project_name or config_manager.default_project
|
||||
|
||||
if permalink and title: # pragma: no cover
|
||||
typer.echo(
|
||||
"Use either --permalink or --title, not both. Exiting.",
|
||||
@@ -273,27 +523,64 @@ def search_notes(
|
||||
)
|
||||
raise typer.Exit(1)
|
||||
|
||||
# Build metadata filters from --filter and --meta
|
||||
metadata_filters = {}
|
||||
if filter_json:
|
||||
try:
|
||||
metadata_filters = json.loads(filter_json)
|
||||
if not isinstance(metadata_filters, dict):
|
||||
raise ValueError("Metadata filter JSON must be an object")
|
||||
except json.JSONDecodeError as e:
|
||||
typer.echo(f"Invalid JSON for --filter: {e}", err=True)
|
||||
raise typer.Exit(1)
|
||||
|
||||
if meta:
|
||||
for item in meta:
|
||||
if "=" not in item:
|
||||
typer.echo(
|
||||
f"Invalid --meta entry '{item}'. Use key=value format.",
|
||||
err=True,
|
||||
)
|
||||
raise typer.Exit(1)
|
||||
key, value = item.split("=", 1)
|
||||
key = key.strip()
|
||||
if not key:
|
||||
typer.echo(f"Invalid --meta entry '{item}'.", err=True)
|
||||
raise typer.Exit(1)
|
||||
metadata_filters[key] = value
|
||||
|
||||
if not metadata_filters:
|
||||
metadata_filters = None
|
||||
|
||||
# set search type
|
||||
search_type = ("permalink" if permalink else None,)
|
||||
search_type = ("permalink_match" if permalink and "*" in query else None,)
|
||||
search_type = ("title" if title else None,)
|
||||
search_type = "text" if search_type is None else search_type
|
||||
search_type = "text"
|
||||
if permalink:
|
||||
search_type = "permalink"
|
||||
if query and "*" in query:
|
||||
search_type = "permalink"
|
||||
if title:
|
||||
search_type = "title"
|
||||
|
||||
results = run_with_cleanup(
|
||||
mcp_search.fn(
|
||||
query,
|
||||
project_name,
|
||||
search_type=search_type,
|
||||
page=page,
|
||||
after_date=after_date,
|
||||
page_size=page_size,
|
||||
with force_routing(local=local, cloud=cloud):
|
||||
results = run_with_cleanup(
|
||||
mcp_search.fn(
|
||||
query or "",
|
||||
project_name,
|
||||
search_type=search_type,
|
||||
page=page,
|
||||
after_date=after_date,
|
||||
page_size=page_size,
|
||||
types=note_types,
|
||||
metadata_filters=metadata_filters,
|
||||
tags=tags,
|
||||
status=status,
|
||||
)
|
||||
)
|
||||
)
|
||||
# Use json module for more controlled serialization
|
||||
import json
|
||||
|
||||
results_dict = results.model_dump(exclude_none=True)
|
||||
print(json.dumps(results_dict, indent=2, ensure_ascii=True, default=str))
|
||||
except ValueError as e:
|
||||
typer.echo(f"Error: {e}", err=True)
|
||||
raise typer.Exit(1)
|
||||
except Exception as e: # pragma: no cover
|
||||
if not isinstance(e, typer.Exit):
|
||||
logger.exception("Error during search", e)
|
||||
@@ -308,12 +595,28 @@ def continue_conversation(
|
||||
timeframe: Annotated[
|
||||
Optional[str], typer.Option(help="How far back to look for activity")
|
||||
] = None,
|
||||
local: bool = typer.Option(
|
||||
False, "--local", help="Force local API routing (ignore cloud mode)"
|
||||
),
|
||||
cloud: bool = typer.Option(False, "--cloud", help="Force cloud API routing"),
|
||||
):
|
||||
"""Prompt to continue a previous conversation or work session."""
|
||||
"""Prompt to continue a previous conversation or work session.
|
||||
|
||||
Use --local to force local routing when cloud mode is enabled.
|
||||
Use --cloud to force cloud routing when cloud mode is disabled.
|
||||
"""
|
||||
try:
|
||||
# Prompt functions return formatted strings directly
|
||||
session = run_with_cleanup(mcp_continue_conversation.fn(topic=topic, timeframe=timeframe)) # type: ignore
|
||||
validate_routing_flags(local, cloud)
|
||||
|
||||
with force_routing(local=local, cloud=cloud):
|
||||
# Prompt functions return formatted strings directly
|
||||
session = run_with_cleanup(
|
||||
mcp_continue_conversation.fn(topic=topic, timeframe=timeframe) # type: ignore[arg-type]
|
||||
)
|
||||
rprint(session)
|
||||
except ValueError as e:
|
||||
typer.echo(f"Error: {e}", err=True)
|
||||
raise typer.Exit(1)
|
||||
except Exception as e: # pragma: no cover
|
||||
if not isinstance(e, typer.Exit):
|
||||
logger.exception("Error continuing conversation", e)
|
||||
|
||||
@@ -6,6 +6,7 @@ from basic_memory.cli.app import app # pragma: no cover
|
||||
from basic_memory.cli.commands import ( # noqa: F401 # pragma: no cover
|
||||
cloud,
|
||||
db,
|
||||
doctor,
|
||||
import_chatgpt,
|
||||
import_claude_conversations,
|
||||
import_claude_projects,
|
||||
|
||||
@@ -91,6 +91,8 @@ from basic_memory.deps.services import (
|
||||
FileServiceV2Dep,
|
||||
get_file_service_v2_external,
|
||||
FileServiceV2ExternalDep,
|
||||
get_task_scheduler,
|
||||
TaskSchedulerDep,
|
||||
get_search_service,
|
||||
SearchServiceDep,
|
||||
get_search_service_v2,
|
||||
@@ -227,6 +229,8 @@ __all__ = [
|
||||
"FileServiceV2Dep",
|
||||
"get_file_service_v2_external",
|
||||
"FileServiceV2ExternalDep",
|
||||
"get_task_scheduler",
|
||||
"TaskSchedulerDep",
|
||||
"get_search_service",
|
||||
"SearchServiceDep",
|
||||
"get_search_service_v2",
|
||||
|
||||
@@ -7,7 +7,8 @@ This module provides service-layer dependencies:
|
||||
- SyncService, ProjectService, DirectoryService
|
||||
"""
|
||||
|
||||
from typing import Annotated
|
||||
import asyncio
|
||||
from typing import Annotated, Any, Callable, Coroutine, Mapping, Protocol
|
||||
|
||||
from fastapi import Depends
|
||||
from loguru import logger
|
||||
@@ -43,7 +44,6 @@ from basic_memory.services.link_resolver import LinkResolver
|
||||
from basic_memory.services.search_service import SearchService
|
||||
from basic_memory.sync import SyncService
|
||||
|
||||
|
||||
# --- Entity Parser ---
|
||||
|
||||
|
||||
@@ -430,6 +430,87 @@ async def get_sync_service_v2_external(
|
||||
SyncServiceV2ExternalDep = Annotated[SyncService, Depends(get_sync_service_v2_external)]
|
||||
|
||||
|
||||
# --- Background Task Scheduler ---
|
||||
|
||||
|
||||
class TaskScheduler(Protocol):
|
||||
def schedule(self, task_name: str, **payload: Any) -> None:
|
||||
"""Schedule a background task by name."""
|
||||
|
||||
|
||||
def _log_task_failure(completed: asyncio.Task) -> None:
|
||||
try:
|
||||
completed.result()
|
||||
except Exception as exc: # pragma: no cover
|
||||
logger.exception("Background task failed", error=str(exc))
|
||||
|
||||
|
||||
class LocalTaskScheduler:
|
||||
"""Default scheduler that runs tasks in-process via asyncio.create_task."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
handlers: Mapping[str, Callable[..., Coroutine[Any, Any, None]]],
|
||||
) -> None:
|
||||
self._handlers = handlers
|
||||
|
||||
def schedule(self, task_name: str, **payload: Any) -> None:
|
||||
handler = self._handlers.get(task_name)
|
||||
# Trigger: task name is not registered
|
||||
# Why: avoid silently dropping background work
|
||||
# Outcome: fail fast to surface misconfiguration
|
||||
if not handler:
|
||||
raise ValueError(f"Unknown task name: {task_name}")
|
||||
task = asyncio.create_task(handler(**payload))
|
||||
task.add_done_callback(_log_task_failure)
|
||||
|
||||
|
||||
async def get_task_scheduler(
|
||||
entity_service: EntityServiceV2ExternalDep,
|
||||
sync_service: SyncServiceV2ExternalDep,
|
||||
search_service: SearchServiceV2ExternalDep,
|
||||
project_config: ProjectConfigV2ExternalDep,
|
||||
) -> TaskScheduler:
|
||||
"""Create a scheduler that maps task specs to coroutines."""
|
||||
|
||||
async def _reindex_entity(
|
||||
entity_id: int,
|
||||
resolve_relations: bool = False,
|
||||
**_: Any,
|
||||
) -> None:
|
||||
await entity_service.reindex_entity(entity_id)
|
||||
# Trigger: caller requests relation resolution
|
||||
# Why: resolve forward references created before the entity existed
|
||||
# Outcome: updates unresolved relations pointing to this entity
|
||||
if resolve_relations:
|
||||
await sync_service.resolve_relations(entity_id=entity_id)
|
||||
|
||||
async def _resolve_relations(entity_id: int, **_: Any) -> None:
|
||||
await sync_service.resolve_relations(entity_id=entity_id)
|
||||
|
||||
async def _sync_project(force_full: bool = False, **_: Any) -> None:
|
||||
await sync_service.sync(
|
||||
project_config.home,
|
||||
project_config.name,
|
||||
force_full=force_full,
|
||||
)
|
||||
|
||||
async def _reindex_project(**_: Any) -> None:
|
||||
await search_service.reindex_all()
|
||||
|
||||
return LocalTaskScheduler(
|
||||
{
|
||||
"reindex_entity": _reindex_entity,
|
||||
"resolve_relations": _resolve_relations,
|
||||
"sync_project": _sync_project,
|
||||
"reindex_project": _reindex_project,
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
TaskSchedulerDep = Annotated[TaskScheduler, Depends(get_task_scheduler)]
|
||||
|
||||
|
||||
# --- Project Service ---
|
||||
|
||||
|
||||
|
||||
@@ -450,16 +450,16 @@ def sanitize_for_filename(text: str, replacement: str = "-") -> str:
|
||||
return text.strip(replacement)
|
||||
|
||||
|
||||
def sanitize_for_folder(folder: str) -> str:
|
||||
def sanitize_for_directory(directory: str) -> str:
|
||||
"""
|
||||
Sanitize folder path to be safe for use in file system paths.
|
||||
Sanitize directory path to be safe for use in file system paths.
|
||||
Removes leading/trailing whitespace, compresses multiple slashes,
|
||||
and removes special characters except for /, -, and _.
|
||||
"""
|
||||
if not folder:
|
||||
if not directory:
|
||||
return ""
|
||||
|
||||
sanitized = folder.strip()
|
||||
sanitized = directory.strip()
|
||||
|
||||
if sanitized.startswith("./"):
|
||||
sanitized = sanitized[2:]
|
||||
|
||||
@@ -241,7 +241,9 @@ class EntityParser:
|
||||
f"Failed to parse YAML frontmatter in {file_path}: {e}. "
|
||||
f"Treating file as plain markdown without frontmatter."
|
||||
)
|
||||
post = frontmatter.Post(content, metadata={})
|
||||
# Use Post(content) not Post(content, metadata={})
|
||||
# The latter creates {"metadata": {}} in the metadata dict (issue #528)
|
||||
post = frontmatter.Post(content)
|
||||
|
||||
# Normalize frontmatter values
|
||||
metadata = normalize_frontmatter_metadata(post.metadata)
|
||||
|
||||
@@ -9,6 +9,7 @@ from frontmatter import Post
|
||||
|
||||
from basic_memory.file_utils import has_frontmatter, remove_frontmatter, parse_frontmatter
|
||||
from basic_memory.markdown import EntityMarkdown
|
||||
from basic_memory.markdown.entity_parser import normalize_frontmatter_metadata
|
||||
from basic_memory.models import Entity
|
||||
from basic_memory.models import Observation as ObservationModel
|
||||
|
||||
@@ -58,9 +59,9 @@ def entity_model_from_markdown(
|
||||
model.created_at = markdown.created
|
||||
model.updated_at = markdown.modified
|
||||
|
||||
# Handle metadata - ensure all values are strings and filter None
|
||||
metadata = markdown.frontmatter.metadata or {}
|
||||
model.entity_metadata = {k: str(v) for k, v in metadata.items() if v is not None}
|
||||
# Handle metadata - normalize values and filter None (preserve structured data)
|
||||
metadata = normalize_frontmatter_metadata(markdown.frontmatter.metadata or {})
|
||||
model.entity_metadata = {k: v for k, v in metadata.items() if v is not None}
|
||||
|
||||
# Get project_id from entity if not provided
|
||||
obs_project_id = project_id or (model.project_id if hasattr(model, "project_id") else None)
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
import os
|
||||
from contextlib import asynccontextmanager, AbstractAsyncContextManager
|
||||
from typing import AsyncIterator, Callable, Optional
|
||||
|
||||
@@ -8,6 +9,19 @@ from basic_memory.api.app import app as fastapi_app
|
||||
from basic_memory.config import ConfigManager
|
||||
|
||||
|
||||
def _force_local_mode() -> bool:
|
||||
"""Check if local mode is forced via environment variable.
|
||||
|
||||
This allows commands like `bm mcp` to force local routing even when
|
||||
cloud_mode_enabled is True in config. The local MCP server should
|
||||
always talk to the local API, not the cloud proxy.
|
||||
|
||||
Returns:
|
||||
True if BASIC_MEMORY_FORCE_LOCAL is set to a truthy value
|
||||
"""
|
||||
return os.environ.get("BASIC_MEMORY_FORCE_LOCAL", "").lower() in ("true", "1", "yes")
|
||||
|
||||
|
||||
# Optional factory override for dependency injection
|
||||
_client_factory: Optional[Callable[[], AbstractAsyncContextManager[AsyncClient]]] = None
|
||||
|
||||
@@ -71,7 +85,17 @@ async def get_client() -> AsyncIterator[AsyncClient]:
|
||||
pool=30.0, # 30 seconds for connection pool
|
||||
)
|
||||
|
||||
if config.cloud_mode_enabled:
|
||||
# Trigger: BASIC_MEMORY_FORCE_LOCAL env var is set
|
||||
# Why: allows local MCP server and CLI commands to route locally
|
||||
# even when cloud_mode_enabled is True
|
||||
# Outcome: uses ASGI transport for in-process local API calls
|
||||
if _force_local_mode():
|
||||
logger.info("Force local mode enabled - using ASGI client for local Basic Memory API")
|
||||
async with AsyncClient(
|
||||
transport=ASGITransport(app=fastapi_app), base_url="http://test", timeout=timeout
|
||||
) as client:
|
||||
yield client
|
||||
elif config.cloud_mode_enabled:
|
||||
# CLI cloud mode: inject auth when creating client
|
||||
from basic_memory.cli.auth import CLIAuth
|
||||
|
||||
@@ -126,7 +150,13 @@ def create_client() -> AsyncClient:
|
||||
pool=30.0, # 30 seconds for connection pool
|
||||
)
|
||||
|
||||
if config.cloud_mode_enabled:
|
||||
# Check force local first (for local MCP server and CLI --local flag)
|
||||
if _force_local_mode():
|
||||
logger.info("Force local mode enabled - using ASGI client for local Basic Memory API")
|
||||
return AsyncClient(
|
||||
transport=ASGITransport(app=fastapi_app), base_url="http://test", timeout=timeout
|
||||
)
|
||||
elif config.cloud_mode_enabled:
|
||||
# Use HTTP transport to proxy endpoint
|
||||
proxy_base_url = f"{config.cloud_host}/proxy"
|
||||
logger.info(f"Creating HTTP client for proxy at: {proxy_base_url}")
|
||||
|
||||
@@ -8,7 +8,12 @@ from typing import Any
|
||||
from httpx import AsyncClient
|
||||
|
||||
from basic_memory.mcp.tools.utils import call_get, call_post, call_put, call_patch, call_delete
|
||||
from basic_memory.schemas.response import EntityResponse, DeleteEntitiesResponse
|
||||
from basic_memory.schemas.response import (
|
||||
EntityResponse,
|
||||
DeleteEntitiesResponse,
|
||||
DirectoryMoveResult,
|
||||
DirectoryDeleteResult,
|
||||
)
|
||||
|
||||
|
||||
class KnowledgeClient:
|
||||
@@ -38,7 +43,9 @@ class KnowledgeClient:
|
||||
|
||||
# --- Entity CRUD Operations ---
|
||||
|
||||
async def create_entity(self, entity_data: dict[str, Any]) -> EntityResponse:
|
||||
async def create_entity(
|
||||
self, entity_data: dict[str, Any], *, fast: bool | None = None
|
||||
) -> EntityResponse:
|
||||
"""Create a new entity.
|
||||
|
||||
Args:
|
||||
@@ -50,14 +57,22 @@ class KnowledgeClient:
|
||||
Raises:
|
||||
ToolError: If the request fails
|
||||
"""
|
||||
params = {"fast": fast} if fast is not None else None
|
||||
response = await call_post(
|
||||
self.http_client,
|
||||
f"{self._base_path}/entities",
|
||||
json=entity_data,
|
||||
params=params,
|
||||
)
|
||||
return EntityResponse.model_validate(response.json())
|
||||
|
||||
async def update_entity(self, entity_id: str, entity_data: dict[str, Any]) -> EntityResponse:
|
||||
async def update_entity(
|
||||
self,
|
||||
entity_id: str,
|
||||
entity_data: dict[str, Any],
|
||||
*,
|
||||
fast: bool | None = None,
|
||||
) -> EntityResponse:
|
||||
"""Update an existing entity (full replacement).
|
||||
|
||||
Args:
|
||||
@@ -70,10 +85,12 @@ class KnowledgeClient:
|
||||
Raises:
|
||||
ToolError: If the request fails
|
||||
"""
|
||||
params = {"fast": fast} if fast is not None else None
|
||||
response = await call_put(
|
||||
self.http_client,
|
||||
f"{self._base_path}/entities/{entity_id}",
|
||||
json=entity_data,
|
||||
params=params,
|
||||
)
|
||||
return EntityResponse.model_validate(response.json())
|
||||
|
||||
@@ -95,7 +112,13 @@ class KnowledgeClient:
|
||||
)
|
||||
return EntityResponse.model_validate(response.json())
|
||||
|
||||
async def patch_entity(self, entity_id: str, patch_data: dict[str, Any]) -> EntityResponse:
|
||||
async def patch_entity(
|
||||
self,
|
||||
entity_id: str,
|
||||
patch_data: dict[str, Any],
|
||||
*,
|
||||
fast: bool | None = None,
|
||||
) -> EntityResponse:
|
||||
"""Partially update an entity.
|
||||
|
||||
Args:
|
||||
@@ -108,10 +131,12 @@ class KnowledgeClient:
|
||||
Raises:
|
||||
ToolError: If the request fails
|
||||
"""
|
||||
params = {"fast": fast} if fast is not None else None
|
||||
response = await call_patch(
|
||||
self.http_client,
|
||||
f"{self._base_path}/entities/{entity_id}",
|
||||
json=patch_data,
|
||||
params=params,
|
||||
)
|
||||
return EntityResponse.model_validate(response.json())
|
||||
|
||||
@@ -153,6 +178,50 @@ class KnowledgeClient:
|
||||
)
|
||||
return EntityResponse.model_validate(response.json())
|
||||
|
||||
async def move_directory(
|
||||
self, source_directory: str, destination_directory: str
|
||||
) -> DirectoryMoveResult:
|
||||
"""Move all entities in a directory to a new location.
|
||||
|
||||
Args:
|
||||
source_directory: Source directory path (relative to project root)
|
||||
destination_directory: Destination directory path (relative to project root)
|
||||
|
||||
Returns:
|
||||
DirectoryMoveResult with counts and details of moved files
|
||||
|
||||
Raises:
|
||||
ToolError: If the request fails
|
||||
"""
|
||||
response = await call_post(
|
||||
self.http_client,
|
||||
f"{self._base_path}/move-directory",
|
||||
json={
|
||||
"source_directory": source_directory,
|
||||
"destination_directory": destination_directory,
|
||||
},
|
||||
)
|
||||
return DirectoryMoveResult.model_validate(response.json())
|
||||
|
||||
async def delete_directory(self, directory: str) -> DirectoryDeleteResult:
|
||||
"""Delete all entities in a directory.
|
||||
|
||||
Args:
|
||||
directory: Directory path to delete (relative to project root)
|
||||
|
||||
Returns:
|
||||
DirectoryDeleteResult with counts and details of deleted files
|
||||
|
||||
Raises:
|
||||
ToolError: If the request fails
|
||||
"""
|
||||
response = await call_post(
|
||||
self.http_client,
|
||||
f"{self._base_path}/delete-directory",
|
||||
json={"directory": directory},
|
||||
)
|
||||
return DirectoryDeleteResult.model_validate(response.json())
|
||||
|
||||
# --- Resolution ---
|
||||
|
||||
async def resolve_entity(self, identifier: str) -> str:
|
||||
|
||||
@@ -47,7 +47,7 @@ class ProjectClient:
|
||||
"""
|
||||
response = await call_get(
|
||||
self.http_client,
|
||||
"/projects/projects",
|
||||
"/v2/projects/",
|
||||
)
|
||||
return ProjectList.model_validate(response.json())
|
||||
|
||||
@@ -65,7 +65,7 @@ class ProjectClient:
|
||||
"""
|
||||
response = await call_post(
|
||||
self.http_client,
|
||||
"/projects/projects",
|
||||
"/v2/projects/",
|
||||
json=project_data,
|
||||
)
|
||||
return ProjectStatusResponse.model_validate(response.json())
|
||||
|
||||
@@ -19,7 +19,7 @@ from fastmcp import Context
|
||||
from basic_memory.config import ConfigManager
|
||||
from basic_memory.project_resolver import ProjectResolver
|
||||
from basic_memory.schemas.project_info import ProjectItem, ProjectList
|
||||
from basic_memory.utils import generate_permalink
|
||||
from basic_memory.schemas.v2 import ProjectResolveResponse
|
||||
|
||||
|
||||
async def resolve_project_parameter(
|
||||
@@ -78,7 +78,7 @@ async def get_project_names(client: AsyncClient, headers: HeaderTypes | None = N
|
||||
# Deferred import to avoid circular dependency with tools
|
||||
from basic_memory.mcp.tools.utils import call_get
|
||||
|
||||
response = await call_get(client, "/projects/projects", headers=headers)
|
||||
response = await call_get(client, "/v2/projects/", headers=headers)
|
||||
project_list = ProjectList.model_validate(response.json())
|
||||
return [project.name for project in project_list.projects]
|
||||
|
||||
@@ -104,7 +104,7 @@ async def get_active_project(
|
||||
HTTPError: If project doesn't exist or is inaccessible
|
||||
"""
|
||||
# Deferred import to avoid circular dependency with tools
|
||||
from basic_memory.mcp.tools.utils import call_get
|
||||
from basic_memory.mcp.tools.utils import call_post
|
||||
|
||||
resolved_project = await resolve_project_parameter(project)
|
||||
if not resolved_project:
|
||||
@@ -126,9 +126,20 @@ async def get_active_project(
|
||||
|
||||
# Validate project exists by calling API
|
||||
logger.debug(f"Validating project: {project}")
|
||||
permalink = generate_permalink(project)
|
||||
response = await call_get(client, f"/{permalink}/project/item", headers=headers)
|
||||
active_project = ProjectItem.model_validate(response.json())
|
||||
response = await call_post(
|
||||
client,
|
||||
"/v2/projects/resolve",
|
||||
json={"identifier": project},
|
||||
headers=headers,
|
||||
)
|
||||
resolved = ProjectResolveResponse.model_validate(response.json())
|
||||
active_project = ProjectItem(
|
||||
id=resolved.project_id,
|
||||
external_id=resolved.external_id,
|
||||
name=resolved.name,
|
||||
path=resolved.path,
|
||||
is_default=resolved.is_default,
|
||||
)
|
||||
|
||||
# Cache in context if available
|
||||
if context:
|
||||
|
||||
@@ -9,11 +9,11 @@ from typing import Annotated, Optional
|
||||
from loguru import logger
|
||||
from pydantic import Field
|
||||
|
||||
from basic_memory.config import get_project_config
|
||||
from basic_memory.config import ConfigManager
|
||||
from basic_memory.mcp.async_client import get_client
|
||||
from basic_memory.mcp.project_context import get_active_project
|
||||
from basic_memory.mcp.server import mcp
|
||||
from basic_memory.mcp.tools.utils import call_post
|
||||
from basic_memory.schemas.base import TimeFrame
|
||||
from basic_memory.schemas.prompt import ContinueConversationRequest
|
||||
|
||||
|
||||
@@ -24,7 +24,7 @@ from basic_memory.schemas.prompt import ContinueConversationRequest
|
||||
async def continue_conversation(
|
||||
topic: Annotated[Optional[str], Field(description="Topic or keyword to search for")] = None,
|
||||
timeframe: Annotated[
|
||||
Optional[TimeFrame],
|
||||
Optional[str],
|
||||
Field(description="How far back to look for activity (e.g. '1d', '1 week')"),
|
||||
] = None,
|
||||
) -> str:
|
||||
@@ -43,17 +43,18 @@ async def continue_conversation(
|
||||
logger.info(f"Continuing session, topic: {topic}, timeframe: {timeframe}")
|
||||
|
||||
async with get_client() as client:
|
||||
config = ConfigManager().config
|
||||
active_project = await get_active_project(client, project=config.default_project)
|
||||
|
||||
# Create request model
|
||||
request = ContinueConversationRequest( # pyright: ignore [reportCallIssue]
|
||||
topic=topic, timeframe=timeframe
|
||||
)
|
||||
|
||||
project_url = get_project_config().project_url
|
||||
|
||||
# Call the prompt API endpoint
|
||||
response = await call_post(
|
||||
client,
|
||||
f"{project_url}/prompt/continue-conversation",
|
||||
f"/v2/projects/{active_project.external_id}/prompt/continue-conversation",
|
||||
json=request.model_dump(exclude_none=True),
|
||||
)
|
||||
|
||||
|
||||
@@ -3,17 +3,14 @@
|
||||
These prompts help users see what has changed in their knowledge base recently.
|
||||
"""
|
||||
|
||||
from textwrap import dedent
|
||||
from typing import Annotated, Optional
|
||||
|
||||
from loguru import logger
|
||||
from pydantic import Field
|
||||
|
||||
from basic_memory.mcp.prompts.utils import format_prompt_context, PromptContext, PromptContextItem
|
||||
from basic_memory.mcp.server import mcp
|
||||
from basic_memory.mcp.tools.recent_activity import recent_activity
|
||||
from basic_memory.schemas.base import TimeFrame
|
||||
from basic_memory.schemas.memory import GraphContext, ProjectActivitySummary
|
||||
from basic_memory.schemas.search import SearchItemType
|
||||
|
||||
|
||||
@mcp.prompt(
|
||||
@@ -22,7 +19,7 @@ from basic_memory.schemas.search import SearchItemType
|
||||
)
|
||||
async def recent_activity_prompt(
|
||||
timeframe: Annotated[
|
||||
TimeFrame,
|
||||
str,
|
||||
Field(description="How far back to look for activity (e.g. '1d', '1 week')"),
|
||||
] = "7d",
|
||||
project: Annotated[
|
||||
@@ -45,144 +42,57 @@ async def recent_activity_prompt(
|
||||
Returns:
|
||||
Formatted summary of recent activity
|
||||
"""
|
||||
timeframe = timeframe or "7d"
|
||||
logger.info(f"Getting recent activity, timeframe: {timeframe}, project: {project}")
|
||||
|
||||
recent = await recent_activity.fn(
|
||||
project=project, timeframe=timeframe, type=[SearchItemType.ENTITY]
|
||||
)
|
||||
# Call the tool function - it returns a well-formatted string
|
||||
activity_summary = await recent_activity.fn(project=project, timeframe=timeframe)
|
||||
|
||||
# Extract primary results from the hierarchical structure
|
||||
primary_results = []
|
||||
related_results = []
|
||||
# Build the prompt response
|
||||
# The tool already returns formatted markdown, so we use it directly
|
||||
# and add prompt-specific guidance
|
||||
target = project if project else "all projects"
|
||||
|
||||
if isinstance(recent, ProjectActivitySummary):
|
||||
# Discovery mode - extract results from all projects
|
||||
for _, project_activity in recent.projects.items():
|
||||
if project_activity.activity.results:
|
||||
# Take up to 2 primary results per project
|
||||
for item in project_activity.activity.results[:2]:
|
||||
primary_results.append(item.primary_result)
|
||||
# Add up to 1 related result per primary item
|
||||
if item.related_results:
|
||||
related_results.extend(item.related_results[:1]) # pragma: no cover
|
||||
prompt_guidance = dedent(f"""
|
||||
# Recent Activity Context
|
||||
|
||||
# Limit total results for readability
|
||||
primary_results = primary_results[:8]
|
||||
related_results = related_results[:6]
|
||||
This is a memory retrieval session showing recent activity from {target}.
|
||||
|
||||
elif isinstance(recent, GraphContext):
|
||||
# Project-specific mode - use existing logic
|
||||
if recent.results:
|
||||
# Take up to 5 primary results
|
||||
for item in recent.results[:5]:
|
||||
primary_results.append(item.primary_result)
|
||||
# Add up to 2 related results per primary item
|
||||
if item.related_results:
|
||||
related_results.extend(item.related_results[:2]) # pragma: no cover
|
||||
{activity_summary}
|
||||
|
||||
# Set topic based on mode
|
||||
if project:
|
||||
topic = f"Recent Activity in {project} ({timeframe})"
|
||||
else:
|
||||
topic = f"Recent Activity Across All Projects ({timeframe})"
|
||||
---
|
||||
|
||||
prompt_context = format_prompt_context(
|
||||
PromptContext(
|
||||
topic=topic,
|
||||
timeframe=timeframe,
|
||||
results=[
|
||||
PromptContextItem(
|
||||
primary_results=primary_results,
|
||||
related_results=related_results[:10], # Limit total related results
|
||||
)
|
||||
],
|
||||
## Next Steps
|
||||
|
||||
Based on this activity, you can:
|
||||
|
||||
1. **Explore specific items** - Use `read_note("permalink")` to dive deeper into any item
|
||||
2. **Search for related content** - Use `search_notes("topic")` to find connected knowledge
|
||||
3. **Build context** - Use `build_context("memory://path")` to see relationships
|
||||
|
||||
## Capture Opportunity
|
||||
|
||||
If you notice patterns or insights from this activity, consider documenting them:
|
||||
|
||||
```python
|
||||
write_note(
|
||||
title="Activity Insights - {timeframe}",
|
||||
content='''
|
||||
# Activity Insights
|
||||
|
||||
## Patterns Observed
|
||||
- [trend] [Pattern you noticed in the activity]
|
||||
|
||||
## Key Developments
|
||||
- [insight] [Important development worth tracking]
|
||||
|
||||
## Relations
|
||||
- summarizes [[Recent Work]]
|
||||
''',
|
||||
folder="insights",
|
||||
project="{project or "default"}"
|
||||
)
|
||||
)
|
||||
```
|
||||
""")
|
||||
|
||||
# Add mode-specific suggestions
|
||||
first_title = "Recent Topic"
|
||||
if primary_results and len(primary_results) > 0:
|
||||
first_title = primary_results[0].title
|
||||
|
||||
if project:
|
||||
# Project-specific suggestions
|
||||
capture_suggestions = f"""
|
||||
## Opportunity to Capture Activity Summary
|
||||
|
||||
Consider creating a summary note of recent activity in {project}:
|
||||
|
||||
```python
|
||||
await write_note(
|
||||
"{project}",
|
||||
title="Activity Summary {timeframe}",
|
||||
content='''
|
||||
# Activity Summary for {project} ({timeframe})
|
||||
|
||||
## Overview
|
||||
[Summary of key changes and developments in this project over this period]
|
||||
|
||||
## Key Updates
|
||||
[List main updates and their significance within this project]
|
||||
|
||||
## Observations
|
||||
- [trend] [Observation about patterns in recent activity]
|
||||
- [insight] [Connection between different activities]
|
||||
|
||||
## Relations
|
||||
- summarizes [[{first_title}]]
|
||||
- relates_to [[{project} Overview]]
|
||||
''',
|
||||
folder="summaries"
|
||||
)
|
||||
```
|
||||
|
||||
Summarizing periodic activity helps create high-level insights and connections within the project.
|
||||
"""
|
||||
else:
|
||||
# Discovery mode suggestions
|
||||
project_count = len(recent.projects) if isinstance(recent, ProjectActivitySummary) else 0
|
||||
most_active = (
|
||||
getattr(recent.summary, "most_active_project", "Unknown")
|
||||
if isinstance(recent, ProjectActivitySummary)
|
||||
else "Unknown"
|
||||
)
|
||||
|
||||
capture_suggestions = f"""
|
||||
## Cross-Project Activity Discovery
|
||||
|
||||
Found activity across {project_count} projects. Most active: **{most_active}**
|
||||
|
||||
Consider creating a cross-project summary:
|
||||
|
||||
```python
|
||||
await write_note(
|
||||
"{most_active if most_active != "Unknown" else "main"}",
|
||||
title="Cross-Project Activity Summary {timeframe}",
|
||||
content='''
|
||||
# Cross-Project Activity Summary ({timeframe})
|
||||
|
||||
## Overview
|
||||
Activity found across {project_count} projects, with {most_active} showing the most activity.
|
||||
|
||||
## Key Developments
|
||||
[Summarize important changes across all projects]
|
||||
|
||||
## Project Insights
|
||||
[Note patterns or connections between projects]
|
||||
|
||||
## Observations
|
||||
- [trend] [Cross-project patterns observed]
|
||||
- [insight] [Connections between different project activities]
|
||||
|
||||
## Relations
|
||||
- summarizes [[{first_title}]]
|
||||
- relates_to [[Project Portfolio Overview]]
|
||||
''',
|
||||
folder="summaries"
|
||||
)
|
||||
```
|
||||
|
||||
Cross-project summaries help identify broader trends and project interconnections.
|
||||
"""
|
||||
|
||||
return prompt_context + capture_suggestions
|
||||
return prompt_guidance
|
||||
|
||||
@@ -8,11 +8,11 @@ from typing import Annotated, Optional
|
||||
from loguru import logger
|
||||
from pydantic import Field
|
||||
|
||||
from basic_memory.config import get_project_config
|
||||
from basic_memory.config import ConfigManager
|
||||
from basic_memory.mcp.async_client import get_client
|
||||
from basic_memory.mcp.project_context import get_active_project
|
||||
from basic_memory.mcp.server import mcp
|
||||
from basic_memory.mcp.tools.utils import call_post
|
||||
from basic_memory.schemas.base import TimeFrame
|
||||
from basic_memory.schemas.prompt import SearchPromptRequest
|
||||
|
||||
|
||||
@@ -23,7 +23,7 @@ from basic_memory.schemas.prompt import SearchPromptRequest
|
||||
async def search_prompt(
|
||||
query: str,
|
||||
timeframe: Annotated[
|
||||
Optional[TimeFrame],
|
||||
Optional[str],
|
||||
Field(description="How far back to search (e.g. '1d', '1 week')"),
|
||||
] = None,
|
||||
) -> str:
|
||||
@@ -42,14 +42,17 @@ async def search_prompt(
|
||||
logger.info(f"Searching knowledge base, query: {query}, timeframe: {timeframe}")
|
||||
|
||||
async with get_client() as client:
|
||||
config = ConfigManager().config
|
||||
active_project = await get_active_project(client, project=config.default_project)
|
||||
|
||||
# Create request model
|
||||
request = SearchPromptRequest(query=query, timeframe=timeframe)
|
||||
|
||||
project_url = get_project_config().project_url
|
||||
|
||||
# Call the prompt API endpoint
|
||||
response = await call_post(
|
||||
client, f"{project_url}/prompt/search", json=request.model_dump(exclude_none=True)
|
||||
client,
|
||||
f"/v2/projects/{active_project.external_id}/prompt/search",
|
||||
json=request.model_dump(exclude_none=True),
|
||||
)
|
||||
|
||||
# Extract the rendered prompt from the response
|
||||
|
||||
@@ -8,7 +8,6 @@ from dataclasses import dataclass
|
||||
from textwrap import dedent
|
||||
from typing import List
|
||||
|
||||
from basic_memory.schemas.base import TimeFrame
|
||||
from basic_memory.schemas.memory import (
|
||||
normalize_memory_url,
|
||||
EntitySummary,
|
||||
@@ -25,7 +24,7 @@ class PromptContextItem:
|
||||
|
||||
@dataclass
|
||||
class PromptContext:
|
||||
timeframe: TimeFrame
|
||||
timeframe: str
|
||||
topic: str
|
||||
results: List[PromptContextItem]
|
||||
|
||||
|
||||
@@ -62,10 +62,9 @@ async def project_info(
|
||||
|
||||
async with get_client() as client:
|
||||
project_config = await get_active_project(client, project, context)
|
||||
project_url = project_config.permalink
|
||||
|
||||
# Call the API endpoint
|
||||
response = await call_get(client, f"{project_url}/project/info")
|
||||
response = await call_get(client, f"/v2/projects/{project_config.external_id}/info")
|
||||
|
||||
# Convert response to ProjectInfoResponse
|
||||
return ProjectInfoResponse.model_validate(response.json())
|
||||
|
||||
@@ -13,7 +13,7 @@ from basic_memory.mcp.tools.recent_activity import recent_activity
|
||||
from basic_memory.mcp.tools.read_note import read_note
|
||||
from basic_memory.mcp.tools.view_note import view_note
|
||||
from basic_memory.mcp.tools.write_note import write_note
|
||||
from basic_memory.mcp.tools.search import search_notes
|
||||
from basic_memory.mcp.tools.search import search_notes, search_by_metadata
|
||||
from basic_memory.mcp.tools.canvas import canvas
|
||||
from basic_memory.mcp.tools.list_directory import list_directory
|
||||
from basic_memory.mcp.tools.edit_note import edit_note
|
||||
@@ -42,6 +42,7 @@ __all__ = [
|
||||
"read_note",
|
||||
"recent_activity",
|
||||
"search",
|
||||
"search_by_metadata",
|
||||
"search_notes",
|
||||
"view_note",
|
||||
"write_note",
|
||||
|
||||
@@ -22,7 +22,7 @@ async def canvas(
|
||||
nodes: List[Dict[str, Any]],
|
||||
edges: List[Dict[str, Any]],
|
||||
title: str,
|
||||
folder: str,
|
||||
directory: str,
|
||||
project: Optional[str] = None,
|
||||
context: Context | None = None,
|
||||
) -> str:
|
||||
@@ -43,8 +43,8 @@ async def canvas(
|
||||
nodes: List of node objects following JSON Canvas 1.0 spec
|
||||
edges: List of edge objects following JSON Canvas 1.0 spec
|
||||
title: The title of the canvas (will be saved as title.canvas)
|
||||
folder: Folder path relative to project root where the canvas should be saved.
|
||||
Use forward slashes (/) as separators. Examples: "diagrams", "projects/2025", "visual/maps"
|
||||
directory: Directory path relative to project root where the canvas should be saved.
|
||||
Use forward slashes (/) as separators. Examples: "diagrams", "projects/2025", "visual/maps"
|
||||
context: Optional FastMCP context for performance caching.
|
||||
|
||||
Returns:
|
||||
@@ -52,7 +52,7 @@ async def canvas(
|
||||
|
||||
Important Notes:
|
||||
- When referencing files, use the exact file path as shown in Obsidian
|
||||
Example: "folder/Document Name.md" (not permalink format)
|
||||
Example: "docs/Document Name.md" (not permalink format)
|
||||
- For file nodes, the "file" attribute must reference an existing file
|
||||
- Nodes require id, type, x, y, width, height properties
|
||||
- Edges require id, fromNode, toNode properties
|
||||
@@ -66,7 +66,7 @@ async def canvas(
|
||||
{
|
||||
"id": "node1",
|
||||
"type": "file", // Options: "file", "text", "link", "group"
|
||||
"file": "folder/Document.md",
|
||||
"file": "docs/Document.md",
|
||||
"x": 0,
|
||||
"y": 0,
|
||||
"width": 400,
|
||||
@@ -85,21 +85,21 @@ async def canvas(
|
||||
```
|
||||
|
||||
Examples:
|
||||
# Create canvas in project
|
||||
canvas("my-project", nodes=[...], edges=[...], title="My Canvas", folder="diagrams")
|
||||
# Create canvas in default/current project
|
||||
canvas(nodes=[...], edges=[...], title="My Canvas", directory="diagrams")
|
||||
|
||||
# Create canvas in work project
|
||||
canvas("work-project", nodes=[...], edges=[...], title="Process Flow", folder="visual/maps")
|
||||
# Create canvas with explicit project
|
||||
canvas(nodes=[...], edges=[...], title="Process Flow", directory="visual/maps", project="work-project")
|
||||
|
||||
Raises:
|
||||
ToolError: If project doesn't exist or folder path is invalid
|
||||
ToolError: If project doesn't exist or directory path is invalid
|
||||
"""
|
||||
async with get_client() as client:
|
||||
active_project = await get_active_project(client, project, context)
|
||||
|
||||
# Ensure path has .canvas extension
|
||||
file_title = title if title.endswith(".canvas") else f"{title}.canvas"
|
||||
file_path = f"{folder}/{file_title}"
|
||||
file_path = f"{directory}/{file_title}"
|
||||
|
||||
# Create canvas data structure
|
||||
canvas_data = {"nodes": nodes, "edges": edges}
|
||||
|
||||
@@ -147,43 +147,58 @@ delete_note("{project}", "correct-identifier-from-search")
|
||||
If the note should be deleted but the operation keeps failing, send a message to support@basicmemory.com."""
|
||||
|
||||
|
||||
@mcp.tool(description="Delete a note by title or permalink")
|
||||
@mcp.tool(description="Delete a note or directory by title, permalink, or path")
|
||||
async def delete_note(
|
||||
identifier: str, project: Optional[str] = None, context: Context | None = None
|
||||
identifier: str,
|
||||
is_directory: bool = False,
|
||||
project: Optional[str] = None,
|
||||
context: Context | None = None,
|
||||
) -> bool | str:
|
||||
"""Delete a note from the knowledge base.
|
||||
"""Delete a note or directory from the knowledge base.
|
||||
|
||||
Permanently removes a note from the specified project. The note is identified
|
||||
by title or permalink. If the note doesn't exist, the operation returns False
|
||||
without error. If deletion fails due to other issues, helpful error messages are provided.
|
||||
Permanently removes a note or directory from the specified project. For single notes,
|
||||
they are identified by title or permalink. For directories, use is_directory=True and
|
||||
provide the directory path. If the note/directory doesn't exist, the operation returns
|
||||
False without error. If deletion fails, helpful error messages are provided.
|
||||
|
||||
Project Resolution:
|
||||
Server resolves projects in this order: Single Project Mode → project parameter → default project.
|
||||
If project unknown, use list_memory_projects() or recent_activity() first.
|
||||
|
||||
Args:
|
||||
identifier: For files: note title or permalink to delete.
|
||||
For directories: the directory path (e.g., "docs", "projects/2025").
|
||||
Can be a title like "Meeting Notes" or permalink like "notes/meeting-notes"
|
||||
is_directory: If True, deletes an entire directory and all its contents.
|
||||
When True, identifier should be a directory path
|
||||
(without file extensions). Defaults to False.
|
||||
project: Project name to delete from. Optional - server will resolve using hierarchy.
|
||||
If unknown, use list_memory_projects() to discover available projects.
|
||||
identifier: Note title or permalink to delete
|
||||
Can be a title like "Meeting Notes" or permalink like "notes/meeting-notes"
|
||||
context: Optional FastMCP context for performance caching.
|
||||
|
||||
Returns:
|
||||
True if note was successfully deleted, False if note was not found.
|
||||
For directories, returns a formatted summary of deleted files.
|
||||
On errors, returns a formatted string with helpful troubleshooting guidance.
|
||||
|
||||
Examples:
|
||||
# Delete by title
|
||||
delete_note("my-project", "Meeting Notes: Project Planning")
|
||||
delete_note("Meeting Notes: Project Planning")
|
||||
|
||||
# Delete by permalink
|
||||
delete_note("work-docs", "notes/project-planning")
|
||||
delete_note("notes/project-planning")
|
||||
|
||||
# Delete with exact path
|
||||
delete_note("research", "experiments/ml-model-results")
|
||||
# Delete with explicit project
|
||||
delete_note("experiments/ml-model-results", project="research")
|
||||
|
||||
# Delete entire directory
|
||||
delete_note("docs", is_directory=True)
|
||||
|
||||
# Delete nested directory
|
||||
delete_note("projects/2024", is_directory=True)
|
||||
|
||||
# Common usage pattern
|
||||
if delete_note("my-project", "old-draft"):
|
||||
if delete_note("old-draft"):
|
||||
print("Note deleted successfully")
|
||||
else:
|
||||
print("Note not found or already deleted")
|
||||
@@ -193,7 +208,7 @@ async def delete_note(
|
||||
SecurityError: If identifier attempts path traversal
|
||||
|
||||
Warning:
|
||||
This operation is permanent and cannot be undone. The note file
|
||||
This operation is permanent and cannot be undone. The note/directory files
|
||||
will be removed from the filesystem and all references will be lost.
|
||||
|
||||
Note:
|
||||
@@ -202,6 +217,10 @@ async def delete_note(
|
||||
commands and alternative formats to try.
|
||||
"""
|
||||
async with get_client() as client:
|
||||
logger.debug(
|
||||
f"Deleting {'directory' if is_directory else 'note'}: {identifier} in project: {project}"
|
||||
)
|
||||
|
||||
active_project = await get_active_project(client, project, context)
|
||||
|
||||
# Import here to avoid circular import
|
||||
@@ -210,6 +229,67 @@ async def delete_note(
|
||||
# Use typed KnowledgeClient for API calls
|
||||
knowledge_client = KnowledgeClient(client, active_project.external_id)
|
||||
|
||||
# Handle directory deletes
|
||||
if is_directory:
|
||||
try:
|
||||
result = await knowledge_client.delete_directory(identifier)
|
||||
|
||||
# Build success message for directory delete
|
||||
result_lines = [
|
||||
"# Directory Deleted Successfully",
|
||||
"",
|
||||
f"**Directory:** `{identifier}`",
|
||||
"",
|
||||
"## Summary",
|
||||
f"- Total files: {result.total_files}",
|
||||
f"- Successfully deleted: {result.successful_deletes}",
|
||||
f"- Failed: {result.failed_deletes}",
|
||||
]
|
||||
|
||||
if result.deleted_files:
|
||||
result_lines.extend(["", "## Deleted Files"])
|
||||
for file_path in result.deleted_files[:10]: # Show first 10
|
||||
result_lines.append(f"- `{file_path}`")
|
||||
if len(result.deleted_files) > 10:
|
||||
result_lines.append(f"- ... and {len(result.deleted_files) - 10} more")
|
||||
|
||||
if result.errors: # pragma: no cover
|
||||
result_lines.extend(["", "## Errors"])
|
||||
for error in result.errors[:5]: # Show first 5 errors
|
||||
result_lines.append(f"- `{error.path}`: {error.error}")
|
||||
if len(result.errors) > 5:
|
||||
result_lines.append(f"- ... and {len(result.errors) - 5} more errors")
|
||||
|
||||
result_lines.extend(["", f"<!-- Project: {active_project.name} -->"])
|
||||
|
||||
logger.info(
|
||||
f"Directory delete completed: {identifier}, "
|
||||
f"deleted={result.successful_deletes}, failed={result.failed_deletes}"
|
||||
)
|
||||
|
||||
return "\n".join(result_lines)
|
||||
|
||||
except Exception as e: # pragma: no cover
|
||||
logger.error(f"Directory delete failed for '{identifier}': {e}")
|
||||
return f"""# Directory Delete Failed
|
||||
|
||||
Error deleting directory '{identifier}': {str(e)}
|
||||
|
||||
## Troubleshooting:
|
||||
1. **Verify the directory exists**: Use `list_directory("{identifier}")` to check
|
||||
2. **Check for permission issues**: Ensure you have delete access to the project
|
||||
3. **Try individual deletes**: Delete files one at a time if bulk delete fails
|
||||
|
||||
## Alternative approach:
|
||||
```
|
||||
# List directory contents first
|
||||
list_directory("{identifier}")
|
||||
|
||||
# Then delete individual files
|
||||
delete_note("path/to/file.md")
|
||||
```"""
|
||||
|
||||
# Handle single note deletes
|
||||
try:
|
||||
# Resolve identifier to entity ID
|
||||
entity_id = await knowledge_client.resolve_entity(identifier)
|
||||
|
||||
@@ -256,7 +256,7 @@ async def edit_note(
|
||||
edit_data["expected_replacements"] = str(expected_replacements)
|
||||
|
||||
# Call the PATCH endpoint
|
||||
result = await knowledge_client.patch_entity(entity_id, edit_data)
|
||||
result = await knowledge_client.patch_entity(entity_id, edit_data, fast=False)
|
||||
|
||||
# Format summary
|
||||
summary = [
|
||||
|
||||
@@ -342,34 +342,41 @@ delete_note("{identifier}")
|
||||
|
||||
|
||||
@mcp.tool(
|
||||
description="Move a note to a new location, updating database and maintaining links.",
|
||||
description="Move a note or directory to a new location, updating database and maintaining links.",
|
||||
)
|
||||
async def move_note(
|
||||
identifier: str,
|
||||
destination_path: str,
|
||||
is_directory: bool = False,
|
||||
project: Optional[str] = None,
|
||||
context: Context | None = None,
|
||||
) -> str:
|
||||
"""Move a note to a new file location within the same project.
|
||||
"""Move a note or directory to a new location within the same project.
|
||||
|
||||
Moves a note from one location to another within the project, updating all
|
||||
database references and maintaining semantic content. Uses stateless architecture -
|
||||
project parameter optional with server resolution.
|
||||
Moves a note or directory from one location to another within the project,
|
||||
updating all database references and maintaining semantic content. Uses stateless
|
||||
architecture - project parameter optional with server resolution.
|
||||
|
||||
Args:
|
||||
identifier: Exact entity identifier (title, permalink, or memory:// URL).
|
||||
identifier: For files: exact entity identifier (title, permalink, or memory:// URL).
|
||||
For directories: the directory path (e.g., "docs", "projects/2025").
|
||||
Must be an exact match - fuzzy matching is not supported for move operations.
|
||||
Use search_notes() or read_note() first to find the correct identifier if uncertain.
|
||||
destination_path: New path relative to project root (e.g., "work/meetings/2025-05-26.md")
|
||||
Use search_notes() or list_directory() first to find the correct path if uncertain.
|
||||
destination_path: For files: new path relative to project root (e.g., "work/meetings/note.md")
|
||||
For directories: new directory path (e.g., "archive/docs")
|
||||
is_directory: If True, moves an entire directory and all its contents.
|
||||
When True, identifier and destination_path should be directory paths
|
||||
(without file extensions). Defaults to False.
|
||||
project: Project name to move within. Optional - server will resolve using hierarchy.
|
||||
If unknown, use list_memory_projects() to discover available projects.
|
||||
context: Optional FastMCP context for performance caching.
|
||||
|
||||
Returns:
|
||||
Success message with move details and project information.
|
||||
For directories, includes count of files moved and any errors.
|
||||
|
||||
Examples:
|
||||
# Move to new folder (exact title match)
|
||||
# Move a single note to new folder (exact title match)
|
||||
move_note("My Note", "work/notes/my-note.md")
|
||||
|
||||
# Move by exact permalink
|
||||
@@ -381,6 +388,12 @@ async def move_note(
|
||||
# Explicit project specification
|
||||
move_note("My Note", "work/notes/my-note.md", project="work-project")
|
||||
|
||||
# Move entire directory
|
||||
move_note("docs", "archive/docs", is_directory=True)
|
||||
|
||||
# Move nested directory
|
||||
move_note("projects/2024", "archive/projects/2024", is_directory=True)
|
||||
|
||||
# If uncertain about identifier, search first:
|
||||
# search_notes("my note") # Find available notes
|
||||
# move_note("docs/my-note-2025", "archive/my-note.md") # Use exact result
|
||||
@@ -400,7 +413,9 @@ async def move_note(
|
||||
- Maintains all observations and relations
|
||||
"""
|
||||
async with get_client() as client:
|
||||
logger.debug(f"Moving note: {identifier} to {destination_path} in project: {project}")
|
||||
logger.debug(
|
||||
f"Moving {'directory' if is_directory else 'note'}: {identifier} to {destination_path} in project: {project}"
|
||||
)
|
||||
|
||||
active_project = await get_active_project(client, project, context)
|
||||
|
||||
@@ -426,7 +441,75 @@ The destination path '{destination_path}' is not allowed - paths must stay withi
|
||||
move_note("{identifier}", "notes/{destination_path.split("/")[-1] if "/" in destination_path else destination_path}")
|
||||
```"""
|
||||
|
||||
# Check for potential cross-project move attempts
|
||||
# Handle directory moves
|
||||
if is_directory:
|
||||
# Import here to avoid circular import
|
||||
from basic_memory.mcp.clients import KnowledgeClient
|
||||
|
||||
knowledge_client = KnowledgeClient(client, active_project.external_id)
|
||||
|
||||
try:
|
||||
result = await knowledge_client.move_directory(identifier, destination_path)
|
||||
|
||||
# Build success message for directory move
|
||||
result_lines = [
|
||||
"# Directory Moved Successfully",
|
||||
"",
|
||||
f"**Source:** `{identifier}`",
|
||||
f"**Destination:** `{destination_path}`",
|
||||
"",
|
||||
"## Summary",
|
||||
f"- Total files: {result.total_files}",
|
||||
f"- Successfully moved: {result.successful_moves}",
|
||||
f"- Failed: {result.failed_moves}",
|
||||
]
|
||||
|
||||
if result.moved_files:
|
||||
result_lines.extend(["", "## Moved Files"])
|
||||
for file_path in result.moved_files[:10]: # Show first 10
|
||||
result_lines.append(f"- `{file_path}`")
|
||||
if len(result.moved_files) > 10:
|
||||
result_lines.append(f"- ... and {len(result.moved_files) - 10} more")
|
||||
|
||||
if result.errors: # pragma: no cover
|
||||
result_lines.extend(["", "## Errors"])
|
||||
for error in result.errors[:5]: # Show first 5 errors
|
||||
result_lines.append(f"- `{error.path}`: {error.error}")
|
||||
if len(result.errors) > 5:
|
||||
result_lines.append(f"- ... and {len(result.errors) - 5} more errors")
|
||||
|
||||
result_lines.extend(["", f"<!-- Project: {active_project.name} -->"])
|
||||
|
||||
logger.info(
|
||||
f"Directory move completed: {identifier} -> {destination_path}, "
|
||||
f"moved={result.successful_moves}, failed={result.failed_moves}"
|
||||
)
|
||||
|
||||
return "\n".join(result_lines)
|
||||
|
||||
except Exception as e: # pragma: no cover
|
||||
logger.error(
|
||||
f"Directory move failed for '{identifier}' to '{destination_path}': {e}"
|
||||
)
|
||||
return f"""# Directory Move Failed
|
||||
|
||||
Error moving directory '{identifier}' to '{destination_path}': {str(e)}
|
||||
|
||||
## Troubleshooting:
|
||||
1. **Verify the directory exists**: Use `list_directory("{identifier}")` to check
|
||||
2. **Check for conflicts**: The destination may already contain files
|
||||
3. **Try individual moves**: Move files one at a time if bulk move fails
|
||||
|
||||
## Alternative approach:
|
||||
```
|
||||
# List directory contents first
|
||||
list_directory("{identifier}")
|
||||
|
||||
# Then move individual files
|
||||
move_note("path/to/file.md", "{destination_path}/file.md")
|
||||
```"""
|
||||
|
||||
# Check for potential cross-project move attempts (file moves only)
|
||||
cross_project_error = await _detect_cross_project_move_attempt(
|
||||
client, identifier, destination_path, active_project.name
|
||||
)
|
||||
|
||||
@@ -144,7 +144,7 @@ async def recent_activity(
|
||||
)
|
||||
|
||||
# Get list of all projects
|
||||
response = await call_get(client, "/projects/projects")
|
||||
response = await call_get(client, "/v2/projects/")
|
||||
project_list = ProjectList.model_validate(response.json())
|
||||
|
||||
projects_activity = {}
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
"""Search tools for Basic Memory MCP server."""
|
||||
|
||||
from textwrap import dedent
|
||||
from typing import List, Optional
|
||||
from typing import List, Optional, Dict, Any
|
||||
|
||||
from loguru import logger
|
||||
from fastmcp import Context
|
||||
@@ -207,6 +207,9 @@ async def search_notes(
|
||||
types: List[str] | None = None,
|
||||
entity_types: List[str] | None = None,
|
||||
after_date: Optional[str] = None,
|
||||
metadata_filters: Optional[Dict[str, Any]] = None,
|
||||
tags: Optional[List[str]] = None,
|
||||
status: Optional[str] = None,
|
||||
context: Context | None = None,
|
||||
) -> SearchResponse | str:
|
||||
"""Search across all content in the knowledge base with comprehensive syntax support.
|
||||
@@ -248,6 +251,27 @@ async def search_notes(
|
||||
- `search_notes("research", "query", entity_types=["observation"])` - Filter by entity type
|
||||
- `search_notes("team-docs", "query", after_date="2024-01-01")` - Recent content only
|
||||
- `search_notes("my-project", "query", after_date="1 week")` - Relative date filtering
|
||||
- `search_notes("my-project", "query", tags=["security"])` - Filter by frontmatter tags
|
||||
- `search_notes("my-project", "query", status="in-progress")` - Filter by frontmatter status
|
||||
- `search_notes("my-project", "query", metadata_filters={"priority": {"$in": ["high"]}})`
|
||||
|
||||
### Structured Metadata Filters
|
||||
Filters are exact matches on frontmatter metadata. Supported forms:
|
||||
- Equality: `{"status": "in-progress"}`
|
||||
- Array contains (all): `{"tags": ["security", "oauth"]}`
|
||||
- Operators:
|
||||
- `$in`: `{"priority": {"$in": ["high", "critical"]}}`
|
||||
- `$gt`, `$gte`, `$lt`, `$lte`: `{"schema.confidence": {"$gt": 0.7}}`
|
||||
- `$between`: `{"schema.confidence": {"$between": [0.3, 0.6]}}`
|
||||
- Nested keys use dot notation (e.g., `"schema.confidence"`).
|
||||
|
||||
### Filter-only Searches
|
||||
You can pass an empty query string when only using structured filters:
|
||||
- `search_notes("my-project", "", metadata_filters={"type": "spec"})`
|
||||
|
||||
### Convenience Filters
|
||||
`tags` and `status` are shorthand for metadata_filters. If the same key exists in
|
||||
metadata_filters, that value wins.
|
||||
|
||||
### Advanced Pattern Examples
|
||||
- `search_notes("work-project", "project AND (meeting OR discussion)")` - Complex boolean logic
|
||||
@@ -265,6 +289,9 @@ async def search_notes(
|
||||
types: Optional list of note types to search (e.g., ["note", "person"])
|
||||
entity_types: Optional list of entity types to filter by (e.g., ["entity", "observation"])
|
||||
after_date: Optional date filter for recent content (e.g., "1 week", "2d", "2024-01-01")
|
||||
metadata_filters: Optional structured frontmatter filters (e.g., {"status": "in-progress"})
|
||||
tags: Optional tag filter (frontmatter tags); shorthand for metadata_filters["tags"]
|
||||
status: Optional status filter (frontmatter status); shorthand for metadata_filters["status"]
|
||||
context: Optional FastMCP context for performance caching.
|
||||
|
||||
Returns:
|
||||
@@ -345,7 +372,7 @@ async def search_notes(
|
||||
search_query.permalink_match = query
|
||||
elif search_type == "permalink":
|
||||
search_query.permalink = query
|
||||
else:
|
||||
else: # pragma: no cover
|
||||
search_query.text = query # Default to text search
|
||||
|
||||
# Add optional filters if provided (empty lists are treated as no filter)
|
||||
@@ -355,6 +382,12 @@ async def search_notes(
|
||||
search_query.types = types
|
||||
if after_date:
|
||||
search_query.after_date = after_date
|
||||
if metadata_filters:
|
||||
search_query.metadata_filters = metadata_filters
|
||||
if tags:
|
||||
search_query.tags = tags
|
||||
if status:
|
||||
search_query.status = status
|
||||
|
||||
async with get_client() as client:
|
||||
active_project = await get_active_project(client, project, context)
|
||||
@@ -387,3 +420,82 @@ async def search_notes(
|
||||
logger.error(f"Search failed for query '{query}': {e}, project: {active_project.name}")
|
||||
# Return formatted error message as string for better user experience
|
||||
return _format_search_error_response(active_project.name, str(e), query, search_type)
|
||||
|
||||
|
||||
@mcp.tool(
|
||||
description="Search entities by structured frontmatter metadata.",
|
||||
)
|
||||
async def search_by_metadata(
|
||||
filters: Dict[str, Any],
|
||||
project: Optional[str] = None,
|
||||
limit: int = 20,
|
||||
offset: int = 0,
|
||||
context: Context | None = None,
|
||||
) -> SearchResponse | str:
|
||||
"""Search entities by structured frontmatter metadata.
|
||||
|
||||
Args:
|
||||
filters: Dictionary of metadata filters (e.g., {"status": "in-progress"})
|
||||
project: Project name to search in. Optional - server will resolve using hierarchy.
|
||||
limit: Maximum number of results to return
|
||||
offset: Number of results to skip (for pagination)
|
||||
context: Optional FastMCP context for performance caching.
|
||||
|
||||
Returns:
|
||||
SearchResponse with results, or helpful error guidance if search fails
|
||||
"""
|
||||
if limit <= 0:
|
||||
return "# Error\n\n`limit` must be greater than 0."
|
||||
|
||||
# Build a structured-only search query
|
||||
search_query = SearchQuery()
|
||||
search_query.metadata_filters = filters
|
||||
search_query.entity_types = [SearchItemType.ENTITY]
|
||||
|
||||
# Convert offset/limit to page/page_size (API uses paging)
|
||||
page_size = limit
|
||||
page = (offset // limit) + 1
|
||||
offset_within_page = offset % limit
|
||||
|
||||
async with get_client() as client:
|
||||
active_project = await get_active_project(client, project, context)
|
||||
logger.info(
|
||||
f"Structured search in project {active_project.name} filters={filters} limit={limit} offset={offset}"
|
||||
)
|
||||
|
||||
try:
|
||||
from basic_memory.mcp.clients import SearchClient
|
||||
|
||||
search_client = SearchClient(client, active_project.external_id)
|
||||
result = await search_client.search(
|
||||
search_query.model_dump(),
|
||||
page=page,
|
||||
page_size=page_size,
|
||||
)
|
||||
|
||||
# Apply offset within page, fetch next page if needed
|
||||
if offset_within_page:
|
||||
remaining = result.results[offset_within_page:]
|
||||
if len(remaining) < limit:
|
||||
next_page = page + 1
|
||||
extra = await search_client.search(
|
||||
search_query.model_dump(),
|
||||
page=next_page,
|
||||
page_size=page_size,
|
||||
)
|
||||
remaining.extend(extra.results[: max(0, limit - len(remaining))])
|
||||
result = SearchResponse(
|
||||
results=remaining[:limit],
|
||||
current_page=page,
|
||||
page_size=page_size,
|
||||
)
|
||||
|
||||
return result
|
||||
|
||||
except Exception as e:
|
||||
logger.error(
|
||||
f"Metadata search failed for filters '{filters}': {e}, project: {active_project.name}"
|
||||
)
|
||||
return _format_search_error_response(
|
||||
active_project.name, str(e), str(filters), "metadata"
|
||||
)
|
||||
|
||||
@@ -21,7 +21,7 @@ TagType = Union[List[str], str, None]
|
||||
async def write_note(
|
||||
title: str,
|
||||
content: str,
|
||||
folder: str,
|
||||
directory: str,
|
||||
project: Optional[str] = None,
|
||||
tags: list[str] | str | None = None,
|
||||
note_type: str = "note",
|
||||
@@ -57,9 +57,9 @@ async def write_note(
|
||||
Args:
|
||||
title: The title of the note
|
||||
content: Markdown content for the note, can include observations and relations
|
||||
folder: Folder path relative to project root where the file should be saved.
|
||||
Use forward slashes (/) as separators. Use "/" or "" to write to project root.
|
||||
Examples: "notes", "projects/2025", "research/ml", "/" (root)
|
||||
directory: Directory path relative to project root where the file should be saved.
|
||||
Use forward slashes (/) as separators. Use "/" or "" to write to project root.
|
||||
Examples: "notes", "projects/2025", "research/ml", "/" (root)
|
||||
project: Project name to write to. Optional - server will resolve using the
|
||||
hierarchy above. If unknown, use list_memory_projects() to discover
|
||||
available projects.
|
||||
@@ -88,7 +88,7 @@ async def write_note(
|
||||
write_note(
|
||||
project="my-research",
|
||||
title="Meeting Notes",
|
||||
folder="meetings",
|
||||
directory="meetings",
|
||||
content="# Weekly Standup\\n\\n- [decision] Use SQLite for storage #tech"
|
||||
)
|
||||
|
||||
@@ -96,45 +96,45 @@ async def write_note(
|
||||
write_note(
|
||||
project="work-project",
|
||||
title="API Design",
|
||||
folder="specs",
|
||||
directory="specs",
|
||||
content="# REST API Specification\\n\\n- implements [[Authentication]]",
|
||||
tags=["api", "design"],
|
||||
note_type="guide"
|
||||
)
|
||||
|
||||
# Update existing note (same title/folder)
|
||||
# Update existing note (same title/directory)
|
||||
write_note(
|
||||
project="my-research",
|
||||
title="Meeting Notes",
|
||||
folder="meetings",
|
||||
directory="meetings",
|
||||
content="# Weekly Standup\\n\\n- [decision] Use PostgreSQL instead #tech"
|
||||
)
|
||||
|
||||
Raises:
|
||||
HTTPError: If project doesn't exist or is inaccessible
|
||||
SecurityError: If folder path attempts path traversal
|
||||
SecurityError: If directory path attempts path traversal
|
||||
"""
|
||||
async with get_client() as client:
|
||||
logger.info(
|
||||
f"MCP tool call tool=write_note project={project} folder={folder}, title={title}, tags={tags}"
|
||||
f"MCP tool call tool=write_note project={project} directory={directory}, title={title}, tags={tags}"
|
||||
)
|
||||
|
||||
# Get and validate the project (supports optional project parameter)
|
||||
active_project = await get_active_project(client, project, context)
|
||||
|
||||
# Normalize "/" to empty string for root folder (must happen before validation)
|
||||
if folder == "/":
|
||||
folder = ""
|
||||
# Normalize "/" to empty string for root directory (must happen before validation)
|
||||
if directory == "/":
|
||||
directory = ""
|
||||
|
||||
# Validate folder path to prevent path traversal attacks
|
||||
# Validate directory path to prevent path traversal attacks
|
||||
project_path = active_project.home
|
||||
if folder and not validate_project_path(folder, project_path):
|
||||
if directory and not validate_project_path(directory, project_path):
|
||||
logger.warning(
|
||||
"Attempted path traversal attack blocked",
|
||||
folder=folder,
|
||||
directory=directory,
|
||||
project=active_project.name,
|
||||
)
|
||||
return f"# Error\n\nFolder path '{folder}' is not allowed - paths must stay within project boundaries"
|
||||
return f"# Error\n\nDirectory path '{directory}' is not allowed - paths must stay within project boundaries"
|
||||
|
||||
# Process tags using the helper function
|
||||
tag_list = parse_tags(tags)
|
||||
@@ -142,7 +142,7 @@ async def write_note(
|
||||
metadata = {"tags": tag_list} if tag_list else None
|
||||
entity = Entity(
|
||||
title=title,
|
||||
folder=folder,
|
||||
directory=directory,
|
||||
entity_type=note_type,
|
||||
content_type="text/markdown",
|
||||
content=content,
|
||||
@@ -159,7 +159,7 @@ async def write_note(
|
||||
logger.debug(f"Attempting to create entity permalink={entity.permalink}")
|
||||
action = "Created" # Default to created
|
||||
try:
|
||||
result = await knowledge_client.create_entity(entity.model_dump())
|
||||
result = await knowledge_client.create_entity(entity.model_dump(), fast=False)
|
||||
action = "Created"
|
||||
except Exception as e:
|
||||
# If creation failed due to conflict (already exists), try to update
|
||||
@@ -175,7 +175,9 @@ async def write_note(
|
||||
"Entity permalink is required for updates"
|
||||
) # pragma: no cover
|
||||
entity_id = await knowledge_client.resolve_entity(entity.permalink)
|
||||
result = await knowledge_client.update_entity(entity_id, entity.model_dump())
|
||||
result = await knowledge_client.update_entity(
|
||||
entity_id, entity.model_dump(), fast=False
|
||||
)
|
||||
action = "Updated"
|
||||
except Exception as update_error: # pragma: no cover
|
||||
# Re-raise the original error if update also fails
|
||||
|
||||
@@ -5,7 +5,7 @@ from typing import List, Optional, Sequence, Union, Any
|
||||
|
||||
|
||||
from loguru import logger
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy import select, func
|
||||
from sqlalchemy.exc import IntegrityError
|
||||
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
|
||||
from sqlalchemy.orm import selectinload
|
||||
@@ -69,12 +69,21 @@ class EntityRepository(Repository[Entity]):
|
||||
return await self.find_one(query)
|
||||
|
||||
async def get_by_title(self, title: str) -> Sequence[Entity]:
|
||||
"""Get entity by title.
|
||||
"""Get entities by title, ordered by shortest path first.
|
||||
|
||||
When multiple entities share the same title (in different folders),
|
||||
returns them ordered by file_path length then alphabetically.
|
||||
This provides "shortest path" resolution for duplicate titles.
|
||||
|
||||
Args:
|
||||
title: Title of the entity to find
|
||||
"""
|
||||
query = self.select().where(Entity.title == title).options(*self.get_load_options())
|
||||
query = (
|
||||
self.select()
|
||||
.where(Entity.title == title)
|
||||
.order_by(func.length(Entity.file_path), Entity.file_path)
|
||||
.options(*self.get_load_options())
|
||||
)
|
||||
result = await self.execute_query(query)
|
||||
return list(result.scalars().all())
|
||||
|
||||
|
||||
@@ -0,0 +1,139 @@
|
||||
"""Helpers for parsing structured metadata filters for search."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from datetime import date, datetime
|
||||
import re
|
||||
from typing import Any, Iterable, List
|
||||
|
||||
|
||||
_KEY_RE = re.compile(r"^[A-Za-z0-9_-]+(\.[A-Za-z0-9_-]+)*$")
|
||||
_NUMERIC_RE = re.compile(r"^-?\d+(\.\d+)?$")
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ParsedMetadataFilter:
|
||||
"""Normalized metadata filter for SQL generation."""
|
||||
|
||||
path_parts: List[str]
|
||||
op: str
|
||||
value: Any
|
||||
comparison: str | None = None # "numeric" or "text" for comparisons
|
||||
|
||||
|
||||
def _is_numeric_value(value: Any) -> bool:
|
||||
if isinstance(value, bool):
|
||||
return False
|
||||
if isinstance(value, (int, float)):
|
||||
return True
|
||||
if isinstance(value, str):
|
||||
return bool(_NUMERIC_RE.match(value.strip()))
|
||||
return False
|
||||
|
||||
|
||||
def _is_numeric_collection(values: Iterable[Any]) -> bool:
|
||||
return all(_is_numeric_value(v) for v in values)
|
||||
|
||||
|
||||
def _normalize_scalar(value: Any) -> Any:
|
||||
if isinstance(value, datetime):
|
||||
return value.isoformat()
|
||||
if isinstance(value, date):
|
||||
return value.isoformat()
|
||||
if isinstance(value, bool):
|
||||
return str(value)
|
||||
if isinstance(value, (int, float)):
|
||||
return str(value)
|
||||
return value
|
||||
|
||||
|
||||
def parse_metadata_filters(filters: dict[str, Any]) -> List[ParsedMetadataFilter]:
|
||||
"""Parse metadata filters into normalized clauses.
|
||||
|
||||
Supported forms:
|
||||
- {"status": "in-progress"}
|
||||
- {"tags": ["security", "oauth"]} # array contains all
|
||||
- {"priority": {"$in": ["high", "critical"]}}
|
||||
- {"schema.confidence": {"$gt": 0.7}}
|
||||
- {"schema.confidence": {"$between": [0.3, 0.6]}}
|
||||
"""
|
||||
parsed: List[ParsedMetadataFilter] = []
|
||||
|
||||
for raw_key, raw_value in (filters or {}).items():
|
||||
if not isinstance(raw_key, str) or not raw_key.strip():
|
||||
raise ValueError("metadata filter keys must be non-empty strings")
|
||||
key = raw_key.strip()
|
||||
if not _KEY_RE.match(key):
|
||||
raise ValueError(f"Unsupported metadata filter key: {raw_key}")
|
||||
|
||||
path_parts = key.split(".")
|
||||
|
||||
# Operator form
|
||||
if isinstance(raw_value, dict):
|
||||
if len(raw_value) != 1:
|
||||
raise ValueError(f"Invalid metadata filter for '{raw_key}': {raw_value}")
|
||||
op, value = next(iter(raw_value.items()))
|
||||
|
||||
if op == "$in":
|
||||
if not isinstance(value, list) or not value:
|
||||
raise ValueError(f"$in requires a non-empty list for '{raw_key}'")
|
||||
parsed.append(
|
||||
ParsedMetadataFilter(path_parts, "in", [_normalize_scalar(v) for v in value])
|
||||
)
|
||||
continue
|
||||
|
||||
if op in {"$gt", "$gte", "$lt", "$lte"}:
|
||||
if _is_numeric_value(value):
|
||||
normalized = float(value)
|
||||
comparison = "numeric"
|
||||
else:
|
||||
normalized = _normalize_scalar(value)
|
||||
comparison = "text"
|
||||
parsed.append(
|
||||
ParsedMetadataFilter(path_parts, op.lstrip("$"), normalized, comparison)
|
||||
)
|
||||
continue
|
||||
|
||||
if op == "$between":
|
||||
if not isinstance(value, list) or len(value) != 2:
|
||||
raise ValueError(f"$between requires [min, max] for '{raw_key}'")
|
||||
if _is_numeric_collection(value):
|
||||
normalized = [float(v) for v in value]
|
||||
comparison = "numeric"
|
||||
else:
|
||||
normalized = [_normalize_scalar(v) for v in value]
|
||||
comparison = "text"
|
||||
parsed.append(ParsedMetadataFilter(path_parts, "between", normalized, comparison))
|
||||
continue
|
||||
|
||||
raise ValueError(f"Unsupported operator '{op}' in metadata filter for '{raw_key}'")
|
||||
|
||||
# Array contains (all)
|
||||
if isinstance(raw_value, list):
|
||||
if not raw_value:
|
||||
raise ValueError(f"Empty list not allowed for metadata filter '{raw_key}'")
|
||||
parsed.append(
|
||||
ParsedMetadataFilter(
|
||||
path_parts, "contains", [_normalize_scalar(v) for v in raw_value]
|
||||
)
|
||||
)
|
||||
continue
|
||||
|
||||
# Simple equality
|
||||
parsed.append(ParsedMetadataFilter(path_parts, "eq", _normalize_scalar(raw_value)))
|
||||
|
||||
return parsed
|
||||
|
||||
|
||||
def build_sqlite_json_path(parts: List[str]) -> str:
|
||||
"""Build a SQLite JSON path for json_extract/json_each."""
|
||||
path = "$"
|
||||
for part in parts:
|
||||
path += f'."{part}"'
|
||||
return path
|
||||
|
||||
|
||||
def build_postgres_json_path(parts: List[str]) -> str:
|
||||
"""Build a Postgres JSON path for #>>/#> operators."""
|
||||
return "{" + ",".join(parts) + "}"
|
||||
@@ -12,9 +12,22 @@ from sqlalchemy import text
|
||||
from basic_memory import db
|
||||
from basic_memory.repository.search_index_row import SearchIndexRow
|
||||
from basic_memory.repository.search_repository_base import SearchRepositoryBase
|
||||
from basic_memory.repository.metadata_filters import (
|
||||
parse_metadata_filters,
|
||||
build_postgres_json_path,
|
||||
)
|
||||
from basic_memory.schemas.search import SearchItemType
|
||||
|
||||
|
||||
def _strip_nul_from_row(row_data: dict) -> dict:
|
||||
"""Strip NUL bytes from all string values in a row dict.
|
||||
|
||||
Secondary defense: PostgreSQL text columns cannot store \\x00.
|
||||
Primary sanitization happens in SearchService.index_entity_markdown().
|
||||
"""
|
||||
return {k: v.replace("\x00", "") if isinstance(v, str) else v for k, v in row_data.items()}
|
||||
|
||||
|
||||
class PostgresSearchRepository(SearchRepositoryBase):
|
||||
"""PostgreSQL tsvector implementation of search repository.
|
||||
|
||||
@@ -215,6 +228,7 @@ class PostgresSearchRepository(SearchRepositoryBase):
|
||||
types: Optional[List[str]] = None,
|
||||
after_date: Optional[datetime] = None,
|
||||
search_item_types: Optional[List[SearchItemType]] = None,
|
||||
metadata_filters: Optional[dict] = None,
|
||||
limit: int = 10,
|
||||
offset: int = 0,
|
||||
) -> List[SearchIndexRow]:
|
||||
@@ -222,6 +236,7 @@ class PostgresSearchRepository(SearchRepositoryBase):
|
||||
conditions = []
|
||||
params = {}
|
||||
order_by_clause = ""
|
||||
from_clause = "search_index"
|
||||
|
||||
# Handle text search for title and content using tsvector
|
||||
if search_text:
|
||||
@@ -233,18 +248,22 @@ class PostgresSearchRepository(SearchRepositoryBase):
|
||||
processed_text = self._prepare_search_term(search_text.strip())
|
||||
params["text"] = processed_text
|
||||
# Use @@ operator for tsvector matching
|
||||
conditions.append("textsearchable_index_col @@ to_tsquery('english', :text)")
|
||||
conditions.append(
|
||||
"search_index.textsearchable_index_col @@ to_tsquery('english', :text)"
|
||||
)
|
||||
|
||||
# Handle title search
|
||||
if title:
|
||||
title_text = self._prepare_search_term(title.strip(), is_prefix=False)
|
||||
params["title_text"] = title_text
|
||||
conditions.append("to_tsvector('english', title) @@ to_tsquery('english', :title_text)")
|
||||
conditions.append(
|
||||
"to_tsvector('english', search_index.title) @@ to_tsquery('english', :title_text)"
|
||||
)
|
||||
|
||||
# Handle permalink exact search
|
||||
if permalink:
|
||||
params["permalink"] = permalink
|
||||
conditions.append("permalink = :permalink")
|
||||
conditions.append("search_index.permalink = :permalink")
|
||||
|
||||
# Handle permalink pattern match
|
||||
if permalink_match:
|
||||
@@ -255,14 +274,14 @@ class PostgresSearchRepository(SearchRepositoryBase):
|
||||
# Convert * to % for SQL LIKE
|
||||
permalink_pattern = permalink_text.replace("*", "%")
|
||||
params["permalink"] = permalink_pattern
|
||||
conditions.append("permalink LIKE :permalink")
|
||||
conditions.append("search_index.permalink LIKE :permalink")
|
||||
else:
|
||||
conditions.append("permalink = :permalink")
|
||||
conditions.append("search_index.permalink = :permalink")
|
||||
|
||||
# Handle search item type filter
|
||||
if search_item_types:
|
||||
type_list = ", ".join(f"'{t.value}'" for t in search_item_types)
|
||||
conditions.append(f"type IN ({type_list})")
|
||||
conditions.append(f"search_index.type IN ({type_list})")
|
||||
|
||||
# Handle entity type filter using JSONB containment
|
||||
if types:
|
||||
@@ -270,19 +289,88 @@ class PostgresSearchRepository(SearchRepositoryBase):
|
||||
type_conditions = []
|
||||
for entity_type in types:
|
||||
# Create JSONB containment condition for each type
|
||||
type_conditions.append(f'metadata @> \'{{"entity_type": "{entity_type}"}}\'')
|
||||
type_conditions.append(
|
||||
f'search_index.metadata @> \'{{"entity_type": "{entity_type}"}}\''
|
||||
)
|
||||
conditions.append(f"({' OR '.join(type_conditions)})")
|
||||
|
||||
# Handle date filter
|
||||
if after_date:
|
||||
params["after_date"] = after_date
|
||||
conditions.append("created_at > :after_date")
|
||||
conditions.append("search_index.created_at > :after_date")
|
||||
# order by most recent first
|
||||
order_by_clause = ", updated_at DESC"
|
||||
order_by_clause = ", search_index.updated_at DESC"
|
||||
|
||||
# Handle structured metadata filters (frontmatter)
|
||||
if metadata_filters:
|
||||
parsed_filters = parse_metadata_filters(metadata_filters)
|
||||
from_clause = "search_index JOIN entity ON search_index.entity_id = entity.id"
|
||||
metadata_expr = "entity.entity_metadata::jsonb"
|
||||
|
||||
for idx, filt in enumerate(parsed_filters):
|
||||
path = build_postgres_json_path(filt.path_parts)
|
||||
text_expr = f"({metadata_expr} #>> '{path}')"
|
||||
json_expr = f"({metadata_expr} #> '{path}')"
|
||||
|
||||
if filt.op == "eq":
|
||||
value_param = f"meta_val_{idx}"
|
||||
params[value_param] = filt.value
|
||||
conditions.append(f"{text_expr} = :{value_param}")
|
||||
continue
|
||||
|
||||
if filt.op == "in":
|
||||
placeholders = []
|
||||
for j, val in enumerate(filt.value):
|
||||
value_param = f"meta_val_{idx}_{j}"
|
||||
params[value_param] = val
|
||||
placeholders.append(f":{value_param}")
|
||||
conditions.append(f"{text_expr} IN ({', '.join(placeholders)})")
|
||||
continue
|
||||
|
||||
if filt.op == "contains":
|
||||
import json as _json
|
||||
|
||||
base_param = f"meta_val_{idx}"
|
||||
tag_conditions = []
|
||||
# Require all values to be present
|
||||
for j, val in enumerate(filt.value):
|
||||
tag_param = f"{base_param}_{j}"
|
||||
params[tag_param] = _json.dumps([val])
|
||||
like_param = f"{base_param}_{j}_like"
|
||||
params[like_param] = f'%"{val}"%'
|
||||
like_param_single = f"{base_param}_{j}_like_single"
|
||||
params[like_param_single] = f"%'{val}'%"
|
||||
tag_conditions.append(
|
||||
f"({json_expr} @> CAST(:{tag_param} AS jsonb) "
|
||||
f"OR {text_expr} LIKE :{like_param} "
|
||||
f"OR {text_expr} LIKE :{like_param_single})"
|
||||
)
|
||||
conditions.append(" AND ".join(tag_conditions))
|
||||
continue
|
||||
|
||||
if filt.op in {"gt", "gte", "lt", "lte", "between"}:
|
||||
compare_expr = (
|
||||
f"({metadata_expr} #>> '{path}')::double precision"
|
||||
if filt.comparison == "numeric"
|
||||
else text_expr
|
||||
)
|
||||
|
||||
if filt.op == "between":
|
||||
min_param = f"meta_val_{idx}_min"
|
||||
max_param = f"meta_val_{idx}_max"
|
||||
params[min_param] = filt.value[0]
|
||||
params[max_param] = filt.value[1]
|
||||
conditions.append(f"{compare_expr} BETWEEN :{min_param} AND :{max_param}")
|
||||
else:
|
||||
value_param = f"meta_val_{idx}"
|
||||
params[value_param] = filt.value
|
||||
operator = {"gt": ">", "gte": ">=", "lt": "<", "lte": "<="}[filt.op]
|
||||
conditions.append(f"{compare_expr} {operator} :{value_param}")
|
||||
continue
|
||||
|
||||
# Always filter by project_id
|
||||
params["project_id"] = self.project_id
|
||||
conditions.append("project_id = :project_id")
|
||||
conditions.append("search_index.project_id = :project_id")
|
||||
|
||||
# set limit and offset
|
||||
params["limit"] = limit
|
||||
@@ -294,31 +382,33 @@ class PostgresSearchRepository(SearchRepositoryBase):
|
||||
# Build SQL with ts_rank() for scoring
|
||||
# Note: If no text search, score will be NULL, so we use COALESCE to default to 0
|
||||
if search_text and search_text.strip() and search_text.strip() != "*":
|
||||
score_expr = "ts_rank(textsearchable_index_col, to_tsquery('english', :text))"
|
||||
score_expr = (
|
||||
"ts_rank(search_index.textsearchable_index_col, to_tsquery('english', :text))"
|
||||
)
|
||||
else:
|
||||
score_expr = "0"
|
||||
|
||||
sql = f"""
|
||||
SELECT
|
||||
project_id,
|
||||
id,
|
||||
title,
|
||||
permalink,
|
||||
file_path,
|
||||
type,
|
||||
metadata,
|
||||
from_id,
|
||||
to_id,
|
||||
relation_type,
|
||||
entity_id,
|
||||
content_snippet,
|
||||
category,
|
||||
created_at,
|
||||
updated_at,
|
||||
search_index.project_id,
|
||||
search_index.id,
|
||||
search_index.title,
|
||||
search_index.permalink,
|
||||
search_index.file_path,
|
||||
search_index.type,
|
||||
search_index.metadata,
|
||||
search_index.from_id,
|
||||
search_index.to_id,
|
||||
search_index.relation_type,
|
||||
search_index.entity_id,
|
||||
search_index.content_snippet,
|
||||
search_index.category,
|
||||
search_index.created_at,
|
||||
search_index.updated_at,
|
||||
{score_expr} as score
|
||||
FROM search_index
|
||||
FROM {from_clause}
|
||||
WHERE {where_clause}
|
||||
ORDER BY score DESC, id ASC {order_by_clause}
|
||||
ORDER BY score DESC, search_index.id ASC {order_by_clause}
|
||||
LIMIT :limit
|
||||
OFFSET :offset
|
||||
"""
|
||||
@@ -408,7 +498,7 @@ class PostgresSearchRepository(SearchRepositoryBase):
|
||||
for row in search_index_rows:
|
||||
insert_data = row.to_insert(serialize_json=True)
|
||||
insert_data["project_id"] = self.project_id
|
||||
insert_data_list.append(insert_data)
|
||||
insert_data_list.append(_strip_nul_from_row(insert_data))
|
||||
|
||||
# Use upsert to handle race conditions during parallel indexing
|
||||
# ON CONFLICT (permalink, project_id) matches the partial unique index
|
||||
|
||||
@@ -40,6 +40,7 @@ class SearchRepository(Protocol):
|
||||
types: Optional[List[str]] = None,
|
||||
after_date: Optional[datetime] = None,
|
||||
search_item_types: Optional[List[SearchItemType]] = None,
|
||||
metadata_filters: Optional[dict] = None,
|
||||
limit: int = 10,
|
||||
offset: int = 0,
|
||||
) -> List[SearchIndexRow]:
|
||||
|
||||
@@ -78,6 +78,7 @@ class SearchRepositoryBase(ABC):
|
||||
types: Optional[List[str]] = None,
|
||||
after_date: Optional[datetime] = None,
|
||||
search_item_types: Optional[List[SearchItemType]] = None,
|
||||
metadata_filters: Optional[Dict[str, Any]] = None,
|
||||
limit: int = 10,
|
||||
offset: int = 0,
|
||||
) -> List[SearchIndexRow]:
|
||||
@@ -91,6 +92,7 @@ class SearchRepositoryBase(ABC):
|
||||
types: Filter by entity types (from metadata.entity_type)
|
||||
after_date: Filter by created_at > after_date
|
||||
search_item_types: Filter by SearchItemType (ENTITY, OBSERVATION, RELATION)
|
||||
metadata_filters: Structured frontmatter metadata filters
|
||||
limit: Maximum results to return
|
||||
offset: Number of results to skip
|
||||
|
||||
|
||||
@@ -13,6 +13,7 @@ from basic_memory import db
|
||||
from basic_memory.models.search import CREATE_SEARCH_INDEX
|
||||
from basic_memory.repository.search_index_row import SearchIndexRow
|
||||
from basic_memory.repository.search_repository_base import SearchRepositoryBase
|
||||
from basic_memory.repository.metadata_filters import parse_metadata_filters, build_sqlite_json_path
|
||||
from basic_memory.schemas.search import SearchItemType
|
||||
|
||||
|
||||
@@ -26,6 +27,17 @@ class SQLiteSearchRepository(SearchRepositoryBase):
|
||||
- Prefix wildcard matching with *
|
||||
"""
|
||||
|
||||
def __init__(self, session_maker, project_id: int):
|
||||
super().__init__(session_maker, project_id)
|
||||
self._entity_columns: set[str] | None = None
|
||||
|
||||
async def _get_entity_columns(self) -> set[str]:
|
||||
if self._entity_columns is None:
|
||||
async with db.scoped_session(self.session_maker) as session:
|
||||
result = await session.execute(text("PRAGMA table_info(entity)"))
|
||||
self._entity_columns = {row[1] for row in result.fetchall()}
|
||||
return self._entity_columns
|
||||
|
||||
async def init_search_index(self):
|
||||
"""Create FTS5 virtual table for search if it doesn't exist.
|
||||
|
||||
@@ -287,6 +299,7 @@ class SQLiteSearchRepository(SearchRepositoryBase):
|
||||
types: Optional[List[str]] = None,
|
||||
after_date: Optional[datetime] = None,
|
||||
search_item_types: Optional[List[SearchItemType]] = None,
|
||||
metadata_filters: Optional[dict] = None,
|
||||
limit: int = 10,
|
||||
offset: int = 0,
|
||||
) -> List[SearchIndexRow]:
|
||||
@@ -294,6 +307,7 @@ class SQLiteSearchRepository(SearchRepositoryBase):
|
||||
conditions = []
|
||||
params = {}
|
||||
order_by_clause = ""
|
||||
from_clause = "search_index"
|
||||
|
||||
# Handle text search for title and content
|
||||
if search_text:
|
||||
@@ -305,18 +319,20 @@ class SQLiteSearchRepository(SearchRepositoryBase):
|
||||
# Use _prepare_search_term to handle both Boolean and non-Boolean queries
|
||||
processed_text = self._prepare_search_term(search_text.strip())
|
||||
params["text"] = processed_text
|
||||
conditions.append("(title MATCH :text OR content_stems MATCH :text)")
|
||||
conditions.append(
|
||||
"(search_index.title MATCH :text OR search_index.content_stems MATCH :text)"
|
||||
)
|
||||
|
||||
# Handle title match search
|
||||
if title:
|
||||
title_text = self._prepare_search_term(title.strip(), is_prefix=False)
|
||||
params["title_text"] = title_text
|
||||
conditions.append("title MATCH :title_text")
|
||||
conditions.append("search_index.title MATCH :title_text")
|
||||
|
||||
# Handle permalink exact search
|
||||
if permalink:
|
||||
params["permalink"] = permalink
|
||||
conditions.append("permalink = :permalink")
|
||||
conditions.append("search_index.permalink = :permalink")
|
||||
|
||||
# Handle permalink match search, supports *
|
||||
if permalink_match:
|
||||
@@ -325,38 +341,122 @@ class SQLiteSearchRepository(SearchRepositoryBase):
|
||||
permalink_text = permalink_match.lower().strip()
|
||||
params["permalink"] = permalink_text
|
||||
if "*" in permalink_match:
|
||||
conditions.append("permalink GLOB :permalink")
|
||||
conditions.append("search_index.permalink GLOB :permalink")
|
||||
else:
|
||||
# For exact matches without *, we can use FTS5 MATCH
|
||||
# but only prepare the term if it doesn't look like a path
|
||||
if "/" in permalink_text:
|
||||
conditions.append("permalink = :permalink")
|
||||
conditions.append("search_index.permalink = :permalink")
|
||||
else:
|
||||
permalink_text = self._prepare_search_term(permalink_text, is_prefix=False)
|
||||
params["permalink"] = permalink_text
|
||||
conditions.append("permalink MATCH :permalink")
|
||||
conditions.append("search_index.permalink MATCH :permalink")
|
||||
|
||||
# Handle entity type filter
|
||||
if search_item_types:
|
||||
type_list = ", ".join(f"'{t.value}'" for t in search_item_types)
|
||||
conditions.append(f"type IN ({type_list})")
|
||||
conditions.append(f"search_index.type IN ({type_list})")
|
||||
|
||||
# Handle type filter
|
||||
if types:
|
||||
type_list = ", ".join(f"'{t}'" for t in types)
|
||||
conditions.append(f"json_extract(metadata, '$.entity_type') IN ({type_list})")
|
||||
conditions.append(
|
||||
f"json_extract(search_index.metadata, '$.entity_type') IN ({type_list})"
|
||||
)
|
||||
|
||||
# Handle date filter using datetime() for proper comparison
|
||||
if after_date:
|
||||
params["after_date"] = after_date
|
||||
conditions.append("datetime(created_at) > datetime(:after_date)")
|
||||
conditions.append("datetime(search_index.created_at) > datetime(:after_date)")
|
||||
|
||||
# order by most recent first
|
||||
order_by_clause = ", updated_at DESC"
|
||||
order_by_clause = ", search_index.updated_at DESC"
|
||||
|
||||
# Handle structured metadata filters (frontmatter)
|
||||
if metadata_filters:
|
||||
parsed_filters = parse_metadata_filters(metadata_filters)
|
||||
from_clause = "search_index JOIN entity ON search_index.entity_id = entity.id"
|
||||
entity_columns = await self._get_entity_columns()
|
||||
|
||||
for idx, filt in enumerate(parsed_filters):
|
||||
path_param = f"meta_path_{idx}"
|
||||
extract_expr = None
|
||||
use_tags_column = False
|
||||
|
||||
if filt.path_parts == ["status"] and "frontmatter_status" in entity_columns:
|
||||
extract_expr = "entity.frontmatter_status"
|
||||
elif filt.path_parts == ["type"] and "frontmatter_type" in entity_columns:
|
||||
extract_expr = "entity.frontmatter_type"
|
||||
elif filt.path_parts == ["tags"] and "tags_json" in entity_columns:
|
||||
extract_expr = "entity.tags_json"
|
||||
use_tags_column = True
|
||||
|
||||
if extract_expr is None:
|
||||
params[path_param] = build_sqlite_json_path(filt.path_parts)
|
||||
extract_expr = f"json_extract(entity.entity_metadata, :{path_param})"
|
||||
|
||||
if filt.op == "eq":
|
||||
value_param = f"meta_val_{idx}"
|
||||
params[value_param] = filt.value
|
||||
conditions.append(f"{extract_expr} = :{value_param}")
|
||||
continue
|
||||
|
||||
if filt.op == "in":
|
||||
placeholders = []
|
||||
for j, val in enumerate(filt.value):
|
||||
value_param = f"meta_val_{idx}_{j}"
|
||||
params[value_param] = val
|
||||
placeholders.append(f":{value_param}")
|
||||
conditions.append(f"{extract_expr} IN ({', '.join(placeholders)})")
|
||||
continue
|
||||
|
||||
if filt.op == "contains":
|
||||
tag_conditions = []
|
||||
for j, val in enumerate(filt.value):
|
||||
value_param = f"meta_val_{idx}_{j}"
|
||||
params[value_param] = val
|
||||
like_param = f"{value_param}_like"
|
||||
params[like_param] = f'%"{val}"%'
|
||||
like_param_single = f"{value_param}_like_single"
|
||||
params[like_param_single] = f"%'{val}'%"
|
||||
json_each_expr = (
|
||||
"json_each(entity.tags_json)"
|
||||
if use_tags_column
|
||||
else f"json_each(entity.entity_metadata, :{path_param})"
|
||||
)
|
||||
tag_conditions.append(
|
||||
"("
|
||||
f"EXISTS (SELECT 1 FROM {json_each_expr} WHERE value = :{value_param}) "
|
||||
f"OR {extract_expr} LIKE :{like_param} "
|
||||
f"OR {extract_expr} LIKE :{like_param_single}"
|
||||
")"
|
||||
)
|
||||
conditions.append(" AND ".join(tag_conditions))
|
||||
continue
|
||||
|
||||
if filt.op in {"gt", "gte", "lt", "lte", "between"}:
|
||||
compare_expr = (
|
||||
f"CAST({extract_expr} AS REAL)"
|
||||
if filt.comparison == "numeric"
|
||||
else extract_expr
|
||||
)
|
||||
|
||||
if filt.op == "between":
|
||||
min_param = f"meta_val_{idx}_min"
|
||||
max_param = f"meta_val_{idx}_max"
|
||||
params[min_param] = filt.value[0]
|
||||
params[max_param] = filt.value[1]
|
||||
conditions.append(f"{compare_expr} BETWEEN :{min_param} AND :{max_param}")
|
||||
else:
|
||||
value_param = f"meta_val_{idx}"
|
||||
params[value_param] = filt.value
|
||||
operator = {"gt": ">", "gte": ">=", "lt": "<", "lte": "<="}[filt.op]
|
||||
conditions.append(f"{compare_expr} {operator} :{value_param}")
|
||||
continue
|
||||
|
||||
# Always filter by project_id
|
||||
params["project_id"] = self.project_id
|
||||
conditions.append("project_id = :project_id")
|
||||
conditions.append("search_index.project_id = :project_id")
|
||||
|
||||
# set limit on search query
|
||||
params["limit"] = limit
|
||||
@@ -367,23 +467,23 @@ class SQLiteSearchRepository(SearchRepositoryBase):
|
||||
|
||||
sql = f"""
|
||||
SELECT
|
||||
project_id,
|
||||
id,
|
||||
title,
|
||||
permalink,
|
||||
file_path,
|
||||
type,
|
||||
metadata,
|
||||
from_id,
|
||||
to_id,
|
||||
relation_type,
|
||||
entity_id,
|
||||
content_snippet,
|
||||
category,
|
||||
created_at,
|
||||
updated_at,
|
||||
search_index.project_id,
|
||||
search_index.id,
|
||||
search_index.title,
|
||||
search_index.permalink,
|
||||
search_index.file_path,
|
||||
search_index.type,
|
||||
search_index.metadata,
|
||||
search_index.from_id,
|
||||
search_index.to_id,
|
||||
search_index.relation_type,
|
||||
search_index.entity_id,
|
||||
search_index.content_snippet,
|
||||
search_index.category,
|
||||
search_index.created_at,
|
||||
search_index.updated_at,
|
||||
bm25(search_index) as score
|
||||
FROM search_index
|
||||
FROM {from_clause}
|
||||
WHERE {where_clause}
|
||||
ORDER BY score ASC {order_by_clause}
|
||||
LIMIT :limit
|
||||
|
||||
@@ -24,7 +24,7 @@ from dateparser import parse
|
||||
from pydantic import BaseModel, BeforeValidator, Field, model_validator, computed_field
|
||||
|
||||
from basic_memory.config import ConfigManager
|
||||
from basic_memory.file_utils import sanitize_for_filename, sanitize_for_folder
|
||||
from basic_memory.file_utils import sanitize_for_filename, sanitize_for_directory
|
||||
from basic_memory.utils import generate_permalink
|
||||
|
||||
|
||||
@@ -227,7 +227,7 @@ class Entity(BaseModel):
|
||||
|
||||
title: str
|
||||
content: Optional[str] = None
|
||||
folder: str
|
||||
directory: str
|
||||
entity_type: EntityType = "note"
|
||||
entity_metadata: Optional[Dict] = Field(default=None, description="Optional metadata")
|
||||
content_type: ContentType = Field(
|
||||
@@ -237,7 +237,7 @@ class Entity(BaseModel):
|
||||
)
|
||||
|
||||
def __init__(self, **data):
|
||||
data["folder"] = sanitize_for_folder(data.get("folder", ""))
|
||||
data["directory"] = sanitize_for_directory(data.get("directory", ""))
|
||||
super().__init__(**data)
|
||||
|
||||
@property
|
||||
@@ -272,10 +272,12 @@ class Entity(BaseModel):
|
||||
safe_title = self.safe_title
|
||||
if self.content_type == "text/markdown":
|
||||
return (
|
||||
os.path.join(self.folder, f"{safe_title}.md") if self.folder else f"{safe_title}.md"
|
||||
os.path.join(self.directory, f"{safe_title}.md")
|
||||
if self.directory
|
||||
else f"{safe_title}.md"
|
||||
)
|
||||
else:
|
||||
return os.path.join(self.folder, safe_title) if self.folder else safe_title
|
||||
return os.path.join(self.directory, safe_title) if self.directory else safe_title
|
||||
|
||||
@property
|
||||
def permalink(self) -> Optional[Permalink]:
|
||||
|
||||
@@ -25,7 +25,7 @@ class CloudProject(BaseModel):
|
||||
|
||||
|
||||
class CloudProjectList(BaseModel):
|
||||
"""Response from /proxy/projects/projects endpoint."""
|
||||
"""Response from /proxy/v2/projects endpoint."""
|
||||
|
||||
projects: list[CloudProject] = Field(default_factory=list, description="List of cloud projects")
|
||||
|
||||
|
||||
@@ -124,6 +124,7 @@ class EntitySummary(BaseModel):
|
||||
"""Simplified entity representation."""
|
||||
|
||||
type: Literal["entity"] = "entity"
|
||||
external_id: str # UUID for v2 API routing
|
||||
entity_id: int # Database ID for v2 API consistency
|
||||
permalink: Optional[str]
|
||||
title: str
|
||||
@@ -150,8 +151,10 @@ class RelationSummary(BaseModel):
|
||||
relation_type: str
|
||||
from_entity: Optional[str] = None
|
||||
from_entity_id: Optional[int] = None # ID of source entity
|
||||
from_entity_external_id: Optional[str] = None # UUID of source entity for v2 API routing
|
||||
to_entity: Optional[str] = None
|
||||
to_entity_id: Optional[int] = None # ID of target entity
|
||||
to_entity_external_id: Optional[str] = None # UUID of target entity for v2 API routing
|
||||
created_at: Annotated[
|
||||
datetime, Field(json_schema_extra={"type": "string", "format": "date-time"})
|
||||
]
|
||||
@@ -167,6 +170,7 @@ class ObservationSummary(BaseModel):
|
||||
type: Literal["observation"] = "observation"
|
||||
observation_id: int # Database ID for v2 API consistency
|
||||
entity_id: Optional[int] = None # ID of the entity this observation belongs to
|
||||
entity_external_id: Optional[str] = None # UUID of parent entity for v2 API routing
|
||||
title: str
|
||||
file_path: str
|
||||
permalink: str
|
||||
|
||||
@@ -110,3 +110,27 @@ class MoveEntityRequest(BaseModel):
|
||||
if not v.strip():
|
||||
raise ValueError("destination_path cannot be empty or whitespace only")
|
||||
return v.strip()
|
||||
|
||||
|
||||
class MoveDirectoryRequest(BaseModel):
|
||||
"""Request schema for moving an entire directory to a new location.
|
||||
|
||||
This moves all entities within a directory to a new location while
|
||||
maintaining project consistency and updating database references.
|
||||
"""
|
||||
|
||||
source_directory: Annotated[str, MinLen(1), MaxLen(500)]
|
||||
destination_directory: Annotated[str, MinLen(1), MaxLen(500)]
|
||||
project: Optional[str] = None
|
||||
|
||||
@field_validator("source_directory", "destination_directory")
|
||||
@classmethod
|
||||
def validate_directory_path(cls, v):
|
||||
"""Ensure directory path is relative and valid."""
|
||||
if v.startswith("/"): # pragma: no cover
|
||||
raise ValueError("directory path must be relative, not absolute")
|
||||
if ".." in v: # pragma: no cover
|
||||
raise ValueError("directory path cannot contain '..' path components")
|
||||
if not v.strip(): # pragma: no cover
|
||||
raise ValueError("directory path cannot be empty or whitespace only")
|
||||
return v.strip()
|
||||
|
||||
@@ -284,3 +284,71 @@ class DeleteEntitiesResponse(SQLAlchemyModel):
|
||||
"""
|
||||
|
||||
deleted: bool
|
||||
|
||||
|
||||
class DirectoryMoveError(BaseModel):
|
||||
"""Error details for a failed file move within a directory move operation."""
|
||||
|
||||
path: str
|
||||
error: str
|
||||
|
||||
|
||||
class DirectoryDeleteError(BaseModel):
|
||||
"""Error details for a failed file delete within a directory delete operation."""
|
||||
|
||||
path: str
|
||||
error: str
|
||||
|
||||
|
||||
class DirectoryMoveResult(SQLAlchemyModel):
|
||||
"""Response schema for directory move operations.
|
||||
|
||||
Returns detailed results of moving all files within a directory,
|
||||
including counts and any errors encountered.
|
||||
|
||||
Example Response:
|
||||
{
|
||||
"total_files": 5,
|
||||
"successful_moves": 5,
|
||||
"failed_moves": 0,
|
||||
"moved_files": [
|
||||
"docs/file1.md",
|
||||
"docs/file2.md",
|
||||
"docs/subdir/file3.md"
|
||||
],
|
||||
"errors": []
|
||||
}
|
||||
"""
|
||||
|
||||
total_files: int
|
||||
successful_moves: int
|
||||
failed_moves: int
|
||||
moved_files: List[str] # List of file paths that were moved
|
||||
errors: List[DirectoryMoveError] # List of errors for failed moves
|
||||
|
||||
|
||||
class DirectoryDeleteResult(SQLAlchemyModel):
|
||||
"""Response schema for directory delete operations.
|
||||
|
||||
Returns detailed results of deleting all files within a directory,
|
||||
including counts and any errors encountered.
|
||||
|
||||
Example Response:
|
||||
{
|
||||
"total_files": 5,
|
||||
"successful_deletes": 5,
|
||||
"failed_deletes": 0,
|
||||
"deleted_files": [
|
||||
"docs/file1.md",
|
||||
"docs/file2.md",
|
||||
"docs/subdir/file3.md"
|
||||
],
|
||||
"errors": []
|
||||
}
|
||||
"""
|
||||
|
||||
total_files: int
|
||||
successful_deletes: int
|
||||
failed_deletes: int
|
||||
deleted_files: List[str] # List of file paths that were deleted
|
||||
errors: List[DirectoryDeleteError] # List of errors for failed deletes
|
||||
|
||||
@@ -6,7 +6,7 @@ The search system supports three primary modes:
|
||||
3. Full-text search across content
|
||||
"""
|
||||
|
||||
from typing import Optional, List, Union
|
||||
from typing import Optional, List, Union, Any
|
||||
from datetime import datetime
|
||||
from enum import Enum
|
||||
from pydantic import BaseModel, field_validator
|
||||
@@ -29,11 +29,15 @@ class SearchQuery(BaseModel):
|
||||
- permalink: Exact permalink match
|
||||
- permalink_match: Path pattern with *
|
||||
- text: Full-text search of title/content (supports boolean operators: AND, OR, NOT)
|
||||
- title: Title only search
|
||||
|
||||
Optionally filter results by:
|
||||
- types: Limit to specific item types
|
||||
- entity_types: Limit to specific entity types
|
||||
- types: Limit to specific entity types (frontmatter "type")
|
||||
- entity_types: Limit to search item types (entity/observation/relation)
|
||||
- after_date: Only items after date
|
||||
- metadata_filters: Structured frontmatter filters (field -> value)
|
||||
- tags: Convenience frontmatter tag filter
|
||||
- status: Convenience frontmatter status filter
|
||||
|
||||
Boolean search examples:
|
||||
- "python AND flask" - Find items with both terms
|
||||
@@ -52,6 +56,9 @@ class SearchQuery(BaseModel):
|
||||
types: Optional[List[str]] = None # Filter by type
|
||||
entity_types: Optional[List[SearchItemType]] = None # Filter by entity type
|
||||
after_date: Optional[Union[datetime, str]] = None # Time-based filter
|
||||
metadata_filters: Optional[dict[str, Any]] = None # Structured frontmatter filters
|
||||
tags: Optional[List[str]] = None # Convenience tag filter
|
||||
status: Optional[str] = None # Convenience status filter
|
||||
|
||||
@field_validator("after_date")
|
||||
@classmethod
|
||||
@@ -62,14 +69,23 @@ class SearchQuery(BaseModel):
|
||||
return v
|
||||
|
||||
def no_criteria(self) -> bool:
|
||||
text_is_empty = self.text is None or (isinstance(self.text, str) and not self.text.strip())
|
||||
metadata_is_empty = not self.metadata_filters
|
||||
tags_is_empty = not self.tags
|
||||
status_is_empty = self.status is None or (isinstance(self.status, str) and not self.status)
|
||||
types_is_empty = not self.types
|
||||
entity_types_is_empty = not self.entity_types
|
||||
return (
|
||||
self.permalink is None
|
||||
and self.permalink_match is None
|
||||
and self.title is None
|
||||
and self.text is None
|
||||
and text_is_empty
|
||||
and self.after_date is None
|
||||
and self.types is None
|
||||
and self.entity_types is None
|
||||
and types_is_empty
|
||||
and entity_types_is_empty
|
||||
and metadata_is_empty
|
||||
and tags_is_empty
|
||||
and status_is_empty
|
||||
)
|
||||
|
||||
def has_boolean_operators(self) -> bool:
|
||||
|
||||
@@ -5,6 +5,8 @@ from basic_memory.schemas.v2.entity import (
|
||||
EntityResolveResponse,
|
||||
EntityResponseV2,
|
||||
MoveEntityRequestV2,
|
||||
MoveDirectoryRequestV2,
|
||||
DeleteDirectoryRequestV2,
|
||||
ProjectResolveRequest,
|
||||
ProjectResolveResponse,
|
||||
)
|
||||
@@ -19,6 +21,8 @@ __all__ = [
|
||||
"EntityResolveResponse",
|
||||
"EntityResponseV2",
|
||||
"MoveEntityRequestV2",
|
||||
"MoveDirectoryRequestV2",
|
||||
"DeleteDirectoryRequestV2",
|
||||
"ProjectResolveRequest",
|
||||
"ProjectResolveResponse",
|
||||
"CreateResourceRequest",
|
||||
|
||||
@@ -15,6 +15,9 @@ class EntityResolveRequest(BaseModel):
|
||||
- Permalinks (e.g., "specs/search")
|
||||
- Titles (e.g., "Search Specification")
|
||||
- File paths (e.g., "specs/search.md")
|
||||
|
||||
When source_path is provided, resolution prefers notes closer to the source
|
||||
(context-aware resolution for duplicate titles).
|
||||
"""
|
||||
|
||||
identifier: str = Field(
|
||||
@@ -23,6 +26,15 @@ class EntityResolveRequest(BaseModel):
|
||||
min_length=1,
|
||||
max_length=500,
|
||||
)
|
||||
source_path: Optional[str] = Field(
|
||||
None,
|
||||
description="Path of the source file containing the link (for context-aware resolution)",
|
||||
max_length=500,
|
||||
)
|
||||
strict: bool = Field(
|
||||
False,
|
||||
description="If True, only exact matches are allowed (no fuzzy search fallback)",
|
||||
)
|
||||
|
||||
|
||||
class EntityResolveResponse(BaseModel):
|
||||
@@ -56,6 +68,42 @@ class MoveEntityRequestV2(BaseModel):
|
||||
)
|
||||
|
||||
|
||||
class MoveDirectoryRequestV2(BaseModel):
|
||||
"""V2 request schema for moving an entire directory to a new location.
|
||||
|
||||
This moves all entities within a source directory to a destination directory
|
||||
while maintaining project consistency and updating database references.
|
||||
"""
|
||||
|
||||
source_directory: str = Field(
|
||||
...,
|
||||
description="Source directory path (relative to project root)",
|
||||
min_length=1,
|
||||
max_length=500,
|
||||
)
|
||||
destination_directory: str = Field(
|
||||
...,
|
||||
description="Destination directory path (relative to project root)",
|
||||
min_length=1,
|
||||
max_length=500,
|
||||
)
|
||||
|
||||
|
||||
class DeleteDirectoryRequestV2(BaseModel):
|
||||
"""V2 request schema for deleting all entities in a directory.
|
||||
|
||||
This deletes all entities within a directory, removing them from the
|
||||
database and file system.
|
||||
"""
|
||||
|
||||
directory: str = Field(
|
||||
...,
|
||||
description="Directory path to delete (relative to project root)",
|
||||
min_length=1,
|
||||
max_length=500,
|
||||
)
|
||||
|
||||
|
||||
class EntityResponseV2(BaseModel):
|
||||
"""V2 entity response with external_id as the primary API identifier.
|
||||
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
"""Service for managing entities in the database."""
|
||||
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
from typing import List, Optional, Sequence, Tuple, Union
|
||||
|
||||
@@ -17,7 +18,7 @@ from basic_memory.file_utils import (
|
||||
dump_frontmatter,
|
||||
)
|
||||
from basic_memory.markdown import EntityMarkdown
|
||||
from basic_memory.markdown.entity_parser import EntityParser
|
||||
from basic_memory.markdown.entity_parser import EntityParser, normalize_frontmatter_metadata
|
||||
from basic_memory.markdown.utils import entity_model_from_markdown, schema_to_markdown
|
||||
from basic_memory.models import Entity as EntityModel
|
||||
from basic_memory.models import Observation, Relation
|
||||
@@ -26,8 +27,18 @@ from basic_memory.repository import ObservationRepository, RelationRepository
|
||||
from basic_memory.repository.entity_repository import EntityRepository
|
||||
from basic_memory.schemas import Entity as EntitySchema
|
||||
from basic_memory.schemas.base import Permalink
|
||||
from basic_memory.schemas.response import (
|
||||
DirectoryMoveResult,
|
||||
DirectoryMoveError,
|
||||
DirectoryDeleteResult,
|
||||
DirectoryDeleteError,
|
||||
)
|
||||
from basic_memory.services import BaseService, FileService
|
||||
from basic_memory.services.exceptions import EntityCreationError, EntityNotFoundError
|
||||
from basic_memory.services.exceptions import (
|
||||
EntityAlreadyExistsError,
|
||||
EntityCreationError,
|
||||
EntityNotFoundError,
|
||||
)
|
||||
from basic_memory.services.link_resolver import LinkResolver
|
||||
from basic_memory.services.search_service import SearchService
|
||||
from basic_memory.utils import generate_permalink
|
||||
@@ -161,6 +172,25 @@ class EntityService(BaseService[EntityModel]):
|
||||
|
||||
return permalink
|
||||
|
||||
def _build_frontmatter_markdown(
|
||||
self, title: str, entity_type: str, permalink: str
|
||||
) -> EntityMarkdown:
|
||||
"""Build a minimal EntityMarkdown object for permalink resolution."""
|
||||
from basic_memory.markdown.schemas import EntityFrontmatter
|
||||
|
||||
frontmatter_metadata = {
|
||||
"title": title,
|
||||
"type": entity_type,
|
||||
"permalink": permalink,
|
||||
}
|
||||
frontmatter_obj = EntityFrontmatter(metadata=frontmatter_metadata)
|
||||
return EntityMarkdown(
|
||||
frontmatter=frontmatter_obj,
|
||||
content="",
|
||||
observations=[],
|
||||
relations=[],
|
||||
)
|
||||
|
||||
async def create_or_update_entity(self, schema: EntitySchema) -> Tuple[EntityModel, bool]:
|
||||
"""Create new entity or update existing one.
|
||||
Returns: (entity, is_new) where is_new is True if a new entity was created
|
||||
@@ -190,8 +220,8 @@ class EntityService(BaseService[EntityModel]):
|
||||
file_path = Path(schema.file_path)
|
||||
|
||||
if await self.file_service.exists(file_path):
|
||||
raise EntityCreationError(
|
||||
f"file for entity {schema.folder}/{schema.title} already exists: {file_path}"
|
||||
raise EntityAlreadyExistsError(
|
||||
f"file for entity {schema.directory}/{schema.title} already exists: {file_path}"
|
||||
)
|
||||
|
||||
# Parse content frontmatter to check for user-specified permalink and entity_type
|
||||
@@ -204,20 +234,8 @@ class EntityService(BaseService[EntityModel]):
|
||||
schema.entity_type = content_frontmatter["type"]
|
||||
|
||||
if "permalink" in content_frontmatter:
|
||||
# Create a minimal EntityMarkdown object for permalink resolution
|
||||
from basic_memory.markdown.schemas import EntityFrontmatter
|
||||
|
||||
frontmatter_metadata = {
|
||||
"title": schema.title,
|
||||
"type": schema.entity_type,
|
||||
"permalink": content_frontmatter["permalink"],
|
||||
}
|
||||
frontmatter_obj = EntityFrontmatter(metadata=frontmatter_metadata)
|
||||
content_markdown = EntityMarkdown(
|
||||
frontmatter=frontmatter_obj,
|
||||
content="", # content not needed for permalink resolution
|
||||
observations=[],
|
||||
relations=[],
|
||||
content_markdown = self._build_frontmatter_markdown(
|
||||
schema.title, schema.entity_type, content_frontmatter["permalink"]
|
||||
)
|
||||
|
||||
# Get unique permalink (prioritizing content frontmatter) unless disabled
|
||||
@@ -242,11 +260,8 @@ class EntityService(BaseService[EntityModel]):
|
||||
content=final_content,
|
||||
)
|
||||
|
||||
# create entity
|
||||
created = await self.create_entity_from_markdown(file_path, entity_markdown)
|
||||
|
||||
# add relations
|
||||
entity = await self.update_entity_relations(created.file_path, entity_markdown)
|
||||
# create entity and relations
|
||||
entity = await self.upsert_entity_from_markdown(file_path, entity_markdown, is_new=True)
|
||||
|
||||
# Set final checksum to mark complete
|
||||
return await self.repository.update(entity.id, {"checksum": checksum})
|
||||
@@ -277,20 +292,8 @@ class EntityService(BaseService[EntityModel]):
|
||||
schema.entity_type = content_frontmatter["type"]
|
||||
|
||||
if "permalink" in content_frontmatter:
|
||||
# Create a minimal EntityMarkdown object for permalink resolution
|
||||
from basic_memory.markdown.schemas import EntityFrontmatter
|
||||
|
||||
frontmatter_metadata = {
|
||||
"title": schema.title,
|
||||
"type": schema.entity_type,
|
||||
"permalink": content_frontmatter["permalink"],
|
||||
}
|
||||
frontmatter_obj = EntityFrontmatter(metadata=frontmatter_metadata)
|
||||
content_markdown = EntityMarkdown(
|
||||
frontmatter=frontmatter_obj,
|
||||
content="", # content not needed for permalink resolution
|
||||
observations=[],
|
||||
relations=[],
|
||||
content_markdown = self._build_frontmatter_markdown(
|
||||
schema.title, schema.entity_type, content_frontmatter["permalink"]
|
||||
)
|
||||
|
||||
# Check if we need to update the permalink based on content frontmatter (unless disabled)
|
||||
@@ -327,17 +330,178 @@ class EntityService(BaseService[EntityModel]):
|
||||
content=final_content,
|
||||
)
|
||||
|
||||
# update entity in db
|
||||
entity = await self.update_entity_and_observations(file_path, entity_markdown)
|
||||
|
||||
# add relations
|
||||
await self.update_entity_relations(file_path.as_posix(), entity_markdown)
|
||||
# update entity and relations
|
||||
entity = await self.upsert_entity_from_markdown(file_path, entity_markdown, is_new=False)
|
||||
|
||||
# Set final checksum to match file
|
||||
entity = await self.repository.update(entity.id, {"checksum": checksum})
|
||||
|
||||
return entity
|
||||
|
||||
async def fast_write_entity(
|
||||
self,
|
||||
schema: EntitySchema,
|
||||
external_id: Optional[str] = None,
|
||||
) -> EntityModel:
|
||||
"""Write file and upsert a minimal entity row for fast responses."""
|
||||
logger.debug(
|
||||
"Fast-writing entity",
|
||||
title=schema.title,
|
||||
external_id=external_id,
|
||||
content_type=schema.content_type,
|
||||
)
|
||||
|
||||
# --- Identity & File Path ---
|
||||
existing = await self.repository.get_by_external_id(external_id) if external_id else None
|
||||
|
||||
# Trigger: external_id already exists
|
||||
# Why: avoid duplicate entities when title-derived paths change
|
||||
# Outcome: update in-place and keep the existing file path
|
||||
file_path = Path(existing.file_path) if existing else Path(schema.file_path)
|
||||
|
||||
if not existing and await self.file_service.exists(file_path):
|
||||
raise EntityAlreadyExistsError(
|
||||
f"file for entity {schema.directory}/{schema.title} already exists: {file_path}"
|
||||
)
|
||||
|
||||
# --- Frontmatter Overrides ---
|
||||
content_markdown = None
|
||||
if schema.content and has_frontmatter(schema.content):
|
||||
content_frontmatter = parse_frontmatter(schema.content)
|
||||
|
||||
if "type" in content_frontmatter:
|
||||
schema.entity_type = content_frontmatter["type"]
|
||||
|
||||
if "permalink" in content_frontmatter:
|
||||
content_markdown = self._build_frontmatter_markdown(
|
||||
schema.title, schema.entity_type, content_frontmatter["permalink"]
|
||||
)
|
||||
|
||||
# --- Permalink Resolution ---
|
||||
if self.app_config and self.app_config.disable_permalinks:
|
||||
schema._permalink = ""
|
||||
else:
|
||||
if existing and not (content_markdown and content_markdown.frontmatter.permalink):
|
||||
schema._permalink = existing.permalink or await self.resolve_permalink(
|
||||
file_path, skip_conflict_check=True
|
||||
)
|
||||
else:
|
||||
schema._permalink = await self.resolve_permalink(
|
||||
file_path, content_markdown, skip_conflict_check=True
|
||||
)
|
||||
|
||||
# --- File Write ---
|
||||
post = await schema_to_markdown(schema)
|
||||
final_content = dump_frontmatter(post)
|
||||
checksum = await self.file_service.write_file(file_path, final_content)
|
||||
|
||||
# --- Minimal DB Upsert ---
|
||||
metadata = normalize_frontmatter_metadata(post.metadata or {})
|
||||
entity_metadata = {k: v for k, v in metadata.items() if v is not None}
|
||||
update_data = {
|
||||
"title": schema.title,
|
||||
"entity_type": schema.entity_type,
|
||||
"file_path": file_path.as_posix(),
|
||||
"content_type": schema.content_type,
|
||||
"entity_metadata": entity_metadata or None,
|
||||
"permalink": schema.permalink,
|
||||
"checksum": checksum,
|
||||
"updated_at": datetime.now().astimezone(),
|
||||
}
|
||||
|
||||
if existing:
|
||||
updated = await self.repository.update(existing.id, update_data)
|
||||
if not updated:
|
||||
raise ValueError(f"Failed to update entity in database: {existing.id}")
|
||||
return updated
|
||||
|
||||
create_data = dict(update_data)
|
||||
if external_id is not None:
|
||||
create_data["external_id"] = external_id
|
||||
return await self.repository.create(create_data)
|
||||
|
||||
async def fast_edit_entity(
|
||||
self,
|
||||
entity: EntityModel,
|
||||
operation: str,
|
||||
content: str,
|
||||
section: Optional[str] = None,
|
||||
find_text: Optional[str] = None,
|
||||
expected_replacements: int = 1,
|
||||
) -> EntityModel:
|
||||
"""Edit an entity quickly and defer full indexing to background."""
|
||||
logger.debug(f"Fast editing entity: {entity.external_id}, operation: {operation}")
|
||||
|
||||
# --- File Edit ---
|
||||
file_path = Path(entity.file_path)
|
||||
current_content, _ = await self.file_service.read_file(file_path)
|
||||
new_content = self.apply_edit_operation(
|
||||
current_content, operation, content, section, find_text, expected_replacements
|
||||
)
|
||||
checksum = await self.file_service.write_file(file_path, new_content)
|
||||
|
||||
# --- Frontmatter Overrides ---
|
||||
update_data = {
|
||||
"checksum": checksum,
|
||||
"updated_at": datetime.now().astimezone(),
|
||||
}
|
||||
content_markdown = None
|
||||
if has_frontmatter(new_content):
|
||||
content_frontmatter = parse_frontmatter(new_content)
|
||||
|
||||
if "title" in content_frontmatter:
|
||||
update_data["title"] = content_frontmatter["title"]
|
||||
if "type" in content_frontmatter:
|
||||
update_data["entity_type"] = content_frontmatter["type"]
|
||||
|
||||
if "permalink" in content_frontmatter:
|
||||
content_markdown = self._build_frontmatter_markdown(
|
||||
update_data.get("title", entity.title),
|
||||
update_data.get("entity_type", entity.entity_type),
|
||||
content_frontmatter["permalink"],
|
||||
)
|
||||
|
||||
metadata = normalize_frontmatter_metadata(content_frontmatter or {})
|
||||
update_data["entity_metadata"] = {k: v for k, v in metadata.items() if v is not None}
|
||||
|
||||
# --- Permalink Resolution ---
|
||||
if self.app_config and self.app_config.disable_permalinks:
|
||||
update_data["permalink"] = None
|
||||
elif content_markdown and content_markdown.frontmatter.permalink:
|
||||
update_data["permalink"] = await self.resolve_permalink(
|
||||
file_path, content_markdown, skip_conflict_check=True
|
||||
)
|
||||
|
||||
updated = await self.repository.update(entity.id, update_data)
|
||||
if not updated:
|
||||
raise ValueError(f"Failed to update entity in database: {entity.id}")
|
||||
return updated
|
||||
|
||||
async def reindex_entity(self, entity_id: int) -> None:
|
||||
"""Parse file content and rebuild observations/relations/search for an entity."""
|
||||
entity = await self.repository.find_by_id(entity_id)
|
||||
if not entity:
|
||||
raise EntityNotFoundError(f"Entity not found: {entity_id}")
|
||||
|
||||
# --- Full Parse ---
|
||||
file_path = Path(entity.file_path)
|
||||
content = await self.file_service.read_file_content(file_path)
|
||||
entity_markdown = await self.entity_parser.parse_markdown_content(
|
||||
file_path=file_path,
|
||||
content=content,
|
||||
)
|
||||
|
||||
# --- DB Reindex ---
|
||||
updated = await self.upsert_entity_from_markdown(file_path, entity_markdown, is_new=False)
|
||||
checksum = await self.file_service.compute_checksum(file_path)
|
||||
updated = await self.repository.update(updated.id, {"checksum": checksum})
|
||||
if not updated:
|
||||
raise ValueError(f"Failed to update entity in database: {entity.id}")
|
||||
|
||||
# --- Search Reindex ---
|
||||
if self.search_service:
|
||||
await self.search_service.index_entity_data(updated, content=content)
|
||||
|
||||
async def delete_entity(self, permalink_or_id: str | int) -> bool:
|
||||
"""Delete entity and its file."""
|
||||
logger.debug(f"Deleting entity: {permalink_or_id}")
|
||||
@@ -459,6 +623,20 @@ class EntityService(BaseService[EntityModel]):
|
||||
db_entity,
|
||||
)
|
||||
|
||||
async def upsert_entity_from_markdown(
|
||||
self,
|
||||
file_path: Path,
|
||||
markdown: EntityMarkdown,
|
||||
*,
|
||||
is_new: bool,
|
||||
) -> EntityModel:
|
||||
"""Create/update entity and relations from parsed markdown."""
|
||||
if is_new:
|
||||
created = await self.create_entity_from_markdown(file_path, markdown)
|
||||
else:
|
||||
created = await self.update_entity_and_observations(file_path, markdown)
|
||||
return await self.update_entity_relations(created.file_path, markdown)
|
||||
|
||||
async def update_entity_relations(
|
||||
self,
|
||||
path: str,
|
||||
@@ -583,8 +761,7 @@ class EntityService(BaseService[EntityModel]):
|
||||
)
|
||||
|
||||
# Update entity and its relationships
|
||||
entity = await self.update_entity_and_observations(file_path, entity_markdown)
|
||||
await self.update_entity_relations(file_path.as_posix(), entity_markdown)
|
||||
entity = await self.upsert_entity_from_markdown(file_path, entity_markdown, is_new=False)
|
||||
|
||||
# Set final checksum to match file
|
||||
entity = await self.repository.update(entity.id, {"checksum": checksum})
|
||||
@@ -862,3 +1039,175 @@ class EntityService(BaseService[EntityModel]):
|
||||
|
||||
# Re-raise the original error with context
|
||||
raise ValueError(f"Move failed: {str(e)}") from e
|
||||
|
||||
async def move_directory(
|
||||
self,
|
||||
source_directory: str,
|
||||
destination_directory: str,
|
||||
project_config: ProjectConfig,
|
||||
app_config: BasicMemoryConfig,
|
||||
) -> DirectoryMoveResult:
|
||||
"""Move all entities in a directory to a new location.
|
||||
|
||||
This operation moves all files within a source directory to a destination
|
||||
directory, updating database records and search indexes. The operation
|
||||
tracks successes and failures individually to provide detailed feedback.
|
||||
|
||||
Args:
|
||||
source_directory: Source directory path relative to project root
|
||||
destination_directory: Destination directory path relative to project root
|
||||
project_config: Project configuration for file operations
|
||||
app_config: App configuration for permalink update settings
|
||||
|
||||
Returns:
|
||||
DirectoryMoveResult with counts and details of moved files
|
||||
|
||||
Raises:
|
||||
ValueError: If source directory is empty or destination conflicts exist
|
||||
"""
|
||||
|
||||
logger.info(f"Moving directory: {source_directory} -> {destination_directory}")
|
||||
|
||||
# Normalize directory paths (remove trailing slashes)
|
||||
source_directory = source_directory.strip("/")
|
||||
destination_directory = destination_directory.strip("/")
|
||||
|
||||
# Find all entities in the source directory
|
||||
entities = await self.repository.find_by_directory_prefix(source_directory)
|
||||
|
||||
if not entities:
|
||||
logger.warning(f"No entities found in directory: {source_directory}")
|
||||
return DirectoryMoveResult(
|
||||
total_files=0,
|
||||
successful_moves=0,
|
||||
failed_moves=0,
|
||||
moved_files=[],
|
||||
errors=[],
|
||||
)
|
||||
|
||||
# Track results
|
||||
moved_files: list[str] = []
|
||||
errors: list[DirectoryMoveError] = []
|
||||
successful_moves = 0
|
||||
failed_moves = 0
|
||||
|
||||
# Process each entity
|
||||
for entity in entities:
|
||||
try:
|
||||
# Calculate new path by replacing source prefix with destination
|
||||
old_path = entity.file_path
|
||||
# Replace only the first occurrence of the source directory prefix
|
||||
if old_path.startswith(f"{source_directory}/"):
|
||||
new_path = old_path.replace(
|
||||
f"{source_directory}/", f"{destination_directory}/", 1
|
||||
)
|
||||
else: # pragma: no cover
|
||||
# Entity is directly in the source directory (shouldn't happen with prefix match)
|
||||
new_path = f"{destination_directory}/{old_path}"
|
||||
|
||||
# Move the individual entity
|
||||
await self.move_entity(
|
||||
identifier=entity.file_path,
|
||||
destination_path=new_path,
|
||||
project_config=project_config,
|
||||
app_config=app_config,
|
||||
)
|
||||
|
||||
moved_files.append(new_path)
|
||||
successful_moves += 1
|
||||
logger.debug(f"Moved entity: {old_path} -> {new_path}")
|
||||
|
||||
except Exception as e: # pragma: no cover
|
||||
failed_moves += 1
|
||||
errors.append(DirectoryMoveError(path=entity.file_path, error=str(e)))
|
||||
logger.error(f"Failed to move entity {entity.file_path}: {e}")
|
||||
|
||||
logger.info(
|
||||
f"Directory move complete: {successful_moves} succeeded, {failed_moves} failed "
|
||||
f"(source={source_directory}, dest={destination_directory})"
|
||||
)
|
||||
|
||||
return DirectoryMoveResult(
|
||||
total_files=len(entities),
|
||||
successful_moves=successful_moves,
|
||||
failed_moves=failed_moves,
|
||||
moved_files=moved_files,
|
||||
errors=errors,
|
||||
)
|
||||
|
||||
async def delete_directory(
|
||||
self,
|
||||
directory: str,
|
||||
) -> DirectoryDeleteResult:
|
||||
"""Delete all entities in a directory.
|
||||
|
||||
This operation deletes all files within a directory, updating database
|
||||
records and search indexes. The operation tracks successes and failures
|
||||
individually to provide detailed feedback.
|
||||
|
||||
Args:
|
||||
directory: Directory path relative to project root
|
||||
|
||||
Returns:
|
||||
DirectoryDeleteResult with counts and details of deleted files
|
||||
"""
|
||||
logger.info(f"Deleting directory: {directory}")
|
||||
|
||||
# Normalize directory path (remove trailing slashes)
|
||||
directory = directory.strip("/")
|
||||
|
||||
# Find all entities in the directory
|
||||
entities = await self.repository.find_by_directory_prefix(directory)
|
||||
|
||||
if not entities:
|
||||
logger.warning(f"No entities found in directory: {directory}")
|
||||
return DirectoryDeleteResult(
|
||||
total_files=0,
|
||||
successful_deletes=0,
|
||||
failed_deletes=0,
|
||||
deleted_files=[],
|
||||
errors=[],
|
||||
)
|
||||
|
||||
# Track results
|
||||
deleted_files: list[str] = []
|
||||
errors: list[DirectoryDeleteError] = []
|
||||
successful_deletes = 0
|
||||
failed_deletes = 0
|
||||
|
||||
# Process each entity
|
||||
for entity in entities:
|
||||
try:
|
||||
file_path = entity.file_path
|
||||
|
||||
# Delete the entity (this handles file deletion and database cleanup)
|
||||
deleted = await self.delete_entity(entity.id)
|
||||
|
||||
if deleted:
|
||||
deleted_files.append(file_path)
|
||||
successful_deletes += 1
|
||||
logger.debug(f"Deleted entity: {file_path}")
|
||||
else: # pragma: no cover
|
||||
failed_deletes += 1
|
||||
errors.append(
|
||||
DirectoryDeleteError(path=file_path, error="Delete returned False")
|
||||
)
|
||||
logger.warning(f"Delete returned False for entity: {file_path}")
|
||||
|
||||
except Exception as e: # pragma: no cover
|
||||
failed_deletes += 1
|
||||
errors.append(DirectoryDeleteError(path=entity.file_path, error=str(e)))
|
||||
logger.error(f"Failed to delete entity {entity.file_path}: {e}")
|
||||
|
||||
logger.info(
|
||||
f"Directory delete complete: {successful_deletes} succeeded, {failed_deletes} failed "
|
||||
f"(directory={directory})"
|
||||
)
|
||||
|
||||
return DirectoryDeleteResult(
|
||||
total_files=len(entities),
|
||||
successful_deletes=successful_deletes,
|
||||
failed_deletes=failed_deletes,
|
||||
deleted_files=deleted_files,
|
||||
errors=errors,
|
||||
)
|
||||
|
||||
@@ -16,6 +16,12 @@ class EntityCreationError(Exception):
|
||||
pass
|
||||
|
||||
|
||||
class EntityAlreadyExistsError(EntityCreationError):
|
||||
"""Raised when an entity file already exists"""
|
||||
|
||||
pass
|
||||
|
||||
|
||||
class DirectoryOperationError(Exception):
|
||||
"""Raised when directory operations fail"""
|
||||
|
||||
|
||||
@@ -28,7 +28,11 @@ class LinkResolver:
|
||||
self.search_service = search_service
|
||||
|
||||
async def resolve_link(
|
||||
self, link_text: str, use_search: bool = True, strict: bool = False
|
||||
self,
|
||||
link_text: str,
|
||||
use_search: bool = True,
|
||||
strict: bool = False,
|
||||
source_path: Optional[str] = None,
|
||||
) -> Optional[Entity]:
|
||||
"""Resolve a markdown link to a permalink.
|
||||
|
||||
@@ -36,12 +40,69 @@ class LinkResolver:
|
||||
link_text: The link text to resolve
|
||||
use_search: Whether to use search-based fuzzy matching as fallback
|
||||
strict: If True, only exact matches are allowed (no fuzzy search fallback)
|
||||
source_path: Optional path of the source file containing the link.
|
||||
Used to prefer notes closer to the source (context-aware resolution).
|
||||
"""
|
||||
logger.trace(f"Resolving link: {link_text}")
|
||||
logger.trace(f"Resolving link: {link_text} (source: {source_path})")
|
||||
|
||||
# Clean link text and extract any alias
|
||||
clean_text, alias = self._normalize_link_text(link_text)
|
||||
|
||||
# --- Path Resolution ---
|
||||
# Note: All paths in Basic Memory are stored as POSIX strings (forward slashes)
|
||||
# for cross-platform compatibility. See entity_repository.py which normalizes
|
||||
# paths using Path().as_posix(). This allows consistent path operations here.
|
||||
|
||||
# --- Relative Path Resolution ---
|
||||
# Trigger: source_path is provided AND link contains "/"
|
||||
# Why: Resolve paths like [[nested/deep-note]] relative to source folder first
|
||||
# Outcome: [[nested/deep-note]] from testing/link-test.md → testing/nested/deep-note.md
|
||||
if source_path and "/" in clean_text:
|
||||
source_folder = source_path.rsplit("/", 1)[0] if "/" in source_path else ""
|
||||
if source_folder:
|
||||
# Construct relative path from source folder
|
||||
relative_path = f"{source_folder}/{clean_text}"
|
||||
|
||||
# Try with .md extension
|
||||
if not relative_path.endswith(".md"):
|
||||
relative_path_md = f"{relative_path}.md"
|
||||
entity = await self.entity_repository.get_by_file_path(relative_path_md)
|
||||
if entity:
|
||||
return entity
|
||||
|
||||
# Try as-is (already has extension or is a permalink)
|
||||
entity = await self.entity_repository.get_by_file_path(relative_path)
|
||||
if entity:
|
||||
return entity
|
||||
|
||||
# When source_path is provided, use context-aware resolution:
|
||||
# Check both permalink and title matches, prefer closest to source.
|
||||
# Example: [[testing]] from folder/note.md prefers folder/testing.md
|
||||
# over a root testing.md with permalink "testing".
|
||||
if source_path:
|
||||
# Gather all potential matches
|
||||
candidates: list[Entity] = []
|
||||
|
||||
# Check permalink match
|
||||
permalink_entity = await self.entity_repository.get_by_permalink(clean_text)
|
||||
if permalink_entity:
|
||||
candidates.append(permalink_entity)
|
||||
|
||||
# Check title matches
|
||||
title_entities = await self.entity_repository.get_by_title(clean_text)
|
||||
for entity in title_entities:
|
||||
# Avoid duplicates (permalink match might also be in title matches)
|
||||
if entity.id not in [c.id for c in candidates]:
|
||||
candidates.append(entity)
|
||||
|
||||
if candidates:
|
||||
if len(candidates) == 1:
|
||||
return candidates[0]
|
||||
else:
|
||||
# Multiple candidates - pick closest to source
|
||||
return self._find_closest_entity(candidates, source_path)
|
||||
|
||||
# Standard resolution (no source context): permalink first, then title
|
||||
# 1. Try exact permalink match first (most efficient)
|
||||
entity = await self.entity_repository.get_by_permalink(clean_text)
|
||||
if entity:
|
||||
@@ -51,7 +112,7 @@ class LinkResolver:
|
||||
# 2. Try exact title match
|
||||
found = await self.entity_repository.get_by_title(clean_text)
|
||||
if found:
|
||||
# Return first match if there are duplicates (consistent behavior)
|
||||
# Return first match (shortest path) if no source context
|
||||
entity = found[0]
|
||||
logger.debug(f"Found title match: {entity.title}")
|
||||
return entity
|
||||
@@ -108,7 +169,7 @@ class LinkResolver:
|
||||
if text.startswith("[[") and text.endswith("]]"):
|
||||
text = text[2:-2]
|
||||
|
||||
# Handle Obsidian-style aliases (format: [[actual|alias]])
|
||||
# Handle wiki link aliases (format: [[actual|alias]])
|
||||
alias = None
|
||||
if "|" in text:
|
||||
text, alias = text.split("|", 1)
|
||||
@@ -119,3 +180,72 @@ class LinkResolver:
|
||||
text = text.strip()
|
||||
|
||||
return text, alias
|
||||
|
||||
def _find_closest_entity(self, entities: list[Entity], source_path: str) -> Entity:
|
||||
"""Find the entity closest to the source file path.
|
||||
|
||||
Context-aware resolution: prefer notes in the same folder or closer in hierarchy.
|
||||
|
||||
Proximity Scoring Algorithm:
|
||||
- Priority 0: Same folder as source (best match)
|
||||
- Priority 1-N: Ancestor folders (N = levels up from source)
|
||||
- Priority 100+N: Descendant folders (N = levels down, deprioritized)
|
||||
- Priority 1000: Completely unrelated paths (least preferred)
|
||||
- Ties are broken by shortest absolute path (consistent behavior)
|
||||
|
||||
Args:
|
||||
entities: List of entities with the same title
|
||||
source_path: Path of the file containing the link
|
||||
|
||||
Returns:
|
||||
The entity closest to the source path
|
||||
"""
|
||||
# Extract source folder (everything before the last /)
|
||||
source_folder = source_path.rsplit("/", 1)[0] if "/" in source_path else ""
|
||||
|
||||
def path_proximity(entity: Entity) -> Tuple[int, int]:
|
||||
"""Return (proximity_score, path_length) for sorting.
|
||||
|
||||
Lower is better for both values.
|
||||
"""
|
||||
entity_path = entity.file_path
|
||||
entity_folder = entity_path.rsplit("/", 1)[0] if "/" in entity_path else ""
|
||||
|
||||
# Trigger: entity is in the same folder as source
|
||||
# Why: same-folder notes are most contextually relevant
|
||||
# Outcome: priority = 0 (best), ties broken by shortest path
|
||||
if entity_folder == source_folder:
|
||||
return (0, len(entity_path))
|
||||
|
||||
# Trigger: entity is in an ancestor folder of source
|
||||
# e.g., source is "a/b/c/file.md", entity is "a/b/note.md" -> ancestor
|
||||
# Why: ancestors are contextually relevant (shared parent context)
|
||||
# Outcome: priority = levels_up (1, 2, 3...), closer ancestors preferred
|
||||
if source_folder.startswith(entity_folder + "/") if entity_folder else source_folder:
|
||||
# Count how many levels up
|
||||
if entity_folder:
|
||||
levels_up = source_folder.count("/") - entity_folder.count("/")
|
||||
else:
|
||||
# Root level
|
||||
levels_up = source_folder.count("/") + 1
|
||||
return (levels_up, len(entity_path))
|
||||
|
||||
# Trigger: entity is in a descendant folder of source
|
||||
# e.g., source is "a/file.md", entity is "a/b/c/note.md" -> descendant
|
||||
# Why: descendants are less contextually relevant than ancestors
|
||||
# Outcome: priority = 100 + levels_down, significantly deprioritized
|
||||
if entity_folder.startswith(source_folder + "/") if source_folder else entity_folder:
|
||||
if source_folder:
|
||||
levels_down = entity_folder.count("/") - source_folder.count("/")
|
||||
else:
|
||||
# Source is at root
|
||||
levels_down = entity_folder.count("/") + 1
|
||||
return (100 + levels_down, len(entity_path))
|
||||
|
||||
# Trigger: entity is in a completely unrelated path
|
||||
# Why: no folder relationship means minimal contextual relevance
|
||||
# Outcome: priority = 1000, only selected if no related paths exist
|
||||
return (1000, len(entity_path))
|
||||
|
||||
# Sort by proximity (lower is better), then by path length (shorter is better)
|
||||
return min(entities, key=path_proximity)
|
||||
|
||||
@@ -241,8 +241,13 @@ class ProjectService:
|
||||
|
||||
project_path = project.path
|
||||
|
||||
# Check if project is default (in cloud mode, check database; in local mode, check config)
|
||||
if project.is_default or name == self.config_manager.config.default_project:
|
||||
# Check if project is default
|
||||
# In cloud mode: database is source of truth
|
||||
# In local mode: also check config file
|
||||
is_default = project.is_default
|
||||
if not self.config_manager.config.cloud_mode:
|
||||
is_default = is_default or name == self.config_manager.config.default_project
|
||||
if is_default:
|
||||
raise ValueError(f"Cannot remove the default project '{name}'") # pragma: no cover
|
||||
|
||||
# Remove from config if it exists there (may not exist in cloud mode)
|
||||
|
||||
@@ -1,8 +1,9 @@
|
||||
"""Service for search operations."""
|
||||
|
||||
import ast
|
||||
import re
|
||||
from datetime import datetime
|
||||
from typing import List, Optional, Set
|
||||
from typing import List, Optional, Set, Dict, Any
|
||||
|
||||
|
||||
from dateparser import parse
|
||||
@@ -21,6 +22,15 @@ from basic_memory.services import FileService
|
||||
MAX_CONTENT_STEMS_SIZE = 6000
|
||||
|
||||
|
||||
def _strip_nul(value: str) -> str:
|
||||
"""Strip NUL bytes that PostgreSQL text columns cannot store.
|
||||
|
||||
rclone preallocation on virtual filesystems (e.g. Google Drive File Stream)
|
||||
can pad files with \\x00 bytes. See: rclone/rclone#6801
|
||||
"""
|
||||
return value.replace("\x00", "")
|
||||
|
||||
|
||||
def _mtime_to_datetime(entity: Entity) -> datetime:
|
||||
"""Convert entity mtime (file modification time) to datetime.
|
||||
|
||||
@@ -79,6 +89,16 @@ class SearchService:
|
||||
2. Pattern match: handles * wildcards in paths
|
||||
3. Text search: full-text search across title/content
|
||||
"""
|
||||
# Support tag:<tag> shorthand by mapping to tags filter
|
||||
if query.text:
|
||||
text = query.text.strip()
|
||||
if text.lower().startswith("tag:"):
|
||||
tag_values = re.split(r"[,\s]+", text[4:].strip())
|
||||
tags = [t for t in tag_values if t]
|
||||
if tags:
|
||||
query.tags = tags
|
||||
query.text = None
|
||||
|
||||
if query.no_criteria():
|
||||
logger.debug("no criteria passed to query")
|
||||
return []
|
||||
@@ -95,6 +115,15 @@ class SearchService:
|
||||
else None
|
||||
)
|
||||
|
||||
# Merge structured metadata filters (explicit + convenience fields)
|
||||
metadata_filters: Optional[Dict[str, Any]] = None
|
||||
if query.metadata_filters or query.tags or query.status:
|
||||
metadata_filters = dict(query.metadata_filters or {})
|
||||
if query.tags:
|
||||
metadata_filters.setdefault("tags", query.tags)
|
||||
if query.status:
|
||||
metadata_filters.setdefault("status", query.status)
|
||||
|
||||
# search
|
||||
results = await self.repository.search(
|
||||
search_text=query.text,
|
||||
@@ -104,6 +133,7 @@ class SearchService:
|
||||
types=query.types,
|
||||
search_item_types=query.entity_types,
|
||||
after_date=after_date,
|
||||
metadata_filters=metadata_filters,
|
||||
limit=limit,
|
||||
offset=offset,
|
||||
)
|
||||
@@ -276,7 +306,7 @@ class SearchService:
|
||||
content = await self.file_service.read_entity_content(entity)
|
||||
if content:
|
||||
content_stems.append(content)
|
||||
content_snippet = f"{content[:250]}"
|
||||
content_snippet = _strip_nul(content[:250])
|
||||
|
||||
if entity.permalink:
|
||||
content_stems.extend(self._generate_variants(entity.permalink))
|
||||
@@ -288,7 +318,7 @@ class SearchService:
|
||||
if entity_tags:
|
||||
content_stems.extend(entity_tags)
|
||||
|
||||
entity_content_stems = "\n".join(p for p in content_stems if p and p.strip())
|
||||
entity_content_stems = _strip_nul("\n".join(p for p in content_stems if p and p.strip()))
|
||||
|
||||
# Truncate to stay under Postgres's 8KB index row limit
|
||||
if len(entity_content_stems) > MAX_CONTENT_STEMS_SIZE: # pragma: no cover
|
||||
@@ -325,8 +355,8 @@ class SearchService:
|
||||
seen_permalinks.add(obs_permalink)
|
||||
|
||||
# Index with parent entity's file path since that's where it's defined
|
||||
obs_content_stems = "\n".join(
|
||||
p for p in self._generate_variants(obs.content) if p and p.strip()
|
||||
obs_content_stems = _strip_nul(
|
||||
"\n".join(p for p in self._generate_variants(obs.content) if p and p.strip())
|
||||
)
|
||||
# Truncate to stay under Postgres's 8KB index row limit
|
||||
if len(obs_content_stems) > MAX_CONTENT_STEMS_SIZE: # pragma: no cover
|
||||
@@ -337,7 +367,7 @@ class SearchService:
|
||||
type=SearchItemType.OBSERVATION.value,
|
||||
title=f"{obs.category}: {obs.content[:100]}...",
|
||||
content_stems=obs_content_stems,
|
||||
content_snippet=obs.content,
|
||||
content_snippet=_strip_nul(obs.content),
|
||||
permalink=obs_permalink,
|
||||
file_path=entity.file_path,
|
||||
category=obs.category,
|
||||
@@ -360,8 +390,8 @@ class SearchService:
|
||||
else f"{rel.from_entity.title}"
|
||||
)
|
||||
|
||||
rel_content_stems = "\n".join(
|
||||
p for p in self._generate_variants(relation_title) if p and p.strip()
|
||||
rel_content_stems = _strip_nul(
|
||||
"\n".join(p for p in self._generate_variants(relation_title) if p and p.strip())
|
||||
)
|
||||
rows_to_index.append(
|
||||
SearchIndexRow(
|
||||
|
||||
@@ -685,19 +685,11 @@ class SyncService:
|
||||
entity_markdown.frontmatter.metadata["permalink"] = permalink
|
||||
await self.file_service.update_frontmatter(path, {"permalink": permalink})
|
||||
|
||||
# if the file is new, create an entity
|
||||
if new:
|
||||
# Create entity with final permalink
|
||||
logger.debug(f"Creating new entity from markdown, path={path}")
|
||||
await self.entity_service.create_entity_from_markdown(Path(path), entity_markdown)
|
||||
|
||||
# otherwise we need to update the entity and observations
|
||||
else:
|
||||
logger.debug(f"Updating entity from markdown, path={path}")
|
||||
await self.entity_service.update_entity_and_observations(Path(path), entity_markdown)
|
||||
|
||||
# Update relations and search index
|
||||
entity = await self.entity_service.update_entity_relations(path, entity_markdown)
|
||||
# Create/update entity and relations in one path
|
||||
logger.debug(f"{'Creating' if new else 'Updating'} entity from markdown, path={path}")
|
||||
entity = await self.entity_service.upsert_entity_from_markdown(
|
||||
Path(path), entity_markdown, is_new=new
|
||||
)
|
||||
|
||||
# After updating relations, we need to compute the checksum again
|
||||
# This is necessary for files with wikilinks to ensure consistent checksums
|
||||
|
||||
@@ -0,0 +1,126 @@
|
||||
"""Integration tests for CLI tool --format json output."""
|
||||
|
||||
import json
|
||||
|
||||
from typer.testing import CliRunner
|
||||
|
||||
from basic_memory.cli.main import app as cli_app
|
||||
|
||||
runner = CliRunner()
|
||||
|
||||
|
||||
def test_write_note_json_format(app, app_config, test_project, config_manager):
|
||||
"""Test write-note --format json returns valid JSON with expected keys."""
|
||||
result = runner.invoke(
|
||||
cli_app,
|
||||
[
|
||||
"tool",
|
||||
"write-note",
|
||||
"--title",
|
||||
"Integration Test Note",
|
||||
"--folder",
|
||||
"test-notes",
|
||||
"--content",
|
||||
"# Test\n\nThis is test content.",
|
||||
"--format",
|
||||
"json",
|
||||
],
|
||||
)
|
||||
|
||||
if result.exit_code != 0:
|
||||
print(f"STDOUT: {result.stdout}")
|
||||
print(f"STDERR: {result.stderr if hasattr(result, 'stderr') else 'N/A'}")
|
||||
print(f"Exception: {result.exception}")
|
||||
assert result.exit_code == 0
|
||||
|
||||
data = json.loads(result.stdout)
|
||||
assert data["title"] == "Integration Test Note"
|
||||
assert "permalink" in data
|
||||
assert data["content"] == "# Test\n\nThis is test content."
|
||||
assert "file_path" in data
|
||||
|
||||
|
||||
def test_read_note_json_format(app, app_config, test_project, config_manager):
|
||||
"""Test read-note --format json returns valid JSON with expected keys."""
|
||||
# First, write a note
|
||||
write_result = runner.invoke(
|
||||
cli_app,
|
||||
[
|
||||
"tool",
|
||||
"write-note",
|
||||
"--title",
|
||||
"Read Test Note",
|
||||
"--folder",
|
||||
"test-notes",
|
||||
"--content",
|
||||
"# Read Test\n\nContent to read back.",
|
||||
"--format",
|
||||
"json",
|
||||
],
|
||||
)
|
||||
assert write_result.exit_code == 0
|
||||
write_data = json.loads(write_result.stdout)
|
||||
permalink = write_data["permalink"]
|
||||
|
||||
# Now read it back
|
||||
result = runner.invoke(
|
||||
cli_app,
|
||||
["tool", "read-note", permalink, "--format", "json"],
|
||||
)
|
||||
|
||||
if result.exit_code != 0:
|
||||
print(f"STDOUT: {result.stdout}")
|
||||
print(f"Exception: {result.exception}")
|
||||
assert result.exit_code == 0
|
||||
|
||||
data = json.loads(result.stdout)
|
||||
assert data["title"] == "Read Test Note"
|
||||
assert data["permalink"] == permalink
|
||||
assert "content" in data
|
||||
assert "file_path" in data
|
||||
|
||||
|
||||
def test_recent_activity_json_format(app, app_config, test_project, config_manager, monkeypatch):
|
||||
"""Test recent-activity --format json returns valid JSON list."""
|
||||
# _recent_activity_json uses resolve_project_parameter which requires either
|
||||
# default_project_mode=True or BASIC_MEMORY_MCP_PROJECT to resolve a project
|
||||
monkeypatch.setenv("BASIC_MEMORY_MCP_PROJECT", test_project.name)
|
||||
|
||||
# Write a note to ensure there's recent activity
|
||||
write_result = runner.invoke(
|
||||
cli_app,
|
||||
[
|
||||
"tool",
|
||||
"write-note",
|
||||
"--title",
|
||||
"Activity Test Note",
|
||||
"--folder",
|
||||
"test-notes",
|
||||
"--content",
|
||||
"# Activity\n\nTest content for activity.",
|
||||
"--format",
|
||||
"json",
|
||||
],
|
||||
)
|
||||
assert write_result.exit_code == 0
|
||||
|
||||
# Get recent activity
|
||||
result = runner.invoke(
|
||||
cli_app,
|
||||
["tool", "recent-activity", "--format", "json"],
|
||||
)
|
||||
|
||||
if result.exit_code != 0:
|
||||
print(f"STDOUT: {result.stdout}")
|
||||
print(f"Exception: {result.exception}")
|
||||
assert result.exit_code == 0
|
||||
|
||||
data = json.loads(result.stdout)
|
||||
assert isinstance(data, list)
|
||||
# Should have at least one entity from the note we just wrote
|
||||
assert len(data) > 0
|
||||
item = data[0]
|
||||
assert "title" in item
|
||||
assert "permalink" in item
|
||||
assert "file_path" in item
|
||||
assert "created_at" in item
|
||||
@@ -0,0 +1,181 @@
|
||||
"""Integration tests for CLI routing flags (--local/--cloud).
|
||||
|
||||
These tests verify that the --local and --cloud flags work correctly
|
||||
across CLI commands, and that the MCP command forces local routing.
|
||||
|
||||
Note: Environment variable behavior during command execution is tested
|
||||
in unit tests (tests/cli/test_routing.py) which can properly monkeypatch
|
||||
the modules before they are imported. These integration tests focus on
|
||||
CLI behavior: flag acceptance and error handling.
|
||||
"""
|
||||
|
||||
import os
|
||||
|
||||
import pytest
|
||||
from typer.testing import CliRunner
|
||||
|
||||
from basic_memory.cli.main import app as cli_app
|
||||
|
||||
|
||||
runner = CliRunner()
|
||||
|
||||
|
||||
class TestRoutingFlagsValidation:
|
||||
"""Tests for --local/--cloud flag validation.
|
||||
|
||||
These tests verify that using both --local and --cloud together
|
||||
produces an appropriate error message.
|
||||
"""
|
||||
|
||||
def test_status_both_flags_error(self):
|
||||
"""Using both --local and --cloud should produce an error."""
|
||||
result = runner.invoke(cli_app, ["status", "--local", "--cloud"])
|
||||
# Exit code can be 1 or 2 depending on how typer handles the exception
|
||||
assert result.exit_code != 0
|
||||
assert "Cannot specify both --local and --cloud" in result.output
|
||||
|
||||
def test_project_list_both_flags_error(self):
|
||||
"""Using both --local and --cloud should produce an error."""
|
||||
result = runner.invoke(cli_app, ["project", "list", "--local", "--cloud"])
|
||||
assert result.exit_code != 0
|
||||
assert "Cannot specify both --local and --cloud" in result.output
|
||||
|
||||
def test_tool_search_both_flags_error(self):
|
||||
"""Using both --local and --cloud should produce an error."""
|
||||
result = runner.invoke(cli_app, ["tool", "search-notes", "test", "--local", "--cloud"])
|
||||
assert result.exit_code != 0
|
||||
assert "Cannot specify both --local and --cloud" in result.output
|
||||
|
||||
def test_tool_read_note_both_flags_error(self):
|
||||
"""Using both --local and --cloud should produce an error."""
|
||||
result = runner.invoke(cli_app, ["tool", "read-note", "test", "--local", "--cloud"])
|
||||
assert result.exit_code != 0
|
||||
assert "Cannot specify both --local and --cloud" in result.output
|
||||
|
||||
def test_tool_build_context_both_flags_error(self):
|
||||
"""Using both --local and --cloud should produce an error."""
|
||||
result = runner.invoke(
|
||||
cli_app, ["tool", "build-context", "memory://test", "--local", "--cloud"]
|
||||
)
|
||||
assert result.exit_code != 0
|
||||
assert "Cannot specify both --local and --cloud" in result.output
|
||||
|
||||
|
||||
class TestMcpCommandForcesLocal:
|
||||
"""Tests that the MCP command forces local routing."""
|
||||
|
||||
def test_mcp_sets_force_local_env(self, monkeypatch):
|
||||
"""MCP command should set BASIC_MEMORY_FORCE_LOCAL before server starts."""
|
||||
# Track what environment variable was set
|
||||
env_set_value = []
|
||||
|
||||
# Mock the MCP server run to capture env state without actually starting server
|
||||
import basic_memory.cli.commands.mcp as mcp_mod
|
||||
|
||||
def mock_run(*args, **kwargs):
|
||||
env_set_value.append(os.environ.get("BASIC_MEMORY_FORCE_LOCAL"))
|
||||
# Don't actually start the server
|
||||
raise SystemExit(0)
|
||||
|
||||
# Get the actual mcp_server from the module
|
||||
monkeypatch.setattr(mcp_mod.mcp_server, "run", mock_run)
|
||||
|
||||
# Also mock init_mcp_logging to avoid file operations
|
||||
monkeypatch.setattr(mcp_mod, "init_mcp_logging", lambda: None)
|
||||
|
||||
runner.invoke(cli_app, ["mcp"])
|
||||
|
||||
# Environment variable should have been set to "true"
|
||||
assert len(env_set_value) == 1
|
||||
assert env_set_value[0] == "true"
|
||||
|
||||
|
||||
class TestToolCommandsAcceptFlags:
|
||||
"""Tests that tool commands accept routing flags without parsing errors."""
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"command,args",
|
||||
[
|
||||
("search-notes", ["test query"]),
|
||||
("recent-activity", []),
|
||||
("read-note", ["test"]),
|
||||
("build-context", ["memory://test"]),
|
||||
("continue-conversation", []),
|
||||
],
|
||||
)
|
||||
def test_tool_commands_accept_local_flag(self, command, args, app_config):
|
||||
"""Tool commands should accept --local flag without parsing error."""
|
||||
full_args = ["tool", command] + args + ["--local"]
|
||||
result = runner.invoke(cli_app, full_args)
|
||||
# Should not fail due to flag parsing (No such option error)
|
||||
assert "No such option: --local" not in result.output
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"command,args",
|
||||
[
|
||||
("search-notes", ["test query"]),
|
||||
("recent-activity", []),
|
||||
("read-note", ["test"]),
|
||||
("build-context", ["memory://test"]),
|
||||
("continue-conversation", []),
|
||||
],
|
||||
)
|
||||
def test_tool_commands_accept_cloud_flag(self, command, args, app_config):
|
||||
"""Tool commands should accept --cloud flag without parsing error."""
|
||||
full_args = ["tool", command] + args + ["--cloud"]
|
||||
result = runner.invoke(cli_app, full_args)
|
||||
# Should not fail due to flag parsing (No such option error)
|
||||
assert "No such option: --cloud" not in result.output
|
||||
|
||||
|
||||
class TestProjectCommandsAcceptFlags:
|
||||
"""Tests that project commands accept routing flags without parsing errors."""
|
||||
|
||||
def test_project_list_accepts_local_flag(self, app_config):
|
||||
"""project list should accept --local flag."""
|
||||
result = runner.invoke(cli_app, ["project", "list", "--local"])
|
||||
assert "No such option: --local" not in result.output
|
||||
|
||||
def test_project_list_accepts_cloud_flag(self, app_config):
|
||||
"""project list should accept --cloud flag."""
|
||||
result = runner.invoke(cli_app, ["project", "list", "--cloud"])
|
||||
assert "No such option: --cloud" not in result.output
|
||||
|
||||
def test_project_info_accepts_local_flag(self, app_config):
|
||||
"""project info should accept --local flag."""
|
||||
result = runner.invoke(cli_app, ["project", "info", "--local"])
|
||||
assert "No such option: --local" not in result.output
|
||||
|
||||
def test_project_info_accepts_cloud_flag(self, app_config):
|
||||
"""project info should accept --cloud flag."""
|
||||
result = runner.invoke(cli_app, ["project", "info", "--cloud"])
|
||||
assert "No such option: --cloud" not in result.output
|
||||
|
||||
def test_project_default_accepts_local_flag(self, app_config):
|
||||
"""project default should accept --local flag."""
|
||||
result = runner.invoke(cli_app, ["project", "default", "test", "--local"])
|
||||
assert "No such option: --local" not in result.output
|
||||
|
||||
def test_project_sync_config_accepts_local_flag(self, app_config):
|
||||
"""project sync-config should accept --local flag."""
|
||||
result = runner.invoke(cli_app, ["project", "sync-config", "test", "--local"])
|
||||
assert "No such option: --local" not in result.output
|
||||
|
||||
def test_project_move_accepts_local_flag(self, app_config):
|
||||
"""project move should accept --local flag."""
|
||||
result = runner.invoke(cli_app, ["project", "move", "test", "/tmp/dest", "--local"])
|
||||
assert "No such option: --local" not in result.output
|
||||
|
||||
|
||||
class TestStatusCommandAcceptsFlags:
|
||||
"""Tests that status command accepts routing flags."""
|
||||
|
||||
def test_status_accepts_local_flag(self, app_config):
|
||||
"""status should accept --local flag."""
|
||||
result = runner.invoke(cli_app, ["status", "--local"])
|
||||
assert "No such option: --local" not in result.output
|
||||
|
||||
def test_status_accepts_cloud_flag(self, app_config):
|
||||
"""status should accept --cloud flag."""
|
||||
result = runner.invoke(cli_app, ["status", "--cloud"])
|
||||
assert "No such option: --cloud" not in result.output
|
||||
@@ -15,7 +15,7 @@ async def test_build_context_underscore_normalization(mcp_server, app, test_proj
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "Parent Entity",
|
||||
"folder": "testing",
|
||||
"directory": "testing",
|
||||
"content": "# Parent Entity\n\nMain entity for testing underscore relations.",
|
||||
"tags": "test,parent",
|
||||
},
|
||||
@@ -27,7 +27,7 @@ async def test_build_context_underscore_normalization(mcp_server, app, test_proj
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "Child with Underscore",
|
||||
"folder": "testing",
|
||||
"directory": "testing",
|
||||
"content": """# Child with Underscore
|
||||
|
||||
- part_of [[Parent Entity]]
|
||||
@@ -42,7 +42,7 @@ async def test_build_context_underscore_normalization(mcp_server, app, test_proj
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "Child with Hyphen",
|
||||
"folder": "testing",
|
||||
"directory": "testing",
|
||||
"content": """# Child with Hyphen
|
||||
|
||||
- part-of [[Parent Entity]]
|
||||
@@ -124,7 +124,7 @@ async def test_build_context_complex_underscore_paths(mcp_server, app, test_proj
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "workflow_manager_agent",
|
||||
"folder": "specs",
|
||||
"directory": "specs",
|
||||
"content": """# Workflow Manager Agent
|
||||
|
||||
Specification for the workflow manager agent.
|
||||
@@ -138,7 +138,7 @@ Specification for the workflow manager agent.
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "task_parser",
|
||||
"folder": "components",
|
||||
"directory": "components",
|
||||
"content": """# Task Parser
|
||||
|
||||
- part_of [[workflow_manager_agent]]
|
||||
|
||||
@@ -15,7 +15,7 @@ async def test_build_context_valid_urls(mcp_server, app, test_project):
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "URL Validation Test",
|
||||
"folder": "testing",
|
||||
"directory": "testing",
|
||||
"content": "# URL Validation Test\n\nThis note tests URL validation.",
|
||||
"tags": "test,validation",
|
||||
},
|
||||
@@ -169,7 +169,7 @@ async def test_build_context_pattern_matching_works(mcp_server, app, test_projec
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": title,
|
||||
"folder": folder,
|
||||
"directory": folder,
|
||||
"content": content,
|
||||
},
|
||||
)
|
||||
|
||||
@@ -36,7 +36,7 @@ async def test_chatgpt_search_basic(mcp_server, app, test_project):
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "Machine Learning Fundamentals",
|
||||
"folder": "ai",
|
||||
"directory": "ai",
|
||||
"content": (
|
||||
"# Machine Learning Fundamentals\n\nIntroduction to ML concepts and algorithms."
|
||||
),
|
||||
@@ -49,7 +49,7 @@ async def test_chatgpt_search_basic(mcp_server, app, test_project):
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "Deep Learning with PyTorch",
|
||||
"folder": "ai",
|
||||
"directory": "ai",
|
||||
"content": (
|
||||
"# Deep Learning with PyTorch\n\n"
|
||||
"Building neural networks using PyTorch framework."
|
||||
@@ -63,7 +63,7 @@ async def test_chatgpt_search_basic(mcp_server, app, test_project):
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "Data Visualization Guide",
|
||||
"folder": "data",
|
||||
"directory": "data",
|
||||
"content": (
|
||||
"# Data Visualization Guide\n\nCreating charts and graphs for data analysis."
|
||||
),
|
||||
@@ -127,7 +127,7 @@ async def test_chatgpt_search_with_boolean_operators(mcp_server, app, test_proje
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "Python Web Frameworks",
|
||||
"folder": "dev",
|
||||
"directory": "dev",
|
||||
"content": (
|
||||
"# Python Web Frameworks\n\nComparing Django and Flask for web development."
|
||||
),
|
||||
@@ -140,7 +140,7 @@ async def test_chatgpt_search_with_boolean_operators(mcp_server, app, test_proje
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "JavaScript Frameworks",
|
||||
"folder": "dev",
|
||||
"directory": "dev",
|
||||
"content": "# JavaScript Frameworks\n\nReact, Vue, and Angular comparison.",
|
||||
"tags": "javascript,web,frameworks",
|
||||
},
|
||||
@@ -191,7 +191,7 @@ def my_decorator(func):
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "Advanced Python Techniques",
|
||||
"folder": "programming",
|
||||
"directory": "programming",
|
||||
"content": note_content,
|
||||
"tags": "python,advanced,programming",
|
||||
},
|
||||
@@ -231,7 +231,7 @@ async def test_chatgpt_fetch_by_permalink(mcp_server, app, test_project):
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "Test Document",
|
||||
"folder": "test",
|
||||
"directory": "test",
|
||||
"content": "# Test Document\n\nThis is test content for permalink fetching.",
|
||||
"tags": "test",
|
||||
},
|
||||
@@ -301,7 +301,7 @@ async def test_chatgpt_fetch_with_empty_title(mcp_server, app, test_project):
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "untitled-note",
|
||||
"folder": "misc",
|
||||
"directory": "misc",
|
||||
"content": "This is content without a markdown header.\n\nJust plain text.",
|
||||
"tags": "misc",
|
||||
},
|
||||
@@ -337,7 +337,7 @@ async def test_chatgpt_search_pagination_default(mcp_server, app, test_project):
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": f"Test Note {i}",
|
||||
"folder": "bulk",
|
||||
"directory": "bulk",
|
||||
"content": f"# Test Note {i}\n\nThis is test content number {i}.",
|
||||
"tags": "test,bulk",
|
||||
},
|
||||
@@ -419,7 +419,7 @@ async def test_chatgpt_integration_workflow(mcp_server, app, test_project):
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": doc["title"],
|
||||
"folder": "architecture",
|
||||
"directory": "architecture",
|
||||
"content": doc["content"],
|
||||
"tags": doc["tags"],
|
||||
},
|
||||
|
||||
@@ -34,7 +34,7 @@ async def test_default_project_mode_enabled_write_note(mcp_server, app, test_pro
|
||||
"write_note",
|
||||
{
|
||||
"title": "Default Mode Test",
|
||||
"folder": "test",
|
||||
"directory": "test",
|
||||
"content": "# Default Mode Test\n\nThis should use the default project automatically.",
|
||||
"tags": "default,mode,test",
|
||||
},
|
||||
@@ -86,7 +86,7 @@ async def test_default_project_mode_explicit_override(
|
||||
"write_note",
|
||||
{
|
||||
"title": "Override Test",
|
||||
"folder": "test",
|
||||
"directory": "test",
|
||||
"content": "# Override Test\n\nThis should go to the explicitly specified project.",
|
||||
"project": other_project.name, # Explicit override
|
||||
},
|
||||
@@ -120,7 +120,7 @@ async def test_default_project_mode_disabled_requires_project(mcp_server, app, t
|
||||
"write_note",
|
||||
{
|
||||
"title": "Should Fail",
|
||||
"folder": "test",
|
||||
"directory": "test",
|
||||
"content": "# Should Fail\n\nThis should fail because no project specified.",
|
||||
},
|
||||
)
|
||||
@@ -173,7 +173,7 @@ async def test_cli_constraint_overrides_default_project_mode(
|
||||
"write_note",
|
||||
{
|
||||
"title": "CLI Constraint Test",
|
||||
"folder": "test",
|
||||
"directory": "test",
|
||||
"content": "# CLI Constraint Test\n\nThis should use CLI constrained project.",
|
||||
},
|
||||
)
|
||||
@@ -210,7 +210,7 @@ async def test_default_project_mode_read_note(mcp_server, app, test_project):
|
||||
"write_note",
|
||||
{
|
||||
"title": "Read Test Note",
|
||||
"folder": "test",
|
||||
"directory": "test",
|
||||
"content": "# Read Test Note\n\nThis note will be read back.",
|
||||
},
|
||||
)
|
||||
@@ -249,7 +249,7 @@ async def test_default_project_mode_edit_note(mcp_server, app, test_project):
|
||||
"write_note",
|
||||
{
|
||||
"title": "Edit Test Note",
|
||||
"folder": "test",
|
||||
"directory": "test",
|
||||
"content": "# Edit Test Note\n\nOriginal content.",
|
||||
},
|
||||
)
|
||||
@@ -325,7 +325,7 @@ async def test_project_resolution_hierarchy(
|
||||
"write_note",
|
||||
{
|
||||
"title": "CLI Priority Test",
|
||||
"folder": "test",
|
||||
"directory": "test",
|
||||
"content": "# CLI Priority Test",
|
||||
"project": explicit_project.name, # Should be ignored
|
||||
},
|
||||
@@ -343,7 +343,7 @@ async def test_project_resolution_hierarchy(
|
||||
"write_note",
|
||||
{
|
||||
"title": "Explicit Priority Test",
|
||||
"folder": "test",
|
||||
"directory": "test",
|
||||
"content": "# Explicit Priority Test",
|
||||
"project": explicit_project.name,
|
||||
},
|
||||
@@ -358,7 +358,7 @@ async def test_project_resolution_hierarchy(
|
||||
"write_note",
|
||||
{
|
||||
"title": "Default Priority Test",
|
||||
"folder": "test",
|
||||
"directory": "test",
|
||||
"content": "# Default Priority Test",
|
||||
# No project specified
|
||||
},
|
||||
|
||||
@@ -0,0 +1,268 @@
|
||||
"""
|
||||
Integration tests for delete_note with is_directory=True.
|
||||
|
||||
Tests the complete directory delete workflow: MCP client -> MCP server -> FastAPI -> database -> file system
|
||||
"""
|
||||
|
||||
import pytest
|
||||
from fastmcp import Client
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_delete_directory_basic(mcp_server, app, test_project):
|
||||
"""Test basic directory delete operation."""
|
||||
|
||||
async with Client(mcp_server) as client:
|
||||
# Create multiple notes in a source directory
|
||||
for i in range(3):
|
||||
await client.call_tool(
|
||||
"write_note",
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": f"Doc {i + 1}",
|
||||
"directory": "delete-dir",
|
||||
"content": f"# Doc {i + 1}\n\nContent for document {i + 1}.",
|
||||
"tags": "test,delete-dir",
|
||||
},
|
||||
)
|
||||
|
||||
# Verify notes exist
|
||||
for i in range(3):
|
||||
read_result = await client.call_tool(
|
||||
"read_note",
|
||||
{
|
||||
"project": test_project.name,
|
||||
"identifier": f"delete-dir/doc-{i + 1}",
|
||||
},
|
||||
)
|
||||
assert f"Content for document {i + 1}" in read_result.content[0].text
|
||||
|
||||
# Delete the entire directory
|
||||
delete_result = await client.call_tool(
|
||||
"delete_note",
|
||||
{
|
||||
"project": test_project.name,
|
||||
"identifier": "delete-dir",
|
||||
"is_directory": True,
|
||||
},
|
||||
)
|
||||
|
||||
# Should return successful delete message with summary
|
||||
assert len(delete_result.content) == 1
|
||||
delete_text = delete_result.content[0].text
|
||||
assert "Directory Deleted Successfully" in delete_text
|
||||
assert "Total files: 3" in delete_text
|
||||
assert "delete-dir" in delete_text
|
||||
|
||||
# Verify all notes are deleted
|
||||
for i in range(3):
|
||||
read_result = await client.call_tool(
|
||||
"read_note",
|
||||
{
|
||||
"project": test_project.name,
|
||||
"identifier": f"delete-dir/doc-{i + 1}",
|
||||
},
|
||||
)
|
||||
assert "Note Not Found" in read_result.content[0].text
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_delete_directory_nested(mcp_server, app, test_project):
|
||||
"""Test deleting a directory with nested subdirectories."""
|
||||
|
||||
async with Client(mcp_server) as client:
|
||||
# Create notes in nested structure
|
||||
directories = [
|
||||
"to-delete/2024",
|
||||
"to-delete/2024/q1",
|
||||
"to-delete/2024/q2",
|
||||
]
|
||||
|
||||
for dir_path in directories:
|
||||
await client.call_tool(
|
||||
"write_note",
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": f"Note in {dir_path.split('/')[-1]}",
|
||||
"directory": dir_path,
|
||||
"content": f"# Note\n\nContent in {dir_path}.",
|
||||
"tags": "test,nested",
|
||||
},
|
||||
)
|
||||
|
||||
# Delete the parent directory
|
||||
delete_result = await client.call_tool(
|
||||
"delete_note",
|
||||
{
|
||||
"project": test_project.name,
|
||||
"identifier": "to-delete/2024",
|
||||
"is_directory": True,
|
||||
},
|
||||
)
|
||||
|
||||
# Should delete all nested files
|
||||
assert len(delete_result.content) == 1
|
||||
delete_text = delete_result.content[0].text
|
||||
assert "Directory Deleted Successfully" in delete_text
|
||||
assert "Total files: 3" in delete_text
|
||||
|
||||
# Verify all nested notes are deleted
|
||||
read_result = await client.call_tool(
|
||||
"read_note",
|
||||
{
|
||||
"project": test_project.name,
|
||||
"identifier": "to-delete/2024/q1/note-in-q1",
|
||||
},
|
||||
)
|
||||
assert "Note Not Found" in read_result.content[0].text
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_delete_directory_empty(mcp_server, app, test_project):
|
||||
"""Test deleting an empty/non-existent directory returns appropriate message."""
|
||||
|
||||
async with Client(mcp_server) as client:
|
||||
# Try to delete a non-existent/empty directory
|
||||
delete_result = await client.call_tool(
|
||||
"delete_note",
|
||||
{
|
||||
"project": test_project.name,
|
||||
"identifier": "nonexistent-dir",
|
||||
"is_directory": True,
|
||||
},
|
||||
)
|
||||
|
||||
# Should return message about no files found
|
||||
assert len(delete_result.content) == 1
|
||||
delete_text = delete_result.content[0].text
|
||||
# Either shows "Directory Deleted Successfully" with 0 files or similar
|
||||
assert "Total files: 0" in delete_text or "0" in delete_text
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_delete_directory_single_file(mcp_server, app, test_project):
|
||||
"""Test deleting a directory with only one file."""
|
||||
|
||||
async with Client(mcp_server) as client:
|
||||
# Create single note in directory
|
||||
await client.call_tool(
|
||||
"write_note",
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "Single Note",
|
||||
"directory": "single-delete-dir",
|
||||
"content": "# Single Note\n\nOnly note in this directory.",
|
||||
"tags": "test,single",
|
||||
},
|
||||
)
|
||||
|
||||
# Delete directory
|
||||
delete_result = await client.call_tool(
|
||||
"delete_note",
|
||||
{
|
||||
"project": test_project.name,
|
||||
"identifier": "single-delete-dir",
|
||||
"is_directory": True,
|
||||
},
|
||||
)
|
||||
|
||||
assert len(delete_result.content) == 1
|
||||
delete_text = delete_result.content[0].text
|
||||
assert "Directory Deleted Successfully" in delete_text
|
||||
assert "Total files: 1" in delete_text
|
||||
|
||||
# Verify note is deleted
|
||||
read_result = await client.call_tool(
|
||||
"read_note",
|
||||
{
|
||||
"project": test_project.name,
|
||||
"identifier": "single-delete-dir/single-note",
|
||||
},
|
||||
)
|
||||
assert "Note Not Found" in read_result.content[0].text
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_delete_directory_search_no_longer_finds(mcp_server, app, test_project):
|
||||
"""Test that deleted directory contents are no longer searchable."""
|
||||
|
||||
async with Client(mcp_server) as client:
|
||||
# Create searchable note
|
||||
await client.call_tool(
|
||||
"write_note",
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "Searchable Delete Doc",
|
||||
"directory": "searchable-delete-dir",
|
||||
"content": "# Searchable Delete Doc\n\nUnique fusionreactor quantum content.",
|
||||
"tags": "search,test",
|
||||
},
|
||||
)
|
||||
|
||||
# Verify searchable before delete
|
||||
search_before = await client.call_tool(
|
||||
"search_notes",
|
||||
{
|
||||
"project": test_project.name,
|
||||
"query": "fusionreactor quantum",
|
||||
},
|
||||
)
|
||||
assert "Searchable Delete Doc" in search_before.content[0].text
|
||||
|
||||
# Delete directory
|
||||
await client.call_tool(
|
||||
"delete_note",
|
||||
{
|
||||
"project": test_project.name,
|
||||
"identifier": "searchable-delete-dir",
|
||||
"is_directory": True,
|
||||
},
|
||||
)
|
||||
|
||||
# Verify no longer searchable after delete
|
||||
search_after = await client.call_tool(
|
||||
"search_notes",
|
||||
{
|
||||
"project": test_project.name,
|
||||
"query": "fusionreactor quantum",
|
||||
},
|
||||
)
|
||||
search_text = search_after.content[0].text
|
||||
# Should not find the deleted note
|
||||
assert "Searchable Delete Doc" not in search_text or "No results" in search_text
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_delete_directory_many_files(mcp_server, app, test_project):
|
||||
"""Test deleting a directory with more than 10 files shows truncated list."""
|
||||
|
||||
async with Client(mcp_server) as client:
|
||||
# Create 12 notes in the directory
|
||||
for i in range(12):
|
||||
await client.call_tool(
|
||||
"write_note",
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": f"DeleteManyDoc {i + 1}",
|
||||
"directory": "delete-many-dir",
|
||||
"content": f"# Delete Many Doc {i + 1}\n\nContent {i + 1}.",
|
||||
"tags": "test,many",
|
||||
},
|
||||
)
|
||||
|
||||
# Delete the entire directory
|
||||
delete_result = await client.call_tool(
|
||||
"delete_note",
|
||||
{
|
||||
"project": test_project.name,
|
||||
"identifier": "delete-many-dir",
|
||||
"is_directory": True,
|
||||
},
|
||||
)
|
||||
|
||||
assert len(delete_result.content) == 1
|
||||
delete_text = delete_result.content[0].text
|
||||
assert "Directory Deleted Successfully" in delete_text
|
||||
assert "Total files: 12" in delete_text
|
||||
# Should show truncation message for >10 files
|
||||
assert "... and 2 more" in delete_text
|
||||
@@ -19,7 +19,7 @@ async def test_delete_note_by_title(mcp_server, app, test_project):
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "Note to Delete",
|
||||
"folder": "test",
|
||||
"directory": "test",
|
||||
"content": "# Note to Delete\n\nThis note will be deleted.",
|
||||
"tags": "test,delete",
|
||||
},
|
||||
@@ -77,7 +77,7 @@ async def test_delete_note_by_permalink(mcp_server, app, test_project):
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "Permalink Delete Test",
|
||||
"folder": "tests",
|
||||
"directory": "tests",
|
||||
"content": "# Permalink Delete Test\n\nTesting deletion by permalink.",
|
||||
"tags": "test,permalink",
|
||||
},
|
||||
@@ -140,7 +140,7 @@ The system handles multiple projects and users."""
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "Project Management System",
|
||||
"folder": "projects",
|
||||
"directory": "projects",
|
||||
"content": complex_content,
|
||||
"tags": "project,management,system",
|
||||
},
|
||||
@@ -208,7 +208,7 @@ async def test_delete_note_special_characters_in_title(mcp_server, app, test_pro
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": title,
|
||||
"folder": "special",
|
||||
"directory": "special",
|
||||
"content": f"# {title}\n\nContent for {title}",
|
||||
"tags": "special,characters",
|
||||
},
|
||||
@@ -275,7 +275,7 @@ async def test_delete_note_by_file_path(mcp_server, app, test_project):
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "File Path Delete",
|
||||
"folder": "docs",
|
||||
"directory": "docs",
|
||||
"content": "# File Path Delete\n\nTesting deletion by file path.",
|
||||
"tags": "test,filepath",
|
||||
},
|
||||
@@ -320,7 +320,7 @@ async def test_delete_note_case_insensitive(mcp_server, app, test_project):
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "CamelCase Note Title",
|
||||
"folder": "test",
|
||||
"directory": "test",
|
||||
"content": "# CamelCase Note Title\n\nTesting case sensitivity.",
|
||||
"tags": "test,case",
|
||||
},
|
||||
@@ -359,7 +359,7 @@ async def test_delete_multiple_notes_sequentially(mcp_server, app, test_project)
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": title,
|
||||
"folder": "batch",
|
||||
"directory": "batch",
|
||||
"content": f"# {title}\n\nContent for {title}",
|
||||
"tags": "batch,test",
|
||||
},
|
||||
@@ -421,7 +421,7 @@ This note contains various Unicode characters:
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "Unicode Test Note",
|
||||
"folder": "unicode",
|
||||
"directory": "unicode",
|
||||
"content": unicode_content,
|
||||
"tags": "unicode,test,emoji",
|
||||
},
|
||||
|
||||
@@ -19,7 +19,7 @@ async def test_edit_note_append_operation(mcp_server, app, test_project):
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "Append Test Note",
|
||||
"folder": "test",
|
||||
"directory": "test",
|
||||
"content": "# Append Test Note\n\nOriginal content here.",
|
||||
"tags": "test,append",
|
||||
},
|
||||
@@ -69,7 +69,7 @@ async def test_edit_note_prepend_operation(mcp_server, app, test_project):
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "Prepend Test Note",
|
||||
"folder": "test",
|
||||
"directory": "test",
|
||||
"content": "# Prepend Test Note\n\nExisting content.",
|
||||
"tags": "test,prepend",
|
||||
},
|
||||
@@ -122,7 +122,7 @@ async def test_edit_note_find_replace_operation(mcp_server, app, test_project):
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "Find Replace Test",
|
||||
"folder": "test",
|
||||
"directory": "test",
|
||||
"content": """# Find Replace Test
|
||||
|
||||
This is version v1.0.0 of the system.
|
||||
@@ -182,7 +182,7 @@ async def test_edit_note_replace_section_operation(mcp_server, app, test_project
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "Section Replace Test",
|
||||
"folder": "test",
|
||||
"directory": "test",
|
||||
"content": """# Section Replace Test
|
||||
|
||||
## Overview
|
||||
@@ -268,7 +268,7 @@ Current endpoints include user management."""
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "API Documentation",
|
||||
"folder": "docs",
|
||||
"directory": "docs",
|
||||
"content": complex_content,
|
||||
"tags": "api,docs",
|
||||
},
|
||||
@@ -356,7 +356,7 @@ async def test_edit_note_error_handling_text_not_found(mcp_server, app, test_pro
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "Error Test Note",
|
||||
"folder": "test",
|
||||
"directory": "test",
|
||||
"content": "# Error Test Note\n\nThis note has specific content.",
|
||||
"tags": "test,error",
|
||||
},
|
||||
@@ -394,7 +394,7 @@ async def test_edit_note_error_handling_wrong_replacement_count(mcp_server, app,
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "Count Test Note",
|
||||
"folder": "test",
|
||||
"directory": "test",
|
||||
"content": """# Count Test Note
|
||||
|
||||
The word "test" appears here.
|
||||
@@ -437,7 +437,7 @@ async def test_edit_note_invalid_operation(mcp_server, app, test_project):
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "Invalid Op Test",
|
||||
"folder": "test",
|
||||
"directory": "test",
|
||||
"content": "# Invalid Op Test\n\nSome content.",
|
||||
"tags": "test",
|
||||
},
|
||||
@@ -472,7 +472,7 @@ async def test_edit_note_missing_required_parameters(mcp_server, app, test_proje
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "Param Test Note",
|
||||
"folder": "test",
|
||||
"directory": "test",
|
||||
"content": "# Param Test Note\n\nContent here.",
|
||||
"tags": "test",
|
||||
},
|
||||
@@ -507,7 +507,7 @@ async def test_edit_note_special_characters_in_content(mcp_server, app, test_pro
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "Special Chars Test",
|
||||
"folder": "test",
|
||||
"directory": "test",
|
||||
"content": "# Special Chars Test\n\nBasic content here.",
|
||||
"tags": "test,unicode",
|
||||
},
|
||||
@@ -582,7 +582,7 @@ async def test_edit_note_using_different_identifiers(mcp_server, app, test_proje
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "Identifier Test Note",
|
||||
"folder": "docs",
|
||||
"directory": "docs",
|
||||
"content": "# Identifier Test Note\n\nOriginal content.",
|
||||
"tags": "test,identifier",
|
||||
},
|
||||
|
||||
@@ -19,7 +19,7 @@ async def test_list_directory_basic_operation(mcp_server, app, test_project):
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "Root Note",
|
||||
"folder": "", # Root folder
|
||||
"directory": "", # Root folder
|
||||
"content": "# Root Note\n\nThis is in the root directory.",
|
||||
"tags": "test,root",
|
||||
},
|
||||
@@ -30,7 +30,7 @@ async def test_list_directory_basic_operation(mcp_server, app, test_project):
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "Project Planning",
|
||||
"folder": "projects",
|
||||
"directory": "projects",
|
||||
"content": "# Project Planning\n\nPlanning document for projects.",
|
||||
"tags": "planning,project",
|
||||
},
|
||||
@@ -41,7 +41,7 @@ async def test_list_directory_basic_operation(mcp_server, app, test_project):
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "Meeting Notes",
|
||||
"folder": "meetings",
|
||||
"directory": "meetings",
|
||||
"content": "# Meeting Notes\n\nNotes from the meeting.",
|
||||
"tags": "meeting,notes",
|
||||
},
|
||||
@@ -83,7 +83,7 @@ async def test_list_directory_specific_folder(mcp_server, app, test_project):
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "Task List",
|
||||
"folder": "work",
|
||||
"directory": "work",
|
||||
"content": "# Task List\n\nWork tasks for today.",
|
||||
"tags": "work,tasks",
|
||||
},
|
||||
@@ -94,7 +94,7 @@ async def test_list_directory_specific_folder(mcp_server, app, test_project):
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "Project Alpha",
|
||||
"folder": "work/projects",
|
||||
"directory": "work/projects",
|
||||
"content": "# Project Alpha\n\nAlpha project documentation.",
|
||||
"tags": "project,alpha",
|
||||
},
|
||||
@@ -105,7 +105,7 @@ async def test_list_directory_specific_folder(mcp_server, app, test_project):
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "Daily Standup",
|
||||
"folder": "work/meetings",
|
||||
"directory": "work/meetings",
|
||||
"content": "# Daily Standup\n\nStandup meeting notes.",
|
||||
"tags": "meeting,standup",
|
||||
},
|
||||
@@ -143,7 +143,7 @@ async def test_list_directory_with_depth(mcp_server, app, test_project):
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "Deep Note",
|
||||
"folder": "research/ml/algorithms/neural-networks",
|
||||
"directory": "research/ml/algorithms/neural-networks",
|
||||
"content": "# Deep Note\n\nDeep learning research.",
|
||||
"tags": "research,ml,deep",
|
||||
},
|
||||
@@ -154,7 +154,7 @@ async def test_list_directory_with_depth(mcp_server, app, test_project):
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "ML Overview",
|
||||
"folder": "research/ml",
|
||||
"directory": "research/ml",
|
||||
"content": "# ML Overview\n\nMachine learning overview.",
|
||||
"tags": "research,ml,overview",
|
||||
},
|
||||
@@ -165,7 +165,7 @@ async def test_list_directory_with_depth(mcp_server, app, test_project):
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "Research Index",
|
||||
"folder": "research",
|
||||
"directory": "research",
|
||||
"content": "# Research Index\n\nIndex of research topics.",
|
||||
"tags": "research,index",
|
||||
},
|
||||
@@ -203,7 +203,7 @@ async def test_list_directory_with_glob_pattern(mcp_server, app, test_project):
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "Meeting 2025-01-15",
|
||||
"folder": "meetings",
|
||||
"directory": "meetings",
|
||||
"content": "# Meeting 2025-01-15\n\nMonday meeting notes.",
|
||||
"tags": "meeting,january",
|
||||
},
|
||||
@@ -214,7 +214,7 @@ async def test_list_directory_with_glob_pattern(mcp_server, app, test_project):
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "Meeting 2025-01-22",
|
||||
"folder": "meetings",
|
||||
"directory": "meetings",
|
||||
"content": "# Meeting 2025-01-22\n\nMonday meeting notes.",
|
||||
"tags": "meeting,january",
|
||||
},
|
||||
@@ -225,7 +225,7 @@ async def test_list_directory_with_glob_pattern(mcp_server, app, test_project):
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "Project Status",
|
||||
"folder": "meetings",
|
||||
"directory": "meetings",
|
||||
"content": "# Project Status\n\nProject status update.",
|
||||
"tags": "meeting,project",
|
||||
},
|
||||
@@ -285,7 +285,7 @@ async def test_list_directory_glob_no_matches(mcp_server, app, test_project):
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "Document One",
|
||||
"folder": "docs",
|
||||
"directory": "docs",
|
||||
"content": "# Document One\n\nFirst document.",
|
||||
"tags": "doc",
|
||||
},
|
||||
@@ -320,7 +320,7 @@ async def test_list_directory_various_file_types(mcp_server, app, test_project):
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "Simple Note",
|
||||
"folder": "mixed",
|
||||
"directory": "mixed",
|
||||
"content": "# Simple Note\n\nA simple note.",
|
||||
"tags": "simple",
|
||||
},
|
||||
@@ -331,7 +331,7 @@ async def test_list_directory_various_file_types(mcp_server, app, test_project):
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "Complex Document with Long Title",
|
||||
"folder": "mixed",
|
||||
"directory": "mixed",
|
||||
"content": "# Complex Document with Long Title\n\nA more complex document.",
|
||||
"tags": "complex,long",
|
||||
},
|
||||
@@ -369,7 +369,7 @@ async def test_list_directory_default_parameters(mcp_server, app, test_project):
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "Default Test",
|
||||
"folder": "default-test",
|
||||
"directory": "default-test",
|
||||
"content": "# Default Test\n\nTesting default parameters.",
|
||||
"tags": "default",
|
||||
},
|
||||
@@ -401,7 +401,7 @@ async def test_list_directory_deep_recursion(mcp_server, app, test_project):
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "Level 5 Note",
|
||||
"folder": "level1/level2/level3/level4/level5",
|
||||
"directory": "level1/level2/level3/level4/level5",
|
||||
"content": "# Level 5 Note\n\nVery deep note.",
|
||||
"tags": "deep,level5",
|
||||
},
|
||||
@@ -412,7 +412,7 @@ async def test_list_directory_deep_recursion(mcp_server, app, test_project):
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "Level 3 Note",
|
||||
"folder": "level1/level2/level3",
|
||||
"directory": "level1/level2/level3",
|
||||
"content": "# Level 3 Note\n\nMid-level note.",
|
||||
"tags": "medium,level3",
|
||||
},
|
||||
@@ -449,7 +449,7 @@ async def test_list_directory_complex_glob_patterns(mcp_server, app, test_projec
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "Project Alpha Plan",
|
||||
"folder": "patterns",
|
||||
"directory": "patterns",
|
||||
"content": "# Project Alpha Plan\n\nAlpha planning.",
|
||||
"tags": "project,alpha",
|
||||
},
|
||||
@@ -460,7 +460,7 @@ async def test_list_directory_complex_glob_patterns(mcp_server, app, test_projec
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "Project Beta Plan",
|
||||
"folder": "patterns",
|
||||
"directory": "patterns",
|
||||
"content": "# Project Beta Plan\n\nBeta planning.",
|
||||
"tags": "project,beta",
|
||||
},
|
||||
@@ -471,7 +471,7 @@ async def test_list_directory_complex_glob_patterns(mcp_server, app, test_projec
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "Meeting Minutes",
|
||||
"folder": "patterns",
|
||||
"directory": "patterns",
|
||||
"content": "# Meeting Minutes\n\nMeeting notes.",
|
||||
"tags": "meeting",
|
||||
},
|
||||
@@ -508,7 +508,7 @@ async def test_list_directory_dot_slash_prefix_paths(mcp_server, app, test_proje
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "Artifact One",
|
||||
"folder": "artifacts",
|
||||
"directory": "artifacts",
|
||||
"content": "# Artifact One\n\nFirst artifact document.",
|
||||
"tags": "artifact,test",
|
||||
},
|
||||
@@ -519,7 +519,7 @@ async def test_list_directory_dot_slash_prefix_paths(mcp_server, app, test_proje
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "Artifact Two",
|
||||
"folder": "artifacts",
|
||||
"directory": "artifacts",
|
||||
"content": "# Artifact Two\n\nSecond artifact document.",
|
||||
"tags": "artifact,test",
|
||||
},
|
||||
|
||||
@@ -0,0 +1,343 @@
|
||||
"""
|
||||
Integration tests for move_note with is_directory=True.
|
||||
|
||||
Tests the complete directory move workflow: MCP client -> MCP server -> FastAPI -> database -> file system
|
||||
"""
|
||||
|
||||
import pytest
|
||||
from fastmcp import Client
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_move_directory_basic(mcp_server, app, test_project):
|
||||
"""Test basic directory move operation."""
|
||||
|
||||
async with Client(mcp_server) as client:
|
||||
# Create multiple notes in a source directory
|
||||
for i in range(3):
|
||||
await client.call_tool(
|
||||
"write_note",
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": f"Doc {i + 1}",
|
||||
"directory": "source-dir",
|
||||
"content": f"# Doc {i + 1}\n\nContent for document {i + 1}.",
|
||||
"tags": "test,move-dir",
|
||||
},
|
||||
)
|
||||
|
||||
# Move the entire directory
|
||||
move_result = await client.call_tool(
|
||||
"move_note",
|
||||
{
|
||||
"project": test_project.name,
|
||||
"identifier": "source-dir",
|
||||
"destination_path": "dest-dir",
|
||||
"is_directory": True,
|
||||
},
|
||||
)
|
||||
|
||||
# Should return successful move message with summary
|
||||
assert len(move_result.content) == 1
|
||||
move_text = move_result.content[0].text
|
||||
assert "Directory Moved Successfully" in move_text
|
||||
assert "Total files: 3" in move_text
|
||||
assert "source-dir" in move_text
|
||||
assert "dest-dir" in move_text
|
||||
|
||||
# Verify all notes can be read from new locations
|
||||
for i in range(3):
|
||||
read_result = await client.call_tool(
|
||||
"read_note",
|
||||
{
|
||||
"project": test_project.name,
|
||||
"identifier": f"dest-dir/doc-{i + 1}",
|
||||
},
|
||||
)
|
||||
content = read_result.content[0].text
|
||||
assert f"Content for document {i + 1}" in content
|
||||
|
||||
# Verify original locations no longer exist
|
||||
for i in range(3):
|
||||
read_original = await client.call_tool(
|
||||
"read_note",
|
||||
{
|
||||
"project": test_project.name,
|
||||
"identifier": f"source-dir/doc-{i + 1}",
|
||||
},
|
||||
)
|
||||
assert "Note Not Found" in read_original.content[0].text
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_move_directory_nested(mcp_server, app, test_project):
|
||||
"""Test moving a directory with nested subdirectories."""
|
||||
|
||||
async with Client(mcp_server) as client:
|
||||
# Create notes in nested structure
|
||||
directories = [
|
||||
"projects/2024",
|
||||
"projects/2024/q1",
|
||||
"projects/2024/q2",
|
||||
]
|
||||
|
||||
for dir_path in directories:
|
||||
await client.call_tool(
|
||||
"write_note",
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": f"Note in {dir_path.split('/')[-1]}",
|
||||
"directory": dir_path,
|
||||
"content": f"# Note\n\nContent in {dir_path}.",
|
||||
"tags": "test,nested",
|
||||
},
|
||||
)
|
||||
|
||||
# Move the parent directory
|
||||
move_result = await client.call_tool(
|
||||
"move_note",
|
||||
{
|
||||
"project": test_project.name,
|
||||
"identifier": "projects/2024",
|
||||
"destination_path": "archive/2024",
|
||||
"is_directory": True,
|
||||
},
|
||||
)
|
||||
|
||||
# Should move all nested files
|
||||
assert len(move_result.content) == 1
|
||||
move_text = move_result.content[0].text
|
||||
assert "Directory Moved Successfully" in move_text
|
||||
assert "Total files: 3" in move_text
|
||||
|
||||
# Verify nested structure is preserved
|
||||
read_result = await client.call_tool(
|
||||
"read_note",
|
||||
{
|
||||
"project": test_project.name,
|
||||
"identifier": "archive/2024/q1/note-in-q1",
|
||||
},
|
||||
)
|
||||
assert "Content in projects/2024/q1" in read_result.content[0].text
|
||||
|
||||
read_result2 = await client.call_tool(
|
||||
"read_note",
|
||||
{
|
||||
"project": test_project.name,
|
||||
"identifier": "archive/2024/q2/note-in-q2",
|
||||
},
|
||||
)
|
||||
assert "Content in projects/2024/q2" in read_result2.content[0].text
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_move_directory_empty(mcp_server, app, test_project):
|
||||
"""Test moving an empty directory returns appropriate message."""
|
||||
|
||||
async with Client(mcp_server) as client:
|
||||
# Try to move a non-existent/empty directory
|
||||
move_result = await client.call_tool(
|
||||
"move_note",
|
||||
{
|
||||
"project": test_project.name,
|
||||
"identifier": "nonexistent-dir",
|
||||
"destination_path": "dest-dir",
|
||||
"is_directory": True,
|
||||
},
|
||||
)
|
||||
|
||||
# Should return message about no files found
|
||||
assert len(move_result.content) == 1
|
||||
move_text = move_result.content[0].text
|
||||
assert "No files found" in move_text or "0" in move_text
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_move_directory_preserves_content(mcp_server, app, test_project):
|
||||
"""Test that directory move preserves all note content including observations and relations."""
|
||||
|
||||
async with Client(mcp_server) as client:
|
||||
# Create note with complex content
|
||||
complex_content = """# Complex Note
|
||||
|
||||
## Observations
|
||||
- [feature] Important feature observation
|
||||
- [tech] Technical detail here
|
||||
|
||||
## Relations
|
||||
- relates_to [[Other Note]]
|
||||
- implements [[Specification]]
|
||||
|
||||
## Content
|
||||
Detailed content that must be preserved."""
|
||||
|
||||
await client.call_tool(
|
||||
"write_note",
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "Complex Note",
|
||||
"directory": "source-complex",
|
||||
"content": complex_content,
|
||||
"tags": "test,complex",
|
||||
},
|
||||
)
|
||||
|
||||
# Move the directory
|
||||
move_result = await client.call_tool(
|
||||
"move_note",
|
||||
{
|
||||
"project": test_project.name,
|
||||
"identifier": "source-complex",
|
||||
"destination_path": "dest-complex",
|
||||
"is_directory": True,
|
||||
},
|
||||
)
|
||||
|
||||
assert "Directory Moved Successfully" in move_result.content[0].text
|
||||
|
||||
# Verify content preservation
|
||||
read_result = await client.call_tool(
|
||||
"read_note",
|
||||
{
|
||||
"project": test_project.name,
|
||||
"identifier": "dest-complex/complex-note",
|
||||
},
|
||||
)
|
||||
|
||||
content = read_result.content[0].text
|
||||
assert "Important feature observation" in content
|
||||
assert "Technical detail here" in content
|
||||
assert "relates_to [[Other Note]]" in content
|
||||
assert "Detailed content that must be preserved" in content
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_move_directory_search_still_works(mcp_server, app, test_project):
|
||||
"""Test that moved directory contents remain searchable."""
|
||||
|
||||
async with Client(mcp_server) as client:
|
||||
# Create searchable note
|
||||
await client.call_tool(
|
||||
"write_note",
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "Searchable Doc",
|
||||
"directory": "searchable-dir",
|
||||
"content": "# Searchable Doc\n\nUnique quantum entanglement research content.",
|
||||
"tags": "search,test",
|
||||
},
|
||||
)
|
||||
|
||||
# Verify searchable before move
|
||||
search_before = await client.call_tool(
|
||||
"search_notes",
|
||||
{
|
||||
"project": test_project.name,
|
||||
"query": "quantum entanglement",
|
||||
},
|
||||
)
|
||||
assert "Searchable Doc" in search_before.content[0].text
|
||||
|
||||
# Move directory
|
||||
await client.call_tool(
|
||||
"move_note",
|
||||
{
|
||||
"project": test_project.name,
|
||||
"identifier": "searchable-dir",
|
||||
"destination_path": "moved-searchable",
|
||||
"is_directory": True,
|
||||
},
|
||||
)
|
||||
|
||||
# Verify still searchable after move
|
||||
search_after = await client.call_tool(
|
||||
"search_notes",
|
||||
{
|
||||
"project": test_project.name,
|
||||
"query": "quantum entanglement",
|
||||
},
|
||||
)
|
||||
search_text = search_after.content[0].text
|
||||
assert "quantum entanglement" in search_text
|
||||
assert "moved-searchable" in search_text or "searchable-doc" in search_text
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_move_directory_single_file(mcp_server, app, test_project):
|
||||
"""Test moving a directory with only one file."""
|
||||
|
||||
async with Client(mcp_server) as client:
|
||||
# Create single note in directory
|
||||
await client.call_tool(
|
||||
"write_note",
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "Single Note",
|
||||
"directory": "single-dir",
|
||||
"content": "# Single Note\n\nOnly note in this directory.",
|
||||
"tags": "test,single",
|
||||
},
|
||||
)
|
||||
|
||||
# Move directory
|
||||
move_result = await client.call_tool(
|
||||
"move_note",
|
||||
{
|
||||
"project": test_project.name,
|
||||
"identifier": "single-dir",
|
||||
"destination_path": "moved-single",
|
||||
"is_directory": True,
|
||||
},
|
||||
)
|
||||
|
||||
assert len(move_result.content) == 1
|
||||
move_text = move_result.content[0].text
|
||||
assert "Directory Moved Successfully" in move_text
|
||||
assert "Total files: 1" in move_text
|
||||
|
||||
# Verify note at new location
|
||||
read_result = await client.call_tool(
|
||||
"read_note",
|
||||
{
|
||||
"project": test_project.name,
|
||||
"identifier": "moved-single/single-note",
|
||||
},
|
||||
)
|
||||
assert "Only note in this directory" in read_result.content[0].text
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_move_directory_many_files(mcp_server, app, test_project):
|
||||
"""Test moving a directory with more than 10 files shows truncated list."""
|
||||
|
||||
async with Client(mcp_server) as client:
|
||||
# Create 12 notes in the directory
|
||||
for i in range(12):
|
||||
await client.call_tool(
|
||||
"write_note",
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": f"ManyDoc {i + 1}",
|
||||
"directory": "many-files-dir",
|
||||
"content": f"# Many Doc {i + 1}\n\nContent {i + 1}.",
|
||||
"tags": "test,many",
|
||||
},
|
||||
)
|
||||
|
||||
# Move the directory
|
||||
move_result = await client.call_tool(
|
||||
"move_note",
|
||||
{
|
||||
"project": test_project.name,
|
||||
"identifier": "many-files-dir",
|
||||
"destination_path": "moved-many",
|
||||
"is_directory": True,
|
||||
},
|
||||
)
|
||||
|
||||
assert len(move_result.content) == 1
|
||||
move_text = move_result.content[0].text
|
||||
assert "Directory Moved Successfully" in move_text
|
||||
assert "Total files: 12" in move_text
|
||||
# Should show truncation message for >10 files
|
||||
assert "... and 2 more" in move_text
|
||||
@@ -19,7 +19,7 @@ async def test_move_note_basic_operation(mcp_server, app, test_project):
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "Move Test Note",
|
||||
"folder": "source",
|
||||
"directory": "source",
|
||||
"content": "# Move Test Note\n\nThis note will be moved to a new location.",
|
||||
"tags": "test,move",
|
||||
},
|
||||
@@ -79,7 +79,7 @@ async def test_move_note_using_permalink(mcp_server, app, test_project):
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "Permalink Move Test",
|
||||
"folder": "test",
|
||||
"directory": "test",
|
||||
"content": "# Permalink Move Test\n\nMoving by permalink.",
|
||||
"tags": "test,permalink",
|
||||
},
|
||||
@@ -142,7 +142,7 @@ This note demonstrates moving complex content."""
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "Complex Note",
|
||||
"folder": "complex",
|
||||
"directory": "complex",
|
||||
"content": complex_content,
|
||||
"tags": "test,complex,move",
|
||||
},
|
||||
@@ -193,7 +193,7 @@ async def test_move_note_to_nested_directory(mcp_server, app, test_project):
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "Nested Move Test",
|
||||
"folder": "root",
|
||||
"directory": "root",
|
||||
"content": "# Nested Move Test\n\nThis will be moved deep.",
|
||||
"tags": "test,nested",
|
||||
},
|
||||
@@ -239,7 +239,7 @@ async def test_move_note_with_special_characters(mcp_server, app, test_project):
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "Special (Chars) & Symbols",
|
||||
"folder": "special",
|
||||
"directory": "special",
|
||||
"content": "# Special (Chars) & Symbols\n\nTesting special characters in move.",
|
||||
"tags": "test,special",
|
||||
},
|
||||
@@ -306,7 +306,7 @@ async def test_move_note_error_handling_invalid_destination(mcp_server, app, tes
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "Invalid Dest Test",
|
||||
"folder": "test",
|
||||
"directory": "test",
|
||||
"content": "# Invalid Dest Test\n\nThis move should fail.",
|
||||
"tags": "test,error",
|
||||
},
|
||||
@@ -340,7 +340,7 @@ async def test_move_note_error_handling_destination_exists(mcp_server, app, test
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "Source Note",
|
||||
"folder": "source",
|
||||
"directory": "source",
|
||||
"content": "# Source Note\n\nThis is the source.",
|
||||
"tags": "test,source",
|
||||
},
|
||||
@@ -352,7 +352,7 @@ async def test_move_note_error_handling_destination_exists(mcp_server, app, test
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "Existing Note",
|
||||
"folder": "destination",
|
||||
"directory": "destination",
|
||||
"content": "# Existing Note\n\nThis already exists.",
|
||||
"tags": "test,existing",
|
||||
},
|
||||
@@ -386,7 +386,7 @@ async def test_move_note_preserves_search_functionality(mcp_server, app, test_pr
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "Searchable Note",
|
||||
"folder": "original",
|
||||
"directory": "original",
|
||||
"content": """# Searchable Note
|
||||
|
||||
This note contains unique search terms:
|
||||
@@ -467,7 +467,7 @@ async def test_move_note_using_different_identifier_formats(mcp_server, app, tes
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "Title ID Note",
|
||||
"folder": "test",
|
||||
"directory": "test",
|
||||
"content": "# Title ID Note\n\nMove by title.",
|
||||
"tags": "test,identifier",
|
||||
},
|
||||
@@ -478,7 +478,7 @@ async def test_move_note_using_different_identifier_formats(mcp_server, app, tes
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "Permalink ID Note",
|
||||
"folder": "test",
|
||||
"directory": "test",
|
||||
"content": "# Permalink ID Note\n\nMove by permalink.",
|
||||
"tags": "test,identifier",
|
||||
},
|
||||
@@ -489,7 +489,7 @@ async def test_move_note_using_different_identifier_formats(mcp_server, app, tes
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "Folder Title Note",
|
||||
"folder": "test",
|
||||
"directory": "test",
|
||||
"content": "# Folder Title Note\n\nMove by folder/title.",
|
||||
"tags": "test,identifier",
|
||||
},
|
||||
@@ -569,7 +569,7 @@ async def test_move_note_cross_project_detection(mcp_server, app, test_project):
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "Cross Project Test Note",
|
||||
"folder": "source",
|
||||
"directory": "source",
|
||||
"content": "# Cross Project Test Note\n\nThis note is in the default project.",
|
||||
"tags": "test,cross-project",
|
||||
},
|
||||
@@ -605,7 +605,7 @@ async def test_move_note_normal_moves_still_work(mcp_server, app, test_project):
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "Normal Move Note",
|
||||
"folder": "source",
|
||||
"directory": "source",
|
||||
"content": "# Normal Move Note\n\nThis should move normally.",
|
||||
"tags": "test,normal-move",
|
||||
},
|
||||
|
||||
@@ -276,7 +276,7 @@ async def test_project_lifecycle_workflow(mcp_server, app, test_project, tmp_pat
|
||||
{
|
||||
"project": project_name,
|
||||
"title": "Lifecycle Test Note",
|
||||
"folder": "test",
|
||||
"directory": "test",
|
||||
"content": "# Lifecycle Test\\n\\nThis note tests the project lifecycle.\\n\\n- [test] Lifecycle testing",
|
||||
"tags": "lifecycle,test",
|
||||
},
|
||||
@@ -390,7 +390,7 @@ async def test_case_insensitive_project_switching(mcp_server, app, test_project,
|
||||
{
|
||||
"project": test_input, # Use different case
|
||||
"title": f"Case Test {test_input}",
|
||||
"folder": "case-test",
|
||||
"directory": "case-test",
|
||||
"content": f"# Case Test\n\nTesting with {test_input}",
|
||||
},
|
||||
)
|
||||
@@ -427,7 +427,7 @@ async def test_case_insensitive_project_operations(mcp_server, app, test_project
|
||||
{
|
||||
"project": project_name,
|
||||
"title": "Case Test Note",
|
||||
"folder": "case-test",
|
||||
"directory": "case-test",
|
||||
"content": "# Case Test Note\n\nTesting case-insensitive operations.\n\n- [test] Case insensitive switch\n- relates_to [[Another Note]]",
|
||||
"tags": "case,test",
|
||||
},
|
||||
@@ -478,7 +478,7 @@ async def test_case_insensitive_error_handling(mcp_server, app, test_project):
|
||||
{
|
||||
"project": test_case,
|
||||
"title": "Test Note",
|
||||
"folder": "test",
|
||||
"directory": "test",
|
||||
"content": "# Test\n\nTest content.",
|
||||
},
|
||||
)
|
||||
@@ -524,7 +524,7 @@ async def test_case_preservation_in_project_list(mcp_server, app, test_project,
|
||||
{
|
||||
"project": project_name, # Use exact project name
|
||||
"title": f"Test Note {project_name}",
|
||||
"folder": "test",
|
||||
"directory": "test",
|
||||
"content": f"# Test\n\nTesting {project_name}",
|
||||
},
|
||||
)
|
||||
|
||||
@@ -41,7 +41,7 @@ async def test_project_state_sync_after_default_change(
|
||||
{
|
||||
"project": "minerva",
|
||||
"title": "Test Consistency Note",
|
||||
"folder": "test",
|
||||
"directory": "test",
|
||||
"content": "# Test Note\n\nThis note tests project state consistency.\n\n- [test] Project state sync working",
|
||||
"tags": "test,consistency",
|
||||
},
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user