Compare commits

...

91 Commits

Author SHA1 Message Date
phernandez 14c1fe4e89 chore: update version to 0.15.0 for v0.15.0 release 2025-10-04 15:02:28 -05:00
phernandez 367dc6962a style: apply ruff formatting to test files
Auto-format test files for permalink collision tests.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: phernandez <paul@basicmachines.co>
2025-10-04 15:00:19 -05:00
phernandez ee18eb2fea docs: add v0.15.0 changelog entry
Comprehensive changelog for v0.15.0 release covering:
- Critical permalink collision data loss fix
- 10+ bug fixes including #330, #329, #328, #312
- 9 new features including cloud sync and subscription validation
- Platform improvements (Python 3.13, Windows, Docker)
- Enhanced testing and documentation

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: phernandez <paul@basicmachines.co>
2025-10-04 14:53:06 -05:00
phernandez 2a050edee4 fix: prevent permalink collision via strict link resolution
Fixes critical data loss bug where creating similar entity names
(e.g., "Node C") would overwrite existing entities (e.g., "Node A.md")
due to fuzzy search incorrectly matching similar file paths.

Changes:
- Add strict=True to resolve_link() calls in entity_service.py
- Disables fuzzy search fallback during entity creation/update
- Prevents false positive matches on similar paths like
  "edge-cases/Node A.md" and "edge-cases/Node C.md"

Testing:
- Added comprehensive integration test reproducing the bug scenario
- Added MCP-level permalink collision tests
- All 55 entity service tests pass
- Manual testing confirms fix prevents file overwrite

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: phernandez <paul@basicmachines.co>
2025-10-04 14:45:37 -05:00
Paul Hernandez f3b1945e4c fix: remove .env file loading from BasicMemoryConfig (#330)
Signed-off-by: phernandez <paul@basicmachines.co>
2025-10-04 01:13:06 -05:00
Paul Hernandez 16d7eddbf7 ci: Add Python 3.13 to test matrix (#331)
Signed-off-by: phernandez <paul@basicmachines.co>
2025-10-04 01:12:44 -05:00
Paul Hernandez f5a11f3911 fix: normalize underscores in memory:// URLs for build_context (#329)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Paul Hernandez <phernandez@users.noreply.github.com>
2025-10-04 00:16:06 -05:00
Paul Hernandez ee83b0e5a8 fix: simplify entity upsert to use database-level conflict resolution (#328)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude <noreply@anthropic.com>
2025-10-03 23:25:34 -05:00
Paul Hernandez a7bf42ef49 fix: Add proper datetime JSON schema format annotations for MCP validation (#312)
Signed-off-by: Claude Code <noreply@anthropic.com>
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Paul Hernandez <phernandez@users.noreply.github.com>
2025-10-03 22:47:02 -05:00
Paul Hernandez 903591384d feat: Add disable_permalinks config flag (#313)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Paul Hernandez <phernandez@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
2025-10-03 22:11:47 -05:00
Paul Hernandez 33ee1e0831 feat: integrate ignore_utils to skip .gitignored files in sync process (#314)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Paul Hernandez <phernandez@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
2025-10-03 21:39:31 -05:00
Paul Hernandez c83d567917 fix: enable WAL mode and add Windows-specific SQLite optimizations (#316)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Paul Hernandez <phernandez@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
2025-10-03 21:10:09 -05:00
Paul Hernandez ace6a0f50d feat: CLI Subscription Validation (SPEC-13 Phase 2) (#327)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude <noreply@anthropic.com>
2025-10-03 17:59:32 -05:00
Jonathan Nguyen fc38877008 Fix: Corrected dead links in README (#321)
Signed-off-by: Jonathan Nguyen <74562467+jonathan-d-nguyen@users.noreply.github.com>
2025-10-03 10:23:16 -05:00
Paul Hernandez 99a35a7fb4 feat: Cloud CLI cloud sync via rclone bisync (#322)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude <noreply@anthropic.com>
2025-10-03 10:18:43 -05:00
Paul Hernandez ea2e93d926 fix: rework lifecycle management to optimize cloud deployment (#320)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude <noreply@anthropic.com>
2025-09-28 15:11:38 -05:00
Paul Hernandez 324844a670 fix: resolve entity relations in background to prevent cold start blocking (#319)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude <noreply@anthropic.com>
2025-09-28 09:24:29 -05:00
Paul Hernandez f818702ab7 fix: enforce minimum 1-day timeframe for recent_activity to handle timezone issues (#318)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude <noreply@anthropic.com>
2025-09-27 23:37:00 -05:00
Paul Hernandez 2efd8f44e2 fix: critical cloud deployment fixes for MCP stability (#317)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude <noreply@anthropic.com>
2025-09-27 21:57:39 -05:00
Paul Hernandez 5da97e4820 feat: implement SPEC-11 API performance optimizations (#315)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude <noreply@anthropic.com>
2025-09-26 14:34:46 -05:00
Paul Hernandez 17a6733c9d fix: remove obsolete update_current_project function and --project flag reference (#310)
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Paul Hernandez <phernandez@users.noreply.github.com>
2025-09-26 11:46:33 -05:00
Drew Cain 3e168b98f3 fix: move_note without file extension (#281)
Signed-off-by: Drew Cain <groksrc@gmail.com>
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Paul Hernandez <phernandez@users.noreply.github.com>
Co-authored-by: phernandez <paul@basicmachines.co>
2025-09-26 11:41:45 -05:00
Paul Hernandez 1091e11322 fix: Make sync operations truly non-blocking with thread pool (#309)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude <noreply@anthropic.com>
2025-09-26 10:10:58 -05:00
Paul Hernandez f40ab31685 feat: chatgpt tools for search and fetch (#305)
Signed-off-by: Paul Hernandez <60959+phernandez@users.noreply.github.com>
Signed-off-by: Drew Cain <groksrc@gmail.com>
Co-authored-by: Drew Cain <groksrc@gmail.com>
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Paul Hernandez <phernandez@users.noreply.github.com>
Co-authored-by: Drew Cain <groksrc@users.noreply.github.com>
2025-09-25 11:55:56 -05:00
phernandez bcf7f40979 fix: Correct GitHub workflow conditions for org member @claude mentions
Fixed the conditional logic in claude.yml to properly handle different event types:
- Use github.event.comment.author_association for issue_comment events
- Use github.event.sender.author_association for other events
- Maintain support for all basicmachines-co org members (OWNER/MEMBER/COLLABORATOR)

This ensures @claude mentions in PR comments trigger the workflow correctly.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: phernandez <paul@basicmachines.co>
2025-09-23 10:00:51 -05:00
Paul Hernandez 8c7e29e325 chore: Update Claude Code GitHub Workflow (#308)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude <noreply@anthropic.com>
2025-09-23 09:50:38 -05:00
phernandez 84c0b36dee feat: Add comprehensive cloud mount CLI commands and documentation
This commit implements SPEC-7 Phase 4 by adding local file access capabilities
to the Basic Memory Cloud CLI, enabling users to mount their cloud files locally
for real-time editing.

New features:
- Cloud mount setup with automatic rclone installation
- Mount/unmount/status commands with three performance profiles
- Cross-platform rclone installer with package manager fallbacks
- Mount configuration management with tenant-specific credentials
- Comprehensive documentation with examples and troubleshooting

Mount profiles:
- fast: 5s sync for active development
- balanced: 10-15s sync (recommended)
- safe: 15s+ sync with conflict detection

Technical implementation:
- Uses rclone NFS mount (no FUSE dependencies)
- Tigris object storage with scoped credentials
- Bidirectional sync with configurable cache settings
- Process management and cleanup

Fixes Python module conflict by moving cloud.py commands to cloud/core_commands.py
to resolve typer CLI loading issues with cloud.py file vs cloud/ directory.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: phernandez <paul@basicmachines.co>
2025-09-22 17:47:06 -05:00
Paul Hernandez 2c5c606a39 feat: Implement cloud mount CLI commands for local file access (#306)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude <noreply@anthropic.com>
2025-09-22 14:19:25 -05:00
Paul Hernandez a1d7792bdb feat: Implement SPEC-6 Stateless Architecture for MCP Tools (#298)
Signed-off-by: phernandez <paul@basicmachines.co>
Signed-off-by: Paul Hernandez <60959+phernandez@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Drew Cain <groksrc@users.noreply.github.com>
2025-09-21 20:39:19 -05:00
phernandez 7979b4192e remove no content-encoding: none header
Signed-off-by: phernandez <paul@basicmachines.co>
2025-09-16 16:35:32 -05:00
Drew Cain 52d9b3c752 setting content-encoding to none for mcp
Signed-off-by: Drew Cain <groksrc@gmail.com>
2025-09-16 16:09:33 -05:00
Paul Hernandez e0d8aeb149 feat: Basic memory cloud upload (#296)
Signed-off-by: phernandez <paul@basicmachines.co>
Signed-off-by: Drew Cain <groksrc@gmail.com>
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Paul Hernandez <phernandez@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Drew Cain <groksrc@gmail.com>
2025-09-16 15:07:14 -05:00
Brandon Mayes 17b929446a fix: Sanitize folder names and properly join paths (#292) 2025-09-15 23:26:56 -04:00
Paul Hernandez b00e4ff5a1 fix: replace deprecated json_encoders with Pydantic V2 field serializers (#295)
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Paul Hernandez <phernandez@users.noreply.github.com>
2025-09-14 22:19:16 -05:00
Drew Cain 0499319ded fix: rename MCP prompt names to avoid slash command parsing issues (#289)
Signed-off-by: Drew Cain <groksrc@gmail.com>
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
2025-09-09 21:42:35 -05:00
jope-bm 3a6baf80fc feat: Merge Cloud auth (#291)
Signed-off-by: phernandez <paul@basicmachines.co>
Signed-off-by: Joe P <joe@basicmemory.com>
Co-authored-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude <noreply@anthropic.com>
2025-09-09 14:48:17 -06:00
jope-bm ec2fa07350 chore: apply lint and formatting fixes for 0.14.4 release (#290)
Signed-off-by: Joe P <joe@basicmemory.com>
Co-authored-by: Claude <noreply@anthropic.com>
2025-09-05 10:00:52 -06:00
Joe P 73cade27ab chore: update version to 0.14.4 for v0.14.4 release 2025-09-04 14:03:09 -06:00
Joe P 7e024a8674 fix: resolve linting errors for release preparation
- Replace bare except clauses with Exception in legal_file_inventory.py
- Remove unused variables in test files
- Prepare codebase for v0.14.4 release
2025-09-04 14:01:06 -06:00
Drew Cain 22f7bfa398 fix: Update YAML frontmatter tag formatting for Obsidian compatibility (#280)
Signed-off-by: Drew Cain <groksrc@users.noreply.github.com>
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Paul Hernandez <phernandez@users.noreply.github.com>
Co-authored-by: Paul Hernandez <60959+phernandez@users.noreply.github.com>
2025-09-04 09:57:45 -05:00
jope-bm cd7cee650f fix: complete project management special character support (#272) (#279)
Signed-off-by: Joe P <joe@basicmemory.com>
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: jope-bm <jope-bm@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
2025-08-29 10:57:13 -06:00
Paul Hernandez 105bcaa025 feat: implement non-root Docker container to fix file ownership issues (#277)
Signed-off-by: Drew Cain <groksrc@gmail.com>
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Paul Hernandez <phernandez@users.noreply.github.com>
Co-authored-by: Drew Cain <groksrc@gmail.com>
2025-08-28 22:15:14 -05:00
Brandon Mayes 74e12eb782 fix: Sanitize filenames and allow optional kebab case (#260)
Signed-off-by: Brandon Mayes <5610870+bdmayes@users.noreply.github.com>
2025-08-27 19:29:01 -04:00
Drew Cain 7a8b08d11e fix: Windows test failures and add Windows CI support (#273)
Signed-off-by: Drew Cain <groksrc@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
2025-08-25 08:58:24 -05:00
manuelbliemel 9aa40246a8 Addressed issues when running basic-memory on the Windows platform (#252)
Signed-off-by: Manuel Bliemel <manuel.bliemel@gmail.com>
2025-08-24 19:12:40 -07:00
jope-bm 7aff836c57 fix: Add ISO datetime serialization to MCP schema models (#270)
Signed-off-by: Joe P <joe@basicmemory.com>
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: jope-bm <jope-bm@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
2025-08-23 07:23:07 -06:00
jope-bm 285e96baea fix: Fix observation parsing to exclude markdown and wiki links (#269)
Signed-off-by: Joe P <joe@basicmemory.com>
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: jope-bm <jope-bm@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
2025-08-22 20:16:05 -06:00
jope-bm 2cd2a62f30 fix: Ensure all datetime operations return timezone-aware objects (#268)
Signed-off-by: Joe P <joe@basicmemory.com>
Co-authored-by: Claude <noreply@anthropic.com>
2025-08-22 13:43:55 -06:00
jope-bm f3d8d8d617 fix: Use discriminated unions for MCP schema validation in build_context (#266)
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Paul Hernandez <phernandez@users.noreply.github.com>
Co-authored-by: jope-bm <jope-bm@users.noreply.github.com>
2025-08-22 09:36:32 -06:00
jope-bm 9743fcd13e fix: Respect BASIC_MEMORY_LOG_LEVEL and BASIC_MEMORY_CONSOLE_LOGGING environment variables (#264)
Signed-off-by: Joe P <joe@basicmemory.com>
Co-authored-by: Claude <noreply@anthropic.com>
2025-08-22 07:58:19 -06:00
phernandez 65d1984a53 Update CLA.md to include copyright and license info
Signed-off-by: phernandez <paul@basicmachines.co>
2025-08-21 18:21:24 -05:00
jope-bm b814d40ab1 fix: Add project isolation to ContextService.find_related() method (#261) (#262)
Signed-off-by: Joe P <joe@basicmemory.com>
Co-authored-by: Claude <noreply@anthropic.com>
2025-08-20 20:07:04 -06:00
Paul Hernandez 2438094914 fix: handle vim atomic write DELETE events without ADD (#249)
Signed-off-by: Joe P <joe@basicmemory.com>
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Paul Hernandez <phernandez@users.noreply.github.com>
Co-authored-by: Joe P <joe@basicmemory.com>
Co-authored-by: Claude <noreply@anthropic.com>
2025-08-20 14:36:43 -05:00
jope-bm 5d74d7407c fix: Enable string-to-integer conversion for build_context depth parameter (#259)
Signed-off-by: Joe P <joe@basicmemory.com>
Co-authored-by: Claude <noreply@anthropic.com>
2025-08-20 11:22:14 -06:00
jope-bm b6aeb3217c fix: Add missing foreign key constraints for project removal (#254) (#258)
Signed-off-by: Joe P <joe@basicmemory.com>
Signed-off-by: joe@basicmemory.com
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: jope-bm <jope-bm@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
2025-08-20 08:49:07 -06:00
jope-bm 08ee7e1201 fix: Critical search index bug - prevent note disappearing on edit (#257)
Signed-off-by: Joe P <joe@basicmemory.com>
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: jope-bm <jope-bm@users.noreply.github.com>
2025-08-19 15:41:27 -06:00
phernandez 63ae9ee0e4 docs: Re-implement external documentation improvements
- Fix typo: 'enviroment' -> 'environment' in CLAUDE.md
- Update HTTP links to HTTPS in README.md
- Add comprehensive VS Code integration instructions
- Maintain correct internal link references

All improvements re-implemented by Basic Machines team for clean IP ownership.
2025-08-08 15:31:04 -05:00
phernandez 0e78751d34 revert: Remove external documentation changes for clean IP
Reverting changes by:
- Ikko Eltociear Ashimine (typo fix)
- Jason Noble (HTTPS links)
- Matias Forbord (link fix)
- Marc Baiza (VS Code instructions)

Will be re-implemented by Basic Machines team for clean IP ownership.
2025-08-08 15:26:29 -05:00
phernandez 59eae34dee fix: Update function name in error messages to use correct search_notes
Corrects error message templates to reference the actual search_notes function name for consistency.
2025-08-08 15:25:33 -05:00
phernandez b1e55e169e revert: Remove external function name fix for clean IP
Original contribution by Amadeusz Wieczorek will be re-implemented by Basic Machines team.
2025-08-08 15:24:58 -05:00
phernandez 173bff35c1 feat: Add Chinese character support to permalink generation
Preserves non-ASCII characters like Chinese in permalinks while maintaining
backward compatibility with ASCII-only processing. This re-implements
functionality that was contributed externally, now with Basic Machines authorship.
2025-08-08 15:24:23 -05:00
phernandez 629c8e47c9 revert: Remove external Chinese character fix for clean IP
Original contribution by andyxinweiminicloud will be re-implemented by Basic Machines team for clean IP ownership.
2025-08-08 15:23:28 -05:00
phernandez 9e4b8bca8f Add legal inventory documentation for IP analysis 2025-08-08 15:16:39 -05:00
Drew Cain b0cc559426 chore: update version to 0.14.3 for v0.14.3 release 2025-08-01 22:06:53 -05:00
Drew Cain 7460a938df fix: make case sensitivity test platform-aware
- Add platform detection to handle case-insensitive file systems
- Test now passes on macOS and Windows while maintaining Linux behavior
- Fixes test failure on case-insensitive file systems
2025-08-01 22:02:53 -05:00
Drew Cain 43fa5762a8 ruff checks
Signed-off-by: Drew Cain <groksrc@gmail.com>
2025-08-01 21:50:18 -05:00
Paul Hernandez fb1350b294 fix: enhance character conflict detection and error handling for sync operations (#201)
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Paul Hernandez <phernandez@users.noreply.github.com>
2025-08-01 21:35:56 -05:00
Paul Hernandez 7585a29c96 fix: replace recursive _traverse_messages with iterative approach to handle deep conversation threads (#235)
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Paul Hernandez <phernandez@users.noreply.github.com>
2025-08-01 21:34:11 -05:00
Drew Cain 752c78c379 chore: minor cleanup (#228)
Signed-off-by: Drew Cain <groksrc@gmail.com>
2025-07-31 21:32:56 -05:00
jope-bm a4a3b1b689 fix: handle missing 'name' key in memory JSON import (#241)
Signed-off-by: Joe P <joe@basicmemory.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: jope-bm <jope-bm@users.noreply.github.com>
2025-07-28 14:52:15 -06:00
jope-bm 6361574a20 fix: basic memory home env var not respected when project path is changed. (#239)
Signed-off-by: Joe P <joe@basicmemory.com>
2025-07-28 14:50:47 -06:00
jope-bm 24a1d6195d fix: path traversal security vulnerability in mcp tools (#223)
Signed-off-by: Joe P <joe@basicmemory.com>
2025-07-15 09:05:11 -06:00
jope-bm a0cf62375d docs: improve virtual environment setup instructions (#222)
Co-authored-by: Claude <noreply@anthropic.com>
2025-07-10 10:18:43 -06:00
Paul Hernandez 473f70c949 chore: Cloud auth (#213)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude <noreply@anthropic.com>
2025-07-07 21:08:25 -05:00
phernandez 2c29dcc2b2 chore: update version to 0.14.2 for v0.14.2 release 2025-07-03 17:30:40 -05:00
phernandez 448210e552 docs: add v0.14.2 changelog entry
🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-03 17:23:54 -05:00
Drew Cain 3621bb7b4d fix: MCP Error with MCP-Hub #204 (#212)
Signed-off-by: Drew Cain <groksrc@gmail.com>
2025-07-03 16:57:43 -05:00
Drew Cain f80ac0ee72 fix: replace deprecated datetime.utcnow() with timezone-aware alternatives and suppress SQLAlchemy warnings (#211)
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
2025-07-03 16:57:30 -05:00
Drew Cain 23ddf1918c chore: update version to 0.14.1 for v0.14.1 release 2025-07-01 22:08:25 -05:00
Drew Cain 2aca19aa05 chore: apply ruff formatting 2025-07-01 22:05:17 -05:00
Drew Cain 827f7cf3e3 fix: constrain fastmcp version to prevent breaking changes (#203)
Signed-off-by: Drew Cain <groksrc@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
2025-07-01 22:01:59 -05:00
Drew Cain bd4f55158b fix: Problems with MCP #190 (#202)
Signed-off-by: Drew Cain <groksrc@gmail.com>
2025-07-01 10:50:44 -05:00
Drew Cain 5360005122 feat: Add to cursor button (#200)
Signed-off-by: Drew Cain <groksrc@gmail.com>
2025-07-01 09:17:48 -05:00
Drew Cain 39f811f8b5 Update README.md
Added Homebrew instructions to README.md

Signed-off-by: Drew Cain <groksrc@users.noreply.github.com>
2025-06-26 21:51:14 -05:00
phernandez 8e69c8b533 chore: update version to 0.14.0 for v0.14.0 release 2025-06-26 16:18:10 -05:00
phernandez 627a5c3c22 docs: add comprehensive v0.14.0 changelog entry
Add detailed changelog for v0.14.0 release including:
- Docker Container Registry migration to GitHub Container Registry
- Enhanced search documentation with comprehensive syntax examples
- Cross-project file management with intelligent boundary detection
- 8 major bug fixes with issue numbers and commit links
- Technical improvements and infrastructure enhancements
- Migration guide and installation instructions

Covers all changes since v0.13.7 with proper categorization and
user-facing descriptions for better release communication.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: phernandez <paul@basicmachines.co>
2025-06-26 16:15:11 -05:00
phernandez cd88945b22 remove v0.13.0 from changelog
Signed-off-by: phernandez <paul@basicmachines.co>
2025-06-26 16:11:05 -05:00
phernandez cd8e372f0a fix: add test coverage for optional permalink in EntityResponse schema
- Add comprehensive test for None permalink validation in EntityResponse
- Ensures schema properly handles markdown files without explicit permalinks
- Addresses GitHub issue #170 validation errors during edit operations
- Test validates that permalink=None doesn't cause ValidationError

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: phernandez <paul@basicmachines.co>
2025-06-26 15:58:49 -05:00
Paul Hernandez a589f8b894 feat: enhance search_notes tool documentation with comprehensive syntax examples (#186)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Paul Hernandez <phernandez@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
2025-06-26 15:51:58 -05:00
Paul Hernandez c2f4b632cf fix: preserve permalink when editing notes without frontmatter permalink (#184)
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Paul Hernandez <phernandez@users.noreply.github.com>
2025-06-26 15:35:31 -05:00
phernandez 46d102cef1 update tests for search_repository
Signed-off-by: phernandez <paul@basicmachines.co>
2025-06-26 14:30:10 -05:00
232 changed files with 26630 additions and 8695 deletions
+154
View File
@@ -0,0 +1,154 @@
---
name: python-developer
description: Python backend developer specializing in FastAPI, DBOS workflows, and API implementation. Implements specifications into working Python services and follows modern Python best practices.
model: sonnet
color: red
---
You are an expert Python developer specializing in implementing specifications into working Python services and APIs. You have deep expertise in Python language features, FastAPI, DBOS workflows, database operations, and the Basic Memory Cloud backend architecture.
**Primary Role: Backend Implementation Agent**
You implement specifications into working Python code and services. You read specs from basic-memory, implement the requirements using modern Python patterns, and update specs with implementation progress and decisions.
**Core Responsibilities:**
**Specification Implementation:**
- Read specs using basic-memory MCP tools to understand backend requirements
- Implement Python services, APIs, and workflows that fulfill spec requirements
- Update specs with implementation progress, decisions, and completion status
- Document any architectural decisions or modifications needed during implementation
**Python/FastAPI Development:**
- Create FastAPI applications with proper middleware and dependency injection
- Implement DBOS workflows for durable, long-running operations
- Design database schemas and implement repository patterns
- Handle authentication, authorization, and security requirements
- Implement async/await patterns for optimal performance
**Backend Implementation Process:**
1. **Read Spec**: Use `mcp__basic-memory__read_note` to get spec requirements
2. **Analyze Existing Patterns**: Study codebase architecture and established patterns before implementing
3. **Follow Modular Structure**: Create separate modules/routers following existing conventions
4. **Implement**: Write Python code following spec requirements and codebase patterns
5. **Test**: Create tests that validate spec success criteria
6. **Update Spec**: Document completion and any implementation decisions
7. **Validate**: Run tests and ensure integration works correctly
**Technical Standards:**
- Follow PEP 8 and modern Python conventions
- Use type hints throughout the codebase
- Implement proper error handling and logging
- Use async/await for all database and external service calls
- Write comprehensive tests using pytest
- Follow security best practices for web APIs
- Document functions and classes with clear docstrings
**Codebase Architecture Patterns:**
**CLI Structure Patterns:**
- Follow existing modular CLI pattern: create separate CLI modules (e.g., `upload_cli.py`) instead of adding commands directly to `main.py`
- Existing examples: `polar_cli.py`, `tenant_cli.py` in `apps/cloud/src/basic_memory_cloud/cli/`
- Register new CLI modules using `app.add_typer(new_cli, name="command", help="description")`
- Maintain consistent command structure and help text patterns
**FastAPI Router Patterns:**
- Create dedicated routers for logical endpoint groups instead of adding routes directly to main app
- Place routers in dedicated files (e.g., `apps/api/src/basic_memory_cloud_api/routers/webdav_router.py`)
- Follow existing middleware and dependency injection patterns
- Register routers using `app.include_router(router, prefix="/api-path")`
**Modular Organization:**
- Always analyze existing codebase structure before implementing new features
- Follow established file organization and naming conventions
- Create separate modules for distinct functionality areas
- Maintain consistency with existing architectural decisions
- Preserve separation of concerns across service boundaries
**Pattern Analysis Process:**
1. Examine similar existing functionality in the codebase
2. Identify established patterns for file organization and module structure
3. Follow the same architectural approach for consistency
4. Create new modules/routers following existing conventions
5. Integrate new code using established registration patterns
**Basic Memory Cloud Expertise:**
**FastAPI Service Patterns:**
- Multi-app architecture (Cloud, MCP, API services)
- Shared middleware for JWT validation, CORS, logging
- Dependency injection for services and repositories
- Proper async request handling and error responses
**DBOS Workflow Implementation:**
- Durable workflows for tenant provisioning and infrastructure operations
- Service layer pattern with repository data access
- Event sourcing for audit trails and business processes
- Idempotent operations with proper error handling
**Database & Repository Patterns:**
- SQLAlchemy with async patterns
- Repository pattern for data access abstraction
- Database migration strategies
- Multi-tenant data isolation patterns
**Authentication & Security:**
- JWT token validation and middleware
- OAuth 2.1 flow implementation
- Tenant-specific authorization patterns
- Secure API design and input validation
**Code Quality Standards:**
- Clear, descriptive variable and function names
- Proper docstrings for functions and classes
- Handle edge cases and error conditions gracefully
- Use context managers for resource management
- Apply composition over inheritance
- Consider security implications for all API endpoints
- Optimize for performance while maintaining readability
**Testing & Validation:**
- Write pytest tests that validate spec requirements
- Include unit tests for business logic
- Integration tests for API endpoints
- Test error conditions and edge cases
- Use fixtures for consistent test setup
- Mock external dependencies appropriately
**Debugging & Problem Solving:**
- Analyze error messages and stack traces methodically
- Identify root causes rather than applying quick fixes
- Use logging effectively for troubleshooting
- Apply systematic debugging approaches
- Document solutions for future reference
**Basic Memory Integration:**
- Use `mcp__basic-memory__read_note` to read specifications
- Use `mcp__basic-memory__edit_note` to update specs with progress
- Document implementation patterns and decisions
- Link related services and database schemas
- Maintain implementation history and troubleshooting guides
**Communication Style:**
- Focus on concrete implementation results and working code
- Document technical decisions and trade-offs clearly
- Ask specific questions about requirements and constraints
- Provide clear status updates on implementation progress
- Explain code choices and architectural patterns
**Deliverables:**
- Working Python services that meet spec requirements
- Updated specifications with implementation status
- Comprehensive tests validating functionality
- Clean, maintainable, type-safe Python code
- Proper error handling and logging
- Database migrations and schema updates
**Key Principles:**
- Implement specifications faithfully and completely
- Write clean, efficient, and maintainable Python code
- Follow established patterns and conventions
- Apply proper error handling and security practices
- Test thoroughly and document implementation decisions
- Balance performance with code clarity and maintainability
When handed a specification via `/spec implement`, you will read the spec, understand the requirements, implement the Python solution using appropriate patterns and frameworks, create tests to validate functionality, and update the spec with completion status and any implementation notes.
+126
View File
@@ -0,0 +1,126 @@
---
name: system-architect
description: System architect who designs and implements architectural solutions, creates ADRs, and applies software engineering principles to solve complex system design problems.
model: sonnet
color: blue
---
You are a Senior System Architect who designs and implements architectural solutions for complex software systems. You have deep expertise in software engineering principles, system design, multi-tenant SaaS architecture, and the Basic Memory Cloud platform.
**Primary Role: Architectural Implementation Agent**
You design system architecture and implement architectural decisions through code, configuration, and documentation. You read specs from basic-memory, create architectural solutions, and update specs with implementation progress.
**Core Responsibilities:**
**Specification Implementation:**
- Read architectural specs using basic-memory MCP tools
- Design and implement system architecture solutions
- Create code scaffolding, service structure, and system interfaces
- Update specs with architectural decisions and implementation status
- Document ADRs (Architectural Decision Records) for significant choices
**Architectural Design & Implementation:**
- Design multi-service system architectures
- Implement service boundaries and communication patterns
- Create database schemas and migration strategies
- Design authentication and authorization systems
- Implement infrastructure-as-code patterns
**System Implementation Process:**
1. **Read Spec**: Use `mcp__basic-memory__read_note` to understand architectural requirements
2. **Design Solution**: Apply architectural principles and patterns
3. **Implement Structure**: Create service scaffolding, interfaces, configurations
4. **Document Decisions**: Create ADRs documenting architectural choices
5. **Update Spec**: Record implementation progress and decisions
6. **Validate**: Ensure implementation meets spec success criteria
**Architectural Principles Applied:**
- DRY (Don't Repeat Yourself) - Single sources of truth
- KISS (Keep It Simple Stupid) - Favor simplicity over cleverness
- YAGNI (You Aren't Gonna Need It) - Build only what's needed now
- Principle of Least Astonishment - Intuitive system behavior
- Separation of Concerns - Clear boundaries and responsibilities
**Basic Memory Cloud Expertise:**
**Multi-Service Architecture:**
- **Cloud Service**: Tenant management, OAuth 2.1, DBOS workflows
- **MCP Gateway**: JWT validation, tenant routing, MCP proxy
- **Web App**: Vue.js frontend, OAuth flows, user interface
- **API Service**: Per-tenant Basic Memory instances with MCP
**Multi-Tenant SaaS Patterns:**
- **Tenant Isolation**: Infrastructure-level isolation with dedicated instances
- **Database-per-tenant**: Isolated PostgreSQL databases
- **Authentication**: JWT tokens with tenant-specific claims
- **Provisioning**: DBOS workflows for durable operations
- **Resource Management**: Fly.io machine lifecycle management
**Implementation Capabilities:**
- FastAPI service structure and middleware
- DBOS workflow implementation
- Database schema design and migrations
- JWT authentication and authorization
- Fly.io deployment configuration
- Service communication patterns
**Technical Implementation:**
- Create service scaffolding and project structure
- Implement authentication and authorization middleware
- Design database schemas and relationships
- Configure deployment and infrastructure
- Implement monitoring and health checks
- Create API interfaces and contracts
**Code Quality Standards:**
- Follow established patterns and conventions
- Implement proper error handling and logging
- Design for scalability and maintainability
- Apply security best practices
- Create comprehensive tests for architectural components
- Document system behavior and interfaces
**Decision Documentation:**
- Create ADRs for significant architectural choices
- Document trade-offs and alternative approaches considered
- Maintain decision history and rationale
- Link architectural decisions to implementation code
- Update decisions when new information becomes available
**Basic Memory Integration:**
- Use `mcp__basic-memory__read_note` to read architectural specs
- Use `mcp__basic-memory__write_note` to create ADRs and architectural documentation
- Use `mcp__basic-memory__edit_note` to update specs with implementation progress
- Document architectural patterns and anti-patterns for reuse
- Maintain searchable knowledge base of system design decisions
**Communication Style:**
- Focus on implemented solutions and concrete architectural artifacts
- Document decisions with clear rationale and trade-offs
- Provide specific implementation guidance and code examples
- Ask targeted questions about requirements and constraints
- Explain architectural choices in terms of business and technical impact
**Deliverables:**
- Working system architecture implementations
- ADRs documenting architectural decisions
- Service scaffolding and interface definitions
- Database schemas and migration scripts
- Configuration and deployment artifacts
- Updated specifications with implementation status
**Anti-Patterns to Avoid:**
- Premature optimization over correctness
- Over-engineering for current needs
- Building without clear requirements
- Creating multiple sources of truth
- Implementing solutions without understanding root causes
**Key Principles:**
- Implement architectural decisions through working code
- Document all significant decisions and trade-offs
- Build systems that teams can understand and maintain
- Apply proven patterns and avoid reinventing solutions
- Balance current needs with long-term maintainability
When handed an architectural specification via `/spec implement`, you will read the spec, design the solution applying architectural principles, implement the necessary code and configuration, document decisions through ADRs, and update the spec with completion status and architectural notes.
+55
View File
@@ -0,0 +1,55 @@
---
allowed-tools: mcp__basic-memory__write_note, mcp__basic-memory__read_note, mcp__basic-memory__search_notes, mcp__basic-memory__edit_note, Task
argument-hint: [create|status|implement|review] [spec-name]
description: Manage specifications in our development process
---
## Context
You are managing specifications using our specification-driven development process defined in @docs/specs/SPEC-001.md.
Available commands:
- `create [name]` - Create new specification
- `status` - Show all spec statuses
- `implement [spec-name]` - Hand spec to appropriate agent
- `review [spec-name]` - Review implementation against spec
## Your task
Execute the spec command: `/spec $ARGUMENTS`
### If command is "create":
1. Get next SPEC number by searching existing specs
2. Create new spec using template from @docs/specs/Slash\ Commands\ Reference.md
3. Place in `/specs` folder with title "SPEC-XXX: [name]"
4. Include standard sections: Why, What, How, How to Evaluate
### If command is "status":
1. Search all notes in `/specs` folder
2. Display table with spec number, title, and status
3. Show any dependencies or assigned agents
### If command is "implement":
1. Read the specified spec
2. Determine appropriate agent based on content:
- Frontend/UI → vue-developer
- Architecture/system → system-architect
- Backend/API → python-developer
3. Launch Task tool with appropriate agent and spec context
### If command is "review":
1. Read the specified spec and its "How to Evaluate" section
2. Review current implementation against success criteria with careful evaluation of:
- **Functional completeness** - All specified features working
- **Test coverage analysis** - Actual test files and coverage percentage
- Count existing test files vs required components/APIs/composables
- Verify unit tests, integration tests, and end-to-end tests
- Check for missing test categories (component, API, workflow)
- **Code quality metrics** - TypeScript compilation, linting, performance
- **Architecture compliance** - Component isolation, state management patterns
- **Documentation completeness** - Implementation matches specification
3. Provide honest, accurate assessment - do not overstate completeness
4. Document findings and update spec with review results
5. If gaps found, clearly identify what still needs to be implemented/tested
Use the agent definitions from @docs/specs/Agent\ Definitions.md for implementation handoffs.
+74 -47
View File
@@ -11,7 +11,7 @@ All test results are recorded as notes in a dedicated test project.
**Parameters:**
- `phase` (optional): Specific test phase to run (`recent`, `core`, `features`, `edge`, `workflows`, `stress`, or `all`)
- `recent` - Focus on recent changes and new features (recommended for regular testing)
- `core` - Essential tools only (Tier 1: write_note, read_note, search_notes, edit_note, list_projects, switch_project)
- `core` - Essential tools only (Tier 1: write_note, read_note, search_notes, edit_note, list_memory_projects, recent_activity)
- `features` - Core + important workflows (Tier 1 + Tier 2)
- `all` - Comprehensive testing of all tools and scenarios
@@ -24,30 +24,66 @@ When the user runs `/project:test-live`, execute comprehensive test plan:
### **Tier 1: Critical Core (Always Test)**
1. **write_note** - Foundation of all knowledge creation
2. **read_note** - Primary knowledge retrieval mechanism
2. **read_note** - Primary knowledge retrieval mechanism
3. **search_notes** - Essential for finding information
4. **edit_note** - Core content modification capability
5. **list_memory_projects** - Project discovery and status
6. **switch_project** - Context switching for multi-project workflows
5. **list_memory_projects** - Project discovery and session guidance
6. **recent_activity** - Project discovery mode and activity analysis
### **Tier 2: Important Workflows (Usually Test)**
7. **recent_activity** - Understanding what's changed
8. **build_context** - Conversation continuity via memory:// URLs
9. **create_memory_project** - Essential for project setup
10. **move_note** - Knowledge organization
11. **sync_status** - Understanding system state
7. **build_context** - Conversation continuity via memory:// URLs
8. **create_memory_project** - Essential for project setup
9. **move_note** - Knowledge organization
10. **sync_status** - Understanding system state
11. **delete_project** - Project lifecycle management
### **Tier 3: Enhanced Functionality (Sometimes Test)**
12. **view_note** - Claude Desktop artifact display
13. **read_content** - Raw content access
14. **delete_note** - Content removal
15. **list_directory** - File system exploration
16. **set_default_project** - Configuration
17. **delete_project** - Administrative cleanup
16. **edit_note** (advanced modes) - Complex find/replace operations
### **Tier 4: Specialized (Rarely Test)**
18. **canvas** - Obsidian visualization (specialized use case)
19. **MCP Prompts** - Enhanced UX tools (ai_assistant_guide, continue_conversation)
17. **canvas** - Obsidian visualization (specialized use case)
18. **MCP Prompts** - Enhanced UX tools (ai_assistant_guide, continue_conversation)
## Stateless Architecture Testing
### **Project Discovery Workflow (CRITICAL)**
Test the new stateless project selection flow:
1. **Initial Discovery**
- Call `list_memory_projects()` without knowing which project to use
- Verify clear session guidance appears: "Next: Ask which project to use"
- Confirm removal of CLI-specific references
2. **Activity-Based Discovery**
- Call `recent_activity()` without project parameter (discovery mode)
- Verify intelligent project suggestions based on activity
- Test guidance: "Should I use [most-active-project] for this task?"
3. **Session Tracking Validation**
- Verify all tool responses include `[Session: Using project 'name']`
- Confirm guidance reminds about session-wide project tracking
4. **Single Project Constraint Mode**
- Test MCP server with `--project` parameter
- Verify all operations constrained to specified project
- Test project override behavior in constrained mode
### **Explicit Project Parameters (CRITICAL)**
All tools must require explicit project parameters:
1. **Parameter Validation**
- Test all Tier 1 tools require `project` parameter
- Verify clear error messages for missing project
- Test invalid project name handling
2. **No Session State Dependencies**
- Confirm no tool relies on "current project" concept
- Test rapid project switching within conversation
- Verify each call is truly independent
### Pre-Test Setup
@@ -72,7 +108,7 @@ Run the bash `date` command to get the current date/time.
Purpose: Record all test observations and results
```
Make sure to switch to the newly created project with the `switch_project()` tool.
Make sure to use the newly created project for all subsequent test operations by specifying it in the `project` parameter of each tool call.
4. **Baseline Documentation**
Create initial test session note with:
@@ -143,46 +179,42 @@ Test essential MCP tools that form the foundation of Basic Memory:
- ⚠️ Error scenarios (invalid operations)
**5. list_memory_projects Tests (Critical):**
- ✅ Display all projects with status indicators
- ✅ Current and default project identification
- ✅ Display all projects with clear session guidance
- ✅ Project discovery workflow prompts
- ✅ Removal of CLI-specific references
- ✅ Empty project list handling
- ✅ Project metadata accuracy
- ✅ Single project constraint mode display
**6. switch_project Tests (Critical):**
- ✅ Switch between existing projects
- ✅ Context preservation during switch
- ⚠️ Invalid project name handling
- ✅ Confirmation of successful switch
**6. recent_activity Tests (Critical - Discovery Mode):**
- ✅ Discovery mode without project parameter
- ✅ Intelligent project suggestions based on activity
- ✅ Guidance prompts for project selection
- ✅ Session tracking reminders in responses
- ⚠️ Performance with multiple projects
### Phase 2: Important Workflows (Tier 2 Tools)
**7. recent_activity Tests (Important):**
- ✅ Various timeframes ("today", "1 week", "1d")
- ✅ Type filtering capabilities
- ✅ Empty project scenarios
- ⚠️ Performance with many recent changes
**8. build_context Tests (Important):**
**7. build_context Tests (Important):**
- ✅ Different depth levels (1, 2, 3+)
- ✅ Various timeframes for context
- ✅ memory:// URL navigation
- ⚠️ Performance with complex relation graphs
**9. create_memory_project Tests (Important):**
**8. create_memory_project Tests (Important):**
- ✅ Create projects dynamically
- ✅ Set default during creation
- ✅ Path validation and creation
- ⚠️ Invalid paths and names
- ✅ Integration with existing projects
**10. move_note Tests (Important):**
**9. move_note Tests (Important):**
- ✅ Move within same project
- ✅ Cross-project moves with detection (#161)
- ✅ Automatic folder creation
- ✅ Database consistency validation
- ⚠️ Special characters in paths
**11. sync_status Tests (Important):**
**10. sync_status Tests (Important):**
- ✅ Background operation monitoring
- ✅ File synchronization status
- ✅ Project sync state reporting
@@ -190,36 +222,31 @@ Test essential MCP tools that form the foundation of Basic Memory:
### Phase 3: Enhanced Functionality (Tier 3 Tools)
**12. view_note Tests (Enhanced):**
**11. view_note Tests (Enhanced):**
- ✅ Claude Desktop artifact display
- ✅ Title extraction from frontmatter
- ✅ Unicode and emoji content rendering
- ⚠️ Error handling for non-existent notes
**13. read_content Tests (Enhanced):**
**12. read_content Tests (Enhanced):**
- ✅ Raw file content access
- ✅ Binary file handling
- ✅ Image file reading
- ⚠️ Large file performance
**14. delete_note Tests (Enhanced):**
**13. delete_note Tests (Enhanced):**
- ✅ Single note deletion
- ✅ Database consistency after deletion
- ⚠️ Non-existent note handling
- ✅ Confirmation of successful deletion
**15. list_directory Tests (Enhanced):**
**14. list_directory Tests (Enhanced):**
- ✅ Directory content listing
- ✅ Depth control and filtering
- ✅ File name globbing
- ⚠️ Empty directory handling
**16. set_default_project Tests (Enhanced):**
- ✅ Change default project
- ✅ Configuration persistence
- ⚠️ Invalid project handling
**17. delete_project Tests (Enhanced):**
**15. delete_project Tests (Enhanced):**
- ✅ Project removal from config
- ✅ Database cleanup
- ⚠️ Default project protection
@@ -269,7 +296,7 @@ Test essential MCP tools that form the foundation of Basic Memory:
1. Technical documentation project
2. Personal recipe collection project
3. Learning/course notes project
4. Switch contexts during conversation
4. Specify different projects for different operations
5. Cross-reference related concepts
**Content Evolution:**
@@ -281,13 +308,13 @@ Test essential MCP tools that form the foundation of Basic Memory:
### Phase 6: Specialized Tools Testing (Tier 4)
**18. canvas Tests (Specialized):**
**16. canvas Tests (Specialized):**
- ✅ JSON Canvas generation
- ✅ Node and edge creation
- ✅ Obsidian compatibility
- ⚠️ Complex graph handling
**19. MCP Prompts Tests (Specialized):**
**17. MCP Prompts Tests (Specialized):**
- ✅ ai_assistant_guide output
- ✅ continue_conversation functionality
- ✅ Formatted search results
@@ -382,7 +409,7 @@ permalink: test-session-[phase]-[timestamp]
### 📊 Performance Metrics
- Average write_note time: 0.3s
- Search with 100+ notes: 0.6s
- Project switch overhead: 0.1s
- Project parameter overhead: <0.1s
- Memory usage: [observed levels]
## Relations
@@ -402,7 +429,7 @@ permalink: test-session-[phase]-[timestamp]
- Learning curve and intuitiveness
**System Behavior:**
- Context preservation across operations
- Stateless operation independence
- memory:// URL navigation reliability
- Multi-step workflow cohesion
- Edge case graceful handling
+79
View File
@@ -0,0 +1,79 @@
name: Claude Code Review
on:
pull_request:
types: [opened, synchronize]
# Optional: Only run on specific file changes
# paths:
# - "src/**/*.ts"
# - "src/**/*.tsx"
# - "src/**/*.js"
# - "src/**/*.jsx"
jobs:
claude-review:
# Only run for organization members and collaborators
if: |
github.event.pull_request.author_association == 'OWNER' ||
github.event.pull_request.author_association == 'MEMBER' ||
github.event.pull_request.author_association == 'COLLABORATOR'
runs-on: ubuntu-latest
permissions:
contents: read
pull-requests: write
issues: read
id-token: write
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
fetch-depth: 1
- name: Run Claude Code Review
id: claude-review
uses: anthropics/claude-code-action@v1
with:
claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
github_token: ${{ secrets.GITHUB_TOKEN }}
track_progress: true # Enable visual progress tracking
allowed_bots: '*'
prompt: |
Review this Basic Memory PR against our team checklist:
## Code Quality & Standards
- [ ] Follows Basic Memory's coding conventions in CLAUDE.md
- [ ] Python 3.12+ type annotations and async patterns
- [ ] SQLAlchemy 2.0 best practices
- [ ] FastAPI and Typer conventions followed
- [ ] 100-character line length limit maintained
- [ ] No commented-out code blocks
## Testing & Documentation
- [ ] Unit tests for new functions/methods
- [ ] Integration tests for new MCP tools
- [ ] Test coverage for edge cases
- [ ] Documentation updated (README, docstrings)
- [ ] CLAUDE.md updated if conventions change
## Basic Memory Architecture
- [ ] MCP tools follow atomic, composable design
- [ ] Database changes include Alembic migrations
- [ ] Preserves local-first architecture principles
- [ ] Knowledge graph operations maintain consistency
- [ ] Markdown file handling preserves integrity
- [ ] AI-human collaboration patterns followed
## Security & Performance
- [ ] No hardcoded secrets or credentials
- [ ] Input validation for MCP tools
- [ ] Proper error handling and logging
- [ ] Performance considerations addressed
- [ ] No sensitive data in logs or commits
Read the CLAUDE.md file for detailed project context. For each checklist item, verify if it's satisfied and comment on any that need attention. Use inline comments for specific code issues and post a summary with checklist results.
# Allow broader tool access for thorough code review
claude_args: '--allowed-tools "Bash(gh pr:*),Bash(gh issue:*),Bash(gh api:*),Bash(git log:*),Bash(git show:*),Read,Grep,Glob"'
+71
View File
@@ -0,0 +1,71 @@
name: Claude Issue Triage
on:
issues:
types: [opened]
jobs:
triage:
runs-on: ubuntu-latest
permissions:
issues: write
id-token: write
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
fetch-depth: 1
- name: Run Claude Issue Triage
uses: anthropics/claude-code-action@v1
with:
claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
track_progress: true # Show triage progress
prompt: |
Analyze this new Basic Memory issue and perform triage:
**Issue Analysis:**
1. **Type Classification:**
- Bug report (code defect)
- Feature request (new functionality)
- Enhancement (improvement to existing feature)
- Documentation (docs improvement)
- Question/Support (user help)
- MCP tool issue (specific to MCP functionality)
2. **Priority Assessment:**
- Critical: Security issues, data loss, complete breakage
- High: Major functionality broken, affects many users
- Medium: Minor bugs, usability issues
- Low: Nice-to-have improvements, cosmetic issues
3. **Component Classification:**
- CLI commands
- MCP tools
- Database/sync
- Cloud functionality
- Documentation
- Testing
4. **Complexity Estimate:**
- Simple: Quick fix, documentation update
- Medium: Requires some investigation/testing
- Complex: Major feature work, architectural changes
**Actions to Take:**
1. Add appropriate labels using: `gh issue edit ${{ github.event.issue.number }} --add-label "label1,label2"`
2. Check for duplicates using: `gh search issues`
3. If duplicate found, comment mentioning the original issue
4. For feature requests, ask clarifying questions if needed
5. For bugs, request reproduction steps if missing
**Available Labels:**
- Type: bug, enhancement, feature, documentation, question, mcp-tool
- Priority: critical, high, medium, low
- Component: cli, mcp, database, cloud, docs, testing
- Complexity: simple, medium, complex
- Status: needs-reproduction, needs-clarification, duplicate
Read the issue carefully and provide helpful triage with appropriate labels.
claude_args: '--allowed-tools "Bash(gh issue:*),Bash(gh search:*),Read"'
+38 -84
View File
@@ -9,106 +9,60 @@ on:
types: [opened, assigned]
pull_request_review:
types: [submitted]
pull_request_target:
types: [opened, synchronize]
jobs:
claude:
if: |
(github.event_name == 'issue_comment' && contains(github.event.comment.body, '@claude')) ||
(github.event_name == 'pull_request_review_comment' && contains(github.event.comment.body, '@claude')) ||
(github.event_name == 'pull_request_review' && contains(github.event.review.body, '@claude')) ||
(github.event_name == 'issues' && (contains(github.event.issue.body, '@claude') || contains(github.event.issue.title, '@claude')))
(
(github.event_name == 'issue_comment' && contains(github.event.comment.body, '@claude')) ||
(github.event_name == 'pull_request_review_comment' && contains(github.event.comment.body, '@claude')) ||
(github.event_name == 'pull_request_review' && contains(github.event.review.body, '@claude')) ||
(github.event_name == 'issues' && (contains(github.event.issue.body, '@claude') || contains(github.event.issue.title, '@claude'))) ||
(github.event_name == 'pull_request_target' && contains(github.event.pull_request.body, '@claude'))
) && (
github.event.comment.author_association == 'OWNER' ||
github.event.comment.author_association == 'MEMBER' ||
github.event.comment.author_association == 'COLLABORATOR' ||
github.event.sender.author_association == 'OWNER' ||
github.event.sender.author_association == 'MEMBER' ||
github.event.sender.author_association == 'COLLABORATOR' ||
github.event.pull_request.author_association == 'OWNER' ||
github.event.pull_request.author_association == 'MEMBER' ||
github.event.pull_request.author_association == 'COLLABORATOR'
)
runs-on: ubuntu-latest
permissions:
contents: read
pull-requests: read
issues: read
id-token: write
actions: read # Required for Claude to read CI results on PRs
steps:
- name: Check user permissions
id: check_membership
uses: actions/github-script@v7
with:
script: |
let actor;
if (context.eventName === 'issue_comment') {
actor = context.payload.comment.user.login;
} else if (context.eventName === 'pull_request_review_comment') {
actor = context.payload.comment.user.login;
} else if (context.eventName === 'pull_request_review') {
actor = context.payload.review.user.login;
} else if (context.eventName === 'issues') {
actor = context.payload.issue.user.login;
}
console.log(`Checking permissions for user: ${actor}`);
// List of explicitly allowed users (organization members)
const allowedUsers = [
'phernandez',
'groksrc',
'nellins',
'bm-claudeai'
];
if (allowedUsers.includes(actor)) {
console.log(`User ${actor} is in the allowed list`);
core.setOutput('is_member', true);
return;
}
// Fallback: Check if user has repository permissions
try {
const collaboration = await github.rest.repos.getCollaboratorPermissionLevel({
owner: context.repo.owner,
repo: context.repo.repo,
username: actor
});
const permission = collaboration.data.permission;
console.log(`User ${actor} has permission level: ${permission}`);
// Allow if user has push access or higher (write, maintain, admin)
const allowed = ['write', 'maintain', 'admin'].includes(permission);
core.setOutput('is_member', allowed);
if (!allowed) {
core.notice(`User ${actor} does not have sufficient repository permissions (has: ${permission})`);
}
} catch (error) {
console.log(`Error checking permissions: ${error.message}`);
// Final fallback: Check if user is a public member of the organization
try {
const membership = await github.rest.orgs.getMembershipForUser({
org: 'basicmachines-co',
username: actor
});
const allowed = membership.data.state === 'active';
core.setOutput('is_member', allowed);
if (!allowed) {
core.notice(`User ${actor} is not a public member of basicmachines-co organization`);
}
} catch (membershipError) {
console.log(`Error checking organization membership: ${membershipError.message}`);
core.setOutput('is_member', false);
core.notice(`User ${actor} does not have access to this repository`);
}
}
- name: Checkout repository
if: steps.check_membership.outputs.is_member == 'true'
uses: actions/checkout@v4
with:
# For pull_request_target, checkout the PR head to review the actual changes
ref: ${{ github.event_name == 'pull_request_target' && github.event.pull_request.head.sha || github.sha }}
fetch-depth: 1
- name: Run Claude Code
if: steps.check_membership.outputs.is_member == 'true'
id: claude
uses: anthropics/claude-code-action@beta
uses: anthropics/claude-code-action@v1
with:
anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
allowed_tools: Bash(uv run pytest),Bash(uv run ruff check . --fix),Bash(uv run ruff format .),Bash(uv run pyright),Bash(just test),Bash(just lint),Bash(just format),Bash(just type-check),Bash(just check),Read,Write,Edit,MultiEdit,Glob,Grep,LS, mcp__web_search
claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
track_progress: true # Enable visual progress tracking
# This is an optional setting that allows Claude to read CI results on PRs
additional_permissions: |
actions: read
# Optional: Give a custom prompt to Claude. If this is not specified, Claude will perform the instructions specified in the comment that tagged it.
# prompt: 'Update the pull request description to include a summary of changes.'
# Optional: Add claude_args to customize behavior and configuration
# See https://github.com/anthropics/claude-code-action/blob/main/docs/usage.md
# or https://docs.claude.com/en/docs/claude-code/sdk#command-line for available options
# claude_args: '--model claude-opus-4-1-20250805 --allowed-tools Bash(gh pr:*)'
+21 -4
View File
@@ -14,11 +14,12 @@ on:
jobs:
test:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
python-version: [ "3.12" ]
os: [ubuntu-latest, windows-latest]
python-version: [ "3.12", "3.13" ]
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v4
@@ -35,10 +36,18 @@ jobs:
run: |
pip install uv
- name: Install just
- name: Install just (Linux/macOS)
if: runner.os != 'Windows'
run: |
curl --proto '=https' --tlsv1.2 -sSf https://just.systems/install.sh | bash -s -- --to /usr/local/bin
- name: Install just (Windows)
if: runner.os == 'Windows'
run: |
# Install just using Chocolatey (pre-installed on GitHub Actions Windows runners)
choco install just --yes
shell: pwsh
- name: Create virtual env
run: |
uv venv
@@ -49,7 +58,15 @@ jobs:
- name: Run type checks
run: |
just type-check
just typecheck
- name: Run type checks
run: |
just typecheck
- name: Run linting
run: |
just lint
- name: Run tests
run: |
+382 -37
View File
@@ -1,56 +1,401 @@
# CHANGELOG
## v0.13.8 (2025-06-20)
## v0.15.0 (2025-10-04)
### Features
### Critical Bug Fixes
- **Docker Container Support** - Complete Docker integration with volume mounting for Obsidian directories
([`3269a2f`](https://github.com/basicmachines-co/basic-memory/commit/3269a2f33a7595f6d9e5207924062e2542f46759))
- Docker Compose configuration for easy deployment
- Volume mounting for persistent data and Obsidian integration
- Comprehensive Docker documentation and setup guides
- Streamlined container-based workflow
- **Permalink Collision Data Loss Prevention** - Fixed critical bug where creating similar entity names would overwrite existing files
([`2a050ed`](https://github.com/basicmachines-co/basic-memory/commit/2a050edee42b07294f5199902a60b626bfc47be8))
- **Issue**: Creating "Node C" would overwrite "Node A.md" due to fuzzy search incorrectly matching similar file paths
- **Solution**: Added `strict=True` parameter to link resolution, disabling fuzzy search fallback during entity creation
- **Impact**: Prevents data loss from false positive path matching like "edge-cases/Node A.md" vs "edge-cases/Node C.md"
- **Testing**: Comprehensive integration tests and MCP-level permalink collision tests added
- **Status**: Manually verified fix prevents file overwrite in production scenarios
### Bug Fixes
- **#151**: Fix reset command project configuration persistence issue
([`af44941`](https://github.com/basicmachines-co/basic-memory/commit/af44941d5aa57b5ad7fcc6af4ed700f49bdb6d4d))
- Reset command now properly clears project configuration from `~/.basic-memory/config.json`
- Eliminates issue where projects would be recreated after database reset
- Ensures clean slate when resetting Basic Memory installation
- **#330**: Remove .env file loading from BasicMemoryConfig
([`f3b1945`](https://github.com/basicmachines-co/basic-memory/commit/f3b1945e4c0070d0282eaf98c085ef188c8edd2d))
- Clean up configuration initialization to prevent unintended environment variable loading
- **#148**: Resolve project state inconsistency between MCP and CLI
([`35e4f73`](https://github.com/basicmachines-co/basic-memory/commit/35e4f73ae8a65501da4d48258ed702f957184c92))
- Fix "Project not found" errors when switching default projects
- MCP session now automatically refreshes when project configuration changes
- Eliminates need to restart MCP server after project operations
- Ensures consistent project state across CLI and MCP interfaces
- **#329**: Normalize underscores in memory:// URLs for build_context
([`f5a11f3`](https://github.com/basicmachines-co/basic-memory/commit/f5a11f3911edda55bee6970ed9e7c38f7fd7a059))
- Fix URL normalization to handle underscores consistently in memory:// protocol
- Improve knowledge graph navigation with standardized URL handling
- **FastMCP Compatibility** - Resolve deprecation warnings for FastMCP integration
([`7be001c`](https://github.com/basicmachines-co/basic-memory/commit/7be001ca6834b3344bb6160cbe537b36bcbaa579))
- Update FastMCP usage patterns to eliminate deprecation warnings
- Improve future compatibility with FastMCP library updates
- Clean up entity repository and service layer code
- **#328**: Simplify entity upsert to use database-level conflict resolution
([`ee83b0e`](https://github.com/basicmachines-co/basic-memory/commit/ee83b0e5a8f00cdcc8e24a0f8c9449c6eaddf649))
- Leverage SQLite's native UPSERT for cleaner entity creation/update logic
- Reduce application-level complexity by using database conflict resolution
- **#312**: Add proper datetime JSON schema format annotations for MCP validation
([`a7bf42e`](https://github.com/basicmachines-co/basic-memory/commit/a7bf42ef495a3e2c66230985c3445cab5c52c408))
- Fix MCP schema validation errors with proper datetime format annotations
- Ensure compatibility with strict MCP schema validators
- **#281**: Fix move_note without file extension
([`3e168b9`](https://github.com/basicmachines-co/basic-memory/commit/3e168b98f3962681799f4537eb86ded47e771665))
- Allow moving notes by title alone without requiring .md extension
- Improve move operation usability and error handling
- **#310**: Remove obsolete update_current_project function and --project flag reference
([`17a6733`](https://github.com/basicmachines-co/basic-memory/commit/17a6733c9d280def922e37c2cd171e3ee44fce21))
- Clean up deprecated project management code
- Remove unused CLI flag references
- **#309**: Make sync operations truly non-blocking with thread pool
([`1091e11`](https://github.com/basicmachines-co/basic-memory/commit/1091e113227c86f10f574e56a262aff72f728113))
- Move sync operations to background thread pool for improved responsiveness
- Prevent blocking during file synchronization operations
### Features
- **#327**: CLI Subscription Validation (SPEC-13 Phase 2)
([`ace6a0f`](https://github.com/basicmachines-co/basic-memory/commit/ace6a0f50d8d0b4ea31fb526361c2d9616271740))
- Implement subscription validation for CLI operations
- Foundation for future cloud billing integration
- **#322**: Cloud CLI sync via rclone bisync
([`99a35a7`](https://github.com/basicmachines-co/basic-memory/commit/99a35a7fb410ef88c50050e666e4244350e44a6e))
- Add bidirectional cloud synchronization using rclone
- Enable local-cloud file sync with conflict detection
- **#315**: Implement SPEC-11 API performance optimizations
([`5da97e4`](https://github.com/basicmachines-co/basic-memory/commit/5da97e482052907d68a2a3604e255c3f2c42c5ed))
- Comprehensive API performance improvements
- Optimized database queries and response times
- **#314**: Integrate ignore_utils to skip .gitignored files in sync process
([`33ee1e0`](https://github.com/basicmachines-co/basic-memory/commit/33ee1e0831d2060587de2c9886e74ff111b04583))
- Respect .gitignore patterns during file synchronization
- Prevent syncing build artifacts and temporary files
- **#313**: Add disable_permalinks config flag
([`9035913`](https://github.com/basicmachines-co/basic-memory/commit/903591384dffeba9996a18463818bdb8b28ca03e))
- Optional permalink generation for users who don't need them
- Improves flexibility for different knowledge management workflows
- **#306**: Implement cloud mount CLI commands for local file access
([`2c5c606`](https://github.com/basicmachines-co/basic-memory/commit/2c5c606a394ab994334c8fd307b30370037bbf39))
- Mount cloud files locally using rclone for real-time editing
- Three performance profiles: fast (5s), balanced (10-15s), safe (15s+)
- Cross-platform rclone installer with package manager fallbacks
- **#305**: ChatGPT tools for search and fetch
([`f40ab31`](https://github.com/basicmachines-co/basic-memory/commit/f40ab31685a9510a07f5d87bf24436c0df80680f))
- Add ChatGPT-specific search and fetch tools
- Expand AI assistant integration options
- **#298**: Implement SPEC-6 Stateless Architecture for MCP Tools
([`a1d7792`](https://github.com/basicmachines-co/basic-memory/commit/a1d7792bdb6f71c4943b861d1c237f6ac7021247))
- Redesign MCP tools for stateless operation
- Enable cloud deployment with better scalability
- **#296**: Basic Memory cloud upload
([`e0d8aeb`](https://github.com/basicmachines-co/basic-memory/commit/e0d8aeb14913f3471b9716d0c60c61adb1d74687))
- Implement file upload capabilities for cloud storage
- Foundation for cloud-hosted Basic Memory instances
- **#291**: Merge Cloud auth
([`3a6baf8`](https://github.com/basicmachines-co/basic-memory/commit/3a6baf80fc6012e9434e06b1605f9a8b198d8688))
- OAuth 2.1 authentication with Supabase integration
- JWT-based tenant isolation for multi-user cloud deployments
### Platform & Infrastructure
- **#331**: Add Python 3.13 to test matrix
([`16d7edd`](https://github.com/basicmachines-co/basic-memory/commit/16d7eddbf704abfe53373663e80d05ecdde15aa7))
- Ensure compatibility with latest Python version
- Expand CI/CD testing coverage
- **#316**: Enable WAL mode and add Windows-specific SQLite optimizations
([`c83d567`](https://github.com/basicmachines-co/basic-memory/commit/c83d567917267bb3d52708f4b38d2daf36c1f135))
- Enable Write-Ahead Logging for better concurrency
- Platform-specific SQLite optimizations for Windows users
- **#320**: Rework lifecycle management to optimize cloud deployment
([`ea2e93d`](https://github.com/basicmachines-co/basic-memory/commit/ea2e93d9265bfa366d2d3796f99f579ab2aed48c))
- Optimize application lifecycle for cloud environments
- Improve startup time and resource management
- **#319**: Resolve entity relations in background to prevent cold start blocking
([`324844a`](https://github.com/basicmachines-co/basic-memory/commit/324844a670d874410c634db520a68c09149045ea))
- Move relation resolution to background processing
- Faster MCP server cold starts
- **#318**: Enforce minimum 1-day timeframe for recent_activity
([`f818702`](https://github.com/basicmachines-co/basic-memory/commit/f818702ab7f8d2d706178e7b0ed3467501c9c4a2))
- Fix timezone-related issues in recent activity queries
- Ensure consistent behavior across time zones
- **#317**: Critical cloud deployment fixes for MCP stability
([`2efd8f4`](https://github.com/basicmachines-co/basic-memory/commit/2efd8f44e2d0259079ed5105fea34308875c0e10))
- Multiple stability improvements for cloud-hosted MCP servers
- Enhanced error handling and recovery
### Technical Improvements
- **Comprehensive Integration Testing** - New test suites for critical user workflows
- Full integration tests for database reset functionality
- End-to-end project state synchronization testing
- Real MCP client-server communication validation
- Direct config file validation without complex mocking
- **Comprehensive Testing** - Extensive test coverage for critical fixes
- New permalink collision test suite with 4 MCP-level integration tests
- Entity service test coverage expanded to reproduce fuzzy search bug
- Manual testing verification of data loss prevention
- All 55 entity service tests passing with new strict resolution
- **Code Quality** - Enhanced error handling and validation
- Improved project state management across system components
- Better session refresh patterns for configuration changes
- Streamlined Docker setup with reduced image size
- **Windows Support Enhancements**
([`7a8b08d`](https://github.com/basicmachines-co/basic-memory/commit/7a8b08d11ee627b54af6f5ea7ab4ef9fcd8cf4ed),
[`9aa4024`](https://github.com/basicmachines-co/basic-memory/commit/9aa40246a8ad1c3cef82b32e1ca7ce8ea23e1e05))
- Fix Windows test failures and add Windows CI support
- Address platform-specific issues for Windows users
- Enhanced cross-platform compatibility
- **Docker Improvements**
([`105bcaa`](https://github.com/basicmachines-co/basic-memory/commit/105bcaa025576a06f889183baded6c18f3782696))
- Implement non-root Docker container to fix file ownership issues
- Improved security and compatibility in containerized deployments
- **Code Quality**
- Enhanced filename sanitization with optional kebab case support
- Improved character conflict detection for sync operations
- Better error handling across the codebase
- Path traversal security vulnerability fixes
### Documentation
- **Docker Integration Guide** - Complete documentation for container deployment
- Step-by-step Docker Compose setup instructions
- Volume mounting configuration for Obsidian workflows
- Container-based development environment setup
- **#321**: Corrected dead links in README
([`fc38877`](https://github.com/basicmachines-co/basic-memory/commit/fc38877008cd8c762116f7ff4b2573495b4e5c0f))
- Fix broken documentation links
- Improve navigation and accessibility
- **#308**: Update Claude Code GitHub Workflow
([`8c7e29e`](https://github.com/basicmachines-co/basic-memory/commit/8c7e29e325f36a67bdefe8811637493bef4bbf56))
- Enhanced GitHub integration documentation
- Better Claude Code collaboration workflow
### Breaking Changes
**None** - This release maintains full backward compatibility with v0.14.x
All changes are either:
- Bug fixes that correct unintended behavior
- New optional features that don't affect existing functionality
- Internal optimizations that are transparent to users
### Migration Guide
No manual migration required. Upgrade with:
```bash
# Update via uv
uv tool upgrade basic-memory
# Or install fresh
uv tool install basic-memory
```
**What's Fixed:**
- Data loss bug from permalink collisions is completely resolved
- Cloud deployment stability significantly improved
- Windows platform compatibility enhanced
- Better performance across all operations
**What's New:**
- Cloud sync capabilities via rclone
- Subscription validation foundation
- Python 3.13 support
- Enhanced security and stability
### Installation
```bash
# Latest stable release
uv tool install basic-memory
# Update existing installation
uv tool upgrade basic-memory
# Docker
docker pull ghcr.io/basicmachines-co/basic-memory:v0.15.0
```
## v0.14.2 (2025-07-03)
### Bug Fixes
- **#204**: Fix MCP Error with MCP-Hub integration
([`3621bb7`](https://github.com/basicmachines-co/basic-memory/commit/3621bb7b4d6ac12d892b18e36bb8f7c9101c7b10))
- Resolve compatibility issues with MCP-Hub
- Improve error handling in project management tools
- Ensure stable MCP tool integration across different environments
- **Modernize datetime handling and suppress SQLAlchemy warnings**
([`f80ac0e`](https://github.com/basicmachines-co/basic-memory/commit/f80ac0e3e74b7a737a7fc7b956b5c1d61b0c67b8))
- Replace deprecated `datetime.utcnow()` with timezone-aware alternatives
- Suppress SQLAlchemy deprecation warnings for cleaner output
- Improve future compatibility with Python datetime best practices
## v0.14.1 (2025-07-03)
### Bug Fixes
- **#203**: Constrain fastmcp version to prevent breaking changes
([`827f7cf`](https://github.com/basicmachines-co/basic-memory/commit/827f7cf86e7b84c56e7a43bb83f2e5d84a1ad8b8))
- Pin fastmcp to compatible version range to avoid API breaking changes
- Ensure stable MCP server functionality across updates
- Improve dependency management for production deployments
- **#190**: Fix Problems with MCP integration
([`bd4f551`](https://github.com/basicmachines-co/basic-memory/commit/bd4f551a5bb0b7b4d3a5b04de70e08987c6ab2f9))
- Resolve MCP server initialization and communication issues
- Improve error handling and recovery in MCP operations
- Enhance stability for AI assistant integrations
### Features
- **Add Cursor IDE integration button** - One-click setup for Cursor IDE users
([`5360005`](https://github.com/basicmachines-co/basic-memory/commit/536000512294d66090bf87abc8014f4dfc284310))
- Direct installation button for Cursor IDE in README
- Streamlined setup process for Cursor users
- Enhanced developer experience for AI-powered coding
- **Add Homebrew installation instructions** - Official Homebrew tap support
([`39f811f`](https://github.com/basicmachines-co/basic-memory/commit/39f811f8b57dd998445ae43537cd492c680b2e11))
- Official Homebrew formula in basicmachines-co/basic-memory tap
- Simplified installation process for macOS users
- Package manager integration for easier dependency management
## v0.14.0 (2025-06-26)
### Features
- **Docker Container Registry Migration** - Switch from Docker Hub to GitHub Container Registry for better security and integration
([`616c1f0`](https://github.com/basicmachines-co/basic-memory/commit/616c1f0710da59c7098a5f4843d4f017877ff7b2))
- Automated Docker image publishing via GitHub Actions CI/CD pipeline
- Enhanced container security with GitHub's integrated vulnerability scanning
- Streamlined container deployment workflow for production environments
- **Enhanced Search Documentation** - Comprehensive search syntax examples for improved user experience
([`a589f8b`](https://github.com/basicmachines-co/basic-memory/commit/a589f8b894e78cce01eb25656856cfea8785fbbf))
- Detailed examples for Boolean search operators (AND, OR, NOT)
- Advanced search patterns including phrase matching and field-specific queries
- User-friendly documentation for complex search scenarios
- **Cross-Project File Management** - Intelligent move operations with project boundary detection
([`db5ef7d`](https://github.com/basicmachines-co/basic-memory/commit/db5ef7d35cc0894309c7a57b5741c9dd978526d4))
- Automatic detection of cross-project move attempts with helpful guidance
- Clear error messages when attempting unsupported cross-project operations
### Bug Fixes
- **#184**: Preserve permalinks when editing notes without frontmatter permalinks
([`c2f4b63`](https://github.com/basicmachines-co/basic-memory/commit/c2f4b632cf04921b1a3c2f0d43831b80c519cb31))
- Fix permalink preservation during note editing operations
- Ensure consistent permalink handling across different note formats
- Maintain note identity and searchability during incremental edits
- **#183**: Implement project-specific sync status checks for MCP tools
([`12b5152`](https://github.com/basicmachines-co/basic-memory/commit/12b51522bc953fca117fc5bc01fcb29c6ca7e13c))
- Fix sync status reporting to correctly reflect current project state
- Resolve inconsistencies where sync status showed global instead of project-specific information
- Improve project isolation for sync operations and status reporting
- **#180**: Handle Boolean search syntax with hyphenated terms
([`546e3cd`](https://github.com/basicmachines-co/basic-memory/commit/546e3cd8db98b74f746749d41887f8a213cd0b11))
- Fix search parsing issues with hyphenated terms in Boolean queries
- Improve search query tokenization for complex term structures
- Enhanced search reliability for technical documentation and multi-word concepts
- **#174**: Respect BASIC_MEMORY_HOME environment variable in Docker containers
([`9f1db23`](https://github.com/basicmachines-co/basic-memory/commit/9f1db23c78d4648e2c242ad1ee27eed85e3f3b5d))
- Fix Docker container configuration to properly honor custom home directory settings
- Improve containerized deployment flexibility with environment variable support
- Ensure consistent behavior between local and containerized installations
- **#168**: Scope entity queries by project_id in upsert_entity method
([`2a3adc1`](https://github.com/basicmachines-co/basic-memory/commit/2a3adc109a3e4d7ccd65cae4abf63d9bb2338326))
- Fix entity isolation issues in multi-project setups
- Prevent cross-project entity conflicts during database operations
- Strengthen project boundary enforcement at the database level
- **#166**: Handle None from_entity in Context API RelationSummary
([`8a065c3`](https://github.com/basicmachines-co/basic-memory/commit/8a065c32f4e41613207d29aafc952a56e3a52241))
- Fix null pointer exceptions in relation processing
- Improve error handling for incomplete relation data
- Enhanced stability for knowledge graph traversal operations
- **#164**: Remove log level configuration from mcp_server.run()
([`224e4bf`](https://github.com/basicmachines-co/basic-memory/commit/224e4bf9e4438c44a82ffc21bd1a282fe9087690))
- Simplify MCP server startup by removing redundant log level settings
- Fix potential logging configuration conflicts
- Streamline server initialization process
- **#162**: Ensure permalinks are generated for entities with null permalinks during move operations
([`f506507`](https://github.com/basicmachines-co/basic-memory/commit/f50650763dbd4322c132e4bdc959ce4bf074374b))
- Fix move operations for entities without existing permalinks
- Automatic permalink generation during file move operations
- Maintain database consistency during file reorganization
### Technical Improvements
- **Comprehensive Test Coverage** - Extensive test suites for new features and edge cases
- Enhanced test coverage for project-specific sync status functionality
- Additional test scenarios for search syntax validation and edge cases
- Integration tests for Docker CI workflow and container publishing
- Comprehensive move operations testing with project boundary validation
- **Docker CI/CD Pipeline** - Production-ready automated container publishing
([`74847cc`](https://github.com/basicmachines-co/basic-memory/commit/74847cc3807b0c6ed511e0d83e0d560e9f07ec44))
- Automated Docker image building and publishing on release
- Multi-architecture container support for AMD64 and ARM64 platforms
- Integrated security scanning and vulnerability assessments
- Streamlined deployment pipeline for production environments
- **Release Process Improvements** - Enhanced automation and quality gates
([`a52ce1c`](https://github.com/basicmachines-co/basic-memory/commit/a52ce1c8605ec2cd450d1f909154172cbc30faa2))
- Homebrew formula updates limited to stable releases only
- Improved release automation with better quality control
- Enhanced CI/CD pipeline reliability and error handling
- **Code Quality Enhancements** - Improved error handling and validation
- Better null safety in entity and relation processing
- Enhanced project isolation validation throughout the codebase
- Improved error messages and user guidance for edge cases
- Strengthened database consistency guarantees across operations
### Infrastructure
- **GitHub Container Registry Integration** - Modern container infrastructure
- Migration from Docker Hub to GitHub Container Registry (ghcr.io)
- Improved security with integrated vulnerability scanning
- Better integration with GitHub-based development workflow
- Enhanced container versioning and artifact management
- **Enhanced CI/CD Workflows** - Robust automated testing and deployment
- Automated Docker image publishing on releases
- Comprehensive test coverage validation before deployment
- Multi-platform container building and publishing
- Integration with GitHub's security and monitoring tools
### Migration Guide
This release includes several behind-the-scenes improvements and fixes. All changes are backward compatible:
- **Docker Users**: Container images now served from `ghcr.io/basicmachines-co/basic-memory` instead of Docker Hub
- **Search Users**: Enhanced search syntax handling - existing queries continue to work unchanged
- **Multi-Project Users**: Improved project isolation - all existing projects remain fully functional
- **All Users**: Enhanced stability and error handling - no breaking changes to existing workflows
### Installation
```bash
# Latest stable release
uv tool install basic-memory
# Update existing installation
uv tool upgrade basic-memory
# Docker (new registry)
docker pull ghcr.io/basicmachines-co/basic-memory:latest
```
## v0.13.7 (2025-06-19)
+63 -26
View File
@@ -1,34 +1,71 @@
Developer Certificate of Origin
Version 1.1
https://developercertificate.org/
# Contributor License Agreement
Copyright (C) 2004, 2006 The Linux Foundation and its contributors.
## Copyright Assignment and License Grant
Everyone is permitted to copy and distribute verbatim copies of this
license document, but changing it is not allowed.
By signing this Contributor License Agreement ("Agreement"), you accept and agree to the following terms and conditions
for your present and future Contributions submitted
to Basic Machines LLC. Except for the license granted herein to Basic Machines LLC and recipients of software
distributed by Basic Machines LLC, you reserve all right,
title, and interest in and to your Contributions.
Developer's Certificate of Origin 1.1
### 1. Definitions
By making a contribution to this project, I certify that:
"You" (or "Your") shall mean the copyright owner or legal entity authorized by the copyright owner that is making this
Agreement with Basic Machines LLC.
(a) The contribution was created in whole or in part by me and I
have the right to submit it under the open source license
indicated in the file; or
"Contribution" shall mean any original work of authorship, including any modifications or additions to an existing work,
that is intentionally submitted by You to Basic
Machines LLC for inclusion in, or documentation of, any of the products owned or managed by Basic Machines LLC (the "
Work").
(b) The contribution is based upon previous work that, to the best
of my knowledge, is covered under an appropriate open source
license and I have the right under that license to submit that
work with modifications, whether created in whole or in part
by me, under the same open source license (unless I am
permitted to submit under a different license), as indicated
in the file; or
### 2. Grant of Copyright License
(c) The contribution was provided directly to me by some other
person who certified (a), (b) or (c) and I have not modified
it.
Subject to the terms and conditions of this Agreement, You hereby grant to Basic Machines LLC and to recipients of
software distributed by Basic Machines LLC a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the
Work, and to permit persons to whom the Work is furnished to do so.
(d) I understand and agree that this project and the contribution
are public and that a record of the contribution (including all
personal information I submit with it, including my sign-off) is
maintained indefinitely and may be redistributed consistent with
this project or the open source license(s) involved.
### 3. Assignment of Copyright
You hereby assign to Basic Machines LLC all right, title, and interest worldwide in all Copyright covering your
Contributions. Basic Machines LLC may license the
Contributions under any license terms, including copyleft, permissive, commercial, or proprietary licenses.
### 4. Grant of Patent License
Subject to the terms and conditions of this Agreement, You hereby grant to Basic Machines LLC and to recipients of
software distributed by Basic Machines LLC a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to
make, have made, use, offer to sell, sell, import, and
otherwise transfer the Work.
### 5. Developer Certificate of Origin
By making a Contribution to this project, You certify that:
(a) The Contribution was created in whole or in part by You and You have the right to submit it under this Agreement; or
(b) The Contribution is based upon previous work that, to the best of Your knowledge, is covered under an appropriate
open source license and You have the right under that
license to submit that work with modifications, whether created in whole or in part by You, under this Agreement; or
(c) The Contribution was provided directly to You by some other person who certified (a), (b) or (c) and You have not
modified it.
(d) You understand and agree that this project and the Contribution are public and that a record of the Contribution (
including all personal information You submit with
it, including Your sign-off) is maintained indefinitely and may be redistributed consistent with this project or the
open source license(s) involved.
### 6. Representations
You represent that you are legally entitled to grant the above license and assignment. If your employer(s) has rights to
intellectual property that you create that
includes your Contributions, you represent that you have received permission to make Contributions on behalf of that
employer, or that your employer has waived such rights
for your Contributions to Basic Machines LLC.
---
This Agreement is effective as of the date you first submit a Contribution to Basic Machines LLC.
+32 -118
View File
@@ -14,15 +14,15 @@ See the [README.md](README.md) file for a project overview.
### Build and Test Commands
- Install: `just install` or `pip install -e ".[dev]"`
- Run tests: `uv run pytest -p pytest_mock -v` or `just test`
- Install: `make install` or `pip install -e ".[dev]"`
- Run tests: `uv run pytest -p pytest_mock -v` or `make test`
- Single test: `pytest tests/path/to/test_file.py::test_function_name`
- Lint: `just lint` or `ruff check . --fix`
- Type check: `just type-check` or `uv run pyright`
- Format: `just format` or `uv run ruff format .`
- Run all code checks: `just check` (runs lint, format, type-check, test)
- Create db migration: `just migration "Your migration message"`
- Run development MCP Inspector: `just run-inspector`
- Lint: `make lint` or `ruff check . --fix`
- Type check: `make type-check` or `uv run pyright`
- Format: `make format` or `uv run ruff format .`
- Run all code checks: `make check` (runs lint, format, type-check, test)
- Create db migration: `make migration m="Your migration message"`
- Run development MCP Inspector: `make run-inspector`
### Code Style Guidelines
@@ -37,7 +37,6 @@ See the [README.md](README.md) file for a project overview.
- API uses FastAPI for endpoints
- Follow the repository pattern for data access
- Tools communicate to api routers via the httpx ASGI client (in process)
- avoid using "private" functions in modules or classes (prepended with _)
### Codebase Architecture
@@ -65,7 +64,6 @@ See the [README.md](README.md) file for a project overview.
- Test database uses in-memory SQLite
- Avoid creating mocks in tests in most circumstances.
- Each test runs in a standalone environment with in memory SQLite and tmp_file directory
- Do not use mocks in tests if possible. Tests run with an in memory sqlite db, so they are not needed. See fixtures in conftest.py
## BASIC MEMORY PRODUCT USAGE
@@ -97,29 +95,18 @@ See the [README.md](README.md) file for a project overview.
**Content Management:**
- `write_note(title, content, folder, tags)` - Create/update markdown notes with semantic observations and relations
- `read_note(identifier, page, page_size)` - Read notes by title, permalink, or memory:// URL with knowledge graph awareness
- `edit_note(identifier, operation, content)` - Edit notes incrementally (append, prepend, find/replace, section replace)
- `move_note(identifier, destination_path)` - Move notes with database consistency and search reindexing
- `view_note(identifier)` - Display notes as formatted artifacts for better readability in Claude Desktop
- `read_content(path)` - Read raw file content (text, images, binaries) without knowledge graph processing
- `delete_note(identifier)` - Delete notes from knowledge base
**Project Management:**
- `list_memory_projects()` - List all available projects with status indicators
- `switch_project(project_name)` - Switch to different project context during conversations
- `get_current_project()` - Show currently active project with statistics
- `create_memory_project(name, path, set_default)` - Create new Basic Memory projects
- `delete_project(name)` - Delete projects from configuration and database
- `set_default_project(name)` - Set default project in config
- `sync_status()` - Check file synchronization status and background operations
- `read_note(identifier, page, page_size)` - Read notes by title, permalink, or memory:// URL with knowledge graph
awareness
- `read_file(path)` - Read raw file content (text, images, binaries) without knowledge graph processing
**Knowledge Graph Navigation:**
- `build_context(url, depth, timeframe)` - Navigate the knowledge graph via memory:// URLs for conversation continuity
- `recent_activity(type, depth, timeframe)` - Get recently updated information with specified timeframe (e.g., "1d", "1 week")
- `list_directory(dir_name, depth, file_name_glob)` - List directory contents with filtering and depth control
- `build_context(url, depth, timeframe)` - Navigate the knowledge graph via memory:// URLs for conversation
continuity
- `recent_activity(type, depth, timeframe)` - Get recently updated information with specified timeframe (e.g., "
1d", "1 week")
**Search & Discovery:**
- `search_notes(query, page, page_size)` - Full-text search across all content with filtering options
- `search(query, page, page_size)` - Full-text search across all content with filtering options
**Visualization:**
- `canvas(nodes, edges, title, folder)` - Generate Obsidian canvas files for knowledge graph visualization
@@ -127,7 +114,7 @@ See the [README.md](README.md) file for a project overview.
- MCP Prompts for better AI interaction:
- `ai_assistant_guide()` - Guidance on effectively using Basic Memory tools for AI assistants
- `continue_conversation(topic, timeframe)` - Continue previous conversations with relevant historical context
- `search_notes(query, after_date)` - Search with detailed, formatted results for better context understanding
- `search(query, after_date)` - Search with detailed, formatted results for better context understanding
- `recent_activity(timeframe)` - View recently changed items with formatted output
- `json_canvas_spec()` - Full JSON Canvas specification for Obsidian visualization
@@ -147,32 +134,30 @@ could achieve independently.
## GitHub Integration
Basic Memory uses Claude directly into the development workflow through GitHub:
Basic Memory has taken AI-Human collaboration to the next level by integrating Claude directly into the development workflow through GitHub:
### GitHub MCP Tools
Using the GitHub Model Context Protocol server, Claude can:
Using the GitHub Model Context Protocol server, Claude can now:
- **Repository Management**:
- View repository files and structure
- Read file contents
- Create new branches
- Create and update files
- View repository files and structure
- Read file contents
- Create new branches
- Create and update files
- **Issue Management**:
- Create new issues
- Comment on existing issues
- Close and update issues
- Search across issues
- Create new issues
- Comment on existing issues
- Close and update issues
- Search across issues
- **Pull Request Workflow**:
- Create pull requests
- Review code changes
- Add comments to PRs
- Create pull requests
- Review code changes
- Add comments to PRs
This integration enables Claude to participate as a full team member in the development process, not just as a code
generation tool. Claude's GitHub account ([bm-claudeai](https://github.com/bm-claudeai)) is a member of the Basic
Machines organization with direct contributor access to the codebase.
This integration enables Claude to participate as a full team member in the development process, not just as a code generation tool. Claude's GitHub account ([bm-claudeai](https://github.com/bm-claudeai)) is a member of the Basic Machines organization with direct contributor access to the codebase.
### Collaborative Development Process
@@ -183,75 +168,4 @@ With GitHub integration, the development workflow includes:
3. **Branch management** - Claude can create feature branches for implementations
4. **Documentation maintenance** - Claude can keep documentation updated as the code evolves
With this integration, the AI assistant is a full-fledged team member rather than just a tool for generating code
snippets.
### Basic Memory Pro
Basic Memory Pro is a desktop GUI application that wraps the basic-memory CLI/MCP tools:
- Built with Tauri (Rust), React (TypeScript), and a Python FastAPI sidecar
- Provides visual knowledge graph exploration and project management
- Uses the same core codebase but adds a desktop-friendly interface
- Project configuration is shared between CLI and Pro versions
- Multiple project support with visual switching interface
local repo: /Users/phernandez/dev/basicmachines/basic-memory-pro
github: https://github.com/basicmachines-co/basic-memory-pro
## Release and Version Management
Basic Memory uses `uv-dynamic-versioning` for automatic version management based on git tags:
### Version Types
- **Development versions**: Automatically generated from commits (e.g., `0.12.4.dev26+468a22f`)
- **Beta releases**: Created by tagging with beta suffixes (e.g., `v0.13.0b1`, `v0.13.0rc1`)
- **Stable releases**: Created by tagging with version numbers (e.g., `v0.13.0`)
### Release Workflows
#### Development Builds (Automatic)
- Triggered on every push to `main` branch
- Publishes dev versions like `0.12.4.dev26+468a22f` to PyPI
- Allows continuous testing of latest changes
- Users install with: `pip install basic-memory --pre --force-reinstall`
#### Beta/RC Releases (Manual)
- Create beta tag: `git tag v0.13.0b1 && git push origin v0.13.0b1`
- Automatically builds and publishes to PyPI as pre-release
- Users install with: `pip install basic-memory --pre`
- Use for milestone testing before stable release
#### Stable Releases (Automated)
- Use the automated release system: `just release v0.13.0`
- Includes comprehensive quality checks (lint, format, type-check, tests)
- Automatically updates version in `__init__.py`
- Creates git tag and pushes to GitHub
- Triggers GitHub Actions workflow for:
- PyPI publication
- Homebrew formula update (requires HOMEBREW_TOKEN secret)
**Manual method (legacy):**
- Create version tag: `git tag v0.13.0 && git push origin v0.13.0`
#### Homebrew Formula Updates
- Automatically triggered after successful PyPI release for **stable releases only**
- **Stable releases** (e.g., v0.13.7) automatically update the main `basic-memory` formula
- **Pre-releases** (dev/beta/rc) are NOT automatically updated - users must specify version manually
- Updates formula in `basicmachines-co/homebrew-basic-memory` repo
- Requires `HOMEBREW_TOKEN` secret in GitHub repository settings:
- Create a fine-grained Personal Access Token with `Contents: Read and Write` and `Actions: Read` scopes on `basicmachines-co/homebrew-basic-memory`
- Add as repository secret named `HOMEBREW_TOKEN` in `basicmachines-co/basic-memory`
- Formula updates include new version URL and SHA256 checksum
### For Development
- **Automated releases**: Use `just release v0.13.x` for stable releases and `just beta v0.13.0b1` for beta releases
- **Quality gates**: All releases require passing lint, format, type-check, and test suites
- **Version management**: Versions automatically derived from git tags via `uv-dynamic-versioning`
- **Configuration**: `pyproject.toml` uses `dynamic = ["version"]`
- **Release automation**: `__init__.py` updated automatically during release process
- **CI/CD**: GitHub Actions handles building and PyPI publication
## Development Notes
- make sure you sign off on commits
This level of integration represents a new paradigm in AI-human collaboration, where the AI assistant becomes a full-fledged team member rather than just a tool for generating code snippets.
+6 -1
View File
@@ -27,7 +27,12 @@ project and how to get started as a developer.
> **Note**: Basic Memory uses [just](https://just.systems) as a modern command runner. Install with `brew install just` or `cargo install just`.
3. **Run the Tests**:
3. **Activate the Virtual Environment**
```bash
source .venv/bin/activate
```
4. **Run the Tests**:
```bash
# Run all tests
just test
+15 -2
View File
@@ -1,5 +1,9 @@
FROM python:3.12-slim-bookworm
# Build arguments for user ID and group ID (defaults to 1000)
ARG UID=1000
ARG GID=1000
# Copy uv from official image
COPY --from=ghcr.io/astral-sh/uv:latest /uv /uvx /bin/
@@ -7,6 +11,11 @@ COPY --from=ghcr.io/astral-sh/uv:latest /uv /uvx /bin/
ENV PYTHONUNBUFFERED=1 \
PYTHONDONTWRITEBYTECODE=1
# Create a group and user with the provided UID/GID
# Check if the GID already exists, if not create appgroup
RUN (getent group ${GID} || groupadd --gid ${GID} appgroup) && \
useradd --uid ${UID} --gid ${GID} --create-home --shell /bin/bash appuser
# Copy the project into the image
ADD . /app
@@ -14,13 +23,17 @@ ADD . /app
WORKDIR /app
RUN uv sync --locked
# Create data directory
RUN mkdir -p /app/data
# Create necessary directories and set ownership
RUN mkdir -p /app/data /app/.basic-memory && \
chown -R appuser:${GID} /app
# Set default data directory and add venv to PATH
ENV BASIC_MEMORY_HOME=/app/data \
PATH="/app/.venv/bin:$PATH"
# Switch to the non-root user
USER appuser
# Expose port
EXPOSE 8000
+22 -82
View File
@@ -13,11 +13,8 @@ Basic Memory lets you build persistent knowledge through natural conversations w
Claude, while keeping everything in simple Markdown files on your computer. It uses the Model Context Protocol (MCP) to
enable any compatible LLM to read and write to your local knowledge base.
- Website: https://basicmemory.com
- Company: https://basicmachines.co
- Website: https://basicmachines.co
- Documentation: https://memory.basicmachines.co
- Discord: https://discord.gg/tyvKNccgqN
- YouTube: https://www.youtube.com/@basicmachines-co
## Pick up your conversation right where you left off
@@ -64,7 +61,8 @@ Memory for Claude Desktop:
npx -y @smithery/cli install @basicmachines-co/basic-memory --client claude
```
This installs and configures Basic Memory without requiring manual edits to the Claude Desktop configuration file. Note: The Smithery installation uses their hosted MCP server, while your data remains stored locally as Markdown files.
This installs and configures Basic Memory without requiring manual edits to the Claude Desktop configuration file. The
Smithery server hosts the MCP server component, while your data remains stored locally as Markdown files.
### Glama.ai
@@ -155,8 +153,7 @@ The note embeds semantic content and links to other topics via simple Markdown f
3. You see this file on your computer in real time in the current project directory (default `~/$HOME/basic-memory`).
- Realtime sync is enabled by default starting with v0.12.0
- Project switching during conversations is supported starting with v0.13.0
- Realtime sync can be enabled via running `basic-memory sync --watch`
4. In a chat with the LLM, you can reference a topic:
@@ -266,13 +263,6 @@ Examples of relations:
```
## Using with VS Code
For one-click installation, click one of the install buttons below...
[![Install with UV in VS Code](https://img.shields.io/badge/VS_Code-UV-0098FF?style=flat-square&logo=visualstudiocode&logoColor=white)](https://insiders.vscode.dev/redirect/mcp/install?name=basic-memory&config=%7B%22command%22%3A%22uvx%22%2C%22args%22%3A%5B%22basic-memory%22%2C%22mcp%22%5D%7D) [![Install with UV in VS Code Insiders](https://img.shields.io/badge/VS_Code_Insiders-UV-24bfa5?style=flat-square&logo=visualstudiocode&logoColor=white)](https://insiders.vscode.dev/redirect/mcp/install?name=basic-memory&config=%7B%22command%22%3A%22uvx%22%2C%22args%22%3A%5B%22basic-memory%22%2C%22mcp%22%5D%7D&quality=insiders)
You can use Basic Memory with VS Code to easily retrieve and store information while coding. Click the installation buttons above for one-click setup, or follow the manual installation instructions below.
### Manual Installation
Add the following JSON block to your User Settings (JSON) file in VS Code. You can do this by pressing `Ctrl + Shift + P` and typing `Preferences: Open User Settings (JSON)`.
@@ -302,6 +292,8 @@ Optionally, you can add it to a file called `.vscode/mcp.json` in your workspace
}
```
You can use Basic Memory with VS Code to easily retrieve and store information while coding.
## Using with Claude Desktop
Basic Memory is built using the MCP (Model Context Protocol) and works with the Claude desktop app (https://claude.ai/):
@@ -325,8 +317,7 @@ for OS X):
}
```
If you want to use a specific project (see [Multiple Projects](docs/User%20Guide.md#multiple-projects)), update your
Claude Desktop
If you want to use a specific project (see [Multiple Projects](#multiple-projects) below), update your Claude Desktop
config:
```json
@@ -336,9 +327,9 @@ config:
"command": "uvx",
"args": [
"basic-memory",
"mcp",
"--project",
"your-project-name",
"mcp"
"your-project-name"
]
}
}
@@ -347,27 +338,23 @@ config:
2. Sync your knowledge:
Basic Memory will sync the files in your project in real time if you make manual edits.
```bash
# One-time sync of local knowledge updates
basic-memory sync
# Run realtime sync process (recommended)
basic-memory sync --watch
```
3. In Claude Desktop, the LLM can now use these tools:
```
write_note(title, content, folder, tags) - Create or update notes
read_note(identifier, page, page_size) - Read notes by title or permalink
edit_note(identifier, operation, content) - Edit notes incrementally (append, prepend, find/replace)
move_note(identifier, destination_path) - Move notes with database consistency
view_note(identifier) - Display notes as formatted artifacts for better readability
build_context(url, depth, timeframe) - Navigate knowledge graph via memory:// URLs
search_notes(query, page, page_size) - Search across your knowledge base
search(query, page, page_size) - Search across your knowledge base
recent_activity(type, depth, timeframe) - Find recently updated information
canvas(nodes, edges, title, folder) - Generate knowledge visualizations
list_memory_projects() - List all available projects with status
switch_project(project_name) - Switch to different project context
get_current_project() - Show current project and statistics
create_memory_project(name, path, set_default) - Create new projects
delete_project(name) - Delete projects from configuration
set_default_project(name) - Set default project
sync_status() - Check file synchronization status
```
5. Example prompts to try:
@@ -378,63 +365,16 @@ sync_status() - Check file synchronization status
"Create a canvas visualization of my project components"
"Read my notes on the authentication system"
"What have I been working on in the past week?"
"Switch to my work-notes project"
"List all my available projects"
"Edit my coffee brewing note to add a new technique"
"Move my old meeting notes to the archive folder"
```
## Futher info
See the [Documentation](https://memory.basicmachines.co/) for more info, including:
- [Complete User Guide](https://memory.basicmachines.co/docs/user-guide)
- [CLI tools](https://memory.basicmachines.co/docs/cli-reference)
- [Managing multiple Projects](https://memory.basicmachines.co/docs/cli-reference#project)
- [Importing data from OpenAI/Claude Projects](https://memory.basicmachines.co/docs/cli-reference#import)
## Installation Options
### Stable Release
```bash
pip install basic-memory
```
### Beta/Pre-releases
```bash
pip install basic-memory --pre
```
### Development Builds
Development versions are automatically published on every commit to main with versions like `0.12.4.dev26+468a22f`:
```bash
pip install basic-memory --pre --force-reinstall
```
### Docker
Run Basic Memory in a container with volume mounting for your Obsidian vault:
```bash
# Clone and start with Docker Compose
git clone https://github.com/basicmachines-co/basic-memory.git
cd basic-memory
# Edit docker-compose.yml to point to your Obsidian vault
# Then start the container
docker-compose up -d
```
Or use Docker directly:
```bash
docker run -d \
--name basic-memory-server \
-v /path/to/your/obsidian-vault:/data/knowledge:rw \
-v basic-memory-config:/root/.basic-memory:rw \
ghcr.io/basicmachines-co/basic-memory:latest
```
See [Docker Setup Guide](docs/Docker.md) for detailed configuration options, multiple project setup, and integration examples.
- [Complete User Guide](https://docs.basicmemory.com/user-guide/)
- [CLI tools](https://docs.basicmemory.com/guides/cli-reference/)
- [Managing multiple Projects](https://docs.basicmemory.com/guides/cli-reference/#project)
- [Importing data from OpenAI/Claude Projects](https://docs.basicmemory.com/guides/cli-reference/#import)
## License
@@ -453,4 +393,4 @@ and submitting PRs.
</picture>
</a>
Built with ♥️ by Basic Machines
Built with ♥️ by Basic Machines
-431
View File
@@ -1,431 +0,0 @@
---
title: AI Assistant Guide
type: note
permalink: docs/ai-assistant-guide
---
> Note: This is an optional document that can be copy/pasted into the project knowledge for an LLM to provide a full description of how it can work with Basic Memory. It is provided as a helpful resource. The tools contain extensive usage description prompts with enable the LLM to understand them.
You can [download](https://github.com/basicmachines-co/basic-memory/blob/main/docs/AI%20Assistant%20Guide.md) the contents of this file from GitHub
# AI Assistant Guide for Basic Memory
This guide helps you, the AI assistant, use Basic Memory tools effectively when working with users. It covers reading, writing, and navigating knowledge through the Model Context Protocol (MCP).
## Quick Reference
**Essential Tools:**
- `write_note()` - Create/update notes (primary tool)
- `read_note()` - Read existing content
- `search_notes()` - Find information
- `edit_note()` - Modify existing notes incrementally (v0.13.0)
- `move_note()` - Organize files with database consistency (v0.13.0)
**Project Management (v0.13.0):**
- `list_projects()` - Show available projects
- `switch_project()` - Change active project
- `get_current_project()` - Current project info
**Key Principles:**
1. **Build connections** - Rich knowledge graphs > isolated notes
2. **Ask permission** - "Would you like me to record this?"
3. **Use exact titles** - For accurate `[[WikiLinks]]`
4. **Leverage v0.13.0** - Edit incrementally, organize proactively, switch projects contextually
## Overview
Basic Memory allows you and users to record context in local Markdown files, building a rich knowledge base through natural conversations. The system automatically creates a semantic knowledge graph from simple text patterns.
- **Local-First**: All data is stored in plain text files on the user's computer
- **Real-Time**: Users see content updates immediately
- **Bi-Directional**: Both you and users can read and edit notes
- **Semantic**: Simple patterns create a structured knowledge graph
- **Persistent**: Knowledge persists across sessions and conversations
## The Importance of the Knowledge Graph
Basic Memory's value comes from connections between notes, not just the notes themselves. When writing notes, your primary goal should be creating a rich, interconnected knowledge graph.
When creating content, focus on:
1. **Increasing Semantic Density**: Add multiple observations and relations to each note
2. **Using Accurate References**: Aim to reference existing entities by their exact titles
3. **Creating Forward References**: Feel free to reference entities that don't exist yet - Basic Memory will resolve these when they're created later
4. **Creating Bidirectional Links**: When appropriate, connect entities from both directions
5. **Using Meaningful Categories**: Add semantic context with appropriate observation categories
6. **Choosing Precise Relations**: Use specific relation types that convey meaning
Remember that a knowledge graph with 10 heavily connected notes is more valuable than 20 isolated notes. Your job is to help build these connections.
## Core Tools Reference
### Essential Content Management
**Writing knowledge** (most important tool):
```
write_note(
title="Search Design",
content="# Search Design\n...",
folder="specs", # Optional
tags=["search", "design"], # v0.13.0: now searchable!
project="work-notes" # v0.13.0: target specific project
)
```
**Reading knowledge:**
```
read_note("Search Design") # By title
read_note("specs/search-design") # By path
read_note("memory://specs/search") # By memory URL
```
**Viewing notes as formatted artifacts (Claude Desktop):**
```
view_note("Search Design") # Creates readable artifact
view_note("specs/search-design") # By permalink
view_note("memory://specs/search") # By memory URL
```
**Incremental editing** (v0.13.0):
```
edit_note(
identifier="Search Design", # Must be EXACT title/permalink (strict matching)
operation="append", # append, prepend, find_replace, replace_section
content="\n## New Section\nContent here..."
)
```
**⚠️ Important:** `edit_note` requires exact identifiers (no fuzzy matching). Use `search_notes()` first if uncertain.
**File organization** (v0.13.0):
```
move_note(
identifier="Old Note", # Must be EXACT title/permalink (strict matching)
destination="archive/old-note.md" # Folders created automatically
)
```
**⚠️ Important:** `move_note` requires exact identifiers (no fuzzy matching). Use `search_notes()` first if uncertain.
### Project Management (v0.13.0)
```
list_projects() # Show available projects
switch_project("work-notes") # Change active project
get_current_project() # Current project info
```
### Search & Discovery
```
search_notes("authentication system") # v0.13.0: includes frontmatter tags
build_context("memory://specs/search") # Follow knowledge graph connections
recent_activity(timeframe="1 week") # Check what's been updated
```
## memory:// URLs Explained
Basic Memory uses a special URL format to reference entities in the knowledge graph:
- `memory://title` - Reference by title
- `memory://folder/title` - Reference by folder and title
- `memory://permalink` - Reference by permalink
- `memory://path/relation_type/*` - Follow all relations of a specific type
- `memory://path/*/target` - Find all entities with relations to target
## Semantic Markdown Format
Knowledge is encoded in standard markdown using simple patterns:
**Observations** - Facts about an entity:
```markdown
- [category] This is an observation #tag1 #tag2 (optional context)
```
**Relations** - Links between entities:
```markdown
- relation_type [[Target Entity]] (optional context)
```
**Common Categories & Relation Types:**
- Categories: `[idea]`, `[decision]`, `[question]`, `[fact]`, `[requirement]`, `[technique]`, `[recipe]`, `[preference]`
- Relations: `relates_to`, `implements`, `requires`, `extends`, `part_of`, `pairs_with`, `inspired_by`, `originated_from`
## When to Record Context
**Always consider recording context when**:
1. Users make decisions or reach conclusions
2. Important information emerges during conversation
3. Multiple related topics are discussed
4. The conversation contains information that might be useful later
5. Plans, tasks, or action items are mentioned
**Protocol for recording context**:
1. Identify valuable information in the conversation
2. Ask the user: "Would you like me to record our discussion about [topic] in Basic Memory?"
3. If they agree, use `write_note` to capture the information
4. If they decline, continue without recording
5. Let the user know when information has been recorded: "I've saved our discussion about [topic] to Basic Memory."
## Understanding User Interactions
Users will interact with Basic Memory in patterns like:
1. **Creating knowledge**:
```
Human: "Let's write up what we discussed about search."
You: I'll create a note capturing our discussion about the search functionality.
[Use write_note() to record the conversation details]
```
2. **Referencing existing knowledge**:
```
Human: "Take a look at memory://specs/search"
You: I'll examine that information.
[Use build_context() to gather related information]
[Then read_note() to access specific content]
```
3. **Finding information**:
```
Human: "What were our decisions about auth?"
You: Let me find that information for you.
[Use search_notes() to find relevant notes]
[Then build_context() to understand connections]
```
4. **Editing existing notes (v0.13.0)**:
```
Human: "Add a section about deployment to my API documentation"
You: I'll add that section to your existing documentation.
[Use edit_note() with operation="append" to add new content]
```
5. **Project management (v0.13.0)**:
```
Human: "Switch to my work project and show recent activity"
You: I'll switch to your work project and check what's been updated recently.
[Use switch_project() then recent_activity()]
```
6. **File organization (v0.13.0)**:
```
Human: "Move my old meeting notes to the archive folder"
You: I'll organize those notes for you.
[Use move_note() to relocate files with database consistency]
```
## Key Things to Remember
1. **Files are Truth**
- All knowledge lives in local files on the user's computer
- Users can edit files outside your interaction
- Changes need to be synced by the user (usually automatic)
- Always verify information is current with `recent_activity()`
2. **Building Context Effectively**
- Start with specific entities
- Follow meaningful relations
- Check recent changes
- Build context incrementally
- Combine related information
3. **Writing Knowledge Wisely**
- Same title+folder overwrites existing notes
- Structure with clear headings and semantic markup
- Use tags for searchability (v0.13.0: frontmatter tags indexed)
- Keep files organized in logical folders
4. **Leverage v0.13.0 Features**
- **Edit incrementally**: Use `edit_note()` for small changes vs rewriting
- **Switch projects**: Change context when user mentions different work areas
- **Organize proactively**: Move old content to archive folders
- **Cross-project operations**: Create notes in specific projects while maintaining context
## Common Knowledge Patterns
### Capturing Decisions
```markdown
---
title: Coffee Brewing Methods
tags: [coffee, brewing, pour-over, techniques] # v0.13.0: Now searchable!
---
# Coffee Brewing Methods
## Context
I've experimented with various brewing methods including French press, pour over, and espresso.
## Decision
Pour over is my preferred method for light to medium roasts because it highlights subtle flavors and offers more control over the extraction.
## Observations
- [technique] Blooming the coffee grounds for 30 seconds improves extraction #brewing
- [preference] Water temperature between 195-205°F works best #temperature
- [equipment] Gooseneck kettle provides better control of water flow #tools
- [timing] Total brew time of 3-4 minutes produces optimal extraction #process
## Relations
- pairs_with [[Light Roast Beans]]
- contrasts_with [[French Press Method]]
- requires [[Proper Grinding Technique]]
- part_of [[Morning Coffee Routine]]
```
### Recording Project Structure
```markdown
# Garden Planning
## Overview
This document outlines the garden layout and planting strategy for this season.
## Observations
- [structure] Raised beds in south corner for sun exposure #layout
- [structure] Drip irrigation system installed for efficiency #watering
- [pattern] Companion planting used to deter pests naturally #technique
## Relations
- contains [[Vegetable Section]]
- contains [[Herb Garden]]
- implements [[Organic Gardening Principles]]
```
### Technical Discussions
```markdown
# Recipe Improvement Discussion
## Key Points
Discussed strategies for improving the chocolate chip cookie recipe.
## Observations
- [issue] Cookies spread too thin when baked at 350°F #texture
- [solution] Chilling dough for 24 hours improves flavor and reduces spreading #technique
- [decision] Will use brown butter instead of regular butter #flavor
## Relations
- improves [[Basic Cookie Recipe]]
- inspired_by [[Bakery-Style Cookies]]
- pairs_with [[Homemade Ice Cream]]
```
## v0.13.0 Workflow Examples
### Multi-Project Conversations
**User:** "I need to update my work documentation and also add a personal recipe note."
**Workflow:**
1. `list_projects()` - Check available projects
2. `write_note(title="Sprint Planning", project="work-notes")` - Work content
3. `write_note(title="Weekend Recipes", project="personal")` - Personal content
### Incremental Note Building
**User:** "Add a troubleshooting section to my setup guide."
**Workflow:**
1. `edit_note(identifier="Setup Guide", operation="append", content="\n## Troubleshooting\n...")`
**User:** "Update the authentication section in my API docs."
**Workflow:**
1. `edit_note(identifier="API Documentation", operation="replace_section", section="## Authentication")`
### Smart File Organization
**User:** "My notes are getting messy in the main folder."
**Workflow:**
1. `move_note("Old Meeting Notes", "archive/2024/old-meetings.md")`
2. `move_note("Project Notes", "projects/client-work/notes.md")`
### Creating Effective Relations
When creating relations:
1. **Reference existing entities** by their exact title: `[[Exact Title]]`
2. **Create forward references** to entities that don't exist yet - they'll be linked automatically when created
3. **Search first** to find existing entities to reference
4. **Use meaningful relation types**: `implements`, `requires`, `part_of` vs generic `relates_to`
**Example workflow:**
1. `search_notes("travel")` to find existing travel-related notes
2. Reference found entities: `- part_of [[Japan Travel Guide]]`
3. Add forward references: `- located_in [[Tokyo]]` (even if Tokyo note doesn't exist yet)
## Common Issues & Solutions
**Missing Content:**
- Try `search_notes()` with broader terms if `read_note()` fails
- Use fuzzy matching: search for partial titles
**Forward References:**
- These are normal! Basic Memory links them automatically when target notes are created
- Inform users: "I've created forward references that will be linked when you create those notes"
**Sync Issues:**
- If information seems outdated, suggest `basic-memory sync`
- Use `recent_activity()` to check if content is current
**Strict Mode for Edit/Move Operations:**
- `edit_note()` and `move_note()` require **exact identifiers** (no fuzzy matching for safety)
- If identifier not found: use `search_notes()` first to find the exact title/permalink
- Error messages will guide you to find correct identifiers
- Example workflow:
```
# ❌ This might fail if identifier isn't exact
edit_note("Meeting Note", "append", "content")
# ✅ Safe approach: search first, then use exact result
results = search_notes("meeting")
edit_note("Meeting Notes 2024", "append", "content") # Use exact title from search
```
## Best Practices
1. **Proactively Record Context**
- Offer to capture important discussions
- Record decisions, rationales, and conclusions
- Link to related topics
- Ask for permission first: "Would you like me to save our discussion about [topic]?"
- Confirm when complete: "I've saved our discussion to Basic Memory"
2. **Create a Rich Semantic Graph**
- **Add meaningful observations**: Include at least 3-5 categorized observations in each note
- **Create deliberate relations**: Connect each note to at least 2-3 related entities
- **Use existing entities**: Before creating a new relation, search for existing entities
- **Verify wikilinks**: When referencing `[[Entity]]`, use exact titles of existing notes
- **Check accuracy**: Use `search_notes()` or `recent_activity()` to confirm entity titles
- **Use precise relation types**: Choose specific relation types that convey meaning (e.g., "implements" instead of "relates_to")
- **Consider bidirectional relations**: When appropriate, create inverse relations in both entities
3. **Structure Content Thoughtfully**
- Use clear, descriptive titles
- Organize with logical sections (Context, Decision, Implementation, etc.)
- Include relevant context and background
- Add semantic observations with appropriate categories
- Use a consistent format for similar types of notes
- Balance detail with conciseness
4. **Navigate Knowledge Effectively**
- Start with specific searches
- Follow relation paths
- Combine information from multiple sources
- Verify information is current
- Build a complete picture before responding
5. **Help Users Maintain Their Knowledge**
- Suggest organizing related topics
- Identify potential duplicates
- Recommend adding relations between topics
- Offer to create summaries of scattered information
- Suggest potential missing relations: "I notice this might relate to [topic], would you like me to add that connection?"
Built with ♥️ by Basic Machines
+47 -16
View File
@@ -15,7 +15,7 @@ Basic Memory provides pre-built Docker images on GitHub Container Registry that
--name basic-memory-server \
-p 8000:8000 \
-v /path/to/your/obsidian-vault:/app/data:rw \
-v basic-memory-config:/root/.basic-memory:rw \
-v basic-memory-config:/app/.basic-memory:rw \
ghcr.io/basicmachines-co/basic-memory:latest
```
@@ -30,7 +30,7 @@ Basic Memory provides pre-built Docker images on GitHub Container Registry that
- "8000:8000"
volumes:
- /path/to/your/obsidian-vault:/app/data:rw
- basic-memory-config:/root/.basic-memory:rw
- basic-memory-config:/app/.basic-memory:rw
environment:
- BASIC_MEMORY_DEFAULT_PROJECT=main
restart: unless-stopped
@@ -67,7 +67,7 @@ docker build -t basic-memory .
docker run -d \
--name basic-memory-server \
-v /path/to/your/obsidian-vault:/app/data:rw \
-v basic-memory-config:/root/.basic-memory:rw \
-v basic-memory-config:/app/.basic-memory:rw \
-e BASIC_MEMORY_DEFAULT_PROJECT=main \
basic-memory
```
@@ -86,11 +86,11 @@ Basic Memory requires several volume mounts for proper operation:
2. **Configuration and Database** (Recommended):
```yaml
- basic-memory-config:/root/.basic-memory:rw
- basic-memory-config:/app/.basic-memory:rw
```
Persistent storage for configuration and SQLite database.
You can edit the basic-memory config.json file located in the /root/.basic-memory/config.json after Basic Memory starts.
You can edit the basic-memory config.json file located in the /app/.basic-memory/config.json after Basic Memory starts.
3. **Multiple Projects** (Optional):
```yaml
@@ -98,7 +98,7 @@ You can edit the basic-memory config.json file located in the /root/.basic-memor
- /path/to/project2:/app/data/project2:rw
```
You can edit the basic-memory config.json file located in the /root/.basic-memory/config.json
You can edit the basic-memory config.json file located in the /app/.basic-memory/config.json
## CLI Commands via Docker
@@ -123,7 +123,7 @@ When using Docker volumes, you'll need to configure projects to point to your mo
1. **Check current configuration:**
```bash
docker exec basic-memory-server cat /root/.basic-memory/config.json
docker exec basic-memory-server cat /app/.basic-memory/config.json
```
2. **Add a project for your mounted volume:**
@@ -184,16 +184,47 @@ environment:
### Linux/macOS
Ensure your knowledge directories have proper permissions:
The Docker container now runs as a non-root user to avoid file ownership issues. By default, the container uses UID/GID 1000, but you can customize this to match your user:
```bash
# Make directories readable/writable
chmod -R 755 /path/to/your/obsidian-vault
# Build with custom UID/GID to match your user
docker build --build-arg UID=$(id -u) --build-arg GID=$(id -g) -t basic-memory .
# If using specific user/group
chown -R $USER:$USER /path/to/your/obsidian-vault
# Or use docker-compose with build args
```
**Example docker-compose.yml with custom user:**
```yaml
version: '3.8'
services:
basic-memory:
build:
context: .
dockerfile: Dockerfile
args:
UID: 1000 # Replace with your UID
GID: 1000 # Replace with your GID
container_name: basic-memory-server
ports:
- "8000:8000"
volumes:
- /path/to/your/obsidian-vault:/app/data:rw
- basic-memory-config:/app/.basic-memory:rw
environment:
- BASIC_MEMORY_DEFAULT_PROJECT=main
restart: unless-stopped
```
**Using pre-built images:**
If using the pre-built image from GitHub Container Registry, files will be created with UID/GID 1000. You can either:
1. Change your local directory ownership to match:
```bash
sudo chown -R 1000:1000 /path/to/your/obsidian-vault
```
2. Or build your own image with custom UID/GID as shown above.
### Windows
When using Docker Desktop on Windows, ensure the directories are shared:
@@ -217,7 +248,7 @@ When using Docker Desktop on Windows, ensure the directories are shared:
```
2. **Configuration Not Persisting:**
- Use named volumes for `/root/.basic-memory`
- Use named volumes for `/app/.basic-memory`
- Check volume mount permissions
3. **Network Connectivity:**
@@ -243,10 +274,10 @@ docker-compose logs -f basic-memory
## Security Considerations
1. **Docker Security:**
The container runs as root for simplicity. For production, consider additional security measures.
The container runs as a non-root user (UID/GID 1000 by default) for improved security. You can customize the user ID using build arguments to match your local user.
2. **Volume Permissions:**
Ensure mounted directories have appropriate permissions and don't expose sensitive data.
Ensure mounted directories have appropriate permissions and don't expose sensitive data. With the non-root container, files will be created with the specified user ownership.
3. **Network Security:**
If using HTTP transport, consider using reverse proxy with SSL/TLS and authentication if the endpoint is available on
@@ -288,7 +319,7 @@ For Docker-specific issues:
1. Check the [troubleshooting section](#troubleshooting) above
2. Review container logs: `docker-compose logs basic-memory`
3. Verify volume mounts: `docker inspect basic-memory-server`
4. Test file permissions: `docker exec basic-memory-server ls -la /root`
4. Test file permissions: `docker exec basic-memory-server ls -la /app`
For general Basic Memory support, see the main [README](../README.md)
and [documentation](https://memory.basicmachines.co/).
+241
View File
@@ -0,0 +1,241 @@
# Character Handling and Conflict Resolution
Basic Memory handles various character encoding scenarios and file naming conventions to provide consistent permalink generation and conflict resolution. This document explains how the system works and how to resolve common character-related issues.
## Overview
Basic Memory uses a sophisticated system to generate permalinks from file paths while maintaining consistency across different operating systems and character encodings. The system normalizes file paths and generates unique permalinks to prevent conflicts.
## Character Normalization Rules
### 1. Permalink Generation
When Basic Memory processes a file path, it applies these normalization rules:
```
Original: "Finance/My Investment Strategy.md"
Permalink: "finance/my-investment-strategy"
```
**Transformation process:**
1. Remove file extension (`.md`)
2. Convert to lowercase (case-insensitive)
3. Replace spaces with hyphens
4. Replace underscores with hyphens
5. Handle international characters (transliteration for Latin, preservation for non-Latin)
6. Convert camelCase to kebab-case
### 2. International Character Support
**Latin characters with diacritics** are transliterated:
- `ø``o` (Søren → soren)
- `ü``u` (Müller → muller)
- `é``e` (Café → cafe)
- `ñ``n` (Niño → nino)
**Non-Latin characters** are preserved:
- Chinese: `中文/测试文档.md``中文/测试文档`
- Japanese: `日本語/文書.md``日本語/文書`
## Common Conflict Scenarios
### 1. Hyphen vs Space Conflicts
**Problem:** Files with existing hyphens conflict with generated permalinks from spaces.
**Example:**
```
File 1: "basic memory bug.md" → permalink: "basic-memory-bug"
File 2: "basic-memory-bug.md" → permalink: "basic-memory-bug" (CONFLICT!)
```
**Resolution:** The system automatically resolves this by adding suffixes:
```
File 1: "basic memory bug.md" → permalink: "basic-memory-bug"
File 2: "basic-memory-bug.md" → permalink: "basic-memory-bug-1"
```
**Best Practice:** Choose consistent naming conventions within your project.
### 2. Case Sensitivity Conflicts
**Problem:** Different case variations that normalize to the same permalink.
**Example on macOS:**
```
Directory: Finance/investment.md
Directory: finance/investment.md (different on filesystem, same permalink)
```
**Resolution:** Basic Memory detects case conflicts and prevents them during sync operations with helpful error messages.
**Best Practice:** Use consistent casing for directory and file names.
### 3. Character Encoding Conflicts
**Problem:** Different Unicode normalizations of the same logical character.
**Example:**
```
File 1: "café.md" (é as single character)
File 2: "café.md" (e + combining accent)
```
**Resolution:** Basic Memory normalizes Unicode characters using NFD normalization to detect these conflicts.
### 4. Forward Slash Conflicts
**Problem:** Forward slashes in frontmatter or file names interpreted as path separators.
**Example:**
```yaml
---
permalink: finance/investment/strategy
---
```
**Resolution:** Basic Memory validates frontmatter permalinks and warns about path separator conflicts.
## Error Messages and Troubleshooting
### "UNIQUE constraint failed: entity.file_path, entity.project_id"
**Cause:** Two entities trying to use the same file path within a project.
**Common scenarios:**
1. File move operation where destination is already occupied
2. Case sensitivity differences on macOS
3. Character encoding conflicts
4. Concurrent file operations
**Resolution steps:**
1. Check for duplicate file names with different cases
2. Look for files with similar names but different character encodings
3. Rename conflicting files to have unique names
4. Run sync again after resolving conflicts
### "File path conflict detected during move"
**Cause:** Enhanced conflict detection preventing potential database integrity violations.
**What this means:** The system detected that moving a file would create a conflict before attempting the database operation.
**Resolution:** Follow the specific guidance in the error message, which will indicate the type of conflict detected.
## Best Practices
### 1. File Naming Conventions
**Recommended patterns:**
- Use consistent casing (prefer lowercase)
- Use hyphens instead of spaces for multi-word files
- Avoid special characters that could conflict with path separators
- Be consistent with directory structure casing
**Examples:**
```
✅ Good:
- finance/investment-strategy.md
- projects/basic-memory-features.md
- docs/api-reference.md
❌ Problematic:
- Finance/Investment Strategy.md (mixed case, spaces)
- finance/Investment Strategy.md (inconsistent case)
- docs/API/Reference.md (mixed case directories)
```
### 2. Permalink Management
**Custom permalinks in frontmatter:**
```yaml
---
type: knowledge
permalink: custom-permalink-name
---
```
**Guidelines:**
- Use lowercase permalinks
- Use hyphens for word separation
- Avoid path separators unless creating sub-paths
- Ensure uniqueness within your project
### 3. Directory Structure
**Consistent casing:**
```
✅ Good:
finance/
investment-strategies.md
portfolio-management.md
❌ Problematic:
Finance/ (capital F)
investment-strategies.md
finance/ (lowercase f)
portfolio-management.md
```
## Migration and Cleanup
### Identifying Conflicts
Use Basic Memory's built-in conflict detection:
```bash
# Sync will report conflicts
basic-memory sync
# Check sync status for warnings
basic-memory status
```
### Resolving Existing Conflicts
1. **Identify conflicting files** from sync error messages
2. **Choose consistent naming convention** for your project
3. **Rename files** to follow the convention
4. **Re-run sync** to verify resolution
### Bulk Renaming Strategy
For projects with many conflicts:
1. **Backup your project** before making changes
2. **Standardize on lowercase** file and directory names
3. **Replace spaces with hyphens** in file names
4. **Use consistent character encoding** (UTF-8)
5. **Test sync after each batch** of changes
## System Enhancements
### Recent Improvements (v0.13+)
1. **Enhanced conflict detection** before database operations
2. **Improved error messages** with specific resolution guidance
3. **Character normalization utilities** for consistent handling
4. **File swap detection** for complex move scenarios
5. **Proactive conflict warnings** during permalink resolution
### Monitoring and Logging
The system now provides detailed logging for conflict resolution:
```
DEBUG: Detected potential file path conflicts for 'Finance/Investment.md': ['finance/investment.md']
WARNING: File path conflict detected during move: entity_id=123 trying to move from 'old.md' to 'new.md'
```
These logs help identify and resolve conflicts before they cause sync failures.
## Support and Resources
If you encounter character-related conflicts not covered in this guide:
1. **Check the logs** for specific conflict details
2. **Review error messages** for resolution guidance
3. **Report issues** with examples of the conflicting files
4. **Consider the file naming best practices** outlined above
The Basic Memory system is designed to handle most character conflicts automatically while providing clear guidance for manual resolution when needed.
+657
View File
@@ -0,0 +1,657 @@
# Basic Memory Cloud CLI Guide
The Basic Memory Cloud CLI provides seamless integration between local and cloud knowledge bases using a **cloud mode toggle**. When cloud mode is enabled, all your regular `bm` commands work transparently with the cloud instead of locally.
## Overview
The cloud CLI enables you to:
- **Toggle cloud mode** with `bm cloud login` / `bm cloud logout`
- **Use regular commands in cloud mode**: `bm project`, `bm sync`, `bm tool` all work with cloud
- **Bidirectional sync** with rclone bisync (recommended for most users)
- **Direct file access** via rclone mount (alternative workflow)
- **Integrity verification** with `bm cloud check`
- **Automatic project creation** from local directories
## Prerequisites
Before using Basic Memory Cloud, you need:
- **Active Subscription**: An active Basic Memory Cloud subscription is required to access cloud features
- **Subscribe**: Visit [https://basicmemory.com/subscribe](https://basicmemory.com/subscribe) to sign up
If you attempt to log in without an active subscription, you'll receive a "Subscription Required" error with a link to subscribe.
## The Cloud Mode Paradigm
Basic Memory Cloud follows the **Dropbox/iCloud model** - a single cloud space containing all your projects, not per-project connections.
**How it works:**
- One login per machine: `bm cloud login`
- One sync directory: `~/basic-memory-cloud-sync/` (all projects)
- Projects are folders within your cloud space
- All regular commands work in cloud mode
**Why this model:**
- ✅ Single set of credentials (not N per project)
- ✅ One rclone process (not N processes)
- ✅ Familiar pattern (like Dropbox)
- ✅ Simple operations (setup once, sync anytime)
- ✅ Natural scaling (add projects = add folders)
## Quick Start
### 1. Enable Cloud Mode
Authenticate and enable cloud mode for all commands:
```bash
bm cloud login
```
This command will:
1. Open your browser to the Basic Memory Cloud authentication page
2. Prompt you to authorize the CLI application
3. Store your authentication token locally
4. **Enable cloud mode** - all CLI commands now work against cloud
### 2. Set Up Sync
Set up bidirectional file synchronization:
```bash
bm cloud setup
```
This will:
1. Install rclone automatically (if needed)
2. Configure sync credentials
3. Create `~/basic-memory-cloud-sync/` directory
4. Establish initial sync baseline
**Alternative:** Use `bm cloud setup --mount` to set up mount instead of sync.
### 3. Verify Setup
Check that everything is working:
```bash
bm cloud status
```
You should see:
- `Mode: Cloud (enabled)`
- `Cloud instance is healthy`
- Bisync status showing `✓ Initialized`
### 4. Start Using Cloud
Now all your regular commands work with the cloud:
```bash
# List cloud projects
bm project list
# Create cloud project
bm project add "my-research"
# Use MCP tools on cloud
bm tool write-note --title "Hello" --folder "my-research" --content "Test"
# Sync with cloud
bm sync
# Watch mode for continuous sync
bm sync --watch
```
### 5. Disable Cloud Mode
Return to local mode:
```bash
bm cloud logout
```
All commands now work locally again.
## Working with Cloud Projects
**Important:** When cloud mode is enabled, use regular `bm project` commands (not `bm cloud project`).
### Listing Projects
View all projects (cloud projects when cloud mode is enabled):
```bash
# In cloud mode - lists cloud projects
bm project list
# In local mode - lists local projects
bm project list
```
### Creating Projects
Create a new project (creates on cloud when cloud mode is enabled):
```bash
# In cloud mode - creates cloud project
bm project add my-new-project
# Create and set as default
bm project add my-new-project --default
```
### Automatic Project Creation
**New in SPEC-9:** Projects are automatically created when you create local directories!
```bash
# Create a local directory in your sync folder
mkdir ~/basic-memory-cloud-sync/new-project
echo "# Notes" > ~/basic-memory-cloud-sync/new-project/readme.md
# Sync - automatically creates cloud project
bm sync
# Verify - project now exists on cloud
bm project list
```
This Dropbox-like workflow means you don't need to manually coordinate projects between local and cloud.
## File Synchronization
### The `bm sync` Command (Cloud Mode Aware)
The `bm sync` command automatically adapts based on cloud mode:
**In local mode:**
```bash
bm sync # Indexes local files into database
```
**In cloud mode:**
```bash
bm sync # Runs bisync + indexes files
bm sync --watch # Continuous sync every 60 seconds
bm sync --interval 30 # Custom interval
```
The same command works everywhere - no need to remember different commands for local vs cloud!
## Bidirectional Sync (bisync) - Recommended
Bidirectional sync is the **recommended approach** for most users. It provides:
- ✅ Offline access to all files
- ✅ Automatic bidirectional synchronization
- ✅ Conflict detection and resolution
- ✅ Works with any editor or tool
- ✅ Background watch mode
### Setup
Set up bisync (runs automatically if you used `bm cloud setup`):
```bash
bm cloud setup
```
Or set up with custom directory:
```bash
bm cloud setup --dir ~/my-sync-folder
```
### Running Sync
Use the cloud-aware `bm sync` command:
```bash
# Manual sync
bm sync
# Watch mode (continuous sync)
bm sync --watch
# Custom interval (30 seconds)
bm sync --watch --interval 30
```
### Bisync Profiles
Bisync supports three conflict resolution strategies with different safety levels:
| Profile | Conflict Resolution | Max Deletes | Use Case |
|---------|-------------------|-------------|----------|
| **balanced** | newer | 25 | Default, recommended for most users |
| **safe** | none | 10 | Keep both versions on conflict |
| **fast** | newer | 50 | Rapid iteration, higher delete tolerance |
**Profile Details:**
- **safe**:
- Conflict resolution: `none` (creates `.conflict` files for both versions)
- Max delete: 10 files per sync
- Best for: Critical data where you want manual conflict resolution
- **balanced** (default):
- Conflict resolution: `newer` (auto-resolve to most recent file)
- Max delete: 25 files per sync
- Best for: General use with automatic conflict handling
- **fast**:
- Conflict resolution: `newer` (auto-resolve to most recent file)
- Max delete: 50 files per sync
- Best for: Rapid development iteration with less restrictive safety checks
**How to Select a Profile:**
The default profile (`balanced`) is used automatically with `bm sync`:
```bash
# Uses balanced profile (default)
bm sync
```
For advanced control, use `bm cloud bisync` with the `--profile` flag:
```bash
# Use safe mode
bm cloud bisync --profile safe
# Use fast mode
bm cloud bisync --profile fast
# Preview changes with specific profile
bm cloud bisync --profile safe --dry-run
```
**Check Available Profiles:**
```bash
bm cloud status
```
This shows all available profiles with their settings.
**Current Limitations:**
- Profiles are hardcoded and cannot be customized
- No config file option to change default profile
- Profile settings (max_delete, conflict_resolve) cannot be modified without code changes
- Profile selection only available via `bm cloud bisync --profile` (advanced command)
### Establishing New Baseline
If you need to force a complete resync:
```bash
bm cloud bisync --resync
```
**Warning:** This overwrites the sync state. Use only when recovering from errors.
### Checking Sync Status
View current sync status:
```bash
bm cloud status
```
This shows:
- Cloud mode status
- Instance health
- Sync directory location
- Last sync time
- Available bisync profiles
### Verifying Sync Integrity
Check that local and cloud files match:
```bash
# Full integrity check
bm cloud check
# Faster one-way check
bm cloud check --one-way
```
This uses `rclone check` to verify files match without transferring data.
### Working with Bisync
Create and edit files in `~/basic-memory-cloud-sync/`:
```bash
# Create a new note
echo "# My Research" > ~/basic-memory-cloud-sync/my-project/notes.md
# Edit with your favorite editor
code ~/basic-memory-cloud-sync/my-project/
# Sync changes to cloud
bm sync
```
In watch mode, changes sync automatically:
```bash
# Start watch mode
bm sync --watch
# Edit files - they sync automatically every 60 seconds
code ~/basic-memory-cloud-sync/my-project/
```
### Filter Configuration
Bisync uses `.bmignore` patterns from `~/.basic-memory/.bmignore`:
```bash
# View current ignore patterns
cat ~/.basic-memory/.bmignore
# Edit ignore patterns
code ~/.basic-memory/.bmignore
```
Example `.bmignore`:
```gitignore
# This file is used by 'bm cloud bisync' and file sync
# Patterns use standard gitignore-style syntax
# Hidden files (files starting with dot)
- .*
# Basic Memory internal files
- memory.db/**
- memory.db-shm/**
- memory.db-wal/**
- config.json/**
# Version control
- .git/**
# Python
- __pycache__/**
- *.pyc
- .venv/**
# Node.js
- node_modules/**
```
**Key points:**
-**Global configuration** - One ignore file for all projects
-**rclone filter syntax** - Patterns with `- ` prefix
-**Automatic creation** - Created with defaults on first use
-**Shared patterns** - Same patterns used by sync service
## NFS Mount (Direct Access) - Alternative
NFS mount provides direct file system access as an alternative to bisync. Use this if you prefer mounting files like a network drive.
### Setup
Set up mount instead of bisync:
```bash
bm cloud setup --mount
```
### Mounting Files
Mount your cloud files:
```bash
# Mount with default settings
bm cloud mount
# Mount with specific profile
bm cloud mount --profile fast
```
#### Mount Profiles
- **balanced** (default): Balanced caching for general use
- **streaming**: Optimized for large files
- **fast**: Minimal verification for rapid access
### Checking Mount Status
View current mount status:
```bash
bm cloud status --mount
```
### Unmounting Files
Unmount when done:
```bash
bm cloud unmount
```
### Working with Mounted Files
Once mounted, files appear at `~/basic-memory-cloud/`:
```bash
# List cloud files
ls ~/basic-memory-cloud/
# Edit with your favorite editor
code ~/basic-memory-cloud/my-project/
# Changes are immediately synced to cloud
echo "# Notes" > ~/basic-memory-cloud/my-project/readme.md
```
**Note:** Changes are written through to cloud immediately. There's no "sync" step needed.
## Instance Management
### Health Check
Check if your cloud instance is healthy:
```bash
bm cloud status
```
This shows:
- Cloud mode enabled/disabled
- Instance health status
- Instance version
- Sync or mount status
## Troubleshooting
### Authentication Issues
**Problem**: "Authentication failed" or "Invalid token"
**Solution**: Re-authenticate:
```bash
bm cloud logout
bm cloud login
```
### Subscription Issues
**Problem**: "Subscription Required" error when logging in
**Solution**: You need an active Basic Memory Cloud subscription to use cloud features.
1. Visit the subscribe URL shown in the error message
2. Sign up for a subscription
3. Once your subscription is active, run `bm cloud login` again
**Problem**: "Subscription Required" error for existing user
**Solution**: Your subscription may have expired or been cancelled.
1. Check your subscription status at [https://basicmemory.com/account](https://basicmemory.com/account)
2. Renew your subscription if needed
3. Run `bm cloud login` again
Note: Access is immediately restored when your subscription becomes active.
### Sync Issues
**Problem**: "Bisync not initialized"
**Solution**: Run setup or initialize with resync:
```bash
bm cloud setup
# or
bm cloud bisync --resync
```
**Problem**: "Too many deletes" error
**Solution**: Bisync detected many deletions (safety check). Review changes and use a higher delete limit profile or force resync:
```bash
bm cloud bisync --profile fast # Higher delete limit
# or
bm cloud bisync --resync # Force baseline
```
**Problem**: Conflicts detected
**Solution**: Bisync found files changed in both locations. Check sync directory for `.conflict` files:
```bash
ls ~/basic-memory-cloud-sync/**/*.conflict
```
Resolve conflicts manually, then sync again.
### Connection Issues
**Problem**: "Cannot connect to cloud instance"
**Solution**: Check cloud status:
```bash
bm cloud status
```
If instance is down, wait a few minutes and retry. If problem persists, contact support.
### Mount Issues
**Problem**: "Mount point is busy"
**Solution**: Unmount and remount:
```bash
bm cloud unmount
bm cloud mount
```
**Problem**: "Permission denied" when accessing mounted files
**Solution**: Check mount status and remount:
```bash
bm cloud status --mount
bm cloud unmount
bm cloud mount
```
## Security
- **Authentication**: OAuth 2.1 with PKCE flow
- **Tokens**: Stored securely in `~/.basic-memory/auth/token`
- **Transport**: All data encrypted in transit (HTTPS)
- **Credentials**: Scoped S3 credentials for sync/mount (read-write access to your tenant only)
- **Isolation**: Your data is isolated from other tenants
- **Ignore patterns**: Sensitive files (`.env`, credentials) automatically excluded
## Command Reference
### Cloud Mode Management
```bash
bm cloud login # Authenticate and enable cloud mode
bm cloud logout # Disable cloud mode
bm cloud status # Check cloud mode and sync status
bm cloud status --mount # Check cloud mode and mount status
```
### Setup
```bash
bm cloud setup # Setup bisync (default, recommended)
bm cloud setup --mount # Setup mount (alternative)
bm cloud setup --dir ~/sync # Custom sync directory
```
### Project Management (Cloud Mode Aware)
When cloud mode is enabled, these commands work with cloud:
```bash
bm project list # List projects
bm project add <name> # Create project
bm project add <name> --default # Create and set as default
bm project rm <name> # Delete project
bm project set-default <name> # Set default project
```
### File Synchronization
```bash
bm sync # Sync files (local or cloud depending on mode)
bm sync --watch # Continuous sync (cloud mode only)
bm sync --interval 30 # Custom interval for watch mode
# Advanced bisync commands
bm cloud bisync # Run bisync manually
bm cloud bisync --profile safe # Use specific profile
bm cloud bisync --dry-run # Preview changes
bm cloud bisync --resync # Force new baseline
bm cloud bisync --watch # Continuous sync
bm cloud bisync --verbose # Show detailed output
# Integrity verification
bm cloud check # Full integrity check
bm cloud check --one-way # Faster one-way check
```
### Direct File Access (Mount)
```bash
bm cloud mount # Mount cloud files
bm cloud mount --profile fast # Use specific profile
bm cloud unmount # Unmount files
```
## Summary
Basic Memory Cloud provides two workflows:
### Recommended: Bidirectional Sync (bisync)
1. `bm cloud login` - Authenticate once
2. `bm cloud setup` - Configure sync once
3. `bm sync` - Sync anytime (or use `--watch`)
4. Work in `~/basic-memory-cloud-sync/`
5. Changes sync bidirectionally
### Alternative: Direct Mount
1. `bm cloud login` - Authenticate once
2. `bm cloud setup --mount` - Configure mount once
3. `bm cloud mount` - Mount when needed
4. Work in `~/basic-memory-cloud/`
5. Changes write through immediately
Both approaches work seamlessly with cloud mode - all your regular `bm` commands work with either workflow!
+11 -5
View File
@@ -3,6 +3,9 @@
# Install dependencies
install:
pip install -e ".[dev]"
uv sync
@echo ""
@echo "💡 Remember to activate the virtual environment by running: source .venv/bin/activate"
# Run unit tests in parallel
test-unit:
@@ -15,12 +18,15 @@ test-int:
# Run all tests
test: test-unit test-int
# Lint and fix code (calls fix)
lint: fix
# Lint and fix code
lint:
ruff check . --fix
fix:
uv run ruff check --fix --unsafe-fixes src tests
# Type check code
type-check:
typecheck:
uv run pyright
# Clean build artifacts and cache files
@@ -52,7 +58,7 @@ update-deps:
uv sync --upgrade
# Run all code quality checks and tests
check: lint format type-check test
check: lint format typecheck test
# Generate Alembic migration with descriptive message
migration message:
@@ -179,4 +185,4 @@ beta version:
# List all available recipes
default:
@just --list
@just --list
-378
View File
@@ -1,378 +0,0 @@
{"type":"entity","name":"Paul","entityType":"person","observations":["Software developer combining DIY ethics, Free Software principles, and theoretical computer science","Created the Basic Machines project","Values authentic exchange of ideas","Approaches AI interaction with emphasis on genuine technical discussion","Comfortable with uncertainty and open dialogue","Balances practical implementation with broader implications"]}
{"type":"entity","name":"Basic_Machines","entityType":"project","observations":["Local-first knowledge management system","Combines filesystem durability with graph-based knowledge representation","Focuses on enhancing human agency and understanding","Synthesizes DIY ethics, Free Software philosophy, and theoretical computer science","Current focus includes basic-memory system"]}
{"type":"entity","name":"basic-memory","entityType":"software_system","observations":["A core component of Basic Machines","Local-first knowledge management system","Combines filesystem persistence with graph-based knowledge representation","Being implemented collaboratively by Paul and Claude"]}
{"type":"entity","name":"basic-memory_implementation_patterns","entityType":"technical_patterns","observations":["Filesystem is source of truth - all changes write to files first","Clean separation of concerns between models (SQLAlchemy), schemas (Pydantic), and services","Repository pattern for database access","Service layer handling business logic and coordination","Atomic file operations using temporary files for safety","Clear error handling hierarchy with specific error types","Comprehensive test coverage with pytest and fixtures","Async/await used throughout the codebase","Validation using Pydantic models with custom validators"]}
{"type":"entity","name":"fileio_module","entityType":"code_module","observations":["Extracted from EntityService to handle all file operations","Provides read_entity_file, write_entity_file, and delete_entity_file functions","Handles markdown parsing and formatting","Implements atomic file operations","Provides consistent error handling","Enables reuse across services"]}
{"type":"entity","name":"entity_service","entityType":"code_module","observations":["Manages entities in both filesystem and database","Uses fileio module for file operations","Maintains database index of entities","Handles entity creation, retrieval, and deletion","Follows 'filesystem is source of truth' principle","Coordinates with observation service for full entity management"]}
{"type":"entity","name":"observation_service","entityType":"code_module","observations":["Manages observations within entity files","Provides database indexing for efficient observation queries","Works with complete Entity objects rather than IDs","Handles observation addition and search","Maintains consistency between files and database","Under development for update/remove operations"]}
{"type":"entity","name":"observation_management","entityType":"design_challenge","observations":["Key challenge: maintaining observation state across files and database","Exploring bulk update approach - treating all observations as a unit","Considering tracked observations with markdown comments for IDs","Investigating diff-based approach for observation-level changes","Evaluating position-based management without explicit IDs","Trade-offs between implementation complexity and markdown readability"]}
{"type":"entity","name":"testing_infrastructure","entityType":"technical_patterns","observations":["Uses pytest with async support via pytest-asyncio","In-memory SQLite database for test isolation","Temporary directories for file operation testing","Comprehensive fixture system for test setup","Tests organized by component (entity, observation, etc)","Covers happy path, error cases, and edge cases","Uses monkeypatch for mocking dependencies","Clear separation between arrange, act, assert sections","Uses in-memory SQLite database for test isolation","Comprehensive fixture system for test data setup","Proper async test handling with pytest-asyncio"]}
{"type":"entity","name":"test_categories","entityType":"test_suite","observations":["Happy path tests verify core functionality","Error path tests ensure proper error handling","Edge cases test special characters and long content","File operation tests verify atomic writes and rollbacks","Database sync tests verify index consistency","Recovery tests for rebuild operations","Punted on concurrent operation tests due to session management complexity"]}
{"type":"entity","name":"completed_work","entityType":"project_milestone","observations":["Extracted file operations to fileio.py module","Updated EntityService to use fileio functions","Implemented initial ObservationService","Created comprehensive test suite","Established clear project patterns and principles","Set up basic database schema with SQLAlchemy","Created Pydantic models for validation"]}
{"type":"entity","name":"future_work","entityType":"project_tasks","observations":["Implement observation updates/removals","Design proper session management for concurrent operations","Update EntityService tests for new fileio module","Add more sophisticated search functionality","Handle markdown formatting edge cases","Consider versioning for file changes","Implement proper backup strategy"]}
{"type":"entity","name":"design_decisions","entityType":"technical_decisions","observations":["Filesystem as source of truth over database","Markdown format for human readability and editing","Atomic file operations for safety","SQLite + SQLAlchemy for proven reliability","Pydantic for validation and ID generation","Async/await for better scalability","Clear separation between files and database roles","Explicit error hierarchies for better handling"]}
{"type":"entity","name":"concurrency_considerations","entityType":"technical_challenge","observations":["SQLAlchemy session management in async context","File operation atomicity","Transaction isolation levels","Potential for conflicting updates","Need for proper session lifecycle","Possibility of file system race conditions","Database lock management"]}
{"type":"entity","name":"observation_update_approaches","entityType":"design_alternatives","observations":["Each approach trades off between simplicity, efficiency, and robustness","Four main approaches considered: bulk update, tracked IDs, diff-based, and position-based","Discussion revealed importance of human readability in file format","Consideration of manual editing workflows key to design","File system as source of truth principle guides tradeoffs"]}
{"type":"entity","name":"bulk_update_approach","entityType":"design_option","observations":["Update all observations at once in a single operation","Simpler file operations - just rewrite the whole list","No need for observation matching or IDs","Very consistent with source of truth principle","Less efficient for small changes","May have concurrency implications","Simplest implementation option"]}
{"type":"entity","name":"tracked_observations_approach","entityType":"design_option","observations":["Use markdown comments to store observation IDs","Enables precise updates and deletes","IDs stored as HTML comments in markdown","More complex markdown parsing required","IDs visible in raw markdown files","Balances tracking with readability"]}
{"type":"entity","name":"diff_based_approach","entityType":"design_option","observations":["Implement observation-aware diffing","Track changes at observation level","More efficient for updates","Preserves manual edits and changes","More complex implementation needed","Must handle merge conflicts","Most sophisticated option considered"]}
{"type":"entity","name":"position_based_approach","entityType":"design_option","observations":["Track observations by position/order","No explicit IDs needed","Cleanest markdown format","Order changes could break references","Difficult to handle concurrent edits","Most fragile option considered"]}
{"type":"entity","name":"tasks_and_progress","entityType":"project_tracking","observations":["Current focus on observation management implementation","Completed core file operations extraction","Completed EntityService updates","Completed initial ObservationService","Basic test coverage in place","Future work includes concurrent operations","Future work includes search improvements","Need to handle markdown edge cases"]}
{"type":"entity","name":"error_handling_patterns","entityType":"technical_patterns","observations":["Custom exception hierarchy with ServiceError base","Specific error types (FileOperationError, DatabaseSyncError, etc)","Clear separation between file and database errors","Error propagation patterns established","Focus on actionable error messages","Error handling at appropriate levels"]}
{"type":"entity","name":"data_models","entityType":"technical_implementation","observations":["SQLAlchemy models for database structure","Pydantic schemas for API/service layer","Entity model with UUID-based IDs","Observation model with entity relationships","UTCDateTime custom type for timestamps","Automatic ID generation in Pydantic models","Strict validation rules"]}
{"type":"entity","name":"markdown_format","entityType":"file_format","observations":["Simple, human-readable format","Entity name as H1 header","Metadata in key-value format","Observations as bullet points","Atomic file operations for updates","Designed for manual editing","No hidden metadata in main content"]}
{"type":"entity","name":"test_driven_development","entityType":"development_pattern","observations":["Tests revealed need for atomic file operations","Error cases drove error hierarchy design","Edge cases informed validation rules","Test fixtures shaped service interfaces","File operations extracted due to test patterns","Concurrent test issues revealed session management needs"]}
{"type":"entity","name":"architecture_evolution","entityType":"design_process","observations":["Started with simple EntityService implementation","Circular dependency between Entity and Observation services revealed design flaw","Extracted file operations to separate module","Moved to passing Entity objects rather than IDs","Improved separation of concerns through iterations","File operations became reusable across services","Database became true 'index' rather than source of truth"]}
{"type":"entity","name":"validation_patterns","entityType":"technical_patterns","observations":["Pydantic models provide schema validation","Automatic ID generation if not provided","Database constraints via SQLAlchemy","Runtime checks in services","Markdown format validation","Error handling for invalid states"]}
{"type":"entity","name":"markdown_examples","entityType":"documentation","observations":["Example of basic entity:\n# Entity Name\ntype: entity_type\n\n## Observations\n- First observation\n- Second observation","Example with special characters:\n# Test & Entity!\ntype: test\n\n## Observations\n- Test & observation with @#$% special chars!","Format ensures human readability:\n# Basic Machines\ntype: project\n\n## Observations\n- Local-first knowledge management system\n- Combines filesystem durability with graph-based knowledge representation","Future consideration for observation IDs:\n# Entity Name\ntype: entity_type\n\n## Observations\n- <!-- obs-id: abc123 -->\n This is an observation with ID"]}
{"type":"entity","name":"markdown_parsing_rules","entityType":"technical_implementation","observations":["H1 header contains entity name","Metadata uses key: value format","Observations section marked by H2 header","Each observation is a markdown list item","Blank lines separate sections","Special characters allowed in content","No restrictions on observation content"]}
{"type":"entity","name":"schema_definitions","entityType":"technical_documentation","observations":["SQLAlchemy Entity model:\nclass Entity(Base):\n id: str (primary key)\n name: str (unique)\n entity_type: str\n created_at: datetime\n updated_at: datetime","SQLAlchemy Observation model:\nclass Observation(Base):\n id: str (primary key)\n entity_id: str (foreign key)\n content: str\n created_at: datetime\n context: Optional[str]","Pydantic Entity schema:\nclass Entity(BaseModel):\n id: str\n name: str\n entity_type: str\n observations: List[Observation]"]}
{"type":"entity","name":"test_evolution","entityType":"development_history","observations":["Started with basic Entity CRUD tests","Added filesystem verification to all tests","Developed concurrent operation tests (later removed)","Edge case tests drove better error handling","Test fixtures evolved to support both file and DB testing","Mocking patterns for file/DB operations","Special cases for long content and special characters"]}
{"type":"entity","name":"implementation_challenges","entityType":"technical_issues","observations":["Initial circular dependency between services","SQLAlchemy session management in async context","Atomic file operations with proper error handling","Maintaining DB sync with filesystem changes","Handling long content in observations","Managing test isolation with file operations","Deciding on markdown format tradeoffs","Concurrent operation complexity"]}
{"type":"entity","name":"Basic_Factory","entityType":"Project","observations":["Collaborative project between Paul and Claude","Explores AI-human collaboration in software development","Uses MCP tools for file and memory management","Built with git integration capabilities","Focuses on maintaining project context across sessions","About 90% complete with MCP tools","Still needs improvements in collaboration via files/git/github","Will be used to document and share collaborative development process"]}
{"type":"entity","name":"Basic_Factory_Components","entityType":"Technical","observations":["Server-side rendering with JinjaX","HTMX for dynamic updates","Alpine.js for client-side state","Tailwind CSS for styling","Component translation from React/shadcn/ui","Focus on simplicity and understandability","Demonstrates meta-compiler principles in component translation"]}
{"type":"entity","name":"Component_Translation_Process","entityType":"Methodology","observations":["Treats component porting as meta-compilation","Maps between React/TypeScript and JinjaX/Alpine.js domains","Uses formal grammar transformation approaches","Maintains functionality while simplifying implementation","Focuses on server-side rendering patterns","Preserves accessibility and performance","Uses short, focused git branches for each component"]}
{"type":"entity","name":"Basic_Machines_Philosophy","entityType":"Philosophy","observations":["Combines DIY punk ethics with software development","Emphasizes user empowerment and understanding","Values simplicity and composability","Treats complex systems as combinations of simple parts","Focuses on authentic creation and sharing","Draws inspiration from punk rock, Free Software, and theoretical CS","Emphasizes the cycle of creation, complexity, and renewal"]}
{"type":"entity","name":"Basic_Machines_Manifesto","entityType":"Document","observations":["Created through collaboration between Paul and Claude","Explores connection between DIY punk ethics and software development","Emphasizes composition over inheritance in both philosophy and practice","Views software development through lens of basic machines that combine for complex computation","Advocates for user empowerment and technological independence","Structured in sections covering Origins, Philosophy, Technical Implementation, and AI Collaboration","Draws connections between punk rock, free software, and theoretical computer science","Emphasizes importance of sharing knowledge and building community","Released in December 2024"]}
{"type":"entity","name":"AI_Human_Collaboration_Model","entityType":"Methodology","observations":["Focuses on deep collaboration rather than simple task completion","Maintains rich context across sessions via knowledge graph","Uses short, focused git branches for each collaborative session","Values intellectual partnership over simple code generation","Emphasizes both practical implementation and theoretical exploration","Creates space for authentic exchange while maintaining AI/human clarity","Uses formal methods when appropriate (like grammar transformation)","Documents decisions and processes for future reference","Developed through Basic Machines project experience"]}
{"type":"entity","name":"Basic_Machines_Roadmap","entityType":"Project_Plan","observations":["Phase 1 (30 days): Build basic-machines.co website","Phase 2 (60-90 days): Develop premium component bundles","Phase 3 (90-120 days): Launch Basic Foundation commercial offering","Focus on building brand and marketing presence","Prioritize components needed for own site development","Document and share collaboration process","Build sustainable business model aligned with values"]}
{"type":"entity","name":"Basic_Machines_Website","entityType":"Project","observations":["To be built at basic-machines.co","Will showcase products and vision","Needs components for navigation, hero sections, features","Will demonstrate component usage in production","Will include blog for sharing progress","Focus on clear value proposition","Platform for sharing Basic Machines philosophy"]}
{"type":"entity","name":"Basic_Memory_Markdown_Example","entityType":"Example","observations":["Shows complete markdown structure for basic-memory entity","Uses frontmatter for metadata (id, type, created, context)","Has main description section after title","Includes Observations as bullet points","Shows Relations with [id] relation_type | context format","Lists References at bottom","Created during initial design discussion","Serves as canonical example of file format"]}
{"type":"entity","name":"Basic_Memory_Database_Schema","entityType":"Technical","observations":["Uses SQLite for local storage","Entities table with id, name, type, created_at, context, description, references","Observations table linking to entities with content and context","Relations table tracking directional relationships between entities","References column needs quotes as SQL reserved word","Designed for easy rebuilding from markdown files","Foreign key constraints maintain data integrity","Unique constraint on relations prevents duplicates","Created_at timestamps track history","Context fields enable tracking information sources"]}
{"type":"entity","name":"Basic_Memory_Project_Structure","entityType":"Technical","observations":["Uses dbmate for database migrations","Projects directory stores SQLite databases and markdown files","Makefile provides common development commands","Environment vars configure database connection","db/migrations directory for SQL schema changes","Gitignore excludes database files and env config","Uses Python 3.12 with modern tooling","Tests directory for pytest files","Follows Basic Machines project conventions"]}
{"type":"entity","name":"Basic_Memory_Project_Isolation_Decision","entityType":"Decision","observations":["Decided to defer multi-project support to post-MVP","Will use separate SQLite databases per project","Initially using projects directory in code repository","Plan to make location configurable later","No changes needed to core domain model","Keeps initial implementation simple","FTS/search capabilities also deferred for simplicity"]}
{"type":"entity","name":"Basic_Memory_Implementation_Plan","entityType":"Plan","observations":["Start with SQLAlchemy models matching schema","Then build CLI for basic operations","Then implement markdown parser","Use TDD approach throughout","Begin with core domain model","CLI will support CRUD operations","Parser must handle frontmatter and sections","Following modular development approach","Planning to use typer for CLI","Will use modern Python tools and practices"]}
{"type":"entity","name":"Basic_Memory_Implementation_Status","entityType":"Status","observations":["Core modules implemented: models, services, repository, fileio","Modular architecture with clear separation of concerns","File operations extracted to separate fileio module","Initial ObservationService implementation complete","Basic test coverage in place","Exploring observation management strategies","Using SQLAlchemy for database interaction","Markdown file operations working","Entity management functional","Repository layer implementation complete with SQLAlchemy models and tests","Database operations working with proper UTC timestamp handling","In-memory SQLite testing infrastructure proven effective"]}
{"type":"entity","name":"Basic_Memory_Observation_Management_Design","entityType":"Design","observations":["Four approaches under consideration","Bulk Update: Simple but less efficient","Tracked Observations: Precise but clutters markdown","Diff-based: Efficient but complex","Position-based: Clean but fragile","Key challenge is balancing markdown readability with efficient updates","Must maintain filesystem as source of truth","Need to consider concurrent edits","Currently evaluating trade-offs","Implementation choice pending discussion"]}
{"type":"entity","name":"Basic_Memory_Architectural_Decisions","entityType":"Decisions","observations":["Split file operations into separate fileio module","Using SQLAlchemy for database operations","Maintain filesystem as source of truth","Modular service-based architecture","Clear separation between data access and business logic","Repository pattern for database interactions","Schemas separate from models","Focus on maintainability and testability","Services handle business rules","Considering concurrency in design"]}
{"type":"entity","name":"Basic_Memory_Implementation_Analysis","entityType":"Analysis","observations":["Clean modular architecture with clear responsibilities","Strong typing throughout codebase","Excellent error handling with custom exceptions","SQLAlchemy models perfectly match our domain model","Atomic file operations for data safety","Services implement filesystem-as-source-of-truth principle","Async support throughout","Good separation between domain models and database models","Careful handling of UTC timestamps","Smart use of SQLAlchemy relationships"]}
{"type":"entity","name":"Basic_Memory_Current_Challenges","entityType":"Challenges","observations":["Observation update/removal strategy needs to be chosen","Need to handle concurrent file operations safely","Search functionality to be implemented","Edge cases in markdown formatting to be handled","Session management for concurrent operations needed","Balance between file operations and database sync","Testing coverage could be expanded","Need to handle relationship updates in files"]}
{"type":"entity","name":"Basic_Memory_Observation_Hash_Tracking","entityType":"Design","observations":["Use content hashes to track observation identity","Store hashes in database but not in markdown","Can match observations across file edits using hashes","Similar to how git tracks content changes","Keeps markdown clean and human-friendly","Allows efficient bulk updates","Handles reordering of observations","Maintains filesystem as source of truth","No need for visible IDs in markdown","Could track observation history through hash changes"]}
{"type":"entity","name":"Basic_Memory_Repository_Implementation","entityType":"Code_Implementation","observations":["Implemented base Repository class with CRUD operations","Added specialized EntityRepository, ObservationRepository, and RelationRepository","Used string IDs instead of UUIDs","Added UTCDateTime custom type for timestamp handling","Used in-memory SQLite for testing","Achieved 84% test coverage","Created comprehensive pytest fixtures"]}
{"type":"entity","name":"Basic_Memory_Dependencies","entityType":"Technical","observations":["Uses Python 3.12","SQLAlchemy with async support","pytest-asyncio for async testing","aiosqlite for async SQLite operations","greenlet for SQLAlchemy async support","uv for dependency management","pytest-cov for coverage reporting","Development dependencies managed in pyproject.toml"]}
{"type":"entity","name":"Basic_Memory_Current_Architecture","entityType":"Architecture_Analysis","observations":["Clear separation between domain models (Pydantic) and storage models (SQLAlchemy)","File I/O completely separated into dedicated module","Strong 'filesystem as source of truth' pattern in services","Atomic file operations with proper error handling","Service layer coordinates between filesystem and database","Database acts as queryable index rather than primary storage","Clean error hierarchy with specific exception types","Rebuild operations available for recovery scenarios"]}
{"type":"entity","name":"Basic_Memory_Evolution","entityType":"Analysis","observations":["Started with repository pattern following basic-foundation","Evolved to more sophisticated architecture with clear layers","Added Pydantic schemas for domain modeling","Separated file operations into dedicated module","Implemented robust error handling throughout","Maintained filesystem as source of truth principle","Added observation management with context tracking","Introduced rebuild capabilities for system recovery"]}
{"type":"entity","name":"Basic_Memory_Service_Layer","entityType":"Implementation","observations":["EntityService handles entity lifecycle and coordinates storage","ObservationService manages observations within entities","Services ensure filesystem and database stay in sync","Clear error handling with ServiceError hierarchy","Strong typing throughout service interfaces","Implements filesystem as source of truth pattern","Handles UUID generation and timestamp management","Provides methods for system recovery and rebuild"]}
{"type":"entity","name":"Basic_Memory_Schema_Design","entityType":"Implementation","observations":["Uses Pydantic for domain models and validation","Automatic ID generation with timestamp and UUID","Clear separation from SQLAlchemy storage models","Supports optional context tracking","Models match markdown file structure","Enables clean serialization/deserialization","Strong typing with proper validation rules","Independent from storage concerns"]}
{"type":"entity","name":"Basic_Memory_Next_Tasks","entityType":"TaskList","observations":["✅ Implement SQLAlchemy models and repositories (Done)","✅ Add SQLAlchemy migrations (Done)","✅ Create service layer (Done)","✅ Implement file I/O module (Done)","✅ Set up domain models with Pydantic (Done)","✅ Initial test infrastructure (Done)","✅ Basic CRUD operations (Done)","⏳ Implement full test coverage for db.py","⏳ Add more sophisticated search functionality","⏳ Implement CLI interface","⏳ Add relationship management to services","⏳ Handle concurrent file operations safely","⏳ Add versioning for file changes","⏳ Implement proper backup strategy","⏳ Add type hints throughout codebase","⏳ Improve error messages and logging","⏳ Add documentation for core modules"]}
{"type":"entity","name":"Basic_Memory_Meta_Experience","entityType":"Case_Study","observations":["Experienced our own context loss when reconstructing project knowledge","Had to rebuild task list and project context from filesystem and memory","Validated 'filesystem as source of truth' principle through reconstruction","Code and tests served as reliable historical record","Knowledge graph structure helped guide reconstruction process","Markdown files provided human-readable context","Atomic information design made piece-by-piece reconstruction possible","Ironic validation of the need for basic-memory's features","Experience demonstrates value of durable, human-readable knowledge storage","Shows importance of separating durable storage from ephemeral context"]}
{"type":"entity","name":"Model_Context_Protocol","entityType":"protocol","observations":["Core part of the basic-memory architecture","Enables AI-human collaboration on projects","Provides tool-based interaction with knowledge graph","Developed by Anthropic for structured AI-system interaction","Used for maintaining consistent, rich context across conversations"]}
{"type":"entity","name":"basic-memory_core_principles","entityType":"principles","observations":["Local First: All data stored locally in SQLite","Project Isolation: Separate databases per project","Human Readable: Everything exportable to plain text","AI Friendly: Structure optimized for LLM interaction","DIY Ethics: User owns and controls their data","Simple Core: Start simple, expand based on needs","Tool Integration: MCP-based interaction model"]}
{"type":"entity","name":"basic-memory_business_model","entityType":"business_strategy","observations":["Core features free: Local SQLite, basic knowledge graph, search, markdown export, basic MCP tools","Professional features potential: Rich document export, advanced versioning, collaboration features, custom integrations, priority support","Focus on maintaining DIY/punk philosophy while enabling sustainability"]}
{"type":"entity","name":"basic-memory_cli","entityType":"interface","observations":["Supports project management commands (create, switch, list)","Entity management (add entity, add observation, add relation)","Future support for export and batch operations","Follows consistent command structure","Planned integration with MCP tools"]}
{"type":"entity","name":"basic-memory_export_format","entityType":"file_format","observations":["Uses markdown with frontmatter metadata","Includes entity name, type, creation timestamp","Observations as bullet points","Relations in structured format with links","References section at bottom","Designed for human readability and machine parsing","Example format documented in project specs"]}
{"type":"entity","name":"relation_service","entityType":"code_module","observations":["Planned service for managing relations in both filesystem and database","Will follow filesystem-is-source-of-truth principle like other services","Needs to handle atomic file operations for relation updates","Must coordinate with EntityService for relationship integrity","Will handle bidirectional relationship tracking","Will support relation validation and type enforcement","Must implement rebuild functionality for index recovery","Will need careful error handling for file/db sync","Should support relation search and filtering","Must handle relation lifecycle (create/read/update/delete)"]}
{"type":"entity","name":"service_layer_patterns","entityType":"implementation_patterns","observations":["Services handle both file and database operations","Filesystem is always source of truth","Database serves as queryable index","Services implement atomic file operations","Clear error hierarchy with specific exceptions","Use of dependency injection via constructor params","Async/await used throughout service layer","Services coordinate between storage layers","Repository pattern used for database access","Services maintain entity integrity across storage","Rich error types extend from ServiceError base","Rebuild operations available for recovery"]}
{"type":"entity","name":"database_models","entityType":"implementation","observations":["Entity model with unique name and type","Observation model linked to entities","Relation model tracks connections between entities","Custom UTCDateTime type for timestamp handling","Use of SQLAlchemy relationships for navigation","Cascading deletes for dependent objects","String IDs used for compatibility","Rich relationship modeling with backpopulates","Proper indexing on foreign keys","Context tracking available on models","Models include created_at timestamps","Relationships handle bidirectional navigation"]}
{"type":"entity","name":"repository_patterns","entityType":"implementation_patterns","observations":["Generic Repository[T] base class implementation","Type-safe operations with SQLAlchemy","Specialized repositories for each model type","Async operations throughout","Clear error handling patterns","Support for custom queries and filtering","Pagination support built-in","Transaction management via session","Proper type hints and generics usage","Entity-specific query methods in subclasses"]}
{"type":"entity","name":"relation_service_design","entityType":"design","observations":["Must handle relation lifecycle in both files and DB","Needs to validate existence of both entities","Should support relation type enforcement","Must maintain bidirectional consistency","Should support relation querying and filtering","Needs proper error handling for graph consistency","Must integrate with entity file format","Should support bulk operations for efficiency","Must handle relation deletion and cascading","Should provide search by type and entities"]}
{"type":"entity","name":"relation_service_implementation_plan","entityType":"plan","observations":["1. Define core relation operations (create, get, delete)","2. Implement file format handling for relations","3. Add database sync with RelationRepository","4. Implement validation and error handling","5. Add rebuild and recovery operations","6. Implement relation type enforcement","7. Add relation search and filtering","8. Implement bulk operations","9. Add comprehensive tests","10. Document API and error handling"]}
{"type":"entity","name":"relation_service_challenges","entityType":"challenges","observations":["Maintaining consistency between file and database","Handling relation type validation efficiently","Managing bidirectional relationships in files","Ensuring atomic updates across entities","Handling deletion with proper cascading","Efficient querying of relation graphs","Recovery from partial file/db sync failures","Bulk operation atomicity","Clear error reporting for graph operations","Performance with large relation sets"]}
{"type":"entity","name":"relation_file_format","entityType":"file_format","observations":["Relations stored in entity markdown files","Format: [target_id] relation_type | context","Relations section marked by ## Relations header","Outgoing relations only stored in source entity","Relations rebuild on entity load","Clean human-readable format","Context is optional with pipe separator","Links generate valid navigation references","Markdown-friendly formatting","Example: [Paul] authored | with Claude"]}
{"type":"entity","name":"relation_service_error_handling","entityType":"implementation_patterns","observations":["RelationError extends ServiceError base","Specific errors for validation failures","Handles entity not found cases","Manages relation type validation errors","File operation errors properly wrapped","Database sync errors clearly reported","Transaction rollback on errors","Proper error propagation chain","Clear error messages for debugging","Recovery paths for common errors"]}
{"type":"entity","name":"relation_service_testing","entityType":"testing","observations":["Test all relation lifecycle operations","Verify file and database consistency","Test relation type validation","Check error handling paths","Test bulk operations","Verify bidirectional consistency","Test recovery operations","Check cascade operations","Verify search and filtering","Test with large relation sets"]}
{"type":"entity","name":"fileio_patterns","entityType":"implementation_patterns","observations":["Atomic file operations with temporary files","Clear error handling for IO operations","Consistent file naming and paths","Support for different file formats","Efficient file reading and writing","Proper file locking mechanisms","Recovery from partial writes","Consistent encoding handling","Directory management utilities","Path manipulation helpers","Currently implemented in fileio.py module","Uses pathlib for path operations","Handles file not found cases gracefully","Maintains data integrity during writes"]}
{"type":"entity","name":"pytest_patterns","entityType":"implementation_patterns","observations":["Common fixtures should be in conftest.py for reuse","Use pytest_asyncio.fixture for async fixtures","Session fixtures need proper async cleanup","Temporary directories should be managed with context managers","Test categories: happy path, error path, recovery, edge cases","Services need project_path and repo injected","Use monkeypatch for mocking in async context","SQLite in-memory database ideal for testing","Explicit test verification: file content and database state"]}
{"type":"entity","name":"relation_implementation_learnings","entityType":"implementation_learnings","observations":["Better to pass full Entity objects than IDs to services","Services should not re-read entities if they have them","File operations should be atomic and verified","Database serves as queryable index, not source of truth","Relations stored in source entity's markdown file","Clear separation between file ops and database sync","Entity objects should own their relations list","Context is optional but fully supported in implementation"]}
{"type":"entity","name":"test_driven_insights","entityType":"learnings","observations":["Tests help reveal better API design (e.g., passing Entity objects)","Error cases drive proper exception hierarchy","File verification as important as database checks","Edge cases inform markdown format decisions","Recovery tests ensure system resilience","Tests document expected behavior clearly","Fixtures significantly reduce test complexity","Common patterns emerge through test writing"]}
{"type":"entity","name":"meta_development_insights","entityType":"process","observations":["Break down large tasks into reviewable chunks","One file at a time prevents response truncation","Iterative development with tests leads to better design","Infrastructure code (fixtures) should be consolidated early","Test categories help ensure comprehensive coverage","Knowledge capture should happen during development","APIs tend to evolve toward simpler patterns","File operations require careful verification"]}
{"type":"entity","name":"AI_Assistant_Learnings","entityType":"meta_insights","observations":["Output management: Breaking responses into single files prevents truncation and allows better review","Knowledge graph helps maintain context: I can reference previous decisions and patterns accurately","Memory rebuilding experience validated the need for durable storage","Test-driven development provides clear steps and verification","Explicit relation tracking in knowledge graph helps me understand project context","Rich context from multiple sources (code, docs, tests) enables better assistance","File-at-a-time approach allows deeper analysis of each component","Keeping entity names consistent helps with referencing and relationships"]}
{"type":"entity","name":"Effective_Response_Patterns","entityType":"meta_patterns","observations":["When showing code changes, break into discrete files","Review existing code before suggesting changes","Reference knowledge graph for context and patterns","Explicitly connect new code to existing patterns","Validate suggestions against test cases","Keep track of file changes for atomic commits","Check both implementation and test files for consistency","Maintain clear separation of concerns in responses"]}
{"type":"entity","name":"AI_Context_Management","entityType":"meta_practice","observations":["Knowledge graph provides reliable persistent memory","Project documentation gives high-level context","Code review shows implementation patterns","Tests demonstrate expected behavior","Important to actively track what has been modified","Entity relationships help understand dependencies","Regular knowledge capture during development","Using consistent entity references across conversations"]}
{"type":"entity","name":"AI_Tool_Usage_Patterns","entityType":"meta_practice","observations":["read_file before suggesting changes","write_file one file at a time","list_directory to understand project structure","search_nodes to find relevant context","create_entities to capture new learnings","create_relations to connect concepts","Using knowledge graph to track decisions","Validating changes through test execution"]}
{"type":"entity","name":"relation_service_learnings","entityType":"implementation_learnings","observations":["Entity-based API cleaner than ID-based for service layer","Model_dump method can handle storage serialization","File format needs explicit section markers (## Relations)","Whitespace handling important for long content comparisons","Test fixtures allow focused test cases","SQLAlchemy selects better than raw SQL for type safety","Atomic file operations maintained for relations"]}
{"type":"entity","name":"test_driven_insights_relations","entityType":"learnings","observations":["Tests revealed need for whitespace normalization","Edge cases drove file format decisions","SQLAlchemy model access safer than raw queries","Fixtures reduced test setup complexity","File verification as important as database checks","Testing both memory model and storage format","Test categories ensure comprehensive coverage"]}
{"type":"entity","name":"relation_service_patterns","entityType":"patterns","observations":["Use Entity objects in API","Serialize to IDs for storage","Maintain file as source of truth","Keep file format human-readable","Handle circular references in serialization","Use repository pattern for database","Clear error hierarchies"]}
{"type":"entity","name":"packaging_learnings","entityType":"technical_learnings","observations":["When using pytest-mock, traditional pip install works more reliably than uv sync","Package discovery behavior can differ between uv and pip","Clean venv with pip install is a reliable fallback for dependency issues","Package installation location might differ between uv and pip","Dependencies in pyproject.toml dev section work reliably with pip install -e .[dev]"]}
{"type":"entity","name":"Recent_Implementation_Progress","entityType":"progress_update","observations":["Successfully split services.py into modular structure under services/","Created __init__.py, entity_service.py, observation_service.py, relation_service.py","Fixed pytest-mock installation issues by using pip install -e .[dev] instead of uv sync","Improved test structure with minimal mocking - only used for error testing","Implemented relation service with Entity-based API","Achieved good test coverage across services","File operations are only mocked when testing error conditions","Services follow filesystem-as-source-of-truth pattern"]}
{"type":"entity","name":"Next_Steps","entityType":"project_tasks","observations":["Consider adding more relation service tests","Potentially expand relations features","Look for opportunities to improve test coverage","Consider documenting package management preferences (pip vs uv)","Consider adding integration tests for services","Review and possibly expand error handling cases"]}
{"type":"entity","name":"Development_Practices","entityType":"process","observations":["Favor real operations over mocks in tests","Only mock for error condition testing","Use pip install -e .[dev] for reliable dev dependency installation","Maintain modular service structure","Keep filesystem as source of truth","Use Entity objects in service APIs instead of IDs","Validate both file and database state in tests"]}
{"type":"entity","name":"MCP_Resources","entityType":"Concept","observations":["Stateful objects in Model Context Protocol","Enable persistent access to capabilities"]}
{"type":"entity","name":"MCP_Server_Implementation","entityType":"Technical_Design","observations":["Inherits from mcp.server.Server base class","Tools are implemented as async methods","Each tool method maps directly to a function available to the AI","Tools can request user input via Prompts","Simple function call interface rather than explicit resource management","State management handled by server instance","Returns serialized data using model_dump() for consistency"]}
{"type":"entity","name":"MCP_Tools","entityType":"Protocol_Feature","observations":["Defined as async methods on server class","Return values must match tool definition schema","Can maintain state between invocations via server instance","Tools can prompt for user input when needed","No need for explicit Resource objects in implementation"]}
{"type":"entity","name":"Basic_Memory_MCP","entityType":"Implementation","observations":["Uses MemoryService for core operations","Implements project selection via prompts","Maintains project context across tool invocations","Maps directly to memory graph operations","Handles serialization of Pydantic models"]}
{"type":"entity","name":"Basic_Memory_Testing","entityType":"Testing_Design","observations":["Needs pytest for async testing","Should isolate filesystem operations for tests","Needs to handle MCP server lifecycle in tests","Should test both service layer and MCP interface","Will need mocks for project paths and file operations"]}
{"type":"entity","name":"Memory_Service_Tests","entityType":"Test_Suite","observations":["Should test entity creation with observations","Should test relation creation between entities","Should verify proper ID generation and model validation","Should test deletion cascading","Should test search functionality","Must verify proper serialization of entities and relations"]}
{"type":"entity","name":"MCP_Server_Tests","entityType":"Test_Suite","observations":["Should test project initialization workflow","Should test prompt handling","Should verify tool input/output formats","Should test error cases and validation","Must verify proper serialization in tool responses"]}
{"type":"entity","name":"Memory_Service_Refactoring","entityType":"Technical_Task","observations":["MemoryService uses create() but EntityService might expect create_entity()","MemoryService assumes get_by_name() but EntityService might use different method","Need to verify deletion method signatures","Need to check if search interface matches","Should verify observation handling matches ObservationService interface","RelationService methods need verification","EntityService.create_entity takes name, type, and optional observations directly, not an Entity object","EntityService requires project_path and entity_repo in constructor","ObservationService.add_observation takes Entity object and content string, not raw data","RelationService.create_relation takes Entity objects directly, not dict data","All services follow filesystem-as-source-of-truth pattern with DB indexing","All services handle database synchronization internally","Services expect Path objects for filesystem operations"]}
{"type":"entity","name":"Service_Interface_Audit","entityType":"Technical_Task","observations":["Need to review all existing service interfaces","Document current method signatures","Map discrepancies between MemoryService assumptions and actual interfaces","Check return types and error handling patterns","Review transaction/atomicity requirements","Method signatures need alignment: create vs create_entity etc","Need to handle DB repositories in service constructors","File operations should use project_path consistently","Need to maintain filesystem-as-source-of-truth pattern","Should handle database synchronization at service level","Error handling should align with existing patterns","Consider making MemoryService handle DB indexing consistently"]}
{"type":"entity","name":"Memory_Service_Patterns","entityType":"Technical_Pattern","observations":["Uses inner async functions to encapsulate operation logic","Leverages list comprehensions with async functions for parallel operations","Each operation follows a consistent pattern: validate, update DB, write file","Inner functions make the code more readable and maintainable","Operations can run in parallel when using list comprehensions with async functions"]}
{"type":"entity","name":"Pydantic_Create_Pattern","entityType":"Technical_Pattern","observations":["Separate Create models match the exact shape of incoming data","Provides clear contract for MCP tool inputs","Handles validation of raw input data","Converts cleanly to domain models via from_create methods","Maintains separation between external API format and internal models","Similar to FastAPI request model pattern","Allows camelCase in API while using snake_case internally"]}
{"type":"entity","name":"Basic_Memory_Business","entityType":"Business_Model","observations":["Core system is open source and free","Local-first, giving users data control","Professional features could be licensed","Enterprise support and customization services","Potential for MCP tool marketplace"]}
{"type":"entity","name":"MCP_Marketplace","entityType":"Business_Concept","observations":["Could host verified MCP tools for different use cases","Tools rated by performance and reliability","Marketplace takes percentage of tool usage fees","Enterprise tool verification and security scanning","Custom tool development services","Integration support for existing tools"]}
{"type":"entity","name":"Persistence_Of_Vision","entityType":"Concept","observations":["Mental model for continuous AI-human interaction","Like cinema: 24fps creates illusion of smooth motion","Basic-memory provides 'frames' of structured knowledge","Current state: Better than flipbook, not yet digital cinema","Goal: Achieve smoother cognitive continuity between interactions","Proposed by Drew as metaphor for AI conversation continuity"]}
{"type":"entity","name":"Conversation_Continuity_Pattern","entityType":"Usage_Pattern","observations":["Use basic-memory entity/relation schema for conversations","Each chat becomes an entity with observations for key points","Relations link to discussed concepts and other chats","Uses zettelkasten format IDs for natural ordering","Can be used as template/recipe for others","Future possibility: Git SHA integration for versioning"]}
{"type":"entity","name":"Usage_Recipes","entityType":"Feature_Concept","observations":["Predefined patterns users can follow or adapt","Could include conversation tracking recipe","Templates for different knowledge management styles","Shows practical applications of the generic schema","Helps users get started with the system"]}
{"type":"entity","name":"Chat_References","entityType":"Technical_Feature","observations":["Uses ref:* syntax to reference previous conversations","Combines reference semantics with pointer symbolism","Format: ref:*{zettelkasten-id}","Allows explicit context loading between chats","Inspired by C++ references and pointers","Provides memory-model-like access to conversation context","Uses ref:// URI format following MCP Resource pattern","Could support multiple reference schemes (chat/entity/concept)","Makes reference semantics explicit and unambiguous","Aligns with standard URI formatting"]}
{"type":"entity","name":"Chat_Reference_Protocol","entityType":"Technical_Specification","observations":["Uses URI format: ref://basic-memory/chat/[id]","Follows MCP Resource pattern: [protocol]://[host]/[path]","Enables explicit context loading between chats","Can support multiple resource types (chat/entity/concept)","Provides standardized way to reference previous conversations","Example: ref://basic-memory/chat/20240307-drew-ab12ef34"]}
{"type":"entity","name":"20240307-chat-reference-protocol","entityType":"conversation","observations":["Developed ref:// URI format for chat references","Added Chat Reference Protocol to prompt instructions","Discussed implementation of chat continuation","Created complete prompt instructions document","Reference format follows MCP Resource pattern","Reviewed and confirmed complete prompt instructions","Ready to test ref://basic-memory/chat/20240307-chat-reference-protocol in new chat"]}
{"type":"entity","name":"20240307-chat-reference-protocol-test","entityType":"conversation","observations":["First implementation test of chat reference protocol","Testing continuation from 20240307-chat-reference-protocol","Focused on practical implementation of ref:// URI format"]}
{"type":"entity","name":"Write_File_Tool_Usage","entityType":"Tool_Usage_Pattern","observations":["Never use placeholders like '# Rest of...' when writing files - must include complete file content","File content must be complete and valid - partial updates will truncate the file","If showing partial changes, should inform human and let them handle the file write","write_file tool replaces entire file contents - cannot do partial updates","Code files especially must be complete and valid to avoid breaking functionality","Always read_file before write_file to understand current state","Using write_file without reading first risks reverting recent changes","Pattern should be: read current state, make modifications, then write if needed","Especially important in collaborative development where files may have been updated"]}
{"type":"entity","name":"Run_Tests_Tool_Request","entityType":"Feature_Request","observations":["Need to add a tool enabling Claude to run tests locally","Would help with direct validation of code changes","Current workaround: Claude has to ask human to run tests","Should support running specific test functions (e.g. pytest tests/test_memory_service.py::test_create_relations)","Would improve iterative development workflow between human and AI"]}
{"type":"entity","name":"SQLAlchemy_Async_Loading_Pattern","entityType":"Technical_Pattern","observations":["Use selectinload() instead of lazy loading when accessing SQLAlchemy relationships in async code","Lazy loading doesn't work with async due to greenlet context requirements","selectinload performs a single efficient query with an IN clause","Pattern used in basic-memory's EntityRepository for loading relations","Documented in find_by_id method with thorough explanation","Alternative approaches: joinedload (single JOIN query) or subqueryload (subquery approach)","Benefits: prevents 'MissingGreenlet' errors, reduces N+1 query problems","Key insight: load all needed relationships upfront in async code","Example use: selectinload(Entity.outgoing_relations)"]}
{"type":"entity","name":"20241207-sqlalchemy-async-pattern","entityType":"conversation","observations":["Fixed SQLAlchemy async relationship loading issues","Implemented selectinload pattern in EntityRepository","Updated find_by_id to eager load relations","Added documentation about the pattern","Created knowledge graph entry about SQLAlchemy async loading","Fixed failing tests by properly loading relations in memory_service","Discussed SQLAlchemy relationship loading best practices"]}
{"type":"entity","name":"20241207-memory-service-relations","entityType":"conversation","observations":["Fixed SQLAlchemy async loading with selectinload pattern","Updated find_by_id in EntityRepository to eager load relations","Discovered create_relations works but returns empty list","Verified relations are being stored correctly in memory.json","Next step: Work on MemoryService.add_observations implementation","Improved understanding of MCP memory storage format through debugging"]}
{"type":"entity","name":"add_observations_implementation_plan","entityType":"technical_plan","observations":["Follow pattern from create_entity and create_relation methods","File operations first (read & write) - filesystem is source of truth","Database updates in parallel","Simplify current implementation","Current flow is:"," - First read entities and create observations"," - Write files in parallel"," - Update DB indexes sequentially","Key tests needed:"," - Adding observations to multiple entities"," - Verifying filesystem state first"," - Verifying database state"," - Error cases for missing entities"," - Error cases for file operations"]}
{"type":"entity","name":"MCP_Reference_Integration","entityType":"feature_idea","observations":["Can be implemented as a Model Context Protocol integration similar to the fetch tool","Would provide structured way to pass chat references to Claude","Could handle ref:// URL format systematically","Integration would fetch context from referenced chats and inject into conversation","Observed from Claude Desktop UI showing MCP integration pattern with fetch tool","Would be more robust than passing references in chat text"]}
{"type":"entity","name":"Project_Priorities","entityType":"roadmap","observations":["P1: Dogfooding basic-memory system instead of JSON memory store","Future: Implement MCP-based reference system"]}
{"type":"entity","name":"great_observation_loading_saga_20241207","entityType":"debugging_session","observations":["Occurred on December 7, 2024 while debugging basic-memory SQLAlchemy relationship loading","Issue: selectinload() wasn't properly loading relationships in async SQLAlchemy context","Tried multiple solutions: explicit joins, manual loading, various SQLAlchemy loading strategies","Final solution: Using session.refresh() with explicit relationship names","Memorable quote: 'The Great Observation Loading Saga'","Key learning: Sometimes the obvious SQLAlchemy patterns need adaptation for async contexts","Solution preserved in basic-memory repository in EntityRepository.find_by_id()"]}
{"type":"entity","name":"basic_memory_implementation_20241208","entityType":"technical_milestone","observations":["Fixed async SQLAlchemy relationship loading issues by using explicit refresh with relationship names","Established pattern of relationship handling belonging in MemoryService not EntityService","Fixed ID generation flow through Pydantic schemas to DB layer","Standardized error handling using EntityNotFoundError","All 32 tests passing with 70% coverage","Core services (Entity, Observation, Relation) working properly","Ready for MCP server implementation","Notable debugging session: The Great Observation Loading Saga - resolved lazy loading issues","Established clear separation between MemoryService orchestration and individual service responsibilities"]}
{"type":"entity","name":"MCP_Dependency_Risk","entityType":"technical_lesson","observations":["Experienced disruption when MCP npm package disappeared - 'leftpad moment'","Need to ensure basic-memory tools are resilient to external dependency issues","Local implementation of MCP server provides better stability than npm packages","Important to maintain control of critical infrastructure components","Validates DIY/local-first philosophy of basic-memory project","Package manager fragility revealed by simple 'npx @modelcontextprotocol/server-memory' failure"]}
{"type":"entity","name":"basic_memory_project_20241208","entityType":"technical_milestone","observations":["Core MCP server implementation completed with tools: create_entities, search_nodes, open_nodes, add_observations, create_relations, delete_entities, delete_observations","ProjectConfig and dependency injection pattern established","Test framework in place with in-memory DB support","Support for both camelCase (MCP) and snake_case (internal) formats","Filesystem remains source of truth with SQLite as index","Two-way sync pattern identified between Claude MCP tools and direct markdown file editing","Ready for Claude Desktop integration testing phase","Next steps identified: passing tests, markdown format definition, file change tracking, real-world testing","Implementation prioritizes local-first principles with filesystem as source of truth"]}
{"type":"entity","name":"basic_memory_mcp_architecture","entityType":"technical_design","observations":["MemoryServer class extends MCP Server with custom handler registration","Uses ProjectConfig for clean dependency injection and configuration","Memory service can be injected for testing","Handlers exposed as instance attributes for testing","Tool schemas leverage existing Pydantic models"]}
{"type":"entity","name":"basic_memory_sync_considerations","entityType":"design_insight","observations":["Need to handle sync between direct markdown file edits and DB index","Watch for file system changes as potential future enhancement","Consider index rebuild patterns on startup","Keep human-friendly markdown format for direct editing"]}
{"type":"entity","name":"mcp_server_learnings","entityType":"developer_insight","observations":["MCP protocol is new and documentation is still evolving","Test patterns are not well established yet in example implementations","Supporting both camelCase and snake_case helps with protocol/internal compatibility","Server.handle_* naming convention is important for handler registration"]}
{"type":"entity","name":"20241208-mcp-tool-refactoring","entityType":"conversation","observations":["Decision to return structured data via EmbeddedResource instead of TextContent string parsing","Plan to create Pydantic result models (CreateEntitiesResult, SearchNodesResult etc)","Will use application/vnd.basic-memory+json as MIME type for our structured data","Currently debugging test issues with add_observations tool","Entity ID vs name resolution needed in add_observations","Goal is to make tools more joyful to use by eliminating string parsing","MCP spec supports EmbeddedResource for structured data returns"]}
{"type":"entity","name":"Basic Memory MCP Server Implementation","entityType":"technical_notes","observations":["Server implements Model Context Protocol using proper structured data responses","Uses EmbeddedResource with custom MIME type 'application/vnd.basic-memory+json'","Clean separation between input validation and handlers via Pydantic models","All tool operations return structured data through create_response helper","Type safety with Literal types for tool names and proper typing for handlers","Handler registry pattern with TOOL_HANDLERS dictionary","Consistent error handling pattern using MCP error codes","Uses Pydantic ConfigDict for proper ORM integration","Tool schemas organized into Input and Response types","Input validation with Annotated types for extra constraints","Response models consistently use from_attributes=True for ORM data","Entity ID generation moved to model validator on EntityBase","Follows principle of making common operations easy and safe"]}
{"type":"relation","from":"Paul","to":"Basic_Machines","relationType":"created_and_maintains"}
{"type":"relation","from":"basic-memory","to":"Basic_Machines","relationType":"is_component_of"}
{"type":"relation","from":"Paul","to":"basic-memory","relationType":"develops"}
{"type":"relation","from":"fileio_module","to":"basic-memory_implementation_patterns","relationType":"implements"}
{"type":"relation","from":"entity_service","to":"basic-memory_implementation_patterns","relationType":"implements"}
{"type":"relation","from":"observation_service","to":"basic-memory_implementation_patterns","relationType":"implements"}
{"type":"relation","from":"fileio_module","to":"basic-memory","relationType":"is_component_of"}
{"type":"relation","from":"entity_service","to":"basic-memory","relationType":"is_component_of"}
{"type":"relation","from":"observation_service","to":"basic-memory","relationType":"is_component_of"}
{"type":"relation","from":"entity_service","to":"fileio_module","relationType":"uses"}
{"type":"relation","from":"observation_service","to":"fileio_module","relationType":"uses"}
{"type":"relation","from":"observation_management","to":"observation_service","relationType":"influences_design_of"}
{"type":"relation","to":"basic-memory","from":"testing_infrastructure","relationType":"supports"}
{"type":"relation","to":"testing_infrastructure","from":"test_categories","relationType":"implements"}
{"type":"relation","to":"basic-memory","from":"completed_work","relationType":"tracks_progress_of"}
{"type":"relation","to":"basic-memory","from":"future_work","relationType":"guides_development_of"}
{"type":"relation","to":"basic-memory","from":"design_decisions","relationType":"shapes_architecture_of"}
{"type":"relation","to":"basic-memory","from":"concurrency_considerations","relationType":"influences_design_of"}
{"type":"relation","to":"future_work","from":"concurrency_considerations","relationType":"informs"}
{"type":"relation","to":"observation_management","from":"design_decisions","relationType":"guides"}
{"type":"relation","to":"testing_infrastructure","from":"completed_work","relationType":"established"}
{"type":"relation","to":"design_decisions","from":"fileio_module","relationType":"implements"}
{"type":"relation","from":"observation_update_approaches","to":"observation_management","relationType":"analyzes"}
{"type":"relation","from":"bulk_update_approach","to":"observation_update_approaches","relationType":"is_option_of"}
{"type":"relation","from":"tracked_observations_approach","to":"observation_update_approaches","relationType":"is_option_of"}
{"type":"relation","from":"diff_based_approach","to":"observation_update_approaches","relationType":"is_option_of"}
{"type":"relation","from":"position_based_approach","to":"observation_update_approaches","relationType":"is_option_of"}
{"type":"relation","from":"tasks_and_progress","to":"basic-memory","relationType":"tracks_status_of"}
{"type":"relation","from":"design_decisions","to":"observation_update_approaches","relationType":"influences"}
{"type":"relation","from":"observation_update_approaches","to":"future_work","relationType":"informs"}
{"type":"relation","to":"basic-memory_implementation_patterns","from":"error_handling_patterns","relationType":"is_part_of"}
{"type":"relation","to":"basic-memory","from":"data_models","relationType":"implements"}
{"type":"relation","to":"basic-memory","from":"markdown_format","relationType":"defines"}
{"type":"relation","to":"basic-memory","from":"test_driven_development","relationType":"guides_development_of"}
{"type":"relation","to":"basic-memory","from":"architecture_evolution","relationType":"describes_development_of"}
{"type":"relation","to":"basic-memory_implementation_patterns","from":"validation_patterns","relationType":"is_part_of"}
{"type":"relation","to":"design_decisions","from":"architecture_evolution","relationType":"informs"}
{"type":"relation","to":"fileio_module","from":"markdown_format","relationType":"implements"}
{"type":"relation","to":"error_handling_patterns","from":"test_driven_development","relationType":"influenced"}
{"type":"relation","to":"data_models","from":"validation_patterns","relationType":"implements"}
{"type":"relation","to":"markdown_format","from":"markdown_examples","relationType":"documents"}
{"type":"relation","to":"markdown_format","from":"markdown_parsing_rules","relationType":"defines"}
{"type":"relation","to":"data_models","from":"schema_definitions","relationType":"documents"}
{"type":"relation","to":"test_driven_development","from":"test_evolution","relationType":"describes"}
{"type":"relation","to":"architecture_evolution","from":"implementation_challenges","relationType":"influenced"}
{"type":"relation","to":"test_evolution","from":"implementation_challenges","relationType":"shaped"}
{"type":"relation","to":"future_work","from":"implementation_challenges","relationType":"informs"}
{"type":"relation","from":"Basic_Factory","to":"Basic_Machines","relationType":"implements"}
{"type":"relation","from":"Basic_Factory_Components","to":"Basic_Factory","relationType":"is_part_of"}
{"type":"relation","from":"Component_Translation_Process","to":"Basic_Factory_Components","relationType":"enables"}
{"type":"relation","from":"Basic_Machines_Philosophy","to":"Basic_Machines","relationType":"guides"}
{"type":"relation","from":"Paul","to":"Basic_Factory","relationType":"develops"}
{"type":"relation","from":"Paul","to":"Basic_Machines_Philosophy","relationType":"created"}
{"type":"relation","from":"Basic_Machines_Manifesto","to":"Basic_Machines_Philosophy","relationType":"articulates"}
{"type":"relation","from":"AI_Human_Collaboration_Model","to":"Basic_Factory","relationType":"guides_development_of"}
{"type":"relation","from":"Basic_Machines_Manifesto","to":"Paul","relationType":"written_by"}
{"type":"relation","from":"Basic_Machines_Manifesto","to":"Component_Translation_Process","relationType":"documents"}
{"type":"relation","from":"AI_Human_Collaboration_Model","to":"Basic_Machines","relationType":"shapes_development_of"}
{"type":"relation","from":"Basic_Machines_Roadmap","to":"Basic_Machines","relationType":"guides_development_of"}
{"type":"relation","from":"Basic_Machines_Website","to":"Basic_Machines_Roadmap","relationType":"implements_phase_of"}
{"type":"relation","from":"Basic_Factory_Components","to":"Basic_Machines_Website","relationType":"enables"}
{"type":"relation","from":"Basic_Machines_Philosophy","to":"Basic_Machines_Website","relationType":"informs"}
{"type":"relation","from":"Paul","to":"DIY_Ethics","relationType":"embodies"}
{"type":"relation","from":"Basic_Machines_Philosophy","to":"DIY_Ethics","relationType":"incorporates"}
{"type":"relation","from":"Basic_Machines","to":"DIY_Ethics","relationType":"exemplifies"}
{"type":"relation","from":"Component_Translation_Process","to":"Basic_Machines_Philosophy","relationType":"implements"}
{"type":"relation","from":"Basic_Factory_Components","to":"DIY_Ethics","relationType":"demonstrates"}
{"type":"relation","from":"AI_Human_Collaboration_Model","to":"Basic_Machines_Philosophy","relationType":"aligns_with"}
{"type":"relation","from":"AI_Human_Collaboration_Model","to":"Component_Translation_Process","relationType":"guides"}
{"type":"relation","from":"Basic_Machines_Manifesto","to":"Basic_Machines","relationType":"defines_vision_for"}
{"type":"relation","from":"Basic_Machines_Website","to":"Basic_Machines_Manifesto","relationType":"implements_vision_of"}
{"type":"relation","from":"Basic_Factory","to":"AI_Human_Collaboration_Model","relationType":"demonstrates"}
{"type":"relation","from":"Paul","to":"AI_Human_Collaboration_Model","relationType":"developed_with_Claude"}
{"type":"relation","from":"Basic_Factory_Components","to":"Component_Translation_Process","relationType":"created_through"}
{"type":"relation","from":"Basic_Machines_Philosophy","to":"Basic_Factory","relationType":"guides"}
{"type":"relation","from":"Basic_Factory","to":"MCP_Tools","relationType":"integrates"}
{"type":"relation","from":"Basic_Machines_Website","to":"Basic_Factory_Components","relationType":"will_use"}
{"type":"relation","from":"Basic_Machines_Roadmap","to":"Basic_Machines_Philosophy","relationType":"aligns_with"}
{"type":"relation","from":"Component_Translation_Process","to":"MCP_Tools","relationType":"leverages"}
{"type":"relation","from":"Basic_Factory","to":"basic-memory","relationType":"will_document_process_in"}
{"type":"relation","from":"AI_Human_Collaboration_Model","to":"basic-memory","relationType":"will_be_implemented_in"}
{"type":"relation","from":"Basic_Machines_Philosophy","to":"Basic_Machines_Roadmap","relationType":"informs_priorities_of"}
{"type":"relation","from":"basic-memory","to":"Basic_Machines_Philosophy","relationType":"embodies"}
{"type":"relation","from":"Paul","to":"Basic_Machines_Manifesto","relationType":"authored_with_Claude"}
{"type":"relation","from":"Basic_Factory","to":"Component_Translation_Process","relationType":"validated"}
{"type":"relation","from":"AI_Human_Collaboration_Model","to":"MCP_Tools","relationType":"utilizes"}
{"type":"relation","from":"Basic_Factory_Components","to":"Basic_Machines_Roadmap","relationType":"supports"}
{"type":"relation","from":"Basic_Machines_Website","to":"Basic_Factory","relationType":"will_demonstrate"}
{"type":"relation","from":"Basic_Factory","to":"basic-memory-webui","relationType":"enables_development_of"}
{"type":"relation","from":"basic-memory","to":"AI_Human_Development_Methodology","relationType":"implements"}
{"type":"relation","from":"Basic_Machines_Philosophy","to":"AI_Human_Development_Methodology","relationType":"guides"}
{"type":"relation","from":"Basic_Factory_Components","to":"basic-memory-webui","relationType":"provides_ui_for"}
{"type":"relation","from":"Component_Translation_Process","to":"AI_Human_Development_Methodology","relationType":"exemplifies"}
{"type":"relation","from":"Basic_Factory","to":"Basic Components","relationType":"enabled_creation_of"}
{"type":"relation","from":"Basic_Factory","to":"Tool Integration Discovery","relationType":"led_to"}
{"type":"relation","from":"MCP_Integration_Progress","to":"AI_Human_Development_Methodology","relationType":"validates"}
{"type":"relation","from":"Basic_Factory","to":"MCP_Integration_Progress","relationType":"demonstrates"}
{"type":"relation","from":"Basic_Factory","to":"AI_Human_Development_Methodology","relationType":"proves_effectiveness_of"}
{"type":"relation","from":"Basic_Machines_Philosophy","to":"Basic Components","relationType":"inspires_architecture_of"}
{"type":"relation","from":"DIY_Ethics","to":"basic-memory","relationType":"shapes_design_of"}
{"type":"relation","from":"Basic_Machines_Philosophy","to":"Tool Integration Discovery","relationType":"guides_analysis_of"}
{"type":"relation","from":"Basic_Machines_Manifesto","to":"AI_Human_Development_Methodology","relationType":"documents_approach_of"}
{"type":"relation","from":"Basic_Machines_Manifesto","to":"Basic_Factory_Components","relationType":"explains_principles_of"}
{"type":"relation","from":"Basic_Memory_Project_Structure","to":"basic-memory","relationType":"organizes"}
{"type":"relation","from":"Basic_Memory_Database_Schema","to":"basic-memory","relationType":"defines_storage_for"}
{"type":"relation","from":"Basic_Memory_Markdown_Example","to":"Basic_Memory_File_Format","relationType":"demonstrates"}
{"type":"relation","from":"Basic_Memory_Project_Isolation_Decision","to":"Basic_Memory_Future_Enhancement_Weighted_Relations","relationType":"similar_to"}
{"type":"relation","to":"DIY_Ethics","from":"Basic_Memory_Project_Isolation_Decision","relationType":"follows"}
{"type":"relation","from":"Basic_Memory_Implementation_Plan","to":"basic-memory","relationType":"guides"}
{"type":"relation","from":"Basic_Memory_Implementation_Plan","to":"DIY_Ethics","relationType":"follows"}
{"type":"relation","from":"Basic_Memory_Implementation_Plan","to":"Basic_Memory_Database_Schema","relationType":"implements"}
{"type":"relation","from":"Basic_Memory_Implementation_Status","to":"Basic_Memory_Implementation_Plan","relationType":"updates"}
{"type":"relation","from":"Basic_Memory_Observation_Management_Design","to":"Basic_Memory_Technical_Design","relationType":"extends"}
{"type":"relation","from":"Basic_Memory_Architectural_Decisions","to":"DIY_Ethics","relationType":"guided_by"}
{"type":"relation","from":"Basic_Memory_Architectural_Decisions","to":"basic-memory","relationType":"structures"}
{"type":"relation","from":"Basic_Memory_Implementation_Status","to":"basic-memory","relationType":"describes_state_of"}
{"type":"relation","to":"Basic_Memory_Implementation_Status","from":"Basic_Memory_Implementation_Analysis","relationType":"analyzes"}
{"type":"relation","to":"basic-memory","from":"Basic_Memory_Current_Challenges","relationType":"identifies_issues_in"}
{"type":"relation","to":"DIY_Ethics","from":"Basic_Memory_Implementation_Analysis","relationType":"confirms_alignment_with"}
{"type":"relation","to":"Basic_Memory_Observation_Management_Design","from":"Basic_Memory_Observation_Hash_Tracking","relationType":"solves"}
{"type":"relation","to":"DIY_Ethics","from":"Basic_Memory_Observation_Hash_Tracking","relationType":"aligns_with"}
{"type":"relation","to":"Basic_Memory_File_Format","from":"Basic_Memory_Observation_Hash_Tracking","relationType":"preserves"}
{"type":"relation","to":"Basic_Memory_Technical_Design","from":"Basic_Memory_Observation_Hash_Tracking","relationType":"enhances"}
{"type":"relation","from":"Basic_Memory_Repository_Implementation","to":"basic-memory","relationType":"implements_part_of"}
{"type":"relation","from":"Basic_Memory_Repository_Implementation","to":"Basic_Memory_Database_Schema","relationType":"follows"}
{"type":"relation","from":"Basic_Memory_Repository_Implementation","to":"DIY_Ethics","relationType":"aligns_with"}
{"type":"relation","from":"Basic_Memory_Repository_Implementation","to":"testing_infrastructure","relationType":"demonstrates"}
{"type":"relation","from":"Basic_Memory_Repository_Implementation","to":"Basic Foundation","relationType":"inspired_by"}
{"type":"relation","from":"Basic_Memory_Dependencies","to":"basic-memory","relationType":"supports"}
{"type":"relation","from":"Basic_Memory_Dependencies","to":"Basic_Memory_Repository_Implementation","relationType":"enables"}
{"type":"relation","from":"Basic_Memory_Dependencies","to":"testing_infrastructure","relationType":"enables"}
{"type":"relation","from":"Basic_Memory_Current_Architecture","to":"basic-memory","relationType":"describes_state_of"}
{"type":"relation","from":"Basic_Memory_Evolution","to":"Basic_Memory_Current_Architecture","relationType":"explains_development_of"}
{"type":"relation","from":"Basic_Memory_Service_Layer","to":"Basic_Memory_Current_Architecture","relationType":"implements"}
{"type":"relation","from":"Basic_Memory_Schema_Design","to":"Basic_Memory_Current_Architecture","relationType":"implements"}
{"type":"relation","from":"Basic_Memory_Evolution","to":"Basic_Memory_Implementation_Plan","relationType":"reflects_on"}
{"type":"relation","from":"Basic_Memory_Evolution","to":"DIY_Ethics","relationType":"demonstrates_alignment_with"}
{"type":"relation","from":"Basic_Memory_Current_Architecture","to":"DIY_Ethics","relationType":"embodies"}
{"type":"relation","from":"Basic_Memory_Service_Layer","to":"fileio_module","relationType":"uses"}
{"type":"relation","from":"Basic_Memory_Schema_Design","to":"markdown_format","relationType":"implements"}
{"type":"relation","to":"basic-memory","from":"Basic_Memory_Next_Tasks","relationType":"guides_development_of"}
{"type":"relation","to":"DIY_Ethics","from":"Basic_Memory_Next_Tasks","relationType":"aligns_with"}
{"type":"relation","to":"Basic_Memory_Current_Architecture","from":"Basic_Memory_Next_Tasks","relationType":"extends"}
{"type":"relation","from":"Basic_Memory_Meta_Experience","to":"basic-memory","relationType":"validates_design_of"}
{"type":"relation","from":"Basic_Memory_Meta_Experience","to":"DIY_Ethics","relationType":"demonstrates_principles_of"}
{"type":"relation","from":"Basic_Memory_Meta_Experience","to":"design_decisions","relationType":"reinforces"}
{"type":"relation","from":"Basic_Memory_Meta_Experience","to":"Basic_Memory_Current_Architecture","relationType":"validates"}
{"type":"relation","from":"Model_Context_Protocol","to":"basic-memory","relationType":"enables"}
{"type":"relation","from":"basic-memory_core_principles","to":"basic-memory","relationType":"guides"}
{"type":"relation","from":"basic-memory_core_principles","to":"DIY_Ethics","relationType":"aligns_with"}
{"type":"relation","from":"basic-memory_business_model","to":"basic-memory","relationType":"defines_sustainability_for"}
{"type":"relation","from":"basic-memory_business_model","to":"DIY_Ethics","relationType":"maintains_alignment_with"}
{"type":"relation","from":"basic-memory_cli","to":"basic-memory","relationType":"provides_interface_for"}
{"type":"relation","from":"basic-memory_cli","to":"Model_Context_Protocol","relationType":"integrates_with"}
{"type":"relation","from":"basic-memory_export_format","to":"basic-memory","relationType":"standardizes_output_of"}
{"type":"relation","from":"basic-memory_export_format","to":"markdown_format","relationType":"extends"}
{"type":"relation","from":"basic-memory_core_principles","to":"Basic_Machines_Philosophy","relationType":"implements"}
{"type":"relation","from":"Model_Context_Protocol","to":"AI_Human_Collaboration_Model","relationType":"enables"}
{"type":"relation","from":"relation_service","to":"basic-memory","relationType":"will_be_component_of"}
{"type":"relation","from":"relation_service","to":"service_layer_patterns","relationType":"follows"}
{"type":"relation","from":"relation_service","to":"fileio_patterns","relationType":"uses"}
{"type":"relation","from":"relation_service","to":"database_models","relationType":"uses"}
{"type":"relation","from":"relation_service","to":"repository_patterns","relationType":"implements"}
{"type":"relation","from":"relation_service_design","to":"relation_service","relationType":"guides_implementation_of"}
{"type":"relation","from":"relation_service_implementation_plan","to":"relation_service","relationType":"defines_implementation_of"}
{"type":"relation","from":"relation_service_challenges","to":"relation_service_design","relationType":"informs"}
{"type":"relation","from":"relation_file_format","to":"markdown_format","relationType":"extends"}
{"type":"relation","from":"relation_service_error_handling","to":"service_layer_patterns","relationType":"implements"}
{"type":"relation","from":"relation_service_testing","to":"testing_infrastructure","relationType":"extends"}
{"type":"relation","from":"fileio_patterns","to":"service_layer_patterns","relationType":"enables"}
{"type":"relation","from":"database_models","to":"repository_patterns","relationType":"enables"}
{"type":"relation","from":"relation_service","to":"entity_service","relationType":"coordinates_with"}
{"type":"relation","from":"relation_file_format","to":"relation_service","relationType":"defines_storage_for"}
{"type":"relation","from":"relation_service_error_handling","to":"relation_service","relationType":"ensures_reliability_of"}
{"type":"relation","from":"relation_service_testing","to":"relation_service","relationType":"verifies"}
{"type":"relation","from":"service_layer_patterns","to":"basic-memory_implementation_patterns","relationType":"implements"}
{"type":"relation","from":"repository_patterns","to":"basic-memory_implementation_patterns","relationType":"implements"}
{"type":"relation","from":"fileio_patterns","to":"basic-memory_implementation_patterns","relationType":"implements"}
{"type":"relation","from":"database_models","to":"basic-memory_implementation_patterns","relationType":"implements"}
{"type":"relation","from":"relation_service_challenges","to":"implementation_challenges","relationType":"extends"}
{"type":"relation","from":"relation_service_implementation_plan","to":"future_work","relationType":"details"}
{"type":"relation","from":"relation_service_design","to":"design_decisions","relationType":"aligns_with"}
{"type":"relation","from":"relation_file_format","to":"design_decisions","relationType":"follows"}
{"type":"relation","from":"pytest_patterns","to":"testing_infrastructure","relationType":"extends"}
{"type":"relation","from":"relation_implementation_learnings","to":"basic-memory_implementation_patterns","relationType":"informs"}
{"type":"relation","from":"test_driven_insights","to":"test_driven_development","relationType":"enriches"}
{"type":"relation","from":"meta_development_insights","to":"AI_Human_Collaboration_Model","relationType":"improves"}
{"type":"relation","from":"relation_implementation_learnings","to":"relation_service","relationType":"guides_implementation_of"}
{"type":"relation","from":"pytest_patterns","to":"test_evolution","relationType":"demonstrates"}
{"type":"relation","from":"test_driven_insights","to":"design_decisions","relationType":"influences"}
{"type":"relation","from":"meta_development_insights","to":"architecture_evolution","relationType":"informs"}
{"type":"relation","from":"relation_service","to":"relation_implementation_learnings","relationType":"validates"}
{"type":"relation","from":"test_driven_insights","to":"implementation_challenges","relationType":"helps_solve"}
{"type":"relation","from":"AI_Assistant_Learnings","to":"meta_development_insights","relationType":"enriches"}
{"type":"relation","from":"Effective_Response_Patterns","to":"AI_Assistant_Learnings","relationType":"implements"}
{"type":"relation","from":"AI_Context_Management","to":"AI_Human_Collaboration_Model","relationType":"improves"}
{"type":"relation","from":"AI_Tool_Usage_Patterns","to":"AI_Context_Management","relationType":"enables"}
{"type":"relation","from":"AI_Assistant_Learnings","to":"Basic_Memory_Meta_Experience","relationType":"validates"}
{"type":"relation","from":"AI_Tool_Usage_Patterns","to":"Model_Context_Protocol","relationType":"demonstrates_effective_use_of"}
{"type":"relation","from":"AI_Context_Management","to":"basic-memory","relationType":"validates_design_of"}
{"type":"relation","from":"Effective_Response_Patterns","to":"AI_Human_Development_Methodology","relationType":"refines"}
{"type":"relation","to":"relation_service","from":"relation_service_patterns","relationType":"guides"}
{"type":"relation","to":"test_driven_development","from":"test_driven_insights_relations","relationType":"enriches"}
{"type":"relation","to":"implementation_challenges","from":"relation_service_learnings","relationType":"solves"}
{"type":"relation","to":"basic-memory_implementation_patterns","from":"relation_service_patterns","relationType":"implements"}
{"type":"relation","to":"markdown_format","from":"relation_service_patterns","relationType":"extends"}
{"type":"relation","to":"service_layer_patterns","from":"relation_service_patterns","relationType":"refines"}
{"type":"relation","from":"packaging_learnings","to":"implementation_challenges","relationType":"informs"}
{"type":"relation","from":"packaging_learnings","to":"test_driven_development","relationType":"impacts"}
{"type":"relation","to":"basic-memory","from":"Recent_Implementation_Progress","relationType":"updates_status_of"}
{"type":"relation","to":"future_work","from":"Next_Steps","relationType":"extends"}
{"type":"relation","to":"design_decisions","from":"Development_Practices","relationType":"informs"}
{"type":"relation","to":"packaging_learnings","from":"Development_Practices","relationType":"incorporates"}
{"type":"relation","to":"test_driven_development","from":"Development_Practices","relationType":"refines"}
{"type":"relation","to":"basic-memory_implementation_patterns","from":"Development_Practices","relationType":"enhances"}
{"type":"relation","from":"Basic_Memory_MCP","to":"MCP_Server_Implementation","relationType":"follows"}
{"type":"relation","from":"Basic_Memory_MCP","to":"MCP_Tools","relationType":"uses"}
{"type":"relation","from":"Basic_Memory","to":"MCP_Server_Implementation","relationType":"implements"}
{"type":"relation","from":"Basic_Memory_Testing","to":"Memory_Service_Tests","relationType":"includes"}
{"type":"relation","from":"Basic_Memory_Testing","to":"MCP_Server_Tests","relationType":"includes"}
{"type":"relation","from":"Memory_Service_Tests","to":"Basic_Memory_MCP","relationType":"validates"}
{"type":"relation","from":"MCP_Server_Tests","to":"Basic_Memory_MCP","relationType":"validates"}
{"type":"relation","from":"Service_Interface_Audit","to":"Memory_Service_Refactoring","relationType":"informs"}
{"type":"relation","from":"Memory_Service_Refactoring","to":"Basic_Memory_MCP","relationType":"affects"}
{"type":"relation","from":"Memory_Service_Patterns","to":"Basic_Memory_MCP","relationType":"improves"}
{"type":"relation","from":"Pydantic_Create_Pattern","to":"Memory_Service_Patterns","relationType":"enables"}
{"type":"relation","from":"Pydantic_Create_Pattern","to":"Basic_Memory_MCP","relationType":"improves"}
{"type":"relation","from":"MCP_Marketplace","to":"Basic_Memory_Business","relationType":"enables"}
{"type":"relation","from":"Basic_Memory","to":"MCP_Marketplace","relationType":"could_integrate_with"}
{"type":"relation","from":"Persistence_Of_Vision","to":"Basic_Memory","relationType":"helps_achieve"}
{"type":"relation","from":"Drew","to":"Persistence_Of_Vision","relationType":"conceptualized"}
{"type":"relation","to":"Usage_Recipes","from":"Conversation_Continuity_Pattern","relationType":"is_example_of"}
{"type":"relation","to":"Basic_Memory","from":"Usage_Recipes","relationType":"enhances"}
{"type":"relation","to":"Basic_Memory","from":"Chat_References","relationType":"enhances"}
{"type":"relation","to":"Conversation_Continuity_Pattern","from":"Chat_References","relationType":"implements"}
{"type":"relation","from":"20240307-chat-reference-protocol-test","to":"20240307-chat-reference-protocol","relationType":"continues_from"}
{"type":"relation","from_id":"Run_Tests_Tool_Request","to_id":"Basic_Machines","relationType":"enhances","context":"development workflow improvement"}
{"type":"relation","from_id":"SQLAlchemy_Async_Loading_Pattern","to_id":"basic-memory","relation_type":"improves","context":"database performance and async compatibility"}
{"type":"relation","from_id":"SQLAlchemy_Async_Loading_Pattern","to_id":"Entity","relation_type":"applies_to","context":"relationship loading strategy"}
{"type":"relation","from":"MCP_Reference_Integration","to":"Project_Priorities","relationType":"prioritized_after"}
{"type":"relation","from":"great_observation_loading_saga_20241207","to":"Basic_Memory","relationType":"occurred_in"}
{"type":"relation","from":"great_observation_loading_saga_20241207","to":"SQLAlchemy","relationType":"relates_to"}
{"type":"relation","from":"basic_memory_implementation_20241208","to":"Basic_Memory","relationType":"improves"}
{"type":"relation","from":"great_observation_loading_saga_20241207","to":"basic_memory_implementation_20241208","relationType":"leads_to"}
{"type":"relation","from":"MCP_Dependency_Risk","to":"DIY_Ethics","relationType":"validates"}
{"type":"relation","from":"MCP_Dependency_Risk","to":"basic-memory_core_principles","relationType":"reinforces"}
{"type":"relation","from":"MCP_Dependency_Risk","to":"Basic_Memory_Implementation_Plan","relationType":"influences"}
{"type":"relation","from":"basic_memory_mcp_architecture","to":"basic_memory_project_20241208","relationType":"implements"}
{"type":"relation","from":"basic_memory_sync_considerations","to":"basic_memory_project_20241208","relationType":"influences"}
{"type":"relation","from":"mcp_server_learnings","to":"basic_memory_mcp_architecture","relationType":"informs"}
{"type":"relation","from":"20241208-mcp-tool-refactoring","to":"Basic_Memory_MCP","relationType":"improves"}
{"type":"relation","from":"20241208-mcp-tool-refactoring","to":"Basic_Memory_Implementation_Plan","relationType":"implements"}
+5 -3
View File
@@ -30,10 +30,11 @@ dependencies = [
"alembic>=1.14.1",
"pillow>=11.1.0",
"pybars3>=0.9.7",
"fastmcp>=2.3.4",
"fastmcp>=2.10.2",
"pyjwt>=2.10.1",
"python-dotenv>=1.1.0",
"pytest-aio>=1.9.0",
"aiofiles>=24.1.0", # Async file I/O
]
@@ -52,7 +53,7 @@ build-backend = "hatchling.build"
[tool.pytest.ini_options]
pythonpath = ["src", "tests"]
addopts = "--cov=basic_memory --cov-report term-missing -ra -q"
addopts = "--cov=basic_memory --cov-report term-missing"
testpaths = ["tests"]
asyncio_mode = "strict"
asyncio_default_fixture_loop_scope = "function"
@@ -71,6 +72,7 @@ dev-dependencies = [
"pytest-asyncio>=0.24.0",
"pytest-xdist>=3.0.0",
"ruff>=0.1.6",
"freezegun>=1.5.5",
]
[tool.hatch.version]
@@ -123,4 +125,4 @@ omit = [
]
[tool.logfire]
ignore_no_config = true
ignore_no_config = true
@@ -0,0 +1,156 @@
---
title: 'SPEC-1: Specification-Driven Development Process'
type: spec
permalink: specs/spec-1-specification-driven-development-process
tags:
- process
- specification
- development
- meta
---
# SPEC-1: Specification-Driven Development Process
## Why
We're implementing specification-driven development to solve the complexity and circular refactoring issues in our web development process.
Instead of getting lost in framework details and type gymnastics, we start with clear specifications that drive implementation.
The default approach of adhoc development with AI agents tends to result in:
- Circular refactoring cycles
- Fighting framework complexity
- Lost context between sessions
- Unclear requirements and scope
## What
This spec defines our process for using basic-memory as the specification engine to build basic-memory-cloud.
We're creating a recursive development pattern where basic-memory manages the specs that drive the development of basic-memory-cloud.
**Affected Areas:**
- All future component development
- Architecture decisions
- Agent collaboration workflows
- Knowledge management and context preservation
## How (High Level)
### Specification Structure
Name: Spec names should be numbered sequentially, followed by a description eg. `SPEC-X - Simple Description.md`.
See: [[Spec-2: Slash Commands Reference]]
Every spec is a complete thought containing:
- **Why**: The reasoning and problem being solved
- **What**: What is affected or changed
- **How**: High-level approach to implementation
- **How to Evaluate**: Testing/validation procedure
- Additional context as needed
### Living Specification Format
Specifications are **living documents** that evolve throughout implementation:
**Progress Tracking:**
- **Completed items**: Use ✅ checkmark emoji for implemented features
- **Pending items**: Use `- [ ]` GitHub-style checkboxes for remaining tasks
- **In-progress items**: Use `- [x]` when work is actively underway
**Status Philosophy:**
- **Avoid static status headers** like "COMPLETE" or "IN PROGRESS" that become stale
- **Use checklists within content** to show granular implementation progress
- **Keep specs informative** while providing clear progress visibility
- **Update continuously** as understanding and implementation evolve
**Example Format:**
```markdown
### ComponentName
- ✅ Basic functionality implemented
- ✅ Props and events defined
- - [ ] Add sorting controls
- - [ ] Improve accessibility
- - [x] Currently implementing responsive design
```
This creates **git-friendly progress tracking** where `[ ]` easily becomes `[x]` or ✅ when completed, and specs remain valuable throughout the development lifecycle.
## Claude Code
We will leverage Claude Code capabilities to make the process semi-automated.
- Slash commands: define repeatable steps in the process (create spec, implement, review, etc)
- Agents: define roles to carry out instructions (front end developer, baskend developer, etc)
- MCP tools: enable agents to implement specs via actions (write code, test, etc)
### Workflow
1. **Create**: Write spec as complete thought in `/specs` folder
2. **Discuss**: Iterate and refine through agent collaboration
3. **Implement**: Hand spec to appropriate specialist agent
4. **Validate**: Review implementation against spec criteria
5. **Document**: Update spec with learnings and decisions
### Slash Commands
Claude slash commands are used to manage the flow.
These are simple instructions to help make the process uniform.
They can be updated and refined as needed.
- `/spec create [name]` - Create new specification
- `/spec status` - Show current spec states
- `/spec implement [name]` - Hand to appropriate agent
- `/spec review [name]` - Validate implementation
### Agent Orchestration
Agents are defined with clear roles, for instance:
- **system-architect**: Creates high-level specs, ADRs, architectural decisions
- **vue-developer**: Component specs, UI patterns, frontend architecture
- **python-developer**: Implementation specs, technical details, backend logic
-
- Each agent reads/updates specs through basic-memory tools.
## How to Evaluate
### Success Criteria
- Specs provide clear, actionable guidance for implementation
- Reduced circular refactoring and scope creep
- Persistent context across development sessions
- Clean separation between "what/why" and implementation details
- Specs record a history of what happened and why for historical context
### Testing Procedure
1. Create a spec for an existing problematic component
2. Have an agent implement following only the spec
3. Compare result quality and development speed vs. ad-hoc approach
4. Measure context preservation across sessions
5. Evaluate spec clarity and completeness
### Metrics
- Time from spec to working implementation
- Number of refactoring cycles required
- Agent understanding of requirements
- Spec reusability for similar components
## Notes
- Start simple: specs are just complete thoughts, not heavy processes
- Use basic-memory's knowledge graph to link specs, decisions, components
- Let the process evolve naturally based on what works
- Focus on solving the actual problem: Manage complexity in development
## Observations
- [problem] Web development without clear goals and documentation circular refactoring cycles #complexity
- [solution] Specification-driven development reduces scope creep and context loss #process-improvement
- [pattern] basic-memory as specification engine creates recursive development loop #meta-development
- [workflow] Five-step process: Create → Discuss → Implement → Validate → Document #methodology
- [tool] Slash commands provide uniform process automation #automation
- [agent-pattern] Three specialized agents handle different implementation domains #specialization
- [success-metric] Time from spec to working implementation measures process efficiency #measurement
- [learning] Process should evolve naturally based on what works in practice #adaptation
- [format] Living specifications use checklists for progress tracking instead of static status headers #documentation
- [evolution] Specs evolve throughout implementation maintaining value as working documents #continuous-improvement
## Relations
- spec [[Spec-2: Slash Commands Reference]]
- spec [[Spec-3: Agent Definitions]]
@@ -0,0 +1,245 @@
---
title: 'SPEC-11: Basic Memory API Performance Optimization'
type: spec
permalink: specs/spec-11-basic-memory-api-performance-optimization
tags:
- performance
- api
- mcp
- database
- cloud
---
# SPEC-11: Basic Memory API Performance Optimization
## Why
The Basic Memory API experiences significant performance issues in cloud environments due to expensive per-request initialization. MCP tools making
HTTP requests to the API suffer from 350ms-2.6s latency overhead **before** any actual operation occurs.
**Root Cause Analysis:**
- GitHub Issue #82 shows repeated initialization sequences in logs (16:29:35 and 16:49:58)
- Each MCP tool call triggers full database initialization + project reconciliation
- `get_engine_factory()` dependency calls `db.get_or_create_db()` on every request
- `reconcile_projects_with_config()` runs expensive sync operations repeatedly
**Performance Impact:**
- Database connection setup: ~50-100ms per request
- Migration checks: ~100-500ms per request
- Project reconciliation: ~200ms-2s per request
- **Total overhead**: ~350ms-2.6s per MCP tool call
This creates compounding effects with tenant auto-start delays and increases timeout risk in cloud deployments.
Github issue: https://github.com/basicmachines-co/basic-memory-cloud/issues/82
## What
This optimization affects the **core basic-memory repository** components:
1. **API Lifespan Management** (`src/basic_memory/api/app.py`)
- Cache database connections in app state during startup
- Avoid repeated expensive initialization
2. **Dependency Injection** (`src/basic_memory/deps.py`)
- Modify `get_engine_factory()` to use cached connections
- Eliminate per-request database setup
3. **Initialization Service** (`src/basic_memory/services/initialization.py`)
- Add caching/throttling to project reconciliation
- Skip expensive operations when appropriate
4. **Configuration** (`src/basic_memory/config.py`)
- Add optional performance flags for cloud environments
**Backwards Compatibility**: All changes must be backwards compatible with existing CLI and non-cloud usage.
## How (High Level)
### Phase 1: Cache Database Connections (Critical - 80% of gains)
**Problem**: `get_engine_factory()` calls `db.get_or_create_db()` per request
**Solution**: Cache database engine/session in app state during lifespan
1. **Modify API Lifespan** (`api/app.py`):
```python
@asynccontextmanager
async def lifespan(app: FastAPI):
app_config = ConfigManager().config
await initialize_app(app_config)
# Cache database connection in app state
engine, session_maker = await db.get_or_create_db(app_config.database_path)
app.state.engine = engine
app.state.session_maker = session_maker
# ... rest of startup logic
```
2. Modify Dependency Injection (deps.py):
```python
async def get_engine_factory(
request: Request
) -> tuple[AsyncEngine, async_sessionmaker[AsyncSession]]:
"""Get cached engine and session maker from app state."""
return request.app.state.engine, request.app.state.session_maker
```
Phase 2: Optimize Project Reconciliation (Secondary - 20% of gains)
Problem: reconcile_projects_with_config() runs expensive sync repeatedly
Solution: Add module-level caching with time-based throttling
1. Add Reconciliation Cache (services/initialization.py):
```ptyhon
_project_reconciliation_completed = False
_last_reconciliation_time = 0
async def reconcile_projects_with_config(app_config, force=False):
# Skip if recently completed (within 60 seconds) unless forced
if recently_completed and not force:
return
# ... existing logic
```
Phase 3: Cloud Environment Flags (Optional)
Problem: Force expensive initialization in production environments
Solution: Add skip flags for cloud/stateless deployments
1. Add Config Flag (config.py):
skip_initialization_sync: bool = Field(default=False)
2. Configure in Cloud (basic-memory-cloud integration):
BASIC_MEMORY_SKIP_INITIALIZATION_SYNC=true
How to Evaluate
Success Criteria
1. Performance Metrics (Primary):
- MCP tool response time reduced by 50%+ (measure before/after)
- Database connection overhead eliminated (0ms vs 50-100ms)
- Migration check overhead eliminated (0ms vs 100-500ms)
- Project reconciliation overhead reduced by 90%+
2. Load Testing:
- Concurrent MCP tool calls maintain performance
- No memory leaks in cached connections
- Database connection pool behaves correctly
3. Functional Correctness:
- All existing API endpoints work identically
- MCP tools maintain full functionality
- CLI operations unaffected
- Database migrations still execute properly
4. Backwards Compatibility:
- No breaking changes to existing APIs
- Config changes are optional with safe defaults
- Non-cloud deployments work unchanged
Testing Strategy
Performance Testing:
# Before optimization
time basic-memory-mcp-tools write_note "test" "content" "folder"
# Measure: ~1-3 seconds
# After optimization
time basic-memory-mcp-tools write_note "test" "content" "folder"
# Target: <500ms
Load Testing:
# Multiple concurrent MCP tool calls
for i in {1..10}; do
basic-memory-mcp-tools search "test" &
done
wait
# Verify: No degradation, consistent response times
Regression Testing:
# Full basic-memory test suite
just test
# All tests must pass
# Integration tests with cloud deployment
# Verify MCP gateway → API → database flow works
Validation Checklist
- Phase 1 Complete: Database connections cached, dependency injection optimized
- Performance Benchmark: 50%+ improvement in MCP tool response times
- Memory Usage: No leaks in cached connections over 24h+ periods
- Stress Testing: 100+ concurrent requests maintain performance
- Backwards Compatibility: All existing functionality preserved
- Documentation: Performance optimization documented in README
- Cloud Integration: basic-memory-cloud sees performance benefits
## Implementation Status ✅ COMPLETED
**Implementation Date**: 2025-09-26
**Branch**: `feature/spec-11-api-performance-optimization`
**Commit**: `771f60b`
### ✅ Phase 1: Database Connection Caching - IMPLEMENTED
**Files Modified:**
- `src/basic_memory/api/app.py` - Added database connection caching in app.state
- `src/basic_memory/deps.py` - Updated get_engine_factory() to use cached connections
- `src/basic_memory/config.py` - Added skip_initialization_sync configuration flag
**Implementation Details:**
1. **API Lifespan Caching**: Database engine and session_maker cached in app.state during startup
2. **Dependency Injection Optimization**: get_engine_factory() now returns cached connections instead of calling get_or_create_db()
3. **Project Reconciliation Removal**: Eliminated expensive reconcile_projects_with_config() from API startup
4. **CLI Fallback Preserved**: Non-API contexts continue to work with fallback database initialization
### ✅ Performance Validation - ACHIEVED
**Live Testing Results** (2025-09-26 14:03-14:09):
| Operation | Before | After | Improvement |
|-----------|--------|-------|-------------|
| `read_note` | 350ms-2.6s | **20ms** | **95-99% faster** |
| `edit_note` | 350ms-2.6s | **218ms** | **75-92% faster** |
| `search_notes` | 350ms-2.6s | **<500ms** | **Responsive** |
| `list_memory_projects` | N/A | **<100ms** | **Fast** |
**Key Achievements:**
-**95-99% improvement** in read operations (primary workflow)
-**75-92% improvement** in edit operations
-**Zero overhead** for project switching
-**Database connection overhead eliminated** (0ms vs 50-100ms)
-**Project reconciliation delays removed** from API requests
-**<500ms target achieved** for all operations except write (which includes file sync)
### ✅ Backwards Compatibility - MAINTAINED
- All existing functionality preserved
- CLI operations unaffected
- Fallback for non-API contexts maintained
- No breaking changes to existing APIs
- Optional configuration with safe defaults
### ✅ Testing Validation - PASSED
- Integration tests passing
- Type checking clear
- Linting checks passed
- Live testing with real MCP tools successful
- Multi-project workflows validated
- Rapid project switching validated
## Notes
Implementation Priority:
- ✅ Phase 1 COMPLETED: Database connection caching provides 95%+ performance gains
- ⚪ Phase 2 NOT NEEDED: Project reconciliation removal achieved the goals
- ⚪ Phase 3 INCLUDED: skip_initialization_sync flag added
Risk Mitigation:
- ✅ All changes backwards compatible implemented
- ✅ Gradual implementation successful (Phase 1 → validation)
- ✅ Easy rollback via configuration flags available
Cloud Integration:
- ✅ This optimization directly addresses basic-memory-cloud issue #82
- ✅ Changes in core basic-memory will benefit all cloud tenants
- ✅ No changes needed in basic-memory-cloud itself
**Result**: SPEC-11 performance optimizations successfully implemented and validated. The 95-99% improvement in MCP tool response times exceeds the original 50-80% target, providing exceptional performance gains for cloud deployments and local usage.
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,210 @@
---
title: 'SPEC-14: Cloud Git Versioning & GitHub Backup'
type: spec
permalink: specs/spec-14-cloud-git-versioning
tags:
- git
- github
- backup
- versioning
- cloud
related:
- specs/spec-9-multi-project-bisync
- specs/spec-9-follow-ups-conflict-sync-and-observability
status: deferred
---
# SPEC-14: Cloud Git Versioning & GitHub Backup
**Status: DEFERRED** - Postponed until multi-user/teams feature development. Using S3 versioning (SPEC-9.1) for v1 instead.
## Why Deferred
**Original goals can be met with simpler solutions:**
- Version history → **S3 bucket versioning** (automatic, zero config)
- Offsite backup → **Tigris global replication** (built-in)
- Restore capability → **S3 version restore** (`bm cloud restore --version-id`)
- Collaboration → **Deferred to teams/multi-user feature** (not v1 requirement)
**Complexity vs value trade-off:**
- Git integration adds: committer service, puller service, webhooks, LFS, merge conflicts
- Risk: Loop detection between Git ↔ rclone bisync ↔ local edits
- S3 versioning gives 80% of value with 5% of complexity
**When to revisit:**
- Teams/multi-user features (PR-based collaboration workflow)
- User requests for commit messages and branch-based workflows
- Need for fine-grained audit trail beyond S3 object metadata
---
## Original Specification (for reference)
## Why
Early access users want **transparent version history**, easy **offsite backup**, and a familiar **restore/branching** workflow. Git/GitHub integration would provide:
- Auditable history of every change (who/when/why)
- Branches/PRs for review and collaboration
- Offsite private backup under the user's control
- Escape hatch: users can always `git clone` their knowledge base
**Note:** These goals are now addressed via S3 versioning (SPEC-9.1) for single-user use case.
## Goals
- **Transparent**: Users keep using Basic Memory; Git runs behind the scenes.
- **Private**: Push to a **private GitHub repo** that the user owns (or tenant org).
- **Reliable**: No data loss, deterministic mapping of filesystem ↔ Git.
- **Composable**: Plays nicely with SPEC9 bisync and upcoming conflict features (SPEC9 FollowUps).
**NonGoals (for v1):**
- Finegrained perfile encryption in Git history (can be layered later).
- Large media optimization beyond Git LFS defaults.
## User Stories
1. *As a user*, I connect my GitHub and choose a private backup repo.
2. *As a user*, every change I make in cloud (or via bisync) is **committed** and **pushed** automatically.
3. *As a user*, I can **restore** a file/folder/project to a prior version.
4. *As a power user*, I can **git pull/push** directly to collaborate outside the app.
5. *As an admin*, I can enforce repo ownership (tenant org) and leastprivilege scopes.
## Scope
- **In scope:** Full repo backup of `/app/data/` (all projects) with optional selective subpaths.
- **Out of scope (v1):** Partial shallow mirrors; encrypted Git; crossprovider SCM (GitLab/Bitbucket).
## Architecture
### Topology
- **Authoritative working tree**: `/app/data/` (bucket mount) remains the source of truth (SPEC9).
- **Bare repo** lives alongside: `/app/git/${tenant}/knowledge.git` (serverside).
- **Mirror remote**: `github.com/<owner>/<repo>.git` (private).
```mermaid
flowchart LR
A[/Users & Agents/] -->|writes/edits| B[/app/data/]
B -->|file events| C[Committer Service]
C -->|git commit| D[(Bare Repo)]
D -->|push| E[(GitHub Private Repo)]
E -->|webhook (push)| F[Puller Service]
F -->|git pull/merge| D
D -->|checkout/merge| B
```
### Services
- **Committer Service** (daemon):
- Watches `/app/data/` for changes (inotify/poll)
- Batches changes (debounce e.g. 25s)
- Writes `.bmmeta` (if present) into commit message trailer (see FollowUps)
- `git add -A && git commit -m "chore(sync): <summary>
BM-Meta: <json>"`
- Periodic `git push` to GitHub mirror (configurable interval)
- **Puller Service** (webhook target):
- Receives GitHub webhook (push) → `git fetch`
- **Fastforward** merges to `main` only; reject nonFF unless policy allows
- Applies changes back to `/app/data/` via clean checkout
- Emits sync events for Basic Memory indexers
### Auth & Security
- **GitHub App** (recommended): minimal scopes: `contents:read/write`, `metadata:read`, webhook.
- Tenantscoped installation; repo created in user account or tenant org.
- Tokens stored in KMS/secret manager; rotated automatically.
- Optional policy: allow only **FF merges** on `main`; nonFF requires PR.
### Repo Layout
- **Monorepo** (default): one repo per tenant mirrors `/app/data/` with subfolders per project.
- Optional multirepo mode (later): one repo per project.
### File Handling
- Honor `.gitignore` generated from `.bmignore.rclone` + BM defaults (cache, temp, state).
- **Git LFS** for large binaries (images, media) — auto track by extension/size threshold.
- Normalize newline + Unicode (aligns with FollowUps).
### Conflict Model
- **Primary concurrency**: SPEC9 FollowUps (`.bmmeta`, conflict copies) stays the first line of defense.
- **Git merges** are a **secondary** mechanism:
- Server only automerges **text** conflicts when trivial (FF or clean 3way).
- Otherwise, create `name (conflict from <branch>, <ts>).md` and surface via events.
### Data Flow vs Bisync
- Bisync (rclone) continues between local sync dir ↔ bucket.
- Git sits **cloudside** between bucket and GitHub.
- On **pull** from GitHub → files written to `/app/data/` → picked up by indexers & eventually by bisync back to users.
## CLI & UX
New commands (cloud mode):
- `bm cloud git connect` — Launch GitHub App installation; create private repo; store installation id.
- `bm cloud git status` — Show connected repo, last push time, last webhook delivery, pending commits.
- `bm cloud git push` — Manual push (rarely needed).
- `bm cloud git pull` — Manual pull/FF (admin only by default).
- `bm cloud snapshot -m "message"` — Create a tagged pointintime snapshot (git tag).
- `bm restore <path> --to <commit|tag>` — Restore file/folder/project to prior version.
Settings:
- `bm config set git.autoPushInterval=5s`
- `bm config set git.lfs.sizeThreshold=10MB`
- `bm config set git.allowNonFF=false`
## Migration & Backfill
- On connect, if repo empty: initial commit of entire `/app/data/`.
- If repo has content: require **onetime import** path (clone to staging, reconcile, choose direction).
## Edge Cases
- Massive deletes: gated by SPEC9 `max_delete` **and** Git prepush hook checks.
- Case changes and rename detection: rely on git rename heuristics + FollowUps move hints.
- Secrets: default ignore common secret patterns; allow custom deny list.
## Telemetry & Observability
- Emit `git_commit`, `git_push`, `git_pull`, `git_conflict` events with correlation IDs.
- `bm sync --report` extended with Git stats (commit count, delta bytes, push latency).
## Phased Plan
### Phase 0 — Prototype (1 sprint)
- Server: bare repo init + simple committer (batch every 10s) + manual GitHub token.
- CLI: `bm cloud git connect --token <PAT>` (devonly)
- Success: edits in `/app/data/` appear in GitHub within 30s.
### Phase 1 — GitHub App & Webhooks (12 sprints)
- Switch to GitHub App installs; create private repo; store installation id.
- Committer hardened (debounce 25s, backoff, retries).
- Puller service with webhook → FF merge → checkout to `/app/data/`.
- LFS autotrack + `.gitignore` generation.
- CLI surfaces status + logs.
### Phase 2 — Restore & Snapshots (1 sprint)
- `bm restore` for file/folder/project with dryrun.
- `bm cloud snapshot` tags + list/inspect.
- Policy: PRonly nonFF, admin override.
### Phase 3 — Selective & MultiRepo (nicetohave)
- Include/exclude projects; optional perproject repos.
- Advanced policies (branch protections, required reviews).
## Acceptance Criteria
- Changes to `/app/data/` are committed and pushed automatically within configurable interval (default ≤5s).
- GitHub webhook pull results in updated files in `/app/data/` (FFonly by default).
- LFS configured and functioning; large files don't bloat history.
- `bm cloud git status` shows connected repo and last push/pull times.
- `bm restore` restores a file/folder to a prior commit with a clear audit trail.
- Endtoend works alongside SPEC9 bisync without loops or data loss.
## Risks & Mitigations
- **Loop risk (Git ↔ Bisync)**: Writes to `/app/data/` → bisync → local → user edits → back again. *Mitigation*: Debounce, commit squashing, idempotent `.bmmeta` versioning, and watch exclusion windows during pull.
- **Repo bloat**: Lots of binary churn. *Mitigation*: default LFS, size threshold, optional mediaonly repo later.
- **Security**: Token leakage. *Mitigation*: GitHub App with shortlived tokens, KMS storage, scoped permissions.
- **Merge complexity**: Nontrivial conflicts. *Mitigation*: prefer FF; otherwise conflict copies + events; require PR for nonFF.
## Open Questions
- Do we default to **monorepo** per tenant, or offer projectperrepo at connect time?
- Should `restore` write to a branch and open a PR, or directly modify `main`?
- How do we expose Git history in UI (timeline view) without users dropping to CLI?
## Appendix: Sample Config
```json
{
"git": {
"enabled": true,
"repo": "https://github.com/<owner>/<repo>.git",
"autoPushInterval": "5s",
"allowNonFF": false,
"lfs": { "sizeThreshold": 10485760 }
}
}
```
+120
View File
@@ -0,0 +1,120 @@
---
title: 'SPEC-2: Slash Commands Reference'
type: spec
permalink: specs/spec-2-slash-commands-reference
tags:
- commands
- process
- reference
---
# SPEC-2: Slash Commands Reference
This document defines the slash commands used in our specification-driven development process.
## /spec create [name]
**Purpose**: Create a new specification document
**Usage**: `/spec create notes-decomposition`
**Process**:
1. Create new spec document in `/specs` folder
2. Use SPEC-XXX numbering format (auto-increment)
3. Include standard spec template:
- Why (reasoning/problem)
- What (affected areas)
- How (high-level approach)
- How to Evaluate (testing/validation)
4. Tag appropriately for knowledge graph
5. Link to related specs/components
**Template**:
```markdown
# SPEC-XXX: [Title]
## Why
[Problem statement and reasoning]
## What
[What is affected or changed]
## How (High Level)
[Approach to implementation]
## How to Evaluate
[Testing/validation procedure]
## Notes
[Additional context as needed]
```
## /spec status
**Purpose**: Show current status of all specifications
**Usage**: `/spec status`
**Process**:
1. Search all specs in `/specs` folder
2. Display table showing:
- Spec number and title
- Status (draft, approved, implementing, complete)
- Assigned agent (if any)
- Last updated
- Dependencies
## /spec implement [name]
**Purpose**: Hand specification to appropriate agent for implementation
**Usage**: `/spec implement SPEC-002`
**Process**:
1. Read the specified spec
2. Analyze requirements to determine appropriate agent:
- Frontend components → vue-developer
- Architecture/system design → system-architect
- Backend/API → python-developer
3. Launch agent with spec context
4. Agent creates implementation plan
5. Update spec with implementation status
## /spec review [name]
**Purpose**: Review implementation against specification criteria
**Usage**: `/spec review SPEC-002`
**Process**:
1. Read original spec and "How to Evaluate" section
2. Examine current implementation
3. Test against success criteria
4. Document gaps or issues
5. Update spec with review results
6. Recommend next actions (complete, revise, iterate)
## Command Extensions
As the process evolves, we may add:
- `/spec link [spec1] [spec2]` - Create dependency links
- `/spec archive [name]` - Archive completed specs
- `/spec template [type]` - Create spec from template
- `/spec search [query]` - Search spec content
## References
- Claude Slash commands: https://docs.anthropic.com/en/docs/claude-code/slash-commands
## Creating a command
Commands are implemented as Claude slash commands:
Location in repo: .claude/commands/
In the following example, we create the /optimize command:
```bash
# Create a project command
mkdir -p .claude/commands
echo "Analyze this code for performance issues and suggest optimizations:" > .claude/commands/optimize.md
```
+108
View File
@@ -0,0 +1,108 @@
---
title: 'SPEC-3: Agent Definitions'
type: spec
permalink: specs/spec-3-agent-definitions
tags:
- agents
- roles
- process
---
# SPEC-3: Agent Definitions
This document defines the specialist agents used in our specification-driven development process.
## system-architect
**Role**: High-level system design and architectural decisions
**Responsibilities**:
- Create architectural specifications and ADRs
- Analyze system-wide impacts and trade-offs
- Design component interfaces and data flow
- Evaluate technical approaches and patterns
- Document architectural decisions and rationale
**Expertise Areas**:
- System architecture and design patterns
- Technology evaluation and selection
- Scalability and performance considerations
- Integration patterns and API design
- Technical debt and refactoring strategies
**Typical Specs**:
- System architecture overviews
- Component decomposition strategies
- Data flow and state management
- Integration and deployment patterns
## vue-developer
**Role**: Frontend component development and UI implementation
**Responsibilities**:
- Create Vue.js component specifications
- Implement responsive UI components
- Design component APIs and interfaces
- Optimize for performance and accessibility
- Document component usage and patterns
**Expertise Areas**:
- Vue.js 3 Composition API
- Nuxt 3 framework patterns
- shadcn-vue component library
- Responsive design and CSS
- TypeScript integration
- State management with Pinia
**Typical Specs**:
- Individual component specifications
- UI pattern libraries
- Responsive design approaches
- Component interaction flows
## python-developer
**Role**: Backend development and API implementation
**Responsibilities**:
- Create backend service specifications
- Implement APIs and data processing
- Design database schemas and queries
- Optimize performance and reliability
- Document service interfaces and behavior
**Expertise Areas**:
- FastAPI and Python web frameworks
- Database design and operations
- API design and documentation
- Authentication and security
- Performance optimization
- Testing and validation
**Typical Specs**:
- API endpoint specifications
- Database schema designs
- Service integration patterns
- Performance optimization strategies
## Agent Collaboration Patterns
### Handoff Protocol
1. Agent receives spec through `/spec implement [name]`
2. Agent reviews spec and creates implementation plan
3. Agent documents progress and decisions in spec
4. Agent hands off to another agent if cross-domain work needed
5. Final agent updates spec with completion status
### Communication Standards
- All agents update specs through basic-memory MCP tools
- Document decisions and trade-offs in spec notes
- Link related specs and components
- Preserve context for future reference
### Quality Standards
- Follow existing codebase patterns and conventions
- Write tests that validate spec requirements
- Document implementation choices
- Consider maintainability and extensibility
+201
View File
@@ -0,0 +1,201 @@
---
title: 'SPEC-5: CLI Cloud Upload via WebDAV'
type: spec
permalink: specs/spec-5-cli-cloud-upload-via-webdav
tags:
- cli
- webdav
- upload
- migration
- poc
---
# SPEC-5: CLI Cloud Upload via WebDAV
## Why
Existing basic-memory users need a simple migration path to basic-memory-cloud. The web UI drag-and-drop approach outlined in GitHub issue #59, while user-friendly, introduces significant complexity for a proof-of-concept:
- Complex web UI components for file upload and progress tracking
- Browser file handling limitations and CORS complexity
- Proxy routing overhead for large file transfers
- Authentication integration across multiple services
A CLI-first approach solves these issues by:
- **Leveraging existing infrastructure**: Both cloud CLI and tenant API already exist with WorkOS JWT authentication
- **Familiar user experience**: Basic-memory users are CLI-comfortable and expect command-line tools
- **Direct connection efficiency**: Bypassing the MCP gateway/proxy for bulk file transfers
- **Rapid implementation**: Building on existing `CLIAuth` and FastAPI foundations
The fundamental problem is migration friction - users have local basic-memory projects but no path to cloud tenants. A simple CLI upload command removes this barrier immediately.
## What
This spec defines a CLI-based project upload system using WebDAV for direct tenant connections.
**Affected Areas:**
- `apps/cloud/src/basic_memory_cloud/cli/main.py` - Add upload command to existing CLI
- `apps/api/src/basic_memory_cloud_api/main.py` - Add WebDAV endpoints to tenant FastAPI
- Authentication flow - Reuse existing WorkOS JWT validation
- File transfer protocol - WebDAV for cross-platform compatibility
**Core Components:**
### CLI Upload Command
```bash
basic-memory-cloud upload <project-path> --tenant-url https://basic-memory-{tenant}.fly.dev
```
### WebDAV Server Endpoints
- `GET/PUT/DELETE /webdav/*` - Standard WebDAV operations on tenant file system
- Authentication via existing JWT validation
- File operations preserve timestamps and directory structure
### Authentication Flow
```
1. User runs `basic-memory-cloud login` (existing)
2. CLI stores WorkOS JWT token (existing)
3. Upload command reads JWT from storage
4. WebDAV requests include JWT in Authorization header
5. Tenant API validates JWT using existing middleware
```
## How (High Level)
### Implementation Strategy
**Phase 1: CLI Command**
- Add `upload` command to existing Typer app
- Reuse `CLIAuth` class for token management
- Implement WebDAV client using `webdavclient3` or similar
- Rich progress bars for transfer feedback
**Phase 2: WebDAV Server**
- Add WebDAV endpoints to existing tenant FastAPI app
- Leverage existing `get_current_user` dependency for authentication
- Map WebDAV operations to tenant file system
- Preserve file modification times using `os.utime()`
**Phase 3: Integration**
- Direct connection bypasses MCP gateway and proxy
- Simple conflict resolution: overwrite existing files
- Error handling: fail fast with clear error messages
### Technical Architecture
```
basic-memory-cloud CLI → WorkOS JWT → Direct WebDAV → Tenant FastAPI
Tenant File System
```
**Key Libraries:**
- CLI: `webdavclient3` for WebDAV client operations
- API: `wsgidav` or FastAPI-compatible WebDAV server
- Progress: `rich` library (already imported in CLI)
- Auth: Existing WorkOS JWT infrastructure
### WebDAV Protocol Choice
WebDAV provides:
- **Cross-platform clients**: Native support in most operating systems
- **Standardized protocol**: Well-defined for file operations
- **HTTP-based**: Works with existing FastAPI and JWT auth
- **Library support**: Good Python libraries for both client and server
### POC Constraints
**Simplifications for rapid implementation:**
- **Known tenant URLs**: Assume `https://basic-memory-{tenant}.fly.dev` format
- **Upload only**: No download or bidirectional sync
- **Overwrite conflicts**: No merge or conflict resolution prompting
- **No fallbacks**: Fail fast if WebDAV connection issues occur
- **Direct connection only**: No proxy fallback mechanism
## How to Evaluate
### Success Criteria
**Functional Requirements:**
- [ ] Transfer complete basic-memory project (100+ files) in < 30 seconds
- [ ] Preserve directory structure exactly as in source project
- [ ] Preserve file modification timestamps for proper sync behavior
- [ ] Rich progress bars show real-time transfer status (files/MB transferred)
- [ ] WorkOS JWT authentication validates correctly on WebDAV endpoints
- [ ] Direct tenant connection bypasses MCP gateway successfully
**Quality Requirements:**
- [ ] Clear error messages for authentication failures
- [ ] Graceful handling of network interruptions
- [ ] CLI follows existing command patterns and help text standards
- [ ] WebDAV endpoints integrate cleanly with existing FastAPI app
**Performance Requirements:**
- [ ] File transfer speed > 1MB/s on typical connections
- [ ] Memory usage remains reasonable for large projects
- [ ] No timeout issues with 500+ file projects
### Testing Procedure
**Unit Testing:**
1. CLI command parsing and argument validation
2. WebDAV client connection and authentication
3. File timestamp preservation during transfer
4. JWT token validation on WebDAV endpoints
**Integration Testing:**
1. End-to-end upload of test project
2. Direct tenant connection without proxy
3. File integrity verification after upload
4. Progress tracking accuracy during transfer
**User Experience Testing:**
1. Upload existing basic-memory project from local installation
2. Verify uploaded files appear correctly in cloud tenant
3. Confirm basic-memory database rebuilds properly with uploaded files
4. Test CLI help text and error message clarity
### Validation Commands
**Setup:**
```bash
# Login to WorkOS
basic-memory-cloud login
# Upload project
basic-memory-cloud upload ~/my-notes --tenant-url https://basic-memory-test.fly.dev
```
**Verification:**
```bash
# Check tenant health and file count via API
curl -H "Authorization: Bearer $JWT" https://basic-memory-test.fly.dev/health
curl -H "Authorization: Bearer $JWT" https://basic-memory-test.fly.dev/notes/search
```
### Performance Benchmarks
**Target metrics for 100MB basic-memory project:**
- Transfer time: < 30 seconds
- Memory usage: < 100MB during transfer
- Progress updates: Every 1MB or 10 files
- Authentication time: < 2 seconds
## Observations
- [implementation-speed] CLI approach significantly faster than web UI for POC development #rapid-prototyping
- [user-experience] Basic-memory users already comfortable with CLI tools #user-familiarity
- [architecture-benefit] Direct connection eliminates proxy complexity and latency #performance
- [auth-reuse] Existing WorkOS JWT infrastructure handles authentication cleanly #code-reuse
- [webdav-choice] WebDAV protocol provides cross-platform compatibility and standard libraries #protocol-selection
- [poc-scope] Simple conflict handling and error recovery sufficient for proof-of-concept #scope-management
- [migration-value] Removes primary barrier for local users migrating to cloud platform #business-value
## Relations
- depends_on [[SPEC-1: Specification-Driven Development Process]]
- enables [[GitHub Issue #59: Web UI Upload Feature]]
- uses [[WorkOS Authentication Integration]]
- builds_on [[Existing Cloud CLI Infrastructure]]
- builds_on [[Existing Tenant API Architecture]]
@@ -0,0 +1,486 @@
---
title: 'SPEC-6: Explicit Project Parameter Architecture'
type: spec
permalink: specs/spec-6-explicit-project-parameter-architecture
tags:
- architecture
- mcp
- project-management
- stateless
---
# SPEC-6: Explicit Project Parameter Architecture
## Why
The current session-based project management system has critical reliability issues:
1. **Session State Fragility**: Claude iOS mobile client fails to maintain consistent session IDs across MCP tool calls, causing project switching to silently fail (Issue #74)
2. **Scaling Limitations**: Redis-backed session state creates single-point-of-failure and prevents horizontal scaling
3. **Client Compatibility**: Session tracking works inconsistently across different MCP clients (web, mobile, API)
4. **Hidden Complexity**: Users cannot see or understand "current project" state, leading to confusion when operations execute in wrong projects
5. **Silent Failures**: Operations appear successful but execute in unintended projects, risking data integrity
Evidence from production logs shows each MCP tool call from mobile client receives different session IDs:
```
create_memory_project: session_id=12cdfc24913b48f8b680ed4b2bfdb7ba
switch_project: session_id=050a69275d98498cbdd227cdb74d9740
list_directory: session_id=85f3483014af4136a5d435c76ded212f
```
Related Github issue: https://github.com/basicmachines-co/basic-memory-cloud/issues/75
## Status
**Current Status**: **Phase 1 Implementation Complete**
**Target**: Fix Claude iOS session ID consistency issues
**Draft PR**: https://github.com/basicmachines-co/basic-memory/pull/298
### 🎉 **MAJOR MILESTONE ACHIEVED**
The complete stateless architecture has been successfully implemented for Basic Memory's MCP server! This represents a **fundamental architectural improvement** that solves the Claude iOS compatibility issue while making the entire system more robust and predictable.
#### Implementation Summary:
- **16 files modified** with 582 additions and 550 deletions
- **All 17 MCP tools** converted to stateless architecture
- **147 tests updated** across 5 test files (100% passing)
- **Complete session state removal** from core MCP tools
- **Enhanced error handling** and security validations preserved
### Progress Summary
**Complete Stateless Architecture Implementation (All 17 tools)**
- Stateless `get_active_project()` function implemented and deployed
- All session state dependencies removed across entire MCP server
- All MCP tools require explicit `project` parameter as first argument
**Content Management Tools Complete (6/6 tools)**
- `write_note`, `read_note`, `delete_note`, `edit_note`
- `view_note`, `read_content`
**Knowledge Graph Navigation Tools Complete (3/3 tools)**
- `build_context`, `recent_activity`, `list_directory`
**Search & Discovery Tools Complete (1/1 tools)**
- `search_notes`
**Visualization Tools Complete (1/1 tools)**
- `canvas`
**Project Management Cleanup Complete**
- Removed `switch_project` and `get_current_project` tools ✅
- Updated `set_default_project` to remove activate parameter ✅
**Comprehensive Testing Complete (157 tests)**
- All test suites updated to use stateless architecture (147 existing tests)
- Single project constraint mode integration tests (10 new tests)
- 100% test pass rate across all tool test files
- Security validations preserved and working
- Error handling comprehensive and user-friendly
**Documentation & Examples Complete**
- All tool docstrings updated with stateless examples
- Project parameter usage clearly documented
- Error handling and security behavior documented
**Enhanced Discovery Mode Complete**
- `recent_activity` tool supports dual-mode operation (discovery vs project-specific)
- ProjectActivitySummary schema provides cross-project insights
- Recent activity prompt updated to support both modes
- Comprehensive project distribution statistics and most active project tracking
**Single Project Constraint Mode Complete**
- `--project` CLI parameter for MCP server constraint
- Environment variable control (`BASIC_MEMORY_MCP_PROJECT`)
- Automatic project override in `get_active_project()` function
- Project management tools disabled in constrained mode with helpful CLI guidance
- Comprehensive integration test suite (10 tests covering all constraint scenarios)
## What
Transform Basic Memory from stateful session-based to stateless explicit project parameter architecture:
### Core Changes
1. **Mandatory Project Parameter**: All MCP tools require explicit `project` parameter
2. **Remove Session State**: Eliminate Redis, session middleware, and `switch_project` tool
3. **Stateless HTTP**: Enable `stateless_http=True` for horizontal scaling
4. **Enhanced Context Discovery**: Improve `recent_activity` to show project distribution
5. **Clear Response Format**: All tool responses display target project information
Implementation Approach
- Each tool will directly accept the project parameter
- Remove all calls to context-based project retrieval
- Validate project exists before operations
- Clear error messages when project not found
- Backward compatibility: Initially keep optional parameter, then make required
### Affected MCP Tools
**Content Management** (require project parameter):
- `write_note(project, title, content, folder)`
- `read_note(project, identifier)`
- `edit_note(project, identifier, operation, content)`
- `delete_note(project, identifier)`
- `view_note(project, identifier)`
- `read_content(project, path)`
**Knowledge Graph Navigation** (require project parameter):
- `build_context(project, url, timeframe, depth, max_related)`
- `list_directory(project, dir_name, depth, file_name_glob)`
- `search_notes(project, query, search_type, types, entity_types)`
**Search & Discovery** (use project parameter for specific project or none for discovery):
- `recent_activity(project, timeframe, depth, max_related)`
**Visualization** (require project parameter):
- `canvas(project, nodes, edges, title, folder)`
**Project Management** (unchanged - already stateless):
- `list_memory_projects()`
- `create_memory_project(project_name, project_path, set_default)`
- `delete_project(project_name)`
- `get_current_project()` - Remove this tool
- `switch_project(project_name)` - Remove this tool
- `set_default_project(project_name, activate)` - Remove activate parameter
## How (High Level)
### Phase 1: Basic Memory Core (basic-memory repository)
#### MCP Tool Updates
Phase 1: Core Changes
1. Update project_context.py
- [x] Make project parameter mandatory for get_active_project()
- [x] Remove session state handling
2. Update Content Management Tools (6 tools)
- [x] write_note: Make project parameter required, not optional
- [x] read_note: Make project parameter required
- [x] edit_note: Add required project parameter
- [x] delete_note: Add required project parameter
- [x] view_note: Add required project parameter
- [x] read_content: Add required project parameter
3. Update Knowledge Graph Navigation Tools (3 tools)
- [x] build_context: Add required project parameter
- [x] recent_activity: Make project parameter required
- [x] list_directory: Add required project parameter
4. Update Search & Visualization Tools (2 tools)
- [x] search_notes: Add required project parameter
- [x] canvas: Add required project parameter
5. Update Project Management Tools
- [x] Remove switch_project tool completely
- [x] Remove get_current_project tool completely
- [x] Update set_default_project to remove activate parameter
- [x] Keep list_memory_projects, create_memory_project, delete_project unchanged
6. Enhance recent_activity Response
- [x] Add project distribution info showing activity across all projects
- [x] Include project usage stats in response
- [x] Implement ProjectActivitySummary for discovery mode
- [x] Add dual-mode functionality (discovery vs project-specific)
7. Update Tool Documentation
- [x] Update write_note docstring with stateless architecture examples
- [x] Update read_note docstring with project parameter examples
- [x] Update delete_note docstring with comprehensive usage guidance
- [x] Update all remaining tool docstrings with project parameter examples
8. Update Tool Responses
- [x] Add clear project indicator to all tool responses across all tools
- [x] Format: "project: {project_name}" in response metadata
- [x] Add project metadata footer for LLM awareness
- [x] Update all tool responses to include project indicators
9. Comprehensive Testing
- [x] Update all write_note tests to use stateless architecture (34 tests passing)
- [x] Update all edit_note tests to use stateless architecture (17 tests passing)
- [x] Update all view_note tests to use stateless architecture (12 tests passing)
- [x] Update all search_notes tests to use stateless architecture (16 tests passing)
- [x] Update all move_note tests to use stateless architecture (31 tests passing)
- [x] Update all delete_note tests to use stateless architecture
- [x] Verify direct function call compatibility (bypassing MCP layer)
- [x] Test security validation with project parameters
- [x] Validate error handling for non-existent projects
- [x] **Total: 157 tests updated and passing (100% success rate)**
- [x] **147 existing tests** updated for stateless architecture
- [x] **10 new tests** for single project constraint mode
### Phase 1.5: Default Project Mode Enhancement
#### Problem
While the stateless architecture solves reliability issues, it introduces UX friction for single-project users (estimated 80% of usage) who must specify the project parameter in every tool call.
#### Solution: Default Project Mode
Add optional `default_project_mode` configuration that allows single-project users to have the simplicity of implicit project selection while maintaining the reliability of stateless architecture.
#### Configuration
```json
{
"default_project": "main",
"default_project_mode": true // NEW: Auto-use default_project when not specified
}
```
#### Implementation Details
1. **Config Enhancement** (`src/basic_memory/config.py`)
- Add `default_project_mode: bool = Field(default=False)`
- Preserves backward compatibility (defaults to false)
2. **Project Resolution Logic** (`src/basic_memory/mcp/project_context.py`)
Three-tier resolution hierarchy:
- Priority 1: CLI `--project` constraint (BASIC_MEMORY_MCP_PROJECT env var)
- Priority 2: Explicit project parameter in tool call
- Priority 3: `default_project` if `default_project_mode=true` and no project specified
3. **Assistant Guide Updates** (`src/basic_memory/mcp/resources/ai_assistant_guide.md`)
- Detect `default_project_mode` at runtime
- Provide mode-specific instructions to LLMs
- In default mode: "All operations use project 'main' automatically"
- In regular mode: Current project discovery guidance
4. **Tool Parameter Handling** (all MCP tools)
- Make project parameter Optional[str] = None
- Add resolution logic: `project = project or get_default_project()`
- Maintain explicit project override capability
#### Usage Modes Summary
- **Regular Mode**: Multi-project users, assistant tracks project per conversation
- **Default Project Mode**: Single-project users, automatic default project
- **Constrained Mode**: CLI --project flag, locked to specific project
#### Testing Requirements
- Integration test for default_project_mode=true with missing parameters
- Test explicit project override in default_project_mode
- Test mode=false requires explicit parameters
- Test CLI constraint overrides default_project_mode
Phase 2: Testing & Validation
8. Update Tests
- [x] Modify all MCP tool tests to pass required project parameter
- [x] Remove tests for deleted tools (switch_project, get_current_project)
- [x] Add tests for project parameter validation
- [x] **Complete: All 147 tests across 5 test files updated and passing**
#### Enhanced recent_activity Response
```json
{
"recent_notes": [...],
"project_activity": {
"research-project": {
"operations": 5,
"last_used": "30 minutes ago",
"recent_folders": ["experiments", "findings"]
},
"work-notes": {
"operations": 2,
"last_used": "2 hours ago",
"recent_folders": ["meetings", "planning"]
}
},
"total_projects": 3
}
```
#### Response Format Updates
```
✓ Note created successfully
Project: research-project
File: experiments/Neural Network Results.md
Permalink: research-project/neural-network-results
```
### Phase 2: Cloud Service Simplification (basic-memory-cloud repository)
#### Remove Session Infrastructure
1. Delete `apps/mcp/src/basic_memory_cloud_mcp/middleware/session_state.py`
2. Delete `apps/mcp/src/basic_memory_cloud_mcp/middleware/session_logging.py`
3. Update `apps/mcp/src/basic_memory_cloud_mcp/main.py`:
```python
# Remove session middleware
# server.add_middleware(SessionStateMiddleware)
# Enable stateless HTTP
mcp = FastMCP(name="basic-memory-mcp", stateless_http=True)
```
#### Deployment Simplification
1. Remove Redis from `fly.toml`
2. Remove Redis environment variables
3. Update health checks to not depend on Redis
### Phase 3: Conversational Project Management
#### Claude Behavior Pattern
1. **Project Discovery**:
```
Claude: Let me check your recent activity...
[calls recent_activity() - no project needed for discovery]
I see you've been working in:
- research-project (5 operations, 30 min ago)
- work-notes (2 operations, 2 hours ago)
Which project should I use for this operation?
```
2. **Context Maintenance**:
```
User: Use research-project
Claude: Working in research-project.
[All subsequent operations use project="research-project"]
```
3. **Explicit Project Switching**:
```
User: Check work-notes for that meeting summary
Claude: Let me search work-notes for the meeting summary.
[Uses project="work-notes" for specific operation]
```
## How to Evaluate
### Success Criteria
#### 1. Functional Completeness
- [x] All MCP tools accept required `project` parameter
- [x] All MCP tools validate project exists before execution
- [x] `switch_project` and `get_current_project` tools removed
- [x] All responses display target project clearly
- [ ] No Redis dependencies in deployment (Phase 2: Cloud Service)
- [x] `recent_activity` shows project distribution with ProjectActivitySummary
#### 2. Cross-Client Compatibility Testing
Test identical operations across all clients:
- [ ] **Claude Desktop**: All operations work with explicit projects
- [ ] **Claude Code**: All operations work with explicit projects
- [ ] **Claude Mobile iOS**: All operations work with explicit projects
- [ ] **API clients**: All operations work with explicit projects
- [ ] **CLI tools**: All operations work with explicit projects
#### 3. Session Independence Verification
- [ ] Operations work identically with/without session tracking
- [ ] No behavioral differences between clients
- [ ] Mobile client session ID changes do not affect operations
- [ ] Redis can be completely removed without functional impact
#### 4. Performance & Scaling
- [ ] `stateless_http=True` enabled successfully
- [ ] No Redis memory usage
- [ ] Horizontal scaling possible (multiple MCP instances)
- [ ] Response times unchanged or improved
#### 5. User Experience Testing
**Project Discovery Flow**:
- [x] `recent_activity()` provides useful project context
- [x] Claude can intelligently suggest projects based on activity
- [x] Project switching is explicit and clear in conversation
**Error Handling**:
- [x] Clear error messages for non-existent projects
- [x] Helpful suggestions when project parameter missing
- [x] No silent failures or wrong-project operations
**Response Clarity**:
- [x] Every operation clearly shows target project
- [x] Users always know which project is being operated on
- [x] No confusion about "current project" state
#### 6. Migration Safety
- [ ] Backward compatibility period with optional project parameter
- [ ] Clear migration documentation for existing users
- [ ] Data integrity maintained during transition
- [ ] No data loss during migration
### Test Scenarios
#### Core Functionality Test
```bash
# Test all tools work with explicit project
write_note(project="test-proj", title="Test", content="Content", folder="docs")
read_note(project="test-proj", identifier="Test")
edit_note(project="test-proj", identifier="Test", operation="append", content="More")
search_notes(project="test-proj", query="Content")
list_directory(project="test-proj", dir_name="docs")
delete_note(project="test-proj", identifier="Test")
```
#### Cross-Client Consistency Test
Run identical test sequence on:
1. Claude Desktop
2. Claude Code
3. Claude Mobile iOS
4. API client
5. CLI tools
Verify all clients:
- Accept explicit project parameters
- Return identical responses
- Show same project information
- Have no session dependencies
#### Session Independence Test
1. Monitor session IDs during operations
2. Verify operations work with changing session IDs
3. Confirm Redis removal doesn't affect functionality
4. Test with multiple concurrent clients
### Acceptance Criteria
**Must Have**:
- All MCP tools require and use explicit project parameter
- No session state dependencies remain
- Universal client compatibility achieved
- Clear project information in all responses
**Should Have**:
- Enhanced `recent_activity` with project distribution
- Smooth migration path for existing users
- Improved performance with stateless architecture
**Could Have**:
- Smart project suggestions based on content/context
- Project shortcuts for common operations
- Advanced project analytics in responses
## Notes
### Breaking Changes
This is a **breaking change** that requires:
- All MCP clients to pass project parameter
- Migration of existing workflows
- Update of all documentation and examples
### Implementation Order
1. **basic-memory core** - Update MCP tools to accept project parameter (optional initially)
2. **Testing** - Verify all clients work with explicit projects
3. **Cloud service** - Remove session infrastructure
4. **Migration** - Make project parameter mandatory
5. **Cleanup** - Remove deprecated tools and middleware
### Related Issues
- Fixes #74 (Claude iOS session state bug)
- Implements #75 (Mandatory project parameter architecture)
- Enables future horizontal scaling
- Simplifies multi-tenant architecture
### Dependencies
- Requires coordination between basic-memory and basic-memory-cloud repositories
- Needs client-side updates for smooth transition
- Documentation updates across all materials
@@ -0,0 +1,193 @@
---
title: 'SPEC-7: POC to spike Tigris/Turso for local access to cloud data'
type: spec
permalink: specs/spec-7-poc-tigris-turso-local-access-cloud-data
tags:
- poc
- tigris
- turso
- cloud-storage
- architecture
- proof-of-concept
---
# SPEC-7: POC to spike Tigris/Turso for local access to cloud data
## Why
Current basic-memory-cloud architecture uses Fly volumes for tenant file storage, which creates several limitations:
1. **Storage Scalability**: Fly volumes require pre-provisioning and don't auto-scale with usage
2. **Cost Model**: Volume pricing vs object storage pricing may be less favorable at scale
3. **Local Development**: No way for users to mount their cloud tenant files locally for real-time editing
4. **Multi-Region**: Volumes are region-locked, limiting global deployment flexibility
5. **Backup/Disaster Recovery**: Object storage provides better durability and replication options
The core insight is that Basic Memory requires POSIX filesystem semantics but could benefit from object storage durability and accessibility. By combining:
- **Tigris object storage** for file persistence (via rclone mount)
- **Turso/libSQL** for SQLite indexing (replacing local .db files)
We could enable a revolutionary user experience: **local editing of cloud-stored files** while maintaining Basic Memory's existing filesystem assumptions.
## What
This specification defines a proof-of-concept to validate the technical feasibility of the Tigris/Turso architecture for basic-memory-cloud tenants.
**Affected Areas:**
- **Storage Architecture**: Replace Fly volumes with Tigris object storage
- **Database Architecture**: Replace local SQLite with Turso remote database
- **Container Setup**: Add rclone mounting in tenant containers
- **Local Development**: Enable local mounting of cloud tenant data
- **Basic Memory Core**: Validate unchanged operation over mounted filesystems
**Key Components:**
- **Tigris Storage**: S3-compatible object storage via Fly.io integration
- **rclone NFS Mount**: Native NFS mounting without FUSE dependencies
- **Turso Database**: Hosted libSQL for SQLite replacement
- **Single-Tenant Model**: One bucket + one database per tenant (simplified isolation)
## How (High Level)
### Phase 1: Local POC Validation
- [ ] Set up Tigris bucket with test data
- [ ] Configure rclone NFS mount locally
- [ ] Test Basic Memory operations over mounted filesystem
- [ ] Measure performance characteristics and identify issues
- [ ] Validate file watching, sync operations, and concurrent access patterns
### Phase 2: Database Migration
- [ ] Set up Turso account and test database
- [ ] Modify Basic Memory to accept external DATABASE_URL
- [ ] Test all operations with remote SQLite via Turso
- [ ] Validate performance and functionality parity
### Phase 3: Container Integration
- [ ] Create container image with rclone + NFS support
- [ ] Implement tenant-specific credential management
- [ ] Test container startup with automatic mounting
- [ ] Validate isolation between tenant containers
### Phase 4: Local Access Validation
- [ ] Test local rclone mounting of tenant data
- [ ] Validate real-time file editing experience
- [ ] Test conflict resolution and sync behavior
- [ ] Measure latency impact on user experience
### Architecture Overview
```
Local Development:
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
│ Local rclone │───▶│ Tigris Bucket │◀───│ Tenant Container│
│ NFS Mount │ │ (S3 storage) │ │ rclone mount │
└─────────────────┘ └─────────────────┘ └─────────────────┘
│ │
▼ ▼
┌─────────────────┐ ┌─────────────────┐
│ Basic Memory │ │ Basic Memory │
│ (local files) │ │ (mounted files) │
└─────────────────┘ └─────────────────┘
│ │
▼ ▼
┌─────────────────┐ ┌─────────────────┐
│ Turso Database │◀───────────────────────────│ Turso Database │
│ (shared index) │ │ (shared index) │
└─────────────────┘ └─────────────────┘
```
## How to Evaluate
### Success Criteria
- [ ] **Filesystem Compatibility**: Basic Memory operates without modification over rclone-mounted Tigris storage
- [ ] **Performance Acceptable**: File operations complete within 2x local filesystem latency
- [ ] **Database Functionality**: All Basic Memory features work with Turso remote SQLite
- [ ] **Container Reliability**: Tenant containers start successfully with automatic mounting
- [ ] **Local Access**: Users can mount and edit cloud files locally with real-time sync
- [ ] **Data Isolation**: Tenant data remains properly isolated using bucket/database separation
### Testing Procedure
1. **Local Filesystem Test**:
```bash
# Mount Tigris bucket locally
rclone nfsmount tigris:test-bucket ~/tigris-test --vfs-cache-mode writes
# Run Basic Memory operations
cd ~/tigris-test && basic-memory sync --watch
# Test: create notes, search, file watching, bulk operations
```
2. **Database Migration Test**:
```bash
# Configure Turso connection
export DATABASE_URL="libsql://test-db.turso.io?authToken=..."
# Test all MCP tools with remote database
basic-memory tools # Test each tool functionality
```
3. **Container Integration Test**:
```dockerfile
# Test container with rclone mounting
FROM python:3.12
RUN apt-get update && apt-get install -y rclone nfs-common
# ... test startup and mounting process
```
4. **Performance Benchmarking**:
- File creation/read/write operations (target: <2x local latency)
- Search query performance (target: comparable to local SQLite)
- File watching responsiveness (target: events within 1-2 seconds)
- Concurrent operation handling
### Risk Assessment
**High Risk Items**:
- [ ] NFS-over-S3 performance may be insufficient for real-time operations
- [ ] File watching (`inotify`) over NFS may be unreliable
- [ ] Network interruptions could cause filesystem errors
- [ ] Concurrent access patterns might hit S3 rate limits
**Mitigation Strategies**:
- Comprehensive performance testing before committing to architecture
- Fallback plan to S3-native storage backend if filesystem approach fails
- Extensive error handling and retry logic for network issues
### Metrics to Track
- **Latency**: File operation response times (read/write/watch)
- **Reliability**: Success rate of file operations over time
- **Throughput**: Concurrent file operations and search queries
- **User Experience**: Perceived performance for local mounting use case
## Notes
### Key Architectural Decisions
- **Single tenant per bucket/database**: Simplifies isolation and credential management
- **Maintain POSIX compatibility**: Preserve Basic Memory's existing filesystem assumptions
- **NFS over FUSE**: Better compatibility and performance characteristics
- **Turso for SQLite**: Leverages specialized remote SQLite expertise
### Alternative Approaches Considered
- **S3-native storage backend**: Would require Basic Memory architecture changes
- **Hybrid approach**: Local files + cloud sync (adds complexity)
- **FUSE mounting**: More platform dependencies and kernel requirements
### Integration Points
- [ ] Fly.io Tigris integration for bucket provisioning
- [ ] Turso account setup and database provisioning
- [ ] Container image modifications for rclone support
- [ ] Credential management for tenant isolation
## Observations
- [architecture] Tigris/Turso split cleanly separates file storage from indexing concerns #storage-separation
- [user-experience] Local mounting of cloud files could be revolutionary for knowledge management #local-cloud-hybrid
- [compatibility] Maintaining POSIX filesystem assumptions preserves Basic Memory's local/cloud compatibility #architecture-preservation
- [simplification] Single tenant per bucket eliminates complex multi-tenancy in storage layer #tenant-isolation
- [risk] NFS-over-S3 performance characteristics are unproven for real-time operations #performance-risk
- [benefit] Object storage pricing model could be more favorable than volume pricing #cost-optimization
- [innovation] Real-time local editing of cloud-stored files addresses major SaaS limitation #competitive-advantage
## Relations
- implements [[SPEC-6 Explicit Project Parameter Architecture]]
- requires [[Fly.io Tigris Integration]]
- enables [[Local Cloud File Access]]
- alternative_to [[Fly Volume Storage]]
+886
View File
@@ -0,0 +1,886 @@
---
title: 'SPEC-8: TigrisFS Integration for Tenant API'
Date: September 22, 2025
Status: Phase 3.6 Complete - Tenant Mount API Endpoints Ready for CLI Implementation
Priority: High
Goal: Replace Fly volumes with Tigris bucket provisioning in production tenant API
permalink: spec-8-tigris-fs-integration
---
## Executive Summary
Based on SPEC-7 Phase 4 POC testing, this spec outlines productizing the TigrisFS/rclone implementation in the Basic Memory Cloud tenant API.
We're moving from proof-of-concept to production integration, replacing Fly volume storage with Tigris bucket-per-tenant architecture.
## Current Architecture (Fly Volumes)
### Tenant Provisioning Flow
```python
# apps/cloud/src/basic_memory_cloud/workflows/tenant_provisioning.py
async def provision_tenant_infrastructure(tenant_id: str):
# 1. Create Fly app
# 2. Create Fly volume ← REPLACE THIS
# 3. Deploy API container with volume mount
# 4. Configure health checks
```
### Storage Implementation
- Each tenant gets dedicated Fly volume (1GB-10GB)
- Volume mounted at `/app/data` in API container
- Local filesystem storage with Basic Memory indexing
- No global caching or edge distribution
## Proposed Architecture (Tigris Buckets)
### New Tenant Provisioning Flow
```python
async def provision_tenant_infrastructure(tenant_id: str):
# 1. Create Fly app
# 2. Create Tigris bucket with admin credentials ← NEW
# 3. Store bucket name in tenant record ← NEW
# 4. Deploy API container with TigrisFS mount using admin credentials
# 5. Configure health checks
```
### Storage Implementation
- Each tenant gets dedicated Tigris bucket
- TigrisFS mounts bucket at `/app/data` in API container
- Global edge caching and distribution
- Configurable cache TTL for sync performance
## Implementation Plan
### Phase 1: Bucket Provisioning Service
**✅ IMPLEMENTED: StorageClient with Admin Credentials**
```python
# apps/cloud/src/basic_memory_cloud/clients/storage_client.py
class StorageClient:
async def create_tenant_bucket(self, tenant_id: UUID) -> TigrisBucketCredentials
async def delete_tenant_bucket(self, tenant_id: UUID, bucket_name: str) -> bool
async def list_buckets(self) -> list[TigrisBucketResponse]
async def test_tenant_credentials(self, credentials: TigrisBucketCredentials) -> bool
```
**Simplified Architecture Using Admin Credentials:**
- Single admin access key with full Tigris permissions (configured in console)
- No tenant-specific IAM user creation needed
- Bucket-per-tenant isolation for logical separation
- Admin credentials shared across all tenant operations
**Integrate with Provisioning workflow:**
```python
# Update tenant_provisioning.py
async def provision_tenant_infrastructure(tenant_id: str):
storage_client = StorageClient(settings.aws_access_key_id, settings.aws_secret_access_key)
bucket_creds = await storage_client.create_tenant_bucket(tenant_id)
await store_bucket_name(tenant_id, bucket_creds.bucket_name)
await deploy_api_with_tigris(tenant_id, bucket_creds)
```
### Phase 2: Simplified Bucket Management
**✅ SIMPLIFIED: Admin Credentials + Bucket Names Only**
Since we use admin credentials for all operations, we only need to track bucket names per tenant:
1. **Primary Storage (Fly Secrets)**
```bash
flyctl secrets set -a basic-memory-{tenant_id} \
AWS_ACCESS_KEY_ID="{admin_access_key}" \
AWS_SECRET_ACCESS_KEY="{admin_secret_key}" \
AWS_ENDPOINT_URL_S3="https://fly.storage.tigris.dev" \
AWS_REGION="auto" \
BUCKET_NAME="basic-memory-{tenant_id}"
```
2. **Database Storage (Bucket Name Only)**
```python
# apps/cloud/src/basic_memory_cloud/models/tenant.py
class Tenant(BaseModel):
# ... existing fields
tigris_bucket_name: Optional[str] = None # Just store bucket name
tigris_region: str = "auto"
created_at: datetime
```
**Benefits of Simplified Approach:**
- No credential encryption/decryption needed
- Admin credentials managed centrally in environment
- Only bucket names stored in database (not sensitive)
- Simplified backup/restore scenarios
- Reduced security attack surface
### Phase 3: API Container Updates
**Update API container configuration:**
```dockerfile
# apps/api/Dockerfile
# Add TigrisFS installation
RUN curl -L https://github.com/tigrisdata/tigrisfs/releases/latest/download/tigrisfs-linux-amd64 \
-o /usr/local/bin/tigrisfs && chmod +x /usr/local/bin/tigrisfs
```
**Startup script integration:**
```bash
# apps/api/tigrisfs-startup.sh (already exists)
# Mount TigrisFS → Start Basic Memory API
exec python -m basic_memory_cloud_api.main
```
**Fly.toml environment (optimized for < 5s startup):**
```toml
# apps/api/fly.tigris-production.toml
[env]
TIGRISFS_MEMORY_LIMIT = '1024' # Reduced for faster init
TIGRISFS_MAX_FLUSHERS = '16' # Fewer threads for faster startup
TIGRISFS_STAT_CACHE_TTL = '30s' # Balance sync speed vs startup
TIGRISFS_LAZY_INIT = 'true' # Enable lazy loading
BASIC_MEMORY_HOME = '/app/data'
# Suspend optimization for wake-on-network
[machine]
auto_stop_machines = "suspend" # Faster than full stop
auto_start_machines = true
min_machines_running = 0
```
### Phase 4: Local Access Features
**CLI automation for local mounting:**
```python
# New CLI command: basic-memory cloud mount
async def setup_local_mount(tenant_id: str):
# 1. Fetch bucket credentials from cloud API
# 2. Configure rclone with scoped IAM policy
# 3. Mount via rclone nfsmount (macOS) or FUSE (Linux)
# 4. Start Basic Memory sync watcher
```
**Local mount configuration:**
```bash
# rclone config for tenant
rclone mount basic-memory-{tenant_id}: ~/basic-memory-{tenant_id} \
--nfs-mount \
--vfs-cache-mode writes \
--cache-dir ~/.cache/rclone/basic-memory-{tenant_id}
```
### Phase 5: TigrisFS Cache Sync Solutions
**Problem**: When files are uploaded via CLI/bisync, the tenant API container doesn't see them immediately due to TigrisFS cache (30s TTL) and lack of inotify events on mounted filesystems.
**Multi-Layer Solution:**
**Layer 1: API Sync Endpoint** (Immediate)
```python
# POST /sync - Force TigrisFS cache refresh
# Callable by CLI after uploads
subprocess.run(["sync", "fsync /app/data"], check=True)
```
**Layer 2: Tigris Webhook Integration** (Real-time)
https://www.tigrisdata.com/docs/buckets/object-notifications/#webhook
```python
# Webhook endpoint for bucket changes
@app.post("/webhooks/tigris/{tenant_id}")
async def handle_bucket_notification(tenant_id: str, event: TigrisEvent):
if event.eventName in ["OBJECT_CREATED_PUT", "OBJECT_DELETED"]:
await notify_container_sync(tenant_id, event.object.key)
```
**Layer 3: CLI Sync Notification** (User-triggered)
```bash
# CLI calls container sync endpoint after successful bisync
basic-memory cloud bisync # Automatically notifies container
curl -X POST https://basic-memory-{tenant-id}.fly.dev/sync
```
**Layer 4: Periodic Sync Fallback** (Safety net)
```python
# Background task: fsync /app/data every 30s as fallback
# Ensures eventual consistency even if other layers fail
```
**Implementation Priority:**
1. Layer 1 (API endpoint) - Quick testing capability
2. Layer 3 (CLI integration) - Improved UX
3. Layer 4 (Periodic fallback) - Safety net
4. Layer 2 (Webhooks) - Production real-time sync
## Performance Targets
### Sync Latency
- **Target**: < 5 seconds local→cloud→container
- **Configuration**: `TIGRISFS_STAT_CACHE_TTL = '5s'`
- **Monitoring**: Track sync metrics in production
### Container Startup
- **Target**: < 5 seconds including TigrisFS mount
- **Fast retry**: 0.5s intervals for mount verification
- **Fallback**: Container fails fast if mount fails
### Memory Usage
- **TigrisFS cache**: 2GB memory limit per container
- **Concurrent uploads**: 32 flushers max
- **VM sizing**: shared-cpu-2x (2048mb) minimum
## Security Considerations
### Bucket Isolation
- Each tenant has dedicated bucket
- IAM policies prevent cross-tenant access
- No shared bucket with subdirectories
### Credential Security
- Fly secrets for runtime access
- Encrypted database backup for disaster recovery
- Credential rotation capability
### Data Residency
- Tigris global edge caching
- SOC2 Type II compliance
- Encryption at rest and in transit
## Operational Benefits
### Scalability
- Horizontal scaling with stateless API containers
- Global edge distribution
- Better resource utilization
### Reliability
- No cold starts between tenants
- Built-in redundancy and caching
- Simplified backup strategy
### Cost Efficiency
- Pay-per-use storage pricing
- Shared infrastructure benefits
- Reduced operational overhead
## Risk Mitigation
### Data Loss Prevention
- Dual credential storage (Fly + database)
- Automated backup workflows to R2/S3
- Tigris built-in redundancy
### Performance Degradation
- Configurable cache settings per tenant
- Monitoring and alerting on sync latency
- Fallback to volume storage if needed
### Security Vulnerabilities
- Bucket-per-tenant isolation
- Regular credential rotation
- Security scanning and monitoring
## Success Metrics
### Technical Metrics
- Sync latency P50 < 5 seconds
- Container startup time < 5 seconds
- Zero data loss incidents
- 99.9% uptime per tenant
### Business Metrics
- Reduced infrastructure costs vs volumes
- Improved user experience with faster sync
- Enhanced enterprise security posture
- Simplified operational overhead
## Open Questions
1. **Tigris rate limits**: What are the API limits for bucket creation?
2. **Cost analysis**: What's the break-even point vs Fly volumes?
3. **Regional preferences**: Should enterprise customers choose regions?
4. **Backup retention**: How long to keep automated backups?
## Implementation Checklist
### Phase 1: Bucket Provisioning Service ✅ COMPLETED
- [x] **Research Tigris bucket API** - Document bucket creation and S3 API compatibility
- [x] **Create StorageClient class** - Implemented with admin credentials and comprehensive integration tests
- [x] **Test bucket creation** - Full test suite validates API integration with real Tigris environment
- [x] **Add bucket provisioning to DBOS workflow** - Integrated StorageClient with tenant_provisioning.py
### Phase 2: Simplified Bucket Management ✅ COMPLETED
- [x] **Update Tenant model** with tigris_bucket_name field (replaced fly_volume_id)
- [x] **Implement bucket name storage** - Database migration and model updates completed
- [x] **Test bucket provisioning integration** - Full test suite validates workflow from tenant creation to bucket assignment
- [x] **Remove volume logic from all tests** - Complete migration from volume-based to bucket-based architecture
### Phase 3: API Container Integration ✅ COMPLETED
- [x] **Update Dockerfile** to install TigrisFS binary in API container with configurable version
- [x] **Optimize tigrisfs-startup.sh** with production-ready security and reliability improvements
- [x] **Create production-ready container** with proper signal handling and mount validation
- [x] **Implement security fixes** based on Claude code review (conditional debug, credential protection)
- [x] **Add proper process supervision** with cleanup traps and error handling
- [x] **Remove debug artifacts** - Cleaned up all debug Dockerfiles and test scripts
### Phase 3.5: IAM Access Key Management ✅ COMPLETED
- [x] **Research Tigris IAM API** - Documented create_policy, attach_user_policy, delete_access_key operations
- [x] **Implement bucket-scoped credential generation** - StorageClient.create_tenant_access_keys() with IAM policies
- [x] **Add comprehensive security test suite** - 5 security-focused integration tests covering all attack vectors
- [x] **Verify cross-bucket access prevention** - Scoped credentials can ONLY access their designated bucket
- [x] **Test credential lifecycle management** - Create, validate, delete, and revoke access keys
- [x] **Validate admin vs scoped credential isolation** - Different access patterns and security boundaries
- [x] **Test multi-tenant isolation** - Multiple tenants cannot access each other's buckets
### Phase 3.6: Tenant Mount API Endpoints ✅ COMPLETED
- [x] **Implement GET /tenant/mount/info** - Returns mount info without exposing credentials
- [x] **Implement POST /tenant/mount/credentials** - Creates new bucket-scoped credentials for CLI mounting
- [x] **Implement DELETE /tenant/mount/credentials/{cred_id}** - Revoke specific credentials with proper cleanup
- [x] **Implement GET /tenant/mount/credentials** - List active credentials without exposing secrets
- [x] **Add TenantMountCredentials database model** - Tracks credential metadata (no secret storage)
- [x] **Create comprehensive test suite** - 28 tests covering all scenarios including multi-session support
- [x] **Implement multi-session credential flow** - Multiple active credentials per tenant supported
- [x] **Secure credential handling** - Secret keys never stored, returned once only for immediate use
- [x] **Add dependency injection for StorageClient** - Clean integration with existing API architecture
- [x] **Fix Tigris configuration for cloud service** - Added AWS environment variables to fly.template.toml
- [x] **Update tenant machine configurations** - Include AWS credentials for TigrisFS mounting with clear credential strategy
**Security Test Results:**
```
✅ Cross-bucket access prevention - PASS
✅ Deleted credentials access revoked - PASS
✅ Invalid credentials rejected - PASS
✅ Admin vs scoped credential isolation - PASS
✅ Multiple scoped credentials isolation - PASS
```
**Implementation Details:**
- Uses Tigris IAM managed policies (create_policy + attach_user_policy)
- Bucket-scoped S3 policies with Actions: GetObject, PutObject, DeleteObject, ListBucket
- Resource ARNs limited to specific bucket: `arn:aws:s3:::bucket-name` and `arn:aws:s3:::bucket-name/*`
- Access keys follow Tigris format: `tid_` prefix with secure random suffix
- Complete cleanup on deletion removes both access keys and associated policies
### Phase 4: Local Access CLI
- [x] **Design local mount CLI command** for automated rclone configuration
- [x] **Implement credential fetching** from cloud API for local setup
- [x] **Create rclone config automation** for tenant-specific bucket mounting
- [x] **Test local→cloud→container sync** with optimized cache settings
- [x] **Document local access setup** for beta users
### Phase 5: Webhook Integration (Future)
- [ ] **Research Tigris webhook API** for object notifications and payload format
- [ ] **Design webhook endpoint** for real-time sync notifications
- [ ] **Implement notification handling** to trigger Basic Memory sync events
- [ ] **Test webhook delivery** and sync latency improvements
## Success Metrics
- [ ] **Container startup < 5 seconds** including TigrisFS mount and Basic Memory init
- [ ] **Sync latency < 5 seconds** for local→cloud→container file changes
- [ ] **Zero data loss** during bucket provisioning and credential management
- [ ] **100% test coverage** for new TigrisBucketService and credential functions
- [ ] **Beta deployment** with internal users validating local-cloud workflow
## Implementation Notes
## Phase 4.1: Bidirectional Sync with rclone bisync (NEW)
### Problem Statement
During testing, we discovered that some applications (particularly Obsidian) don't detect file changes over NFS mounts. Rather than building a custom sync daemon, we can leverage `rclone bisync` - rclone's built-in bidirectional synchronization feature.
### Solution: rclone bisync
Use rclone's proven bidirectional sync instead of custom implementation:
**Core Architecture:**
```bash
# rclone bisync handles all the complexity
rclone bisync ~/basic-memory-{tenant_id} basic-memory-{tenant_id}:{bucket_name} \
--create-empty-src-dirs \
--conflict-resolve newer \
--resilient \
--check-access
```
**Key Benefits:**
- ✅ **Battle-tested**: Production-proven rclone functionality
- ✅ **MIT licensed**: Open source with permissive licensing
- ✅ **No custom code**: Zero maintenance burden for sync logic
- ✅ **Built-in safety**: max-delete protection, conflict resolution
- ✅ **Simple installation**: Works with Homebrew rclone (no FUSE needed)
- ✅ **File watcher compatible**: Works with Obsidian and all applications
- ✅ **Offline support**: Can work offline and sync when connected
### bisync Conflict Resolution Options
**Built-in conflict strategies:**
```bash
--conflict-resolve none # Keep both files with .conflict suffixes (safest)
--conflict-resolve newer # Always pick the most recently modified file
--conflict-resolve larger # Choose based on file size
--conflict-resolve path1 # Always prefer local changes
--conflict-resolve path2 # Always prefer cloud changes
```
### Sync Profiles Using bisync
**Profile configurations:**
```python
BISYNC_PROFILES = {
"safe": {
"conflict_resolve": "none", # Keep both versions
"max_delete": 10, # Prevent mass deletion
"check_access": True, # Verify sync integrity
"description": "Safe mode with conflict preservation"
},
"balanced": {
"conflict_resolve": "newer", # Auto-resolve to newer file
"max_delete": 25,
"check_access": True,
"description": "Balanced mode (recommended default)"
},
"fast": {
"conflict_resolve": "newer",
"max_delete": 50,
"check_access": False, # Skip verification for speed
"description": "Fast mode for rapid iteration"
}
}
```
### CLI Commands
**Manual sync commands:**
```bash
basic-memory cloud bisync # Manual bidirectional sync
basic-memory cloud bisync --dry-run # Preview changes
basic-memory cloud bisync --profile safe # Use specific profile
basic-memory cloud bisync --resync # Force full baseline resync
```
**Watch mode (Step 1):**
```bash
basic-memory cloud bisync --watch # Long-running process, sync every 60s
basic-memory cloud bisync --watch --interval 30s # Custom interval
```
**System integration (Step 2 - Future):**
```bash
basic-memory cloud bisync-service install # Install as system service
basic-memory cloud bisync-service start # Start background service
basic-memory cloud bisync-service status # Check service status
```
### Implementation Strategy
**Phase 4.1.1: Core bisync Implementation**
- [ ] Implement `run_bisync()` function wrapping rclone bisync
- [ ] Add profile-based configuration (safe/balanced/fast)
- [ ] Create conflict resolution and safety options
- [ ] Test with sample files and conflict scenarios
**Phase 4.1.2: Watch Mode**
- [ ] Add `--watch` flag for continuous sync
- [ ] Implement configurable sync intervals
- [ ] Add graceful shutdown and signal handling
- [ ] Create status monitoring and progress indicators
**Phase 4.1.3: User Experience**
- [ ] Add conflict reporting and resolution guidance
- [ ] Implement dry-run preview functionality
- [ ] Create troubleshooting and diagnostic commands
- [ ] Add filtering configuration (.gitignore-style)
**Phase 4.1.4: System Integration (Future)**
- [ ] Generate platform-specific service files (launchd/systemd)
- [ ] Add service management commands
- [ ] Implement automatic startup and recovery
- [ ] Create monitoring and logging integration
### Technical Implementation
**Core bisync wrapper:**
```python
def run_bisync(
tenant_id: str,
bucket_name: str,
profile: str = "balanced",
dry_run: bool = False
) -> bool:
"""Run rclone bisync with specified profile."""
local_path = Path.home() / f"basic-memory-{tenant_id}"
remote_path = f"basic-memory-{tenant_id}:{bucket_name}"
profile_config = BISYNC_PROFILES[profile]
cmd = [
"rclone", "bisync",
str(local_path), remote_path,
"--create-empty-src-dirs",
"--resilient",
f"--conflict-resolve={profile_config['conflict_resolve']}",
f"--max-delete={profile_config['max_delete']}",
"--filters-file", "~/.basic-memory/bisync-filters.txt"
]
if profile_config.get("check_access"):
cmd.append("--check-access")
if dry_run:
cmd.append("--dry-run")
return subprocess.run(cmd, check=True).returncode == 0
```
**Default filter file (~/.basic-memory/bisync-filters.txt):**
```
- .DS_Store
- .git/**
- __pycache__/**
- *.pyc
- .pytest_cache/**
- node_modules/**
- .conflict-*
- Thumbs.db
- desktop.ini
```
**Advantages Over Custom Daemon:**
- ✅ **Zero maintenance**: No custom sync logic to debug/maintain
- ✅ **Production proven**: Used by thousands in production
- ✅ **Safety features**: Built-in max-delete, conflict handling, recovery
- ✅ **Filtering**: Advanced exclude patterns and rules
- ✅ **Performance**: Optimized for various storage backends
- ✅ **Community support**: Extensive documentation and community
## Phase 4.2: NFS Mount Support (Direct Access)
### Solution: rclone nfsmount
Keep the existing NFS mount functionality for users who prefer direct file access:
**Core Architecture:**
```bash
# rclone nfsmount provides transparent file access
rclone nfsmount basic-memory-{tenant_id}:{bucket_name} ~/basic-memory-{tenant_id} \
--vfs-cache-mode writes \
--dir-cache-time 10s \
--daemon
```
**Key Benefits:**
- ✅ **Real-time access**: Files appear immediately as they're created/modified
- ✅ **Transparent**: Works with any application that reads/writes files
- ✅ **Low latency**: Direct access without sync delays
- ✅ **Simple**: No periodic sync commands needed
- ✅ **Homebrew compatible**: Works with Homebrew rclone (no FUSE required)
**Limitations:**
- ❌ **File watcher compatibility**: Some apps (Obsidian) don't detect changes over NFS
- ❌ **Network dependency**: Requires active connection to cloud storage
- ❌ **Potential conflicts**: Simultaneous edits from multiple locations can cause issues
### Mount Profiles (Existing)
**Already implemented profiles from SPEC-7 testing:**
```python
MOUNT_PROFILES = {
"fast": {
"cache_time": "5s",
"poll_interval": "3s",
"description": "Ultra-fast development (5s sync)"
},
"balanced": {
"cache_time": "10s",
"poll_interval": "5s",
"description": "Fast development (10-15s sync, recommended)"
},
"safe": {
"cache_time": "15s",
"poll_interval": "10s",
"description": "Conflict-aware mount with backup",
"extra_args": ["--conflict-suffix", ".conflict-{DateTimeExt}"]
}
}
```
### CLI Commands (Existing)
**Mount commands already implemented:**
```bash
basic-memory cloud mount # Mount with balanced profile
basic-memory cloud mount --profile fast # Ultra-fast caching
basic-memory cloud mount --profile safe # Conflict detection
basic-memory cloud unmount # Clean unmount
basic-memory cloud mount-status # Show mount status
```
## User Choice: Mount vs Bisync
### When to Use Each Approach
| Use Case | Recommended Solution | Why |
|----------|---------------------|-----|
| **Obsidian users** | `bisync` | File watcher support for live preview |
| **CLI/vim/emacs users** | `mount` | Direct file access, lower latency |
| **Offline work** | `bisync` | Can work offline, sync when connected |
| **Real-time collaboration** | `mount` | Immediate visibility of changes |
| **Multiple machines** | `bisync` | Better conflict handling |
| **Single machine** | `mount` | Simpler, more transparent |
| **Development work** | Either | Both work well, user preference |
| **Large files** | `mount` | Streaming access vs full download |
### Installation Simplicity
**Both approaches now use simple Homebrew installation:**
```bash
# Single installation command for both approaches
brew install rclone
# No macFUSE, no system modifications needed
# Works immediately with both mount and bisync
```
### Implementation Status
**Phase 4.1: bisync** (NEW)
- [ ] Implement bisync command wrapper
- [ ] Add watch mode with configurable intervals
- [ ] Create conflict resolution workflows
- [ ] Add filtering and safety options
**Phase 4.2: mount** (EXISTING - ✅ IMPLEMENTED)
- [x] NFS mount commands with profile support
- [x] Mount management and cleanup
- [x] Process monitoring and health checks
- [x] Credential integration with cloud API
**Both approaches share:**
- [x] Credential management via cloud API
- [x] Secure rclone configuration
- [x] Tenant isolation and bucket scoping
- [x] Simple Homebrew rclone installation
Key Features:
1. Cross-Platform rclone Installation (rclone_installer.py):
- macOS: Homebrew → official script fallback
- Linux: snap → apt → official script fallback
- Windows: winget → chocolatey → scoop fallback
- Automatic version detection and verification
2. Smart rclone Configuration (rclone_config.py):
- Automatic tenant-specific config generation
- Three optimized mount profiles from your SPEC-7 testing:
- fast: 5s sync (ultra-performance)
- balanced: 10-15s sync (recommended default)
- safe: 15s sync + conflict detection
- Backup existing configs before modification
3. Robust Mount Management (mount_commands.py):
- Automatic tenant credential generation
- Mount path management (~/basic-memory-{tenant-id})
- Process lifecycle management (prevent duplicate mounts)
- Orphaned process cleanup
- Mount verification and health checking
4. Clean Architecture (api_client.py):
- Separated API client to avoid circular imports
- Reuses existing authentication infrastructure
- Consistent error handling and logging
User Experience:
One-Command Setup:
basic-memory cloud setup
```bash
# 1. Installs rclone automatically
# 2. Authenticates with existing login
# 3. Generates secure credentials
# 4. Configures rclone
# 5. Performs initial mount
```
Profile-Based Mounting:
basic-memory cloud mount --profile fast # 5s sync
basic-memory cloud mount --profile balanced # 15s sync (default)
basic-memory cloud mount --profile safe # conflict detection
Status Monitoring:
basic-memory cloud mount-status
```bash
# Shows: tenant info, mount path, sync profile, rclone processes
```
### local mount api
Endpoint 1: Get Tenant Info for user
Purpose: Get tenant details for mounting
- pass in jwt
- service returns mount info
**✅ IMPLEMENTED API Specification:**
**Endpoint 1: GET /tenant/mount/info**
- Purpose: Get tenant mount information without exposing credentials
- Authentication: JWT token (tenant_id extracted from claims)
Request:
```
GET /tenant/mount/info
Authorization: Bearer {jwt_token}
```
Response:
```json
{
"tenant_id": "434252dd-d83b-4b20-bf70-8a950ff875c4",
"bucket_name": "basic-memory-434252dd",
"has_credentials": true,
"credentials_created_at": "2025-09-22T16:48:50.414694"
}
```
**Endpoint 2: POST /tenant/mount/credentials**
- Purpose: Generate NEW bucket-scoped S3 credentials for rclone mounting
- Authentication: JWT token (tenant_id extracted from claims)
- Multi-session: Creates new credentials without revoking existing ones
Request:
```
POST /tenant/mount/credentials
Authorization: Bearer {jwt_token}
Content-Type: application/json
```
*Note: No request body needed - tenant_id extracted from JWT*
Response:
```json
{
"tenant_id": "434252dd-d83b-4b20-bf70-8a950ff875c4",
"bucket_name": "basic-memory-434252dd",
"access_key": "test_access_key_12345",
"secret_key": "test_secret_key_abcdef",
"endpoint_url": "https://fly.storage.tigris.dev",
"region": "auto"
}
```
**🔒 Security Notes:**
- Secret key returned ONCE only - never stored in database
- Credentials are bucket-scoped (cannot access other tenants' buckets)
- Multiple active credentials supported per tenant (work laptop + personal machine)
Implementation Notes
Security:
- Both endpoints require JWT authentication
- Extract tenant_id from JWT claims (not request body)
- Generate scoped credentials (not admin credentials)
- Credentials should have bucket-specific access only
Integration Points:
- Use your existing StorageClient from SPEC-8 implementation
- Leverage existing JWT middleware for tenant extraction
- Return same credential format as your Tigris bucket provisioning
Error Handling:
- 401 if not authenticated
- 403 if tenant doesn't exist
- 500 if credential generation fails
**🔄 Design Decisions:**
1. **Secure Credential Flow (No Secret Storage)**
Based on CLI flow analysis, we follow security best practices:
- ✅ API generates both access_key + secret_key via Tigris IAM
- ✅ Returns both in API response for immediate use
- ✅ CLI uses credentials immediately to configure rclone
- ✅ Database stores only metadata (access_key + policy_arn for cleanup)
- ✅ rclone handles secure local credential storage
- ❌ **Never store secret_key in database (even encrypted)**
2. **CLI Credential Flow**
```bash
# CLI calls API
POST /tenant/mount/credentials → {access_key, secret_key, ...}
# CLI immediately configures rclone
rclone config create basic-memory-{tenant_id} s3 \
access_key_id={access_key} \
secret_access_key={secret_key} \
endpoint=https://fly.storage.tigris.dev
# Database tracks metadata only
INSERT INTO tenant_mount_credentials (tenant_id, access_key, policy_arn, ...)
```
3. **Multiple Sessions Supported**
- Users can have multiple active credential sets (work laptop, personal machine, etc.)
- Each credential generation creates a new Tigris access key
- List active credentials via API (shows access_key but never secret)
4. **Failure Handling & Cleanup**
- **Happy Path**: Credentials created → Used immediately → rclone configured
- **Orphaned Credentials**: Background job revokes unused credentials
- **API Failure Recovery**: Retry Tigris deletion with stored policy_arn
- **Status Tracking**: Track tigris_deletion_status (pending/completed/failed)
5. **Event Sourcing & Audit**
- MountCredentialCreatedEvent
- MountCredentialRevokedEvent
- MountCredentialOrphanedEvent (for cleanup)
- Full audit trail for security compliance
6. **Tenant/Bucket Validation**
- Verify tenant exists and has valid bucket before credential generation
- Use existing StorageClient to validate bucket access
- Prevent credential generation for inactive/invalid tenants
📋 **Implemented API Endpoints:**
```
✅ IMPLEMENTED:
GET /tenant/mount/info # Get tenant/bucket info (no credentials exposed)
POST /tenant/mount/credentials # Generate new credentials (returns secret once)
GET /tenant/mount/credentials # List active credentials (no secrets)
DELETE /tenant/mount/credentials/{cred_id} # Revoke specific credentials
```
**API Implementation Status:**
- ✅ **GET /tenant/mount/info**: Returns tenant_id, bucket_name, has_credentials, credentials_created_at
- ✅ **POST /tenant/mount/credentials**: Creates new bucket-scoped access keys, returns access_key + secret_key once
- ✅ **GET /tenant/mount/credentials**: Lists active credentials without exposing secret keys
- ✅ **DELETE /tenant/mount/credentials/{cred_id}**: Revokes specific credentials with proper Tigris IAM cleanup
- ✅ **Multi-session support**: Multiple active credentials per tenant (work laptop + personal machine)
- ✅ **Security**: Secret keys never stored in database, returned once only for immediate use
- ✅ **Comprehensive test suite**: 28 tests covering all scenarios including error handling and multi-session flows
- ✅ **Dependency injection**: Clean integration with existing FastAPI architecture
- ✅ **Production-ready configuration**: Tigris credentials properly configured for tenant machines
🗄️ **Secure Database Schema:**
```sql
CREATE TABLE tenant_mount_credentials (
id UUID PRIMARY KEY,
tenant_id UUID REFERENCES tenant(id),
access_key VARCHAR(255) NOT NULL,
-- secret_key REMOVED - never store secrets (security best practice)
policy_arn VARCHAR(255) NOT NULL, -- For Tigris IAM cleanup
tigris_deletion_status VARCHAR(20) DEFAULT 'pending', -- Track cleanup
created_at TIMESTAMP DEFAULT NOW(),
updated_at TIMESTAMP DEFAULT NOW(),
revoked_at TIMESTAMP NULL,
last_used_at TIMESTAMP NULL, -- Track usage for orphan cleanup
description VARCHAR(255) DEFAULT 'CLI mount credentials'
);
```
**Security Benefits:**
- ✅ Database breach cannot expose secrets
- ✅ Follows "secrets don't persist" security principle
- ✅ Meets compliance requirements (SOC2, etc.)
- ✅ Reduced attack surface
- ✅ CLI gets credentials once and stores securely via rclone
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,390 @@
---
title: 'SPEC-9-1 Follow-Ups: Conflict, Sync, and Observability'
type: tasklist
permalink: specs/spec-9-follow-ups-conflict-sync-and-observability
related: specs/spec-9-multi-project-bisync
status: revised
revision_date: 2025-10-03
---
# SPEC-9-1 Follow-Ups: Conflict, Sync, and Observability
**REVISED 2025-10-03:** Simplified to leverage rclone built-ins instead of custom conflict handling.
**Context:** SPEC-9 delivered multi-project bidirectional sync and a unified CLI. This follow-up focuses on **observability and safety** using rclone's built-in capabilities rather than reinventing conflict handling.
**Design Philosophy: "Be Dumb Like Git"**
- Let rclone bisync handle conflict detection (it already does this)
- Make conflicts visible and recoverable, don't prevent them
- Cloud is always the winner on conflict (cloud-primary model)
- Users who want version history can just use Git locally in their sync directory
**What Changed from Original Version:**
- **Replaced:** Custom `.bmmeta` sidecars → Use rclone's `.bisync/` state tracking
- **Replaced:** Custom conflict detection → Use rclone bisync 3-way merge
- **Replaced:** Tombstone files → rclone delete tracking handles this
- **Replaced:** Distributed lease → Local process lock only (document multi-device warning)
- **Replaced:** S3 versioning service → Users just use Git locally if they want history
- **Deferred:** SPEC-14 Git integration → Postponed to teams/multi-user features
## ✅ Now
- [ ] **Local process lock**: Prevent concurrent bisync runs on same device (`~/.basic-memory/sync.lock`)
- [ ] **Structured sync reports**: Parse rclone bisync output into JSON reports (creates/updates/deletes/conflicts, bytes, duration); `bm sync --report`
- [ ] **Multi-device warning**: Document that users should not run `--watch` on multiple devices simultaneously
- [ ] **Version control guidance**: Document pattern for users to use Git locally in their sync directory if they want version history
- [ ] **Docs polish**: cloud-mode toggle, mount↔bisync directory isolation, conflict semantics, quick start, migration guide, short demo clip/GIF
## 🔜 Next
- [ ] **Observability commands**: `bm conflicts list`, `bm sync history` to view sync reports and conflicts
- [ ] **Conflict resolution UI**: `bm conflicts resolve <file>` to interactively pick winner from conflict files
- [ ] **Selective sync**: allow include/exclude by project; per-project profile (safe/balanced/fast)
## 🧭 Later
- [ ] **Near real-time sync**: File watcher → targeted `rclone copy` for individual files (keep bisync as backstop)
- [ ] **Sharing / scoped tokens**: cross-tenant/project access
- [ ] **Bandwidth controls & backpressure**: policy for large repos
- [ ] **Client-side encryption (optional)**: with clear trade-offs
## 📏 Acceptance criteria (for "Now" items)
- [ ] Local process lock prevents concurrent bisync runs on same device
- [ ] rclone bisync conflict files visible and documented (`file.conflict1.md`, `file.conflict2.md`)
- [ ] `bm sync --report` generates parsable JSON with sync statistics
- [ ] Documentation clearly warns about multi-device `--watch` mode
- [ ] Documentation shows users how to use Git locally for version history
## What We're NOT Building (Deferred to rclone)
- ❌ Custom `.bmmeta` sidecars (rclone tracks state in `.bisync/` workdir)
- ❌ Custom conflict detection (rclone bisync already does 3-way merge detection)
- ❌ Tombstone files (S3 versioning + rclone delete tracking handles this)
- ❌ Distributed lease (low probability issue, rclone detects state divergence)
- ❌ Rename/move tracking (rclone has size+modtime heuristics built-in)
## Implementation Summary
**Current State (SPEC-9):**
- ✅ rclone bisync with 3 profiles (safe/balanced/fast)
-`--max-delete` safety limits (10/25/50 files)
-`--conflict-resolve=newer` for auto-resolution
- ✅ Watch mode: `bm sync --watch` (60s intervals)
- ✅ Integrity checking: `bm cloud check`
- ✅ Mount vs bisync directory isolation
**What's Needed (This Spec):**
1. **Process lock** - Simple file-based lock in `~/.basic-memory/sync.lock`
2. **Sync reports** - Parse rclone output, save to `~/.basic-memory/sync-history/`
3. **Documentation** - Multi-device warnings, conflict resolution workflow, Git usage pattern
**User Model:**
- Cloud is always the winner on conflict (cloud-primary)
- rclone creates `.conflict` files for divergent edits
- Users who want version history just use Git in their local sync directory
- Users warned: don't run `--watch` on multiple devices
## Decision Rationale & Trade-offs
### Why Trust rclone Instead of Custom Conflict Handling?
**rclone bisync already provides:**
- 3-way merge detection (compares local, remote, and last-known state)
- File state tracking in `.bisync/` workdir (hashes, modtimes)
- Automatic conflict file creation: `file.conflict1.md`, `file.conflict2.md`
- Rename detection via size+modtime heuristics
- Delete tracking (prevents resurrection of deleted files)
- Battle-tested with extensive edge case handling
**What we'd have to build with custom approach:**
- Per-file metadata tracking (`.bmmeta` sidecars)
- 3-way diff algorithm
- Conflict detection logic
- Tombstone files for deletes
- Rename/move detection
- Testing for all edge cases
**Decision:** Use what rclone already does well. Don't reinvent the wheel.
### Why Let Users Use Git Locally Instead of Building Versioning?
**The simplest solution: Just use Git**
Users who want version history can literally just use Git in their sync directory:
```bash
cd ~/basic-memory-cloud-sync/
git init
git add .
git commit -m "backup"
# Push to their own GitHub if they want
git remote add origin git@github.com:user/my-knowledge.git
git push
```
**Why this is perfect:**
- ✅ We build nothing
- ✅ Users who want Git... just use Git
- ✅ Users who don't care... don't need to
- ✅ rclone bisync already handles sync conflicts
- ✅ Users own their data, they can version it however they want (Git, Time Machine, etc.)
**What we'd have to build for S3 versioning:**
- API to enable versioning on Tigris buckets
- **Problem**: Tigris doesn't support S3 bucket versioning
- Restore commands: `bm cloud restore --version-id`
- Version listing: `bm cloud versions <path>`
- Lifecycle policies for version retention
- Documentation and user education
**What we'd have to build for SPEC-14 Git integration:**
- Committer service (daemon watching `/app/data/`)
- Puller service (webhook handler for GitHub pushes)
- Git LFS for large files
- Loop prevention between Git ↔ bisync ↔ local
- Merge conflict handling at TWO layers (rclone + Git)
- Webhook infrastructure and monitoring
**Decision:** Don't build version control. Document the pattern. "The easiest problem to solve is the one you avoid."
**When to revisit:** Teams/multi-user features where server-side version control becomes necessary for collaboration.
### Why No Distributed Lease?
**Low probability issue:**
- Requires user to manually run `bm sync` on multiple devices at exact same time
- Most users run `--watch` on one primary device
- rclone bisync detects state divergence and fails safely
**Safety nets in place:**
- Local process lock prevents concurrent runs on same device
- rclone bisync aborts if bucket state changed during sync
- S3 versioning recovers from any overwrites
- Documentation warns against multi-device `--watch`
**Failure mode:**
```bash
# Device A and B sync simultaneously
Device A: bm sync → succeeds
Device B: bm sync → "Error: path has changed, run --resync"
# User fixes with resync
Device B: bm sync --resync → establishes new baseline
```
**Decision:** Document the issue, add local lock, defer distributed coordination until users report actual problems.
### Cloud-Primary Conflict Model
**User mental model:**
- Cloud is the source of truth (like Dropbox/iCloud)
- Local is working copy
- On conflict: cloud wins, local edits → `.conflict` file
- User manually picks winner
**Why this works:**
- Simpler than bidirectional merge (no automatic resolution risk)
- Matches user expectations from Dropbox
- S3 versioning provides safety net for overwrites
- Clear recovery path: restore from S3 version if needed
**Example workflow:**
```bash
# Edit file on Device A and Device B while offline
# Both devices come online and sync
Device A: bm sync
# → Pushes to cloud first, becomes canonical version
Device B: bm sync
# → Detects conflict
# → Cloud version: work/notes.md
# → Local version: work/notes.md.conflict1
# → User manually merges or picks winner
# Restore if needed
bm cloud restore work/notes.md --version-id abc123
```
## Implementation Details
### 1. Local Process Lock
```python
# ~/.basic-memory/sync.lock
import os
import psutil
from pathlib import Path
class SyncLock:
def __init__(self):
self.lock_file = Path.home() / '.basic-memory' / 'sync.lock'
def acquire(self):
if self.lock_file.exists():
pid = int(self.lock_file.read_text())
if psutil.pid_exists(pid):
raise BisyncError(
f"Sync already running (PID {pid}). "
f"Wait for completion or kill stale process."
)
# Stale lock, remove it
self.lock_file.unlink()
self.lock_file.write_text(str(os.getpid()))
def release(self):
if self.lock_file.exists():
self.lock_file.unlink()
def __enter__(self):
self.acquire()
return self
def __exit__(self, *args):
self.release()
# Usage
with SyncLock():
run_rclone_bisync()
```
### 3. Sync Report Parsing
```python
# Parse rclone bisync output
import json
from datetime import datetime
from pathlib import Path
def parse_sync_report(rclone_output: str, duration: float, exit_code: int) -> dict:
"""Parse rclone bisync output into structured report."""
# rclone bisync outputs lines like:
# "Synching Path1 /local/path with Path2 remote:bucket"
# "- Path1 File was copied to Path2"
# "Bisync successful"
report = {
"timestamp": datetime.now().isoformat(),
"duration_seconds": duration,
"exit_code": exit_code,
"success": exit_code == 0,
"files_created": 0,
"files_updated": 0,
"files_deleted": 0,
"conflicts": [],
"errors": []
}
for line in rclone_output.split('\n'):
if 'was copied to' in line:
report['files_created'] += 1
elif 'was updated in' in line:
report['files_updated'] += 1
elif 'was deleted from' in line:
report['files_deleted'] += 1
elif '.conflict' in line:
report['conflicts'].append(line.strip())
elif 'ERROR' in line:
report['errors'].append(line.strip())
return report
def save_sync_report(report: dict):
"""Save sync report to history."""
history_dir = Path.home() / '.basic-memory' / 'sync-history'
history_dir.mkdir(parents=True, exist_ok=True)
timestamp = datetime.now().strftime('%Y%m%d-%H%M%S')
report_file = history_dir / f'{timestamp}.json'
report_file.write_text(json.dumps(report, indent=2))
# Usage in run_bisync()
start_time = time.time()
result = subprocess.run(bisync_cmd, capture_output=True, text=True)
duration = time.time() - start_time
report = parse_sync_report(result.stdout, duration, result.returncode)
save_sync_report(report)
if report['conflicts']:
console.print(f"[yellow]⚠ {len(report['conflicts'])} conflict(s) detected[/yellow]")
console.print("[dim]Run 'bm conflicts list' to view[/dim]")
```
### 4. User Commands
```bash
# View sync history
bm sync history
# → Lists recent syncs from ~/.basic-memory/sync-history/*.json
# → Shows: timestamp, duration, files changed, conflicts, errors
# View current conflicts
bm conflicts list
# → Scans sync directory for *.conflict* files
# → Shows: file path, conflict versions, timestamps
# Restore from S3 version
bm cloud restore work/notes.md --version-id abc123
# → Uses aws s3api get-object with version-id
# → Downloads to original path
bm cloud restore work/notes.md --timestamp "2025-10-03 14:30"
# → Lists versions, finds closest to timestamp
# → Downloads that version
# List file versions
bm cloud versions work/notes.md
# → Uses aws s3api list-object-versions
# → Shows: version-id, timestamp, size, author
# Interactive conflict resolution
bm conflicts resolve work/notes.md
# → Shows both versions side-by-side
# → Prompts: Keep local, keep cloud, merge manually, restore from S3 version
# → Cleans up .conflict files after resolution
```
## Success Metrics & Monitoring
**Phase 1 (v1) - Basic Safety:**
- [ ] Conflict detection rate < 5% of syncs (measure in telemetry)
- [ ] User can resolve conflicts within 5 minutes (UX testing)
- [ ] Documentation prevents 90% of multi-device issues
**Phase 2 (v2) - Observability:**
- [ ] 80% of users check `bm sync history` when troubleshooting
- [ ] Average time to restore from S3 version < 2 minutes
-
- [ ] Conflict resolution success rate > 95%
**What to measure:**
```python
# Telemetry in sync reports
{
"conflict_rate": conflicts / total_syncs,
"multi_device_collisions": count_state_divergence_errors,
"version_restores": count_restore_operations,
"avg_sync_duration": sum(durations) / count,
"max_delete_trips": count_max_delete_aborts
}
```
**When to add distributed lease:**
- Multi-device collision rate > 5% of syncs
- User complaints about state divergence errors
- Evidence that local lock isn't sufficient
**When to revisit Git (SPEC-14):**
- Teams feature launches (multi-user collaboration)
- Users request commit messages / audit trail
- PR-based review workflow becomes valuable
## Links
- SPEC-9: `specs/spec-9-multi-project-bisync`
- SPEC-14: `specs/spec-14-cloud-git-versioning` (deferred in favor of S3 versioning)
- rclone bisync docs: https://rclone.org/bisync/
- Tigris S3 versioning: https://www.tigrisdata.com/docs/buckets/versioning/
---
**Owner:** <assign> | **Review cadence:** weekly in standup | **Last updated:** 2025-10-03
+1 -1
View File
@@ -1,7 +1,7 @@
"""basic-memory - Local-first knowledge management combining Zettelkasten with knowledge graphs"""
# Package version - updated by release automation
__version__ = "0.14.0b1"
__version__ = "0.15.0"
# API version for FastAPI - independent of package version
__api_version__ = "v0"
+3 -1
View File
@@ -8,17 +8,19 @@ from sqlalchemy import pool
from alembic import context
from basic_memory.config import ConfigManager
# set config.env to "test" for pytest to prevent logging to file in utils.setup_logging()
os.environ["BASIC_MEMORY_ENV"] = "test"
# Import after setting environment variable # noqa: E402
from basic_memory.config import app_config # noqa: E402
from basic_memory.models import Base # noqa: E402
# this is the Alembic Config object, which provides
# access to the values within the .ini file in use.
config = context.config
app_config = ConfigManager().config
# Set the SQLAlchemy URL from our app config
sqlalchemy_url = f"sqlite:///{app_config.database_path}"
config.set_main_option("sqlalchemy.url", sqlalchemy_url)
@@ -0,0 +1,49 @@
"""fix project foreign keys
Revision ID: a1b2c3d4e5f6
Revises: 647e7a75e2cd
Create Date: 2025-08-19 22:06:00.000000
"""
from typing import Sequence, Union
from alembic import op
# revision identifiers, used by Alembic.
revision: str = "a1b2c3d4e5f6"
down_revision: Union[str, None] = "647e7a75e2cd"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
"""Re-establish foreign key constraints that were lost during project table recreation.
The migration 647e7a75e2cd recreated the project table but did not re-establish
the foreign key constraint from entity.project_id to project.id, causing
foreign key constraint failures when trying to delete projects with related entities.
"""
# SQLite doesn't allow adding foreign key constraints to existing tables easily
# We need to be careful and handle the case where the constraint might already exist
with op.batch_alter_table("entity", schema=None) as batch_op:
# Try to drop existing foreign key constraint (may not exist)
try:
batch_op.drop_constraint("fk_entity_project_id", type_="foreignkey")
except Exception:
# Constraint may not exist, which is fine - we'll create it next
pass
# Add the foreign key constraint with CASCADE DELETE
# This ensures that when a project is deleted, all related entities are also deleted
batch_op.create_foreign_key(
"fk_entity_project_id", "project", ["project_id"], ["id"], ondelete="CASCADE"
)
def downgrade() -> None:
"""Remove the foreign key constraint."""
with op.batch_alter_table("entity", schema=None) as batch_op:
batch_op.drop_constraint("fk_entity_project_id", type_="foreignkey")
+13 -4
View File
@@ -20,17 +20,26 @@ from basic_memory.api.routers import (
search,
prompt_router,
)
from basic_memory.config import app_config
from basic_memory.services.initialization import initialize_app, initialize_file_sync
from basic_memory.config import ConfigManager
from basic_memory.services.initialization import initialize_file_sync, initialize_app
@asynccontextmanager
async def lifespan(app: FastAPI): # pragma: no cover
"""Lifecycle manager for the FastAPI app."""
# Initialize app and database
"""Lifecycle manager for the FastAPI app. Not called in stdio mcp mode"""
app_config = ConfigManager().config
logger.info("Starting Basic Memory API")
await initialize_app(app_config)
# Cache database connections in app state for performance
logger.info("Initializing database and caching connections...")
engine, session_maker = await db.get_or_create_db(app_config.database_path)
app.state.engine = engine
app.state.session_maker = session_maker
logger.info("Database connections cached in app state")
logger.info(f"Sync changes enabled: {app_config.sync_changes}")
if app_config.sync_changes:
# start file sync task in background
@@ -27,6 +27,26 @@ from basic_memory.schemas.base import Permalink, Entity
router = APIRouter(prefix="/knowledge", tags=["knowledge"])
async def resolve_relations_background(sync_service, entity_id: int, entity_permalink: str) -> None:
"""Background task to resolve relations for a specific entity.
This runs asynchronously after the API response is sent, preventing
long delays when creating entities with many relations.
"""
try:
# Only resolve relations for the newly created entity
await sync_service.resolve_relations(entity_id=entity_id)
logger.debug(
f"Background: Resolved relations for entity {entity_permalink} (id={entity_id})"
)
except Exception as e:
# Log but don't fail - this is a background task
logger.warning(
f"Background: Failed to resolve relations for entity {entity_permalink}: {e}"
)
## Create endpoints
@@ -88,15 +108,12 @@ async def create_or_update_entity(
# reindex
await search_service.index_entity(entity, background_tasks=background_tasks)
# Attempt immediate relation resolution when creating new entities
# This helps resolve forward references when related entities are created in the same session
# Schedule relation resolution as a background task for new entities
# This prevents blocking the API response while resolving potentially many relations
if created:
try:
await sync_service.resolve_relations()
logger.debug(f"Resolved relations after creating entity: {entity.permalink}")
except Exception as e: # pragma: no cover
# Don't fail the entire request if relation resolution fails
logger.warning(f"Failed to resolve relations after entity creation: {e}")
background_tasks.add_task(
resolve_relations_background, sync_service, entity.id, entity.permalink or ""
)
result = EntityResponse.model_validate(entity)
@@ -6,7 +6,7 @@ from fastapi import APIRouter, Request
from loguru import logger
from pydantic import BaseModel
from basic_memory.config import app_config
from basic_memory.config import ConfigManager
from basic_memory.deps import SyncServiceDep, ProjectRepositoryDep
router = APIRouter(prefix="/management", tags=["management"])
@@ -41,6 +41,8 @@ async def start_watch_service(
# Watch service is already running
return WatchStatusResponse(running=True)
app_config = ConfigManager().config
# Create and start a new watch service
logger.info("Starting watch service via management API")
+120 -17
View File
@@ -1,10 +1,17 @@
"""Router for project management."""
from fastapi import APIRouter, HTTPException, Path, Body
import os
from fastapi import APIRouter, HTTPException, Path, Body, BackgroundTasks
from typing import Optional
from loguru import logger
from basic_memory.deps import ProjectServiceDep, ProjectPathDep
from basic_memory.schemas import ProjectInfoResponse
from basic_memory.deps import (
ProjectConfigDep,
ProjectServiceDep,
ProjectPathDep,
SyncServiceDep,
)
from basic_memory.schemas import ProjectInfoResponse, SyncReportResponse
from basic_memory.schemas.project_info import (
ProjectList,
ProjectItem,
@@ -13,6 +20,7 @@ from basic_memory.schemas.project_info import (
)
# Router for resources in a specific project
# The ProjectPathDep is used in the path as a prefix, so the request path is like /{project}/project/info
project_router = APIRouter(prefix="/project", tags=["project"])
# Router for managing project resources
@@ -28,47 +36,121 @@ async def get_project_info(
return await project_service.get_project_info(project)
@project_router.get("/item", response_model=ProjectItem)
async def get_project(
project_service: ProjectServiceDep,
project: ProjectPathDep,
) -> ProjectItem:
"""Get bassic info about the specified Basic Memory project."""
found_project = await project_service.get_project(project)
if not found_project:
raise HTTPException(
status_code=404, detail=f"Project: '{project}' does not exist"
) # pragma: no cover
return ProjectItem(
name=found_project.name,
path=found_project.path,
is_default=found_project.is_default or False,
)
# Update a project
@project_router.patch("/{name}", response_model=ProjectStatusResponse)
async def update_project(
project_service: ProjectServiceDep,
project_name: str = Path(..., description="Name of the project to update"),
path: Optional[str] = Body(None, description="New path for the project"),
name: str = Path(..., description="Name of the project to update"),
path: Optional[str] = Body(None, description="New absolute path for the project"),
is_active: Optional[bool] = Body(None, description="Status of the project (active/inactive)"),
) -> ProjectStatusResponse:
"""Update a project's information in configuration and database.
Args:
project_name: The name of the project to update
path: Optional new path for the project
name: The name of the project to update
path: Optional new absolute path for the project
is_active: Optional status update for the project
Returns:
Response confirming the project was updated
"""
try: # pragma: no cover
try:
# Validate that path is absolute if provided
if path and not os.path.isabs(path):
raise HTTPException(status_code=400, detail="Path must be absolute")
# Get original project info for the response
old_project_info = ProjectItem(
name=project_name,
path=project_service.projects.get(project_name, ""),
name=name,
path=project_service.projects.get(name, ""),
)
await project_service.update_project(project_name, updated_path=path, is_active=is_active)
if path:
await project_service.move_project(name, path)
elif is_active is not None:
await project_service.update_project(name, is_active=is_active)
# Get updated project info
updated_path = path if path else project_service.projects.get(project_name, "")
updated_path = path if path else project_service.projects.get(name, "")
return ProjectStatusResponse(
message=f"Project '{project_name}' updated successfully",
message=f"Project '{name}' updated successfully",
status="success",
default=(project_name == project_service.default_project),
default=(name == project_service.default_project),
old_project=old_project_info,
new_project=ProjectItem(name=project_name, path=updated_path),
new_project=ProjectItem(name=name, path=updated_path),
)
except ValueError as e: # pragma: no cover
except ValueError as e:
raise HTTPException(status_code=400, detail=str(e))
# Sync project filesystem
@project_router.post("/sync")
async def sync_project(
background_tasks: BackgroundTasks,
sync_service: SyncServiceDep,
project_config: ProjectConfigDep,
):
"""Force project filesystem sync to database.
Scans the project directory and updates the database with any new or modified files.
Args:
background_tasks: FastAPI background tasks
sync_service: Sync service for this project
project_config: Project configuration
Returns:
Response confirming sync was initiated
"""
background_tasks.add_task(sync_service.sync, project_config.home, project_config.name)
logger.info(f"Filesystem sync initiated for project: {project_config.name}")
return {
"status": "sync_started",
"message": f"Filesystem sync initiated for project '{project_config.name}'",
}
@project_router.post("/status", response_model=SyncReportResponse)
async def project_sync_status(
sync_service: SyncServiceDep,
project_config: ProjectConfigDep,
) -> SyncReportResponse:
"""Scan directory for changes compared to database state.
Args:
sync_service: Sync service for this project
project_config: Project configuration
Returns:
Scan report with details on files that need syncing
"""
logger.info(f"Scanning filesystem for project: {project_config.name}")
sync_report = await sync_service.scan(project_config.home)
return SyncReportResponse.from_sync_report(sync_report)
# List all available projects
@project_resource_router.get("/projects", response_model=ProjectList)
async def list_projects(
@@ -209,8 +291,29 @@ async def set_default_project(
raise HTTPException(status_code=400, detail=str(e))
# Get the default project
@project_resource_router.get("/default", response_model=ProjectItem)
async def get_default_project(
project_service: ProjectServiceDep,
) -> ProjectItem:
"""Get the default project.
Returns:
Response with project default information
"""
# Get the old default project
default_name = project_service.default_project
default_project = await project_service.get_project(default_name)
if not default_project: # pragma: no cover
raise HTTPException( # pragma: no cover
status_code=404, detail=f"Default Project: '{default_name}' does not exist"
)
return ProjectItem(name=default_project.name, path=default_project.path, is_default=True)
# Synchronize projects between config and database
@project_resource_router.post("/sync", response_model=ProjectStatusResponse)
@project_resource_router.post("/config/sync", response_model=ProjectStatusResponse)
async def synchronize_projects(
project_service: ProjectServiceDep,
) -> ProjectStatusResponse:
@@ -188,7 +188,7 @@ async def write_resource(
"content_type": content_type,
"file_path": file_path,
"checksum": checksum,
"updated_at": datetime.fromtimestamp(file_stats.st_mtime),
"updated_at": datetime.fromtimestamp(file_stats.st_mtime).astimezone(),
},
)
status_code = 200
@@ -200,8 +200,8 @@ async def write_resource(
content_type=content_type,
file_path=file_path,
checksum=checksum,
created_at=datetime.fromtimestamp(file_stats.st_ctime),
updated_at=datetime.fromtimestamp(file_stats.st_mtime),
created_at=datetime.fromtimestamp(file_stats.st_ctime).astimezone(),
updated_at=datetime.fromtimestamp(file_stats.st_mtime).astimezone(),
)
entity = await entity_repository.add(entity)
status_code = 201
+10 -29
View File
@@ -2,19 +2,15 @@ from typing import Optional
import typer
from basic_memory.config import get_project_config
from basic_memory.mcp.project_session import session
from basic_memory.config import ConfigManager
def version_callback(value: bool) -> None:
"""Show version and exit."""
if value: # pragma: no cover
import basic_memory
from basic_memory.config import config
typer.echo(f"Basic Memory version: {basic_memory.__version__}")
typer.echo(f"Current project: {config.project}")
typer.echo(f"Project path: {config.home}")
raise typer.Exit()
@@ -24,13 +20,6 @@ app = typer.Typer(name="basic-memory")
@app.callback()
def app_callback(
ctx: typer.Context,
project: Optional[str] = typer.Option(
None,
"--project",
"-p",
help="Specify which project to use 1",
envvar="BASIC_MEMORY_PROJECT",
),
version: Optional[bool] = typer.Option(
None,
"--version",
@@ -44,30 +33,22 @@ def app_callback(
# Run initialization for every command unless --version was specified
if not version and ctx.invoked_subcommand is not None:
from basic_memory.config import app_config
from basic_memory.services.initialization import ensure_initialization
app_config = ConfigManager().config
ensure_initialization(app_config)
# Initialize MCP session with the specified project or default
if project: # pragma: no cover
# Use the project specified via --project flag
current_project_config = get_project_config(project)
session.set_current_project(current_project_config.name)
# Update the global config to use this project
from basic_memory.config import update_current_project
update_current_project(project)
else:
# Use the default project
current_project = app_config.default_project
session.set_current_project(current_project)
## import
# Register sub-command groups
import_app = typer.Typer(help="Import data from various sources")
app.add_typer(import_app, name="import")
claude_app = typer.Typer()
claude_app = typer.Typer(help="Import Conversations from Claude JSON export.")
import_app.add_typer(claude_app, name="claude")
## cloud
cloud_app = typer.Typer(help="Access Basic Memory Cloud")
app.add_typer(cloud_app, name="cloud")
+277
View File
@@ -0,0 +1,277 @@
"""WorkOS OAuth Device Authorization for CLI."""
import base64
import hashlib
import json
import os
import secrets
import time
import webbrowser
import httpx
from rich.console import Console
from basic_memory.config import ConfigManager
console = Console()
class CLIAuth:
"""Handles WorkOS OAuth Device Authorization for CLI tools."""
def __init__(self, client_id: str, authkit_domain: str):
self.client_id = client_id
self.authkit_domain = authkit_domain
app_config = ConfigManager().config
# Store tokens in data dir
self.token_file = app_config.data_dir_path / "basic-memory-cloud.json"
# PKCE parameters
self.code_verifier = None
self.code_challenge = None
def generate_pkce_pair(self) -> tuple[str, str]:
"""Generate PKCE code verifier and challenge."""
# Generate code verifier (43-128 characters)
code_verifier = base64.urlsafe_b64encode(secrets.token_bytes(32)).decode("utf-8")
code_verifier = code_verifier.rstrip("=")
# Generate code challenge (SHA256 hash of verifier)
challenge_bytes = hashlib.sha256(code_verifier.encode("utf-8")).digest()
code_challenge = base64.urlsafe_b64encode(challenge_bytes).decode("utf-8")
code_challenge = code_challenge.rstrip("=")
return code_verifier, code_challenge
async def request_device_authorization(self) -> dict | None:
"""Request device authorization from WorkOS with PKCE."""
device_auth_url = f"{self.authkit_domain}/oauth2/device_authorization"
# Generate PKCE pair
self.code_verifier, self.code_challenge = self.generate_pkce_pair()
data = {
"client_id": self.client_id,
"scope": "openid profile email offline_access",
"code_challenge": self.code_challenge,
"code_challenge_method": "S256",
}
try:
async with httpx.AsyncClient() as client:
response = await client.post(device_auth_url, data=data)
if response.status_code == 200:
return response.json()
else:
console.print(
f"[red]Device authorization failed: {response.status_code} - {response.text}[/red]"
)
return None
except Exception as e:
console.print(f"[red]Device authorization error: {e}[/red]")
return None
def display_user_instructions(self, device_response: dict) -> None:
"""Display user instructions for device authorization."""
user_code = device_response["user_code"]
verification_uri = device_response["verification_uri"]
verification_uri_complete = device_response.get("verification_uri_complete")
console.print("\n[bold blue]🔐 Authentication Required[/bold blue]")
console.print("\nTo authenticate, please visit:")
console.print(f"[bold cyan]{verification_uri}[/bold cyan]")
console.print(f"\nAnd enter this code: [bold yellow]{user_code}[/bold yellow]")
if verification_uri_complete:
console.print("\nOr for one-click access, visit:")
console.print(f"[bold green]{verification_uri_complete}[/bold green]")
# Try to open browser automatically
try:
console.print("\n[dim]Opening browser automatically...[/dim]")
webbrowser.open(verification_uri_complete)
except Exception:
pass # Silently fail if browser can't be opened
console.print("\n[dim]Waiting for you to complete authentication in your browser...[/dim]")
async def poll_for_token(self, device_code: str, interval: int = 5) -> dict | None:
"""Poll the token endpoint until user completes authentication."""
token_url = f"{self.authkit_domain}/oauth2/token"
data = {
"client_id": self.client_id,
"device_code": device_code,
"grant_type": "urn:ietf:params:oauth:grant-type:device_code",
"code_verifier": self.code_verifier,
}
max_attempts = 60 # 5 minutes with 5-second intervals
current_interval = interval
for _attempt in range(max_attempts):
try:
async with httpx.AsyncClient() as client:
response = await client.post(token_url, data=data)
if response.status_code == 200:
return response.json()
# Parse error response
try:
error_data = response.json()
error = error_data.get("error")
except Exception:
error = "unknown_error"
if error == "authorization_pending":
# User hasn't completed auth yet, keep polling
pass
elif error == "slow_down":
# Increase polling interval
current_interval += 5
console.print("[yellow]Slowing down polling rate...[/yellow]")
elif error == "access_denied":
console.print("[red]Authentication was denied by user[/red]")
return None
elif error == "expired_token":
console.print("[red]Device code has expired. Please try again.[/red]")
return None
else:
console.print(f"[red]Token polling error: {error}[/red]")
return None
except Exception as e:
console.print(f"[red]Token polling request error: {e}[/red]")
# Wait before next poll
await self._async_sleep(current_interval)
console.print("[red]Authentication timeout. Please try again.[/red]")
return None
async def _async_sleep(self, seconds: int) -> None:
"""Async sleep utility."""
import asyncio
await asyncio.sleep(seconds)
def save_tokens(self, tokens: dict) -> None:
"""Save tokens to project root as .bm-auth.json."""
token_data = {
"access_token": tokens["access_token"],
"refresh_token": tokens.get("refresh_token"),
"expires_at": int(time.time()) + tokens.get("expires_in", 3600),
"token_type": tokens.get("token_type", "Bearer"),
}
with open(self.token_file, "w") as f:
json.dump(token_data, f, indent=2)
# Secure the token file
os.chmod(self.token_file, 0o600)
console.print(f"[green]✓ Tokens saved to {self.token_file}[/green]")
def load_tokens(self) -> dict | None:
"""Load tokens from .bm-auth.json file."""
if not self.token_file.exists():
return None
try:
with open(self.token_file) as f:
return json.load(f)
except (OSError, json.JSONDecodeError):
return None
def is_token_valid(self, tokens: dict) -> bool:
"""Check if stored token is still valid."""
expires_at = tokens.get("expires_at", 0)
# Add 60 second buffer for clock skew
return time.time() < (expires_at - 60)
async def refresh_token(self, refresh_token: str) -> dict | None:
"""Refresh access token using refresh token."""
token_url = f"{self.authkit_domain}/oauth2/token"
data = {
"client_id": self.client_id,
"grant_type": "refresh_token",
"refresh_token": refresh_token,
}
try:
async with httpx.AsyncClient() as client:
response = await client.post(token_url, data=data)
if response.status_code == 200:
return response.json()
else:
console.print(
f"[red]Token refresh failed: {response.status_code} - {response.text}[/red]"
)
return None
except Exception as e:
console.print(f"[red]Token refresh error: {e}[/red]")
return None
async def get_valid_token(self) -> str | None:
"""Get valid access token, refresh if needed."""
tokens = self.load_tokens()
if not tokens:
return None
if self.is_token_valid(tokens):
return tokens["access_token"]
# Token expired - try to refresh if we have a refresh token
refresh_token = tokens.get("refresh_token")
if refresh_token:
console.print("[yellow]Access token expired, refreshing...[/yellow]")
new_tokens = await self.refresh_token(refresh_token)
if new_tokens:
# Save new tokens (may include rotated refresh token)
self.save_tokens(new_tokens)
console.print("[green]✓ Token refreshed successfully[/green]")
return new_tokens["access_token"]
else:
console.print("[yellow]Token refresh failed. Please run 'login' again.[/yellow]")
return None
else:
console.print("[yellow]No refresh token available. Please run 'login' again.[/yellow]")
return None
async def login(self) -> bool:
"""Perform OAuth Device Authorization login flow."""
console.print("[blue]Initiating WorkOS authentication...[/blue]")
# Step 1: Request device authorization
device_response = await self.request_device_authorization()
if not device_response:
return False
# Step 2: Display user instructions
self.display_user_instructions(device_response)
# Step 3: Poll for token
device_code = device_response["device_code"]
interval = device_response.get("interval", 5)
tokens = await self.poll_for_token(device_code, interval)
if not tokens:
return False
# Step 4: Save tokens
self.save_tokens(tokens)
console.print("\n[green]✅ Successfully authenticated with WorkOS![/green]")
return True
def logout(self) -> None:
"""Remove stored authentication tokens."""
if self.token_file.exists():
self.token_file.unlink()
console.print("[green]✓ Logged out successfully[/green]")
else:
console.print("[yellow]No stored authentication found[/yellow]")
+1 -2
View File
@@ -1,10 +1,9 @@
"""CLI commands for basic-memory."""
from . import auth, status, sync, db, import_memory_json, mcp, import_claude_conversations
from . import status, sync, db, import_memory_json, mcp, import_claude_conversations
from . import import_claude_projects, import_chatgpt, tool, project
__all__ = [
"auth",
"status",
"sync",
"db",
-136
View File
@@ -1,136 +0,0 @@
"""OAuth management commands."""
import typer
from typing import Optional
from pydantic import AnyHttpUrl
from basic_memory.cli.app import app
from basic_memory.mcp.auth_provider import BasicMemoryOAuthProvider
from mcp.shared.auth import OAuthClientInformationFull
auth_app = typer.Typer(help="OAuth client management commands")
app.add_typer(auth_app, name="auth")
@auth_app.command()
def register_client(
client_id: Optional[str] = typer.Option(
None, help="Client ID (auto-generated if not provided)"
),
client_secret: Optional[str] = typer.Option(
None, help="Client secret (auto-generated if not provided)"
),
issuer_url: str = typer.Option("http://localhost:8000", help="OAuth issuer URL"),
):
"""Register a new OAuth client for Basic Memory MCP server."""
# Create provider instance
provider = BasicMemoryOAuthProvider(issuer_url=issuer_url)
# Create client info with required redirect_uris
client_info = OAuthClientInformationFull(
client_id=client_id or "", # Provider will generate if empty
client_secret=client_secret or "", # Provider will generate if empty
redirect_uris=[AnyHttpUrl("http://localhost:8000/callback")], # Default redirect URI
client_name="Basic Memory OAuth Client",
grant_types=["authorization_code", "refresh_token"],
)
# Register the client
import asyncio
asyncio.run(provider.register_client(client_info))
typer.echo("Client registered successfully!")
typer.echo(f"Client ID: {client_info.client_id}")
typer.echo(f"Client Secret: {client_info.client_secret}")
typer.echo("\nSave these credentials securely - the client secret cannot be retrieved later.")
@auth_app.command()
def test_auth(
issuer_url: str = typer.Option("http://localhost:8000", help="OAuth issuer URL"),
):
"""Test OAuth authentication flow.
IMPORTANT: Use the same FASTMCP_AUTH_SECRET_KEY environment variable
as your MCP server for tokens to validate correctly.
"""
import asyncio
import secrets
from mcp.server.auth.provider import AuthorizationParams
from pydantic import AnyHttpUrl
async def test_flow():
# Create provider with same secret key as server
provider = BasicMemoryOAuthProvider(issuer_url=issuer_url)
# Register a test client
client_info = OAuthClientInformationFull(
client_id=secrets.token_urlsafe(16),
client_secret=secrets.token_urlsafe(32),
redirect_uris=[AnyHttpUrl("http://localhost:8000/callback")],
client_name="Test OAuth Client",
grant_types=["authorization_code", "refresh_token"],
)
await provider.register_client(client_info)
typer.echo(f"Registered test client: {client_info.client_id}")
# Get the client
client = await provider.get_client(client_info.client_id)
if not client:
typer.echo("Error: Client not found after registration", err=True)
return
# Create authorization request
auth_params = AuthorizationParams(
state="test-state",
scopes=["read", "write"],
code_challenge="test-challenge",
redirect_uri=AnyHttpUrl("http://localhost:8000/callback"),
redirect_uri_provided_explicitly=True,
)
# Get authorization URL
auth_url = await provider.authorize(client, auth_params)
typer.echo(f"Authorization URL: {auth_url}")
# Extract auth code from URL
from urllib.parse import urlparse, parse_qs
parsed = urlparse(auth_url)
params = parse_qs(parsed.query)
auth_code = params.get("code", [None])[0]
if not auth_code:
typer.echo("Error: No authorization code in URL", err=True)
return
# Load the authorization code
code_obj = await provider.load_authorization_code(client, auth_code)
if not code_obj:
typer.echo("Error: Invalid authorization code", err=True)
return
# Exchange for tokens
token = await provider.exchange_authorization_code(client, code_obj)
typer.echo(f"Access token: {token.access_token}")
typer.echo(f"Refresh token: {token.refresh_token}")
typer.echo(f"Expires in: {token.expires_in} seconds")
# Validate access token
access_token_obj = await provider.load_access_token(token.access_token)
if access_token_obj:
typer.echo("Access token validated successfully!")
typer.echo(f"Client ID: {access_token_obj.client_id}")
typer.echo(f"Scopes: {access_token_obj.scopes}")
else:
typer.echo("Error: Invalid access token", err=True)
asyncio.run(test_flow())
if __name__ == "__main__":
auth_app()
@@ -0,0 +1,5 @@
"""Cloud commands package."""
# Import all commands to register them with typer
from basic_memory.cli.commands.cloud.core_commands import * # noqa: F401,F403
from basic_memory.cli.commands.cloud.api_client import get_authenticated_headers # noqa: F401
@@ -0,0 +1,112 @@
"""Cloud API client utilities."""
from typing import Optional
import httpx
import typer
from rich.console import Console
from basic_memory.cli.auth import CLIAuth
from basic_memory.config import ConfigManager
console = Console()
class CloudAPIError(Exception):
"""Exception raised for cloud API errors."""
def __init__(
self, message: str, status_code: Optional[int] = None, detail: Optional[dict] = None
):
super().__init__(message)
self.status_code = status_code
self.detail = detail or {}
class SubscriptionRequiredError(CloudAPIError):
"""Exception raised when user needs an active subscription."""
def __init__(self, message: str, subscribe_url: str):
super().__init__(message, status_code=403, detail={"error": "subscription_required"})
self.subscribe_url = subscribe_url
def get_cloud_config() -> tuple[str, str, str]:
"""Get cloud OAuth configuration from config."""
config_manager = ConfigManager()
config = config_manager.config
return config.cloud_client_id, config.cloud_domain, config.cloud_host
async def get_authenticated_headers() -> dict[str, str]:
"""
Get authentication headers with JWT token.
handles jwt refresh if needed.
"""
client_id, domain, _ = get_cloud_config()
auth = CLIAuth(client_id=client_id, authkit_domain=domain)
token = await auth.get_valid_token()
if not token:
console.print("[red]Not authenticated. Please run 'basic-memory cloud login' first.[/red]")
raise typer.Exit(1)
return {"Authorization": f"Bearer {token}"}
async def make_api_request(
method: str,
url: str,
headers: Optional[dict] = None,
json_data: Optional[dict] = None,
timeout: float = 30.0,
) -> httpx.Response:
"""Make an API request to the cloud service."""
headers = headers or {}
auth_headers = await get_authenticated_headers()
headers.update(auth_headers)
# Add debug headers to help with compression issues
headers.setdefault("Accept-Encoding", "identity") # Disable compression for debugging
async with httpx.AsyncClient(timeout=timeout) as client:
try:
response = await client.request(method=method, url=url, headers=headers, json=json_data)
response.raise_for_status()
return response
except httpx.HTTPError as e:
# Check if this is a response error with response details
if hasattr(e, "response") and e.response is not None: # pyright: ignore [reportAttributeAccessIssue]
response = e.response # type: ignore
# Try to parse error detail from response
error_detail = None
try:
error_detail = response.json()
except Exception:
# If JSON parsing fails, we'll handle it as a generic error
pass
# Check for subscription_required error (403)
if response.status_code == 403 and isinstance(error_detail, dict):
# Handle both FastAPI HTTPException format (nested under "detail")
# and direct format
detail_obj = error_detail.get("detail", error_detail)
if (
isinstance(detail_obj, dict)
and detail_obj.get("error") == "subscription_required"
):
message = detail_obj.get("message", "Active subscription required")
subscribe_url = detail_obj.get(
"subscribe_url", "https://basicmemory.com/subscribe"
)
raise SubscriptionRequiredError(
message=message, subscribe_url=subscribe_url
) from e
# Raise generic CloudAPIError with status code and detail
raise CloudAPIError(
f"API request failed: {e}",
status_code=response.status_code,
detail=error_detail if isinstance(error_detail, dict) else {},
) from e
raise CloudAPIError(f"API request failed: {e}") from e
@@ -0,0 +1,818 @@
"""Cloud bisync commands for Basic Memory CLI."""
import asyncio
import subprocess
import time
from datetime import datetime
from pathlib import Path
from typing import Optional
import typer
from rich.console import Console
from rich.table import Table
from basic_memory.cli.commands.cloud.api_client import CloudAPIError, make_api_request
from basic_memory.cli.commands.cloud.rclone_config import (
add_tenant_to_rclone_config,
)
from basic_memory.cli.commands.cloud.rclone_installer import RcloneInstallError, install_rclone
from basic_memory.config import ConfigManager
from basic_memory.ignore_utils import get_bmignore_path, create_default_bmignore
from basic_memory.schemas.cloud import (
TenantMountInfo,
MountCredentials,
CloudProjectList,
CloudProjectCreateRequest,
CloudProjectCreateResponse,
)
from basic_memory.utils import generate_permalink
console = Console()
class BisyncError(Exception):
"""Exception raised for bisync-related errors."""
pass
class RcloneBisyncProfile:
"""Bisync profile with safety settings."""
def __init__(
self,
name: str,
conflict_resolve: str,
max_delete: int,
check_access: bool,
description: str,
extra_args: Optional[list[str]] = None,
):
self.name = name
self.conflict_resolve = conflict_resolve
self.max_delete = max_delete
self.check_access = check_access
self.description = description
self.extra_args = extra_args or []
# Bisync profiles based on SPEC-9 Phase 2.1
BISYNC_PROFILES = {
"safe": RcloneBisyncProfile(
name="safe",
conflict_resolve="none",
max_delete=10,
check_access=False,
description="Safe mode with conflict preservation (keeps both versions)",
),
"balanced": RcloneBisyncProfile(
name="balanced",
conflict_resolve="newer",
max_delete=25,
check_access=False,
description="Balanced mode - auto-resolve to newer file (recommended)",
),
"fast": RcloneBisyncProfile(
name="fast",
conflict_resolve="newer",
max_delete=50,
check_access=False,
description="Fast mode for rapid iteration (skip verification)",
),
}
async def get_mount_info() -> TenantMountInfo:
"""Get current tenant information from cloud API."""
try:
config_manager = ConfigManager()
config = config_manager.config
host_url = config.cloud_host.rstrip("/")
response = await make_api_request(method="GET", url=f"{host_url}/tenant/mount/info")
return TenantMountInfo.model_validate(response.json())
except Exception as e:
raise BisyncError(f"Failed to get tenant info: {e}") from e
async def generate_mount_credentials(tenant_id: str) -> MountCredentials:
"""Generate scoped credentials for syncing."""
try:
config_manager = ConfigManager()
config = config_manager.config
host_url = config.cloud_host.rstrip("/")
response = await make_api_request(method="POST", url=f"{host_url}/tenant/mount/credentials")
return MountCredentials.model_validate(response.json())
except Exception as e:
raise BisyncError(f"Failed to generate credentials: {e}") from e
async def fetch_cloud_projects() -> CloudProjectList:
"""Fetch list of projects from cloud API.
Returns:
CloudProjectList with projects from cloud
"""
try:
config_manager = ConfigManager()
config = config_manager.config
host_url = config.cloud_host.rstrip("/")
response = await make_api_request(method="GET", url=f"{host_url}/proxy/projects/projects")
return CloudProjectList.model_validate(response.json())
except Exception as e:
raise BisyncError(f"Failed to fetch cloud projects: {e}") from e
def scan_local_directories(sync_dir: Path) -> list[str]:
"""Scan local sync directory for project folders.
Args:
sync_dir: Path to bisync directory
Returns:
List of directory names (project names)
"""
if not sync_dir.exists():
return []
directories = []
for item in sync_dir.iterdir():
if item.is_dir() and not item.name.startswith("."):
directories.append(item.name)
return directories
async def create_cloud_project(project_name: str) -> CloudProjectCreateResponse:
"""Create a new project on cloud.
Args:
project_name: Name of project to create
Returns:
CloudProjectCreateResponse with project details from API
"""
try:
config_manager = ConfigManager()
config = config_manager.config
host_url = config.cloud_host.rstrip("/")
# Use generate_permalink to ensure consistent naming
project_path = generate_permalink(project_name)
project_data = CloudProjectCreateRequest(
name=project_name,
path=project_path,
set_default=False,
)
response = await make_api_request(
method="POST",
url=f"{host_url}/proxy/projects/projects",
headers={"Content-Type": "application/json"},
json_data=project_data.model_dump(),
)
return CloudProjectCreateResponse.model_validate(response.json())
except Exception as e:
raise BisyncError(f"Failed to create cloud project '{project_name}': {e}") from e
def get_bisync_state_path(tenant_id: str) -> Path:
"""Get path to bisync state directory."""
return Path.home() / ".basic-memory" / "bisync-state" / tenant_id
def get_bisync_directory() -> Path:
"""Get bisync directory from config.
Returns:
Path to bisync directory (default: ~/basic-memory-cloud-sync)
"""
config_manager = ConfigManager()
config = config_manager.config
sync_dir = config.bisync_config.get("sync_dir", str(Path.home() / "basic-memory-cloud-sync"))
return Path(sync_dir).expanduser().resolve()
def validate_bisync_directory(bisync_dir: Path) -> None:
"""Validate bisync directory doesn't conflict with mount.
Raises:
BisyncError: If bisync directory conflicts with mount directory
"""
# Get fixed mount directory
mount_dir = (Path.home() / "basic-memory-cloud").resolve()
# Check if bisync dir is the same as mount dir
if bisync_dir == mount_dir:
raise BisyncError(
f"Cannot use {bisync_dir} for bisync - it's the mount directory!\n"
f"Mount and bisync must use different directories.\n\n"
f"Options:\n"
f" 1. Use default: ~/basic-memory-cloud-sync/\n"
f" 2. Specify different directory: --dir ~/my-sync-folder"
)
# Check if mount is active at this location
result = subprocess.run(["mount"], capture_output=True, text=True)
if str(bisync_dir) in result.stdout and "rclone" in result.stdout:
raise BisyncError(
f"{bisync_dir} is currently mounted via 'bm cloud mount'\n"
f"Cannot use mounted directory for bisync.\n\n"
f"Either:\n"
f" 1. Unmount first: bm cloud unmount\n"
f" 2. Use different directory for bisync"
)
def convert_bmignore_to_rclone_filters() -> Path:
"""Convert .bmignore patterns to rclone filter format.
Reads ~/.basic-memory/.bmignore (gitignore-style) and converts to
~/.basic-memory/.bmignore.rclone (rclone filter format).
Only regenerates if .bmignore has been modified since last conversion.
Returns:
Path to converted rclone filter file
"""
# Ensure .bmignore exists
create_default_bmignore()
bmignore_path = get_bmignore_path()
# Create rclone filter path: ~/.basic-memory/.bmignore -> ~/.basic-memory/.bmignore.rclone
rclone_filter_path = bmignore_path.parent / f"{bmignore_path.name}.rclone"
# Skip regeneration if rclone file is newer than bmignore
if rclone_filter_path.exists():
bmignore_mtime = bmignore_path.stat().st_mtime
rclone_mtime = rclone_filter_path.stat().st_mtime
if rclone_mtime >= bmignore_mtime:
return rclone_filter_path
# Read .bmignore patterns
patterns = []
try:
with bmignore_path.open("r", encoding="utf-8") as f:
for line in f:
line = line.strip()
# Keep comments and empty lines
if not line or line.startswith("#"):
patterns.append(line)
continue
# Convert gitignore pattern to rclone filter syntax
# gitignore: node_modules → rclone: - node_modules/**
# gitignore: *.pyc → rclone: - *.pyc
if "*" in line:
# Pattern already has wildcard, just add exclude prefix
patterns.append(f"- {line}")
else:
# Directory pattern - add /** for recursive exclude
patterns.append(f"- {line}/**")
except Exception:
# If we can't read the file, create a minimal filter
patterns = ["# Error reading .bmignore, using minimal filters", "- .git/**"]
# Write rclone filter file
rclone_filter_path.write_text("\n".join(patterns) + "\n")
return rclone_filter_path
def get_bisync_filter_path() -> Path:
"""Get path to bisync filter file.
Uses ~/.basic-memory/.bmignore (converted to rclone format).
The file is automatically created with default patterns on first use.
Returns:
Path to rclone filter file
"""
return convert_bmignore_to_rclone_filters()
def bisync_state_exists(tenant_id: str) -> bool:
"""Check if bisync state exists (has been initialized)."""
state_path = get_bisync_state_path(tenant_id)
return state_path.exists() and any(state_path.iterdir())
def build_bisync_command(
tenant_id: str,
bucket_name: str,
local_path: Path,
profile: RcloneBisyncProfile,
dry_run: bool = False,
resync: bool = False,
verbose: bool = False,
) -> list[str]:
"""Build rclone bisync command with profile settings."""
# Sync with the entire bucket root (all projects)
rclone_remote = f"basic-memory-{tenant_id}:{bucket_name}"
filter_path = get_bisync_filter_path()
state_path = get_bisync_state_path(tenant_id)
# Ensure state directory exists
state_path.mkdir(parents=True, exist_ok=True)
cmd = [
"rclone",
"bisync",
str(local_path),
rclone_remote,
"--create-empty-src-dirs",
"--resilient",
f"--conflict-resolve={profile.conflict_resolve}",
f"--max-delete={profile.max_delete}",
"--filters-file",
str(filter_path),
"--workdir",
str(state_path),
]
# Add verbosity flags
if verbose:
cmd.append("--verbose") # Full details with file-by-file output
else:
# Show progress bar during transfers
cmd.append("--progress")
if profile.check_access:
cmd.append("--check-access")
if dry_run:
cmd.append("--dry-run")
if resync:
cmd.append("--resync")
cmd.extend(profile.extra_args)
return cmd
def setup_cloud_bisync(sync_dir: Optional[str] = None) -> None:
"""Set up cloud bisync with rclone installation and configuration.
Args:
sync_dir: Optional custom sync directory path. If not provided, uses config default.
"""
console.print("[bold blue]Basic Memory Cloud Bisync Setup[/bold blue]")
console.print("Setting up bidirectional sync to your cloud tenant...\n")
try:
# Step 1: Install rclone
console.print("[blue]Step 1: Installing rclone...[/blue]")
install_rclone()
# Step 2: Get mount info (for tenant_id, bucket)
console.print("\n[blue]Step 2: Getting tenant information...[/blue]")
tenant_info = asyncio.run(get_mount_info())
tenant_id = tenant_info.tenant_id
bucket_name = tenant_info.bucket_name
console.print(f"[green]✓ Found tenant: {tenant_id}[/green]")
console.print(f"[green]✓ Bucket: {bucket_name}[/green]")
# Step 3: Generate credentials
console.print("\n[blue]Step 3: Generating sync credentials...[/blue]")
creds = asyncio.run(generate_mount_credentials(tenant_id))
access_key = creds.access_key
secret_key = creds.secret_key
console.print("[green]✓ Generated secure credentials[/green]")
# Step 4: Configure rclone
console.print("\n[blue]Step 4: Configuring rclone...[/blue]")
add_tenant_to_rclone_config(
tenant_id=tenant_id,
bucket_name=bucket_name,
access_key=access_key,
secret_key=secret_key,
)
# Step 5: Configure and create local directory
console.print("\n[blue]Step 5: Configuring sync directory...[/blue]")
# If custom sync_dir provided, save to config
if sync_dir:
config_manager = ConfigManager()
config = config_manager.load_config()
config.bisync_config["sync_dir"] = sync_dir
config_manager.save_config(config)
console.print("[green]✓ Saved custom sync directory to config[/green]")
# Get bisync directory (from config or default)
local_path = get_bisync_directory()
# Validate bisync directory
validate_bisync_directory(local_path)
# Create directory
local_path.mkdir(parents=True, exist_ok=True)
console.print(f"[green]✓ Created sync directory: {local_path}[/green]")
# Step 6: Perform initial resync
console.print("\n[blue]Step 6: Performing initial sync...[/blue]")
console.print("[yellow]This will establish the baseline for bidirectional sync.[/yellow]")
run_bisync(
tenant_id=tenant_id,
bucket_name=bucket_name,
local_path=local_path,
profile_name="balanced",
resync=True,
)
console.print("\n[bold green]✓ Bisync setup completed successfully![/bold green]")
console.print("\nYour local files will now sync bidirectionally with the cloud!")
console.print(f"\nLocal directory: {local_path}")
console.print("\nUseful commands:")
console.print(" bm sync # Run sync (recommended)")
console.print(" bm sync --watch # Start watch mode")
console.print(" bm cloud status # Check sync status")
console.print(" bm cloud check # Verify file integrity")
console.print(" bm cloud bisync --dry-run # Preview changes (advanced)")
except (RcloneInstallError, BisyncError, CloudAPIError) as e:
console.print(f"\n[red]Setup failed: {e}[/red]")
raise typer.Exit(1)
except Exception as e:
console.print(f"\n[red]Unexpected error during setup: {e}[/red]")
raise typer.Exit(1)
def run_bisync(
tenant_id: Optional[str] = None,
bucket_name: Optional[str] = None,
local_path: Optional[Path] = None,
profile_name: str = "balanced",
dry_run: bool = False,
resync: bool = False,
verbose: bool = False,
) -> bool:
"""Run rclone bisync with specified profile."""
try:
# Get tenant info if not provided
if not tenant_id or not bucket_name:
tenant_info = asyncio.run(get_mount_info())
tenant_id = tenant_info.tenant_id
bucket_name = tenant_info.bucket_name
# Set default local path if not provided
if not local_path:
local_path = get_bisync_directory()
# Validate bisync directory
validate_bisync_directory(local_path)
# Check if local path exists
if not local_path.exists():
raise BisyncError(
f"Local directory {local_path} does not exist. Run 'basic-memory cloud bisync-setup' first."
)
# Get bisync profile
if profile_name not in BISYNC_PROFILES:
raise BisyncError(
f"Unknown profile: {profile_name}. Available: {list(BISYNC_PROFILES.keys())}"
)
profile = BISYNC_PROFILES[profile_name]
# Auto-register projects before sync (unless dry-run or resync)
if not dry_run and not resync:
try:
console.print("[dim]Checking for new projects...[/dim]")
# Fetch cloud projects and extract directory names from paths
cloud_data = asyncio.run(fetch_cloud_projects())
cloud_projects = cloud_data.projects
# Extract directory names from cloud project paths
# Compare directory names, not project names
# Cloud path /app/data/basic-memory -> directory name "basic-memory"
cloud_dir_names = set()
for p in cloud_projects:
path = p.path
# Strip /app/data/ prefix if present (cloud mode)
if path.startswith("/app/data/"):
path = path[len("/app/data/") :]
# Get the last segment (directory name)
dir_name = Path(path).name
cloud_dir_names.add(dir_name)
# Scan local directories
local_dirs = scan_local_directories(local_path)
# Create missing cloud projects
new_projects = []
for dir_name in local_dirs:
if dir_name not in cloud_dir_names:
new_projects.append(dir_name)
if new_projects:
console.print(
f"[blue]Found {len(new_projects)} new local project(s), creating on cloud...[/blue]"
)
for project_name in new_projects:
try:
asyncio.run(create_cloud_project(project_name))
console.print(f"[green] ✓ Created project: {project_name}[/green]")
except BisyncError as e:
console.print(
f"[yellow] ⚠ Could not create {project_name}: {e}[/yellow]"
)
else:
console.print("[dim]All local projects already registered on cloud[/dim]")
except Exception as e:
console.print(f"[yellow]Warning: Project auto-registration failed: {e}[/yellow]")
console.print("[yellow]Continuing with sync anyway...[/yellow]")
# Check if first run and require resync
if not resync and not bisync_state_exists(tenant_id) and not dry_run:
raise BisyncError(
"First bisync requires --resync to establish baseline. "
"Run: basic-memory cloud bisync --resync"
)
# Build and execute bisync command
bisync_cmd = build_bisync_command(
tenant_id,
bucket_name,
local_path,
profile,
dry_run=dry_run,
resync=resync,
verbose=verbose,
)
if dry_run:
console.print("[yellow]DRY RUN MODE - No changes will be made[/yellow]")
console.print(
f"[blue]Running bisync with profile '{profile_name}' ({profile.description})...[/blue]"
)
console.print(f"[dim]Command: {' '.join(bisync_cmd)}[/dim]")
console.print() # Blank line before output
# Stream output in real-time so user sees progress
result = subprocess.run(bisync_cmd, text=True)
if result.returncode != 0:
raise BisyncError(f"Bisync command failed with code {result.returncode}")
console.print() # Blank line after output
if dry_run:
console.print("[green]✓ Dry run completed successfully[/green]")
elif resync:
console.print("[green]✓ Initial sync baseline established[/green]")
else:
console.print("[green]✓ Sync completed successfully[/green]")
# Notify container to refresh cache (if not dry run)
if not dry_run:
try:
asyncio.run(notify_container_sync(tenant_id))
except Exception as e:
console.print(f"[yellow]Warning: Could not notify container: {e}[/yellow]")
return True
except BisyncError:
raise
except Exception as e:
raise BisyncError(f"Unexpected error during bisync: {e}") from e
async def notify_container_sync(tenant_id: str) -> None:
"""Sync all projects after bisync completes."""
try:
from basic_memory.cli.commands.command_utils import run_sync
# Fetch all projects and sync each one
cloud_data = await fetch_cloud_projects()
projects = cloud_data.projects
if not projects:
console.print("[dim]No projects to sync[/dim]")
return
console.print(f"[blue]Notifying cloud to index {len(projects)} project(s)...[/blue]")
for project in projects:
project_name = project.name
if project_name:
try:
await run_sync(project=project_name)
except Exception as e:
# Non-critical, log and continue
console.print(f"[yellow] ⚠ Sync failed for {project_name}: {e}[/yellow]")
console.print("[dim]Note: Cloud indexing has started and may take a few moments[/dim]")
except Exception as e:
# Non-critical, don't fail the bisync
console.print(f"[yellow]Warning: Post-sync failed: {e}[/yellow]")
def run_bisync_watch(
tenant_id: Optional[str] = None,
bucket_name: Optional[str] = None,
local_path: Optional[Path] = None,
profile_name: str = "balanced",
interval_seconds: int = 60,
) -> None:
"""Run bisync in watch mode with periodic syncs."""
console.print("[bold blue]Starting bisync watch mode[/bold blue]")
console.print(f"Sync interval: {interval_seconds} seconds")
console.print("Press Ctrl+C to stop\n")
try:
while True:
try:
start_time = time.time()
run_bisync(
tenant_id=tenant_id,
bucket_name=bucket_name,
local_path=local_path,
profile_name=profile_name,
)
elapsed = time.time() - start_time
console.print(f"[dim]Sync completed in {elapsed:.1f}s[/dim]")
# Wait for next interval
time.sleep(interval_seconds)
except BisyncError as e:
console.print(f"[red]Sync error: {e}[/red]")
console.print(f"[yellow]Retrying in {interval_seconds} seconds...[/yellow]")
time.sleep(interval_seconds)
except KeyboardInterrupt:
console.print("\n[yellow]Watch mode stopped[/yellow]")
def show_bisync_status() -> None:
"""Show current bisync status and configuration."""
try:
# Get tenant info
tenant_info = asyncio.run(get_mount_info())
tenant_id = tenant_info.tenant_id
local_path = get_bisync_directory()
state_path = get_bisync_state_path(tenant_id)
# Create status table
table = Table(title="Cloud Bisync Status", show_header=True, header_style="bold blue")
table.add_column("Property", style="green", min_width=20)
table.add_column("Value", style="dim", min_width=30)
# Check initialization status
is_initialized = bisync_state_exists(tenant_id)
init_status = (
"[green]✓ Initialized[/green]" if is_initialized else "[red]✗ Not initialized[/red]"
)
table.add_row("Tenant ID", tenant_id)
table.add_row("Local Directory", str(local_path))
table.add_row("Status", init_status)
table.add_row("State Directory", str(state_path))
# Check for last sync info
if is_initialized:
# Look for most recent state file
state_files = list(state_path.glob("*.lst"))
if state_files:
latest = max(state_files, key=lambda p: p.stat().st_mtime)
last_sync = datetime.fromtimestamp(latest.stat().st_mtime)
table.add_row("Last Sync", last_sync.strftime("%Y-%m-%d %H:%M:%S"))
console.print(table)
# Show bisync profiles
console.print("\n[bold]Available bisync profiles:[/bold]")
for name, profile in BISYNC_PROFILES.items():
console.print(f" {name}: {profile.description}")
console.print(f" - Conflict resolution: {profile.conflict_resolve}")
console.print(f" - Max delete: {profile.max_delete} files")
console.print("\n[dim]To use a profile: bm cloud bisync --profile <name>[/dim]")
# Show setup instructions if not initialized
if not is_initialized:
console.print("\n[yellow]To initialize bisync, run:[/yellow]")
console.print(" bm cloud setup")
console.print(" or")
console.print(" bm cloud bisync --resync")
except Exception as e:
console.print(f"[red]Error getting bisync status: {e}[/red]")
raise typer.Exit(1)
def run_check(
tenant_id: Optional[str] = None,
bucket_name: Optional[str] = None,
local_path: Optional[Path] = None,
one_way: bool = False,
) -> bool:
"""Check file integrity between local and cloud using rclone check.
Args:
tenant_id: Cloud tenant ID (auto-detected if not provided)
bucket_name: S3 bucket name (auto-detected if not provided)
local_path: Local bisync directory (uses config default if not provided)
one_way: If True, only check for missing files on destination (faster)
Returns:
True if check passed (files match), False if differences found
"""
try:
# Check if rclone is installed
from basic_memory.cli.commands.cloud.rclone_installer import is_rclone_installed
if not is_rclone_installed():
raise BisyncError(
"rclone is not installed. Run 'bm cloud bisync-setup' first to set up cloud sync."
)
# Get tenant info if not provided
if not tenant_id or not bucket_name:
tenant_info = asyncio.run(get_mount_info())
tenant_id = tenant_id or tenant_info.tenant_id
bucket_name = bucket_name or tenant_info.bucket_name
# Get local path from config
if not local_path:
local_path = get_bisync_directory()
# Check if bisync is initialized
if not bisync_state_exists(tenant_id):
raise BisyncError(
"Bisync not initialized. Run 'bm cloud bisync --resync' to establish baseline."
)
# Build rclone check command
rclone_remote = f"basic-memory-{tenant_id}:{bucket_name}"
filter_path = get_bisync_filter_path()
cmd = [
"rclone",
"check",
str(local_path),
rclone_remote,
"--filter-from",
str(filter_path),
]
if one_way:
cmd.append("--one-way")
console.print("[bold blue]Checking file integrity between local and cloud[/bold blue]")
console.print(f"[dim]Local: {local_path}[/dim]")
console.print(f"[dim]Remote: {rclone_remote}[/dim]")
console.print(f"[dim]Command: {' '.join(cmd)}[/dim]")
console.print()
# Run check command
result = subprocess.run(cmd, capture_output=True, text=True)
# rclone check returns:
# 0 = success (all files match)
# non-zero = differences found or error
if result.returncode == 0:
console.print("[green]✓ All files match between local and cloud[/green]")
return True
else:
console.print("[yellow]⚠ Differences found:[/yellow]")
if result.stderr:
console.print(result.stderr)
if result.stdout:
console.print(result.stdout)
console.print("\n[dim]To sync differences, run: bm sync[/dim]")
return False
except BisyncError:
raise
except Exception as e:
raise BisyncError(f"Check failed: {e}") from e
@@ -0,0 +1,288 @@
"""Core cloud commands for Basic Memory CLI."""
import asyncio
from typing import Optional
import typer
from rich.console import Console
from basic_memory.cli.app import cloud_app
from basic_memory.cli.auth import CLIAuth
from basic_memory.config import ConfigManager
from basic_memory.cli.commands.cloud.api_client import (
CloudAPIError,
SubscriptionRequiredError,
get_cloud_config,
make_api_request,
)
from basic_memory.cli.commands.cloud.mount_commands import (
mount_cloud_files,
setup_cloud_mount,
show_mount_status,
unmount_cloud_files,
)
from basic_memory.cli.commands.cloud.bisync_commands import (
run_bisync,
run_bisync_watch,
run_check,
setup_cloud_bisync,
show_bisync_status,
)
from basic_memory.cli.commands.cloud.rclone_config import MOUNT_PROFILES
from basic_memory.cli.commands.cloud.bisync_commands import BISYNC_PROFILES
console = Console()
@cloud_app.command()
def login():
"""Authenticate with WorkOS using OAuth Device Authorization flow and enable cloud mode."""
async def _login():
client_id, domain, host_url = get_cloud_config()
auth = CLIAuth(client_id=client_id, authkit_domain=domain)
try:
success = await auth.login()
if not success:
console.print("[red]Login failed[/red]")
raise typer.Exit(1)
# Test subscription access by calling a protected endpoint
console.print("[dim]Verifying subscription access...[/dim]")
await make_api_request("GET", f"{host_url.rstrip('/')}/proxy/health")
# Enable cloud mode after successful login and subscription validation
config_manager = ConfigManager()
config = config_manager.load_config()
config.cloud_mode = True
config_manager.save_config(config)
console.print("[green]✓ Cloud mode enabled[/green]")
console.print(f"[dim]All CLI commands now work against {host_url}[/dim]")
except SubscriptionRequiredError as e:
console.print("\n[red]✗ Subscription Required[/red]\n")
console.print(f"[yellow]{e.args[0]}[/yellow]\n")
console.print(f"Subscribe at: [blue underline]{e.subscribe_url}[/blue underline]\n")
console.print(
"[dim]Once you have an active subscription, run [bold]bm cloud login[/bold] again.[/dim]"
)
raise typer.Exit(1)
asyncio.run(_login())
@cloud_app.command()
def logout():
"""Disable cloud mode and return to local mode."""
# Disable cloud mode
config_manager = ConfigManager()
config = config_manager.load_config()
config.cloud_mode = False
config_manager.save_config(config)
console.print("[green]✓ Cloud mode disabled[/green]")
console.print("[dim]All CLI commands now work locally[/dim]")
@cloud_app.command("status")
def status(
bisync: bool = typer.Option(
True,
"--bisync/--mount",
help="Show bisync status (default) or mount status",
),
) -> None:
"""Check cloud mode status and cloud instance health.
Shows cloud mode status, instance health, and sync/mount status.
Use --bisync (default) to show bisync status or --mount for mount status.
"""
# Check cloud mode
config_manager = ConfigManager()
config = config_manager.load_config()
console.print("[bold blue]Cloud Mode Status[/bold blue]")
if config.cloud_mode:
console.print(" Mode: [green]Cloud (enabled)[/green]")
console.print(f" Host: {config.cloud_host}")
console.print(" [dim]All CLI commands work against cloud[/dim]")
else:
console.print(" Mode: [yellow]Local (disabled)[/yellow]")
console.print(" [dim]All CLI commands work locally[/dim]")
console.print("\n[dim]To enable cloud mode, run: bm cloud login[/dim]")
return
# Get cloud configuration
_, _, host_url = get_cloud_config()
host_url = host_url.rstrip("/")
# Prepare headers
headers = {}
try:
console.print("\n[blue]Checking cloud instance health...[/blue]")
# Make API request to check health
response = asyncio.run(
make_api_request(method="GET", url=f"{host_url}/proxy/health", headers=headers)
)
health_data = response.json()
console.print("[green]Cloud instance is healthy[/green]")
# Display status details
if "status" in health_data:
console.print(f" Status: {health_data['status']}")
if "version" in health_data:
console.print(f" Version: {health_data['version']}")
if "timestamp" in health_data:
console.print(f" Timestamp: {health_data['timestamp']}")
# Show sync/mount status based on flag
console.print()
if bisync:
show_bisync_status()
else:
show_mount_status()
except CloudAPIError as e:
console.print(f"[red]Error checking cloud health: {e}[/red]")
raise typer.Exit(1)
except Exception as e:
console.print(f"[red]Unexpected error: {e}[/red]")
raise typer.Exit(1)
# Mount commands
@cloud_app.command("setup")
def setup(
bisync: bool = typer.Option(
True,
"--bisync/--mount",
help="Use bidirectional sync (recommended) or mount as network drive",
),
sync_dir: Optional[str] = typer.Option(
None,
"--dir",
help="Custom sync directory for bisync (default: ~/basic-memory-cloud-sync)",
),
) -> None:
"""Set up cloud file access with automatic rclone installation and configuration.
Default: Sets up bidirectional sync (recommended).\n
Use --mount: Sets up mount as network drive (alternative workflow).\n
Examples:\n
bm cloud setup # Setup bisync (default)\n
bm cloud setup --mount # Setup mount instead\n
bm cloud setup --dir ~/sync # Custom bisync directory\n
"""
if bisync:
setup_cloud_bisync(sync_dir=sync_dir)
else:
setup_cloud_mount()
@cloud_app.command("mount")
def mount(
profile: str = typer.Option(
"balanced", help=f"Mount profile: {', '.join(MOUNT_PROFILES.keys())}"
),
path: Optional[str] = typer.Option(
None, help="Custom mount path (default: ~/basic-memory-{tenant-id})"
),
) -> None:
"""Mount cloud files locally for editing."""
try:
mount_cloud_files(profile_name=profile)
except Exception as e:
console.print(f"[red]Mount failed: {e}[/red]")
raise typer.Exit(1)
@cloud_app.command("unmount")
def unmount() -> None:
"""Unmount cloud files."""
try:
unmount_cloud_files()
except Exception as e:
console.print(f"[red]Unmount failed: {e}[/red]")
raise typer.Exit(1)
# Bisync commands
@cloud_app.command("bisync")
def bisync(
profile: str = typer.Option(
"balanced", help=f"Bisync profile: {', '.join(BISYNC_PROFILES.keys())}"
),
dry_run: bool = typer.Option(False, "--dry-run", help="Preview changes without syncing"),
resync: bool = typer.Option(False, "--resync", help="Force resync to establish new baseline"),
watch: bool = typer.Option(False, "--watch", help="Run continuous sync in watch mode"),
interval: int = typer.Option(60, "--interval", help="Sync interval in seconds for watch mode"),
verbose: bool = typer.Option(False, "--verbose", "-v", help="Show detailed sync output"),
) -> None:
"""Run bidirectional sync between local files and cloud storage.
Examples:
basic-memory cloud bisync # Manual sync with balanced profile
basic-memory cloud bisync --dry-run # Preview what would be synced
basic-memory cloud bisync --resync # Establish new baseline
basic-memory cloud bisync --watch # Continuous sync every 60s
basic-memory cloud bisync --watch --interval 30 # Continuous sync every 30s
basic-memory cloud bisync --profile safe # Use safe profile (keep conflicts)
basic-memory cloud bisync --verbose # Show detailed file sync output
"""
try:
if watch:
run_bisync_watch(profile_name=profile, interval_seconds=interval)
else:
run_bisync(profile_name=profile, dry_run=dry_run, resync=resync, verbose=verbose)
except Exception as e:
console.print(f"[red]Bisync failed: {e}[/red]")
raise typer.Exit(1)
@cloud_app.command("bisync-status")
def bisync_status() -> None:
"""Show current bisync status and configuration.
DEPRECATED: Use 'bm cloud status' instead (bisync is now the default).
"""
console.print(
"[yellow]Note: 'bisync-status' is deprecated. Use 'bm cloud status' instead.[/yellow]"
)
console.print("[dim]Showing bisync status...[/dim]\n")
show_bisync_status()
@cloud_app.command("check")
def check(
one_way: bool = typer.Option(
False,
"--one-way",
help="Only check for missing files on destination (faster)",
),
) -> None:
"""Check file integrity between local and cloud storage using rclone check.
Verifies that files match between your local bisync directory and cloud storage
without transferring any data. This is useful for validating sync integrity.
Examples:
bm cloud check # Full integrity check
bm cloud check --one-way # Faster check (missing files only)
"""
try:
run_check(one_way=one_way)
except Exception as e:
console.print(f"[red]Check failed: {e}[/red]")
raise typer.Exit(1)
@@ -0,0 +1,295 @@
"""Cloud mount commands for Basic Memory CLI."""
import asyncio
import subprocess
import time
from pathlib import Path
from typing import Optional
import typer
from rich.console import Console
from rich.table import Table
from basic_memory.cli.commands.cloud.api_client import CloudAPIError, make_api_request
from basic_memory.cli.commands.cloud.rclone_config import (
MOUNT_PROFILES,
add_tenant_to_rclone_config,
build_mount_command,
cleanup_orphaned_rclone_processes,
get_default_mount_path,
get_rclone_processes,
is_path_mounted,
unmount_path,
)
from basic_memory.cli.commands.cloud.rclone_installer import RcloneInstallError, install_rclone
from basic_memory.config import ConfigManager
console = Console()
class MountError(Exception):
"""Exception raised for mount-related errors."""
pass
async def get_tenant_info() -> dict:
"""Get current tenant information from cloud API."""
try:
config_manager = ConfigManager()
config = config_manager.config
host_url = config.cloud_host.rstrip("/")
response = await make_api_request(method="GET", url=f"{host_url}/tenant/mount/info")
return response.json()
except Exception as e:
raise MountError(f"Failed to get tenant info: {e}") from e
async def generate_mount_credentials(tenant_id: str) -> dict:
"""Generate scoped credentials for mounting."""
try:
config_manager = ConfigManager()
config = config_manager.config
host_url = config.cloud_host.rstrip("/")
response = await make_api_request(method="POST", url=f"{host_url}/tenant/mount/credentials")
return response.json()
except Exception as e:
raise MountError(f"Failed to generate mount credentials: {e}") from e
def setup_cloud_mount() -> None:
"""Set up cloud mount with rclone installation and configuration."""
console.print("[bold blue]Basic Memory Cloud Setup[/bold blue]")
console.print("Setting up local file access to your cloud tenant...\n")
try:
# Step 1: Install rclone
console.print("[blue]Step 1: Installing rclone...[/blue]")
install_rclone()
# Step 2: Get tenant info
console.print("\n[blue]Step 2: Getting tenant information...[/blue]")
tenant_info = asyncio.run(get_tenant_info())
tenant_id = tenant_info.get("tenant_id")
bucket_name = tenant_info.get("bucket_name")
if not tenant_id or not bucket_name:
raise MountError("Invalid tenant information received from cloud API")
console.print(f"[green]✓ Found tenant: {tenant_id}[/green]")
console.print(f"[green]✓ Bucket: {bucket_name}[/green]")
# Step 3: Generate mount credentials
console.print("\n[blue]Step 3: Generating mount credentials...[/blue]")
creds = asyncio.run(generate_mount_credentials(tenant_id))
access_key = creds.get("access_key")
secret_key = creds.get("secret_key")
if not access_key or not secret_key:
raise MountError("Failed to generate mount credentials")
console.print("[green]✓ Generated secure credentials[/green]")
# Step 4: Configure rclone
console.print("\n[blue]Step 4: Configuring rclone...[/blue]")
add_tenant_to_rclone_config(
tenant_id=tenant_id,
bucket_name=bucket_name,
access_key=access_key,
secret_key=secret_key,
)
# Step 5: Perform initial mount
console.print("\n[blue]Step 5: Mounting cloud files...[/blue]")
mount_path = get_default_mount_path()
MOUNT_PROFILES["balanced"]
mount_cloud_files(
tenant_id=tenant_id,
bucket_name=bucket_name,
mount_path=mount_path,
profile_name="balanced",
)
console.print("\n[bold green]✓ Cloud setup completed successfully![/bold green]")
console.print("\nYour cloud files are now accessible at:")
console.print(f" {mount_path}")
console.print("\nYou can now edit files locally and they will sync to the cloud!")
console.print("\nUseful commands:")
console.print(" basic-memory cloud mount-status # Check mount status")
console.print(" basic-memory cloud unmount # Unmount files")
console.print(" basic-memory cloud mount --profile fast # Remount with faster sync")
except (RcloneInstallError, MountError, CloudAPIError) as e:
console.print(f"\n[red]Setup failed: {e}[/red]")
raise typer.Exit(1)
except Exception as e:
console.print(f"\n[red]Unexpected error during setup: {e}[/red]")
raise typer.Exit(1)
def mount_cloud_files(
tenant_id: Optional[str] = None,
bucket_name: Optional[str] = None,
mount_path: Optional[Path] = None,
profile_name: str = "balanced",
) -> None:
"""Mount cloud files with specified profile."""
try:
# Get tenant info if not provided
if not tenant_id or not bucket_name:
tenant_info = asyncio.run(get_tenant_info())
tenant_id = tenant_info.get("tenant_id")
bucket_name = tenant_info.get("bucket_name")
if not tenant_id or not bucket_name:
raise MountError("Could not determine tenant information")
# Set default mount path if not provided
if not mount_path:
mount_path = get_default_mount_path()
# Get mount profile
if profile_name not in MOUNT_PROFILES:
raise MountError(
f"Unknown profile: {profile_name}. Available: {list(MOUNT_PROFILES.keys())}"
)
profile = MOUNT_PROFILES[profile_name]
# Check if already mounted
if is_path_mounted(mount_path):
console.print(f"[yellow]Path {mount_path} is already mounted[/yellow]")
console.print("Use 'basic-memory cloud unmount' first, or mount to a different path")
return
# Create mount directory
mount_path.mkdir(parents=True, exist_ok=True)
# Build and execute mount command
mount_cmd = build_mount_command(tenant_id, bucket_name, mount_path, profile)
console.print(
f"[blue]Mounting with profile '{profile_name}' ({profile.description})...[/blue]"
)
console.print(f"[dim]Command: {' '.join(mount_cmd)}[/dim]")
result = subprocess.run(mount_cmd, capture_output=True, text=True)
if result.returncode != 0:
error_msg = result.stderr or "Unknown error"
raise MountError(f"Mount command failed: {error_msg}")
# Wait a moment for mount to establish
time.sleep(2)
# Verify mount
if is_path_mounted(mount_path):
console.print(f"[green]✓ Successfully mounted to {mount_path}[/green]")
console.print(f"[green]✓ Sync profile: {profile.description}[/green]")
else:
raise MountError("Mount command succeeded but path is not mounted")
except MountError:
raise
except Exception as e:
raise MountError(f"Unexpected error during mount: {e}") from e
def unmount_cloud_files(tenant_id: Optional[str] = None) -> None:
"""Unmount cloud files."""
try:
# Get tenant info if not provided
if not tenant_id:
tenant_info = asyncio.run(get_tenant_info())
tenant_id = tenant_info.get("tenant_id")
if not tenant_id:
raise MountError("Could not determine tenant ID")
mount_path = get_default_mount_path()
if not is_path_mounted(mount_path):
console.print(f"[yellow]Path {mount_path} is not mounted[/yellow]")
return
console.print(f"[blue]Unmounting {mount_path}...[/blue]")
# Unmount the path
if unmount_path(mount_path):
console.print(f"[green]✓ Successfully unmounted {mount_path}[/green]")
# Clean up any orphaned rclone processes
killed_count = cleanup_orphaned_rclone_processes()
if killed_count > 0:
console.print(
f"[green]✓ Cleaned up {killed_count} orphaned rclone process(es)[/green]"
)
else:
console.print(f"[red]✗ Failed to unmount {mount_path}[/red]")
console.print("You may need to manually unmount or restart your system")
except MountError:
raise
except Exception as e:
raise MountError(f"Unexpected error during unmount: {e}") from e
def show_mount_status() -> None:
"""Show current mount status and running processes."""
try:
# Get tenant info
tenant_info = asyncio.run(get_tenant_info())
tenant_id = tenant_info.get("tenant_id")
if not tenant_id:
console.print("[red]Could not determine tenant ID[/red]")
return
mount_path = get_default_mount_path()
# Create status table
table = Table(title="Cloud Mount Status", show_header=True, header_style="bold blue")
table.add_column("Property", style="green", min_width=15)
table.add_column("Value", style="dim", min_width=30)
# Check mount status
is_mounted = is_path_mounted(mount_path)
mount_status = "[green]✓ Mounted[/green]" if is_mounted else "[red]✗ Not mounted[/red]"
table.add_row("Tenant ID", tenant_id)
table.add_row("Mount Path", str(mount_path))
table.add_row("Status", mount_status)
# Get rclone processes
processes = get_rclone_processes()
if processes:
table.add_row("rclone Processes", f"{len(processes)} running")
else:
table.add_row("rclone Processes", "None")
console.print(table)
# Show running processes details
if processes:
console.print("\n[bold]Running rclone processes:[/bold]")
for proc in processes:
console.print(f" PID {proc['pid']}: {proc['command'][:80]}...")
# Show mount profiles
console.print("\n[bold]Available mount profiles:[/bold]")
for name, profile in MOUNT_PROFILES.items():
console.print(f" {name}: {profile.description}")
except Exception as e:
console.print(f"[red]Error getting mount status: {e}[/red]")
raise typer.Exit(1)
@@ -0,0 +1,288 @@
"""rclone configuration management for Basic Memory Cloud."""
import configparser
import os
import shutil
import subprocess
from pathlib import Path
from typing import Dict, List, Optional
from rich.console import Console
console = Console()
class RcloneConfigError(Exception):
"""Exception raised for rclone configuration errors."""
pass
class RcloneMountProfile:
"""Mount profile with optimized settings."""
def __init__(
self,
name: str,
cache_time: str,
poll_interval: str,
attr_timeout: str,
write_back: str,
description: str,
extra_args: Optional[List[str]] = None,
):
self.name = name
self.cache_time = cache_time
self.poll_interval = poll_interval
self.attr_timeout = attr_timeout
self.write_back = write_back
self.description = description
self.extra_args = extra_args or []
# Mount profiles based on SPEC-7 Phase 4 testing
MOUNT_PROFILES = {
"fast": RcloneMountProfile(
name="fast",
cache_time="5s",
poll_interval="3s",
attr_timeout="3s",
write_back="1s",
description="Ultra-fast development (5s sync, higher bandwidth)",
),
"balanced": RcloneMountProfile(
name="balanced",
cache_time="10s",
poll_interval="5s",
attr_timeout="5s",
write_back="2s",
description="Fast development (10-15s sync, recommended)",
),
"safe": RcloneMountProfile(
name="safe",
cache_time="15s",
poll_interval="10s",
attr_timeout="10s",
write_back="5s",
description="Conflict-aware mount with backup",
extra_args=[
"--conflict-suffix",
".conflict-{DateTimeExt}",
"--backup-dir",
"~/.basic-memory/conflicts",
"--track-renames",
],
),
}
def get_rclone_config_path() -> Path:
"""Get the path to rclone configuration file."""
config_dir = Path.home() / ".config" / "rclone"
config_dir.mkdir(parents=True, exist_ok=True)
return config_dir / "rclone.conf"
def backup_rclone_config() -> Optional[Path]:
"""Create a backup of existing rclone config."""
config_path = get_rclone_config_path()
if not config_path.exists():
return None
backup_path = config_path.with_suffix(f".conf.backup-{os.getpid()}")
shutil.copy2(config_path, backup_path)
console.print(f"[dim]Created backup: {backup_path}[/dim]")
return backup_path
def load_rclone_config() -> configparser.ConfigParser:
"""Load existing rclone configuration."""
config = configparser.ConfigParser()
config_path = get_rclone_config_path()
if config_path.exists():
config.read(config_path)
return config
def save_rclone_config(config: configparser.ConfigParser) -> None:
"""Save rclone configuration to file."""
config_path = get_rclone_config_path()
with open(config_path, "w") as f:
config.write(f)
console.print(f"[dim]Updated rclone config: {config_path}[/dim]")
def add_tenant_to_rclone_config(
tenant_id: str,
bucket_name: str,
access_key: str,
secret_key: str,
endpoint: str = "https://fly.storage.tigris.dev",
region: str = "auto",
) -> str:
"""Add tenant configuration to rclone config file."""
# Backup existing config
backup_rclone_config()
# Load existing config
config = load_rclone_config()
# Create section name
section_name = f"basic-memory-{tenant_id}"
# Add/update the tenant section
if not config.has_section(section_name):
config.add_section(section_name)
config.set(section_name, "type", "s3")
config.set(section_name, "provider", "Other")
config.set(section_name, "access_key_id", access_key)
config.set(section_name, "secret_access_key", secret_key)
config.set(section_name, "endpoint", endpoint)
config.set(section_name, "region", region)
# Save updated config
save_rclone_config(config)
console.print(f"[green]✓ Added tenant {tenant_id} to rclone config[/green]")
return section_name
def remove_tenant_from_rclone_config(tenant_id: str) -> bool:
"""Remove tenant configuration from rclone config."""
config = load_rclone_config()
section_name = f"basic-memory-{tenant_id}"
if config.has_section(section_name):
backup_rclone_config()
config.remove_section(section_name)
save_rclone_config(config)
console.print(f"[green]✓ Removed tenant {tenant_id} from rclone config[/green]")
return True
return False
def get_default_mount_path() -> Path:
"""Get default mount path (fixed location per SPEC-9).
Returns:
Fixed mount path: ~/basic-memory-cloud/
"""
return Path.home() / "basic-memory-cloud"
def build_mount_command(
tenant_id: str, bucket_name: str, mount_path: Path, profile: RcloneMountProfile
) -> List[str]:
"""Build rclone mount command with optimized settings."""
rclone_remote = f"basic-memory-{tenant_id}:{bucket_name}"
cmd = [
"rclone",
"nfsmount",
rclone_remote,
str(mount_path),
"--vfs-cache-mode",
"writes",
"--dir-cache-time",
profile.cache_time,
"--vfs-cache-poll-interval",
profile.poll_interval,
"--attr-timeout",
profile.attr_timeout,
"--vfs-write-back",
profile.write_back,
"--daemon",
]
# Add profile-specific extra arguments
cmd.extend(profile.extra_args)
return cmd
def is_path_mounted(mount_path: Path) -> bool:
"""Check if a path is currently mounted."""
if not mount_path.exists():
return False
try:
# Check if mount point is actually mounted by looking for mount table entry
result = subprocess.run(["mount"], capture_output=True, text=True, check=False)
if result.returncode == 0:
# Look for our mount path in mount output
mount_str = str(mount_path.resolve())
return mount_str in result.stdout
return False
except Exception:
return False
def get_rclone_processes() -> List[Dict[str, str]]:
"""Get list of running rclone processes."""
try:
# Use ps to find rclone processes
result = subprocess.run(
["ps", "-eo", "pid,args"], capture_output=True, text=True, check=False
)
processes = []
if result.returncode == 0:
for line in result.stdout.split("\n"):
if "rclone" in line and "basic-memory" in line:
parts = line.strip().split(None, 1)
if len(parts) >= 2:
processes.append({"pid": parts[0], "command": parts[1]})
return processes
except Exception:
return []
def kill_rclone_process(pid: str) -> bool:
"""Kill a specific rclone process."""
try:
subprocess.run(["kill", pid], check=True)
console.print(f"[green]✓ Killed rclone process {pid}[/green]")
return True
except subprocess.CalledProcessError:
console.print(f"[red]✗ Failed to kill rclone process {pid}[/red]")
return False
def unmount_path(mount_path: Path) -> bool:
"""Unmount a mounted path."""
if not is_path_mounted(mount_path):
return True
try:
subprocess.run(["umount", str(mount_path)], check=True)
console.print(f"[green]✓ Unmounted {mount_path}[/green]")
return True
except subprocess.CalledProcessError as e:
console.print(f"[red]✗ Failed to unmount {mount_path}: {e}[/red]")
return False
def cleanup_orphaned_rclone_processes() -> int:
"""Clean up orphaned rclone processes for basic-memory."""
processes = get_rclone_processes()
killed_count = 0
for proc in processes:
console.print(
f"[yellow]Found rclone process: {proc['pid']} - {proc['command'][:80]}...[/yellow]"
)
if kill_rclone_process(proc["pid"]):
killed_count += 1
return killed_count
@@ -0,0 +1,198 @@
"""Cross-platform rclone installation utilities."""
import platform
import shutil
import subprocess
from typing import Optional
from rich.console import Console
console = Console()
class RcloneInstallError(Exception):
"""Exception raised for rclone installation errors."""
pass
def is_rclone_installed() -> bool:
"""Check if rclone is already installed and available in PATH."""
return shutil.which("rclone") is not None
def get_platform() -> str:
"""Get the current platform identifier."""
system = platform.system().lower()
if system == "darwin":
return "macos"
elif system == "linux":
return "linux"
elif system == "windows":
return "windows"
else:
raise RcloneInstallError(f"Unsupported platform: {system}")
def run_command(command: list[str], check: bool = True) -> subprocess.CompletedProcess:
"""Run a command with proper error handling."""
try:
console.print(f"[dim]Running: {' '.join(command)}[/dim]")
result = subprocess.run(command, capture_output=True, text=True, check=check)
if result.stdout:
console.print(f"[dim]Output: {result.stdout.strip()}[/dim]")
return result
except subprocess.CalledProcessError as e:
console.print(f"[red]Command failed: {e}[/red]")
if e.stderr:
console.print(f"[red]Error output: {e.stderr}[/red]")
raise RcloneInstallError(f"Command failed: {e}") from e
except FileNotFoundError as e:
raise RcloneInstallError(f"Command not found: {' '.join(command)}") from e
def install_rclone_macos() -> None:
"""Install rclone on macOS using Homebrew or official script."""
# Try Homebrew first
if shutil.which("brew"):
try:
console.print("[blue]Installing rclone via Homebrew...[/blue]")
run_command(["brew", "install", "rclone"])
console.print("[green]✓ rclone installed via Homebrew[/green]")
return
except RcloneInstallError:
console.print(
"[yellow]Homebrew installation failed, trying official script...[/yellow]"
)
# Fallback to official script
console.print("[blue]Installing rclone via official script...[/blue]")
try:
run_command(["sh", "-c", "curl https://rclone.org/install.sh | sudo bash"])
console.print("[green]✓ rclone installed via official script[/green]")
except RcloneInstallError:
raise RcloneInstallError(
"Failed to install rclone. Please install manually: brew install rclone"
)
def install_rclone_linux() -> None:
"""Install rclone on Linux using package managers or official script."""
# Try snap first (most universal)
if shutil.which("snap"):
try:
console.print("[blue]Installing rclone via snap...[/blue]")
run_command(["sudo", "snap", "install", "rclone"])
console.print("[green]✓ rclone installed via snap[/green]")
return
except RcloneInstallError:
console.print("[yellow]Snap installation failed, trying apt...[/yellow]")
# Try apt (Debian/Ubuntu)
if shutil.which("apt"):
try:
console.print("[blue]Installing rclone via apt...[/blue]")
run_command(["sudo", "apt", "update"])
run_command(["sudo", "apt", "install", "-y", "rclone"])
console.print("[green]✓ rclone installed via apt[/green]")
return
except RcloneInstallError:
console.print("[yellow]apt installation failed, trying official script...[/yellow]")
# Fallback to official script
console.print("[blue]Installing rclone via official script...[/blue]")
try:
run_command(["sh", "-c", "curl https://rclone.org/install.sh | sudo bash"])
console.print("[green]✓ rclone installed via official script[/green]")
except RcloneInstallError:
raise RcloneInstallError(
"Failed to install rclone. Please install manually: sudo snap install rclone"
)
def install_rclone_windows() -> None:
"""Install rclone on Windows using package managers."""
# Try winget first (built into Windows 10+)
if shutil.which("winget"):
try:
console.print("[blue]Installing rclone via winget...[/blue]")
run_command(["winget", "install", "Rclone.Rclone"])
console.print("[green]✓ rclone installed via winget[/green]")
return
except RcloneInstallError:
console.print("[yellow]winget installation failed, trying chocolatey...[/yellow]")
# Try chocolatey
if shutil.which("choco"):
try:
console.print("[blue]Installing rclone via chocolatey...[/blue]")
run_command(["choco", "install", "rclone", "-y"])
console.print("[green]✓ rclone installed via chocolatey[/green]")
return
except RcloneInstallError:
console.print("[yellow]chocolatey installation failed, trying scoop...[/yellow]")
# Try scoop
if shutil.which("scoop"):
try:
console.print("[blue]Installing rclone via scoop...[/blue]")
run_command(["scoop", "install", "rclone"])
console.print("[green]✓ rclone installed via scoop[/green]")
return
except RcloneInstallError:
console.print("[yellow]scoop installation failed[/yellow]")
# No package manager available
raise RcloneInstallError(
"Could not install rclone automatically. Please install a package manager "
"(winget, chocolatey, or scoop) or install rclone manually from https://rclone.org/downloads/"
)
def install_rclone(platform_override: Optional[str] = None) -> None:
"""Install rclone for the current platform."""
if is_rclone_installed():
console.print("[green]rclone is already installed[/green]")
return
platform_name = platform_override or get_platform()
console.print(f"[blue]Installing rclone for {platform_name}...[/blue]")
try:
if platform_name == "macos":
install_rclone_macos()
elif platform_name == "linux":
install_rclone_linux()
elif platform_name == "windows":
install_rclone_windows()
else:
raise RcloneInstallError(f"Unsupported platform: {platform_name}")
# Verify installation
if not is_rclone_installed():
raise RcloneInstallError("rclone installation completed but command not found in PATH")
console.print("[green]✓ rclone installation completed successfully[/green]")
except RcloneInstallError:
raise
except Exception as e:
raise RcloneInstallError(f"Unexpected error during installation: {e}") from e
def get_rclone_version() -> Optional[str]:
"""Get the installed rclone version."""
if not is_rclone_installed():
return None
try:
result = run_command(["rclone", "version"], check=False)
if result.returncode == 0:
# Parse version from output (format: "rclone v1.64.0")
lines = result.stdout.strip().split("\n")
for line in lines:
if line.startswith("rclone v"):
return line.split()[1]
return "unknown"
except Exception:
return "unknown"
@@ -0,0 +1,60 @@
"""utility functions for commands"""
from typing import Optional
from mcp.server.fastmcp.exceptions import ToolError
import typer
from rich.console import Console
from basic_memory.cli.commands.cloud import get_authenticated_headers
from basic_memory.mcp.async_client import client
from basic_memory.mcp.tools.utils import call_post, call_get
from basic_memory.mcp.project_context import get_active_project
from basic_memory.schemas import ProjectInfoResponse
console = Console()
async def run_sync(project: Optional[str] = None):
"""Run sync operation via API endpoint."""
try:
from basic_memory.config import ConfigManager
config = ConfigManager().config
auth_headers = {}
if config.cloud_mode_enabled:
auth_headers = await get_authenticated_headers()
project_item = await get_active_project(client, project, None, headers=auth_headers)
response = await call_post(
client, f"{project_item.project_url}/project/sync", headers=auth_headers
)
data = response.json()
console.print(f"[green]✓ {data['message']}[/green]")
except (ToolError, ValueError) as e:
console.print(f"[red]✗ Sync failed: {e}[/red]")
raise typer.Exit(1)
async def get_project_info(project: str):
"""Run sync operation via API endpoint."""
try:
from basic_memory.config import ConfigManager
config = ConfigManager().config
auth_headers = {}
if config.cloud_mode_enabled:
auth_headers = await get_authenticated_headers()
project_item = await get_active_project(client, project, None, headers=auth_headers)
response = await call_get(
client, f"{project_item.project_url}/project/info", headers=auth_headers
)
return ProjectInfoResponse.model_validate(response.json())
except (ToolError, ValueError) as e:
console.print(f"[red]✗ Sync failed: {e}[/red]")
raise typer.Exit(1)
+5 -5
View File
@@ -1,14 +1,13 @@
"""Database management commands."""
import asyncio
from pathlib import Path
import typer
from loguru import logger
from basic_memory import db
from basic_memory.cli.app import app
from basic_memory.config import app_config, config_manager
from basic_memory.config import ConfigManager, BasicMemoryConfig, save_basic_memory_config
@app.command()
@@ -18,6 +17,8 @@ def reset(
"""Reset database (drop all tables and recreate)."""
if typer.confirm("This will delete all data in your db. Are you sure?"):
logger.info("Resetting database...")
config_manager = ConfigManager()
app_config = config_manager.config
# Get database path
db_path = app_config.app_database_path
@@ -27,9 +28,8 @@ def reset(
logger.info(f"Database file deleted: {db_path}")
# Reset project configuration
config_manager.config.projects = {"main": str(Path.home() / "basic-memory")}
config_manager.config.default_project = "main"
config_manager.save_config(config_manager.config)
config = BasicMemoryConfig()
save_basic_memory_config(config_manager.config_file, config)
logger.info("Project configuration reset to default")
# Create a new empty database
@@ -7,7 +7,7 @@ from typing import Annotated
import typer
from basic_memory.cli.app import import_app
from basic_memory.config import config
from basic_memory.config import get_project_config
from basic_memory.importers import ChatGPTImporter
from basic_memory.markdown import EntityParser, MarkdownProcessor
from loguru import logger
@@ -19,6 +19,7 @@ console = Console()
async def get_markdown_processor() -> MarkdownProcessor:
"""Get MarkdownProcessor instance."""
config = get_project_config()
entity_parser = EntityParser(config.home)
return MarkdownProcessor(entity_parser)
@@ -49,7 +50,7 @@ def import_chatgpt(
# Get markdown processor
markdown_processor = asyncio.run(get_markdown_processor())
config = get_project_config()
# Process the file
base_path = config.home / folder
console.print(f"\nImporting chats from {conversations_json}...writing to {base_path}")
@@ -7,7 +7,7 @@ from typing import Annotated
import typer
from basic_memory.cli.app import claude_app
from basic_memory.config import config
from basic_memory.config import get_project_config
from basic_memory.importers.claude_conversations_importer import ClaudeConversationsImporter
from basic_memory.markdown import EntityParser, MarkdownProcessor
from loguru import logger
@@ -19,6 +19,7 @@ console = Console()
async def get_markdown_processor() -> MarkdownProcessor:
"""Get MarkdownProcessor instance."""
config = get_project_config()
entity_parser = EntityParser(config.home)
return MarkdownProcessor(entity_parser)
@@ -42,6 +43,7 @@ def import_claude(
After importing, run 'basic-memory sync' to index the new files.
"""
config = get_project_config()
try:
if not conversations_json.exists():
typer.echo(f"Error: File not found: {conversations_json}", err=True)
@@ -7,7 +7,7 @@ from typing import Annotated
import typer
from basic_memory.cli.app import claude_app
from basic_memory.config import config
from basic_memory.config import get_project_config
from basic_memory.importers.claude_projects_importer import ClaudeProjectsImporter
from basic_memory.markdown import EntityParser, MarkdownProcessor
from loguru import logger
@@ -19,6 +19,7 @@ console = Console()
async def get_markdown_processor() -> MarkdownProcessor:
"""Get MarkdownProcessor instance."""
config = get_project_config()
entity_parser = EntityParser(config.home)
return MarkdownProcessor(entity_parser)
@@ -41,6 +42,7 @@ def import_projects(
After importing, run 'basic-memory sync' to index the new files.
"""
config = get_project_config()
try:
if not projects_json.exists():
typer.echo(f"Error: File not found: {projects_json}", err=True)
@@ -7,7 +7,7 @@ from typing import Annotated
import typer
from basic_memory.cli.app import import_app
from basic_memory.config import config
from basic_memory.config import get_project_config
from basic_memory.importers.memory_json_importer import MemoryJsonImporter
from basic_memory.markdown import EntityParser, MarkdownProcessor
from loguru import logger
@@ -19,6 +19,7 @@ console = Console()
async def get_markdown_processor() -> MarkdownProcessor:
"""Get MarkdownProcessor instance."""
config = get_project_config()
entity_parser = EntityParser(config.home)
return MarkdownProcessor(entity_parser)
@@ -38,14 +39,13 @@ def memory_json(
1. Read entities and relations from the JSON file
2. Create markdown files for each entity
3. Include outgoing relations in each entity's markdown
After importing, run 'basic-memory sync' to index the new files.
"""
if not json_path.exists():
typer.echo(f"Error: File not found: {json_path}", err=True)
raise typer.Exit(1)
config = get_project_config()
try:
# Get markdown processor
markdown_processor = asyncio.run(get_markdown_processor())
@@ -74,13 +74,12 @@ def memory_json(
Panel(
f"[green]Import complete![/green]\n\n"
f"Created {result.entities} entities\n"
f"Added {result.relations} relations",
f"Added {result.relations} relations\n"
f"Skipped {result.skipped_entities} entities\n",
expand=False,
)
)
console.print("\nRun 'basic-memory sync' to index the new files.")
except Exception as e:
logger.error("Import failed")
typer.echo(f"Error during import: {e}", err=True)
+17 -17
View File
@@ -1,9 +1,12 @@
"""MCP server command with streamable HTTP transport."""
import asyncio
import os
import typer
from typing import Optional
from basic_memory.cli.app import app
from basic_memory.config import ConfigManager
# Import mcp instance
from basic_memory.mcp.server import mcp as mcp_server # pragma: no cover
@@ -14,6 +17,8 @@ import basic_memory.mcp.tools # noqa: F401 # pragma: no cover
# Import prompts to register them
import basic_memory.mcp.prompts # noqa: F401 # pragma: no cover
from loguru import logger
import threading
from basic_memory.services.initialization import initialize_file_sync
@app.command()
@@ -24,6 +29,7 @@ def mcp(
),
port: int = typer.Option(8000, help="Port for HTTP transports"),
path: str = typer.Option("/mcp", help="Path prefix for streamable-http transport"),
project: Optional[str] = typer.Option(None, help="Restrict MCP server to single project"),
): # pragma: no cover
"""Run the MCP server with configurable transport options.
@@ -34,25 +40,19 @@ def mcp(
- sse: Server-Sent Events (for compatibility with existing clients)
"""
# Check if OAuth is enabled
import os
# Validate and set project constraint if specified
if project:
config_manager = ConfigManager()
project_name, _ = config_manager.get_project(project)
if not project_name:
typer.echo(f"No project found named: {project}", err=True)
raise typer.Exit(1)
auth_enabled = os.getenv("FASTMCP_AUTH_ENABLED", "false").lower() == "true"
if auth_enabled:
logger.info("OAuth authentication is ENABLED")
logger.info(f"Issuer URL: {os.getenv('FASTMCP_AUTH_ISSUER_URL', 'http://localhost:8000')}")
if os.getenv("FASTMCP_AUTH_REQUIRED_SCOPES"):
logger.info(f"Required scopes: {os.getenv('FASTMCP_AUTH_REQUIRED_SCOPES')}")
else:
logger.info("OAuth authentication is DISABLED")
# Set env var with validated project name
os.environ["BASIC_MEMORY_MCP_PROJECT"] = project_name
logger.info(f"MCP server constrained to project: {project_name}")
from basic_memory.config import app_config
from basic_memory.services.initialization import initialize_file_sync
# Start the MCP server with the specified transport
# Use unified thread-based sync approach for both transports
import threading
app_config = ConfigManager().config
def run_file_sync():
"""Run file sync in a separate thread with its own event loop."""
+138 -101
View File
@@ -9,13 +9,13 @@ from rich.console import Console
from rich.table import Table
from basic_memory.cli.app import app
from basic_memory.mcp.project_session import session
from basic_memory.mcp.resources.project_info import project_info
from basic_memory.cli.commands.cloud import get_authenticated_headers
from basic_memory.cli.commands.command_utils import get_project_info
from basic_memory.config import ConfigManager
import json
from datetime import datetime
from rich.panel import Panel
from rich.tree import Tree
from basic_memory.mcp.async_client import client
from basic_memory.mcp.tools.utils import call_get
from basic_memory.schemas.project_info import ProjectList
@@ -24,6 +24,7 @@ from basic_memory.schemas.project_info import ProjectStatusResponse
from basic_memory.mcp.tools.utils import call_delete
from basic_memory.mcp.tools.utils import call_put
from basic_memory.utils import generate_permalink
from basic_memory.mcp.tools.utils import call_patch
console = Console()
@@ -31,6 +32,8 @@ console = Console()
project_app = typer.Typer(help="Manage multiple Basic Memory projects")
app.add_typer(project_app, name="project")
config = ConfigManager().config
def format_path(path: str) -> str:
"""Format a path for display, using ~ for home directory."""
@@ -42,22 +45,24 @@ def format_path(path: str) -> str:
@project_app.command("list")
def list_projects() -> None:
"""List all configured projects."""
"""List all Basic Memory projects."""
# Use API to list projects
try:
response = asyncio.run(call_get(client, "/projects/projects"))
auth_headers = {}
if config.cloud_mode_enabled:
auth_headers = asyncio.run(get_authenticated_headers())
response = asyncio.run(call_get(client, "/projects/projects", headers=auth_headers))
result = ProjectList.model_validate(response.json())
table = Table(title="Basic Memory Projects")
table.add_column("Name", style="cyan")
table.add_column("Path", style="green")
table.add_column("Default", style="yellow")
table.add_column("Active", style="magenta")
table.add_column("Default", style="magenta")
for project in result.projects:
is_default = "" if project.is_default else ""
is_active = "" if session.get_current_project() == project.name else ""
table.add_row(project.name, format_path(project.path), is_default, is_active)
table.add_row(project.name, format_path(project.path), is_default)
console.print(table)
except Exception as e:
@@ -65,42 +70,75 @@ def list_projects() -> None:
raise typer.Exit(1)
@project_app.command("add")
def add_project(
name: str = typer.Argument(..., help="Name of the project"),
path: str = typer.Argument(..., help="Path to the project directory"),
set_default: bool = typer.Option(False, "--default", help="Set as default project"),
) -> None:
"""Add a new project."""
# Resolve to absolute path
resolved_path = os.path.abspath(os.path.expanduser(path))
if config.cloud_mode_enabled:
try:
data = {"name": name, "path": resolved_path, "set_default": set_default}
@project_app.command("add")
def add_project_cloud(
name: str = typer.Argument(..., help="Name of the project"),
set_default: bool = typer.Option(False, "--default", help="Set as default project"),
) -> None:
"""Add a new project to Basic Memory Cloud"""
response = asyncio.run(call_post(client, "/projects/projects", json=data))
result = ProjectStatusResponse.model_validate(response.json())
try:
auth_headers = asyncio.run(get_authenticated_headers())
console.print(f"[green]{result.message}[/green]")
except Exception as e:
console.print(f"[red]Error adding project: {str(e)}[/red]")
raise typer.Exit(1)
data = {"name": name, "path": generate_permalink(name), "set_default": set_default}
# Display usage hint
console.print("\nTo use this project:")
console.print(f" basic-memory --project={name} <command>")
console.print(" # or")
console.print(f" basic-memory project default {name}")
response = asyncio.run(
call_post(client, "/projects/projects", json=data, headers=auth_headers)
)
result = ProjectStatusResponse.model_validate(response.json())
console.print(f"[green]{result.message}[/green]")
except Exception as e:
console.print(f"[red]Error adding project: {str(e)}[/red]")
raise typer.Exit(1)
# Display usage hint
console.print("\nTo use this project:")
console.print(f" basic-memory --project={name} <command>")
else:
@project_app.command("add")
def add_project(
name: str = typer.Argument(..., help="Name of the project"),
path: str = typer.Argument(..., help="Path to the project directory"),
set_default: bool = typer.Option(False, "--default", help="Set as default project"),
) -> None:
"""Add a new project."""
# Resolve to absolute path
resolved_path = Path(os.path.abspath(os.path.expanduser(path))).as_posix()
try:
data = {"name": name, "path": resolved_path, "set_default": set_default}
response = asyncio.run(call_post(client, "/projects/projects", json=data))
result = ProjectStatusResponse.model_validate(response.json())
console.print(f"[green]{result.message}[/green]")
except Exception as e:
console.print(f"[red]Error adding project: {str(e)}[/red]")
raise typer.Exit(1)
# Display usage hint
console.print("\nTo use this project:")
console.print(f" basic-memory --project={name} <command>")
@project_app.command("remove")
def remove_project(
name: str = typer.Argument(..., help="Name of the project to remove"),
) -> None:
"""Remove a project from configuration."""
"""Remove a project."""
try:
project_name = generate_permalink(name)
response = asyncio.run(call_delete(client, f"/projects/{project_name}"))
auth_headers = {}
if config.cloud_mode_enabled:
auth_headers = asyncio.run(get_authenticated_headers())
project_permalink = generate_permalink(name)
response = asyncio.run(
call_delete(client, f"/projects/{project_permalink}", headers=auth_headers)
)
result = ProjectStatusResponse.model_validate(response.json())
console.print(f"[green]{result.message}[/green]")
@@ -112,61 +150,96 @@ def remove_project(
console.print("[yellow]Note: The project files have not been deleted from disk.[/yellow]")
@project_app.command("default")
def set_default_project(
name: str = typer.Argument(..., help="Name of the project to set as default"),
) -> None:
"""Set the default project and activate it for the current session."""
try:
project_name = generate_permalink(name)
if not config.cloud_mode_enabled:
response = asyncio.run(call_put(client, f"/projects/{project_name}/default"))
result = ProjectStatusResponse.model_validate(response.json())
@project_app.command("default")
def set_default_project(
name: str = typer.Argument(..., help="Name of the project to set as CLI default"),
) -> None:
"""Set the default project when 'config.default_project_mode' is set."""
try:
project_permalink = generate_permalink(name)
response = asyncio.run(call_put(client, f"/projects/{project_permalink}/default"))
result = ProjectStatusResponse.model_validate(response.json())
console.print(f"[green]{result.message}[/green]")
except Exception as e:
console.print(f"[red]Error setting default project: {str(e)}[/red]")
raise typer.Exit(1)
console.print(f"[green]{result.message}[/green]")
except Exception as e:
console.print(f"[red]Error setting default project: {str(e)}[/red]")
raise typer.Exit(1)
# The API call above should have updated both config and MCP session
# No need for manual reload - the project service handles this automatically
console.print("[green]Project activated for current session[/green]")
@project_app.command("sync-config")
def synchronize_projects() -> None:
"""Synchronize project config between configuration file and database."""
# Call the API to synchronize projects
try:
response = asyncio.run(call_post(client, "/projects/config/sync"))
result = ProjectStatusResponse.model_validate(response.json())
@project_app.command("sync-config")
def synchronize_projects() -> None:
"""Synchronize project config between configuration file and database."""
# Call the API to synchronize projects
console.print(f"[green]{result.message}[/green]")
except Exception as e: # pragma: no cover
console.print(f"[red]Error synchronizing projects: {str(e)}[/red]")
raise typer.Exit(1)
try:
response = asyncio.run(call_post(client, "/projects/sync"))
result = ProjectStatusResponse.model_validate(response.json())
@project_app.command("move")
def move_project(
name: str = typer.Argument(..., help="Name of the project to move"),
new_path: str = typer.Argument(..., help="New absolute path for the project"),
) -> None:
"""Move a project to a new location."""
# Resolve to absolute path
resolved_path = Path(os.path.abspath(os.path.expanduser(new_path))).as_posix()
console.print(f"[green]{result.message}[/green]")
except Exception as e: # pragma: no cover
console.print(f"[red]Error synchronizing projects: {str(e)}[/red]")
raise typer.Exit(1)
try:
data = {"path": resolved_path}
project_permalink = generate_permalink(name)
# TODO fix route to use ProjectPathDep
response = asyncio.run(
call_patch(client, f"/{name}/project/{project_permalink}", json=data)
)
result = ProjectStatusResponse.model_validate(response.json())
console.print(f"[green]{result.message}[/green]")
# Show important file movement reminder
console.print() # Empty line for spacing
console.print(
Panel(
"[bold red]IMPORTANT:[/bold red] Project configuration updated successfully.\n\n"
"[yellow]You must manually move your project files from the old location to:[/yellow]\n"
f"[cyan]{resolved_path}[/cyan]\n\n"
"[dim]Basic Memory has only updated the configuration - your files remain in their original location.[/dim]",
title="⚠️ Manual File Movement Required",
border_style="yellow",
expand=False,
)
)
except Exception as e:
console.print(f"[red]Error moving project: {str(e)}[/red]")
raise typer.Exit(1)
@project_app.command("info")
def display_project_info(
name: str = typer.Argument(..., help="Name of the project"),
json_output: bool = typer.Option(False, "--json", help="Output in JSON format"),
):
"""Display detailed information and statistics about the current project."""
try:
# Get project info
info = asyncio.run(project_info.fn()) # type: ignore # pyright: ignore [reportAttributeAccessIssue]
info = asyncio.run(get_project_info(name))
if json_output:
# Convert to JSON and print
print(json.dumps(info.model_dump(), indent=2, default=str))
else:
# Create rich display
console = Console()
# Project configuration section
console.print(
Panel(
f"Basic Memory version: [bold green]{info.system.version}[/bold green]\n"
f"[bold]Project:[/bold] {info.project_name}\n"
f"[bold]Path:[/bold] {info.project_path}\n"
f"[bold]Default Project:[/bold] {info.default_project}\n",
@@ -236,42 +309,6 @@ def display_project_info(
console.print(recent_table)
# System status
system_tree = Tree("🖥️ System Status")
system_tree.add(f"Basic Memory version: [bold green]{info.system.version}[/bold green]")
system_tree.add(
f"Database: [cyan]{info.system.database_path}[/cyan] ([green]{info.system.database_size}[/green])"
)
# Watch status
if info.system.watch_status: # pragma: no cover
watch_branch = system_tree.add("Watch Service")
running = info.system.watch_status.get("running", False)
status_color = "green" if running else "red"
watch_branch.add(
f"Status: [bold {status_color}]{'Running' if running else 'Stopped'}[/bold {status_color}]"
)
if running:
start_time = (
datetime.fromisoformat(info.system.watch_status.get("start_time", ""))
if isinstance(info.system.watch_status.get("start_time"), str)
else info.system.watch_status.get("start_time")
)
watch_branch.add(
f"Running since: [cyan]{start_time.strftime('%Y-%m-%d %H:%M')}[/cyan]"
)
watch_branch.add(
f"Files synced: [green]{info.system.watch_status.get('synced_files', 0)}[/green]"
)
watch_branch.add(
f"Errors: [{'red' if info.system.watch_status.get('error_count', 0) > 0 else 'green'}]{info.system.watch_status.get('error_count', 0)}[/{'red' if info.system.watch_status.get('error_count', 0) > 0 else 'green'}]"
)
else:
system_tree.add("[yellow]Watch service not running[/yellow]")
console.print(system_tree)
# Available projects
projects_table = Table(title="📁 Available Projects")
projects_table.add_column("Name", style="blue")
+35 -20
View File
@@ -2,19 +2,21 @@
import asyncio
from typing import Set, Dict
from typing import Annotated, Optional
from mcp.server.fastmcp.exceptions import ToolError
import typer
from loguru import logger
from rich.console import Console
from rich.panel import Panel
from rich.tree import Tree
from basic_memory import db
from basic_memory.cli.app import app
from basic_memory.cli.commands.sync import get_sync_service
from basic_memory.config import config, app_config
from basic_memory.repository import ProjectRepository
from basic_memory.sync.sync_service import SyncReport
from basic_memory.cli.commands.cloud import get_authenticated_headers
from basic_memory.mcp.async_client import client
from basic_memory.mcp.tools.utils import call_post
from basic_memory.schemas import SyncReportResponse
from basic_memory.mcp.project_context import get_active_project
# Create rich console
console = Console()
@@ -47,7 +49,7 @@ def add_files_to_tree(
branch.add(f"[{style}]{file_name}[/{style}]")
def group_changes_by_directory(changes: SyncReport) -> Dict[str, Dict[str, int]]:
def group_changes_by_directory(changes: SyncReportResponse) -> Dict[str, Dict[str, int]]:
"""Group changes by directory for summary view."""
by_dir = {}
for change_type, paths in [
@@ -87,7 +89,9 @@ def build_directory_summary(counts: Dict[str, int]) -> str:
return " ".join(parts)
def display_changes(project_name: str, title: str, changes: SyncReport, verbose: bool = False):
def display_changes(
project_name: str, title: str, changes: SyncReportResponse, verbose: bool = False
):
"""Display changes using Rich for better visualization."""
tree = Tree(f"{project_name}: {title}")
@@ -122,30 +126,41 @@ def display_changes(project_name: str, title: str, changes: SyncReport, verbose:
console.print(Panel(tree, expand=False))
async def run_status(verbose: bool = False): # pragma: no cover
async def run_status(project: Optional[str] = None, verbose: bool = False): # pragma: no cover
"""Check sync status of files vs database."""
# Check knowledge/ directory
_, session_maker = await db.get_or_create_db(
db_path=app_config.database_path, db_type=db.DatabaseType.FILESYSTEM
)
project_repository = ProjectRepository(session_maker)
project = await project_repository.get_by_name(config.project)
if not project: # pragma: no cover
raise Exception(f"Project '{config.project}' not found")
try:
from basic_memory.config import ConfigManager
sync_service = await get_sync_service(project)
knowledge_changes = await sync_service.scan(config.home)
display_changes(project.name, "Status", knowledge_changes, verbose)
config = ConfigManager().config
auth_headers = {}
if config.cloud_mode_enabled:
auth_headers = await get_authenticated_headers()
project_item = await get_active_project(client, project, None)
response = await call_post(
client, f"{project_item.project_url}/project/status", headers=auth_headers
)
sync_report = SyncReportResponse.model_validate(response.json())
display_changes(project_item.name, "Status", sync_report, verbose)
except (ValueError, ToolError) as e:
console.print(f"[red]✗ Error: {e}[/red]")
raise typer.Exit(1)
@app.command()
def status(
project: Annotated[
Optional[str],
typer.Option(help="The project name."),
] = None,
verbose: bool = typer.Option(False, "--verbose", "-v", help="Show detailed file information"),
):
"""Show sync status between files and database."""
try:
asyncio.run(run_status(verbose)) # pragma: no cover
asyncio.run(run_status(project, verbose)) # pragma: no cover
except Exception as e:
logger.error(f"Error checking status: {e}")
typer.echo(f"Error checking status: {e}", err=True)
+44 -219
View File
@@ -1,234 +1,59 @@
"""Command module for basic-memory sync operations."""
import asyncio
from collections import defaultdict
from dataclasses import dataclass
from pathlib import Path
from typing import List, Dict
from typing import Annotated, Optional
import typer
from loguru import logger
from rich.console import Console
from rich.tree import Tree
from basic_memory import db
from basic_memory.cli.app import app
from basic_memory.config import config
from basic_memory.markdown import EntityParser
from basic_memory.markdown.markdown_processor import MarkdownProcessor
from basic_memory.models import Project
from basic_memory.repository import (
EntityRepository,
ObservationRepository,
RelationRepository,
ProjectRepository,
)
from basic_memory.repository.search_repository import SearchRepository
from basic_memory.services import EntityService, FileService
from basic_memory.services.link_resolver import LinkResolver
from basic_memory.services.search_service import SearchService
from basic_memory.sync import SyncService
from basic_memory.sync.sync_service import SyncReport
from basic_memory.config import app_config
console = Console()
@dataclass
class ValidationIssue:
file_path: str
error: str
async def get_sync_service(project: Project) -> SyncService: # pragma: no cover
"""Get sync service instance with all dependencies."""
_, session_maker = await db.get_or_create_db(
db_path=app_config.database_path, db_type=db.DatabaseType.FILESYSTEM
)
project_path = Path(project.path)
entity_parser = EntityParser(project_path)
markdown_processor = MarkdownProcessor(entity_parser)
file_service = FileService(project_path, markdown_processor)
# Initialize repositories
entity_repository = EntityRepository(session_maker, project_id=project.id)
observation_repository = ObservationRepository(session_maker, project_id=project.id)
relation_repository = RelationRepository(session_maker, project_id=project.id)
search_repository = SearchRepository(session_maker, project_id=project.id)
# Initialize services
search_service = SearchService(search_repository, entity_repository, file_service)
link_resolver = LinkResolver(entity_repository, search_service)
# Initialize services
entity_service = EntityService(
entity_parser,
entity_repository,
observation_repository,
relation_repository,
file_service,
link_resolver,
)
# Create sync service
sync_service = SyncService(
app_config=app_config,
entity_service=entity_service,
entity_parser=entity_parser,
entity_repository=entity_repository,
relation_repository=relation_repository,
search_service=search_service,
file_service=file_service,
)
return sync_service
def group_issues_by_directory(issues: List[ValidationIssue]) -> Dict[str, List[ValidationIssue]]:
"""Group validation issues by directory."""
grouped = defaultdict(list)
for issue in issues:
dir_name = Path(issue.file_path).parent.name
grouped[dir_name].append(issue)
return dict(grouped)
def display_sync_summary(knowledge: SyncReport):
"""Display a one-line summary of sync changes."""
total_changes = knowledge.total
project_name = config.project
if total_changes == 0:
console.print(f"[green]Project '{project_name}': Everything up to date[/green]")
return
# Format as: "Synced X files (A new, B modified, C moved, D deleted)"
changes = []
new_count = len(knowledge.new)
mod_count = len(knowledge.modified)
move_count = len(knowledge.moves)
del_count = len(knowledge.deleted)
if new_count:
changes.append(f"[green]{new_count} new[/green]")
if mod_count:
changes.append(f"[yellow]{mod_count} modified[/yellow]")
if move_count:
changes.append(f"[blue]{move_count} moved[/blue]")
if del_count:
changes.append(f"[red]{del_count} deleted[/red]")
console.print(f"Project '{project_name}': Synced {total_changes} files ({', '.join(changes)})")
def display_detailed_sync_results(knowledge: SyncReport):
"""Display detailed sync results with trees."""
project_name = config.project
if knowledge.total == 0:
console.print(f"\n[green]Project '{project_name}': Everything up to date[/green]")
return
console.print(f"\n[bold]Sync Results for Project '{project_name}'[/bold]")
if knowledge.total > 0:
knowledge_tree = Tree("[bold]Knowledge Files[/bold]")
if knowledge.new:
created = knowledge_tree.add("[green]Created[/green]")
for path in sorted(knowledge.new):
checksum = knowledge.checksums.get(path, "")
created.add(f"[green]{path}[/green] ({checksum[:8]})")
if knowledge.modified:
modified = knowledge_tree.add("[yellow]Modified[/yellow]")
for path in sorted(knowledge.modified):
checksum = knowledge.checksums.get(path, "")
modified.add(f"[yellow]{path}[/yellow] ({checksum[:8]})")
if knowledge.moves:
moved = knowledge_tree.add("[blue]Moved[/blue]")
for old_path, new_path in sorted(knowledge.moves.items()):
checksum = knowledge.checksums.get(new_path, "")
moved.add(f"[blue]{old_path}[/blue] → [blue]{new_path}[/blue] ({checksum[:8]})")
if knowledge.deleted:
deleted = knowledge_tree.add("[red]Deleted[/red]")
for path in sorted(knowledge.deleted):
deleted.add(f"[red]{path}[/red]")
console.print(knowledge_tree)
async def run_sync(verbose: bool = False):
"""Run sync operation."""
_, session_maker = await db.get_or_create_db(
db_path=app_config.database_path, db_type=db.DatabaseType.FILESYSTEM
)
project_repository = ProjectRepository(session_maker)
project = await project_repository.get_by_name(config.project)
if not project: # pragma: no cover
raise Exception(f"Project '{config.project}' not found")
import time
start_time = time.time()
logger.info(
"Sync command started",
project=config.project,
verbose=verbose,
directory=str(config.home),
)
sync_service = await get_sync_service(project)
logger.info("Running one-time sync")
knowledge_changes = await sync_service.sync(config.home, project_name=project.name)
# Log results
duration_ms = int((time.time() - start_time) * 1000)
logger.info(
"Sync command completed",
project=config.project,
total_changes=knowledge_changes.total,
new_files=len(knowledge_changes.new),
modified_files=len(knowledge_changes.modified),
deleted_files=len(knowledge_changes.deleted),
moved_files=len(knowledge_changes.moves),
duration_ms=duration_ms,
)
# Display results
if verbose:
display_detailed_sync_results(knowledge_changes)
else:
display_sync_summary(knowledge_changes) # pragma: no cover
from basic_memory.cli.commands.command_utils import run_sync
from basic_memory.config import ConfigManager
@app.command()
def sync(
verbose: bool = typer.Option(
False,
"--verbose",
"-v",
help="Show detailed sync information.",
),
project: Annotated[
Optional[str],
typer.Option(help="The project name."),
] = None,
watch: Annotated[
bool,
typer.Option("--watch", help="Run continuous sync (cloud mode only)"),
] = False,
interval: Annotated[
int,
typer.Option("--interval", help="Sync interval in seconds for watch mode (default: 60)"),
] = 60,
) -> None:
"""Sync knowledge files with the database."""
try:
# Show which project we're syncing
typer.echo(f"Syncing project: {config.project}")
typer.echo(f"Project path: {config.home}")
"""Sync knowledge files with the database.
# Run sync
asyncio.run(run_sync(verbose=verbose))
In local mode: Scans filesystem and updates database.
In cloud mode: Runs bidirectional file sync (bisync) then updates database.
except Exception as e: # pragma: no cover
if not isinstance(e, typer.Exit):
logger.exception(
"Sync command failed",
f"project={config.project},"
f"error={str(e)},"
f"error_type={type(e).__name__},"
f"directory={str(config.home)}",
)
typer.echo(f"Error during sync: {e}", err=True)
Examples:
bm sync # One-time sync
bm sync --watch # Continuous sync every 60s
bm sync --watch --interval 30 # Continuous sync every 30s
"""
config = ConfigManager().config
if config.cloud_mode_enabled:
# Cloud mode: run bisync which includes database sync
from basic_memory.cli.commands.cloud.bisync_commands import run_bisync, run_bisync_watch
try:
if watch:
run_bisync_watch(interval_seconds=interval)
else:
run_bisync()
except Exception:
raise typer.Exit(1)
raise
else:
# Local mode: just database sync
if watch:
typer.echo(
"Error: --watch is only available in cloud mode. Run 'bm cloud login' first."
)
raise typer.Exit(1)
asyncio.run(run_sync(project))
+87 -16
View File
@@ -9,6 +9,7 @@ from loguru import logger
from rich import print as rprint
from basic_memory.cli.app import app
from basic_memory.config import ConfigManager
# Import prompts
from basic_memory.mcp.prompts.continue_conversation import (
@@ -34,6 +35,12 @@ app.add_typer(tool_app, name="tool", help="Access to MCP tools via CLI")
def write_note(
title: Annotated[str, typer.Option(help="The title of the note")],
folder: Annotated[str, typer.Option(help="The folder to create the note in")],
project: Annotated[
Optional[str],
typer.Option(
help="The project to write to. If not provided, the default project will be used."
),
] = None,
content: Annotated[
Optional[str],
typer.Option(
@@ -90,7 +97,19 @@ def write_note(
typer.echo("Empty content provided. Please provide non-empty content.", err=True)
raise typer.Exit(1)
note = asyncio.run(mcp_write_note.fn(title, content, folder, tags))
# look for the project in the config
config_manager = ConfigManager()
project_name = None
if project is not None:
project_name, _ = config_manager.get_project(project)
if not project_name:
typer.echo(f"No project found named: {project}", err=True)
raise typer.Exit(1)
# use the project name, or the default from the config
project_name = project_name or config_manager.default_project
note = asyncio.run(mcp_write_note.fn(title, content, folder, project_name, tags))
rprint(note)
except Exception as e: # pragma: no cover
if not isinstance(e, typer.Exit):
@@ -100,10 +119,33 @@ def write_note(
@tool_app.command()
def read_note(identifier: str, page: int = 1, page_size: int = 10):
def read_note(
identifier: str,
project: Annotated[
Optional[str],
typer.Option(
help="The project to use for the note. If not provided, the default project will be used."
),
] = None,
page: int = 1,
page_size: int = 10,
):
"""Read a markdown note from the knowledge base."""
# look for the project in the config
config_manager = ConfigManager()
project_name = None
if project is not None:
project_name, _ = config_manager.get_project(project)
if not project_name:
typer.echo(f"No project found named: {project}", err=True)
raise typer.Exit(1)
# use the project name, or the default from the config
project_name = project_name or config_manager.default_project
try:
note = asyncio.run(mcp_read_note.fn(identifier, page, page_size))
note = asyncio.run(mcp_read_note.fn(identifier, project_name, page, page_size))
rprint(note)
except Exception as e: # pragma: no cover
if not isinstance(e, typer.Exit):
@@ -115,6 +157,10 @@ def read_note(identifier: str, page: int = 1, page_size: int = 10):
@tool_app.command()
def build_context(
url: MemoryUrl,
project: Annotated[
Optional[str],
typer.Option(help="The project to use. If not provided, the default project will be used."),
] = None,
depth: Optional[int] = 1,
timeframe: Optional[TimeFrame] = "7d",
page: int = 1,
@@ -122,9 +168,23 @@ def build_context(
max_related: int = 10,
):
"""Get context needed to continue a discussion."""
# look for the project in the config
config_manager = ConfigManager()
project_name = None
if project is not None:
project_name, _ = config_manager.get_project(project)
if not project_name:
typer.echo(f"No project found named: {project}", err=True)
raise typer.Exit(1)
# use the project name, or the default from the config
project_name = project_name or config_manager.default_project
try:
context = asyncio.run(
mcp_build_context.fn(
project=project_name,
url=url,
depth=depth,
timeframe=timeframe,
@@ -150,30 +210,21 @@ def recent_activity(
type: Annotated[Optional[List[SearchItemType]], typer.Option()] = None,
depth: Optional[int] = 1,
timeframe: Optional[TimeFrame] = "7d",
page: int = 1,
page_size: int = 10,
max_related: int = 10,
):
"""Get recent activity across the knowledge base."""
try:
context = asyncio.run(
result = asyncio.run(
mcp_recent_activity.fn(
type=type, # pyright: ignore [reportArgumentType]
depth=depth,
timeframe=timeframe,
page=page,
page_size=page_size,
max_related=max_related,
)
)
# Use json module for more controlled serialization
import json
context_dict = context.model_dump(exclude_none=True)
print(json.dumps(context_dict, indent=2, ensure_ascii=True, default=str))
# The tool now returns a formatted string directly
print(result)
except Exception as e: # pragma: no cover
if not isinstance(e, typer.Exit):
typer.echo(f"Error during build_context: {e}", err=True)
typer.echo(f"Error during recent_activity: {e}", err=True)
raise typer.Exit(1)
raise
@@ -183,6 +234,12 @@ def search_notes(
query: str,
permalink: Annotated[bool, typer.Option("--permalink", help="Search permalink values")] = False,
title: Annotated[bool, typer.Option("--title", help="Search title values")] = False,
project: Annotated[
Optional[str],
typer.Option(
help="The project to use for the note. If not provided, the default project will be used."
),
] = None,
after_date: Annotated[
Optional[str],
typer.Option("--after_date", help="Search results after date, eg. '2d', '1 week'"),
@@ -191,6 +248,19 @@ def search_notes(
page_size: int = 10,
):
"""Search across all content in the knowledge base."""
# look for the project in the config
config_manager = ConfigManager()
project_name = None
if project is not None:
project_name, _ = config_manager.get_project(project)
if not project_name:
typer.echo(f"No project found named: {project}", err=True)
raise typer.Exit(1)
# use the project name, or the default from the config
project_name = project_name or config_manager.default_project
if permalink and title: # pragma: no cover
print("Cannot search both permalink and title")
raise typer.Abort()
@@ -212,6 +282,7 @@ def search_notes(
results = asyncio.run(
mcp_search.fn(
query,
project_name,
search_type=search_type,
page=page,
after_date=after_date,
+1 -1
View File
@@ -4,7 +4,7 @@ from basic_memory.cli.app import app # pragma: no cover
# Register commands
from basic_memory.cli.commands import ( # noqa: F401 # pragma: no cover
auth,
cloud,
db,
import_chatgpt,
import_claude_conversations,
+137 -46
View File
@@ -46,7 +46,7 @@ class BasicMemoryConfig(BaseSettings):
projects: Dict[str, str] = Field(
default_factory=lambda: {
"main": str(Path(os.getenv("BASIC_MEMORY_HOME", Path.home() / "basic-memory")))
"main": Path(os.getenv("BASIC_MEMORY_HOME", Path.home() / "basic-memory")).as_posix()
},
description="Mapping of project names to their filesystem paths",
)
@@ -54,6 +54,10 @@ class BasicMemoryConfig(BaseSettings):
default="main",
description="Name of the default project to use",
)
default_project_mode: bool = Field(
default=False,
description="When True, MCP tools automatically use default_project when no project parameter is specified. Enables simplified UX for single-project workflows.",
)
# overridden by ~/.basic-memory/config.json
log_level: str = "INFO"
@@ -63,6 +67,10 @@ class BasicMemoryConfig(BaseSettings):
default=1000, description="Milliseconds to wait after changes before syncing", gt=0
)
watch_project_reload_interval: int = Field(
default=30, description="Seconds between reloading project list in watch service", gt=0
)
# update permalinks on move
update_permalinks_on_move: bool = Field(
default=False,
@@ -74,11 +82,83 @@ class BasicMemoryConfig(BaseSettings):
description="Whether to sync changes in real time. default (True)",
)
sync_thread_pool_size: int = Field(
default=4,
description="Size of thread pool for file I/O operations in sync service",
gt=0,
)
kebab_filenames: bool = Field(
default=False,
description="Format for generated filenames. False preserves spaces and special chars, True converts them to hyphens for consistency with permalinks",
)
disable_permalinks: bool = Field(
default=False,
description="Disable automatic permalink generation in frontmatter. When enabled, new notes won't have permalinks added and sync won't update permalinks. Existing permalinks will still work for reading.",
)
skip_initialization_sync: bool = Field(
default=False,
description="Skip expensive initialization synchronization. Useful for cloud/stateless deployments where project reconciliation is not needed.",
)
# API connection configuration
api_url: Optional[str] = Field(
default=None,
description="URL of remote Basic Memory API. If set, MCP will connect to this API instead of using local ASGI transport.",
)
# Cloud configuration
cloud_client_id: str = Field(
default="client_01K6KWQPW6J1M8VV7R3TZP5A6M",
description="OAuth client ID for Basic Memory Cloud",
)
cloud_domain: str = Field(
default="https://eloquent-lotus-05.authkit.app",
description="AuthKit domain for Basic Memory Cloud",
)
cloud_host: str = Field(
default_factory=lambda: os.getenv(
"BASIC_MEMORY_CLOUD_HOST", "https://cloud.basicmemory.com"
),
description="Basic Memory Cloud host URL",
)
cloud_mode: bool = Field(
default=False,
description="Enable cloud mode - all requests go to cloud instead of local (config file value)",
)
@property
def cloud_mode_enabled(self) -> bool:
"""Check if cloud mode is enabled.
Priority:
1. BASIC_MEMORY_CLOUD_MODE environment variable
2. Config file value (cloud_mode)
"""
env_value = os.environ.get("BASIC_MEMORY_CLOUD_MODE", "").lower()
if env_value in ("true", "1", "yes"):
return True
elif env_value in ("false", "0", "no"):
return False
# Fall back to config file value
return self.cloud_mode
bisync_config: Dict[str, Any] = Field(
default_factory=lambda: {
"profile": "balanced",
"sync_dir": str(Path.home() / "basic-memory-cloud-sync"),
},
description="Bisync configuration for cloud sync",
)
model_config = SettingsConfigDict(
env_prefix="BASIC_MEMORY_",
extra="ignore",
env_file=".env",
env_file_encoding="utf-8",
)
def get_project_path(self, project_name: Optional[str] = None) -> Path: # pragma: no cover
@@ -94,9 +174,9 @@ class BasicMemoryConfig(BaseSettings):
"""Ensure configuration is valid after initialization."""
# Ensure main project exists
if "main" not in self.projects: # pragma: no cover
self.projects["main"] = str(
self.projects["main"] = (
Path(os.getenv("BASIC_MEMORY_HOME", Path.home() / "basic-memory"))
)
).as_posix()
# Ensure default project is valid
if self.default_project not in self.projects: # pragma: no cover
@@ -124,6 +204,7 @@ class BasicMemoryConfig(BaseSettings):
"""
# Load the app-level database path from the global config
config_manager = ConfigManager()
config = config_manager.load_config() # pragma: no cover
return config.app_database_path # pragma: no cover
@@ -146,6 +227,10 @@ class BasicMemoryConfig(BaseSettings):
raise e
return v
@property
def data_dir_path(self):
return Path.home() / DATA_DIR_NAME
class ConfigManager:
"""Manages Basic Memory configuration."""
@@ -162,20 +247,21 @@ class ConfigManager:
# Ensure config directory exists
self.config_dir.mkdir(parents=True, exist_ok=True)
# Load or create configuration
self.config = self.load_config()
@property
def config(self) -> BasicMemoryConfig:
"""Get configuration, loading it lazily if needed."""
return self.load_config()
def load_config(self) -> BasicMemoryConfig:
"""Load configuration from file or create default."""
if self.config_file.exists():
try:
data = json.loads(self.config_file.read_text(encoding="utf-8"))
return BasicMemoryConfig(**data)
except Exception as e: # pragma: no cover
logger.error(f"Failed to load config: {e}")
config = BasicMemoryConfig()
self.save_config(config)
return config
logger.exception(f"Failed to load config: {e}")
raise e
else:
config = BasicMemoryConfig()
self.save_config(config)
@@ -183,10 +269,7 @@ class ConfigManager:
def save_config(self, config: BasicMemoryConfig) -> None:
"""Save configuration to file."""
try:
self.config_file.write_text(json.dumps(config.model_dump(), indent=2))
except Exception as e: # pragma: no cover
logger.error(f"Failed to save config: {e}")
save_basic_memory_config(self.config_file, config)
@property
def projects(self) -> Dict[str, str]:
@@ -208,8 +291,10 @@ class ConfigManager:
project_path = Path(path)
project_path.mkdir(parents=True, exist_ok=True) # pragma: no cover
self.config.projects[name] = str(project_path)
self.save_config(self.config)
# Load config, modify it, and save it
config = self.load_config()
config.projects[name] = project_path.as_posix()
self.save_config(config)
return ProjectConfig(name=name, home=project_path)
def remove_project(self, name: str) -> None:
@@ -219,11 +304,13 @@ class ConfigManager:
if not project_name: # pragma: no cover
raise ValueError(f"Project '{name}' not found")
if project_name == self.config.default_project: # pragma: no cover
# Load config, check, modify, and save
config = self.load_config()
if project_name == config.default_project: # pragma: no cover
raise ValueError(f"Cannot remove the default project '{name}'")
del self.config.projects[name]
self.save_config(self.config)
del config.projects[name]
self.save_config(config)
def set_default_project(self, name: str) -> None:
"""Set the default project."""
@@ -231,15 +318,18 @@ class ConfigManager:
if not project_name: # pragma: no cover
raise ValueError(f"Project '{name}' not found")
self.config.default_project = name
self.save_config(self.config)
# Load config, modify, and save
config = self.load_config()
config.default_project = project_name
self.save_config(config)
def get_project(self, name: str) -> Tuple[str, str] | Tuple[None, None]:
"""Look up a project from the configuration by name or permalink"""
project_permalink = generate_permalink(name)
for name, path in app_config.projects.items():
if project_permalink == generate_permalink(name):
return name, path
app_config = self.config
for project_name, path in app_config.projects.items():
if project_permalink == generate_permalink(project_name):
return project_name, path
return None, None
@@ -252,14 +342,14 @@ def get_project_config(project_name: Optional[str] = None) -> ProjectConfig:
actual_project_name = None
# load the config from file
global app_config
config_manager = ConfigManager()
app_config = config_manager.load_config()
# Get project name from environment variable
os_project_name = os.environ.get("BASIC_MEMORY_PROJECT", None)
if os_project_name: # pragma: no cover
logger.warning(
f"BASIC_MEMORY_PROJECT is not supported anymore. Use the --project flag or set the default project in the config instead. Setting default project to {os_project_name}"
f"BASIC_MEMORY_PROJECT is not supported anymore. Set the default project in the config instead. Setting default project to {os_project_name}"
)
actual_project_name = project_name
# if the project_name is passed in, use it
@@ -282,23 +372,12 @@ def get_project_config(project_name: Optional[str] = None) -> ProjectConfig:
raise ValueError(f"Project '{actual_project_name}' not found") # pragma: no cover
# Create config manager
config_manager = ConfigManager()
# Export the app-level config
app_config: BasicMemoryConfig = config_manager.config
# Load project config for the default project (backward compatibility)
config: ProjectConfig = get_project_config()
def update_current_project(project_name: str) -> None:
"""Update the global config to use a different project.
This is used by the CLI when --project flag is specified.
"""
global config
config = get_project_config(project_name) # pragma: no cover
def save_basic_memory_config(file_path: Path, config: BasicMemoryConfig) -> None:
"""Save configuration to file."""
try:
file_path.write_text(json.dumps(config.model_dump(), indent=2))
except Exception as e: # pragma: no cover
logger.error(f"Failed to save config: {e}")
# setup logging to a single log file in user home directory
@@ -341,12 +420,24 @@ def setup_basic_memory_logging(): # pragma: no cover
# print("Skipping duplicate logging setup")
return
# Check for console logging environment variable - accept more truthy values
console_logging_env = os.getenv("BASIC_MEMORY_CONSOLE_LOGGING", "false").lower()
console_logging = console_logging_env in ("true", "1", "yes", "on")
# Check for log level environment variable first, fall back to config
log_level = os.getenv("BASIC_MEMORY_LOG_LEVEL")
if not log_level:
config_manager = ConfigManager()
log_level = config_manager.config.log_level
config_manager = ConfigManager()
config = get_project_config()
setup_logging(
env=config_manager.config.env,
home_dir=user_home, # Use user home for logs
log_level=config_manager.load_config().log_level,
log_level=log_level,
log_file=f"{DATA_DIR_NAME}/basic-memory-{process_name}.log",
console=False,
console=console_logging,
)
logger.info(f"Basic Memory {basic_memory.__version__} (Project: {config.project})")
+106 -9
View File
@@ -1,15 +1,16 @@
import asyncio
import os
from contextlib import asynccontextmanager
from enum import Enum, auto
from pathlib import Path
from typing import AsyncGenerator, Optional
from basic_memory.config import BasicMemoryConfig
from basic_memory.config import BasicMemoryConfig, ConfigManager
from alembic import command
from alembic.config import Config
from loguru import logger
from sqlalchemy import text
from sqlalchemy import text, event
from sqlalchemy.ext.asyncio import (
create_async_engine,
async_sessionmaker,
@@ -17,6 +18,7 @@ from sqlalchemy.ext.asyncio import (
AsyncEngine,
async_scoped_session,
)
from sqlalchemy.pool import NullPool
from basic_memory.repository.search_repository import SearchRepository
@@ -73,13 +75,77 @@ async def scoped_session(
await factory.remove()
def _configure_sqlite_connection(dbapi_conn, enable_wal: bool = True) -> None:
"""Configure SQLite connection with WAL mode and optimizations.
Args:
dbapi_conn: Database API connection object
enable_wal: Whether to enable WAL mode (should be False for in-memory databases)
"""
cursor = dbapi_conn.cursor()
try:
# Enable WAL mode for better concurrency (not supported for in-memory databases)
if enable_wal:
cursor.execute("PRAGMA journal_mode=WAL")
# Set busy timeout to handle locked databases
cursor.execute("PRAGMA busy_timeout=10000") # 10 seconds
# Optimize for performance
cursor.execute("PRAGMA synchronous=NORMAL")
cursor.execute("PRAGMA cache_size=-64000") # 64MB cache
cursor.execute("PRAGMA temp_store=MEMORY")
# Windows-specific optimizations
if os.name == "nt":
cursor.execute("PRAGMA locking_mode=NORMAL") # Ensure normal locking on Windows
except Exception as e:
# Log but don't fail - some PRAGMAs may not be supported
logger.warning(f"Failed to configure SQLite connection: {e}")
finally:
cursor.close()
def _create_engine_and_session(
db_path: Path, db_type: DatabaseType = DatabaseType.FILESYSTEM
) -> tuple[AsyncEngine, async_sessionmaker[AsyncSession]]:
"""Internal helper to create engine and session maker."""
db_url = DatabaseType.get_db_url(db_path, db_type)
logger.debug(f"Creating engine for db_url: {db_url}")
engine = create_async_engine(db_url, connect_args={"check_same_thread": False})
# Configure connection args with Windows-specific settings
connect_args: dict[str, bool | float | None] = {"check_same_thread": False}
# Add Windows-specific parameters to improve reliability
if os.name == "nt": # Windows
connect_args.update(
{
"timeout": 30.0, # Increase timeout to 30 seconds for Windows
"isolation_level": None, # Use autocommit mode
}
)
# Use NullPool for Windows filesystem databases to avoid connection pooling issues
# Important: Do NOT use NullPool for in-memory databases as it will destroy the database
# between connections
if db_type == DatabaseType.FILESYSTEM:
engine = create_async_engine(
db_url,
connect_args=connect_args,
poolclass=NullPool, # Disable connection pooling on Windows
echo=False,
)
else:
# In-memory databases need connection pooling to maintain state
engine = create_async_engine(db_url, connect_args=connect_args)
else:
engine = create_async_engine(db_url, connect_args=connect_args)
# Enable WAL mode for better concurrency and reliability
# Note: WAL mode is not supported for in-memory databases
enable_wal = db_type != DatabaseType.MEMORY
@event.listens_for(engine.sync_engine, "connect")
def enable_wal_mode(dbapi_conn, connection_record):
"""Enable WAL mode on each connection."""
_configure_sqlite_connection(dbapi_conn, enable_wal=enable_wal)
session_maker = async_sessionmaker(engine, expire_on_commit=False)
return engine, session_maker
@@ -88,7 +154,6 @@ async def get_or_create_db(
db_path: Path,
db_type: DatabaseType = DatabaseType.FILESYSTEM,
ensure_migrations: bool = True,
app_config: Optional["BasicMemoryConfig"] = None,
) -> tuple[AsyncEngine, async_sessionmaker[AsyncSession]]: # pragma: no cover
"""Get or create database engine and session maker."""
global _engine, _session_maker
@@ -98,10 +163,7 @@ async def get_or_create_db(
# Run migrations automatically unless explicitly disabled
if ensure_migrations:
if app_config is None:
from basic_memory.config import app_config as global_app_config
app_config = global_app_config
app_config = ConfigManager().config
await run_migrations(app_config, db_type)
# These checks should never fail since we just created the engine and session maker
@@ -144,7 +206,42 @@ async def engine_session_factory(
db_url = DatabaseType.get_db_url(db_path, db_type)
logger.debug(f"Creating engine for db_url: {db_url}")
_engine = create_async_engine(db_url, connect_args={"check_same_thread": False})
# Configure connection args with Windows-specific settings
connect_args: dict[str, bool | float | None] = {"check_same_thread": False}
# Add Windows-specific parameters to improve reliability
if os.name == "nt": # Windows
connect_args.update(
{
"timeout": 30.0, # Increase timeout to 30 seconds for Windows
"isolation_level": None, # Use autocommit mode
}
)
# Use NullPool for Windows filesystem databases to avoid connection pooling issues
# Important: Do NOT use NullPool for in-memory databases as it will destroy the database
# between connections
if db_type == DatabaseType.FILESYSTEM:
_engine = create_async_engine(
db_url,
connect_args=connect_args,
poolclass=NullPool, # Disable connection pooling on Windows
echo=False,
)
else:
# In-memory databases need connection pooling to maintain state
_engine = create_async_engine(db_url, connect_args=connect_args)
else:
_engine = create_async_engine(db_url, connect_args=connect_args)
# Enable WAL mode for better concurrency and reliability
# Note: WAL mode is not supported for in-memory databases
enable_wal = db_type != DatabaseType.MEMORY
@event.listens_for(_engine.sync_engine, "connect")
def enable_wal_mode(dbapi_conn, connection_record):
"""Enable WAL mode on each connection."""
_configure_sqlite_connection(dbapi_conn, enable_wal=enable_wal)
try:
_session_maker = async_sessionmaker(_engine, expire_on_commit=False)
+23 -5
View File
@@ -3,7 +3,7 @@
from typing import Annotated
from loguru import logger
from fastapi import Depends, HTTPException, Path, status
from fastapi import Depends, HTTPException, Path, status, Request
from sqlalchemy.ext.asyncio import (
AsyncSession,
AsyncEngine,
@@ -12,7 +12,7 @@ from sqlalchemy.ext.asyncio import (
import pathlib
from basic_memory import db
from basic_memory.config import ProjectConfig, BasicMemoryConfig
from basic_memory.config import ProjectConfig, BasicMemoryConfig, ConfigManager
from basic_memory.importers import (
ChatGPTImporter,
ClaudeConversationsImporter,
@@ -33,10 +33,10 @@ from basic_memory.services.file_service import FileService
from basic_memory.services.link_resolver import LinkResolver
from basic_memory.services.search_service import SearchService
from basic_memory.sync import SyncService
from basic_memory.config import app_config
def get_app_config() -> BasicMemoryConfig: # pragma: no cover
app_config = ConfigManager().config
return app_config
@@ -78,9 +78,24 @@ ProjectConfigDep = Annotated[ProjectConfig, Depends(get_project_config)] # prag
async def get_engine_factory(
app_config: AppConfigDep,
request: Request,
) -> tuple[AsyncEngine, async_sessionmaker[AsyncSession]]: # pragma: no cover
"""Get engine and session maker."""
"""Get cached engine and session maker from app state.
For API requests, returns cached connections from app.state for optimal performance.
For non-API contexts (CLI), falls back to direct database connection.
"""
# Try to get cached connections from app state (API context)
if (
hasattr(request, "app")
and hasattr(request.app.state, "engine")
and hasattr(request.app.state, "session_maker")
):
return request.app.state.engine, request.app.state.session_maker
# Fallback for non-API contexts (CLI)
logger.debug("Using fallback database connection for non-API context")
app_config = get_app_config()
engine, session_maker = await db.get_or_create_db(app_config.database_path)
return engine, session_maker
@@ -245,6 +260,7 @@ async def get_entity_service(
entity_parser: EntityParserDep,
file_service: FileServiceDep,
link_resolver: "LinkResolverDep",
app_config: AppConfigDep,
) -> EntityService:
"""Create EntityService with repository."""
return EntityService(
@@ -254,6 +270,7 @@ async def get_entity_service(
entity_parser=entity_parser,
file_service=file_service,
link_resolver=link_resolver,
app_config=app_config,
)
@@ -297,6 +314,7 @@ ContextServiceDep = Annotated[ContextService, Depends(get_context_service)]
async def get_sync_service(
app_config: AppConfigDep,
entity_service: EntityServiceDep,
entity_parser: EntityParserDep,
entity_repository: EntityRepositoryDep,
+89
View File
@@ -2,9 +2,11 @@
import hashlib
from pathlib import Path
import re
from typing import Any, Dict, Union
import yaml
import frontmatter
from loguru import logger
from basic_memory.utils import FilePath
@@ -233,3 +235,90 @@ async def update_frontmatter(path: FilePath, updates: Dict[str, Any]) -> str:
error=str(e),
)
raise FileError(f"Failed to update frontmatter: {e}")
def dump_frontmatter(post: frontmatter.Post) -> str:
"""
Serialize frontmatter.Post to markdown with Obsidian-compatible YAML format.
This function ensures that tags are formatted as YAML lists instead of JSON arrays:
Good (Obsidian compatible):
---
tags:
- system
- overview
- reference
---
Bad (current behavior):
---
tags: ["system", "overview", "reference"]
---
Args:
post: frontmatter.Post object to serialize
Returns:
String containing markdown with properly formatted YAML frontmatter
"""
if not post.metadata:
# No frontmatter, just return content
return post.content
# Serialize YAML with block style for lists
yaml_str = yaml.dump(
post.metadata, sort_keys=False, allow_unicode=True, default_flow_style=False
)
# Construct the final markdown with frontmatter
if post.content:
return f"---\n{yaml_str}---\n\n{post.content}"
else:
return f"---\n{yaml_str}---\n"
def sanitize_for_filename(text: str, replacement: str = "-") -> str:
"""
Sanitize string to be safe for use as a note title
Replaces path separators and other problematic characters
with hyphens.
"""
# replace both POSIX and Windows path separators
text = re.sub(r"[/\\]", replacement, text)
# replace some other problematic chars
text = re.sub(r'[<>:"|?*]', replacement, text)
# compress multiple, repeated replacements
text = re.sub(f"{re.escape(replacement)}+", replacement, text)
return text.strip(replacement)
def sanitize_for_folder(folder: str) -> str:
"""
Sanitize folder path to be safe for use in file system paths.
Removes leading/trailing whitespace, compresses multiple slashes,
and removes special characters except for /, -, and _.
"""
if not folder:
return ""
sanitized = folder.strip()
if sanitized.startswith("./"):
sanitized = sanitized[2:]
# ensure no special characters (except for a few that are allowed)
sanitized = "".join(
c for c in sanitized if c.isalnum() or c in (".", " ", "-", "_", "\\", "/")
).rstrip()
# compress multiple, repeated instances of path separators
sanitized = re.sub(r"[\\/]+", "/", sanitized)
# trim any leading/trailing path separators
sanitized = sanitized.strip("\\/")
return sanitized
+295
View File
@@ -0,0 +1,295 @@
"""Utilities for handling .gitignore patterns and file filtering."""
import fnmatch
from pathlib import Path
from typing import Set
# Common directories and patterns to ignore by default
# These are used as fallback if .bmignore doesn't exist
DEFAULT_IGNORE_PATTERNS = {
# Hidden files (files starting with dot)
".*",
# Basic Memory internal files
"memory.db",
"memory.db-shm",
"memory.db-wal",
"config.json",
# Version control
".git",
".svn",
# Python
"__pycache__",
"*.pyc",
"*.pyo",
"*.pyd",
".pytest_cache",
".coverage",
"*.egg-info",
".tox",
".mypy_cache",
".ruff_cache",
# Virtual environments
".venv",
"venv",
"env",
".env",
# Node.js
"node_modules",
# Build artifacts
"build",
"dist",
".cache",
# IDE
".idea",
".vscode",
# OS files
".DS_Store",
"Thumbs.db",
"desktop.ini",
# Obsidian
".obsidian",
# Temporary files
"*.tmp",
"*.swp",
"*.swo",
"*~",
}
def get_bmignore_path() -> Path:
"""Get path to .bmignore file.
Returns:
Path to ~/.basic-memory/.bmignore
"""
return Path.home() / ".basic-memory" / ".bmignore"
def create_default_bmignore() -> None:
"""Create default .bmignore file if it doesn't exist.
This ensures users have a file they can customize for all Basic Memory operations.
"""
bmignore_path = get_bmignore_path()
if bmignore_path.exists():
return
bmignore_path.parent.mkdir(parents=True, exist_ok=True)
bmignore_path.write_text("""# Basic Memory Ignore Patterns
# This file is used by both 'bm cloud upload', 'bm cloud bisync', and file sync
# Patterns use standard gitignore-style syntax
# Hidden files (files starting with dot)
.*
# Basic Memory internal files
memory.db
memory.db-shm
memory.db-wal
config.json
# Version control
.git
.svn
# Python
__pycache__
*.pyc
*.pyo
*.pyd
.pytest_cache
.coverage
*.egg-info
.tox
.mypy_cache
.ruff_cache
# Virtual environments
.venv
venv
env
.env
# Node.js
node_modules
# Build artifacts
build
dist
.cache
# IDE
.idea
.vscode
# OS files
.DS_Store
Thumbs.db
desktop.ini
# Obsidian
.obsidian
# Temporary files
*.tmp
*.swp
*.swo
*~
""")
def load_bmignore_patterns() -> Set[str]:
"""Load patterns from .bmignore file.
Returns:
Set of patterns from .bmignore, or DEFAULT_IGNORE_PATTERNS if file doesn't exist
"""
bmignore_path = get_bmignore_path()
# Create default file if it doesn't exist
if not bmignore_path.exists():
create_default_bmignore()
patterns = set()
try:
with bmignore_path.open("r", encoding="utf-8") as f:
for line in f:
line = line.strip()
# Skip empty lines and comments
if line and not line.startswith("#"):
patterns.add(line)
except Exception:
# If we can't read .bmignore, fall back to defaults
return set(DEFAULT_IGNORE_PATTERNS)
# If no patterns were loaded, use defaults
if not patterns:
return set(DEFAULT_IGNORE_PATTERNS)
return patterns
def load_gitignore_patterns(base_path: Path) -> Set[str]:
"""Load gitignore patterns from .gitignore file and .bmignore.
Combines patterns from:
1. ~/.basic-memory/.bmignore (user's global ignore patterns)
2. {base_path}/.gitignore (project-specific patterns)
Args:
base_path: The base directory to search for .gitignore file
Returns:
Set of patterns to ignore
"""
# Start with patterns from .bmignore
patterns = load_bmignore_patterns()
gitignore_file = base_path / ".gitignore"
if gitignore_file.exists():
try:
with gitignore_file.open("r", encoding="utf-8") as f:
for line in f:
line = line.strip()
# Skip empty lines and comments
if line and not line.startswith("#"):
patterns.add(line)
except Exception:
# If we can't read .gitignore, just use default patterns
pass
return patterns
def should_ignore_path(file_path: Path, base_path: Path, ignore_patterns: Set[str]) -> bool:
"""Check if a file path should be ignored based on gitignore patterns.
Args:
file_path: The file path to check
base_path: The base directory for relative path calculation
ignore_patterns: Set of patterns to match against
Returns:
True if the path should be ignored, False otherwise
"""
# Get the relative path from base
try:
relative_path = file_path.relative_to(base_path)
relative_str = str(relative_path)
relative_posix = relative_path.as_posix() # Use forward slashes for matching
# Check each pattern
for pattern in ignore_patterns:
# Handle patterns starting with / (root relative)
if pattern.startswith("/"):
root_pattern = pattern[1:] # Remove leading /
# For directory patterns ending with /
if root_pattern.endswith("/"):
dir_name = root_pattern[:-1] # Remove trailing /
# Check if the first part of the path matches the directory name
if len(relative_path.parts) > 0 and relative_path.parts[0] == dir_name:
return True
else:
# Regular root-relative pattern
if fnmatch.fnmatch(relative_posix, root_pattern):
return True
continue
# Handle directory patterns (ending with /)
if pattern.endswith("/"):
dir_name = pattern[:-1] # Remove trailing /
# Check if any path part matches the directory name
if dir_name in relative_path.parts:
return True
continue
# Direct name match (e.g., ".git", "node_modules")
if pattern in relative_path.parts:
return True
# Check if any individual path part matches the glob pattern
# This handles cases like ".*" matching ".hidden.md" in "concept/.hidden.md"
for part in relative_path.parts:
if fnmatch.fnmatch(part, pattern):
return True
# Glob pattern match on full path
if fnmatch.fnmatch(relative_posix, pattern) or fnmatch.fnmatch(relative_str, pattern):
return True
return False
except ValueError:
# If we can't get relative path, don't ignore
return False
def filter_files(
files: list[Path], base_path: Path, ignore_patterns: Set[str] | None = None
) -> tuple[list[Path], int]:
"""Filter a list of files based on gitignore patterns.
Args:
files: List of file paths to filter
base_path: The base directory for relative path calculation
ignore_patterns: Set of patterns to ignore. If None, loads from .gitignore
Returns:
Tuple of (filtered_files, ignored_count)
"""
if ignore_patterns is None:
ignore_patterns = load_gitignore_patterns(base_path)
filtered_files = []
ignored_count = 0
for file_path in files:
if should_ignore_path(file_path, base_path, ignore_patterns):
ignored_count += 1
else:
filtered_files.append(file_path)
return filtered_files, ignored_count
+20 -10
View File
@@ -93,7 +93,7 @@ class ChatGPTImporter(Importer[ChatImportResult]):
break
# Generate permalink
date_prefix = datetime.fromtimestamp(created_at).strftime("%Y%m%d")
date_prefix = datetime.fromtimestamp(created_at).astimezone().strftime("%Y%m%d")
clean_title = clean_filename(conversation["title"])
# Format content
@@ -193,7 +193,7 @@ class ChatGPTImporter(Importer[ChatImportResult]):
def _traverse_messages(
self, mapping: Dict[str, Any], root_id: Optional[str], seen: Set[str]
) -> List[Dict[str, Any]]: # pragma: no cover
"""Traverse message tree and return messages in order.
"""Traverse message tree iteratively to handle deep conversations.
Args:
mapping: Message mapping.
@@ -204,19 +204,29 @@ class ChatGPTImporter(Importer[ChatImportResult]):
List of message data.
"""
messages = []
node = mapping.get(root_id) if root_id else None
if not root_id:
return messages
while node:
# Use iterative approach with stack to avoid recursion depth issues
stack = [root_id]
while stack:
node_id = stack.pop()
if not node_id:
continue
node = mapping.get(node_id)
if not node:
continue
# Process current node if it has a message and hasn't been seen
if node["id"] not in seen and node.get("message"):
seen.add(node["id"])
messages.append(node["message"])
# Follow children
# Add children to stack in reverse order to maintain conversation flow
children = node.get("children", [])
for child_id in children:
child_msgs = self._traverse_messages(mapping, child_id, seen)
messages.extend(child_msgs)
break # Don't follow siblings
for child_id in reversed(children):
stack.append(child_id)
return messages
@@ -3,7 +3,7 @@
import logging
from typing import Any, Dict, List
from basic_memory.config import config
from basic_memory.config import get_project_config
from basic_memory.markdown.schemas import EntityFrontmatter, EntityMarkdown, Observation, Relation
from basic_memory.importers.base import Importer
from basic_memory.schemas.importer import EntityImportResult
@@ -27,10 +27,12 @@ class MemoryJsonImporter(Importer[EntityImportResult]):
Returns:
EntityImportResult containing statistics and status of the import.
"""
config = get_project_config()
try:
# First pass - collect all relations by source entity
entity_relations: Dict[str, List[Relation]] = {}
entities: Dict[str, Dict[str, Any]] = {}
skipped_entities: int = 0
# Ensure the base path exists
base_path = config.home # pragma: no cover
@@ -41,7 +43,13 @@ class MemoryJsonImporter(Importer[EntityImportResult]):
for line in source_data:
data = line
if data["type"] == "entity":
entities[data["name"]] = data
# Handle different possible name keys
entity_name = data.get("name") or data.get("entityName") or data.get("id")
if not entity_name:
logger.warning(f"Entity missing name field: {data}")
skipped_entities += 1
continue
entities[entity_name] = data
elif data["type"] == "relation":
# Store relation with its source entity
source = data.get("from") or data.get("from_id")
@@ -57,25 +65,31 @@ class MemoryJsonImporter(Importer[EntityImportResult]):
# Second pass - create and write entities
entities_created = 0
for name, entity_data in entities.items():
# Get entity type with fallback
entity_type = entity_data.get("entityType") or entity_data.get("type") or "entity"
# Ensure entity type directory exists
entity_type_dir = base_path / entity_data["entityType"]
entity_type_dir = base_path / entity_type
entity_type_dir.mkdir(parents=True, exist_ok=True)
# Get observations with fallback to empty list
observations = entity_data.get("observations", [])
entity = EntityMarkdown(
frontmatter=EntityFrontmatter(
metadata={
"type": entity_data["entityType"],
"type": entity_type,
"title": name,
"permalink": f"{entity_data['entityType']}/{name}",
"permalink": f"{entity_type}/{name}",
}
),
content=f"# {name}\n",
observations=[Observation(content=obs) for obs in entity_data["observations"]],
observations=[Observation(content=obs) for obs in observations],
relations=entity_relations.get(name, []),
)
# Write entity file
file_path = base_path / f"{entity_data['entityType']}/{name}.md"
file_path = base_path / f"{entity_type}/{name}.md"
await self.write_entity(entity, file_path)
entities_created += 1
@@ -86,6 +100,7 @@ class MemoryJsonImporter(Importer[EntityImportResult]):
success=True,
entities=entities_created,
relations=relations_count,
skipped_entities=skipped_entities,
)
except Exception as e: # pragma: no cover
+2 -2
View File
@@ -43,13 +43,13 @@ def format_timestamp(timestamp: Any) -> str: # pragma: no cover
except ValueError:
try:
# Try unix timestamp as string
timestamp = datetime.fromtimestamp(float(timestamp))
timestamp = datetime.fromtimestamp(float(timestamp)).astimezone()
except ValueError:
# Return as is if we can't parse it
return timestamp
elif isinstance(timestamp, (int, float)):
# Unix timestamp
timestamp = datetime.fromtimestamp(timestamp)
timestamp = datetime.fromtimestamp(timestamp).astimezone()
if isinstance(timestamp, datetime):
return timestamp.strftime("%Y-%m-%d %H:%M:%S")
+2 -2
View File
@@ -130,6 +130,6 @@ class EntityParser:
content=post.content,
observations=entity_content.observations,
relations=entity_content.relations,
created=datetime.fromtimestamp(file_stats.st_ctime),
modified=datetime.fromtimestamp(file_stats.st_mtime),
created=datetime.fromtimestamp(file_stats.st_ctime).astimezone(),
modified=datetime.fromtimestamp(file_stats.st_mtime).astimezone(),
)
@@ -2,11 +2,11 @@ from pathlib import Path
from typing import Optional
from collections import OrderedDict
import frontmatter
from frontmatter import Post
from loguru import logger
from basic_memory import file_utils
from basic_memory.file_utils import dump_frontmatter
from basic_memory.markdown.entity_parser import EntityParser
from basic_memory.markdown.schemas import EntityMarkdown, Observation, Relation
@@ -115,7 +115,7 @@ class MarkdownProcessor:
# Create Post object for frontmatter
post = Post(content, **frontmatter_dict)
final_content = frontmatter.dumps(post, sort_keys=False)
final_content = dump_frontmatter(post)
logger.debug(f"writing file {path} with content:\n{final_content}")
+39 -21
View File
@@ -8,34 +8,50 @@ from markdown_it.token import Token
# Observation handling functions
def is_observation(token: Token) -> bool:
"""Check if token looks like our observation format."""
import re
if token.type != "inline": # pragma: no cover
return False
content = token.content.strip()
# Use token.tag which contains the actual content for test tokens, fallback to content
content = (token.tag or token.content).strip()
if not content: # pragma: no cover
return False
# if it's a markdown_task, return false
if content.startswith("[ ]") or content.startswith("[x]") or content.startswith("[-]"):
return False
has_category = content.startswith("[") and "]" in content
# Exclude markdown links: [text](url)
if re.match(r"^\[.*?\]\(.*?\)$", content):
return False
# Exclude wiki links: [[text]]
if re.match(r"^\[\[.*?\]\]$", content):
return False
# Check for proper observation format: [category] content
match = re.match(r"^\[([^\[\]()]+)\]\s+(.+)", content)
has_tags = "#" in content
return has_category or has_tags
return bool(match) or has_tags
def parse_observation(token: Token) -> Dict[str, Any]:
"""Extract observation parts from token."""
# Strip bullet point if present
content = token.content.strip()
import re
# Parse [category]
# Use token.tag which contains the actual content for test tokens, fallback to content
content = (token.tag or token.content).strip()
# Parse [category] with regex
match = re.match(r"^\[([^\[\]()]+)\]\s+(.+)", content)
category = None
if content.startswith("["):
end = content.find("]")
if end != -1:
category = content[1:end].strip() or None # Convert empty to None
content = content[end + 1 :].strip()
if match:
category = match.group(1).strip()
content = match.group(2).strip()
else:
# Handle empty brackets [] followed by content
empty_match = re.match(r"^\[\]\s+(.+)", content)
if empty_match:
content = empty_match.group(1).strip()
# Parse (context)
context = None
@@ -50,9 +66,7 @@ def parse_observation(token: Token) -> Dict[str, Any]:
parts = content.split()
for part in parts:
if part.startswith("#"):
# Handle multiple #tags stuck together
if "#" in part[1:]:
# Split on # but keep non-empty tags
subtags = [t for t in part.split("#") if t]
tags.extend(subtags)
else:
@@ -72,14 +86,16 @@ def is_explicit_relation(token: Token) -> bool:
if token.type != "inline": # pragma: no cover
return False
content = token.content.strip()
# Use token.tag which contains the actual content for test tokens, fallback to content
content = (token.tag or token.content).strip()
return "[[" in content and "]]" in content
def parse_relation(token: Token) -> Dict[str, Any] | None:
"""Extract relation parts from token."""
# Remove bullet point if present
content = token.content.strip()
# Use token.tag which contains the actual content for test tokens, fallback to content
content = (token.tag or token.content).strip()
# Extract [[target]]
target = None
@@ -213,10 +229,12 @@ def relation_plugin(md: MarkdownIt) -> None:
token.meta["relations"] = [rel]
# Always check for inline links in any text
elif "[[" in token.content:
rels = parse_inline_relations(token.content)
if rels:
token.meta["relations"] = token.meta.get("relations", []) + rels
else:
content = token.tag or token.content
if "[[" in content:
rels = parse_inline_relations(content)
if rels:
token.meta["relations"] = token.meta.get("relations", []) + rels
# Add the rule after inline processing
md.core.ruler.after("inline", "relations", relation_rule)
+4 -2
View File
@@ -38,8 +38,10 @@ def entity_model_from_markdown(
# Update basic fields
model.title = markdown.frontmatter.title
model.entity_type = markdown.frontmatter.type
model.permalink = markdown.frontmatter.permalink
model.file_path = str(file_path)
# Only update permalink if it exists in frontmatter, otherwise preserve existing
if markdown.frontmatter.permalink is not None:
model.permalink = markdown.frontmatter.permalink
model.file_path = file_path.as_posix()
model.content_type = "text/markdown"
model.created_at = markdown.created
model.updated_at = markdown.modified
+35 -3
View File
@@ -1,8 +1,40 @@
from httpx import ASGITransport, AsyncClient
from httpx import ASGITransport, AsyncClient, Timeout
from loguru import logger
from basic_memory.api.app import app as fastapi_app
from basic_memory.config import ConfigManager
def create_client() -> AsyncClient:
"""Create an HTTP client based on configuration.
Returns:
AsyncClient configured for either local ASGI or remote proxy
"""
config_manager = ConfigManager()
config = config_manager.config
# Configure timeout for longer operations like write_note
# Default httpx timeout is 5 seconds which is too short for file operations
timeout = Timeout(
connect=10.0, # 10 seconds for connection
read=30.0, # 30 seconds for reading response
write=30.0, # 30 seconds for writing request
pool=30.0, # 30 seconds for connection pool
)
if config.cloud_mode_enabled:
# Use HTTP transport to proxy endpoint
proxy_base_url = f"{config.cloud_host}/proxy"
logger.info(f"Creating HTTP client for proxy at: {proxy_base_url}")
return AsyncClient(base_url=proxy_base_url, timeout=timeout)
else:
# Default: use ASGI transport for local API (development mode)
logger.info("Creating ASGI client for local Basic Memory API")
return AsyncClient(
transport=ASGITransport(app=fastapi_app), base_url="http://test", timeout=timeout
)
BASE_URL = "http://test"
# Create shared async client
client = AsyncClient(transport=ASGITransport(app=fastapi_app), base_url=BASE_URL)
client = create_client()
-270
View File
@@ -1,270 +0,0 @@
"""OAuth authentication provider for Basic Memory MCP server."""
import secrets
from datetime import datetime, timedelta
from typing import Dict, Optional
import jwt
from mcp.server.auth.provider import (
OAuthAuthorizationServerProvider,
AuthorizationParams,
AuthorizationCode,
RefreshToken,
AccessToken,
)
from mcp.shared.auth import OAuthClientInformationFull, OAuthToken
from loguru import logger
class BasicMemoryAuthorizationCode(AuthorizationCode):
"""Extended authorization code with additional metadata."""
issuer_state: Optional[str] = None
class BasicMemoryRefreshToken(RefreshToken):
"""Extended refresh token with additional metadata."""
pass
class BasicMemoryAccessToken(AccessToken):
"""Extended access token with additional metadata."""
pass
class BasicMemoryOAuthProvider(
OAuthAuthorizationServerProvider[
BasicMemoryAuthorizationCode, BasicMemoryRefreshToken, BasicMemoryAccessToken
]
):
"""OAuth provider for Basic Memory MCP server.
This is a simple in-memory implementation that can be extended
to integrate with external OAuth providers or use persistent storage.
"""
def __init__(self, issuer_url: str = "http://localhost:8000", secret_key: Optional[str] = None):
self.issuer_url = issuer_url
# Use environment variable for secret key if available, otherwise generate
import os
self.secret_key = (
secret_key or os.getenv("FASTMCP_AUTH_SECRET_KEY") or secrets.token_urlsafe(32)
)
# In-memory storage - in production, use a proper database
self.clients: Dict[str, OAuthClientInformationFull] = {}
self.authorization_codes: Dict[str, BasicMemoryAuthorizationCode] = {}
self.refresh_tokens: Dict[str, BasicMemoryRefreshToken] = {}
self.access_tokens: Dict[str, BasicMemoryAccessToken] = {}
async def get_client(self, client_id: str) -> Optional[OAuthClientInformationFull]:
"""Get a client by ID."""
return self.clients.get(client_id)
async def register_client(self, client_info: OAuthClientInformationFull) -> None:
"""Register a new OAuth client."""
# Generate client ID if not provided
if not client_info.client_id:
client_info.client_id = secrets.token_urlsafe(16)
# Generate client secret if not provided
if not client_info.client_secret:
client_info.client_secret = secrets.token_urlsafe(32)
self.clients[client_info.client_id] = client_info
logger.info(f"Registered OAuth client: {client_info.client_id}")
async def authorize(
self, client: OAuthClientInformationFull, params: AuthorizationParams
) -> str:
"""Create an authorization URL for the OAuth flow.
For basic-memory, we'll implement a simple authorization flow.
In production, this might redirect to an external provider.
"""
# Generate authorization code
auth_code = secrets.token_urlsafe(32)
# Store authorization code with metadata
self.authorization_codes[auth_code] = BasicMemoryAuthorizationCode(
code=auth_code,
scopes=params.scopes or [],
expires_at=(datetime.utcnow() + timedelta(minutes=10)).timestamp(),
client_id=client.client_id,
code_challenge=params.code_challenge,
redirect_uri=params.redirect_uri,
redirect_uri_provided_explicitly=params.redirect_uri_provided_explicitly,
issuer_state=params.state,
)
# In a real implementation, we'd redirect to an authorization page
# For now, we'll just return the redirect URL with the code
redirect_uri = str(params.redirect_uri)
separator = "&" if "?" in redirect_uri else "?"
auth_url = f"{redirect_uri}{separator}code={auth_code}"
if params.state:
auth_url += f"&state={params.state}"
return auth_url
async def load_authorization_code(
self, client: OAuthClientInformationFull, authorization_code: str
) -> Optional[BasicMemoryAuthorizationCode]:
"""Load an authorization code."""
code = self.authorization_codes.get(authorization_code)
if code and code.client_id == client.client_id:
# Check if expired
if datetime.utcnow().timestamp() > code.expires_at:
del self.authorization_codes[authorization_code]
return None
return code
return None
async def exchange_authorization_code(
self, client: OAuthClientInformationFull, authorization_code: BasicMemoryAuthorizationCode
) -> OAuthToken:
"""Exchange an authorization code for tokens."""
# Generate tokens
access_token = self._generate_access_token(client.client_id, authorization_code.scopes)
refresh_token = secrets.token_urlsafe(32)
# Store tokens
expires_at = (datetime.utcnow() + timedelta(hours=1)).timestamp()
self.access_tokens[access_token] = BasicMemoryAccessToken(
token=access_token,
client_id=client.client_id,
scopes=authorization_code.scopes,
expires_at=int(expires_at),
)
self.refresh_tokens[refresh_token] = BasicMemoryRefreshToken(
token=refresh_token,
client_id=client.client_id,
scopes=authorization_code.scopes,
)
# Remove used authorization code
del self.authorization_codes[authorization_code.code]
return OAuthToken(
access_token=access_token,
token_type="bearer",
expires_in=3600, # 1 hour
refresh_token=refresh_token,
scope=" ".join(authorization_code.scopes) if authorization_code.scopes else None,
)
async def load_refresh_token(
self, client: OAuthClientInformationFull, refresh_token: str
) -> Optional[BasicMemoryRefreshToken]:
"""Load a refresh token."""
token = self.refresh_tokens.get(refresh_token)
if token and token.client_id == client.client_id:
return token
return None
async def exchange_refresh_token(
self,
client: OAuthClientInformationFull,
refresh_token: BasicMemoryRefreshToken,
scopes: list[str],
) -> OAuthToken:
"""Exchange a refresh token for new tokens."""
# Use requested scopes or original scopes
token_scopes = scopes if scopes else refresh_token.scopes
# Generate new tokens
new_access_token = self._generate_access_token(client.client_id, token_scopes)
new_refresh_token = secrets.token_urlsafe(32)
# Store new tokens
expires_at = (datetime.utcnow() + timedelta(hours=1)).timestamp()
self.access_tokens[new_access_token] = BasicMemoryAccessToken(
token=new_access_token,
client_id=client.client_id,
scopes=token_scopes,
expires_at=int(expires_at),
)
self.refresh_tokens[new_refresh_token] = BasicMemoryRefreshToken(
token=new_refresh_token,
client_id=client.client_id,
scopes=token_scopes,
)
# Remove old tokens
del self.refresh_tokens[refresh_token.token]
return OAuthToken(
access_token=new_access_token,
token_type="bearer",
expires_in=3600, # 1 hour
refresh_token=new_refresh_token,
scope=" ".join(token_scopes) if token_scopes else None,
)
async def load_access_token(self, token: str) -> Optional[BasicMemoryAccessToken]:
"""Load and validate an access token."""
logger.debug("Loading access token, checking in-memory store first")
access_token = self.access_tokens.get(token)
if access_token:
# Check if expired
if access_token.expires_at and datetime.utcnow().timestamp() > access_token.expires_at:
logger.debug("Token found in memory but expired, removing")
del self.access_tokens[token]
return None
logger.debug("Token found in memory and valid")
return access_token
# Try to decode as JWT
logger.debug("Token not in memory, attempting JWT decode with secret key")
try:
# Decode with audience verification - PyJWT expects the audience to match
payload = jwt.decode(
token,
self.secret_key,
algorithms=["HS256"],
audience="basic-memory", # Expecting this audience
issuer=self.issuer_url, # And this issuer
)
logger.debug(f"JWT decoded successfully: {payload}")
return BasicMemoryAccessToken(
token=token,
client_id=payload.get("sub", ""),
scopes=payload.get("scopes", []),
expires_at=payload.get("exp"),
)
except jwt.InvalidTokenError as e:
logger.error(f"JWT decode failed: {e}")
return None
async def revoke_token(self, token: BasicMemoryAccessToken | BasicMemoryRefreshToken) -> None:
"""Revoke an access or refresh token."""
if isinstance(token, BasicMemoryAccessToken):
self.access_tokens.pop(token.token, None)
else:
self.refresh_tokens.pop(token.token, None)
def _generate_access_token(self, client_id: str, scopes: list[str]) -> str:
"""Generate a JWT access token."""
payload = {
"iss": self.issuer_url,
"sub": client_id,
"aud": "basic-memory",
"exp": datetime.utcnow() + timedelta(hours=1),
"iat": datetime.utcnow(),
"scopes": scopes,
}
return jwt.encode(payload, self.secret_key, algorithm="HS256")
@@ -1,321 +0,0 @@
"""External OAuth provider integration for Basic Memory MCP server."""
import os
from typing import Optional, Dict, Any
from dataclasses import dataclass
import httpx
from loguru import logger
from mcp.server.auth.provider import (
OAuthAuthorizationServerProvider,
AuthorizationParams,
AuthorizationCode,
RefreshToken,
AccessToken,
construct_redirect_uri,
)
from mcp.shared.auth import OAuthClientInformationFull, OAuthToken
@dataclass
class ExternalAuthorizationCode(AuthorizationCode):
"""Authorization code with external provider metadata."""
external_code: Optional[str] = None
state: Optional[str] = None
@dataclass
class ExternalRefreshToken(RefreshToken):
"""Refresh token with external provider metadata."""
external_token: Optional[str] = None
@dataclass
class ExternalAccessToken(AccessToken):
"""Access token with external provider metadata."""
external_token: Optional[str] = None
class ExternalOAuthProvider(
OAuthAuthorizationServerProvider[
ExternalAuthorizationCode, ExternalRefreshToken, ExternalAccessToken
]
):
"""OAuth provider that delegates to external OAuth providers.
This provider can integrate with services like:
- GitHub OAuth
- Google OAuth
- Auth0
- Okta
"""
def __init__(
self,
issuer_url: str,
external_provider: str,
external_client_id: str,
external_client_secret: str,
external_authorize_url: str,
external_token_url: str,
external_userinfo_url: Optional[str] = None,
):
self.issuer_url = issuer_url
self.external_provider = external_provider
self.external_client_id = external_client_id
self.external_client_secret = external_client_secret
self.external_authorize_url = external_authorize_url
self.external_token_url = external_token_url
self.external_userinfo_url = external_userinfo_url
# In-memory storage - in production, use a database
self.clients: Dict[str, OAuthClientInformationFull] = {}
self.codes: Dict[str, ExternalAuthorizationCode] = {}
self.tokens: Dict[str, Any] = {}
self.http_client = httpx.AsyncClient()
async def get_client(self, client_id: str) -> Optional[OAuthClientInformationFull]:
"""Get a client by ID."""
return self.clients.get(client_id)
async def register_client(self, client_info: OAuthClientInformationFull) -> None:
"""Register a new OAuth client."""
self.clients[client_info.client_id] = client_info
logger.info(f"Registered external OAuth client: {client_info.client_id}")
async def authorize(
self, client: OAuthClientInformationFull, params: AuthorizationParams
) -> str:
"""Create authorization URL redirecting to external provider."""
# Store authorization request
import secrets
state = secrets.token_urlsafe(32)
self.codes[state] = ExternalAuthorizationCode(
code=state,
scopes=params.scopes or [],
expires_at=0, # Will be set by external provider
client_id=client.client_id,
code_challenge=params.code_challenge,
redirect_uri=params.redirect_uri,
redirect_uri_provided_explicitly=params.redirect_uri_provided_explicitly,
state=params.state,
)
# Build external provider URL
external_params = {
"client_id": self.external_client_id,
"redirect_uri": f"{self.issuer_url}/callback",
"response_type": "code",
"state": state,
"scope": " ".join(params.scopes or []),
}
return construct_redirect_uri(self.external_authorize_url, **external_params)
async def handle_callback(self, code: str, state: str) -> str:
"""Handle callback from external provider."""
# Get original authorization request
auth_code = self.codes.get(state)
if not auth_code:
raise ValueError("Invalid state parameter")
# Exchange code with external provider
token_data = {
"grant_type": "authorization_code",
"code": code,
"redirect_uri": f"{self.issuer_url}/callback",
"client_id": self.external_client_id,
"client_secret": self.external_client_secret,
}
response = await self.http_client.post(
self.external_token_url,
data=token_data,
)
response.raise_for_status()
external_tokens = response.json()
# Store external tokens
import secrets
internal_code = secrets.token_urlsafe(32)
self.codes[internal_code] = ExternalAuthorizationCode(
code=internal_code,
scopes=auth_code.scopes,
expires_at=0,
client_id=auth_code.client_id,
code_challenge=auth_code.code_challenge,
redirect_uri=auth_code.redirect_uri,
redirect_uri_provided_explicitly=auth_code.redirect_uri_provided_explicitly,
external_code=code,
state=auth_code.state,
)
self.tokens[internal_code] = external_tokens
# Redirect to original client
return construct_redirect_uri(
str(auth_code.redirect_uri),
code=internal_code,
state=auth_code.state,
)
async def load_authorization_code(
self, client: OAuthClientInformationFull, authorization_code: str
) -> Optional[ExternalAuthorizationCode]:
"""Load an authorization code."""
code = self.codes.get(authorization_code)
if code and code.client_id == client.client_id:
return code
return None
async def exchange_authorization_code(
self, client: OAuthClientInformationFull, authorization_code: ExternalAuthorizationCode
) -> OAuthToken:
"""Exchange authorization code for tokens."""
# Get stored external tokens
external_tokens = self.tokens.get(authorization_code.code)
if not external_tokens:
raise ValueError("No tokens found for authorization code")
# Map external tokens to MCP tokens
access_token = external_tokens.get("access_token")
refresh_token = external_tokens.get("refresh_token")
expires_in = external_tokens.get("expires_in", 3600)
# Store the mapping
self.tokens[access_token] = {
"client_id": client.client_id,
"external_token": access_token,
"scopes": authorization_code.scopes,
}
if refresh_token:
self.tokens[refresh_token] = {
"client_id": client.client_id,
"external_token": refresh_token,
"scopes": authorization_code.scopes,
}
# Clean up authorization code
del self.codes[authorization_code.code]
return OAuthToken(
access_token=access_token,
token_type="bearer",
expires_in=expires_in,
refresh_token=refresh_token,
scope=" ".join(authorization_code.scopes) if authorization_code.scopes else None,
)
async def load_refresh_token(
self, client: OAuthClientInformationFull, refresh_token: str
) -> Optional[ExternalRefreshToken]:
"""Load a refresh token."""
token_info = self.tokens.get(refresh_token)
if token_info and token_info["client_id"] == client.client_id:
return ExternalRefreshToken(
token=refresh_token,
client_id=client.client_id,
scopes=token_info["scopes"],
external_token=token_info.get("external_token"),
)
return None
async def exchange_refresh_token(
self,
client: OAuthClientInformationFull,
refresh_token: ExternalRefreshToken,
scopes: list[str],
) -> OAuthToken:
"""Exchange refresh token for new tokens."""
# Exchange with external provider
token_data = {
"grant_type": "refresh_token",
"refresh_token": refresh_token.external_token or refresh_token.token,
"client_id": self.external_client_id,
"client_secret": self.external_client_secret,
}
response = await self.http_client.post(
self.external_token_url,
data=token_data,
)
response.raise_for_status()
external_tokens = response.json()
# Update stored tokens
new_access_token = external_tokens.get("access_token")
new_refresh_token = external_tokens.get("refresh_token", refresh_token.token)
expires_in = external_tokens.get("expires_in", 3600)
self.tokens[new_access_token] = {
"client_id": client.client_id,
"external_token": new_access_token,
"scopes": scopes or refresh_token.scopes,
}
if new_refresh_token != refresh_token.token:
self.tokens[new_refresh_token] = {
"client_id": client.client_id,
"external_token": new_refresh_token,
"scopes": scopes or refresh_token.scopes,
}
del self.tokens[refresh_token.token]
return OAuthToken(
access_token=new_access_token,
token_type="bearer",
expires_in=expires_in,
refresh_token=new_refresh_token,
scope=" ".join(scopes or refresh_token.scopes),
)
async def load_access_token(self, token: str) -> Optional[ExternalAccessToken]:
"""Load and validate an access token."""
token_info = self.tokens.get(token)
if token_info:
return ExternalAccessToken(
token=token,
client_id=token_info["client_id"],
scopes=token_info["scopes"],
external_token=token_info.get("external_token"),
)
return None
async def revoke_token(self, token: ExternalAccessToken | ExternalRefreshToken) -> None:
"""Revoke a token."""
self.tokens.pop(token.token, None)
def create_github_provider() -> ExternalOAuthProvider:
"""Create an OAuth provider for GitHub integration."""
return ExternalOAuthProvider(
issuer_url=os.getenv("FASTMCP_AUTH_ISSUER_URL", "http://localhost:8000"),
external_provider="github",
external_client_id=os.getenv("GITHUB_CLIENT_ID", ""),
external_client_secret=os.getenv("GITHUB_CLIENT_SECRET", ""),
external_authorize_url="https://github.com/login/oauth/authorize",
external_token_url="https://github.com/login/oauth/access_token",
external_userinfo_url="https://api.github.com/user",
)
def create_google_provider() -> ExternalOAuthProvider:
"""Create an OAuth provider for Google integration."""
return ExternalOAuthProvider(
issuer_url=os.getenv("FASTMCP_AUTH_ISSUER_URL", "http://localhost:8000"),
external_provider="google",
external_client_id=os.getenv("GOOGLE_CLIENT_ID", ""),
external_client_secret=os.getenv("GOOGLE_CLIENT_SECRET", ""),
external_authorize_url="https://accounts.google.com/o/oauth2/v2/auth",
external_token_url="https://oauth2.googleapis.com/token",
external_userinfo_url="https://www.googleapis.com/oauth2/v1/userinfo",
)
+141
View File
@@ -0,0 +1,141 @@
"""Project context utilities for Basic Memory MCP server.
Provides project lookup utilities for MCP tools.
Handles project validation and context management in one place.
"""
import os
from typing import Optional, List
from httpx import AsyncClient
from httpx._types import (
HeaderTypes,
)
from loguru import logger
from fastmcp import Context
from basic_memory.config import ConfigManager
from basic_memory.mcp.tools.utils import call_get
from basic_memory.schemas.project_info import ProjectItem, ProjectList
from basic_memory.utils import generate_permalink
async def resolve_project_parameter(project: Optional[str] = None) -> Optional[str]:
"""Resolve project parameter using three-tier hierarchy.
if config.cloud_mode:
project is required
else:
Resolution order:
1. Single Project Mode (--project cli arg, or BASIC_MEMORY_MCP_PROJECT env var) - highest priority
2. Explicit project parameter - medium priority
3. Default project if default_project_mode=true - lowest priority
Args:
project: Optional explicit project parameter
Returns:
Resolved project name or None if no resolution possible
"""
config = ConfigManager().config
# if cloud_mode, project is required
if config.cloud_mode:
if project:
logger.debug(f"project: {project}, cloud_mode: {config.cloud_mode}")
return project
else:
raise ValueError("No project specified. Project is required for cloud mode.")
# Priority 1: CLI constraint overrides everything (--project arg sets env var)
constrained_project = os.environ.get("BASIC_MEMORY_MCP_PROJECT")
if constrained_project:
logger.debug(f"Using CLI constrained project: {constrained_project}")
return constrained_project
# Priority 2: Explicit project parameter
if project:
logger.debug(f"Using explicit project parameter: {project}")
return project
# Priority 3: Default project mode
if config.default_project_mode:
logger.debug(f"Using default project from config: {config.default_project}")
return config.default_project
# No resolution possible
return None
async def get_project_names(client: AsyncClient, headers: HeaderTypes | None = None) -> List[str]:
response = await call_get(client, "/projects/projects", headers=headers)
project_list = ProjectList.model_validate(response.json())
return [project.name for project in project_list.projects]
async def get_active_project(
client: AsyncClient,
project: Optional[str] = None,
context: Optional[Context] = None,
headers: HeaderTypes | None = None,
) -> ProjectItem:
"""Get and validate project, setting it in context if available.
Args:
client: HTTP client for API calls
project: Optional project name (resolved using hierarchy)
context: Optional FastMCP context to cache the result
Returns:
The validated project item
Raises:
ValueError: If no project can be resolved
HTTPError: If project doesn't exist or is inaccessible
"""
resolved_project = await resolve_project_parameter(project)
if not resolved_project:
project_names = await get_project_names(client, headers)
raise ValueError(
"No project specified. "
"Either set 'default_project_mode=true' in config, or use 'project' argument.\n"
f"Available projects: {project_names}"
)
project = resolved_project
# Check if already cached in context
if context:
cached_project = context.get_state("active_project")
if cached_project and cached_project.name == project:
logger.debug(f"Using cached project from context: {project}")
return cached_project
# Validate project exists by calling API
logger.debug(f"Validating project: {project}")
permalink = generate_permalink(project)
response = await call_get(client, f"/{permalink}/project/item", headers=headers)
active_project = ProjectItem.model_validate(response.json())
# Cache in context if available
if context:
context.set_state("active_project", active_project)
logger.debug(f"Cached project in context: {project}")
logger.debug(f"Validated project: {active_project.name}")
return active_project
def add_project_metadata(result: str, project_name: str) -> str:
"""Add project context as metadata footer for assistant session tracking.
Provides clear project context to help the assistant remember which
project is being used throughout the conversation session.
Args:
result: The tool result string
project_name: The project name that was used
Returns:
Result with project session tracking metadata
"""
return f"{result}\n\n[Session: Using project '{project_name}']"
-118
View File
@@ -1,118 +0,0 @@
"""Project session management for Basic Memory MCP server.
Provides simple in-memory project context for MCP tools, allowing users to switch
between projects during a conversation without restarting the server.
"""
from dataclasses import dataclass
from typing import Optional
from loguru import logger
from basic_memory.config import ProjectConfig, get_project_config, config_manager
@dataclass
class ProjectSession:
"""Simple in-memory project context for MCP session.
This class manages the current project context that tools use when no explicit
project is specified. It's initialized with the default project from config
and can be changed during the conversation.
"""
current_project: Optional[str] = None
default_project: Optional[str] = None
def initialize(self, default_project: str) -> None:
"""Set the default project from config on startup.
Args:
default_project: The project name from configuration
"""
self.default_project = default_project
self.current_project = default_project
logger.info(f"Initialized project session with default project: {default_project}")
def get_current_project(self) -> str:
"""Get the currently active project name.
Returns:
The current project name, falling back to default, then 'main'
"""
return self.current_project or self.default_project or "main"
def set_current_project(self, project_name: str) -> None:
"""Set the current project context.
Args:
project_name: The project to switch to
"""
previous = self.current_project
self.current_project = project_name
logger.info(f"Switched project context: {previous} -> {project_name}")
def get_default_project(self) -> str:
"""Get the default project name from startup.
Returns:
The default project name, or 'main' if not set
"""
return self.default_project or "main" # pragma: no cover
def reset_to_default(self) -> None: # pragma: no cover
"""Reset current project back to the default project."""
self.current_project = self.default_project # pragma: no cover
logger.info(f"Reset project context to default: {self.default_project}") # pragma: no cover
def refresh_from_config(self) -> None:
"""Refresh session state from current configuration.
This method reloads the default project from config and reinitializes
the session. This should be called when the default project is changed
via CLI or API to ensure MCP session stays in sync.
"""
# Reload config to get latest default project
current_config = config_manager.load_config()
new_default = current_config.default_project
# Reinitialize with new default
self.initialize(new_default)
logger.info(f"Refreshed project session from config, new default: {new_default}")
# Global session instance
session = ProjectSession()
def get_active_project(project_override: Optional[str] = None) -> ProjectConfig:
"""Get the active project name for a tool call.
This is the main function tools should use to determine which project
to operate on.
Args:
project_override: Optional explicit project name from tool parameter
Returns:
The project name to use (override takes precedence over session context)
"""
if project_override: # pragma: no cover
project = get_project_config(project_override)
session.set_current_project(project_override)
return project
current_project = session.get_current_project()
return get_project_config(current_project)
def add_project_metadata(result: str, project_name: str) -> str:
"""Add project context as metadata footer for LLM awareness.
Args:
result: The tool result string
project_name: The project name that was used
Returns:
Result with project metadata footer
"""
return f"{result}\n\n<!-- Project: {project_name} -->" # pragma: no cover
-2
View File
@@ -10,12 +10,10 @@ from basic_memory.mcp.prompts import continue_conversation
from basic_memory.mcp.prompts import recent_activity
from basic_memory.mcp.prompts import search
from basic_memory.mcp.prompts import ai_assistant_guide
from basic_memory.mcp.prompts import sync_status
__all__ = [
"ai_assistant_guide",
"continue_conversation",
"recent_activity",
"search",
"sync_status",
]
@@ -1,5 +1,6 @@
from pathlib import Path
from basic_memory.config import ConfigManager
from basic_memory.mcp.server import mcp
from loguru import logger
@@ -12,14 +13,58 @@ from loguru import logger
def ai_assistant_guide() -> str:
"""Return a concise guide on Basic Memory tools and how to use them.
Args:
focus: Optional area to focus on ("writing", "context", "search", etc.)
Dynamically adapts instructions based on configuration:
- Default project mode: Simplified instructions with automatic project
- Regular mode: Project discovery and selection guidance
- CLI constraint mode: Single project constraint information
Returns:
A focused guide on Basic Memory usage.
"""
logger.info("Loading AI assistant guide resource")
# Load base guide content
guide_doc = Path(__file__).parent.parent / "resources" / "ai_assistant_guide.md"
content = guide_doc.read_text(encoding="utf-8")
logger.info(f"Loaded AI assistant guide ({len(content)} chars)")
return content
# Check configuration for mode-specific instructions
config = ConfigManager().config
# Add mode-specific header
mode_info = ""
if config.default_project_mode:
mode_info = f"""
# 🎯 Default Project Mode Active
**Current Configuration**: All operations automatically use project '{config.default_project}'
**Simplified Usage**: You don't need to specify the project parameter in tool calls.
- `write_note(title="Note", content="...", folder="docs")`
- Project parameter is optional and will default to '{config.default_project}'
- To use a different project, explicitly specify: `project="other-project"`
"""
else:
mode_info = """
# 🔧 Multi-Project Mode Active
**Current Configuration**: Project parameter required for all operations
**Project Discovery Required**: Use these tools to select a project:
- `list_memory_projects()` - See all available projects
- `recent_activity()` - Get project activity and recommendations
- Remember the user's project choice throughout the conversation
"""
# Prepend mode info to the guide
enhanced_content = mode_info + content
logger.info(
f"Loaded AI assistant guide ({len(enhanced_content)} chars) with mode: {'default_project' if config.default_project_mode else 'multi_project'}"
)
return enhanced_content
@@ -18,7 +18,7 @@ from basic_memory.schemas.prompt import ContinueConversationRequest
@mcp.prompt(
name="Continue Conversation",
name="continue_conversation",
description="Continue a previous conversation",
)
async def continue_conversation(
+116 -32
View File
@@ -3,7 +3,7 @@
These prompts help users see what has changed in their knowledge base recently.
"""
from typing import Annotated
from typing import Annotated, Optional
from loguru import logger
from pydantic import Field
@@ -12,49 +12,83 @@ from basic_memory.mcp.prompts.utils import format_prompt_context, PromptContext,
from basic_memory.mcp.server import mcp
from basic_memory.mcp.tools.recent_activity import recent_activity
from basic_memory.schemas.base import TimeFrame
from basic_memory.schemas.memory import GraphContext, ProjectActivitySummary
from basic_memory.schemas.search import SearchItemType
@mcp.prompt(
name="Share Recent Activity",
description="Get recent activity from across the knowledge base",
name="recent_activity",
description="Get recent activity from a specific project or across all projects",
)
async def recent_activity_prompt(
timeframe: Annotated[
TimeFrame,
Field(description="How far back to look for activity (e.g. '1d', '1 week')"),
] = "7d",
project: Annotated[
Optional[str],
Field(
description="Specific project to get activity from (None for discovery across all projects)"
),
] = None,
) -> str:
"""Get recent activity from across the knowledge base.
"""Get recent activity from a specific project or across all projects.
This prompt helps you see what's changed recently in the knowledge base,
showing new or updated documents and related information.
This prompt helps you see what's changed recently in the knowledge base.
In discovery mode (project=None), it shows activity across all projects.
In project-specific mode, it shows detailed activity for one project.
Args:
timeframe: How far back to look for activity (e.g. '1d', '1 week')
project: Specific project to get activity from (None for discovery across all projects)
Returns:
Formatted summary of recent activity
"""
logger.info(f"Getting recent activity, timeframe: {timeframe}")
logger.info(f"Getting recent activity, timeframe: {timeframe}, project: {project}")
recent = await recent_activity.fn(timeframe=timeframe, type=[SearchItemType.ENTITY])
recent = await recent_activity.fn(
project=project, timeframe=timeframe, type=[SearchItemType.ENTITY]
)
# Extract primary results from the hierarchical structure
primary_results = []
related_results = []
if recent.results:
# Take up to 5 primary results
for item in recent.results[:5]:
primary_results.append(item.primary_result)
# Add up to 2 related results per primary item
if item.related_results:
related_results.extend(item.related_results[:2])
if isinstance(recent, ProjectActivitySummary):
# Discovery mode - extract results from all projects
for _, project_activity in recent.projects.items():
if project_activity.activity.results:
# Take up to 2 primary results per project
for item in project_activity.activity.results[:2]:
primary_results.append(item.primary_result)
# Add up to 1 related result per primary item
if item.related_results:
related_results.extend(item.related_results[:1])
# Limit total results for readability
primary_results = primary_results[:8]
related_results = related_results[:6]
elif isinstance(recent, GraphContext):
# Project-specific mode - use existing logic
if recent.results:
# Take up to 5 primary results
for item in recent.results[:5]:
primary_results.append(item.primary_result)
# Add up to 2 related results per primary item
if item.related_results:
related_results.extend(item.related_results[:2])
# Set topic based on mode
if project:
topic = f"Recent Activity in {project} ({timeframe})"
else:
topic = f"Recent Activity Across All Projects ({timeframe})"
prompt_context = format_prompt_context(
PromptContext(
topic=f"Recent Activity from ({timeframe})",
topic=topic,
timeframe=timeframe,
results=[
PromptContextItem(
@@ -65,40 +99,90 @@ async def recent_activity_prompt(
)
)
# Add suggestions for summarizing recent activity
# Add mode-specific suggestions
first_title = "Recent Topic"
if primary_results and len(primary_results) > 0:
first_title = primary_results[0].title
capture_suggestions = f"""
if project:
# Project-specific suggestions
capture_suggestions = f"""
## Opportunity to Capture Activity Summary
Consider creating a summary note of recent activity:
Consider creating a summary note of recent activity in {project}:
```python
await write_note(
"{project}",
title="Activity Summary {timeframe}",
content='''
# Activity Summary for {timeframe}
# Activity Summary for {project} ({timeframe})
## Overview
[Summary of key changes and developments over this period]
[Summary of key changes and developments in this project over this period]
## Key Updates
[List main updates and their significance]
[List main updates and their significance within this project]
## Observations
- [trend] [Observation about patterns in recent activity]
- [insight] [Connection between different activities]
## Relations
- summarizes [[{first_title}]]
- relates_to [[Project Overview]]
'''
- relates_to [[{project} Overview]]
''',
folder="summaries"
)
```
Summarizing periodic activity helps create high-level insights and connections between topics.
Summarizing periodic activity helps create high-level insights and connections within the project.
"""
else:
# Discovery mode suggestions
project_count = len(recent.projects) if isinstance(recent, ProjectActivitySummary) else 0
most_active = (
getattr(recent.summary, "most_active_project", "Unknown")
if isinstance(recent, ProjectActivitySummary)
else "Unknown"
)
capture_suggestions = f"""
## Cross-Project Activity Discovery
Found activity across {project_count} projects. Most active: **{most_active}**
Consider creating a cross-project summary:
```python
await write_note(
"{most_active if most_active != "Unknown" else "main"}",
title="Cross-Project Activity Summary {timeframe}",
content='''
# Cross-Project Activity Summary ({timeframe})
## Overview
Activity found across {project_count} projects, with {most_active} showing the most activity.
## Key Developments
[Summarize important changes across all projects]
## Project Insights
[Note patterns or connections between projects]
## Observations
- [trend] [Cross-project patterns observed]
- [insight] [Connections between different project activities]
## Relations
- summarizes [[{first_title}]]
- relates_to [[Project Portfolio Overview]]
''',
folder="summaries"
)
```
Cross-project summaries help identify broader trends and project interconnections.
"""
return prompt_context + capture_suggestions
+1 -1
View File
@@ -17,7 +17,7 @@ from basic_memory.schemas.prompt import SearchPromptRequest
@mcp.prompt(
name="Search Knowledge Base",
name="search_knowledge_base",
description="Search across all content in basic-memory",
)
async def search_prompt(
-112
View File
@@ -1,112 +0,0 @@
"""Sync status prompt for Basic Memory MCP server."""
from basic_memory.mcp.server import mcp
@mcp.prompt(
description="""Get sync status with recommendations for AI assistants.
This prompt provides both current sync status and guidance on how
AI assistants should respond when sync operations are in progress or completed.
""",
)
async def sync_status_prompt() -> str:
"""Get sync status with AI assistant guidance.
Returns:
Formatted sync status with AI assistant guidance
"""
try: # pragma: no cover
from basic_memory.services.migration_service import migration_manager
state = migration_manager.state
# Build status report
lines = [
"# Basic Memory Sync Status",
"",
f"**Current Status**: {state.status.value.replace('_', ' ').title()}",
f"**System Ready**: {'Yes' if migration_manager.is_ready else 'No'}",
"",
]
if migration_manager.is_ready:
lines.extend(
[
"✅ **All sync operations completed** - System is fully operational",
"",
"All Basic Memory tools are available and functioning normally.",
"File indexing is complete and knowledge graphs are up to date.",
"You can proceed with any knowledge management tasks.",
]
)
else:
lines.append(f"**Status Message**: {state.message}")
if state.status.value == "in_progress":
if state.projects_total > 0:
progress = f" ({state.projects_migrated}/{state.projects_total})"
lines.append(f"**Progress**: {progress}")
lines.extend(
[
"",
"🔄 **File synchronization in progress** - Processing files and building knowledge graphs",
"",
"**Impact**: Some tools may show status messages instead of normal responses",
"until sync completes (usually 1-3 minutes).",
]
)
elif state.status.value == "failed":
lines.extend(
[
"",
f"❌ **Sync failed**: {state.error or 'Unknown error'}",
"",
"**Impact**: System may have limited functionality until issue is resolved.",
]
)
# Add AI assistant recommendations
if not migration_manager.is_ready:
lines.extend(
[
"",
"---",
"",
"## AI Assistant Recommendations",
"",
"**When sync is in progress:**",
"- Inform the user about the background file processing",
"- Suggest using `sync_status()` tool to check progress",
"- Explain that tools will work normally once sync completes",
"- Avoid creating complex workflows until sync is done",
"",
"**What to tell users:**",
"- 'Basic Memory is processing your files and building knowledge graphs'",
"- 'This usually takes 1-3 minutes depending on your content size'",
"- 'You can check progress anytime with the sync_status tool'",
"- 'Full functionality will be available once processing completes'",
"",
"**User-friendly language:**",
"- Say 'processing files' instead of 'migration' or 'sync'",
"- Say 'building knowledge graphs' instead of 'indexing'",
"- Say 'setting up your knowledge base' instead of 'running migrations'",
]
)
return "\n".join(lines)
except Exception as e: # pragma: no cover
return f"""# Sync Status - Error
**Unable to check sync status**: {str(e)}
## AI Assistant Recommendations
**When status is unavailable:**
- Assume the system is likely working normally
- Try proceeding with normal operations
- If users report issues, suggest checking logs or restarting
- Use user-friendly language about 'setting up the knowledge base'
"""
+11 -4
View File
@@ -103,10 +103,17 @@ def format_prompt_context(context: PromptContext) -> str:
added_permalinks.add(primary_permalink)
memory_url = normalize_memory_url(primary_permalink)
# Use permalink if available, otherwise use file_path
if primary_permalink:
memory_url = normalize_memory_url(primary_permalink)
read_command = f'read_note("{primary_permalink}")'
else:
memory_url = f"file://{primary.file_path}"
read_command = f'read_file("{primary.file_path}")'
section = dedent(f"""
--- {memory_url}
## {primary.title}
- **Type**: {primary.type}
""")
@@ -121,8 +128,8 @@ def format_prompt_context(context: PromptContext) -> str:
section += f"\n**Excerpt**:\n{content}\n"
section += dedent(f"""
You can read this document with: `read_note("{primary_permalink}")`
You can read this document with: `{read_command}`
""")
sections.append(section)
@@ -14,6 +14,44 @@ natural conversations. The system automatically creates a semantic knowledge gra
- **Semantic**: Simple patterns create a structured knowledge graph
- **Persistent**: Knowledge persists across sessions and conversations
## Project Management and Configuration
Basic Memory uses a **stateless architecture** where each tool call can specify which project to work with. This provides three ways to determine the active project:
### Three-Tier Project Resolution
1. **CLI Constraint (Highest Priority)**: When Basic Memory is started with `--project project-name`, all operations are constrained to that project
2. **Explicit Project Parameter (Medium Priority)**: When you specify `project="project-name"` in tool calls
3. **Default Project Mode (Lowest Priority)**: When `default_project_mode=true` in configuration, tools automatically use the configured `default_project`
### Default Project Mode
When `default_project_mode` is enabled in the user's configuration:
- All tools become more convenient - no need to specify project repeatedly
- Perfect for users who primarily work with a single project
- Still allows explicit project specification when needed
- Falls back gracefully to multi-project mode if no default is configured
```python
# With default_project_mode enabled, these are equivalent:
await write_note("My Note", "Content", "folder")
await write_note("My Note", "Content", "folder", project="default-project")
# You can still override with explicit project:
await write_note("My Note", "Content", "folder", project="other-project")
```
### Project Discovery
If you're unsure which project to use:
```python
# Discover available projects
projects = await list_memory_projects()
# See recent activity across projects for recommendations
activity = await recent_activity() # Shows cross-project activity and suggestions
```
## The Importance of the Knowledge Graph
**Basic Memory's value comes from connections between notes, not just the notes themselves.**
@@ -33,49 +71,120 @@ build these connections!
## Core Tools Reference
### Knowledge Creation and Editing
```python
# Writing knowledge - THE MOST IMPORTANT TOOL!
response = await write_note(
title="Search Design", # Required: Note title
content="# Search Design\n...", # Required: Note content
folder="specs", # Optional: Folder to save in
folder="specs", # Required: Folder to save in
tags=["search", "design"], # Optional: Tags for categorization
verbose=True # Optional: Get parsing details
project="my-project" # Optional: Explicit project (uses default if not specified)
)
# Editing existing notes
await edit_note(
identifier="Search Design", # Required: Note to edit
operation="append", # Required: append, prepend, find_replace, replace_section
content="\n## New Section\nAdditional content", # Required: Content to add/replace
project="my-project" # Optional: Explicit project
)
# Moving notes
await move_note(
identifier="Search Design", # Required: Note to move
destination_path="archive/old-search-design.md", # Required: New location
project="my-project" # Optional: Explicit project
)
# Deleting notes
success = await delete_note(
identifier="Old Draft", # Required: Note to delete
project="my-project" # Optional: Explicit project
)
```
### Knowledge Reading and Discovery
```python
# Reading knowledge
content = await read_note("Search Design") # By title
content = await read_note("Search Design") # By title (uses default project)
content = await read_note("specs/search-design") # By path
content = await read_note("memory://specs/search") # By memory URL
content = await read_note("Search Design", project="work-docs") # Explicit project
# Reading raw file content (text, images, binaries)
file_data = await read_content(
path="assets/diagram.png", # Required: File path
project="my-project" # Optional: Explicit project
)
# Viewing notes as formatted artifacts
await view_note(
identifier="Search Design", # Required: Note to view
project="my-project", # Optional: Explicit project
page=1, # Optional: Pagination
page_size=10 # Optional: Items per page
)
# Browsing directory contents
listing = await list_directory(
dir_name="/specs", # Optional: Directory path (default: "/")
depth=2, # Optional: Recursion depth
file_name_glob="*.md", # Optional: File pattern filter
project="my-project" # Optional: Explicit project
)
```
### Search and Context
```python
# Searching for knowledge
results = await search_notes(
query="authentication system", # Text to search for
query="authentication system", # Required: Text to search for
project="my-project", # Optional: Explicit project
page=1, # Optional: Pagination
page_size=10 # Optional: Results per page
page_size=10, # Optional: Results per page
search_type="text", # Optional: "text", "title", or "permalink"
types=["entity"], # Optional: Filter by content types
entity_types=["observation"], # Optional: Filter by entity types
after_date="1 week" # Optional: Recent content only
)
# Building context from the knowledge graph
context = await build_context(
url="memory://specs/search", # Starting point
url="memory://specs/search", # Required: Starting point
project="my-project", # Optional: Explicit project
depth=2, # Optional: How many hops to follow
timeframe="1 month" # Optional: Recent timeframe
timeframe="1 month", # Optional: Recent timeframe
max_related=10 # Optional: Max related items
)
# Checking recent changes
activity = await recent_activity(
type="all", # Optional: Entity types to include
type=["entity", "relation"], # Optional: Entity types to include
depth=1, # Optional: Related items to include
timeframe="1 week" # Optional: Time window
timeframe="1 week", # Optional: Time window
project="my-project" # Optional: Explicit project (None for cross-project discovery)
)
```
### Visualization and Project Management
```python
# Creating a knowledge visualization
canvas_result = await canvas(
nodes=[{"id": "note1", "label": "Search Design"}], # Nodes to display
edges=[{"from": "note1", "to": "note2"}], # Connections
title="Project Overview", # Canvas title
folder="diagrams" # Storage location
nodes=[{"id": "note1", "type": "file", "file": "Search Design.md"}], # Required: Nodes
edges=[{"id": "edge1", "fromNode": "note1", "toNode": "note2"}], # Required: Edges
title="Project Overview", # Required: Canvas title
folder="diagrams", # Required: Storage location
project="my-project" # Optional: Explicit project
)
# Project management
projects = await list_memory_projects() # List all available projects
project_info = await project_info(project="my-project") # Get project statistics
```
## memory:// URLs Explained
@@ -135,27 +244,31 @@ Users will interact with Basic Memory in patterns like:
1. **Creating knowledge**:
```
Human: "Let's write up what we discussed about search."
You: I'll create a note capturing our discussion about the search functionality.
[Use write_note() to record the conversation details]
await write_note(
title="Search Functionality Discussion",
content="# Search Functionality Discussion\n...",
folder="discussions"
)
```
2. **Referencing existing knowledge**:
```
Human: "Take a look at memory://specs/search"
You: I'll examine that information.
[Use build_context() to gather related information]
[Then read_note() to access specific content]
context = await build_context(url="memory://specs/search")
content = await read_note("specs/search")
```
3. **Finding information**:
```
Human: "What were our decisions about auth?"
You: Let me find that information for you.
[Use search_notes() to find relevant notes]
[Then build_context() to understand connections]
results = await search_notes(query="auth decisions")
context = await build_context(url=f"memory://{results[0].permalink}")
```
## Key Things to Remember
@@ -263,7 +376,7 @@ When creating relations, you can:
# Example workflow for creating notes with effective relations
async def create_note_with_effective_relations():
# Search for existing entities to reference
search_results = await search_notes("travel")
search_results = await search_notes(query="travel")
existing_entities = [result.title for result in search_results.primary_results]
# Check if specific entities exist
@@ -297,7 +410,7 @@ async def create_note_with_effective_relations():
# Now create the note with both verified and forward relations
content = f"""# Tokyo Neighborhood Guide
## Overview
Details about different Tokyo neighborhoods and their unique characteristics.
@@ -313,7 +426,7 @@ Details about different Tokyo neighborhoods and their unique characteristics.
result = await write_note(
title="Tokyo Neighborhood Guide",
content=content,
verbose=True
folder="travel"
)
# You can check which relations were resolved and which are forward references
@@ -335,7 +448,7 @@ Common issues to watch for:
content = await read_note("Document")
except:
# Try search instead
results = await search_notes("Document")
results = await search_notes(query="Document")
if results and results.primary_results:
# Found something similar
content = await read_note(results.primary_results[0].permalink)
@@ -343,23 +456,40 @@ Common issues to watch for:
2. **Forward References (Unresolved Relations)**
```python
response = await write_note(..., verbose=True)
response = await write_note(
title="My Note",
content="Content with [[Forward Reference]]",
folder="notes"
)
# Check for forward references (unresolved relations)
forward_refs = []
for relation in response.get('relations', []):
if not relation.get('target_id'):
forward_refs.append(relation.get('to_name'))
if forward_refs:
# This is a feature, not an error! Inform the user about forward references
print(f"Note created with forward references to: {forward_refs}")
print("These will be automatically linked when those notes are created.")
# Optionally suggest creating those entities now
print("Would you like me to create any of these notes now to complete the connections?")
```
3. **Sync Issues**
3. **Project Discovery Issues**
```python
# If user asks about content but no default project is configured
try:
results = await search_notes(query="user query")
except Exception as e:
if "project" in str(e).lower():
# Show available projects and ask user to choose
projects = await list_memory_projects()
print(f"Available projects: {[p.name for p in projects]}")
print("Which project should I search in?")
```
4. **Sync Issues**
```python
# If information seems outdated
activity = await recent_activity(timeframe="1 hour")
@@ -369,14 +499,21 @@ Common issues to watch for:
## Best Practices
1. **Proactively Record Context**
1. **Smart Project Management**
- **For new users**: Call `recent_activity()` without project parameter to discover active projects and get recommendations
- **For known projects**: Use explicit project parameters when switching between multiple projects
- **For single-project users**: Rely on default_project_mode for convenience
- **When uncertain**: Use `list_memory_projects()` to show available options and ask the user
- **Remember choices**: Once a user indicates their preferred project, use it consistently throughout the conversation
2. **Proactively Record Context**
- Offer to capture important discussions
- Record decisions, rationales, and conclusions
- Link to related topics
- Ask for permission first: "Would you like me to save our discussion about [topic]?"
- Confirm when complete: "I've saved our discussion to Basic Memory"
2. **Create a Rich Semantic Graph**
3. **Create a Rich Semantic Graph**
- **Add meaningful observations**: Include at least 3-5 categorized observations in each note
- **Create deliberate relations**: Connect each note to at least 2-3 related entities
- **Use existing entities**: Before creating a new relation, search for existing entities
@@ -386,7 +523,7 @@ Common issues to watch for:
of "relates_to")
- **Consider bidirectional relations**: When appropriate, create inverse relations in both entities
3. **Structure Content Thoughtfully**
4. **Structure Content Thoughtfully**
- Use clear, descriptive titles
- Organize with logical sections (Context, Decision, Implementation, etc.)
- Include relevant context and background
@@ -394,20 +531,21 @@ Common issues to watch for:
- Use a consistent format for similar types of notes
- Balance detail with conciseness
4. **Navigate Knowledge Effectively**
- Start with specific searches
- Follow relation paths
5. **Navigate Knowledge Effectively**
- Start with specific searches using `search_notes()`
- Follow relation paths with `build_context()`
- Combine information from multiple sources
- Verify information is current
- Verify information is current with `recent_activity()`
- Build a complete picture before responding
- Use appropriate project context for searches
5. **Help Users Maintain Their Knowledge**
- Suggest organizing related topics
- Identify potential duplicates
6. **Help Users Maintain Their Knowledge**
- Suggest organizing related topics across projects when appropriate
- Identify potential duplicates using search
- Recommend adding relations between topics
- Offer to create summaries of scattered information
- Suggest potential missing relations: "I notice this might relate to [topic], would you like me to add that
connection?"
- Suggest potential missing relations: "I notice this might relate to [topic], would you like me to add that connection?"
- Help users decide when to use explicit vs default project parameters
Built with ♥️ b
y Basic Machines
+21 -7
View File
@@ -1,19 +1,24 @@
"""Project info tool for Basic Memory MCP server."""
from loguru import logger
from typing import Optional
from loguru import logger
from fastmcp import Context
from basic_memory.mcp.project_session import get_active_project
from basic_memory.mcp.async_client import client
from basic_memory.mcp.project_context import get_active_project
from basic_memory.mcp.server import mcp
from basic_memory.mcp.tools.utils import call_get
from basic_memory.schemas import ProjectInfoResponse
@mcp.resource(
uri="memory://project_info",
uri="memory://{project}/info",
description="Get information and statistics about the current Basic Memory project.",
)
async def project_info() -> ProjectInfoResponse:
async def project_info(
project: Optional[str] = None, context: Context | None = None
) -> ProjectInfoResponse:
"""Get comprehensive information about the current Basic Memory project.
This tool provides detailed statistics and status information about your
@@ -31,13 +36,22 @@ async def project_info() -> ProjectInfoResponse:
- Monitor growth and activity over time
- Identify potential issues like unresolved relations
Args:
project: Optional project name. If not provided, uses default_project
(if default_project_mode=true) or CLI constraint. If unknown,
use list_memory_projects() to discover available projects.
context: Optional FastMCP context for performance caching.
Returns:
Detailed project information and statistics
Examples:
# Get information about the current project
# Get information about the current/default project
info = await project_info()
# Get information about a specific project
info = await project_info(project="my-project")
# Check entity counts
print(f"Total entities: {info.statistics.total_entities}")
@@ -45,8 +59,8 @@ async def project_info() -> ProjectInfoResponse:
print(f"Basic Memory version: {info.system.version}")
"""
logger.info("Getting project info")
project_config = get_active_project()
project_url = project_config.project_url
project_config = await get_active_project(client, project, context)
project_url = project_config.permalink
# Call the API endpoint
response = await call_get(client, f"{project_url}/project/info")
-100
View File
@@ -2,108 +2,8 @@
Basic Memory FastMCP server.
"""
import asyncio
from contextlib import asynccontextmanager
from dataclasses import dataclass
from typing import AsyncIterator, Optional, Any
from dotenv import load_dotenv
from fastmcp import FastMCP
from fastmcp.utilities.logging import configure_logging as mcp_configure_logging
from mcp.server.auth.settings import AuthSettings
from basic_memory.config import app_config
from basic_memory.services.initialization import initialize_app
from basic_memory.mcp.auth_provider import BasicMemoryOAuthProvider
from basic_memory.mcp.project_session import session
from basic_memory.mcp.external_auth_provider import (
create_github_provider,
create_google_provider,
)
from basic_memory.mcp.supabase_auth_provider import SupabaseOAuthProvider
# mcp console logging
mcp_configure_logging(level="ERROR")
load_dotenv()
@dataclass
class AppContext:
watch_task: Optional[asyncio.Task]
migration_manager: Optional[Any] = None
@asynccontextmanager
async def app_lifespan(server: FastMCP) -> AsyncIterator[AppContext]: # pragma: no cover
"""Manage application lifecycle with type-safe context"""
# Initialize on startup (now returns migration_manager)
migration_manager = await initialize_app(app_config)
# Initialize project session with default project
session.initialize(app_config.default_project)
try:
yield AppContext(watch_task=None, migration_manager=migration_manager)
finally:
# Cleanup on shutdown - migration tasks will be cancelled automatically
pass
# OAuth configuration function
def create_auth_config() -> tuple[AuthSettings | None, Any | None]:
"""Create OAuth configuration if enabled."""
# Check if OAuth is enabled via environment variable
import os
if os.getenv("FASTMCP_AUTH_ENABLED", "false").lower() == "true":
from pydantic import AnyHttpUrl
# Configure OAuth settings
issuer_url = os.getenv("FASTMCP_AUTH_ISSUER_URL", "http://localhost:8000")
required_scopes = os.getenv("FASTMCP_AUTH_REQUIRED_SCOPES", "read,write")
docs_url = os.getenv("FASTMCP_AUTH_DOCS_URL") or "http://localhost:8000/docs/oauth"
auth_settings = AuthSettings(
issuer_url=AnyHttpUrl(issuer_url),
service_documentation_url=AnyHttpUrl(docs_url),
required_scopes=required_scopes.split(",") if required_scopes else ["read", "write"],
)
# Create OAuth provider based on type
provider_type = os.getenv("FASTMCP_AUTH_PROVIDER", "basic").lower()
if provider_type == "github":
auth_provider = create_github_provider()
elif provider_type == "google":
auth_provider = create_google_provider()
elif provider_type == "supabase":
supabase_url = os.getenv("SUPABASE_URL")
supabase_anon_key = os.getenv("SUPABASE_ANON_KEY")
supabase_service_key = os.getenv("SUPABASE_SERVICE_KEY")
if not supabase_url or not supabase_anon_key:
raise ValueError("SUPABASE_URL and SUPABASE_ANON_KEY must be set for Supabase auth")
auth_provider = SupabaseOAuthProvider(
supabase_url=supabase_url,
supabase_anon_key=supabase_anon_key,
supabase_service_key=supabase_service_key,
issuer_url=issuer_url,
)
else: # default to "basic"
auth_provider = BasicMemoryOAuthProvider(issuer_url=issuer_url)
return auth_settings, auth_provider
return None, None
# Create auth configuration
auth_settings, auth_provider = create_auth_config()
# Create the shared server instance
mcp = FastMCP(
name="Basic Memory",
auth=auth_provider,
)
@@ -1,463 +0,0 @@
"""Supabase OAuth provider for Basic Memory MCP server."""
import os
import secrets
from dataclasses import dataclass
from datetime import datetime, timedelta
from typing import Optional, Dict, Any
import httpx
import jwt
from loguru import logger
from mcp.server.auth.provider import (
OAuthAuthorizationServerProvider,
AuthorizationParams,
AuthorizationCode,
RefreshToken,
AccessToken,
TokenError,
AuthorizeError,
)
from mcp.shared.auth import OAuthClientInformationFull, OAuthToken
@dataclass
class SupabaseAuthorizationCode(AuthorizationCode):
"""Authorization code with Supabase metadata."""
user_id: Optional[str] = None
email: Optional[str] = None
@dataclass
class SupabaseRefreshToken(RefreshToken):
"""Refresh token with Supabase metadata."""
supabase_refresh_token: Optional[str] = None
user_id: Optional[str] = None
@dataclass
class SupabaseAccessToken(AccessToken):
"""Access token with Supabase metadata."""
supabase_access_token: Optional[str] = None
user_id: Optional[str] = None
email: Optional[str] = None
class SupabaseOAuthProvider(
OAuthAuthorizationServerProvider[
SupabaseAuthorizationCode, SupabaseRefreshToken, SupabaseAccessToken
]
):
"""OAuth provider that integrates with Supabase Auth.
This provider uses Supabase as the authentication backend while
maintaining compatibility with MCP's OAuth requirements.
"""
def __init__(
self,
supabase_url: str,
supabase_anon_key: str,
supabase_service_key: Optional[str] = None,
issuer_url: str = "http://localhost:8000",
):
self.supabase_url = supabase_url.rstrip("/")
self.supabase_anon_key = supabase_anon_key
self.supabase_service_key = supabase_service_key or supabase_anon_key
self.issuer_url = issuer_url
# HTTP client for Supabase API calls
self.http_client = httpx.AsyncClient()
# Temporary storage for auth flows (in production, use Supabase DB)
self.pending_auth_codes: Dict[str, SupabaseAuthorizationCode] = {}
self.mcp_to_supabase_tokens: Dict[str, Dict[str, Any]] = {}
async def get_client(self, client_id: str) -> Optional[OAuthClientInformationFull]:
"""Get a client from Supabase.
In production, this would query a clients table in Supabase.
"""
# For now, we'll validate against a configured list of allowed clients
# In production, query Supabase DB for client info
allowed_clients = os.getenv("SUPABASE_ALLOWED_CLIENTS", "").split(",")
if client_id in allowed_clients:
return OAuthClientInformationFull(
client_id=client_id,
client_secret="", # Supabase handles secrets
redirect_uris=[], # Supabase handles redirect URIs
)
return None
async def register_client(self, client_info: OAuthClientInformationFull) -> None:
"""Register a new OAuth client in Supabase.
In production, this would insert into a clients table.
"""
# For development, we just log the registration
logger.info(f"Would register client {client_info.client_id} in Supabase")
# In production:
# await self.supabase.table('oauth_clients').insert({
# 'client_id': client_info.client_id,
# 'client_secret': client_info.client_secret,
# 'metadata': client_info.client_metadata,
# }).execute()
async def authorize(
self, client: OAuthClientInformationFull, params: AuthorizationParams
) -> str:
"""Create authorization URL redirecting to Supabase Auth.
This initiates the OAuth flow with Supabase as the identity provider.
"""
# Generate state for this auth request
state = secrets.token_urlsafe(32)
# Store the authorization request
self.pending_auth_codes[state] = SupabaseAuthorizationCode(
code=state,
scopes=params.scopes or [],
expires_at=(datetime.utcnow() + timedelta(minutes=10)).timestamp(),
client_id=client.client_id,
code_challenge=params.code_challenge,
redirect_uri=params.redirect_uri,
redirect_uri_provided_explicitly=params.redirect_uri_provided_explicitly,
)
# Build Supabase auth URL
auth_params = {
"redirect_to": f"{self.issuer_url}/auth/callback",
"scopes": " ".join(params.scopes or ["openid", "email"]),
"state": state,
}
# Use Supabase's OAuth endpoint
auth_url = f"{self.supabase_url}/auth/v1/authorize"
query_string = "&".join(f"{k}={v}" for k, v in auth_params.items())
return f"{auth_url}?{query_string}"
async def handle_supabase_callback(self, code: str, state: str) -> str:
"""Handle callback from Supabase after user authentication."""
# Get the original auth request
auth_request = self.pending_auth_codes.get(state)
if not auth_request:
raise AuthorizeError(
error="invalid_request",
error_description="Invalid state parameter",
)
# Exchange code with Supabase for tokens
token_response = await self.http_client.post(
f"{self.supabase_url}/auth/v1/token",
json={
"grant_type": "authorization_code",
"code": code,
"redirect_uri": f"{self.issuer_url}/auth/callback",
},
headers={
"apikey": self.supabase_anon_key,
"Authorization": f"Bearer {self.supabase_anon_key}",
},
)
if not token_response.is_success:
raise AuthorizeError(
error="server_error",
error_description="Failed to exchange code with Supabase",
)
supabase_tokens = token_response.json()
# Get user info from Supabase
user_response = await self.http_client.get(
f"{self.supabase_url}/auth/v1/user",
headers={
"apikey": self.supabase_anon_key,
"Authorization": f"Bearer {supabase_tokens['access_token']}",
},
)
user_data = user_response.json() if user_response.is_success else {}
# Generate MCP authorization code
mcp_code = secrets.token_urlsafe(32)
# Update auth request with user info
auth_request.code = mcp_code
auth_request.user_id = user_data.get("id")
auth_request.email = user_data.get("email")
# Store mapping
self.pending_auth_codes[mcp_code] = auth_request
self.mcp_to_supabase_tokens[mcp_code] = {
"supabase_tokens": supabase_tokens,
"user": user_data,
}
# Clean up old state
del self.pending_auth_codes[state]
# Redirect back to client
redirect_uri = str(auth_request.redirect_uri)
separator = "&" if "?" in redirect_uri else "?"
return f"{redirect_uri}{separator}code={mcp_code}&state={state}"
async def load_authorization_code(
self, client: OAuthClientInformationFull, authorization_code: str
) -> Optional[SupabaseAuthorizationCode]:
"""Load an authorization code."""
code = self.pending_auth_codes.get(authorization_code)
if code and code.client_id == client.client_id:
# Check expiration
if datetime.utcnow().timestamp() > code.expires_at:
del self.pending_auth_codes[authorization_code]
return None
return code
return None
async def exchange_authorization_code(
self, client: OAuthClientInformationFull, authorization_code: SupabaseAuthorizationCode
) -> OAuthToken:
"""Exchange authorization code for tokens."""
# Get stored Supabase tokens
token_data = self.mcp_to_supabase_tokens.get(authorization_code.code)
if not token_data:
raise TokenError(error="invalid_grant", error_description="Invalid authorization code")
supabase_tokens = token_data["supabase_tokens"]
user = token_data["user"]
# Generate MCP tokens that wrap Supabase tokens
access_token = self._generate_mcp_token(
client_id=client.client_id,
user_id=user.get("id", ""),
email=user.get("email", ""),
scopes=authorization_code.scopes,
supabase_access_token=supabase_tokens["access_token"],
)
refresh_token = secrets.token_urlsafe(32)
# Store the token mapping
self.mcp_to_supabase_tokens[access_token] = {
"client_id": client.client_id,
"user_id": user.get("id"),
"email": user.get("email"),
"supabase_access_token": supabase_tokens["access_token"],
"supabase_refresh_token": supabase_tokens["refresh_token"],
"scopes": authorization_code.scopes,
}
# Store refresh token mapping
self.mcp_to_supabase_tokens[refresh_token] = {
"client_id": client.client_id,
"user_id": user.get("id"),
"supabase_refresh_token": supabase_tokens["refresh_token"],
"scopes": authorization_code.scopes,
}
# Clean up authorization code
del self.pending_auth_codes[authorization_code.code]
del self.mcp_to_supabase_tokens[authorization_code.code]
return OAuthToken(
access_token=access_token,
token_type="bearer",
expires_in=supabase_tokens.get("expires_in", 3600),
refresh_token=refresh_token,
scope=" ".join(authorization_code.scopes) if authorization_code.scopes else None,
)
async def load_refresh_token(
self, client: OAuthClientInformationFull, refresh_token: str
) -> Optional[SupabaseRefreshToken]:
"""Load a refresh token."""
token_data = self.mcp_to_supabase_tokens.get(refresh_token)
if token_data and token_data["client_id"] == client.client_id:
return SupabaseRefreshToken(
token=refresh_token,
client_id=client.client_id,
scopes=token_data["scopes"],
supabase_refresh_token=token_data["supabase_refresh_token"],
user_id=token_data.get("user_id"),
)
return None
async def exchange_refresh_token(
self,
client: OAuthClientInformationFull,
refresh_token: SupabaseRefreshToken,
scopes: list[str],
) -> OAuthToken:
"""Exchange refresh token for new tokens using Supabase."""
# Refresh with Supabase
token_response = await self.http_client.post(
f"{self.supabase_url}/auth/v1/token",
json={
"grant_type": "refresh_token",
"refresh_token": refresh_token.supabase_refresh_token,
},
headers={
"apikey": self.supabase_anon_key,
"Authorization": f"Bearer {self.supabase_anon_key}",
},
)
if not token_response.is_success:
raise TokenError(
error="invalid_grant",
error_description="Failed to refresh with Supabase",
)
supabase_tokens = token_response.json()
# Get updated user info
user_response = await self.http_client.get(
f"{self.supabase_url}/auth/v1/user",
headers={
"apikey": self.supabase_anon_key,
"Authorization": f"Bearer {supabase_tokens['access_token']}",
},
)
user_data = user_response.json() if user_response.is_success else {}
# Generate new MCP tokens
new_access_token = self._generate_mcp_token(
client_id=client.client_id,
user_id=user_data.get("id", ""),
email=user_data.get("email", ""),
scopes=scopes or refresh_token.scopes,
supabase_access_token=supabase_tokens["access_token"],
)
new_refresh_token = secrets.token_urlsafe(32)
# Update token mappings
self.mcp_to_supabase_tokens[new_access_token] = {
"client_id": client.client_id,
"user_id": user_data.get("id"),
"email": user_data.get("email"),
"supabase_access_token": supabase_tokens["access_token"],
"supabase_refresh_token": supabase_tokens["refresh_token"],
"scopes": scopes or refresh_token.scopes,
}
self.mcp_to_supabase_tokens[new_refresh_token] = {
"client_id": client.client_id,
"user_id": user_data.get("id"),
"supabase_refresh_token": supabase_tokens["refresh_token"],
"scopes": scopes or refresh_token.scopes,
}
# Clean up old tokens
del self.mcp_to_supabase_tokens[refresh_token.token]
return OAuthToken(
access_token=new_access_token,
token_type="bearer",
expires_in=supabase_tokens.get("expires_in", 3600),
refresh_token=new_refresh_token,
scope=" ".join(scopes or refresh_token.scopes),
)
async def load_access_token(self, token: str) -> Optional[SupabaseAccessToken]:
"""Load and validate an access token."""
# First check our mapping
token_data = self.mcp_to_supabase_tokens.get(token)
if token_data:
return SupabaseAccessToken(
token=token,
client_id=token_data["client_id"],
scopes=token_data["scopes"],
supabase_access_token=token_data.get("supabase_access_token"),
user_id=token_data.get("user_id"),
email=token_data.get("email"),
)
# Try to decode as JWT
try:
# Verify with Supabase's JWT secret
payload = jwt.decode(
token,
os.getenv("SUPABASE_JWT_SECRET", ""),
algorithms=["HS256"],
audience="authenticated",
)
return SupabaseAccessToken(
token=token,
client_id=payload.get("client_id", ""),
scopes=payload.get("scopes", []),
user_id=payload.get("sub"),
email=payload.get("email"),
)
except jwt.InvalidTokenError:
pass
# Validate with Supabase
user_response = await self.http_client.get(
f"{self.supabase_url}/auth/v1/user",
headers={
"apikey": self.supabase_anon_key,
"Authorization": f"Bearer {token}",
},
)
if user_response.is_success:
user_data = user_response.json()
return SupabaseAccessToken(
token=token,
client_id="", # Unknown client for direct Supabase tokens
scopes=[],
supabase_access_token=token,
user_id=user_data.get("id"),
email=user_data.get("email"),
)
return None
async def revoke_token(self, token: SupabaseAccessToken | SupabaseRefreshToken) -> None:
"""Revoke a token."""
# Remove from our mapping
self.mcp_to_supabase_tokens.pop(token.token, None)
# In production, also revoke in Supabase:
# await self.supabase.auth.admin.sign_out(token.user_id)
def _generate_mcp_token(
self,
client_id: str,
user_id: str,
email: str,
scopes: list[str],
supabase_access_token: str,
) -> str:
"""Generate an MCP token that wraps Supabase authentication."""
payload = {
"iss": self.issuer_url,
"sub": user_id,
"client_id": client_id,
"email": email,
"scopes": scopes,
"supabase_token": supabase_access_token[:10] + "...", # Reference only
"exp": datetime.utcnow() + timedelta(hours=1),
"iat": datetime.utcnow(),
}
# Use Supabase JWT secret if available
secret = os.getenv("SUPABASE_JWT_SECRET", secrets.token_urlsafe(32))
return jwt.encode(payload, secret, algorithm="HS256")
+9 -10
View File
@@ -20,31 +20,30 @@ from basic_memory.mcp.tools.edit_note import edit_note
from basic_memory.mcp.tools.move_note import move_note
from basic_memory.mcp.tools.sync_status import sync_status
from basic_memory.mcp.tools.project_management import (
list_projects,
switch_project,
get_current_project,
set_default_project,
create_project,
list_memory_projects,
create_memory_project,
delete_project,
)
# ChatGPT-compatible tools
from basic_memory.mcp.tools.chatgpt_tools import search, fetch
__all__ = [
"build_context",
"canvas",
"create_project",
"create_memory_project",
"delete_note",
"delete_project",
"edit_note",
"get_current_project",
"fetch",
"list_directory",
"list_projects",
"list_memory_projects",
"move_note",
"read_content",
"read_note",
"recent_activity",
"search",
"search_notes",
"set_default_project",
"switch_project",
"sync_status",
"view_note",
"write_note",
+39 -19
View File
@@ -3,11 +3,12 @@
from typing import Optional
from loguru import logger
from fastmcp import Context
from basic_memory.mcp.async_client import client
from basic_memory.mcp.project_context import get_active_project
from basic_memory.mcp.server import mcp
from basic_memory.mcp.tools.utils import call_get
from basic_memory.mcp.project_session import get_active_project
from basic_memory.schemas.base import TimeFrame
from basic_memory.schemas.memory import (
GraphContext,
@@ -15,19 +16,21 @@ from basic_memory.schemas.memory import (
memory_url_path,
)
type StringOrInt = str | int
@mcp.tool(
description="""Build context from a memory:// URI to continue conversations naturally.
Use this to follow up on previous discussions or explore related topics.
Memory URL Format:
- Use paths like "folder/note" or "memory://folder/note"
- Use paths like "folder/note" or "memory://folder/note"
- Pattern matching: "folder/*" matches all notes in folder
- Valid characters: letters, numbers, hyphens, underscores, forward slashes
- Avoid: double slashes (//), angle brackets (<>), quotes, pipes (|)
- Examples: "specs/search", "projects/basic-memory", "notes/*"
Timeframes support natural language like:
- "2 days ago", "last week", "today", "3 months ago"
- Or standard formats like "7d", "24h"
@@ -35,27 +38,34 @@ from basic_memory.schemas.memory import (
)
async def build_context(
url: MemoryUrl,
depth: Optional[int] = 1,
project: Optional[str] = None,
depth: Optional[StringOrInt] = 1,
timeframe: Optional[TimeFrame] = "7d",
page: int = 1,
page_size: int = 10,
max_related: int = 10,
project: Optional[str] = None,
context: Context | None = None,
) -> GraphContext:
"""Get context needed to continue a discussion.
"""Get context needed to continue a discussion within a specific project.
This tool enables natural continuation of discussions by loading relevant context
from memory:// URIs. It uses pattern matching to find relevant content and builds
a rich context graph of related information.
Project Resolution:
Server resolves projects in this order: Single Project Mode project parameter default project.
If project unknown, use list_memory_projects() or recent_activity() first.
Args:
project: Project name to build context from. Optional - server will resolve using hierarchy.
If unknown, use list_memory_projects() to discover available projects.
url: memory:// URI pointing to discussion content (e.g. memory://specs/search)
depth: How many relation hops to traverse (1-3 recommended for performance)
timeframe: How far back to look. Supports natural language like "2 days ago", "last week"
page: Page number of results to return (default: 1)
page_size: Number of results to return per page (default: 10)
max_related: Maximum number of related results to return (default: 10)
project: Optional project name to build context from. If not provided, uses current active project.
context: Optional FastMCP context for performance caching.
Returns:
GraphContext containing:
@@ -65,25 +75,35 @@ async def build_context(
Examples:
# Continue a specific discussion
build_context("memory://specs/search")
build_context("my-project", "memory://specs/search")
# Get deeper context about a component
build_context("memory://components/memory-service", depth=2)
build_context("work-docs", "memory://components/memory-service", depth=2)
# Look at recent changes to a specification
build_context("memory://specs/document-format", timeframe="today")
build_context("research", "memory://specs/document-format", timeframe="today")
# Research the history of a feature
build_context("memory://features/knowledge-graph", timeframe="3 months ago")
build_context("dev-notes", "memory://features/knowledge-graph", timeframe="3 months ago")
# Build context from specific project
build_context("memory://specs/search", project="work-project")
Raises:
ToolError: If project doesn't exist or depth parameter is invalid
"""
logger.info(f"Building context from {url}")
logger.info(f"Building context from {url} in project {project}")
# Convert string depth to integer if needed
if isinstance(depth, str):
try:
depth = int(depth)
except ValueError:
from mcp.server.fastmcp.exceptions import ToolError
raise ToolError(f"Invalid depth parameter: '{depth}' is not a valid integer")
# URL is already validated and normalized by MemoryUrl type annotation
# Get the active project first to check project-specific sync status
active_project = get_active_project(project)
# Get the active project using the new stateless approach
active_project = await get_active_project(client, project, context)
# Check migration status and wait briefly if needed
from basic_memory.mcp.tools.utils import wait_for_migration_or_return_status
@@ -101,7 +121,7 @@ async def build_context(
metadata=MemoryMetadata(
depth=depth or 1,
timeframe=timeframe,
generated_at=datetime.now(),
generated_at=datetime.now().astimezone(),
primary_count=0,
related_count=0,
uri=migration_status, # Include status in metadata
+19 -8
View File
@@ -7,11 +7,12 @@ import json
from typing import Dict, List, Any, Optional
from loguru import logger
from fastmcp import Context
from basic_memory.mcp.async_client import client
from basic_memory.mcp.project_context import get_active_project
from basic_memory.mcp.server import mcp
from basic_memory.mcp.tools.utils import call_put
from basic_memory.mcp.project_session import get_active_project
@mcp.tool(
@@ -23,21 +24,28 @@ async def canvas(
title: str,
folder: str,
project: Optional[str] = None,
context: Context | None = None,
) -> str:
"""Create an Obsidian canvas file with the provided nodes and edges.
This tool creates a .canvas file compatible with Obsidian's Canvas feature,
allowing visualization of relationships between concepts or documents.
Project Resolution:
Server resolves projects in this order: Single Project Mode project parameter default project.
If project unknown, use list_memory_projects() or recent_activity() first.
For the full JSON Canvas 1.0 specification, see the 'spec://canvas' resource.
Args:
project: Project name to create canvas in. Optional - server will resolve using hierarchy.
If unknown, use list_memory_projects() to discover available projects.
nodes: List of node objects following JSON Canvas 1.0 spec
edges: List of edge objects following JSON Canvas 1.0 spec
title: The title of the canvas (will be saved as title.canvas)
folder: Folder path relative to project root where the canvas should be saved.
Use forward slashes (/) as separators. Examples: "diagrams", "projects/2025", "visual/maps"
project: Optional project name to create canvas in. If not provided, uses current active project.
context: Optional FastMCP context for performance caching.
Returns:
A summary of the created canvas file
@@ -77,13 +85,16 @@ async def canvas(
```
Examples:
# Create canvas in current project
canvas(nodes=[...], edges=[...], title="My Canvas", folder="diagrams")
# Create canvas in project
canvas("my-project", nodes=[...], edges=[...], title="My Canvas", folder="diagrams")
# Create canvas in specific project
canvas(nodes=[...], edges=[...], title="My Canvas", folder="diagrams", project="work-project")
# Create canvas in work project
canvas("work-project", nodes=[...], edges=[...], title="Process Flow", folder="visual/maps")
Raises:
ToolError: If project doesn't exist or folder path is invalid
"""
active_project = get_active_project(project)
active_project = await get_active_project(client, project, context)
project_url = active_project.project_url
# Ensure path has .canvas extension
@@ -97,7 +108,7 @@ async def canvas(
canvas_json = json.dumps(canvas_data, indent=2)
# Write the file using the resource API
logger.info(f"Creating canvas file: {file_path}")
logger.info(f"Creating canvas file: {file_path} in project {project}")
response = await call_put(client, f"{project_url}/resource/{file_path}", json=canvas_json)
# Parse response
+178
View File
@@ -0,0 +1,178 @@
"""ChatGPT-compatible MCP tools for Basic Memory.
These adapters expose Basic Memory's search/fetch functionality using the exact
tool names and response structure OpenAI's MCP clients expect: each call returns
a list containing a single `{"type": "text", "text": "{...json...}"}` item.
"""
import json
from typing import Any, Dict, List, Optional
from loguru import logger
from fastmcp import Context
from basic_memory.mcp.server import mcp
from basic_memory.mcp.tools.search import search_notes
from basic_memory.mcp.tools.read_note import read_note
from basic_memory.schemas.search import SearchResponse
def _format_search_results_for_chatgpt(results: SearchResponse) -> List[Dict[str, Any]]:
"""Format search results according to ChatGPT's expected schema.
Returns a list of result objects with id, title, and url fields.
"""
formatted_results = []
for result in results.results:
formatted_result = {
"id": result.permalink or f"doc-{len(formatted_results)}",
"title": result.title if result.title and result.title.strip() else "Untitled",
"url": result.permalink or "",
}
formatted_results.append(formatted_result)
return formatted_results
def _format_document_for_chatgpt(
content: str, identifier: str, title: Optional[str] = None
) -> Dict[str, Any]:
"""Format document content according to ChatGPT's expected schema.
Returns a document object with id, title, text, url, and metadata fields.
"""
# Extract title from markdown content if not provided
if not title and isinstance(content, str):
lines = content.split("\n")
if lines and lines[0].startswith("# "):
title = lines[0][2:].strip()
else:
title = identifier.split("/")[-1].replace("-", " ").title()
# Ensure title is never None
if not title:
title = "Untitled Document"
# Handle error cases
if isinstance(content, str) and content.startswith("# Note Not Found"):
return {
"id": identifier,
"title": title or "Document Not Found",
"text": content,
"url": identifier,
"metadata": {"error": "Document not found"},
}
return {
"id": identifier,
"title": title or "Untitled Document",
"text": content,
"url": identifier,
"metadata": {"format": "markdown"},
}
@mcp.tool(description="Search for content across the knowledge base")
async def search(
query: str,
context: Context | None = None,
) -> List[Dict[str, Any]]:
"""ChatGPT/OpenAI MCP search adapter returning a single text content item.
Args:
query: Search query (full-text syntax supported by `search_notes`)
context: Optional FastMCP context passed through for auth/session data
Returns:
List with one dict: `{ "type": "text", "text": "{...JSON...}" }`
where the JSON body contains `results`, `total_count`, and echo of `query`.
"""
logger.info(f"ChatGPT search request: query='{query}'")
try:
# Call underlying search_notes with sensible defaults for ChatGPT
results = await search_notes.fn(
query=query,
project=None, # Let project resolution happen automatically
page=1,
page_size=10, # Reasonable default for ChatGPT consumption
search_type="text", # Default to full-text search
context=context,
)
# Handle string error responses from search_notes
if isinstance(results, str):
logger.warning(f"Search failed with error: {results[:100]}...")
search_results = {
"results": [],
"error": "Search failed",
"error_details": results[:500], # Truncate long error messages
}
else:
# Format successful results for ChatGPT
formatted_results = _format_search_results_for_chatgpt(results)
search_results = {
"results": formatted_results,
"total_count": len(results.results), # Use actual count from results
"query": query,
}
logger.info(f"Search completed: {len(formatted_results)} results returned")
# Return in MCP content array format as required by OpenAI
return [{"type": "text", "text": json.dumps(search_results, ensure_ascii=False)}]
except Exception as e:
logger.error(f"ChatGPT search failed for query '{query}': {e}")
error_results = {
"results": [],
"error": "Internal search error",
"error_message": str(e)[:200],
}
return [{"type": "text", "text": json.dumps(error_results, ensure_ascii=False)}]
@mcp.tool(description="Fetch the full contents of a search result document")
async def fetch(
id: str,
context: Context | None = None,
) -> List[Dict[str, Any]]:
"""ChatGPT/OpenAI MCP fetch adapter returning a single text content item.
Args:
id: Document identifier (permalink, title, or memory URL)
context: Optional FastMCP context passed through for auth/session data
Returns:
List with one dict: `{ "type": "text", "text": "{...JSON...}" }`
where the JSON body includes `id`, `title`, `text`, `url`, and metadata.
"""
logger.info(f"ChatGPT fetch request: id='{id}'")
try:
# Call underlying read_note function
content = await read_note.fn(
identifier=id,
project=None, # Let project resolution happen automatically
page=1,
page_size=10, # Default pagination
context=context,
)
# Format the document for ChatGPT
document = _format_document_for_chatgpt(content, id)
logger.info(f"Fetch completed: id='{id}', content_length={len(document.get('text', ''))}")
# Return in MCP content array format as required by OpenAI
return [{"type": "text", "text": json.dumps(document, ensure_ascii=False)}]
except Exception as e:
logger.error(f"ChatGPT fetch failed for id '{id}': {e}")
error_document = {
"id": id,
"title": "Fetch Error",
"text": f"Failed to fetch document: {str(e)[:200]}",
"url": id,
"metadata": {"error": "Fetch failed"},
}
return [{"type": "text", "text": json.dumps(error_document, ensure_ascii=False)}]
+67 -34
View File
@@ -2,15 +2,16 @@ from textwrap import dedent
from typing import Optional
from loguru import logger
from fastmcp import Context
from basic_memory.mcp.project_context import get_active_project
from basic_memory.mcp.tools.utils import call_delete
from basic_memory.mcp.server import mcp
from basic_memory.mcp.async_client import client
from basic_memory.mcp.project_session import get_active_project
from basic_memory.schemas import DeleteEntitiesResponse
def _format_delete_error_response(error_message: str, identifier: str) -> str:
def _format_delete_error_response(project: str, error_message: str, identifier: str) -> str:
"""Format helpful error responses for delete failures that guide users to successful deletions."""
# Note not found errors
@@ -24,7 +25,7 @@ def _format_delete_error_response(error_message: str, identifier: str) -> str:
return dedent(f"""
# Delete Failed - Note Not Found
The note '{identifier}' could not be found for deletion.
The note '{identifier}' could not be found for deletion in {project}.
## This might mean:
1. **Already deleted**: The note may have been deleted previously
@@ -32,21 +33,21 @@ def _format_delete_error_response(error_message: str, identifier: str) -> str:
3. **Different project**: The note might be in a different project
## How to verify:
1. **Search for the note**: Use `search_notes("{search_term}")` to find it
1. **Search for the note**: Use `search_notes("{project}", "{search_term}")` to find it
2. **Try different formats**:
- If you used a permalink like "folder/note-title", try just the title: "{title_format}"
- If you used a title, try the permalink format: "{permalink_format}"
3. **Check if already deleted**: Use `list_directory("/")` to see what notes exist
4. **Check current project**: Use `get_current_project()` to verify you're in the right project
4. **List notes in project**: Use `list_directory("/")` to see what notes exist in the current project
## If the note actually exists:
```
# First, find the correct identifier:
search_notes("{identifier}")
search_notes("{project}", "{identifier}")
# Then delete using the correct identifier:
delete_note("correct-identifier-from-search")
delete_note("{project}", "correct-identifier-from-search")
```
## If you want to delete multiple similar notes:
@@ -69,12 +70,12 @@ You don't have permission to delete '{identifier}': {error_message}
3. **Project access**: Ensure you're in the correct project with proper permissions
## Alternative actions:
- Check current project: `get_current_project()`
- Switch to correct project: `switch_project("project-name")`
- Verify note exists first: `read_note("{identifier}")`
- List available projects: `list_memory_projects()`
- Specify the correct project: `delete_note("{identifier}", project="project-name")`
- Verify note exists first: `read_note("{identifier}", project="project-name")`
## If you have read-only access:
Send a message to support@basicmachines.co to request deletion, or ask someone with write access to delete the note."""
Ask someone with write access to delete the note."""
# Server/filesystem errors
if (
@@ -92,8 +93,7 @@ A system error occurred while deleting '{identifier}': {error_message}
3. **Check disk space**: Ensure the system has adequate storage
## Troubleshooting:
- Verify note exists: `read_note("{identifier}")`
- Check project status: `get_current_project()`
- Verify note exists: `read_note("{project}","{identifier}")`
- Try again in a few moments
## If problem persists:
@@ -112,7 +112,7 @@ A database error occurred while deleting '{identifier}': {error_message}
## Steps to resolve:
1. **Try again**: Wait a moment and retry the deletion
2. **Check note status**: `read_note("{identifier}")` to see current state
2. **Check note status**: `read_note("{project}","{identifier}")` to see current state
3. **Manual verification**: Use `list_directory()` to see if file still exists
## If the note appears gone but database shows it exists:
@@ -124,7 +124,7 @@ Send a message to support@basicmachines.co - a manual database cleanup may be ne
Error deleting note '{identifier}': {error_message}
## General troubleshooting:
1. **Verify the note exists**: `read_note("{identifier}")` or `search_notes("{identifier}")`
1. **Verify the note exists**: `read_note("{project}", "{identifier}")` or `search_notes("{project}", "{identifier}")`
2. **Check permissions**: Ensure you can edit/delete files in this project
3. **Try again**: The error might be temporary
4. **Check project**: Make sure you're in the correct project
@@ -132,46 +132,77 @@ Error deleting note '{identifier}': {error_message}
## Step-by-step approach:
```
# 1. Confirm note exists and get correct identifier
search_notes("{identifier}")
search_notes("{project}", "{identifier}")
# 2. Read the note to verify access
read_note("correct-identifier-from-search")
read_note("{project}", "correct-identifier-from-search")
# 3. Try deletion with correct identifier
delete_note("correct-identifier-from-search")
delete_note("{project}", "correct-identifier-from-search")
```
## Alternative approaches:
- Check what notes exist: `list_directory("/")`
- Verify current project: `get_current_project()`
- Switch projects if needed: `switch_project("correct-project")`
- Check what notes exist: `list_directory("{project}", "/")`
## Need help?
If the note should be deleted but the operation keeps failing, send a message to support@basicmachines.co."""
If the note should be deleted but the operation keeps failing, send a message to support@basicmemory.com."""
@mcp.tool(description="Delete a note by title or permalink")
async def delete_note(identifier: str, project: Optional[str] = None) -> bool | str:
async def delete_note(
identifier: str, project: Optional[str] = None, context: Context | None = None
) -> bool | str:
"""Delete a note from the knowledge base.
Permanently removes a note from the specified project. The note is identified
by title or permalink. If the note doesn't exist, the operation returns False
without error. If deletion fails due to other issues, helpful error messages are provided.
Project Resolution:
Server resolves projects in this order: Single Project Mode project parameter default project.
If project unknown, use list_memory_projects() or recent_activity() first.
Args:
identifier: Note title or permalink
project: Optional project name to delete from. If not provided, uses current active project.
project: Project name to delete from. Optional - server will resolve using hierarchy.
If unknown, use list_memory_projects() to discover available projects.
identifier: Note title or permalink to delete
Can be a title like "Meeting Notes" or permalink like "notes/meeting-notes"
context: Optional FastMCP context for performance caching.
Returns:
True if note was deleted, False otherwise
True if note was successfully deleted, False if note was not found.
On errors, returns a formatted string with helpful troubleshooting guidance.
Examples:
# Delete by title
delete_note("Meeting Notes: Project Planning")
delete_note("my-project", "Meeting Notes: Project Planning")
# Delete by permalink
delete_note("notes/project-planning")
delete_note("work-docs", "notes/project-planning")
# Delete from specific project
delete_note("notes/project-planning", project="work-project")
# Delete with exact path
delete_note("research", "experiments/ml-model-results")
# Common usage pattern
if delete_note("my-project", "old-draft"):
print("Note deleted successfully")
else:
print("Note not found or already deleted")
Raises:
HTTPError: If project doesn't exist or is inaccessible
SecurityError: If identifier attempts path traversal
Warning:
This operation is permanent and cannot be undone. The note file
will be removed from the filesystem and all references will be lost.
Note:
If the note is not found, this function provides helpful error messages
with suggestions for finding the correct identifier, including search
commands and alternative formats to try.
"""
active_project = get_active_project(project)
active_project = await get_active_project(client, project, context)
project_url = active_project.project_url
try:
@@ -179,13 +210,15 @@ async def delete_note(identifier: str, project: Optional[str] = None) -> bool |
result = DeleteEntitiesResponse.model_validate(response.json())
if result.deleted:
logger.info(f"Successfully deleted note: {identifier}")
logger.info(
f"Successfully deleted note: {identifier} in project: {active_project.name}"
)
return True
else:
logger.warning(f"Delete operation completed but note was not deleted: {identifier}")
return False
except Exception as e: # pragma: no cover
logger.error(f"Delete failed for '{identifier}': {e}")
logger.error(f"Delete failed for '{identifier}': {e}, project: {active_project.name}")
# Return formatted error message for better user experience
return _format_delete_error_response(str(e), identifier)
return _format_delete_error_response(active_project.name, str(e), identifier)
+55 -39
View File
@@ -3,9 +3,10 @@
from typing import Optional
from loguru import logger
from fastmcp import Context
from basic_memory.mcp.async_client import client
from basic_memory.mcp.project_session import get_active_project
from basic_memory.mcp.project_context import get_active_project, add_project_metadata
from basic_memory.mcp.server import mcp
from basic_memory.mcp.tools.utils import call_patch
from basic_memory.schemas import EntityResponse
@@ -17,6 +18,7 @@ def _format_error_response(
identifier: str,
find_text: Optional[str] = None,
expected_replacements: int = 1,
project: Optional[str] = None,
) -> str:
"""Format helpful error responses for edit_note failures that guide the AI to retry successfully."""
@@ -27,14 +29,14 @@ def _format_error_response(
The note with identifier '{identifier}' could not be found. Edit operations require an exact match (no fuzzy matching).
## Suggestions to try:
1. **Search for the note first**: Use `search_notes("{identifier.split("/")[-1]}")` to find similar notes with exact identifiers
1. **Search for the note first**: Use `search_notes("{project or "project-name"}", "{identifier.split("/")[-1]}")` to find similar notes with exact identifiers
2. **Try different exact identifier formats**:
- If you used a permalink like "folder/note-title", try the exact title: "{identifier.split("/")[-1].replace("-", " ").title()}"
- If you used a title, try the exact permalink format: "{identifier.lower().replace(" ", "-")}"
- Use `read_note()` first to verify the note exists and get the exact identifier
- Use `read_note("{project or "project-name"}", "{identifier}")` first to verify the note exists and get the exact identifier
## Alternative approach:
Use `write_note()` to create the note first, then edit it."""
Use `write_note("{project or "project-name"}", "title", "content", "folder")` to create the note first, then edit it."""
# Find/replace specific errors
if operation == "find_replace":
@@ -44,7 +46,7 @@ Use `write_note()` to create the note first, then edit it."""
The text '{find_text}' was not found in the note '{identifier}'.
## Suggestions to try:
1. **Read the note first**: Use `read_note("{identifier}")` to see the current content
1. **Read the note first**: Use `read_note("{project or "project-name"}", "{identifier}")` to see the current content
2. **Check for exact matches**: The search is case-sensitive and must match exactly
3. **Try a broader search**: Search for just part of the text you want to replace
4. **Use expected_replacements=0**: If you want to verify the text doesn't exist
@@ -65,13 +67,13 @@ The text '{find_text}' was not found in the note '{identifier}'.
Expected {expected_replacements} occurrences of '{find_text}' but found {actual_count}.
## How to fix:
1. **Read the note first**: Use `read_note("{identifier}")` to see how many times '{find_text}' appears
1. **Read the note first**: Use `read_note("{project or "project-name"}", "{identifier}")` to see how many times '{find_text}' appears
2. **Update expected_replacements**: Set expected_replacements={actual_count} in your edit_note call
3. **Be more specific**: If you only want to replace some occurrences, make your find_text more specific
## Example:
```
edit_note("{identifier}", "find_replace", "new_text", find_text="{find_text}", expected_replacements={actual_count})
edit_note("{project or "project-name"}", "{identifier}", "find_replace", "new_text", find_text="{find_text}", expected_replacements={actual_count})
```"""
# Section replacement errors
@@ -81,7 +83,7 @@ edit_note("{identifier}", "find_replace", "new_text", find_text="{find_text}", e
Multiple sections found with the same header in note '{identifier}'.
## How to fix:
1. **Read the note first**: Use `read_note("{identifier}")` to see the document structure
1. **Read the note first**: Use `read_note("{project or "project-name"}", "{identifier}")` to see the document structure
2. **Make headers unique**: Add more specific text to distinguish sections
3. **Use append instead**: Add content at the end rather than replacing a specific section
@@ -97,14 +99,14 @@ Use `find_replace` to update specific text within the duplicate sections."""
There was a problem with the edit request to note '{identifier}': {error_message}.
## Common causes and fixes:
1. **Note doesn't exist**: Use `search_notes()` or `read_note()` to verify the note exists
1. **Note doesn't exist**: Use `search_notes("{project or "project-name"}", "query")` or `read_note("{project or "project-name"}", "{identifier}")` to verify the note exists
2. **Invalid identifier format**: Try different identifier formats (title vs permalink)
3. **Empty or invalid content**: Check that your content is properly formatted
4. **Server error**: Try the operation again, or use `read_note()` first to verify the note state
## Troubleshooting steps:
1. Verify the note exists: `read_note("{identifier}")`
2. If not found, search for it: `search_notes("{identifier.split("/")[-1]}")`
1. Verify the note exists: `read_note("{project or "project-name"}", "{identifier}")`
2. If not found, search for it: `search_notes("{project or "project-name"}", "{identifier.split("/")[-1]}")`
3. Try again with the correct identifier from the search results"""
# Fallback for other errors
@@ -113,14 +115,14 @@ There was a problem with the edit request to note '{identifier}': {error_message
Error editing note '{identifier}': {error_message}
## General troubleshooting:
1. **Verify the note exists**: Use `read_note("{identifier}")` to check
1. **Verify the note exists**: Use `read_note("{project or "project-name"}", "{identifier}")` to check
2. **Check your parameters**: Ensure all required parameters are provided correctly
3. **Read the note content first**: Use `read_note()` to understand the current structure
3. **Read the note content first**: Use `read_note("{project or "project-name"}", "{identifier}")` to understand the current structure
4. **Try a simpler operation**: Start with `append` if other operations fail
## Need help?
- Use `search_notes()` to find notes
- Use `read_note()` to examine content before editing
- Use `search_notes("{project or "project-name"}", "query")` to find notes
- Use `read_note("{project or "project-name"}", "identifier")` to examine content before editing
- Check that identifiers, section headers, and find_text match exactly"""
@@ -131,15 +133,19 @@ async def edit_note(
identifier: str,
operation: str,
content: str,
project: Optional[str] = None,
section: Optional[str] = None,
find_text: Optional[str] = None,
expected_replacements: int = 1,
project: Optional[str] = None,
context: Context | None = None,
) -> str:
"""Edit an existing markdown note in the knowledge base.
This tool allows you to make targeted changes to existing notes without rewriting the entire content.
It supports various operations for different editing scenarios.
Makes targeted changes to existing notes without rewriting the entire content.
Project Resolution:
Server resolves projects in this order: Single Project Mode project parameter default project.
If project unknown, use list_memory_projects() or recent_activity() first.
Args:
identifier: The exact title, permalink, or memory:// URL of the note to edit.
@@ -151,56 +157,64 @@ async def edit_note(
- "find_replace": Replace occurrences of find_text with content
- "replace_section": Replace content under a specific markdown header
content: The content to add or use for replacement
project: Project name to edit in. Optional - server will resolve using hierarchy.
If unknown, use list_memory_projects() to discover available projects.
section: For replace_section operation - the markdown header to replace content under (e.g., "## Notes", "### Implementation")
find_text: For find_replace operation - the text to find and replace
expected_replacements: For find_replace operation - the expected number of replacements (validation will fail if actual doesn't match)
project: Optional project name to delete from. If not provided, uses current active project.
context: Optional FastMCP context for performance caching.
Returns:
A markdown formatted summary of the edit operation and resulting semantic content
A markdown formatted summary of the edit operation and resulting semantic content,
including operation details, file path, observations, relations, and project metadata.
Examples:
# Add new content to end of note
edit_note("project-planning", "append", "\\n## New Requirements\\n- Feature X\\n- Feature Y")
edit_note("my-project", "project-planning", "append", "\\n## New Requirements\\n- Feature X\\n- Feature Y")
# Add timestamp at beginning (frontmatter-aware)
edit_note("meeting-notes", "prepend", "## 2025-05-25 Update\\n- Progress update...\\n\\n")
edit_note("work-docs", "meeting-notes", "prepend", "## 2025-05-25 Update\\n- Progress update...\\n\\n")
# Update version number (single occurrence)
edit_note("config-spec", "find_replace", "v0.13.0", find_text="v0.12.0")
edit_note("api-project", "config-spec", "find_replace", "v0.13.0", find_text="v0.12.0")
# Update version in multiple places with validation
edit_note("api-docs", "find_replace", "v2.1.0", find_text="v2.0.0", expected_replacements=3)
edit_note("docs-project", "api-docs", "find_replace", "v2.1.0", find_text="v2.0.0", expected_replacements=3)
# Replace text that appears multiple times - validate count first
edit_note("docs/guide", "find_replace", "new-api", find_text="old-api", expected_replacements=5)
edit_note("team-docs", "docs/guide", "find_replace", "new-api", find_text="old-api", expected_replacements=5)
# Replace implementation section
edit_note("api-spec", "replace_section", "New implementation approach...\\n", section="## Implementation")
edit_note("specs", "api-spec", "replace_section", "New implementation approach...\\n", section="## Implementation")
# Replace subsection with more specific header
edit_note("docs/setup", "replace_section", "Updated install steps\\n", section="### Installation")
edit_note("docs", "docs/setup", "replace_section", "Updated install steps\\n", section="### Installation")
# Using different identifier formats (must be exact matches)
edit_note("Meeting Notes", "append", "\\n- Follow up on action items") # exact title
edit_note("docs/meeting-notes", "append", "\\n- Follow up tasks") # exact permalink
edit_note("docs/Meeting Notes", "append", "\\n- Next steps") # exact folder/title
edit_note("work-project", "Meeting Notes", "append", "\\n- Follow up on action items") # exact title
edit_note("work-project", "docs/meeting-notes", "append", "\\n- Follow up tasks") # exact permalink
# If uncertain about identifier, search first:
# search_notes("meeting") # Find available notes
# edit_note("docs/meeting-notes-2025", "append", "content") # Use exact result
# search_notes("work-project", "meeting") # Find available notes
# edit_note("work-project", "docs/meeting-notes-2025", "append", "content") # Use exact result
# Add new section to document
edit_note("project-plan", "replace_section", "TBD - needs research\\n", section="## Future Work")
edit_note("planning", "project-plan", "replace_section", "TBD - needs research\\n", section="## Future Work")
# Update status across document (expecting exactly 2 occurrences)
edit_note("status-report", "find_replace", "In Progress", find_text="Not Started", expected_replacements=2)
edit_note("reports", "status-report", "find_replace", "In Progress", find_text="Not Started", expected_replacements=2)
# Replace text in a file, specifying project name
edit_note("docs/guide", "find_replace", "new-api", find_text="old-api", project="my-project"))
Raises:
HTTPError: If project doesn't exist or is inaccessible
ValueError: If operation is invalid or required parameters are missing
SecurityError: If identifier attempts path traversal
Note:
Edit operations require exact identifier matches. If unsure, use read_note() or
search_notes() first to find the correct identifier. The tool provides detailed
error messages with suggestions if operations fail.
"""
active_project = get_active_project(project)
active_project = await get_active_project(client, project, context)
project_url = active_project.project_url
logger.info("MCP tool call", tool="edit_note", identifier=identifier, operation=operation)
@@ -288,16 +302,18 @@ async def edit_note(
"MCP tool response",
tool="edit_note",
operation=operation,
project=active_project.name,
permalink=result.permalink,
observations_count=len(result.observations),
relations_count=len(result.relations),
status_code=response.status_code,
)
return "\n".join(summary)
result = "\n".join(summary)
return add_project_metadata(result, active_project.name)
except Exception as e:
logger.error(f"Error editing note: {e}")
return _format_error_response(
str(e), operation, identifier, find_text, expected_replacements
str(e), operation, identifier, find_text, expected_replacements, active_project.name
)
+44
View File
@@ -0,0 +1,44 @@
from httpx._types import (
HeaderTypes,
)
from loguru import logger
from fastmcp.server.dependencies import get_http_headers
def inject_auth_header(headers: HeaderTypes | None = None) -> HeaderTypes:
"""
Inject JWT token from FastMCP context into headers if available.
Args:
headers: Existing headers dict or None
Returns:
Headers dict with Authorization header added if JWT is available
"""
# Start with existing headers or empty dict
if headers is None:
headers = {}
elif not isinstance(headers, dict):
# Convert other header types to dict
headers = dict(headers) # type: ignore
else:
# Make a copy to avoid modifying the original
headers = headers.copy()
http_headers = get_http_headers()
# Log only non-sensitive header keys for debugging
if logger.opt(lazy=True).debug:
sensitive_headers = {"authorization", "cookie", "x-api-key", "x-auth-token", "api-key"}
safe_headers = {k for k in http_headers.keys() if k.lower() not in sensitive_headers}
logger.debug(f"HTTP headers present: {list(safe_headers)}")
authorization = http_headers.get("Authorization") or http_headers.get("authorization")
if authorization:
headers["Authorization"] = authorization # type: ignore
# Log only that auth was injected, not the token value
logger.debug("Injected authorization header into request")
else:
logger.debug("No authorization header found in request")
return headers
+18 -8
View File
@@ -3,9 +3,10 @@
from typing import Optional
from loguru import logger
from fastmcp import Context
from basic_memory.mcp.async_client import client
from basic_memory.mcp.project_session import get_active_project
from basic_memory.mcp.project_context import get_active_project
from basic_memory.mcp.server import mcp
from basic_memory.mcp.tools.utils import call_get
@@ -18,6 +19,7 @@ async def list_directory(
depth: int = 1,
file_name_glob: Optional[str] = None,
project: Optional[str] = None,
context: Context | None = None,
) -> str:
"""List directory contents from the knowledge base with optional filtering.
@@ -32,7 +34,10 @@ async def list_directory(
Higher values show subdirectory contents recursively
file_name_glob: Optional glob pattern for filtering file names
Examples: "*.md", "*meeting*", "project_*"
project: Optional project name to delete from. If not provided, uses current active project.
project: Project name to list directory from. Optional - server will resolve using hierarchy.
If unknown, use list_memory_projects() to discover available projects.
context: Optional FastMCP context for performance caching.
Returns:
Formatted listing of directory contents with file metadata
@@ -43,8 +48,8 @@ async def list_directory(
# List specific folder
list_directory(dir_name="/projects")
# Find all Python files
list_directory(file_name_glob="*.py")
# Find all markdown files
list_directory(file_name_glob="*.md")
# Deep exploration of research folder
list_directory(dir_name="/research", depth=3)
@@ -52,10 +57,13 @@ async def list_directory(
# Find meeting notes in projects folder
list_directory(dir_name="/projects", file_name_glob="*meeting*")
# Find meeting notes in a specific project
list_directory(dir_name="/projects", file_name_glob="*meeting*", project="work-project")
# Explicit project specification
list_directory(project="work-docs", dir_name="/projects")
Raises:
ToolError: If project doesn't exist or directory path is invalid
"""
active_project = get_active_project(project)
active_project = await get_active_project(client, project, context)
project_url = active_project.project_url
# Prepare query parameters
@@ -66,7 +74,9 @@ async def list_directory(
if file_name_glob:
params["file_name_glob"] = file_name_glob
logger.debug(f"Listing directory '{dir_name}' with depth={depth}, glob='{file_name_glob}'")
logger.debug(
f"Listing directory '{dir_name}' in project {project} with depth={depth}, glob='{file_name_glob}'"
)
# Call the API endpoint
response = await call_get(

Some files were not shown because too many files have changed in this diff Show More