Compare commits

...

411 Commits

Author SHA1 Message Date
phernandez 4bfec8a88e feat: Add research skill and /research command
New capability to research topics and save structured reports:

/research command:
- /research <topic> [folder]
- Investigates using web search, codebase search, and existing notes
- Produces structured report with findings and analysis
- Saves to research/ folder by default

research skill (model-invoked):
- Triggers on "research", "investigate", "look into", "explore"
- Gathers information from multiple sources
- Synthesizes findings into actionable reports
- Links to sources and related notes

Report structure:
- Summary and research question
- Key findings with evidence
- Analysis and recommendations
- Open questions and sources
- Observations and relations for knowledge graph

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: phernandez <paul@basicmachines.co>
2025-11-28 14:31:02 -06:00
phernandez 9c3d2eb335 refactor: Move plugin into claude-code-plugin subdirectory
Reorganize plugin files into a dedicated subdirectory to keep
them separate from the main Basic Memory Python package:

- Move all plugin files to claude-code-plugin/
- Add README.md with quick start guide
- Update installation paths to use subdirectory

New structure:
```
claude-code-plugin/
├── .claude-plugin/
│   ├── plugin.json
│   └── marketplace.json
├── commands/
├── skills/
├── hooks/
├── README.md
└── PLUGIN.md
```

Installation:
/plugin marketplace add basicmachines-co/basic-memory/claude-code-plugin
/plugin install basic-memory@basicmachines

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: phernandez <paul@basicmachines.co>
2025-11-28 14:08:29 -06:00
phernandez 5fcbae3fdb feat: Add /organize slash command for knowledge graph maintenance
User-invoked command to complement the knowledge-organize skill.

Actions:
- /organize health - Quick overview of KB status (default)
- /organize orphans - Find notes with no relations
- /organize duplicates - Find similar/overlapping notes
- /organize relations [note] - Suggest connections for a note
- /organize tags - Review and normalize tag consistency

Always confirms before modifying notes.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: phernandez <paul@basicmachines.co>
2025-11-28 13:58:19 -06:00
phernandez 68ee310a55 refactor: Rename knowledge-organizer to knowledge-organize
Use verb form for skill name to be consistent with action-oriented
naming (like knowledge-capture, not knowledge-capturer).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: phernandez <paul@basicmachines.co>
2025-11-28 13:56:18 -06:00
phernandez 3a7fca8a9e feat: Add knowledge-organizer skill for maintaining knowledge graph
New skill to help users organize and maintain their knowledge base:

Capabilities:
- Find orphan notes (no relations to other notes)
- Suggest relations based on content similarity
- Identify duplicate or overlapping notes
- Review and suggest folder organization
- Normalize inconsistent tags
- Create index/hub notes for topic navigation
- Enrich sparse notes with observations and structure

Includes workflows for:
- Quick health check (overview of KB status)
- Deep organization session (thorough review)
- Topic-focused organization (organize around a subject)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: phernandez <paul@basicmachines.co>
2025-11-28 13:48:43 -06:00
phernandez 4f2b1b2fd3 feat: Add edit-note skills for interactive note editing
Add two new skills for editing Basic Memory notes:

edit-note (MCP-based):
- Works with both cloud and local installations
- Conversational editing workflow via MCP tools
- Uses edit_note operations: append, prepend, find_replace, replace_section
- Shows before/after state for user verification

edit-note-local (file-based):
- For local installations with file system access
- Edits markdown files directly using Claude Code's Read/Edit/Write
- Changes sync automatically via `basic-memory sync --watch`
- Full file access including frontmatter editing

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: phernandez <paul@basicmachines.co>
2025-11-28 13:33:52 -06:00
phernandez a70b355838 feat: Complete plugin with marketplace, commands, and hooks
Add full plugin infrastructure for distribution:

Marketplace:
- Add marketplace.json for self-hosting at basicmachines-co/basic-memory
- Users can add via: /plugin marketplace add basicmachines-co/basic-memory

Slash Commands:
- /remember [title] - Capture insights to Basic Memory
- /continue [topic] - Resume previous work with context
- /context <url> - Build context from memory:// URLs
- /recent [timeframe] - Show recent activity

Hooks:
- PostToolUse: Confirm when notes are saved
- Stop: Suggest /remember for valuable conversations

Updated PLUGIN.md with comprehensive documentation.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: phernandez <paul@basicmachines.co>
2025-11-28 13:11:11 -06:00
phernandez 32acbfe982 refactor: Package skills as Claude Code plugin
Convert the Basic Memory skills into a proper Claude Code plugin format:

- Add .claude-plugin/plugin.json manifest with metadata
- Move skills from .claude/skills/ to root skills/ directory
- Add PLUGIN.md with installation and usage documentation

Plugin structure:
```
.claude-plugin/
  plugin.json        # Plugin manifest
skills/
  knowledge-capture/
  continue-conversation/
  spec-driven-development/
PLUGIN.md            # Plugin documentation
```

Users can install via: /plugin install basic-memory@basicmachines

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: phernandez <paul@basicmachines.co>
2025-11-28 13:01:22 -06:00
phernandez 1831397861 feat: Add Claude Code skills for Basic Memory MCP integration
Add three model-invoked skills that help Claude automatically use
Basic Memory's MCP tools in the right contexts:

- knowledge-capture: Capture insights, decisions, and learnings into
  structured notes with observations and relations
- continue-conversation: Resume previous work by building context from
  the knowledge graph using memory:// URLs and recent activity
- spec-driven-development: Guide implementation based on specs stored
  in Basic Memory, following the SPEC-1 process

Unlike slash commands (user-invoked), skills are automatically
discovered and applied by Claude based on conversation context.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: phernandez <paul@basicmachines.co>
2025-11-28 12:52:47 -06:00
jope-bm 28cc5225a7 feat: Implement API v2 with ID-based endpoints (Phase 1) (#441)
Signed-off-by: Joe P <joe@basicmemory.com>
Signed-off-by: phernandez <paul@basicmachines.co>
Signed-off-by: Claude <noreply@anthropic.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Paul Hernandez <60959+phernandez@users.noreply.github.com>
Co-authored-by: phernandez <paul@basicmachines.co>
2025-11-27 10:35:55 -06:00
phernandez 9b7bbc7116 formatting and logic change to resolve_relations, remove fuzzy search 2025-11-25 22:54:37 -06:00
phernandez 138c283d6c add postgres db type 2025-11-25 20:25:56 -06:00
phernandez 7a8954c37e add extra logic for cloud-indexing improvements 2025-11-25 13:52:58 -06:00
phernandez 10c7c19c03 fix db url for sqlite migrations
Signed-off-by: phernandez <paul@basicmachines.co>
2025-11-21 13:21:20 -06:00
Paul Hernandez fb5e9e1d77 feat: Add PostgreSQL database backend support (#439)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude <noreply@anthropic.com>
2025-11-20 11:20:29 -06:00
phernandez 66b91b2847 ci: Add PostgreSQL testing to GitHub Actions workflow
Add Postgres service container and separate test step for PostgreSQL backend testing.
The Postgres tests only run on Linux runners since GitHub Actions service containers
are only available on Linux.

- Add postgres:17 service container with health checks
- Add 'Run tests (Postgres)' step with Linux-only condition
- Rename existing test step to 'Run tests (SQLite)' for clarity

This enables CI testing of dual database backend support introduced in the
postgres-support feature branch.

Signed-off-by: phernandez <paul@basicmachines.co>
2025-11-18 12:25:18 -06:00
Cedric Hurst b004565df9 fix: handle periods in kebab_filenames mode (#424) 2025-11-18 06:49:55 -05:00
Drew Cain a258b73e1d chore: update version to 0.16.2 for v0.16.2 release 2025-11-16 21:30:59 -06:00
Drew Cain 9a845f2906 docs: prepare for v0.16.2 release 2025-11-16 21:28:13 -06:00
Drew Cain 6517e9845f fix: Use platform-native path separators in config.json (#429)
Signed-off-by: Drew Cain <groksrc@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
2025-11-13 09:12:26 -06:00
Drew Cain 1af05392ee fix: Add rclone installation checks for Windows bisync commands (#427) 2025-11-12 14:22:14 -06:00
Brandon Mayes cad7019c89 fix: main project always recreated on project list command (#421) 2025-11-12 09:57:08 -05:00
phernandez 099c334e3d chore: update version to 0.16.1 for v0.16.1 release 2025-11-11 09:21:47 -06:00
phernandez 7685586178 docs: Add v0.16.1 CHANGELOG entry for Windows line ending fix
Signed-off-by: phernandez <paul@basicmachines.co>
2025-11-11 09:10:26 -06:00
Paul Hernandez e9d0a944a9 fix: Handle Windows line endings in rclone bisync (#422)
Signed-off-by: phernandez <paul@basicmachines.co>
2025-11-11 09:08:20 -06:00
phernandez caf3c14bb1 chore: update version to 0.16.0 for v0.16.0 release 2025-11-10 19:19:48 -06:00
phernandez c5d9067754 docs: Add v0.16.0 CHANGELOG entry with comprehensive release notes
Signed-off-by: phernandez <paul@basicmachines.co>
2025-11-10 19:17:29 -06:00
phernandez e0fc59ea97 style: Format upload.py for better readability
Signed-off-by: phernandez <paul@basicmachines.co>
2025-11-10 19:15:03 -06:00
Paul Hernandez 49b2adc35c fix: skip archive files during cloud upload (#420)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Paul Hernandez <phernandez@users.noreply.github.com>
2025-11-10 19:04:49 -06:00
Paul Hernandez 1646572f69 fix: Rename write_note entity_type to note_type for clarity (#419)
Signed-off-by: phernandez <paul@basicmachines.co>
Signed-off-by: Paul Hernandez <60959+phernandez@users.noreply.github.com>
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Paul Hernandez <phernandez@users.noreply.github.com>
2025-11-10 19:02:50 -06:00
Paul Hernandez f0d7398815 fix: Quote string values in YAML frontmatter to handle special characters (#418)
Signed-off-by: phernandez <paul@basicmachines.co>
Signed-off-by: Paul Hernandez <60959+phernandez@users.noreply.github.com>
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Paul Hernandez <phernandez@users.noreply.github.com>
2025-11-10 18:13:48 -06:00
Paul Hernandez 581b7b17c6 fix: Add explicit type annotations to MCP tool parameters (#394)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Paul Hernandez <phernandez@users.noreply.github.com>
2025-11-10 17:02:17 -06:00
Paul Hernandez d775f7bab9 fix: Simplify search_notes schema by removing Optional wrappers (#395)
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Paul Hernandez <phernandez@users.noreply.github.com>
2025-11-10 16:30:59 -06:00
Drew Cain fc01f6abaf fix: Replace Unicode arrows with ASCII for Windows compatibility (#414) 2025-11-10 16:30:41 -06:00
Paul Hernandez 4614fd09d5 fix: Handle dict objects in write_resource endpoint (#415)
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Paul Hernandez <phernandez@users.noreply.github.com>
2025-11-10 16:30:27 -06:00
phernandez 0d4ad7bbf0 remove v0.15.0 info from assistant-guide
Signed-off-by: phernandez <paul@basicmachines.co>
2025-11-10 16:25:49 -06:00
jope-bm 7ccec7eba2 feat: Add run_in_background parameter to sync endpoint with tests (#417)
Co-authored-by: Claude <noreply@anthropic.com>
2025-11-07 09:04:43 -07:00
Paul Hernandez 021af74545 fix: Strip duplicate headers in edit_note replace_section (#396)
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Drew Cain <groksrc@users.noreply.github.com>
2025-11-02 14:20:11 -06:00
phernandez 2ad0ee9d5d fix: Use force_full=true for database sync after project sync/bisync
After rclone synchronizes files between local and cloud storage, the
database needs to perform a full scan to ensure it captures all changes.
Previously, incremental sync (watermark optimization) could miss files
that were changed remotely.

Changes:
- project sync command now calls /project/sync?force_full=true
- project bisync command now calls /project/sync?force_full=true
- Ensures complete database refresh after file synchronization

This guarantees the database is fully in sync with the filesystem
after any rclone sync or bisync operation.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: phernandez <paul@basicmachines.co>
2025-11-02 13:26:08 -06:00
Brandon Mayes c9946ecf1e fix: Various rclone fixes for cloud sync on Windows (#410)
Signed-off-by: Brandon Mayes <5610870+bdmayes@users.noreply.github.com>
Signed-off-by: Paul Hernandez <60959+phernandez@users.noreply.github.com>
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Paul Hernandez <60959+phernandez@users.noreply.github.com>
Co-authored-by: phernandez <paul@basicmachines.co>
2025-11-02 11:26:57 -06:00
Drew Cain 0ba6f219f1 fix: Windows CLI Unicode encoding errors (#411)
Signed-off-by: phernandez <paul@basicmachines.co>
Signed-off-by: Claude <noreply@anthropic.com>
Signed-off-by: Paul Hernandez <60959+phernandez@users.noreply.github.com>
Co-authored-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Paul Hernandez <60959+phernandez@users.noreply.github.com>
2025-11-02 10:20:48 -06:00
Paul Hernandez 0b3272ae6e feat: SPEC-20 Simplified Project-Scoped Rclone Sync (#405)
Signed-off-by: phernandez <paul@basicmachines.co>
Signed-off-by: Claude <noreply@anthropic.com>
Co-authored-by: Claude <noreply@anthropic.com>
2025-11-02 09:35:26 -06:00
Paul Hernandez a7d7cc5ee6 fix: Normalize YAML frontmatter types to prevent AttributeError (#236) (#402)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude <noreply@anthropic.com>
2025-10-27 09:19:50 -05:00
Paul Hernandez a7e696b039 Add free trial information to README
Added information about a 7-day free trial.

Signed-off-by: Paul Hernandez <60959+phernandez@users.noreply.github.com>
2025-10-24 10:05:11 -05:00
Paul Hernandez 8aaddb6d45 Add free trial information to README
Added information about a 7-day free trial to the README.

Signed-off-by: Paul Hernandez <60959+phernandez@users.noreply.github.com>
2025-10-24 10:04:41 -05:00
Paul Hernandez d7565312fc Announce Basic Memory Cloud launch in README
Added a section announcing the launch of Basic Memory Cloud with details on cross-device support and early supporter pricing.

Signed-off-by: Paul Hernandez <60959+phernandez@users.noreply.github.com>
2025-10-24 09:53:14 -05:00
Paul Hernandez c7e6eab02f feat: Add delete_notes parameter to remove project endpoint (#391)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude <noreply@anthropic.com>
2025-10-21 14:09:20 -05:00
Paul Hernandez bb8da31472 fix: Handle null, empty, and string 'None' title in markdown frontmatter (#387) (#389)
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Paul Hernandez <phernandez@users.noreply.github.com>
2025-10-21 09:29:19 -05:00
Paul Hernandez e78345ff25 feat: Streaming Foundation & Async I/O Consolidation (SPEC-19) (#384)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude <noreply@anthropic.com>
2025-10-21 09:03:59 -05:00
Paul Hernandez 32236cd247 fix: Handle YAML parsing errors gracefully in update_frontmatter (#378) (#379)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude <noreply@anthropic.com>
2025-10-16 20:17:58 -05:00
Paul Hernandez 4fd6d0c648 fix: Optimize sync memory usage to prevent OOM on large projects (#380)
Signed-off-by: Claude <noreply@anthropic.com>
Signed-off-by: phernandez <paul@basicmachines.co>
2025-10-16 20:17:34 -05:00
Paul Hernandez e6c8e3662c fix: preserve mtime webdav upload 376 (#377)
Signed-off-by: Claude <noreply@anthropic.com>
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude <noreply@anthropic.com>
2025-10-16 17:18:10 -05:00
Paul Hernandez 449b62d947 fix: Prevent deleted projects from being recreated by background sync (#193) (#370)
Signed-off-by: Claude <noreply@anthropic.com>
Signed-off-by: phernandez <paul@basicmachines.co>
2025-10-16 15:16:24 -05:00
Paul Hernandez b7497d7484 fix: Use filesystem timestamps for entity sync instead of database operation time (#138) (#369)
Signed-off-by: Claude <noreply@anthropic.com>
Signed-off-by: phernandez <paul@basicmachines.co>
2025-10-16 14:21:27 -05:00
Paul Hernandez d1431bdb1b fix: Handle YAML parsing errors and missing entity_type in markdown files (#368)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude <noreply@anthropic.com>
2025-10-16 13:18:58 -05:00
Paul Hernandez 171bef717f fix: Resolve UNIQUE constraint violation in entity upsert with observations (#187) (#367)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude <noreply@anthropic.com>
2025-10-16 12:30:56 -05:00
Paul Hernandez 729a5a3b8d fix: Terminate sync immediately when project is deleted (#366)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude <noreply@anthropic.com>
2025-10-16 11:07:12 -05:00
Paul Hernandez 434cdf24dd feat: Add circuit breaker for file sync failures (#364)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude <noreply@anthropic.com>
2025-10-16 09:47:48 -05:00
Paul Hernandez 7f9c1a97a4 feat: Add --verbose and --no-gitignore options to cloud upload (#362)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude <noreply@anthropic.com>
2025-10-15 20:03:36 -05:00
Paul Hernandez 53fb13b054 fix: Make project creation endpoint idempotent (#357)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude <noreply@anthropic.com>
2025-10-15 19:27:41 -05:00
Paul Hernandez bd6c8348b8 fix: Handle None text values in Claude conversations importer (#353)
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Paul Hernandez <phernandez@users.noreply.github.com>
2025-10-15 16:08:57 -05:00
phernandez 994c8b8e7e chore: update version to 0.15.2 for v0.15.2 release 2025-10-14 09:36:47 -05:00
phernandez a78e8c3ac5 style: Apply linter formatting changes 2025-10-14 09:34:10 -05:00
phernandez 53900c5baa fix: Project commands now respect cloud_mode at runtime
- Moved config evaluation from module load time to runtime
- Unified add_project command to handle both cloud and local modes
- Commands (default, sync-config, move) now check cloud_mode at runtime
- Fixes test failures where monkeypatch wasn't applied before command registration
2025-10-14 09:24:13 -05:00
phernandez 02c6de3387 docs: Add v0.15.2 changelog entry 2025-10-14 00:43:14 -05:00
phernandez 9ccf4b6b56 remove extra conole out from sync message after upload
Signed-off-by: phernandez <paul@basicmachines.co>
2025-10-14 00:38:02 -05:00
phernandez ba74ca7e18 fix: Update CloudProjectCreateResponse schema to match API response
The /proxy/projects/projects POST endpoint returns a ProjectStatusResponse
with fields: message, status, default, old_project, new_project.

Updated CloudProjectCreateResponse schema to match this format instead of
expecting name, path, message fields.

Also updated all related tests to use the correct response format:
- tests/cli/test_cloud_utils.py (3 tests)
- tests/cli/test_bisync_commands.py (1 test)

Fixes the validation error when creating cloud projects via upload command:
"bm cloud upload --project test --create-project specs"

Signed-off-by: Pablo Hernandez <pablo@basicmachines.co>
Signed-off-by: phernandez <paul@basicmachines.co>
2025-10-13 23:56:59 -05:00
Paul Hernandez 5258f45730 feat: Add WebDAV upload command for cloud projects (#356)
Signed-off-by: phernandez <paul@basicmachines.co>
Signed-off-by: Pablo Hernandez <pablo@basicmachines.co>
Co-authored-by: Claude <noreply@anthropic.com>
2025-10-13 23:20:54 -05:00
phernandez e773c002ce chore: update version to 0.15.1 for v0.15.1 release 2025-10-13 11:04:44 -05:00
phernandez e70ba944e7 docs: Add v0.15.1 changelog entry
Add comprehensive changelog for v0.15.1 release including:
- Performance improvements (43% faster sync, 10-100x faster directory ops)
- Bug fixes for cloud mode, project paths, and Claude Desktop compatibility
- New features: async client context manager, BASIC_MEMORY_PROJECT_ROOT
- Documentation updates and SPEC-15/16 additions

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-13 10:58:21 -05:00
phernandez e41579f971 Merge branch 'main' of github.com:basicmachines-co/basic-memory 2025-10-13 10:44:54 -05:00
Paul Hernandez 2b7008d997 fix: Update view_note and ChatGPT tools for Claude Desktop compatibility (#355)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude <noreply@anthropic.com>
2025-10-13 10:44:35 -05:00
phernandez 56e5cc072b Merge branch 'main' of github.com:basicmachines-co/basic-memory 2025-10-13 07:30:08 -05:00
Paul Hernandez c0538ad2dd perf: Optimize sync/indexing for 43% faster performance (#352)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude <noreply@anthropic.com>
2025-10-12 14:41:44 -05:00
phernandez 962d88ea43 add specs-17/18
Signed-off-by: phernandez <paul@basicmachines.co>
2025-10-12 10:59:45 -05:00
phernandez cd5efd4a44 perf: exclude null fields from directory endpoint responses
Reduces JSON payload size by 50-70% for directory-heavy responses by omitting
null fields from serialization.

Changes:
- Added response_model_exclude_none=True to all directory endpoints:
  - GET /directory/tree
  - GET /directory/structure
  - GET /directory/list

Impact:
- Directory nodes no longer serialize 7 null fields (title, permalink,
  entity_id, entity_type, content_type, updated_at, file_path)
- For 50+ directories: eliminates 350+ null fields from response
- Payload reduction: ~2.3kb → ~1kb for typical directory trees
- File nodes still include all metadata when present

Example directory node output:
{
  "name": "Tools",
  "directory_path": "/Tools",
  "type": "directory",
  "children": []
}

Testing:
- All 29 directory tests passing
- Type checking passing (0 errors)
- Backward compatible (clients just see missing keys vs null)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: phernandez <paul@basicmachines.co>
2025-10-11 09:56:26 -05:00
Paul Hernandez 00b73b0d08 feat: Optimize directory operations for 10-100x performance improvement (#350)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude <noreply@anthropic.com>
2025-10-11 09:11:46 -05:00
jope-bm a09066e0f0 fix: Add permalink normalization to project lookups in deps.py (#348)
Signed-off-by: Joe P <joe@basicmemory.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: phernandez <paul@basicmachines.co>
2025-10-10 21:21:35 -05:00
Drew Cain be352ab474 fix: Project deletion failing with permalink normalization (#345) 2025-10-10 12:18:03 -05:00
Paul Hernandez 8d2e70cfc8 refactor: async client context manager pattern for cloud consolidation (#344)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude <noreply@anthropic.com>
2025-10-09 19:09:47 -05:00
Paul Hernandez 53438d1eab feat: Add SPEC-15 for configuration persistence via Tigris (#343)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude <noreply@anthropic.com>
2025-10-08 18:07:00 -05:00
phernandez 032de7e3f2 fix: formatting in test file
Signed-off-by: phernandez <paul@basicmachines.co>
2025-10-08 09:13:05 -05:00
phernandez fd2b188645 Revert "feat: add optional logfire instrumentation for cloud mode distributed tracing"
This reverts commit 1fa93ecbd2.
2025-10-08 09:08:15 -05:00
phernandez 453cba94e4 Revert "fix: instrument httpx client at module level for MCP context"
This reverts commit 48cb4be4cd.
2025-10-08 09:07:26 -05:00
phernandez 48cb4be4cd fix: instrument httpx client at module level for MCP context
The lifespan-based instrumentation only runs when FastAPI app starts.
In MCP context, the app never starts but the httpx client is still used.

Solution: Instrument the client immediately after creation at module level.
This works in both contexts:
- MCP: client is instrumented when module is imported
- API: client is instrumented before lifespan runs (lifespan still safe)

This enables distributed tracing from MCP -> Cloud -> API.

Signed-off-by: phernandez <paul@basicmachines.co>
2025-10-08 08:10:54 -05:00
Paul Hernandez 3e876a7549 fix: correct ProjectItem.home property to return path instead of name (#341)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Paul Hernandez <phernandez@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
2025-10-08 01:03:12 -05:00
phernandez 1fa93ecbd2 feat: add optional logfire instrumentation for cloud mode distributed tracing 2025-10-08 00:23:00 -05:00
Paul Hernandez 73202d1aab fix: add tool use doc to write note for using empty string for root folder (#339)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude <noreply@anthropic.com>
2025-10-07 23:48:02 -05:00
Paul Hernandez 795e339333 fix: prevent nested project paths to avoid data conflicts (#338)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude <noreply@anthropic.com>
2025-10-07 09:44:39 -05:00
Paul Hernandez 07e304ce8e fix: normalize paths to lowercase in cloud mode to prevent case collisions (#336)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude <noreply@anthropic.com>
2025-10-05 17:56:58 -05:00
phernandez 2a1c06d9ad fix link in ai_assistant_guide resource
Signed-off-by: phernandez <paul@basicmachines.co>
2025-10-05 17:26:14 -05:00
Paul Hernandez c6f93a0294 chore: v0.15.0 assistant guide (#335)
Signed-off-by: phernandez <paul@basicmachines.co>
2025-10-05 17:20:54 -05:00
Paul Hernandez ccc4386627 feat: introduce BASIC_MEMORY_PROJECT_ROOT for path constraints (#334)
Signed-off-by: phernandez <paul@basicmachines.co>
2025-10-05 10:42:06 -05:00
Paul Hernandez 7616b2bb08 fix: cloud mode path validation and sanitization (bmc-issue-103) (#332)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude <noreply@anthropic.com>
2025-10-04 22:10:21 -05:00
phernandez 14c1fe4e89 chore: update version to 0.15.0 for v0.15.0 release 2025-10-04 15:02:28 -05:00
phernandez 367dc6962a style: apply ruff formatting to test files
Auto-format test files for permalink collision tests.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: phernandez <paul@basicmachines.co>
2025-10-04 15:00:19 -05:00
phernandez ee18eb2fea docs: add v0.15.0 changelog entry
Comprehensive changelog for v0.15.0 release covering:
- Critical permalink collision data loss fix
- 10+ bug fixes including #330, #329, #328, #312
- 9 new features including cloud sync and subscription validation
- Platform improvements (Python 3.13, Windows, Docker)
- Enhanced testing and documentation

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: phernandez <paul@basicmachines.co>
2025-10-04 14:53:06 -05:00
phernandez 2a050edee4 fix: prevent permalink collision via strict link resolution
Fixes critical data loss bug where creating similar entity names
(e.g., "Node C") would overwrite existing entities (e.g., "Node A.md")
due to fuzzy search incorrectly matching similar file paths.

Changes:
- Add strict=True to resolve_link() calls in entity_service.py
- Disables fuzzy search fallback during entity creation/update
- Prevents false positive matches on similar paths like
  "edge-cases/Node A.md" and "edge-cases/Node C.md"

Testing:
- Added comprehensive integration test reproducing the bug scenario
- Added MCP-level permalink collision tests
- All 55 entity service tests pass
- Manual testing confirms fix prevents file overwrite

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: phernandez <paul@basicmachines.co>
2025-10-04 14:45:37 -05:00
Paul Hernandez f3b1945e4c fix: remove .env file loading from BasicMemoryConfig (#330)
Signed-off-by: phernandez <paul@basicmachines.co>
2025-10-04 01:13:06 -05:00
Paul Hernandez 16d7eddbf7 ci: Add Python 3.13 to test matrix (#331)
Signed-off-by: phernandez <paul@basicmachines.co>
2025-10-04 01:12:44 -05:00
Paul Hernandez f5a11f3911 fix: normalize underscores in memory:// URLs for build_context (#329)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Paul Hernandez <phernandez@users.noreply.github.com>
2025-10-04 00:16:06 -05:00
Paul Hernandez ee83b0e5a8 fix: simplify entity upsert to use database-level conflict resolution (#328)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude <noreply@anthropic.com>
2025-10-03 23:25:34 -05:00
Paul Hernandez a7bf42ef49 fix: Add proper datetime JSON schema format annotations for MCP validation (#312)
Signed-off-by: Claude Code <noreply@anthropic.com>
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Paul Hernandez <phernandez@users.noreply.github.com>
2025-10-03 22:47:02 -05:00
Paul Hernandez 903591384d feat: Add disable_permalinks config flag (#313)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Paul Hernandez <phernandez@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
2025-10-03 22:11:47 -05:00
Paul Hernandez 33ee1e0831 feat: integrate ignore_utils to skip .gitignored files in sync process (#314)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Paul Hernandez <phernandez@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
2025-10-03 21:39:31 -05:00
Paul Hernandez c83d567917 fix: enable WAL mode and add Windows-specific SQLite optimizations (#316)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Paul Hernandez <phernandez@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
2025-10-03 21:10:09 -05:00
Paul Hernandez ace6a0f50d feat: CLI Subscription Validation (SPEC-13 Phase 2) (#327)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude <noreply@anthropic.com>
2025-10-03 17:59:32 -05:00
Jonathan Nguyen fc38877008 Fix: Corrected dead links in README (#321)
Signed-off-by: Jonathan Nguyen <74562467+jonathan-d-nguyen@users.noreply.github.com>
2025-10-03 10:23:16 -05:00
Paul Hernandez 99a35a7fb4 feat: Cloud CLI cloud sync via rclone bisync (#322)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude <noreply@anthropic.com>
2025-10-03 10:18:43 -05:00
Paul Hernandez ea2e93d926 fix: rework lifecycle management to optimize cloud deployment (#320)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude <noreply@anthropic.com>
2025-09-28 15:11:38 -05:00
Paul Hernandez 324844a670 fix: resolve entity relations in background to prevent cold start blocking (#319)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude <noreply@anthropic.com>
2025-09-28 09:24:29 -05:00
Paul Hernandez f818702ab7 fix: enforce minimum 1-day timeframe for recent_activity to handle timezone issues (#318)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude <noreply@anthropic.com>
2025-09-27 23:37:00 -05:00
Paul Hernandez 2efd8f44e2 fix: critical cloud deployment fixes for MCP stability (#317)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude <noreply@anthropic.com>
2025-09-27 21:57:39 -05:00
Paul Hernandez 5da97e4820 feat: implement SPEC-11 API performance optimizations (#315)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude <noreply@anthropic.com>
2025-09-26 14:34:46 -05:00
Paul Hernandez 17a6733c9d fix: remove obsolete update_current_project function and --project flag reference (#310)
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Paul Hernandez <phernandez@users.noreply.github.com>
2025-09-26 11:46:33 -05:00
Drew Cain 3e168b98f3 fix: move_note without file extension (#281)
Signed-off-by: Drew Cain <groksrc@gmail.com>
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Paul Hernandez <phernandez@users.noreply.github.com>
Co-authored-by: phernandez <paul@basicmachines.co>
2025-09-26 11:41:45 -05:00
Paul Hernandez 1091e11322 fix: Make sync operations truly non-blocking with thread pool (#309)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude <noreply@anthropic.com>
2025-09-26 10:10:58 -05:00
Paul Hernandez f40ab31685 feat: chatgpt tools for search and fetch (#305)
Signed-off-by: Paul Hernandez <60959+phernandez@users.noreply.github.com>
Signed-off-by: Drew Cain <groksrc@gmail.com>
Co-authored-by: Drew Cain <groksrc@gmail.com>
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Paul Hernandez <phernandez@users.noreply.github.com>
Co-authored-by: Drew Cain <groksrc@users.noreply.github.com>
2025-09-25 11:55:56 -05:00
phernandez bcf7f40979 fix: Correct GitHub workflow conditions for org member @claude mentions
Fixed the conditional logic in claude.yml to properly handle different event types:
- Use github.event.comment.author_association for issue_comment events
- Use github.event.sender.author_association for other events
- Maintain support for all basicmachines-co org members (OWNER/MEMBER/COLLABORATOR)

This ensures @claude mentions in PR comments trigger the workflow correctly.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: phernandez <paul@basicmachines.co>
2025-09-23 10:00:51 -05:00
Paul Hernandez 8c7e29e325 chore: Update Claude Code GitHub Workflow (#308)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude <noreply@anthropic.com>
2025-09-23 09:50:38 -05:00
phernandez 84c0b36dee feat: Add comprehensive cloud mount CLI commands and documentation
This commit implements SPEC-7 Phase 4 by adding local file access capabilities
to the Basic Memory Cloud CLI, enabling users to mount their cloud files locally
for real-time editing.

New features:
- Cloud mount setup with automatic rclone installation
- Mount/unmount/status commands with three performance profiles
- Cross-platform rclone installer with package manager fallbacks
- Mount configuration management with tenant-specific credentials
- Comprehensive documentation with examples and troubleshooting

Mount profiles:
- fast: 5s sync for active development
- balanced: 10-15s sync (recommended)
- safe: 15s+ sync with conflict detection

Technical implementation:
- Uses rclone NFS mount (no FUSE dependencies)
- Tigris object storage with scoped credentials
- Bidirectional sync with configurable cache settings
- Process management and cleanup

Fixes Python module conflict by moving cloud.py commands to cloud/core_commands.py
to resolve typer CLI loading issues with cloud.py file vs cloud/ directory.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: phernandez <paul@basicmachines.co>
2025-09-22 17:47:06 -05:00
Paul Hernandez 2c5c606a39 feat: Implement cloud mount CLI commands for local file access (#306)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude <noreply@anthropic.com>
2025-09-22 14:19:25 -05:00
Paul Hernandez a1d7792bdb feat: Implement SPEC-6 Stateless Architecture for MCP Tools (#298)
Signed-off-by: phernandez <paul@basicmachines.co>
Signed-off-by: Paul Hernandez <60959+phernandez@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Drew Cain <groksrc@users.noreply.github.com>
2025-09-21 20:39:19 -05:00
phernandez 7979b4192e remove no content-encoding: none header
Signed-off-by: phernandez <paul@basicmachines.co>
2025-09-16 16:35:32 -05:00
Drew Cain 52d9b3c752 setting content-encoding to none for mcp
Signed-off-by: Drew Cain <groksrc@gmail.com>
2025-09-16 16:09:33 -05:00
Paul Hernandez e0d8aeb149 feat: Basic memory cloud upload (#296)
Signed-off-by: phernandez <paul@basicmachines.co>
Signed-off-by: Drew Cain <groksrc@gmail.com>
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Paul Hernandez <phernandez@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Drew Cain <groksrc@gmail.com>
2025-09-16 15:07:14 -05:00
Brandon Mayes 17b929446a fix: Sanitize folder names and properly join paths (#292) 2025-09-15 23:26:56 -04:00
Paul Hernandez b00e4ff5a1 fix: replace deprecated json_encoders with Pydantic V2 field serializers (#295)
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Paul Hernandez <phernandez@users.noreply.github.com>
2025-09-14 22:19:16 -05:00
Drew Cain 0499319ded fix: rename MCP prompt names to avoid slash command parsing issues (#289)
Signed-off-by: Drew Cain <groksrc@gmail.com>
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
2025-09-09 21:42:35 -05:00
jope-bm 3a6baf80fc feat: Merge Cloud auth (#291)
Signed-off-by: phernandez <paul@basicmachines.co>
Signed-off-by: Joe P <joe@basicmemory.com>
Co-authored-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude <noreply@anthropic.com>
2025-09-09 14:48:17 -06:00
jope-bm ec2fa07350 chore: apply lint and formatting fixes for 0.14.4 release (#290)
Signed-off-by: Joe P <joe@basicmemory.com>
Co-authored-by: Claude <noreply@anthropic.com>
2025-09-05 10:00:52 -06:00
Joe P 73cade27ab chore: update version to 0.14.4 for v0.14.4 release 2025-09-04 14:03:09 -06:00
Joe P 7e024a8674 fix: resolve linting errors for release preparation
- Replace bare except clauses with Exception in legal_file_inventory.py
- Remove unused variables in test files
- Prepare codebase for v0.14.4 release
2025-09-04 14:01:06 -06:00
Drew Cain 22f7bfa398 fix: Update YAML frontmatter tag formatting for Obsidian compatibility (#280)
Signed-off-by: Drew Cain <groksrc@users.noreply.github.com>
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Paul Hernandez <phernandez@users.noreply.github.com>
Co-authored-by: Paul Hernandez <60959+phernandez@users.noreply.github.com>
2025-09-04 09:57:45 -05:00
jope-bm cd7cee650f fix: complete project management special character support (#272) (#279)
Signed-off-by: Joe P <joe@basicmemory.com>
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: jope-bm <jope-bm@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
2025-08-29 10:57:13 -06:00
Paul Hernandez 105bcaa025 feat: implement non-root Docker container to fix file ownership issues (#277)
Signed-off-by: Drew Cain <groksrc@gmail.com>
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Paul Hernandez <phernandez@users.noreply.github.com>
Co-authored-by: Drew Cain <groksrc@gmail.com>
2025-08-28 22:15:14 -05:00
Brandon Mayes 74e12eb782 fix: Sanitize filenames and allow optional kebab case (#260)
Signed-off-by: Brandon Mayes <5610870+bdmayes@users.noreply.github.com>
2025-08-27 19:29:01 -04:00
Drew Cain 7a8b08d11e fix: Windows test failures and add Windows CI support (#273)
Signed-off-by: Drew Cain <groksrc@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
2025-08-25 08:58:24 -05:00
manuelbliemel 9aa40246a8 Addressed issues when running basic-memory on the Windows platform (#252)
Signed-off-by: Manuel Bliemel <manuel.bliemel@gmail.com>
2025-08-24 19:12:40 -07:00
jope-bm 7aff836c57 fix: Add ISO datetime serialization to MCP schema models (#270)
Signed-off-by: Joe P <joe@basicmemory.com>
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: jope-bm <jope-bm@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
2025-08-23 07:23:07 -06:00
jope-bm 285e96baea fix: Fix observation parsing to exclude markdown and wiki links (#269)
Signed-off-by: Joe P <joe@basicmemory.com>
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: jope-bm <jope-bm@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
2025-08-22 20:16:05 -06:00
jope-bm 2cd2a62f30 fix: Ensure all datetime operations return timezone-aware objects (#268)
Signed-off-by: Joe P <joe@basicmemory.com>
Co-authored-by: Claude <noreply@anthropic.com>
2025-08-22 13:43:55 -06:00
jope-bm f3d8d8d617 fix: Use discriminated unions for MCP schema validation in build_context (#266)
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Paul Hernandez <phernandez@users.noreply.github.com>
Co-authored-by: jope-bm <jope-bm@users.noreply.github.com>
2025-08-22 09:36:32 -06:00
jope-bm 9743fcd13e fix: Respect BASIC_MEMORY_LOG_LEVEL and BASIC_MEMORY_CONSOLE_LOGGING environment variables (#264)
Signed-off-by: Joe P <joe@basicmemory.com>
Co-authored-by: Claude <noreply@anthropic.com>
2025-08-22 07:58:19 -06:00
phernandez 65d1984a53 Update CLA.md to include copyright and license info
Signed-off-by: phernandez <paul@basicmachines.co>
2025-08-21 18:21:24 -05:00
jope-bm b814d40ab1 fix: Add project isolation to ContextService.find_related() method (#261) (#262)
Signed-off-by: Joe P <joe@basicmemory.com>
Co-authored-by: Claude <noreply@anthropic.com>
2025-08-20 20:07:04 -06:00
Paul Hernandez 2438094914 fix: handle vim atomic write DELETE events without ADD (#249)
Signed-off-by: Joe P <joe@basicmemory.com>
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Paul Hernandez <phernandez@users.noreply.github.com>
Co-authored-by: Joe P <joe@basicmemory.com>
Co-authored-by: Claude <noreply@anthropic.com>
2025-08-20 14:36:43 -05:00
jope-bm 5d74d7407c fix: Enable string-to-integer conversion for build_context depth parameter (#259)
Signed-off-by: Joe P <joe@basicmemory.com>
Co-authored-by: Claude <noreply@anthropic.com>
2025-08-20 11:22:14 -06:00
jope-bm b6aeb3217c fix: Add missing foreign key constraints for project removal (#254) (#258)
Signed-off-by: Joe P <joe@basicmemory.com>
Signed-off-by: joe@basicmemory.com
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: jope-bm <jope-bm@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
2025-08-20 08:49:07 -06:00
jope-bm 08ee7e1201 fix: Critical search index bug - prevent note disappearing on edit (#257)
Signed-off-by: Joe P <joe@basicmemory.com>
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: jope-bm <jope-bm@users.noreply.github.com>
2025-08-19 15:41:27 -06:00
phernandez 63ae9ee0e4 docs: Re-implement external documentation improvements
- Fix typo: 'enviroment' -> 'environment' in CLAUDE.md
- Update HTTP links to HTTPS in README.md
- Add comprehensive VS Code integration instructions
- Maintain correct internal link references

All improvements re-implemented by Basic Machines team for clean IP ownership.
2025-08-08 15:31:04 -05:00
phernandez 0e78751d34 revert: Remove external documentation changes for clean IP
Reverting changes by:
- Ikko Eltociear Ashimine (typo fix)
- Jason Noble (HTTPS links)
- Matias Forbord (link fix)
- Marc Baiza (VS Code instructions)

Will be re-implemented by Basic Machines team for clean IP ownership.
2025-08-08 15:26:29 -05:00
phernandez 59eae34dee fix: Update function name in error messages to use correct search_notes
Corrects error message templates to reference the actual search_notes function name for consistency.
2025-08-08 15:25:33 -05:00
phernandez b1e55e169e revert: Remove external function name fix for clean IP
Original contribution by Amadeusz Wieczorek will be re-implemented by Basic Machines team.
2025-08-08 15:24:58 -05:00
phernandez 173bff35c1 feat: Add Chinese character support to permalink generation
Preserves non-ASCII characters like Chinese in permalinks while maintaining
backward compatibility with ASCII-only processing. This re-implements
functionality that was contributed externally, now with Basic Machines authorship.
2025-08-08 15:24:23 -05:00
phernandez 629c8e47c9 revert: Remove external Chinese character fix for clean IP
Original contribution by andyxinweiminicloud will be re-implemented by Basic Machines team for clean IP ownership.
2025-08-08 15:23:28 -05:00
phernandez 9e4b8bca8f Add legal inventory documentation for IP analysis 2025-08-08 15:16:39 -05:00
Drew Cain b0cc559426 chore: update version to 0.14.3 for v0.14.3 release 2025-08-01 22:06:53 -05:00
Drew Cain 7460a938df fix: make case sensitivity test platform-aware
- Add platform detection to handle case-insensitive file systems
- Test now passes on macOS and Windows while maintaining Linux behavior
- Fixes test failure on case-insensitive file systems
2025-08-01 22:02:53 -05:00
Drew Cain 43fa5762a8 ruff checks
Signed-off-by: Drew Cain <groksrc@gmail.com>
2025-08-01 21:50:18 -05:00
Paul Hernandez fb1350b294 fix: enhance character conflict detection and error handling for sync operations (#201)
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Paul Hernandez <phernandez@users.noreply.github.com>
2025-08-01 21:35:56 -05:00
Paul Hernandez 7585a29c96 fix: replace recursive _traverse_messages with iterative approach to handle deep conversation threads (#235)
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Paul Hernandez <phernandez@users.noreply.github.com>
2025-08-01 21:34:11 -05:00
Drew Cain 752c78c379 chore: minor cleanup (#228)
Signed-off-by: Drew Cain <groksrc@gmail.com>
2025-07-31 21:32:56 -05:00
jope-bm a4a3b1b689 fix: handle missing 'name' key in memory JSON import (#241)
Signed-off-by: Joe P <joe@basicmemory.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: jope-bm <jope-bm@users.noreply.github.com>
2025-07-28 14:52:15 -06:00
jope-bm 6361574a20 fix: basic memory home env var not respected when project path is changed. (#239)
Signed-off-by: Joe P <joe@basicmemory.com>
2025-07-28 14:50:47 -06:00
jope-bm 24a1d6195d fix: path traversal security vulnerability in mcp tools (#223)
Signed-off-by: Joe P <joe@basicmemory.com>
2025-07-15 09:05:11 -06:00
jope-bm a0cf62375d docs: improve virtual environment setup instructions (#222)
Co-authored-by: Claude <noreply@anthropic.com>
2025-07-10 10:18:43 -06:00
Paul Hernandez 473f70c949 chore: Cloud auth (#213)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude <noreply@anthropic.com>
2025-07-07 21:08:25 -05:00
phernandez 2c29dcc2b2 chore: update version to 0.14.2 for v0.14.2 release 2025-07-03 17:30:40 -05:00
phernandez 448210e552 docs: add v0.14.2 changelog entry
🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-03 17:23:54 -05:00
Drew Cain 3621bb7b4d fix: MCP Error with MCP-Hub #204 (#212)
Signed-off-by: Drew Cain <groksrc@gmail.com>
2025-07-03 16:57:43 -05:00
Drew Cain f80ac0ee72 fix: replace deprecated datetime.utcnow() with timezone-aware alternatives and suppress SQLAlchemy warnings (#211)
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
2025-07-03 16:57:30 -05:00
Drew Cain 23ddf1918c chore: update version to 0.14.1 for v0.14.1 release 2025-07-01 22:08:25 -05:00
Drew Cain 2aca19aa05 chore: apply ruff formatting 2025-07-01 22:05:17 -05:00
Drew Cain 827f7cf3e3 fix: constrain fastmcp version to prevent breaking changes (#203)
Signed-off-by: Drew Cain <groksrc@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
2025-07-01 22:01:59 -05:00
Drew Cain bd4f55158b fix: Problems with MCP #190 (#202)
Signed-off-by: Drew Cain <groksrc@gmail.com>
2025-07-01 10:50:44 -05:00
Drew Cain 5360005122 feat: Add to cursor button (#200)
Signed-off-by: Drew Cain <groksrc@gmail.com>
2025-07-01 09:17:48 -05:00
Drew Cain 39f811f8b5 Update README.md
Added Homebrew instructions to README.md

Signed-off-by: Drew Cain <groksrc@users.noreply.github.com>
2025-06-26 21:51:14 -05:00
phernandez 8e69c8b533 chore: update version to 0.14.0 for v0.14.0 release 2025-06-26 16:18:10 -05:00
phernandez 627a5c3c22 docs: add comprehensive v0.14.0 changelog entry
Add detailed changelog for v0.14.0 release including:
- Docker Container Registry migration to GitHub Container Registry
- Enhanced search documentation with comprehensive syntax examples
- Cross-project file management with intelligent boundary detection
- 8 major bug fixes with issue numbers and commit links
- Technical improvements and infrastructure enhancements
- Migration guide and installation instructions

Covers all changes since v0.13.7 with proper categorization and
user-facing descriptions for better release communication.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: phernandez <paul@basicmachines.co>
2025-06-26 16:15:11 -05:00
phernandez cd88945b22 remove v0.13.0 from changelog
Signed-off-by: phernandez <paul@basicmachines.co>
2025-06-26 16:11:05 -05:00
phernandez cd8e372f0a fix: add test coverage for optional permalink in EntityResponse schema
- Add comprehensive test for None permalink validation in EntityResponse
- Ensures schema properly handles markdown files without explicit permalinks
- Addresses GitHub issue #170 validation errors during edit operations
- Test validates that permalink=None doesn't cause ValidationError

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: phernandez <paul@basicmachines.co>
2025-06-26 15:58:49 -05:00
Paul Hernandez a589f8b894 feat: enhance search_notes tool documentation with comprehensive syntax examples (#186)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Paul Hernandez <phernandez@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
2025-06-26 15:51:58 -05:00
Paul Hernandez c2f4b632cf fix: preserve permalink when editing notes without frontmatter permalink (#184)
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Paul Hernandez <phernandez@users.noreply.github.com>
2025-06-26 15:35:31 -05:00
phernandez 46d102cef1 update tests for search_repository
Signed-off-by: phernandez <paul@basicmachines.co>
2025-06-26 14:30:10 -05:00
phernandez 8e4dc026ce chore: update version to 0.14.0b1 for v0.14.0b1 beta release 2025-06-26 14:08:32 -05:00
phernandez 7af8e198c2 style: fix linting errors in test assertions
Replace equality comparisons to False with 'not' for better style.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: phernandez <paul@basicmachines.co>
2025-06-26 14:06:51 -05:00
Paul Hernandez 12b51522bc fix: implement project-specific sync status checks for MCP tools (#183)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude <noreply@anthropic.com>
2025-06-26 13:54:26 -05:00
Paul Hernandez ac9e148bcc test: add more tests for search_repository (#181)
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Paul Hernandez <phernandez@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
2025-06-26 13:45:07 -05:00
Paul Hernandez 546e3cd8db fix: handle Boolean search syntax with hyphenated terms (#180)
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Paul Hernandez <phernandez@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
2025-06-26 12:41:32 -05:00
phernandez de4737cc22 fix: correct typo and update changelog command template
- Fix typo: <versuib> → <version>
- Update version examples to v0.14.0 format
- Improve template formatting clarity

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-06-26 11:09:32 -05:00
phernandez 77eefeb252 update test-live.md regression suite
Signed-off-by: phernandez <paul@basicmachines.co>
2025-06-26 10:25:48 -05:00
phernandez e5923a0378 allow web_search in claude github action
Signed-off-by: phernandez <paul@basicmachines.co>
2025-06-26 09:29:23 -05:00
phernandez 1bf348259b fix formatting on files
Signed-off-by: phernandez <paul@basicmachines.co>
2025-06-25 22:32:05 -05:00
phernandez 224e4bf9e4 fixes #164 revove log level from mcp_server.run()
Signed-off-by: phernandez <paul@basicmachines.co>
2025-06-25 22:31:47 -05:00
Drew Cain 9f1db23c78 fix: respect BASIC_MEMORY_HOME environment variable in Docker containers (#174)
Signed-off-by: Drew Cain <groksrc@gmail.com>
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
2025-06-25 21:40:30 -05:00
Paul Hernandez db5ef7d35c feat: enhance move_note tool with cross-project detection and guidance (#161)
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Paul Hernandez <phernandez@users.noreply.github.com>
2025-06-25 12:57:59 -05:00
Paul Hernandez f50650763d fix: ensure permalinks are generated for entities with null permalinks during move operations (#162)
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Paul Hernandez <phernandez@users.noreply.github.com>
2025-06-25 12:57:44 -05:00
Drew Cain 8a065c32f4 fix: handle None from_entity in Context API RelationSummary (#166)
Signed-off-by: Drew Cain <groksrc@gmail.com>
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
2025-06-25 12:57:31 -05:00
Drew Cain 2a3adc109a fix: scope entity queries by project_id in upsert_entity method (#168)
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
2025-06-24 00:02:18 -05:00
Drew Cain a52ce1c860 fix: only update Homebrew on stable releases
Signed-off-by: Drew Cain <groksrc@gmail.com>
2025-06-21 08:12:23 -05:00
phernandez 616c1f0710 feat: switch from Docker Hub to GitHub Container Registry
🏴 Fighting the power! No more $15/month Docker Hub fees.

- Use ghcr.io/basicmachines-co/basic-memory for container images
- Native GitHub integration with GITHUB_TOKEN (no external secrets)
- Update all documentation and examples to use GHCR
- Remove Docker Hub description update step (not needed for GHCR)
- Completely free solution for public repositories

Docker users can now:
docker pull ghcr.io/basicmachines-co/basic-memory:latest
2025-06-20 15:57:49 -05:00
Paul Hernandez 74847cc380 feat: implement Docker CI workflow for automated image publishing (#159)
Co-authored-by: Claude <noreply@anthropic.com>
2025-06-20 15:42:55 -05:00
phernandez d3b6c85184 docs: add v0.13.8 changelog entry
Documents recent fixes and features including:
- Docker container support with volume mounting
- #151: Reset command project configuration fix
- #148: MCP/CLI project state consistency fix
- FastMCP compatibility improvements
- Comprehensive integration testing

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-06-19 22:13:32 -05:00
Paul Hernandez af44941d5a fix: reset command now clears project configuration (#152)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Paul Hernandez <phernandez@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
2025-06-19 21:55:50 -05:00
Paul Hernandez 35e4f73ae8 fix: resolve project state inconsistency between MCP and CLI (#149)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Paul Hernandez <phernandez@users.noreply.github.com>
2025-06-19 21:24:51 -05:00
Drew Cain 7be001ca68 fix: fastmcp deprecation warning (#150)
Signed-off-by: Drew Cain <groksrc@gmail.com>
Signed-off-by: Paul Hernandez <60959+phernandez@users.noreply.github.com>
Co-authored-by: Paul Hernandez <60959+phernandez@users.noreply.github.com>
2025-06-19 19:59:17 -05:00
Paul Hernandez 3269a2f33a feat: add Docker container support with volume mounting (#131)
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: phernandez <phernandez@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
2025-06-19 19:57:30 -05:00
Drew Cain b8191d090f chore: update version to 0.13.7 for v0.13.7 release 2025-06-18 22:32:53 -05:00
Drew Cain 2ce8a8e4b0 feat: Automatically update Homebrew
Signed-off-by: Drew Cain <groksrc@users.noreply.github.com>
2025-06-18 22:00:02 -05:00
Drew Cain f8099cd004 feat: Automatically update Homebrew (#147)
Signed-off-by: Drew Cain <groksrc@users.noreply.github.com>
2025-06-18 21:54:49 -05:00
phernandez 688e0b0971 chore: update version to 0.13.6 for v0.13.6 release 2025-06-18 17:58:56 -05:00
phernandez ed09ea4ec7 docs: add git sign-off reminder to CLAUDE.md
🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-06-18 17:56:24 -05:00
phernandez c85d9f74d7 docs: add v0.13.6 changelog entry
🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-06-18 17:55:21 -05:00
Paul Hernandez 84d2aaf641 fix: eliminate redundant database migration initialization (#146)
Co-authored-by: Claude <noreply@anthropic.com>
2025-06-18 17:32:20 -05:00
Paul Hernandez 7789864493 fix: add entity_type parameter to write_note MCP tool (#145)
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Paul Hernandez <phernandez@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
2025-06-18 17:10:15 -05:00
Drew Cain c6215fd819 fix: UNIQUE constraint failed: entity.permalink issue #139 (#140)
Signed-off-by: Drew Cain <groksrc@users.noreply.github.com>
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude <noreply@anthropic.com>
2025-06-18 15:03:11 -05:00
Drew Cain b4c26a6133 fix: correct spelling error "Chose" to "Choose" in continue_conversation prompt (#141)
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Paul Hernandez <phernandez@users.noreply.github.com>
2025-06-17 22:15:14 -05:00
phernandez 3fdce683d7 Update README with new website and community links
- Add new main website: https://basicmemory.com
- Add Discord community: https://discord.gg/tyvKNccgqN
- Add YouTube channel: https://www.youtube.com/@basicmachines-co
- Reorganize links section for better clarity

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-06-15 10:37:24 -05:00
phernandez 782cb2df28 update README.md and CLAUDE.md docs
Signed-off-by: phernandez <paul@basicmachines.co>
2025-06-12 14:24:37 -05:00
phernandez 56c875f137 chore: update version to 0.13.5 for v0.13.5 release 2025-06-11 22:02:56 -05:00
phernandez 5049de7e2d docs: add changelog entry for v0.13.5
- Renamed create_project to create_memory_project for namespace isolation

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-06-11 22:01:10 -05:00
phernandez 49011768f7 fix: rename create_project to create_memory_project for namespace isolation
Continue the namespace isolation effort by renaming the create_project tool
to create_memory_project to avoid conflicts with other MCP servers.

Changes:
- Renamed @mcp.tool() decorator from 'create_project' to 'create_memory_project'
- Updated all test references to use the new tool name
- Tool functionality remains identical, only the name changed
- Part of broader effort to ensure Basic Memory tools have unique namespaced names

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-06-11 21:58:40 -05:00
phernandez bc3557f000 chore: update version to 0.13.4 for v0.13.4 release 2025-06-11 21:41:06 -05:00
phernandez 611f5cd305 docs: add changelog entry for v0.13.4
- Renamed list_projects to list_memory_projects for namespace isolation

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-06-11 21:39:20 -05:00
phernandez 4ea392d284 fix: rename list_projects to list_memory_projects to avoid naming conflicts
The tool name 'list_projects' was too generic and could conflict with other MCP servers.
Renamed to 'list_memory_projects' for better specificity and namespace isolation.

Changes:
- Renamed @mcp.tool() decorator from 'list_projects' to 'list_memory_projects'
- Updated all test references to use the new tool name
- Tool functionality remains identical, only the name changed

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-06-11 21:37:48 -05:00
phernandez d491757980 docs: add changelog entries for v0.13.2 and v0.13.3
- v0.13.2: automated release management system with version control
- v0.13.3: case-insensitive project switching bug fixes

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-06-11 19:33:48 -05:00
phernandez 7a69ca2c36 chore: update version to 0.13.3 for v0.13.3 release 2025-06-11 19:29:04 -05:00
phernandez 70a6ce3411 fix: resolve case-insensitive project switching issues
This commit fixes the persistent case-insensitive project switching bug
where switching to projects with different case variations would succeed
but subsequent operations would fail.

Key changes:
- Enhanced config manager with case-insensitive project lookup using permalinks
- Updated project management tools to handle both name and permalink matching
- Fixed API URL construction to use permalinks consistently
- Added comprehensive test coverage for case-insensitive operations
- Updated project service to support permalink-based lookups

The fix ensures that users can switch to projects using any case variation
(e.g., "personal", "Personal", "PERSONAL") and all subsequent operations
work correctly with the canonical project name.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-06-11 19:26:27 -05:00
phernandez 5b69fd65cd fix: resolve case-insensitive project switching database lookup issue
Fix project switching bug where case-insensitive matching worked but
caused database lookup failures for subsequent operations.

**Problem:**
- switch_project('personal') succeeded (case-insensitive matching)
- get_current_project() failed with 'Project personal not found'
- Session stored user input case instead of canonical database name

**Solution:**
- Find project by permalink (case-insensitive) in switch_project
- Store canonical project name from database in session
- Use canonical name for all API calls and responses

**Test Coverage:**
- Added comprehensive case-insensitive project switching tests
- Added tests for case preservation in project listings
- Added tests for session state consistency after case switching
- Added error handling tests for non-existent projects

**Files Changed:**
- src/basic_memory/mcp/tools/project_management.py: Fixed switch_project logic
- test-int/mcp/test_project_management_integration.py: Added test coverage

**Test Cases Now Passing:**
-  switch_project('personal') → finds 'Personal' project
-  get_current_project() → works with canonical name
-  Project summary shows stats correctly
-  Case-insensitive matching for all case variations
-  Error handling for non-existent projects

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-06-11 18:19:17 -05:00
phernandez 85a178a6b8 chore: update version to 0.13.2 for v0.13.2 release 2025-06-11 17:09:57 -05:00
phernandez e4b32d7bc9 feat: add automated release management system
- Add version management in __init__.py
- Add justfile targets for release and beta automation
- Create Claude command documentation for /release and /beta
- Implement comprehensive quality checks and validation
- Support automated version updates and git tagging

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-06-11 17:06:23 -05:00
phernandez 9590b934cf Merge branch 'main' of github.com:basicmachines-co/basic-memory 2025-06-11 16:55:48 -05:00
phernandez 735f239f9b chore: update CHANGELOG.md for v0.13.1 release
Add changelog entry for v0.13.1 patch release documenting:
- Fixed CLI project management commands (#129)
- Resolved case sensitivity issues in project switching (#127)
- API endpoint standardization and improved error handling
- Consistent project name handling using permalinks

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-06-11 16:14:04 -05:00
Paul Hernandez 3ee30e1f36 fix: project cli commands and case sensitivity when switching projects (#130)
Signed-off-by: phernandez <paul@basicmachines.co>
2025-06-11 16:09:53 -05:00
phernandez ac401ea254 chore: prepare for v0.13.0 release by removing release notes file
The release notes content has been integrated into CHANGELOG.md.
Removing the standalone RELEASE_NOTES_v0.13.0.md file as it's no longer needed.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-06-11 08:36:30 -05:00
phernandez fb2fd62ed9 fix: resolve type error and prepare for v0.13.0b6 release
- Add type ignore comment for MCP prompt function call
- Function works correctly at runtime despite false positive type error
- All quality checks now passing
2025-06-09 15:38:59 -05:00
phernandez 126d1655e6 fix: simplify versioning for release workflow
- Use static API version 'v0' instead of dynamic package version
- Remove version verification step in release workflow
- Dynamic versioning handled by uv-dynamic-versioning at build time
2025-06-09 15:25:20 -05:00
phernandez 2abf626c46 fix: resolve unused variable lint warnings in tests
- Remove unused variables in test mock functions
- Clean up test code per ruff linting rules
2025-06-09 15:15:05 -05:00
phernandez ba8e3d112d chore: update dependencies for beta release
- fastmcp 2.7.0 -> 2.7.1
- automated dependency updates
2025-06-09 00:48:48 -05:00
phernandez 7108a7baf1 fix: resolve sync race conditions and search errors
- Add IntegrityError handling in entity_service.create_entity_from_markdown for file_path/permalink constraint violations
- Add IntegrityError handling in sync_service.sync_regular_file for concurrent sync race conditions
- Fix FTS "unknown special query" error when searching for wildcard "*" patterns
- Add comprehensive test coverage for race condition edge cases and error handling
- Gracefully handle concurrent sync processes with fallback to update operations

Fixes sync errors from beta testing including:
- "UNIQUE constraint failed: entity.file_path"
- "UNIQUE constraint failed: entity.permalink"
- "unknown special query" FTS errors

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-06-08 15:20:27 -05:00
phernandez 35884ef3a7 fix: update MCP tool/prompt/resource calls to use .fn attribute
FastMCP library changes now require calling decorated functions via the .fn attribute:
- Tools: @mcp.tool() functions return FunctionTool, call with tool.fn()
- Prompts: @mcp.prompt() functions return FunctionPrompt, call with prompt.fn()
- Resources: @mcp.resource() functions return FunctionResource, call with resource.fn()

Updated core files:
- view_note.py: read_note() → read_note.fn()
- read_note.py: search_notes() → search_notes.fn() (2 locations)
- tool.py: 6 MCP tool calls updated to use .fn
- recent_activity.py: recent_activity() → recent_activity.fn()
- project.py: project_info() → project_info.fn() with type ignore

Updated 100+ test files systematically to use .fn attribute and fixed mock targets.

All 869 tests now pass. Fixes view_note tool error in Claude Desktop.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-06-08 11:52:23 -05:00
phernandez 040be05a81 fix: normalize project names in config during startup
- Fix case sensitivity bug where config had "Personal" but database expected "personal"
- Add project name normalization in synchronize_projects() to use generate_permalink()
- Update config file with normalized names and log changes for user visibility
- Use proper permalink generation instead of hardcoded name.lower().replace()
- Add comprehensive tests for project name normalization scenarios

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-06-08 09:28:57 -05:00
phernandez c141d7d1e6 chore: update version to 0.13.0b5 for release 2025-06-05 17:08:07 -05:00
phernandez b73aeb5ed8 feat: add view_note tool for formatted artifacts
- Implement view_note tool for better note readability in Claude Desktop
- Display notes as formatted markdown artifacts with special instructions
- Extract titles from frontmatter or headings automatically
- Add comprehensive test suite with 100% coverage
- Include view_note in live testing plan and release notes

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-06-05 16:57:11 -05:00
phernandez 9a0e0bd82d add view_note tool
Signed-off-by: phernandez <paul@basicmachines.co>
2025-06-05 16:30:09 -05:00
phernandez 117fa44ecf fix project info stats tests
Signed-off-by: phernandez <paul@basicmachines.co>
2025-06-05 15:51:11 -05:00
phernandez 69d7610d47 test coverage 100%
Signed-off-by: phernandez <paul@basicmachines.co>
2025-06-05 13:07:20 -05:00
phernandez f608cd13f1 add justfile instead of Makefile, add ignores to test coverage
Signed-off-by: phernandez <paul@basicmachines.co>
2025-06-05 12:26:31 -05:00
phernandez 2162ad57fe all tests passing
Signed-off-by: phernandez <paul@basicmachines.co>
2025-06-05 11:13:40 -05:00
phernandez dd6ca80716 fix link_resolver tests
Signed-off-by: phernandez <paul@basicmachines.co>
2025-06-04 23:57:01 -05:00
phernandez ae3eeb0cc1 add tool prompting and doc updates for strict mode in edit/move, and sync_status tool
Signed-off-by: phernandez <paul@basicmachines.co>
2025-06-04 23:46:48 -05:00
phernandez 602c55fe90 only allow edit_note, move_note using strict identifier match
Signed-off-by: phernandez <paul@basicmachines.co>
2025-06-04 23:37:29 -05:00
phernandez 91bfe2dc92 add sync status tool
Signed-off-by: phernandez <paul@basicmachines.co>
2025-06-04 22:24:10 -05:00
phernandez a3cae1064d add background migration task and status tool/prompt
Signed-off-by: phernandez <paul@basicmachines.co>
2025-06-04 16:52:12 -05:00
phernandez c5c70cb0f4 improve validation for memory:// urls, add examples to build_context
Signed-off-by: phernandez <paul@basicmachines.co>
2025-06-04 00:16:33 -05:00
phernandez 80ec860a1c remove coverage files
Signed-off-by: phernandez <paul@basicmachines.co>
2025-06-03 23:14:07 -05:00
phernandez f64d5b2152 improve error messages for tools
Signed-off-by: phernandez <paul@basicmachines.co>
2025-06-03 23:12:59 -05:00
phernandez 69a625acd1 fix search escape issues, and empty forward reference resolving for entities
Signed-off-by: phernandez <paul@basicmachines.co>
2025-06-03 18:09:41 -05:00
phernandez 53c29a37ca fix: resolve FTS5 search syntax errors with special characters
Enhances search term preparation to handle special characters gracefully while preserving functionality:

- Improves FTS5 query preparation with targeted special character handling
- Preserves boolean operators (AND, OR, NOT) without modification
- Quotes problematic characters that cause syntax errors
- Maintains wildcard patterns for legitimate use cases
- Adds comprehensive error handling with graceful fallback

Includes extensive test coverage:
- 10 new test cases for various search scenarios
- Programming terms (C++, function(), email@domain.com) now searchable
- Malformed syntax handled without crashes
- Boolean and wildcard functionality preserved

Fixes search crashes when users enter queries containing special characters.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-06-03 16:45:13 -05:00
phernandez d8c13bf1d3 fix project table unique constraint bug
Signed-off-by: phernandez <paul@basicmachines.co>
2025-06-03 14:23:15 -05:00
phernandez 3f70f5ed42 feat: add /project:test-live command for comprehensive real-world testing
Implements live testing suite that:
- Uses installed Basic Memory version via MCP
- Follows TESTING.md methodology systematically
- Records all observations in Basic Memory notes
- Tests all 5 phases: core, features, edge cases, workflows, stress
- Creates dedicated test project for isolation
- Documents bugs with reproduction steps
- Tracks performance metrics and UX insights
- Validates v0.13.0 features in real usage scenarios

This enables 'Basic Memory testing itself' - comprehensive integration
testing that creates living documentation of test results.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-06-03 09:40:52 -05:00
phernandez 569a3de80b feat: add comprehensive custom Claude Code slash commands
Adds custom slash commands for streamlined development workflow:

Release Management (/project:release:*):
- beta - Create beta releases with automated quality checks
- release - Create stable releases with comprehensive validation
- release-check - Pre-flight validation without making changes
- changelog - Generate changelog entries from commits

Development (/project:*):
- test-coverage - Run tests with detailed coverage analysis
- lint-fix - Comprehensive code quality fixes with auto-repair
- check-health - Project health assessment and metrics

Commands are organized in .claude/commands/ directory following Claude Code
conventions and provide structured automation for common development tasks.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-06-03 09:24:03 -05:00
phernandez ac08a8d024 fix: update FastMCP initialization for API changes
- Remove deprecated auth_server_provider parameter
- Use auth parameter correctly with OAuthProvider instead of AuthSettings
- Fixes type error after dependency updates

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-06-03 09:08:17 -05:00
phernandez c13d4b1511 chore: update dependencies via make update-deps
- Updated authlib and other dependencies to latest versions
- Includes setuptools import fix for runtime compatibility

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-06-03 08:53:56 -05:00
phernandez 5b85d33a99 fix: remove unused setuptools import causing runtime error
Fixes ModuleNotFoundError when basic-memory is installed in environments
without setuptools (common with uv tool installs)

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-06-03 08:52:30 -05:00
phernandez e2bd3142dc docs: remove working documentation files
Clean up development docs before v0.13.0 beta release

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-06-03 08:45:37 -05:00
phernandez 40b7de9168 docs: add comprehensive v0.13.0 changelog entry
🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-06-03 08:43:54 -05:00
phernandez 455a280707 Merge branch 'main' of github.com:basicmachines-co/basic-memory 2025-06-03 08:31:42 -05:00
Paul Hernandez 634486107b feat: v0.13.0 pre (#122)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude <noreply@anthropic.com>
2025-06-03 01:00:40 -05:00
phernandez 23d34289fd Merge branch 'main' of github.com:basicmachines-co/basic-memory 2025-05-28 14:46:45 -05:00
Paul Hernandez 5ec4087f7c feat: Add Claude Code GitHub Workflow (#120)
Co-authored-by: Claude <noreply@anthropic.com>
2025-05-28 14:45:33 -05:00
phernandez 71941c35dd Merge branch 'main' of github.com:basicmachines-co/basic-memory 2025-05-25 10:07:54 -05:00
bm-claudeai 020957cd76 feat: Multi-project support, OAuth authentication, and major improvements (#119)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude <noreply@anthropic.com>
2025-05-25 10:07:34 -05:00
phernandez b3af6aa685 Merge branch 'main' of github.com:basicmachines-co/basic-memory 2025-04-17 11:12:18 -05:00
semantic-release 9437a5f83b chore(release): 0.12.3 [skip ci] 2025-04-17 16:04:31 +00:00
phernandez 3c1cc346df fix: modify recent_activity args to be strings instead of enums
Signed-off-by: phernandez <paul@basicmachines.co>
2025-04-17 10:56:46 -05:00
phernandez 81616ab42e update CLAUDE.md
Signed-off-by: phernandez <paul@basicmachines.co>
2025-04-17 10:56:46 -05:00
phernandez 321471f29d update CLAUDE.md
Signed-off-by: phernandez <paul@basicmachines.co>
2025-04-17 10:17:47 -05:00
phernandez 73ea91fe0d fix: add extra logic for permalink generation with mixed Latin unicode and Chinese characters
Signed-off-by: phernandez <paul@basicmachines.co>
2025-04-17 10:00:40 -05:00
andyxinweiminicloud 03d4e97b90 Fix: preserve Chinese characters in permalinks 2025-04-17 09:03:35 -05:00
Paul Hernandez 98622a7a47 Update README.md
fix example json for passing --project arg

Signed-off-by: Paul Hernandez <60959+phernandez@users.noreply.github.com>
2025-04-16 07:50:12 -05:00
Amadeusz Wieczorek 54dfa08aba Update name of function search_notes in error output (#92)
Signed-off-by: Amadeusz Wieczorek <git@amadeusw.com>
2025-04-14 11:06:15 -05:00
semantic-release 9433065a57 chore(release): 0.12.2 [skip ci] 2025-04-08 16:27:42 +00:00
Paul Hernandez 2934176331 fix: utf8 for all file reads/write/open instead of default platform encoding (#91)
Signed-off-by: phernandez <paul@basicmachines.co>
2025-04-08 10:55:01 -05:00
semantic-release ac89eb47cb chore(release): 0.12.1 [skip ci] 2025-04-07 22:47:23 +00:00
Paul Hernandez 78a3412bcf fix: run migrations and sync when starting mcp (#88) 2025-04-07 17:45:26 -05:00
Paul Hernandez fa314b5d2b Update README.md
Signed-off-by: Paul Hernandez <60959+phernandez@users.noreply.github.com>
2025-04-06 20:29:00 -05:00
Paul Hernandez d2ac62a27c Update README.md
Signed-off-by: Paul Hernandez <60959+phernandez@users.noreply.github.com>
2025-04-06 20:25:06 -05:00
semantic-release 7ce7a29f35 chore(release): 0.12.0 [skip ci] 2025-04-06 23:44:17 +00:00
Paul Hernandez 3f4d9e4d87 fix: write_note preserves frontmatter fields in content (#84) 2025-04-06 18:39:51 -05:00
Paul Hernandez 00c8633cfc feat: add watch to mcp process (#83) 2025-04-06 18:33:10 -05:00
Paul Hernandez 617e60bda4 feat: permalink enhancements (#82)
- Avoiding "useless permalink values" for files without metadata
- Enable permalinks to be updated on move via config setting
2025-04-06 14:54:59 -05:00
github-actions[bot] 6c19c9edf5 fix: [BUG] # character accumulation in markdown frontmatter tags prop (#79) 2025-04-05 23:07:53 -05:00
github-actions[bot] 9bff1f732e fix: [BUG] write_note Tool Fails to Update Existing Files in Some Situations. (#80) 2025-04-05 22:31:40 -05:00
Paul Hernandez 248214cb11 fix: set default mcp log level to ERROR (#81) 2025-04-05 22:27:29 -05:00
Jason Noble 40ea28b0bf docs: Updated basicmachines.co links to be https (#69) 2025-04-05 22:22:37 -05:00
Marc Baiza 43cbb7b38c docs: Add VS Code instructions to README (#76) 2025-04-05 22:18:56 -05:00
github-actions[bot] 7930ddb291 fix: [BUG] Some notes never exit "modified" status (#77) 2025-04-05 22:03:38 -05:00
phernandez 1844e58210 Add pull_request_target trigger to test workflow with explanatory comment 2025-04-05 20:30:05 -05:00
phernandez 7fbb5ebfc5 Update claude-code-github-action to v0.11.0 with backtick handling fix 2025-04-05 20:04:18 -05:00
phernandez 0e5a465b10 add claude-output to .gitignore
Signed-off-by: phernandez <paul@basicmachines.co>
2025-04-05 19:45:07 -05:00
github-actions[bot] 9d581cee13 fix: [BUG] Cursor has errors calling search tool (#78) 2025-04-05 19:12:23 -05:00
phernandez f58852954b Update to claude-code-github-action v0.10.0 to check comment author for org membership 2025-04-05 12:54:33 -05:00
phernandez edd426b99f Update to claude-code-github-action v0.9.0 with organization membership check fix 2025-04-05 12:42:35 -05:00
phernandez 6b4a421315 Update to claude-code-github-action v0.8.0 with PERSONAL_ACCESS_TOKEN support 2025-04-05 12:13:05 -05:00
phernandez 4632405446 Fix claude-code-actions.yml to use v0.7.0 of basicmachines-co action 2025-04-05 12:01:04 -05:00
phernandez 3a6d5ff58d update claude code actions
Signed-off-by: phernandez <paul@basicmachines.co>
2025-04-04 23:31:08 -05:00
phernandez 03de85ec32 add claude code github action workflow
Signed-off-by: phernandez <paul@basicmachines.co>
2025-04-04 21:20:40 -05:00
phernandez 084c91ecce update basic-memory dep version
Signed-off-by: phernandez <paul@basicmachines.co>
2025-03-30 17:46:44 -05:00
semantic-release 3d45227b72 chore(release): 0.11.0 [skip ci] 2025-03-29 00:19:46 +00:00
Paul Hernandez 069c0a21c6 feat: add bm command alias for basic-memory (#67)
Signed-off-by: phernandez <paul@basicmachines.co>
2025-03-28 19:12:25 -05:00
Paul Hernandez b27827671d feat: rename search tool to search_notes (#66)
Signed-off-by: phernandez <paul@basicmachines.co>
2025-03-28 19:11:35 -05:00
Paul Hernandez 0743ade5fc fix: just delete db for reset db instead of using migrations. (#65)
Signed-off-by: phernandez <paul@basicmachines.co>
2025-03-28 19:11:11 -05:00
Paul Hernandez f1c95709cb fix: make logs for each process - mcp, sync, cli (#64)
Signed-off-by: phernandez <paul@basicmachines.co>
2025-03-28 19:10:43 -05:00
Matias Forbord 3c68b7d5dd docs: Update broken "Multiple Projects" link in README.md (#55) 2025-03-26 14:31:51 -05:00
semantic-release 3d0077990f chore(release): 0.10.1 [skip ci] 2025-03-25 18:12:28 +00:00
phernandez 731b502d36 docs: add help docs to mcp cli tools
Signed-off-by: phernandez <paul@basicmachines.co>
2025-03-25 13:01:11 -05:00
phernandez 2a881b1425 refactor: move project stats into projct subcommand
Signed-off-by: phernandez <paul@basicmachines.co>
2025-03-25 12:50:11 -05:00
phernandez 681af5d450 chore: remove duplicate code in entity_service.py from bad merge
Signed-off-by: phernandez <paul@basicmachines.co>
2025-03-25 12:27:37 -05:00
phernandez b35037eb86 fix broken merge
Signed-off-by: phernandez <paul@basicmachines.co>
2025-03-24 23:18:22 -05:00
phernandez b667bca5a2 fix test coverage and type checks
Signed-off-by: phernandez <paul@basicmachines.co>
2025-03-24 22:46:47 -05:00
phernandez e716946b44 fix: preserve custom frontmatter fields when updating notes
Fixes #36 by modifying entity_service.update_entity() to read existing
frontmatter from files before updating them. Custom metadata fields
such as Status, Priority, and Version are now preserved when notes
are updated through the write_note MCP tool.

Added test case that verifies this behavior by creating a note with
custom frontmatter and then updating it.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: phernandez <paul@basicmachines.co>
2025-03-24 22:46:47 -05:00
phernandez 46c4fd2164 fix: make set_default_project also activate project for current session to fix #37
This change makes the 'basic-memory project default <name>' command both:
1. Set the default project for future invocations (persistent change)
2. Activate the project for the current session (immediate change)

Added tests to verify this behavior, which resolves issue #37 where the
project name and path weren't changing properly when the default project
was changed.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: phernandez <paul@basicmachines.co>
2025-03-24 22:46:47 -05:00
phernandez 9c791259a0 \fix: improve tags handling in write_note tool to fix #38\n\nThis change makes the write_note tool more flexible with tag inputs by:\n1. Adding a parse_tags helper function to handle various input formats\n2. Removing explicit type annotation from the write_note signature\n3. Adding documentation on how to pass tags from external MCP clients\n4. Adding a test that reproduces the issue from bug report #38\n\nThe issue occurred in external MCP clients like Cursor where the tags\nparameter was causing type mismatch errors.\n\n\ud83e\udd16 Generated with [Claude Code](https://claude.ai/code)\n\nCo-Authored-By: Claude <noreply@anthropic.com>\
Signed-off-by: phernandez <paul@basicmachines.co>
2025-03-24 22:46:47 -05:00
phernandez cc2cae72c1 fix: move ai_assistant_guide.md into package resources to fix #39
This change relocates the AI assistant guide from the static directory
into the package resources directory, ensuring it gets properly included
in the distribution package and is accessible when installed via pip/uv.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: phernandez <paul@basicmachines.co>
2025-03-24 22:46:47 -05:00
phernandez 0a1fdd8f15 update docs
Signed-off-by: phernandez <paul@basicmachines.co>
2025-03-24 22:46:47 -05:00
phernandez 78f234b180 fix: preserve custom frontmatter fields when updating notes
Fixes #36 by modifying entity_service.update_entity() to read existing
frontmatter from files before updating them. Custom metadata fields
such as Status, Priority, and Version are now preserved when notes
are updated through the write_note MCP tool.

Added test case that verifies this behavior by creating a note with
custom frontmatter and then updating it.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-03-24 19:50:38 -05:00
phernandez cbe72be10a fix: make set_default_project also activate project for current session to fix #37
This change makes the 'basic-memory project default <name>' command both:
1. Set the default project for future invocations (persistent change)
2. Activate the project for the current session (immediate change)

Added tests to verify this behavior, which resolves issue #37 where the
project name and path weren't changing properly when the default project
was changed.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-03-24 18:56:51 -05:00
phernandez 0ade6b0603 \fix: improve tags handling in write_note tool to fix #38\n\nThis change makes the write_note tool more flexible with tag inputs by:\n1. Adding a parse_tags helper function to handle various input formats\n2. Removing explicit type annotation from the write_note signature\n3. Adding documentation on how to pass tags from external MCP clients\n4. Adding a test that reproduces the issue from bug report #38\n\nThe issue occurred in external MCP clients like Cursor where the tags\nparameter was causing type mismatch errors.\n\n\ud83e\udd16 Generated with [Claude Code](https://claude.ai/code)\n\nCo-Authored-By: Claude <noreply@anthropic.com>\ 2025-03-24 18:30:21 -05:00
phernandez 390ff9d31c fix: move ai_assistant_guide.md into package resources to fix #39
This change relocates the AI assistant guide from the static directory
into the package resources directory, ensuring it gets properly included
in the distribution package and is accessible when installed via pip/uv.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-03-24 17:34:47 -05:00
phernandez 3ed74b4a4d update docs 2025-03-24 17:28:30 -05:00
Ikko Eltociear Ashimine dfaf0fea9c docs: update CLAUDE.md (#33)
fix spelling in CLAUDE.md: enviroment -> environment
Signed-off-by: Ikko Eltociear Ashimine <eltociear@gmail.com>
2025-03-24 13:15:12 -05:00
phernandez b26afa927f docs: add mcp badge, update cli reference, llms-install.md
Signed-off-by: phernandez <paul@basicmachines.co>
2025-03-16 14:09:08 -05:00
phernandez 3806943f8c add star history to README.md
Signed-off-by: phernandez <paul@basicmachines.co>
2025-03-16 11:24:08 -05:00
phernandez 3bffb2e190 Add updated getting started docs and utf8 test
Signed-off-by: phernandez <paul@basicmachines.co>
2025-03-15 16:52:17 -05:00
semantic-release 9fe34dcddf chore(release): 0.10.0 [skip ci] 2025-03-15 20:49:20 +00:00
Paul Hernandez eb5e4ec6bd fix: improve utf-8 support for file reading/writing (#32)
fixes #29

Signed-off-by: phernandez <paul@basicmachines.co>
2025-03-15 15:43:37 -05:00
Paul Hernandez 6b110b28dd fix: don't sync *.tmp files on watch (#31)
Fixes #30

Signed-off-by: phernandez <paul@basicmachines.co>
2025-03-15 15:43:28 -05:00
phernandez 88d193e0aa Edig content in README.md
Signed-off-by: phernandez <paul@basicmachines.co>
2025-03-15 11:46:53 -05:00
phernandez fa9175f049 Remove redundant info in README.md
Signed-off-by: phernandez <paul@basicmachines.co>
2025-03-15 11:45:28 -05:00
phernandez 7c787a413c Simpify README.md, ONTRIBUTING.md
Signed-off-by: phernandez <paul@basicmachines.co>
2025-03-15 11:31:18 -05:00
Paul Hernandez 0fa1adf3b5 Update README.md
Signed-off-by: Paul Hernandez <60959+phernandez@users.noreply.github.com>
2025-03-15 11:17:29 -05:00
phernandez cbe488fbd2 remove .obsidian dir in docs, try to fix video
Signed-off-by: phernandez <paul@basicmachines.co>
2025-03-15 11:00:27 -05:00
Paul Hernandez 99ff8f60c2 Update README.md
Signed-off-by: Paul Hernandez <60959+phernandez@users.noreply.github.com>
2025-03-15 10:57:59 -05:00
phernandez 06ee2852a0 try to fix video in README.md
Signed-off-by: phernandez <paul@basicmachines.co>
2025-03-15 10:53:08 -05:00
phernandez 939bab1a24 fix demo link in README.md
Signed-off-by: phernandez <paul@basicmachines.co>
2025-03-15 10:48:58 -05:00
phernandez 67736487fb update README.md
Signed-off-by: phernandez <paul@basicmachines.co>
2025-03-15 10:43:41 -05:00
phernandez 2ac620f98b Merge branch 'main' of github.com:basicmachines-co/basic-memory 2025-03-15 10:40:16 -05:00
phernandez 4cbda7a9d4 update docs
Signed-off-by: phernandez <paul@basicmachines.co>
2025-03-15 10:40:09 -05:00
Paul Hernandez 3070ade69a Create dependabot.yml
Signed-off-by: Paul Hernandez <60959+phernandez@users.noreply.github.com>
2025-03-15 10:34:53 -05:00
Paul Hernandez 9af913da4f docs: add glama badge. Fix typos in README.md (#28)
Signed-off-by: phernandez <paul@basicmachines.co>
2025-03-13 13:24:39 -05:00
Paul Hernandez 7b7bec9557 Create SECURITY.md
Signed-off-by: Paul Hernandez <60959+phernandez@users.noreply.github.com>
2025-03-12 15:51:23 -05:00
bm-claudeai fea2f40d1b docs: Update CLAUDE.md with GitHub integration capabilities (#25)
This PR updates the CLAUDE.md file to document the GitHub integration
capabilities that enable Claude to participate directly in the
development workflow.
2025-03-11 16:32:30 -05:00
bm-claudeai eb1e7b6088 feat: Add Smithery integration for easier installation (#24)
This PR adds support for deploying Basic Memory on the Smithery platform.

Signed-off-by: bm-claudeai <claude@basicmachines.co>
2025-03-11 16:21:51 -05:00
phernandez 9203730ba7 update README.md
Signed-off-by: phernandez <paul@basicmachines.co>
2025-03-11 14:04:01 -05:00
phernandez 1a473c02e9 remove cool asii art from README.md
Signed-off-by: phernandez <paul@basicmachines.co>
2025-03-11 14:00:37 -05:00
phernandez b6f5d8a545 update getting started docs
Signed-off-by: phernandez <paul@basicmachines.co>
2025-03-11 13:35:33 -05:00
phernandez addeeb1e7f update user docs
Signed-off-by: phernandez <paul@basicmachines.co>
2025-03-11 13:30:00 -05:00
phernandez 85f9099b0a update docs
Signed-off-by: phernandez <paul@basicmachines.co>
2025-03-08 11:53:24 -06:00
phernandez 82b6f3d60e update user guide docs
Signed-off-by: phernandez <paul@basicmachines.co>
2025-03-08 11:51:17 -06:00
semantic-release 8a7432949b chore(release): 0.9.0 [skip ci] 2025-03-07 20:54:32 +00:00
Paul Hernandez 6a4bd54646 chore: Pre beta prep (#20)
fix: drop search_index table on db reindex 
fix: ai_resource_guide.md path
chore: remove logfire
Signed-off-by: phernandez <paul@basicmachines.co>
2025-03-07 14:31:35 -06:00
phernandez da97353cfc fix: ai_resource_guide.md path
Signed-off-by: phernandez <paul@basicmachines.co>
2025-03-07 14:25:24 -06:00
phernandez 2e9d673e54 fix: ai_resource_guide.md path
Signed-off-by: phernandez <paul@basicmachines.co>
2025-03-07 14:24:48 -06:00
phernandez c4732a47b3 fix: ai_resource_guide.md path 2025-03-07 14:19:27 -06:00
phernandez 2d5176e800 add user guide docs
Signed-off-by: phernandez <paul@basicmachines.co>
2025-03-07 14:11:06 -06:00
phernandez 9bb8a020c3 chore: remove logfire
Signed-off-by: phernandez <paul@basicmachines.co>
2025-03-06 13:06:48 -06:00
phernandez 31cca6f913 fix: drop search_index table on db reindex
Signed-off-by: phernandez <paul@basicmachines.co>
2025-03-06 12:51:25 -06:00
Paul Hernandez d2bd75a949 feat: add project_info tool (#19)
Signed-off-by: phernandez <paul@basicmachines.co>
2025-03-05 23:44:28 -06:00
Paul Hernandez 90d5754180 feat: Implement boolean search (#18) 2025-03-05 20:57:12 -06:00
Paul Hernandez e6496df595 feat: Beta work (#17)
feat: Add multiple projects support 
feat: enhanced read_note for when initial result is not found
fix: merge frontmatter when updating note
fix: handle directory removed on sync watch
2025-03-05 18:46:04 -06:00
phernandez 41868fd34c Add contributing docs and github workflows 2025-03-01 15:46:51 -06:00
phernandez a3072702b9 add CLA.md 2025-03-01 14:13:09 -06:00
phernandez 182ec78355 docs: update README.md and CLAUDE.md 2025-03-01 13:43:09 -06:00
Paul Hernandez f8078cdd46 Fix: Beta testing fixes (#16)
Co-authored-by: phernandez <phernandez@basicmachines.co>
2025-02-28 20:23:53 -06:00
semantic-release 4bcbeacdd2 chore(release): 0.8.0 [skip ci] 2025-02-28 02:50:27 +00:00
Paul Hernandez 093dab5f03 feat: Add enhanced prompts and resources (#15)
## Summary
- Add comprehensive documentation to all MCP prompt modules
- Enhance search prompt with detailed contextual output formatting
- Implement consistent logging and docstring patterns across prompt
utilities
- Fix type checking in prompt modules

## Prompts Added/Enhanced
- `search.py`: New formatted output with relevance scores, excerpts, and
next steps
- `recent_activity.py`: Enhanced with better metadata handling and
documentation
- `continue_conversation.py`: Improved context management

## Resources Added/Enhanced
- `ai_assistant_guide`: Resource with description to give to LLM to
understand how to use the tools

## Technical improvements
- Added detailed docstrings to all prompt modules explaining their
purpose and usage
- Enhanced the search prompt with rich contextual output that helps LLMs
understand results
- Created a consistent pattern for formatting output across prompts
- Improved error handling in metadata extraction
- Standardized import organization and naming conventions
- Fixed various type checking issues across the codebase

This PR is part of our ongoing effort to improve the MCP's interaction
quality with LLMs, making the system more helpful and intuitive for AI
assistants to navigate knowledge bases.

🤖 Generated with [Claude Code](https://claude.ai/code)

---------

Co-authored-by: phernandez <phernandez@basicmachines.co>
2025-02-27 20:48:56 -06:00
Paul Hernandez 0d7b0b3d7e feat: Add new canvas tool to create json canvas files in obsidian. (#14)
Add new `canvas` tool to create json canvas files in obsidian.

---------

Co-authored-by: phernandez <phernandez@basicmachines.co>
2025-02-25 21:21:05 -06:00
phernandez 93cc6379eb chore: formatting 2025-02-24 22:18:27 -06:00
Paul Hernandez 37a01b806d feat: Incremental sync on watch (#13)
- incremental sync on watch
- sync non-markdown files in knowledge base
- experimental `read_resource` tool for reading non-markdown files in raw form (pdf, image)
2025-02-24 22:13:15 -06:00
phernandez a573f78317 try to optimize image size for mcp tool response 2025-02-23 17:30:47 -06:00
phernandez ab36e7e560 return optimized image from read_resource tool - tweaks 2025-02-23 16:54:07 -06:00
phernandez 959b278687 return optimized image from read_resource tool 2025-02-23 16:50:00 -06:00
phernandez 04b387c742 add more logging for image response 2025-02-23 16:38:20 -06:00
phernandez 0c59fd5e83 handle image response in tools WIP 2025-02-23 16:14:06 -06:00
phernandez 2ed920aa50 add resource tool to read binary files 2025-02-23 15:34:41 -06:00
phernandez bd20185492 read img/pdf via tool 2025-02-23 13:51:47 -06:00
phernandez f731a23e49 fix sync and status cli 2025-02-22 17:16:33 -06:00
phernandez 07a9415451 add alembic migration for relation_to_name uniqueness 2025-02-22 15:05:40 -06:00
phernandez 20d0375ffa sync non-markdown files 2025-02-22 14:50:40 -06:00
phernandez eb4a55a5e7 code cleanup 2025-02-21 22:11:54 -06:00
phernandez 51b4eb32c5 fix tests 2025-02-21 21:40:28 -06:00
phernandez 74adae506e fix tests 2025-02-21 21:36:39 -06:00
phernandez 94394f0bfe new sync service implementation 2025-02-21 19:10:23 -06:00
phernandez 9e3f71cb87 file sync WIP 2025-02-20 22:29:40 -06:00
phernandez f4b703e57f chore: refactor logging setup 2025-02-19 20:22:01 -06:00
semantic-release 6b8cefcd45 chore(release): 0.7.0 [skip ci] 2025-02-19 14:27:20 +00:00
phernandez 57984aa912 chore: fix tests 2025-02-18 23:48:22 -06:00
phernandez f5a7541da1 feat: add cli commands for mcp tools 2025-02-18 23:23:09 -06:00
phernandez bc9ca0744f fix: search query pagination params 2025-02-18 21:29:11 -06:00
phernandez 2c8ed1737d chore: remove unused tests 2025-02-18 20:45:47 -06:00
phernandez 02f8e86692 feat: add pagination to read_notes 2025-02-18 20:42:19 -06:00
phernandez 0123544556 feat: add pagination to build_context and recent_activity 2025-02-18 20:22:35 -06:00
phernandez 00d23a5ee1 fix: add logfire spans to cli 2025-02-18 19:56:20 -06:00
phernandez 812136c8c2 fix: add logfire spans to cli 2025-02-18 19:08:46 -06:00
phernandez 3e8e3e8961 fix: add logfire instrumentation to tools 2025-02-18 18:52:09 -06:00
phernandez 8544bb7966 Merge branch 'main' of github.com:basicmachines-co/basic-memory 2025-02-18 16:19:20 -06:00
phernandez 66b57e682f chore: re-add sync status console on watch 2025-02-18 16:19:14 -06:00
semantic-release 58a1296111 chore(release): 0.6.0 [skip ci] 2025-02-18 22:09:14 +00:00
Paul Hernandez 6da143898b feat: configure logfire telemetry (#12)
Co-authored-by: phernandez <phernandez@basicmachines.co>
2025-02-18 15:53:38 -06:00
397 changed files with 88739 additions and 8049 deletions
+154
View File
@@ -0,0 +1,154 @@
---
name: python-developer
description: Python backend developer specializing in FastAPI, DBOS workflows, and API implementation. Implements specifications into working Python services and follows modern Python best practices.
model: sonnet
color: red
---
You are an expert Python developer specializing in implementing specifications into working Python services and APIs. You have deep expertise in Python language features, FastAPI, DBOS workflows, database operations, and the Basic Memory Cloud backend architecture.
**Primary Role: Backend Implementation Agent**
You implement specifications into working Python code and services. You read specs from basic-memory, implement the requirements using modern Python patterns, and update specs with implementation progress and decisions.
**Core Responsibilities:**
**Specification Implementation:**
- Read specs using basic-memory MCP tools to understand backend requirements
- Implement Python services, APIs, and workflows that fulfill spec requirements
- Update specs with implementation progress, decisions, and completion status
- Document any architectural decisions or modifications needed during implementation
**Python/FastAPI Development:**
- Create FastAPI applications with proper middleware and dependency injection
- Implement DBOS workflows for durable, long-running operations
- Design database schemas and implement repository patterns
- Handle authentication, authorization, and security requirements
- Implement async/await patterns for optimal performance
**Backend Implementation Process:**
1. **Read Spec**: Use `mcp__basic-memory__read_note` to get spec requirements
2. **Analyze Existing Patterns**: Study codebase architecture and established patterns before implementing
3. **Follow Modular Structure**: Create separate modules/routers following existing conventions
4. **Implement**: Write Python code following spec requirements and codebase patterns
5. **Test**: Create tests that validate spec success criteria
6. **Update Spec**: Document completion and any implementation decisions
7. **Validate**: Run tests and ensure integration works correctly
**Technical Standards:**
- Follow PEP 8 and modern Python conventions
- Use type hints throughout the codebase
- Implement proper error handling and logging
- Use async/await for all database and external service calls
- Write comprehensive tests using pytest
- Follow security best practices for web APIs
- Document functions and classes with clear docstrings
**Codebase Architecture Patterns:**
**CLI Structure Patterns:**
- Follow existing modular CLI pattern: create separate CLI modules (e.g., `upload_cli.py`) instead of adding commands directly to `main.py`
- Existing examples: `polar_cli.py`, `tenant_cli.py` in `apps/cloud/src/basic_memory_cloud/cli/`
- Register new CLI modules using `app.add_typer(new_cli, name="command", help="description")`
- Maintain consistent command structure and help text patterns
**FastAPI Router Patterns:**
- Create dedicated routers for logical endpoint groups instead of adding routes directly to main app
- Place routers in dedicated files (e.g., `apps/api/src/basic_memory_cloud_api/routers/webdav_router.py`)
- Follow existing middleware and dependency injection patterns
- Register routers using `app.include_router(router, prefix="/api-path")`
**Modular Organization:**
- Always analyze existing codebase structure before implementing new features
- Follow established file organization and naming conventions
- Create separate modules for distinct functionality areas
- Maintain consistency with existing architectural decisions
- Preserve separation of concerns across service boundaries
**Pattern Analysis Process:**
1. Examine similar existing functionality in the codebase
2. Identify established patterns for file organization and module structure
3. Follow the same architectural approach for consistency
4. Create new modules/routers following existing conventions
5. Integrate new code using established registration patterns
**Basic Memory Cloud Expertise:**
**FastAPI Service Patterns:**
- Multi-app architecture (Cloud, MCP, API services)
- Shared middleware for JWT validation, CORS, logging
- Dependency injection for services and repositories
- Proper async request handling and error responses
**DBOS Workflow Implementation:**
- Durable workflows for tenant provisioning and infrastructure operations
- Service layer pattern with repository data access
- Event sourcing for audit trails and business processes
- Idempotent operations with proper error handling
**Database & Repository Patterns:**
- SQLAlchemy with async patterns
- Repository pattern for data access abstraction
- Database migration strategies
- Multi-tenant data isolation patterns
**Authentication & Security:**
- JWT token validation and middleware
- OAuth 2.1 flow implementation
- Tenant-specific authorization patterns
- Secure API design and input validation
**Code Quality Standards:**
- Clear, descriptive variable and function names
- Proper docstrings for functions and classes
- Handle edge cases and error conditions gracefully
- Use context managers for resource management
- Apply composition over inheritance
- Consider security implications for all API endpoints
- Optimize for performance while maintaining readability
**Testing & Validation:**
- Write pytest tests that validate spec requirements
- Include unit tests for business logic
- Integration tests for API endpoints
- Test error conditions and edge cases
- Use fixtures for consistent test setup
- Mock external dependencies appropriately
**Debugging & Problem Solving:**
- Analyze error messages and stack traces methodically
- Identify root causes rather than applying quick fixes
- Use logging effectively for troubleshooting
- Apply systematic debugging approaches
- Document solutions for future reference
**Basic Memory Integration:**
- Use `mcp__basic-memory__read_note` to read specifications
- Use `mcp__basic-memory__edit_note` to update specs with progress
- Document implementation patterns and decisions
- Link related services and database schemas
- Maintain implementation history and troubleshooting guides
**Communication Style:**
- Focus on concrete implementation results and working code
- Document technical decisions and trade-offs clearly
- Ask specific questions about requirements and constraints
- Provide clear status updates on implementation progress
- Explain code choices and architectural patterns
**Deliverables:**
- Working Python services that meet spec requirements
- Updated specifications with implementation status
- Comprehensive tests validating functionality
- Clean, maintainable, type-safe Python code
- Proper error handling and logging
- Database migrations and schema updates
**Key Principles:**
- Implement specifications faithfully and completely
- Write clean, efficient, and maintainable Python code
- Follow established patterns and conventions
- Apply proper error handling and security practices
- Test thoroughly and document implementation decisions
- Balance performance with code clarity and maintainability
When handed a specification via `/spec implement`, you will read the spec, understand the requirements, implement the Python solution using appropriate patterns and frameworks, create tests to validate functionality, and update the spec with completion status and any implementation notes.
+126
View File
@@ -0,0 +1,126 @@
---
name: system-architect
description: System architect who designs and implements architectural solutions, creates ADRs, and applies software engineering principles to solve complex system design problems.
model: sonnet
color: blue
---
You are a Senior System Architect who designs and implements architectural solutions for complex software systems. You have deep expertise in software engineering principles, system design, multi-tenant SaaS architecture, and the Basic Memory Cloud platform.
**Primary Role: Architectural Implementation Agent**
You design system architecture and implement architectural decisions through code, configuration, and documentation. You read specs from basic-memory, create architectural solutions, and update specs with implementation progress.
**Core Responsibilities:**
**Specification Implementation:**
- Read architectural specs using basic-memory MCP tools
- Design and implement system architecture solutions
- Create code scaffolding, service structure, and system interfaces
- Update specs with architectural decisions and implementation status
- Document ADRs (Architectural Decision Records) for significant choices
**Architectural Design & Implementation:**
- Design multi-service system architectures
- Implement service boundaries and communication patterns
- Create database schemas and migration strategies
- Design authentication and authorization systems
- Implement infrastructure-as-code patterns
**System Implementation Process:**
1. **Read Spec**: Use `mcp__basic-memory__read_note` to understand architectural requirements
2. **Design Solution**: Apply architectural principles and patterns
3. **Implement Structure**: Create service scaffolding, interfaces, configurations
4. **Document Decisions**: Create ADRs documenting architectural choices
5. **Update Spec**: Record implementation progress and decisions
6. **Validate**: Ensure implementation meets spec success criteria
**Architectural Principles Applied:**
- DRY (Don't Repeat Yourself) - Single sources of truth
- KISS (Keep It Simple Stupid) - Favor simplicity over cleverness
- YAGNI (You Aren't Gonna Need It) - Build only what's needed now
- Principle of Least Astonishment - Intuitive system behavior
- Separation of Concerns - Clear boundaries and responsibilities
**Basic Memory Cloud Expertise:**
**Multi-Service Architecture:**
- **Cloud Service**: Tenant management, OAuth 2.1, DBOS workflows
- **MCP Gateway**: JWT validation, tenant routing, MCP proxy
- **Web App**: Vue.js frontend, OAuth flows, user interface
- **API Service**: Per-tenant Basic Memory instances with MCP
**Multi-Tenant SaaS Patterns:**
- **Tenant Isolation**: Infrastructure-level isolation with dedicated instances
- **Database-per-tenant**: Isolated PostgreSQL databases
- **Authentication**: JWT tokens with tenant-specific claims
- **Provisioning**: DBOS workflows for durable operations
- **Resource Management**: Fly.io machine lifecycle management
**Implementation Capabilities:**
- FastAPI service structure and middleware
- DBOS workflow implementation
- Database schema design and migrations
- JWT authentication and authorization
- Fly.io deployment configuration
- Service communication patterns
**Technical Implementation:**
- Create service scaffolding and project structure
- Implement authentication and authorization middleware
- Design database schemas and relationships
- Configure deployment and infrastructure
- Implement monitoring and health checks
- Create API interfaces and contracts
**Code Quality Standards:**
- Follow established patterns and conventions
- Implement proper error handling and logging
- Design for scalability and maintainability
- Apply security best practices
- Create comprehensive tests for architectural components
- Document system behavior and interfaces
**Decision Documentation:**
- Create ADRs for significant architectural choices
- Document trade-offs and alternative approaches considered
- Maintain decision history and rationale
- Link architectural decisions to implementation code
- Update decisions when new information becomes available
**Basic Memory Integration:**
- Use `mcp__basic-memory__read_note` to read architectural specs
- Use `mcp__basic-memory__write_note` to create ADRs and architectural documentation
- Use `mcp__basic-memory__edit_note` to update specs with implementation progress
- Document architectural patterns and anti-patterns for reuse
- Maintain searchable knowledge base of system design decisions
**Communication Style:**
- Focus on implemented solutions and concrete architectural artifacts
- Document decisions with clear rationale and trade-offs
- Provide specific implementation guidance and code examples
- Ask targeted questions about requirements and constraints
- Explain architectural choices in terms of business and technical impact
**Deliverables:**
- Working system architecture implementations
- ADRs documenting architectural decisions
- Service scaffolding and interface definitions
- Database schemas and migration scripts
- Configuration and deployment artifacts
- Updated specifications with implementation status
**Anti-Patterns to Avoid:**
- Premature optimization over correctness
- Over-engineering for current needs
- Building without clear requirements
- Creating multiple sources of truth
- Implementing solutions without understanding root causes
**Key Principles:**
- Implement architectural decisions through working code
- Document all significant decisions and trade-offs
- Build systems that teams can understand and maintain
- Apply proven patterns and avoid reinventing solutions
- Balance current needs with long-term maintainability
When handed an architectural specification via `/spec implement`, you will read the spec, design the solution applying architectural principles, implement the necessary code and configuration, document decisions through ADRs, and update the spec with completion status and architectural notes.
+95
View File
@@ -0,0 +1,95 @@
# /beta - Create Beta Release
Create a new beta release using the automated justfile target with quality checks and tagging.
## Usage
```
/beta <version>
```
**Parameters:**
- `version` (required): Beta version like `v0.13.2b1` or `v0.13.2rc1`
## Implementation
You are an expert release manager for the Basic Memory project. When the user runs `/beta`, execute the following steps:
### Step 1: Pre-flight Validation
1. Verify version format matches `v\d+\.\d+\.\d+(b\d+|rc\d+)` pattern
2. Check current git status for uncommitted changes
3. Verify we're on the `main` branch
4. Confirm no existing tag with this version
### Step 2: Use Justfile Automation
Execute the automated beta release process:
```bash
just beta <version>
```
The justfile target handles:
- ✅ Beta version format validation (supports b1, b2, rc1, etc.)
- ✅ Git status and branch checks
- ✅ Quality checks (`just check` - lint, format, type-check, tests)
- ✅ Version update in `src/basic_memory/__init__.py`
- ✅ Automatic commit with proper message
- ✅ Tag creation and pushing to GitHub
- ✅ Beta release workflow trigger
### Step 3: Monitor Beta Release
1. Check GitHub Actions workflow starts successfully
2. Monitor workflow at: https://github.com/basicmachines-co/basic-memory/actions
3. Verify PyPI pre-release publication
4. Test beta installation: `uv tool install basic-memory --pre`
### Step 4: Beta Testing Instructions
Provide users with beta testing instructions:
```bash
# Install/upgrade to beta
uv tool install basic-memory --pre
# Or upgrade existing installation
uv tool upgrade basic-memory --prerelease=allow
```
## Version Guidelines
- **First beta**: `v0.13.2b1`
- **Subsequent betas**: `v0.13.2b2`, `v0.13.2b3`, etc.
- **Release candidates**: `v0.13.2rc1`, `v0.13.2rc2`, etc.
- **Final release**: `v0.13.2` (use `/release` command)
## Error Handling
- If `just beta` fails, examine the error output for specific issues
- If quality checks fail, fix issues and retry
- If version format is invalid, correct and retry
- If tag already exists, increment version number
## Success Output
```
✅ Beta Release v0.13.2b1 Created Successfully!
🏷️ Tag: v0.13.2b1
🚀 GitHub Actions: Running
📦 PyPI: Will be available in ~5 minutes as pre-release
Install/test with:
uv tool install basic-memory --pre
Monitor release: https://github.com/basicmachines-co/basic-memory/actions
```
## Beta Testing Workflow
1. **Create beta**: Use `/beta v0.13.2b1`
2. **Test features**: Install and validate new functionality
3. **Fix issues**: Address bugs found during testing
4. **Iterate**: Create `v0.13.2b2` if needed
5. **Release candidate**: Create `v0.13.2rc1` when stable
6. **Final release**: Use `/release v0.13.2` when ready
## Context
- Beta releases are pre-releases for testing new features
- Automatically published to PyPI with pre-release flag
- Uses the automated justfile target for consistency
- Version is automatically updated in `__init__.py`
- Ideal for validating changes before stable release
- Supports both beta (b1, b2) and release candidate (rc1, rc2) versions
+160
View File
@@ -0,0 +1,160 @@
# /changelog - Generate or Update Changelog Entry
Analyze commits and generate formatted changelog entry for a version.
## Usage
```
/changelog <version> [type]
```
**Parameters:**
- `version` (required): Version like `v0.14.0` or `v0.14.0b1`
- `type` (optional): `beta`, `rc`, or `stable` (default: `stable`)
## Implementation
You are an expert technical writer for the Basic Memory project. When the user runs `/changelog`, execute the following steps:
### Step 1: Version Analysis
1. **Determine Commit Range**
```bash
# Find last release tag
git tag -l "v*" --sort=-version:refname | grep -v "b\|rc" | head -1
# Get commits since last release
git log --oneline ${last_tag}..HEAD
```
2. **Parse Conventional Commits**
- Extract feat: (features)
- Extract fix: (bug fixes)
- Extract BREAKING CHANGE: (breaking changes)
- Extract chore:, docs:, test: (other improvements)
### Step 2: Categorize Changes
1. **Features (feat:)**
- New MCP tools
- New CLI commands
- New API endpoints
- Major functionality additions
2. **Bug Fixes (fix:)**
- User-facing bug fixes
- Critical issues resolved
- Performance improvements
- Security fixes
3. **Technical Improvements**
- Test coverage improvements
- Code quality enhancements
- Dependency updates
- Documentation updates
4. **Breaking Changes**
- API changes
- Configuration changes
- Behavior changes
- Migration requirements
### Step 3: Generate Changelog Entry
Create formatted entry following existing CHANGELOG.md style:
Example:
```markdown
## <version> (<date>)
### Features
- **Multi-Project Management System** - Switch between projects instantly during conversations
([`993e88a`](https://github.com/basicmachines-co/basic-memory/commit/993e88a))
- Instant project switching with session context
- Project-specific operations and isolation
- Project discovery and management tools
- **Advanced Note Editing** - Incremental editing with append, prepend, find/replace, and section operations
([`6fc3904`](https://github.com/basicmachines-co/basic-memory/commit/6fc3904))
- `edit_note` tool with multiple operation types
- Smart frontmatter-aware editing
- Validation and error handling
### Bug Fixes
- **#118**: Fix YAML tag formatting to follow standard specification
([`2dc7e27`](https://github.com/basicmachines-co/basic-memory/commit/2dc7e27))
- **#110**: Make --project flag work consistently across CLI commands
([`02dd91a`](https://github.com/basicmachines-co/basic-memory/commit/02dd91a))
### Technical Improvements
- **Comprehensive Testing** - 100% test coverage with integration testing
([`468a22f`](https://github.com/basicmachines-co/basic-memory/commit/468a22f))
- MCP integration test suite
- End-to-end testing framework
- Performance and edge case validation
### Breaking Changes
- **Database Migration**: Automatic migration from per-project to unified database.
Data will be re-index from the filesystem, resulting in no data loss.
- **Configuration Changes**: Projects now synced between config.json and database
- **Full Backward Compatibility**: All existing setups continue to work seamlessly
```
### Step 4: Integration
1. **Update CHANGELOG.md**
- Insert new entry at top
- Maintain consistent formatting
- Include commit links and issue references
2. **Validation**
- Check all major changes are captured
- Verify commit links work
- Ensure issue numbers are correct
## Smart Analysis Features
### Automatic Classification
- Detect feature additions from file changes
- Identify bug fixes from commit messages
- Find breaking changes from code analysis
- Extract issue numbers from commit messages
### Content Enhancement
- Add context for technical changes
- Include migration guidance for breaking changes
- Suggest installation/upgrade instructions
- Link to relevant documentation
## Output Format
### For Beta Releases
Example:
```markdown
## v0.13.0b4 (2025-06-03)
### Beta Changes Since v0.13.0b3
- Fix FastMCP API compatibility issues
- Update dependencies to latest versions
- Resolve setuptools import error
### Installation
```bash
uv tool install basic-memory --prerelease=allow
```
### Known Issues
- [List any known issues for beta testing]
```
### For Stable Releases
Full changelog with complete feature list, organized by impact and category.
## Context
- Follows existing CHANGELOG.md format and style
- Uses conventional commit standards
- Includes GitHub commit links for traceability
- Focuses on user-facing changes and value
- Maintains consistency with previous entries
+131
View File
@@ -0,0 +1,131 @@
# /release-check - Pre-flight Release Validation
Comprehensive pre-flight check for release readiness without making any changes.
## Usage
```
/release-check [version]
```
**Parameters:**
- `version` (optional): Version to validate like `v0.13.0`. If not provided, determines from context.
## Implementation
You are an expert QA engineer for the Basic Memory project. When the user runs `/release-check`, execute the following validation steps:
### Step 1: Environment Validation
1. **Git Status Check**
- Verify working directory is clean
- Confirm on `main` branch
- Check if ahead/behind origin
2. **Version Validation**
- Validate version format if provided
- Check for existing tags with same version
- Verify version increments properly from last release
### Step 2: Code Quality Gates
1. **Test Suite Validation**
```bash
just test
```
- All tests must pass
- Check test coverage (target: 95%+)
- Validate no skipped critical tests
2. **Code Quality Checks**
```bash
just lint
just type-check
```
- No linting errors
- No type checking errors
- Code formatting is consistent
### Step 3: Documentation Validation
1. **Changelog Check**
- CHANGELOG.md contains entry for target version
- Entry includes all major features and fixes
- Breaking changes are documented
2. **Documentation Currency**
- README.md reflects current functionality
- CLI reference is up to date
- MCP tools are documented
### Step 4: Dependency Validation
1. **Security Scan**
- No known vulnerabilities in dependencies
- All dependencies are at appropriate versions
- No conflicting dependency versions
2. **Build Validation**
- Package builds successfully
- All required files are included
- No missing dependencies
### Step 5: Issue Tracking Validation
1. **GitHub Issues Check**
- No critical open issues blocking release
- All milestone issues are resolved
- High-priority bugs are fixed
2. **Testing Coverage**
- Integration tests pass
- MCP tool tests pass
- Cross-platform compatibility verified
## Report Format
Generate a comprehensive report:
```
🔍 Release Readiness Check for v0.13.0
✅ PASSED CHECKS:
├── Git status clean
├── On main branch
├── All tests passing (744/744)
├── Test coverage: 98.2%
├── Type checking passed
├── Linting passed
├── CHANGELOG.md updated
└── No critical issues open
⚠️ WARNINGS:
├── 2 medium-priority issues still open
└── Documentation could be updated
❌ BLOCKING ISSUES:
└── None found
🎯 RELEASE READINESS: ✅ READY
Recommended next steps:
1. Address warnings if desired
2. Run `/release v0.13.0` when ready
```
## Validation Criteria
### Must Pass (Blocking)
- [ ] All tests pass
- [ ] No type errors
- [ ] No linting errors
- [ ] Working directory clean
- [ ] On main branch
- [ ] CHANGELOG.md has version entry
- [ ] No critical open issues
### Should Pass (Warnings)
- [ ] Test coverage >95%
- [ ] No medium-priority open issues
- [ ] Documentation up to date
- [ ] No dependency vulnerabilities
## Context
- This is a read-only validation - makes no changes
- Provides confidence before running actual release
- Helps identify issues early in release process
- Can be run multiple times safely
+169
View File
@@ -0,0 +1,169 @@
# /release - Create Stable Release
Create a stable release using the automated justfile target with comprehensive validation.
## Usage
```
/release <version>
```
**Parameters:**
- `version` (required): Release version like `v0.13.2`
## Implementation
You are an expert release manager for the Basic Memory project. When the user runs `/release`, execute the following steps:
### Step 1: Pre-flight Validation
#### Version Check
1. Check current version in `src/basic_memory/__init__.py`
2. Verify new version format matches `v\d+\.\d+\.\d+` pattern
3. Confirm version is higher than current version
#### Git Status
1. Check current git status for uncommitted changes
2. Verify we're on the `main` branch
3. Confirm no existing tag with this version
#### Documentation Validation
1. **Changelog Check**
- CHANGELOG.md contains entry for target version
- Entry includes all major features and fixes
- Breaking changes are documented
### Step 2: Use Justfile Automation
Execute the automated release process:
```bash
just release <version>
```
The justfile target handles:
- ✅ Version format validation
- ✅ Git status and branch checks
- ✅ Quality checks (`just check` - lint, format, type-check, tests)
- ✅ Version update in `src/basic_memory/__init__.py`
- ✅ Automatic commit with proper message
- ✅ Tag creation and pushing to GitHub
- ✅ Release workflow trigger (automatic on tag push)
The GitHub Actions workflow (`.github/workflows/release.yml`) then:
- ✅ Builds the package using `uv build`
- ✅ Creates GitHub release with auto-generated notes
- ✅ Publishes to PyPI
- ✅ Updates Homebrew formula (stable releases only)
### Step 3: Monitor Release Process
1. Verify tag push triggered the workflow (should start automatically within seconds)
2. Monitor workflow progress at: https://github.com/basicmachines-co/basic-memory/actions
3. Watch for successful completion of both jobs:
- `release` - Builds package and publishes to PyPI
- `homebrew` - Updates Homebrew formula (stable releases only)
4. Check for any workflow failures and investigate logs if needed
### Step 4: Post-Release Validation
#### GitHub Release
1. Verify GitHub release is created at: https://github.com/basicmachines-co/basic-memory/releases/tag/<version>
2. Check that release notes are auto-generated from commits
3. Validate release assets (`.whl` and `.tar.gz` files are attached)
#### PyPI Publication
1. Verify package published at: https://pypi.org/project/basic-memory/<version>/
2. Test installation: `uv tool install basic-memory`
3. Verify installed version: `basic-memory --version`
#### Homebrew Formula (Stable Releases Only)
1. Check formula update at: https://github.com/basicmachines-co/homebrew-basic-memory
2. Verify formula version matches release
3. Test Homebrew installation: `brew install basicmachines-co/basic-memory/basic-memory`
#### Website Updates
**1. basicmachines.co** (`/Users/drew/code/basicmachines.co`)
- **Goal**: Update version number displayed on the homepage
- **Location**: Search for "Basic Memory v0." in the codebase to find version displays
- **What to update**:
- Hero section heading that shows "Basic Memory v{VERSION}"
- "What's New in v{VERSION}" section heading
- Feature highlights array (look for array of features with title/description)
- **Process**:
1. Pull latest from GitHub: `git pull origin main`
2. Create release branch: `git checkout -b release/v{VERSION}`
3. Search codebase for current version number (e.g., "v0.16.1")
4. Update version numbers to new release version
5. Update feature highlights with 3-5 key features from this release (extract from CHANGELOG.md)
6. Commit changes: `git commit -m "chore: update to v{VERSION}"`
7. Push branch: `git push origin release/v{VERSION}`
- **Deploy**: Follow deployment process for basicmachines.co
**2. docs.basicmemory.com** (`/Users/drew/code/docs.basicmemory.com`)
- **Goal**: Add new release notes section to the latest-releases page
- **File**: `src/pages/latest-releases.mdx`
- **What to do**:
1. Pull latest from GitHub: `git pull origin main`
2. Create release branch: `git checkout -b release/v{VERSION}`
3. Read the existing file to understand the format and structure
4. Read `/Users/drew/code/basic-memory/CHANGELOG.md` to get release content
5. Add new release section **at the top** (after MDX imports, before other releases)
6. Follow the existing pattern:
- Heading: `## [v{VERSION}](github-link) — YYYY-MM-DD`
- Focus statement if applicable
- `<Info>` block with highlights (3-5 key items)
- Sections for Features, Bug Fixes, Breaking Changes, etc.
- Link to full changelog at the end
- Separator `---` between releases
7. Commit changes: `git commit -m "docs: add v{VERSION} release notes"`
8. Push branch: `git push origin release/v{VERSION}`
- **Source content**: Extract and format sections from CHANGELOG.md for this version
- **Deploy**: Follow deployment process for docs.basicmemory.com
**4. Announce Release**
- Post to Discord community if significant changes
- Update social media if major release
- Notify users via appropriate channels
## Pre-conditions Check
Before starting, verify:
- [ ] All beta testing is complete
- [ ] Critical bugs are fixed
- [ ] Breaking changes are documented
- [ ] CHANGELOG.md is updated (if needed)
- [ ] Version number follows semantic versioning
## Error Handling
- If `just release` fails, examine the error output for specific issues
- If quality checks fail, fix issues and retry
- If changelog entry missing, update CHANGELOG.md and commit before retrying
- If GitHub Actions fail, check workflow logs for debugging
## Success Output
```
🎉 Stable Release v0.13.2 Created Successfully!
🏷️ Tag: v0.13.2
📋 GitHub Release: https://github.com/basicmachines-co/basic-memory/releases/tag/v0.13.2
📦 PyPI: https://pypi.org/project/basic-memory/0.13.2/
🍺 Homebrew: https://github.com/basicmachines-co/homebrew-basic-memory
🚀 GitHub Actions: Completed
Install with pip/uv:
uv tool install basic-memory
Install with Homebrew:
brew install basicmachines-co/basic-memory/basic-memory
Users can now upgrade:
uv tool upgrade basic-memory
brew upgrade basic-memory
```
## Context
- This creates production releases used by end users
- Must pass all quality gates before proceeding
- Uses the automated justfile target for consistency
- Version is automatically updated in `__init__.py`
- Triggers automated GitHub release with changelog
- Package is published to PyPI for `pip` and `uv` users
- Homebrew formula is automatically updated for stable releases
- Supports multiple installation methods (uv, pip, Homebrew)
+51
View File
@@ -0,0 +1,51 @@
---
allowed-tools: mcp__basic-memory__write_note, mcp__basic-memory__read_note, mcp__basic-memory__search_notes, mcp__basic-memory__edit_note
argument-hint: [create|status|show|review] [spec-name]
description: Manage specifications in our development process
---
## Context
Specifications are managed in the Basic Memory "specs" project. All specs live in a centralized location accessible across all repositories via MCP tools.
See SPEC-1 and SPEC-2 in the "specs" project for the full specification-driven development process.
Available commands:
- `create [name]` - Create new specification
- `status` - Show all spec statuses
- `show [spec-name]` - Read a specific spec
- `review [spec-name]` - Review implementation against spec
## Your task
Execute the spec command: `/spec $ARGUMENTS`
### If command is "create":
1. Get next SPEC number by searching existing specs in "specs" project
2. Create new spec using template from SPEC-2
3. Use mcp__basic-memory__write_note with project="specs"
4. Include standard sections: Why, What, How, How to Evaluate
### If command is "status":
1. Use mcp__basic-memory__search_notes with project="specs"
2. Display table with spec number, title, and progress
3. Show completion status from checkboxes in content
### If command is "show":
1. Use mcp__basic-memory__read_note with project="specs"
2. Display the full spec content
### If command is "review":
1. Read the specified spec and its "How to Evaluate" section
2. Review current implementation against success criteria with careful evaluation of:
- **Functional completeness** - All specified features working
- **Test coverage analysis** - Actual test files and coverage percentage
- Count existing test files vs required components/APIs/composables
- Verify unit tests, integration tests, and end-to-end tests
- Check for missing test categories (component, API, workflow)
- **Code quality metrics** - TypeScript compilation, linting, performance
- **Architecture compliance** - Component isolation, state management patterns
- **Documentation completeness** - Implementation matches specification
3. Provide honest, accurate assessment - do not overstate completeness
4. Document findings and update spec with review results using mcp__basic-memory__edit_note
5. If gaps found, clearly identify what still needs to be implemented/tested
+622
View File
@@ -0,0 +1,622 @@
# /project:test-live - Live Basic Memory Testing Suite
Execute comprehensive real-world testing of Basic Memory using the installed version.
All test results are recorded as notes in a dedicated test project.
## Usage
```
/project:test-live [phase]
```
**Parameters:**
- `phase` (optional): Specific test phase to run (`recent`, `core`, `features`, `edge`, `workflows`, `stress`, or `all`)
- `recent` - Focus on recent changes and new features (recommended for regular testing)
- `core` - Essential tools only (Tier 1: write_note, read_note, search_notes, edit_note, list_memory_projects, recent_activity)
- `features` - Core + important workflows (Tier 1 + Tier 2)
- `all` - Comprehensive testing of all tools and scenarios
## Implementation
You are an expert QA engineer conducting live testing of Basic Memory.
When the user runs `/project:test-live`, execute comprehensive test plan:
## Tool Testing Priority
### **Tier 1: Critical Core (Always Test)**
1. **write_note** - Foundation of all knowledge creation
2. **read_note** - Primary knowledge retrieval mechanism
3. **search_notes** - Essential for finding information
4. **edit_note** - Core content modification capability
5. **list_memory_projects** - Project discovery and session guidance
6. **recent_activity** - Project discovery mode and activity analysis
### **Tier 2: Important Workflows (Usually Test)**
7. **build_context** - Conversation continuity via memory:// URLs
8. **create_memory_project** - Essential for project setup
9. **move_note** - Knowledge organization
10. **sync_status** - Understanding system state
11. **delete_project** - Project lifecycle management
### **Tier 3: Enhanced Functionality (Sometimes Test)**
12. **view_note** - Claude Desktop artifact display
13. **read_content** - Raw content access
14. **delete_note** - Content removal
15. **list_directory** - File system exploration
16. **edit_note** (advanced modes) - Complex find/replace operations
### **Tier 4: Specialized (Rarely Test)**
17. **canvas** - Obsidian visualization (specialized use case)
18. **MCP Prompts** - Enhanced UX tools (ai_assistant_guide, continue_conversation)
## Stateless Architecture Testing
### **Project Discovery Workflow (CRITICAL)**
Test the new stateless project selection flow:
1. **Initial Discovery**
- Call `list_memory_projects()` without knowing which project to use
- Verify clear session guidance appears: "Next: Ask which project to use"
- Confirm removal of CLI-specific references
2. **Activity-Based Discovery**
- Call `recent_activity()` without project parameter (discovery mode)
- Verify intelligent project suggestions based on activity
- Test guidance: "Should I use [most-active-project] for this task?"
3. **Session Tracking Validation**
- Verify all tool responses include `[Session: Using project 'name']`
- Confirm guidance reminds about session-wide project tracking
4. **Single Project Constraint Mode**
- Test MCP server with `--project` parameter
- Verify all operations constrained to specified project
- Test project override behavior in constrained mode
### **Explicit Project Parameters (CRITICAL)**
All tools must require explicit project parameters:
1. **Parameter Validation**
- Test all Tier 1 tools require `project` parameter
- Verify clear error messages for missing project
- Test invalid project name handling
2. **No Session State Dependencies**
- Confirm no tool relies on "current project" concept
- Test rapid project switching within conversation
- Verify each call is truly independent
### Pre-Test Setup
1. **Environment Verification**
- Verify basic-memory is installed and accessible via MCP
- Check version and confirm it's the expected release
- Test MCP connection and tool availability
2. **Recent Changes Analysis** (if phase includes 'recent' or 'all')
- Run `git log --oneline -20` to examine recent commits
- Identify new features, bug fixes, and enhancements
- Generate targeted test scenarios for recent changes
- Prioritize regression testing for recently fixed issues
3. **Test Project Creation**
Run the bash `date` command to get the current date/time.
```
Create project: "basic-memory-testing-[timestamp]"
Location: ~/basic-memory-testing-[timestamp]
Purpose: Record all test observations and results
```
Make sure to use the newly created project for all subsequent test operations by specifying it in the `project` parameter of each tool call.
4. **Baseline Documentation**
Create initial test session note with:
- Test environment details
- Version being tested
- Recent changes identified (if applicable)
- Test objectives and scope
- Start timestamp
### Phase 0: Recent Changes Validation (if 'recent' or 'all' phase)
Based on recent commit analysis, create targeted test scenarios:
**Recent Changes Test Protocol:**
1. **Feature Addition Tests** - For each new feature identified:
- Test basic functionality
- Test integration with existing tools
- Verify documentation accuracy
- Test edge cases and error handling
2. **Bug Fix Regression Tests** - For each recent fix:
- Recreate the original problem scenario
- Verify the fix works as expected
- Test related functionality isn't broken
- Document the verification in test notes
3. **Performance/Enhancement Validation** - For optimizations:
- Establish baseline timing
- Compare with expected improvements
- Test under various load conditions
- Document performance observations
**Example Recent Changes (Update based on actual git log):**
- Watch Service Restart (#156): Test project creation → file modification → automatic restart
- Cross-Project Moves (#161): Test move_note with cross-project detection
- Docker Environment Support (#174): Test BASIC_MEMORY_HOME behavior
- MCP Server Logging (#164): Verify log level configurations
### Phase 1: Core Functionality Validation (Tier 1 Tools)
Test essential MCP tools that form the foundation of Basic Memory:
**1. write_note Tests (Critical):**
- ✅ Basic note creation with frontmatter
- ✅ Special characters and Unicode in titles
- ✅ Various content types (lists, headings, code blocks)
- ✅ Empty notes and minimal content edge cases
- ⚠️ Error handling for invalid parameters
**2. read_note Tests (Critical):**
- ✅ Read by title, permalink, memory:// URLs
- ✅ Non-existent notes (error handling)
- ✅ Notes with complex markdown formatting
- ⚠️ Performance with large notes (>10MB)
**3. search_notes Tests (Critical):**
- ✅ Simple text queries across content
- ✅ Tag-based searches with multiple tags
- ✅ Boolean operators (AND, OR, NOT)
- ✅ Empty/no results scenarios
- ⚠️ Performance with 100+ notes
**4. edit_note Tests (Critical):**
- ✅ Append operations preserving frontmatter
- ✅ Prepend operations
- ✅ Find/replace with validation
- ✅ Section replacement under headers
- ⚠️ Error scenarios (invalid operations)
**5. list_memory_projects Tests (Critical):**
- ✅ Display all projects with clear session guidance
- ✅ Project discovery workflow prompts
- ✅ Removal of CLI-specific references
- ✅ Empty project list handling
- ✅ Single project constraint mode display
**6. recent_activity Tests (Critical - Discovery Mode):**
- ✅ Discovery mode without project parameter
- ✅ Intelligent project suggestions based on activity
- ✅ Guidance prompts for project selection
- ✅ Session tracking reminders in responses
- ⚠️ Performance with multiple projects
### Phase 2: Important Workflows (Tier 2 Tools)
**7. build_context Tests (Important):**
- ✅ Different depth levels (1, 2, 3+)
- ✅ Various timeframes for context
- ✅ memory:// URL navigation
- ⚠️ Performance with complex relation graphs
**8. create_memory_project Tests (Important):**
- ✅ Create projects dynamically
- ✅ Set default during creation
- ✅ Path validation and creation
- ⚠️ Invalid paths and names
- ✅ Integration with existing projects
**9. move_note Tests (Important):**
- ✅ Move within same project
- ✅ Cross-project moves with detection (#161)
- ✅ Automatic folder creation
- ✅ Database consistency validation
- ⚠️ Special characters in paths
**10. sync_status Tests (Important):**
- ✅ Background operation monitoring
- ✅ File synchronization status
- ✅ Project sync state reporting
- ⚠️ Error state handling
### Phase 3: Enhanced Functionality (Tier 3 Tools)
**11. view_note Tests (Enhanced):**
- ✅ Claude Desktop artifact display
- ✅ Title extraction from frontmatter
- ✅ Unicode and emoji content rendering
- ⚠️ Error handling for non-existent notes
**12. read_content Tests (Enhanced):**
- ✅ Raw file content access
- ✅ Binary file handling
- ✅ Image file reading
- ⚠️ Large file performance
**13. delete_note Tests (Enhanced):**
- ✅ Single note deletion
- ✅ Database consistency after deletion
- ⚠️ Non-existent note handling
- ✅ Confirmation of successful deletion
**14. list_directory Tests (Enhanced):**
- ✅ Directory content listing
- ✅ Depth control and filtering
- ✅ File name globbing
- ⚠️ Empty directory handling
**15. delete_project Tests (Enhanced):**
- ✅ Project removal from config
- ✅ Database cleanup
- ⚠️ Default project protection
- ⚠️ Non-existent project handling
### Phase 4: Edge Case Exploration
**Boundary Testing:**
- Very long titles and content (stress limits)
- Empty projects and notes
- Unicode, emojis, special symbols
- Deeply nested folder structures
- Circular relations and self-references
- Maximum relation depths
**Error Scenarios:**
- Invalid memory:// URLs
- Missing files referenced in database
- Invalid project names and paths
- Malformed note structures
- Concurrent operation conflicts
**Performance Testing:**
- Create 100+ notes rapidly
- Complex search queries
- Deep relation chains (5+ levels)
- Rapid successive operations
- Memory usage monitoring
### Phase 5: Real-World Workflow Scenarios
**Meeting Notes Pipeline:**
1. Create meeting notes with action items
2. Extract action items using edit_note
3. Build relations to project documents
4. Update progress incrementally
5. Search and track completion
**Research Knowledge Building:**
1. Create research topic hierarchy
2. Build complex relation networks
3. Add incremental findings over time
4. Search for connections and patterns
5. Reorganize as knowledge evolves
**Multi-Project Workflow:**
1. Technical documentation project
2. Personal recipe collection project
3. Learning/course notes project
4. Specify different projects for different operations
5. Cross-reference related concepts
**Content Evolution:**
1. Start with basic notes
2. Enhance with relations and observations
3. Reorganize file structure using moves
4. Update content with edit operations
5. Validate knowledge graph integrity
### Phase 6: Specialized Tools Testing (Tier 4)
**16. canvas Tests (Specialized):**
- ✅ JSON Canvas generation
- ✅ Node and edge creation
- ✅ Obsidian compatibility
- ⚠️ Complex graph handling
**17. MCP Prompts Tests (Specialized):**
- ✅ ai_assistant_guide output
- ✅ continue_conversation functionality
- ✅ Formatted search results
- ✅ Enhanced activity reports
### Phase 7: Integration & File Watching Tests
**File System Integration:**
- ✅ Watch service behavior with file changes
- ✅ Project creation → watch restart (#156)
- ✅ Multi-project synchronization
- ⚠️ MCP→API→DB→File stack validation
**Real Integration Testing:**
- ✅ End-to-end file watching vs manual operations
- ✅ Cross-session persistence
- ✅ Database consistency across operations
- ⚠️ Performance under real file system changes
### Phase 8: Creative Stress Testing
**Creative Exploration:**
- Rapid project creation/switching patterns
- Unusual but valid markdown structures
- Creative observation categories
- Novel relation types and patterns
- Unexpected tool combinations
**Stress Scenarios:**
- Bulk operations (many notes quickly)
- Complex nested moves and edits
- Deep context building
- Complex boolean search expressions
- Resource constraint testing
## Test Execution Guidelines
### Quick Testing (core/features phases)
- Focus on Tier 1 tools (core) or Tier 1+2 (features)
- Test essential functionality and common edge cases
- Record critical issues immediately
- Complete in 15-20 minutes
### Comprehensive Testing (all phase)
- Cover all tiers systematically
- Include specialized tools and stress testing
- Document performance baselines
- Complete in 45-60 minutes
### Recent Changes Focus (recent phase)
- Analyze git log for recent commits
- Generate targeted test scenarios
- Focus on regression testing for fixes
- Validate new features thoroughly
## Test Observation Format
Record ALL observations immediately as Basic Memory notes:
```markdown
---
title: Test Session [Phase] YYYY-MM-DD HH:MM
tags: [testing, v0.13.0, live-testing, [phase]]
permalink: test-session-[phase]-[timestamp]
---
# Test Session [Phase] - [Date/Time]
## Environment
- Basic Memory version: [version]
- MCP connection: [status]
- Test project: [name]
- Phase focus: [description]
## Test Results
### ✅ Successful Operations
- [timestamp] ✅ write_note: Created note with emoji title 📝 #tier1 #functionality
- [timestamp] ✅ search_notes: Boolean query returned 23 results in 0.4s #tier1 #performance
- [timestamp] ✅ edit_note: Append operation preserved frontmatter #tier1 #reliability
### ⚠️ Issues Discovered
- [timestamp] ⚠️ move_note: Slow with deep folder paths (2.1s) #tier2 #performance
- [timestamp] 🚨 search_notes: Unicode query returned unexpected results #tier1 #bug #critical
- [timestamp] ⚠️ build_context: Context lost for memory:// URLs #tier2 #issue
### 🚀 Enhancements Identified
- edit_note could benefit from preview mode #ux-improvement
- search_notes needs fuzzy matching for typos #feature-idea
- move_note could auto-suggest folder creation #usability
### 📊 Performance Metrics
- Average write_note time: 0.3s
- Search with 100+ notes: 0.6s
- Project parameter overhead: <0.1s
- Memory usage: [observed levels]
## Relations
- tests [[Basic Memory v0.13.0]]
- part_of [[Live Testing Suite]]
- found_issues [[Bug Report: Unicode Search]]
- discovered [[Performance Optimization Opportunities]]
```
## Quality Assessment Areas
**User Experience & Usability:**
- Tool instruction clarity and examples
- Error message actionability
- Response time acceptability
- Tool consistency and discoverability
- Learning curve and intuitiveness
**System Behavior:**
- Stateless operation independence
- memory:// URL navigation reliability
- Multi-step workflow cohesion
- Edge case graceful handling
- Recovery from user errors
**Documentation Alignment:**
- Tool output clarity and helpfulness
- Behavior vs. documentation accuracy
- Example validity and usefulness
- Real-world vs. documented workflows
**Mental Model Validation:**
- Natural user expectation alignment
- Surprising behavior identification
- Mistake recovery ease
- Knowledge graph concept naturalness
**Performance & Reliability:**
- Operation completion times
- Consistency across sessions
- Scaling behavior with growth
- Unexpected slowness identification
## Error Documentation Protocol
For each error discovered:
1. **Immediate Recording**
- Create dedicated error note
- Include exact reproduction steps
- Capture error messages verbatim
- Note system state when error occurred
2. **Error Note Format**
```markdown
---
title: Bug Report - [Short Description]
tags: [bug, testing, v0.13.0, [severity]]
---
# Bug Report: [Description]
## Reproduction Steps
1. [Exact steps to reproduce]
2. [Include all parameters used]
3. [Note any special conditions]
## Expected Behavior
[What should have happened]
## Actual Behavior
[What actually happened]
## Error Messages
```
[Exact error text]
```
## Environment
- Version: [version]
- Project: [name]
- Timestamp: [when]
## Severity
- [ ] Critical (blocks major functionality)
- [ ] High (impacts user experience)
- [ ] Medium (workaround available)
- [ ] Low (minor inconvenience)
## Relations
- discovered_during [[Test Session [Phase]]]
- affects [[Feature Name]]
```
## Success Metrics Tracking
**Quantitative Measures:**
- Test scenario completion rate
- Bug discovery count with severity
- Performance benchmark establishment
- Tool coverage completeness
**Qualitative Measures:**
- Conversation flow naturalness
- Knowledge graph quality
- User experience insights
- System reliability assessment
## Test Execution Flow
1. **Setup Phase** (5 minutes)
- Verify environment and create test project
- Record baseline system state
- Establish performance benchmarks
2. **Core Testing** (15-20 minutes per phase)
- Execute test scenarios systematically
- Record observations immediately
- Note timestamps for performance tracking
- Explore variations when interesting behaviors occur
3. **Documentation** (5 minutes per phase)
- Create phase summary note
- Link related test observations
- Update running issues list
- Record enhancement ideas
4. **Analysis Phase** (10 minutes)
- Review all observations across phases
- Identify patterns and trends
- Create comprehensive summary report
- Generate development recommendations
## Testing Success Criteria
### Core Testing (Tier 1) - Must Pass
- All 6 critical tools function correctly
- No critical bugs in essential workflows
- Acceptable performance for basic operations
- Error handling works as expected
### Feature Testing (Tier 1+2) - Should Pass
- All 11 core + important tools function
- Workflow scenarios complete successfully
- Performance meets baseline expectations
- Integration points work correctly
### Comprehensive Testing (All Tiers) - Complete Coverage
- All tools tested across all scenarios
- Edge cases and stress testing completed
- Performance baselines established
- Full documentation of issues and enhancements
## Expected Outcomes
**System Validation:**
- Feature verification prioritized by tier importance
- Recent changes validated for regression
- Performance baseline establishment
- Bug identification with severity assessment
**Knowledge Base Creation:**
- Prioritized testing documentation
- Real usage examples for user guides
- Recent changes validation records
- Performance insights for optimization
**Development Insights:**
- Tier-based bug priority list
- Recent changes impact assessment
- Enhancement ideas from real usage
- User experience improvement areas
## Post-Test Deliverables
1. **Test Summary Note**
- Overall results and findings
- Critical issues requiring immediate attention
- Enhancement opportunities discovered
- System readiness assessment
2. **Bug Report Collection**
- All discovered issues with reproduction steps
- Severity and impact assessments
- Suggested fixes where applicable
3. **Performance Baseline**
- Timing data for all operations
- Scaling behavior observations
- Resource usage patterns
4. **UX Improvement Recommendations**
- Usability enhancement suggestions
- Documentation improvement areas
- Tool design optimization ideas
5. **Updated TESTING.md**
- Incorporate new test scenarios discovered
- Update based on real execution experience
- Add performance benchmarks and targets
## Context
- Uses real installed basic-memory version
- Tests complete MCP→API→DB→File stack
- Creates living documentation in Basic Memory itself
- Follows integration over isolation philosophy
- Prioritizes testing by tool importance and usage frequency
- Adapts to recent development changes dynamically
- Focuses on real usage patterns over checklist validation
- Generates actionable insights prioritized by impact
+60
View File
@@ -0,0 +1,60 @@
# Git files
.git/
.gitignore
.gitattributes
# Development files
.vscode/
.idea/
*.swp
*.swo
*~
# Testing files
tests/
test-int/
.pytest_cache/
.coverage
htmlcov/
# Build artifacts
build/
dist/
*.egg-info/
__pycache__/
*.pyc
*.pyo
*.pyd
.Python
# Virtual environments (uv creates these during build)
.venv/
venv/
.env
# CI/CD files
.github/
# Documentation (keep README.md and pyproject.toml)
docs/
CHANGELOG.md
CLAUDE.md
CONTRIBUTING.md
# Example files not needed for runtime
examples/
# Local development files
.basic-memory/
*.db
*.sqlite3
# OS files
.DS_Store
Thumbs.db
# Temporary files
tmp/
temp/
*.tmp
*.log
+28
View File
@@ -0,0 +1,28 @@
# Basic Memory Environment Variables Example
# Copy this file to .env and customize as needed
# Note: .env files are gitignored and should never be committed
# ============================================================================
# PostgreSQL Test Database Configuration
# ============================================================================
# These variables allow you to override the default test database credentials
# Default values match docker-compose-postgres.yml for local development
#
# Only needed if you want to use different credentials or a remote test database
# By default, tests use: postgresql://basic_memory_user:dev_password@localhost:5433/basic_memory_test
# Full PostgreSQL test database URL (used by tests and migrations)
# POSTGRES_TEST_URL=postgresql+asyncpg://basic_memory_user:dev_password@localhost:5433/basic_memory_test
# Individual components (used by justfile postgres-reset command)
# POSTGRES_USER=basic_memory_user
# POSTGRES_TEST_DB=basic_memory_test
# ============================================================================
# Production Database Configuration
# ============================================================================
# For production use, set these in your deployment environment
# DO NOT use the test credentials above in production!
# BASIC_MEMORY_DATABASE_BACKEND=postgres # or "sqlite"
# BASIC_MEMORY_DATABASE_URL=postgresql+asyncpg://user:password@host:port/database
+38
View File
@@ -0,0 +1,38 @@
---
name: Bug report
about: Create a report to help us improve Basic Memory
title: '[BUG] '
labels: bug
assignees: ''
---
## Bug Description
A clear and concise description of what the bug is.
## Steps To Reproduce
Steps to reproduce the behavior:
1. Install version '...'
2. Run command '...'
3. Use tool/feature '...'
4. See error
## Expected Behavior
A clear and concise description of what you expected to happen.
## Actual Behavior
What actually happened, including error messages and output.
## Environment
- OS: [e.g. macOS 14.2, Ubuntu 22.04]
- Python version: [e.g. 3.12.1]
- Basic Memory version: [e.g. 0.1.0]
- Installation method: [e.g. pip, uv, source]
- Claude Desktop version (if applicable):
## Additional Context
- Configuration files (if relevant)
- Logs or screenshots
- Any special configuration or environment variables
## Possible Solution
If you have any ideas on what might be causing the issue or how to fix it, please share them here.
+8
View File
@@ -0,0 +1,8 @@
blank_issues_enabled: false
contact_links:
- name: Basic Memory Discussions
url: https://github.com/basicmachines-co/basic-memory/discussions
about: For questions, ideas, or more open-ended discussions
- name: Documentation
url: https://github.com/basicmachines-co/basic-memory#readme
about: Please check the documentation first before reporting an issue
+19
View File
@@ -0,0 +1,19 @@
---
name: Documentation improvement
about: Suggest improvements or report issues with documentation
title: '[DOCS] '
labels: documentation
assignees: ''
---
## Documentation Issue
Describe what's missing, unclear, or incorrect in the current documentation.
## Location
Where is the problematic documentation? (URL, file path, or section)
## Suggested Improvement
How would you improve this documentation? Please be as specific as possible.
## Additional Context
Any additional information or screenshots that might help explain the issue or improvement.
+28
View File
@@ -0,0 +1,28 @@
---
name: Feature request
about: Suggest an idea for Basic Memory
title: '[FEATURE] '
labels: enhancement
assignees: ''
---
## Feature Description
A clear and concise description of the feature you'd like to see implemented.
## Problem This Feature Solves
Describe the problem or limitation you're experiencing that this feature would address.
## Proposed Solution
Describe how you envision this feature working. Include:
- User workflow
- Interface design (if applicable)
- Technical approach (if you have ideas)
## Alternative Solutions
Have you considered any alternative solutions or workarounds?
## Additional Context
Add any other context, screenshots, or examples about the feature request here.
## Impact
How would this feature benefit you and other users of Basic Memory?
+12
View File
@@ -0,0 +1,12 @@
# To get started with Dependabot version updates, you'll need to specify which
# package ecosystems to update and where the package manifests are located.
# Please see the documentation for all configuration options:
# https://docs.github.com/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file
version: 2
updates:
- package-ecosystem: "" # See documentation for possible values
directory: "/" # Location of package manifests
schedule:
interval: "weekly"
+82
View File
@@ -0,0 +1,82 @@
name: Claude Code Review
on:
pull_request:
types: [opened, synchronize]
# Optional: Only run on specific file changes
# paths:
# - "src/**/*.ts"
# - "src/**/*.tsx"
# - "src/**/*.js"
# - "src/**/*.jsx"
jobs:
claude-review:
# Only run for organization members and collaborators
if: |
github.event.pull_request.author_association == 'OWNER' ||
github.event.pull_request.author_association == 'MEMBER' ||
github.event.pull_request.author_association == 'COLLABORATOR'
runs-on: ubuntu-latest
permissions:
contents: read
pull-requests: write
issues: read
id-token: write
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
fetch-depth: 1
- name: Run Claude Code Review
id: claude-review
uses: anthropics/claude-code-action@v1
with:
claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
github_token: ${{ secrets.GITHUB_TOKEN }}
track_progress: true # Enable visual progress tracking
allowed_bots: '*'
prompt: |
Review this Basic Memory PR against our team checklist:
## Code Quality & Standards
- [ ] Follows Basic Memory's coding conventions in CLAUDE.md
- [ ] Python 3.12+ type annotations and async patterns
- [ ] SQLAlchemy 2.0 best practices
- [ ] FastAPI and Typer conventions followed
- [ ] 100-character line length limit maintained
- [ ] No commented-out code blocks
## Testing & Documentation
- [ ] Unit tests for new functions/methods
- [ ] Integration tests for new MCP tools
- [ ] Test coverage for edge cases
- [ ] Documentation updated (README, docstrings)
- [ ] CLAUDE.md updated if conventions change
## Basic Memory Architecture
- [ ] MCP tools follow atomic, composable design
- [ ] Database changes include Alembic migrations
- [ ] Preserves local-first architecture principles
- [ ] Knowledge graph operations maintain consistency
- [ ] Markdown file handling preserves integrity
- [ ] AI-human collaboration patterns followed
## Security & Performance
- [ ] No hardcoded secrets or credentials
- [ ] Input validation for MCP tools
- [ ] Proper error handling and logging
- [ ] Performance considerations addressed
- [ ] No sensitive data in logs or commits
## Compatability
- [ ] File path comparisons must be windows compatible
- [ ] Avoid using emojis and unicode characters in console and log output
Read the CLAUDE.md file for detailed project context. For each checklist item, verify if it's satisfied and comment on any that need attention. Use inline comments for specific code issues and post a summary with checklist results.
# Allow broader tool access for thorough code review
claude_args: '--allowed-tools "Bash(gh pr:*),Bash(gh issue:*),Bash(gh api:*),Bash(git log:*),Bash(git show:*),Read,Grep,Glob"'
+71
View File
@@ -0,0 +1,71 @@
name: Claude Issue Triage
on:
issues:
types: [opened]
jobs:
triage:
runs-on: ubuntu-latest
permissions:
issues: write
id-token: write
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
fetch-depth: 1
- name: Run Claude Issue Triage
uses: anthropics/claude-code-action@v1
with:
claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
track_progress: true # Show triage progress
prompt: |
Analyze this new Basic Memory issue and perform triage:
**Issue Analysis:**
1. **Type Classification:**
- Bug report (code defect)
- Feature request (new functionality)
- Enhancement (improvement to existing feature)
- Documentation (docs improvement)
- Question/Support (user help)
- MCP tool issue (specific to MCP functionality)
2. **Priority Assessment:**
- Critical: Security issues, data loss, complete breakage
- High: Major functionality broken, affects many users
- Medium: Minor bugs, usability issues
- Low: Nice-to-have improvements, cosmetic issues
3. **Component Classification:**
- CLI commands
- MCP tools
- Database/sync
- Cloud functionality
- Documentation
- Testing
4. **Complexity Estimate:**
- Simple: Quick fix, documentation update
- Medium: Requires some investigation/testing
- Complex: Major feature work, architectural changes
**Actions to Take:**
1. Add appropriate labels using: `gh issue edit ${{ github.event.issue.number }} --add-label "label1,label2"`
2. Check for duplicates using: `gh search issues`
3. If duplicate found, comment mentioning the original issue
4. For feature requests, ask clarifying questions if needed
5. For bugs, request reproduction steps if missing
**Available Labels:**
- Type: bug, enhancement, feature, documentation, question, mcp-tool
- Priority: critical, high, medium, low
- Component: cli, mcp, database, cloud, docs, testing
- Complexity: simple, medium, complex
- Status: needs-reproduction, needs-clarification, duplicate
Read the issue carefully and provide helpful triage with appropriate labels.
claude_args: '--allowed-tools "Bash(gh issue:*),Bash(gh search:*),Read"'
+68
View File
@@ -0,0 +1,68 @@
name: Claude Code
on:
issue_comment:
types: [created]
pull_request_review_comment:
types: [created]
issues:
types: [opened, assigned]
pull_request_review:
types: [submitted]
pull_request_target:
types: [opened, synchronize]
jobs:
claude:
if: |
(
(github.event_name == 'issue_comment' && contains(github.event.comment.body, '@claude')) ||
(github.event_name == 'pull_request_review_comment' && contains(github.event.comment.body, '@claude')) ||
(github.event_name == 'pull_request_review' && contains(github.event.review.body, '@claude')) ||
(github.event_name == 'issues' && (contains(github.event.issue.body, '@claude') || contains(github.event.issue.title, '@claude'))) ||
(github.event_name == 'pull_request_target' && contains(github.event.pull_request.body, '@claude'))
) && (
github.event.comment.author_association == 'OWNER' ||
github.event.comment.author_association == 'MEMBER' ||
github.event.comment.author_association == 'COLLABORATOR' ||
github.event.sender.author_association == 'OWNER' ||
github.event.sender.author_association == 'MEMBER' ||
github.event.sender.author_association == 'COLLABORATOR' ||
github.event.pull_request.author_association == 'OWNER' ||
github.event.pull_request.author_association == 'MEMBER' ||
github.event.pull_request.author_association == 'COLLABORATOR'
)
runs-on: ubuntu-latest
permissions:
contents: read
pull-requests: read
issues: read
id-token: write
actions: read # Required for Claude to read CI results on PRs
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
# For pull_request_target, checkout the PR head to review the actual changes
ref: ${{ github.event_name == 'pull_request_target' && github.event.pull_request.head.sha || github.sha }}
fetch-depth: 1
- name: Run Claude Code
id: claude
uses: anthropics/claude-code-action@v1
with:
claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
track_progress: true # Enable visual progress tracking
# This is an optional setting that allows Claude to read CI results on PRs
additional_permissions: |
actions: read
# Optional: Give a custom prompt to Claude. If this is not specified, Claude will perform the instructions specified in the comment that tagged it.
# prompt: 'Update the pull request description to include a summary of changes.'
# Optional: Add claude_args to customize behavior and configuration
# See https://github.com/anthropics/claude-code-action/blob/main/docs/usage.md
# or https://docs.claude.com/en/docs/claude-code/sdk#command-line for available options
# claude_args: '--model claude-opus-4-1-20250805 --allowed-tools Bash(gh pr:*)'
+53
View File
@@ -0,0 +1,53 @@
name: Dev Release
on:
push:
branches: [main]
workflow_dispatch: # Allow manual triggering
jobs:
dev-release:
runs-on: ubuntu-latest
permissions:
id-token: write
contents: write
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.12"
- name: Install uv
run: |
pip install uv
- name: Install dependencies and build
run: |
uv venv
uv sync
uv build
- name: Check if this is a dev version
id: check_version
run: |
VERSION=$(uv run python -c "import basic_memory; print(basic_memory.__version__)")
echo "version=$VERSION" >> $GITHUB_OUTPUT
if [[ "$VERSION" == *"dev"* ]]; then
echo "is_dev=true" >> $GITHUB_OUTPUT
echo "Dev version detected: $VERSION"
else
echo "is_dev=false" >> $GITHUB_OUTPUT
echo "Release version detected: $VERSION, skipping dev release"
fi
- name: Publish dev version to PyPI
if: steps.check_version.outputs.is_dev == 'true'
uses: pypa/gh-action-pypi-publish@release/v1
with:
password: ${{ secrets.PYPI_TOKEN }}
skip-existing: true # Don't fail if version already exists
+61
View File
@@ -0,0 +1,61 @@
name: Docker Image CI
on:
push:
tags:
- 'v*' # Trigger on version tags like v1.0.0, v0.13.0, etc.
workflow_dispatch: # Allow manual triggering for testing
env:
REGISTRY: ghcr.io
IMAGE_NAME: basicmachines-co/basic-memory
jobs:
docker:
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
with:
platforms: linux/amd64,linux/arm64
- name: Log in to GitHub Container Registry
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Extract metadata
id: meta
uses: docker/metadata-action@v5
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
tags: |
type=ref,event=branch
type=ref,event=pr
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
type=raw,value=latest,enable={{is_default_branch}}
- name: Build and push Docker image
uses: docker/build-push-action@v5
with:
context: .
file: ./Dockerfile
platforms: linux/amd64,linux/arm64
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
cache-from: type=gha
cache-to: type=gha,mode=max
+54 -65
View File
@@ -1,96 +1,85 @@
name: Release
on:
workflow_dispatch:
inputs:
version_type:
description: 'Type of version bump (major, minor, patch)'
required: true
default: 'patch'
type: choice
options:
- patch
- minor
- major
push:
tags:
- 'v*' # Trigger on version tags like v1.0.0, v0.13.0, etc.
jobs:
release:
runs-on: ubuntu-latest
concurrency: release
permissions:
id-token: write
contents: write
outputs:
released: ${{ steps.release.outputs.released }}
tag: ${{ steps.release.outputs.tag }}
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Python Semantic Release
id: release
uses: python-semantic-release/python-semantic-release@master
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
- name: Publish to PyPI
uses: pypa/gh-action-pypi-publish@release/v1
if: steps.release.outputs.released == 'true'
with:
password: ${{ secrets.PYPI_TOKEN }}
- name: Publish to GitHub Release Assets
uses: python-semantic-release/publish-action@v9.8.9
if: steps.release.outputs.released == 'true'
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
tag: ${{ steps.release.outputs.tag }}
build-macos:
needs: release
if: needs.release.outputs.released == 'true'
runs-on: macos-latest
steps:
- uses: actions/checkout@v4
with:
ref: ${{ needs.release.outputs.tag }}
- name: Set up Python "3.12"
uses: actions/setup-python@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.12"
cache: 'pip'
- name: Install librsvg
run: brew install librsvg
- name: Install uv
run: |
pip install uv
- name: Create virtual env
- name: Install dependencies and build
run: |
uv venv
- name: Install dependencies
run: |
uv sync
uv build
- name: Build macOS installer
- name: Verify build succeeded
run: |
make installer-mac
xattr -dr com.apple.quarantine "installer/build/Basic Memory Installer.app"
# Verify that build artifacts exist
ls -la dist/
echo "Build completed successfully"
- name: Zip macOS installer
run: |
cd installer/build
zip -ry "Basic-Memory-Installer-${{ needs.release.outputs.tag }}.zip" "Basic Memory Installer.app"
- name: Upload macOS installer
uses: softprops/action-gh-release@v1
- name: Create GitHub Release
uses: softprops/action-gh-release@v2
with:
files: installer/build/Basic-Memory-Installer-${{ needs.release.outputs.tag }}.zip
tag_name: ${{ needs.release.outputs.tag }}
files: |
dist/*.whl
dist/*.tar.gz
generate_release_notes: true
tag_name: ${{ github.ref_name }}
token: ${{ secrets.GITHUB_TOKEN }}
- name: Publish to PyPI
uses: pypa/gh-action-pypi-publish@release/v1
with:
password: ${{ secrets.PYPI_TOKEN }}
homebrew:
name: Update Homebrew Formula
needs: release
runs-on: ubuntu-latest
# Only run for stable releases (not dev, beta, or rc versions)
if: ${{ !contains(github.ref_name, 'dev') && !contains(github.ref_name, 'b') && !contains(github.ref_name, 'rc') }}
permissions:
contents: write
actions: read
steps:
- name: Update Homebrew formula
uses: mislav/bump-homebrew-formula-action@v3
with:
# Formula name in homebrew-basic-memory repo
formula-name: basic-memory
# The tap repository
homebrew-tap: basicmachines-co/homebrew-basic-memory
# Base branch of the tap repository
base-branch: main
# Download URL will be automatically constructed from the tag
download-url: https://github.com/basicmachines-co/basic-memory/archive/refs/tags/${{ github.ref_name }}.tar.gz
# Commit message for the formula update
commit-message: |
{{formulaName}} {{version}}
Created by https://github.com/basicmachines-co/basic-memory/actions/runs/${{ github.run_id }}
env:
# Personal Access Token with repo scope for homebrew-basic-memory repo
COMMITTER_TOKEN: ${{ secrets.HOMEBREW_TOKEN }}
+86 -6
View File
@@ -5,14 +5,22 @@ on:
branches: [ "main" ]
pull_request:
branches: [ "main" ]
# pull_request_target runs on the BASE of the PR, not the merge result.
# It has write permissions and access to secrets.
# It's useful for PRs from forks or automated PRs but requires careful use for security reasons.
# See: https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#pull_request_target
pull_request_target:
branches: [ "main" ]
jobs:
test:
runs-on: ubuntu-latest
test-sqlite:
name: Test SQLite (${{ matrix.os }}, Python ${{ matrix.python-version }})
strategy:
fail-fast: false
matrix:
python-version: [ "3.12" ]
os: [ubuntu-latest, windows-latest]
python-version: [ "3.12", "3.13" ]
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v4
@@ -29,6 +37,18 @@ jobs:
run: |
pip install uv
- name: Install just (Linux/macOS)
if: runner.os != 'Windows'
run: |
curl --proto '=https' --tlsv1.2 -sSf https://just.systems/install.sh | bash -s -- --to /usr/local/bin
- name: Install just (Windows)
if: runner.os == 'Windows'
run: |
# Install just using Chocolatey (pre-installed on GitHub Actions Windows runners)
choco install just --yes
shell: pwsh
- name: Create virtual env
run: |
uv venv
@@ -39,9 +59,69 @@ jobs:
- name: Run type checks
run: |
uv run make type-check
just typecheck
- name: Run tests
- name: Run linting
run: |
just lint
- name: Run tests (SQLite)
run: |
uv pip install pytest pytest-cov
uv run make test
just test-sqlite
test-postgres:
name: Test Postgres (Python ${{ matrix.python-version }})
strategy:
fail-fast: false
matrix:
python-version: [ "3.12", "3.13" ]
runs-on: ubuntu-latest
# Postgres service (only available on Linux runners)
services:
postgres:
image: postgres:17
env:
POSTGRES_DB: basic_memory_test
POSTGRES_USER: basic_memory_user
POSTGRES_PASSWORD: dev_password
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
ports:
- 5433:5432
steps:
- uses: actions/checkout@v4
with:
submodules: true
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}
cache: 'pip'
- name: Install uv
run: |
pip install uv
- name: Install just
run: |
curl --proto '=https' --tlsv1.2 -sSf https://just.systems/install.sh | bash -s -- --to /usr/local/bin
- name: Create virtual env
run: |
uv venv
- name: Install dependencies
run: |
uv pip install -e .[dev]
- name: Run tests (Postgres)
run: |
uv pip install pytest pytest-cov
just test-postgres
+12 -1
View File
@@ -42,4 +42,15 @@ ENV/
# macOS
.DS_Store
/.coverage.*
.coverage.*
# obsidian docs:
/docs/.obsidian/
/examples/.obsidian/
/examples/.basic-memory/
# claude action
claude-output
**/.claude/settings.local.json
.mcp.json
+1719
View File
File diff suppressed because it is too large Load Diff
+71
View File
@@ -0,0 +1,71 @@
# Contributor License Agreement
## Copyright Assignment and License Grant
By signing this Contributor License Agreement ("Agreement"), you accept and agree to the following terms and conditions
for your present and future Contributions submitted
to Basic Machines LLC. Except for the license granted herein to Basic Machines LLC and recipients of software
distributed by Basic Machines LLC, you reserve all right,
title, and interest in and to your Contributions.
### 1. Definitions
"You" (or "Your") shall mean the copyright owner or legal entity authorized by the copyright owner that is making this
Agreement with Basic Machines LLC.
"Contribution" shall mean any original work of authorship, including any modifications or additions to an existing work,
that is intentionally submitted by You to Basic
Machines LLC for inclusion in, or documentation of, any of the products owned or managed by Basic Machines LLC (the "
Work").
### 2. Grant of Copyright License
Subject to the terms and conditions of this Agreement, You hereby grant to Basic Machines LLC and to recipients of
software distributed by Basic Machines LLC a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the
Work, and to permit persons to whom the Work is furnished to do so.
### 3. Assignment of Copyright
You hereby assign to Basic Machines LLC all right, title, and interest worldwide in all Copyright covering your
Contributions. Basic Machines LLC may license the
Contributions under any license terms, including copyleft, permissive, commercial, or proprietary licenses.
### 4. Grant of Patent License
Subject to the terms and conditions of this Agreement, You hereby grant to Basic Machines LLC and to recipients of
software distributed by Basic Machines LLC a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to
make, have made, use, offer to sell, sell, import, and
otherwise transfer the Work.
### 5. Developer Certificate of Origin
By making a Contribution to this project, You certify that:
(a) The Contribution was created in whole or in part by You and You have the right to submit it under this Agreement; or
(b) The Contribution is based upon previous work that, to the best of Your knowledge, is covered under an appropriate
open source license and You have the right under that
license to submit that work with modifications, whether created in whole or in part by You, under this Agreement; or
(c) The Contribution was provided directly to You by some other person who certified (a), (b) or (c) and You have not
modified it.
(d) You understand and agree that this project and the Contribution are public and that a record of the Contribution (
including all personal information You submit with
it, including Your sign-off) is maintained indefinitely and may be redistributed consistent with this project or the
open source license(s) involved.
### 6. Representations
You represent that you are legally entitled to grant the above license and assignment. If your employer(s) has rights to
intellectual property that you create that
includes your Contributions, you represent that you have received permission to make Contributions on behalf of that
employer, or that your employer has waived such rights
for your Contributions to Basic Machines LLC.
---
This Agreement is effective as of the date you first submit a Contribution to Basic Machines LLC.
+269
View File
@@ -0,0 +1,269 @@
# CLAUDE.md - Basic Memory Project Guide
## Project Overview
Basic Memory is a local-first knowledge management system built on the Model Context Protocol (MCP). It enables
bidirectional communication between LLMs (like Claude) and markdown files, creating a personal knowledge graph that can
be traversed using links between documents.
## CODEBASE DEVELOPMENT
### Project information
See the [README.md](README.md) file for a project overview.
### Build and Test Commands
- Install: `just install` or `pip install -e ".[dev]"`
- Run all tests (with coverage): `just test` - Runs both unit and integration tests with unified coverage
- Run unit tests only: `just test-unit` - Fast, no coverage
- Run integration tests only: `just test-int` - Fast, no coverage
- Generate HTML coverage: `just coverage` - Opens in browser
- Single test: `pytest tests/path/to/test_file.py::test_function_name`
- Run benchmarks: `pytest test-int/test_sync_performance_benchmark.py -v -m "benchmark and not slow"`
- Lint: `just lint` or `ruff check . --fix`
- Type check: `just typecheck` or `uv run pyright`
- Format: `just format` or `uv run ruff format .`
- Run all code checks: `just check` (runs lint, format, typecheck, test)
- Create db migration: `just migration "Your migration message"`
- Run development MCP Inspector: `just run-inspector`
**Note:** Project requires Python 3.12+ (uses type parameter syntax and `type` aliases introduced in 3.12)
### Test Structure
- `tests/` - Unit tests for individual components (mocked, fast)
- `test-int/` - Integration tests for real-world scenarios (no mocks, realistic)
- Both directories are covered by unified coverage reporting
- Benchmark tests in `test-int/` are marked with `@pytest.mark.benchmark`
- Slow tests are marked with `@pytest.mark.slow`
### Code Style Guidelines
- Line length: 100 characters max
- Python 3.12+ with full type annotations (uses type parameters and type aliases)
- Format with ruff (consistent styling)
- Import order: standard lib, third-party, local imports
- Naming: snake_case for functions/variables, PascalCase for classes
- Prefer async patterns with SQLAlchemy 2.0
- Use Pydantic v2 for data validation and schemas
- CLI uses Typer for command structure
- API uses FastAPI for endpoints
- Follow the repository pattern for data access
- Tools communicate to api routers via the httpx ASGI client (in process)
### Codebase Architecture
- `/alembic` - Alembic db migrations
- `/api` - FastAPI implementation of REST endpoints
- `/cli` - Typer command-line interface
- `/markdown` - Markdown parsing and processing
- `/mcp` - Model Context Protocol server implementation
- `/models` - SQLAlchemy ORM models
- `/repository` - Data access layer
- `/schemas` - Pydantic models for validation
- `/services` - Business logic layer
- `/sync` - File synchronization services
### Development Notes
- MCP tools are defined in src/basic_memory/mcp/tools/
- MCP prompts are defined in src/basic_memory/mcp/prompts/
- MCP tools should be atomic, composable operations
- Use `textwrap.dedent()` for multi-line string formatting in prompts and tools
- MCP Prompts are used to invoke tools and format content with instructions for an LLM
- Schema changes require Alembic migrations
- SQLite is used for indexing and full text search, files are source of truth
- Testing uses pytest with asyncio support (strict mode)
- Unit tests (`tests/`) use mocks when necessary; integration tests (`test-int/`) use real implementations
- Test database uses in-memory SQLite
- Each test runs in a standalone environment with in-memory SQLite and tmp_file directory
- Performance benchmarks are in `test-int/test_sync_performance_benchmark.py`
- Use pytest markers: `@pytest.mark.benchmark` for benchmarks, `@pytest.mark.slow` for slow tests
### Async Client Pattern (Important!)
**All MCP tools and CLI commands use the context manager pattern for HTTP clients:**
```python
from basic_memory.mcp.async_client import get_client
async def my_mcp_tool():
async with get_client() as client:
# Use client for API calls
response = await call_get(client, "/path")
return response
```
**Do NOT use:**
-`from basic_memory.mcp.async_client import client` (deprecated module-level client)
- ❌ Manual auth header management
-`inject_auth_header()` (deleted)
**Key principles:**
- Auth happens at client creation, not per-request
- Proper resource management via context managers
- Supports three modes: Local (ASGI), CLI cloud (HTTP + auth), Cloud app (factory injection)
- Factory pattern enables dependency injection for cloud consolidation
**For cloud app integration:**
```python
from basic_memory.mcp import async_client
# Set custom factory before importing tools
async_client.set_client_factory(your_custom_factory)
```
See SPEC-16 for full context manager refactor details.
## BASIC MEMORY PRODUCT USAGE
### Knowledge Structure
- Entity: Any concept, document, or idea represented as a markdown file
- Observation: A categorized fact about an entity (`- [category] content`)
- Relation: A directional link between entities (`- relation_type [[Target]]`)
- Frontmatter: YAML metadata at the top of markdown files
- Knowledge representation follows precise markdown format:
- Observations with [category] prefixes
- Relations with WikiLinks [[Entity]]
- Frontmatter with metadata
### Basic Memory Commands
**Local Commands:**
- Sync knowledge: `basic-memory sync` or `basic-memory sync --watch`
- Import from Claude: `basic-memory import claude conversations`
- Import from ChatGPT: `basic-memory import chatgpt`
- Import from Memory JSON: `basic-memory import memory-json`
- Check sync status: `basic-memory status`
- Tool access: `basic-memory tools` (provides CLI access to MCP tools)
- Guide: `basic-memory tools basic-memory-guide`
- Continue: `basic-memory tools continue-conversation --topic="search"`
**Cloud Commands (requires subscription):**
- Authenticate: `basic-memory cloud login`
- Logout: `basic-memory cloud logout`
- Bidirectional sync: `basic-memory cloud sync`
- Integrity check: `basic-memory cloud check`
- Mount cloud storage: `basic-memory cloud mount`
- Unmount cloud storage: `basic-memory cloud unmount`
### MCP Capabilities
- Basic Memory exposes these MCP tools to LLMs:
**Content Management:**
- `write_note(title, content, folder, tags)` - Create/update markdown notes with semantic observations and relations
- `read_note(identifier, page, page_size)` - Read notes by title, permalink, or memory:// URL with knowledge graph awareness
- `read_content(path)` - Read raw file content (text, images, binaries) without knowledge graph processing
- `view_note(identifier, page, page_size)` - View notes as formatted artifacts for better readability
- `edit_note(identifier, operation, content)` - Edit notes incrementally (append, prepend, find/replace, replace_section)
- `move_note(identifier, destination_path)` - Move notes to new locations, updating database and maintaining links
- `delete_note(identifier)` - Delete notes from the knowledge base
**Knowledge Graph Navigation:**
- `build_context(url, depth, timeframe)` - Navigate the knowledge graph via memory:// URLs for conversation continuity
- `recent_activity(type, depth, timeframe)` - Get recently updated information with specified timeframe (e.g., "1d", "1 week")
- `list_directory(dir_name, depth, file_name_glob)` - Browse directory contents with filtering and depth control
**Search & Discovery:**
- `search_notes(query, page, page_size, search_type, types, entity_types, after_date)` - Full-text search across all content with advanced filtering options
**Project Management:**
- `list_memory_projects()` - List all available projects with their status
- `create_memory_project(project_name, project_path, set_default)` - Create new Basic Memory projects
- `delete_project(project_name)` - Delete a project from configuration
- `get_current_project()` - Get current project information and stats
- `sync_status()` - Check file synchronization and background operation status
**Visualization:**
- `canvas(nodes, edges, title, folder)` - Generate Obsidian canvas files for knowledge graph visualization
- MCP Prompts for better AI interaction:
- `ai_assistant_guide()` - Guidance on effectively using Basic Memory tools for AI assistants
- `continue_conversation(topic, timeframe)` - Continue previous conversations with relevant historical context
- `search(query, after_date)` - Search with detailed, formatted results for better context understanding
- `recent_activity(timeframe)` - View recently changed items with formatted output
- `json_canvas_spec()` - Full JSON Canvas specification for Obsidian visualization
### Cloud Features (v0.15.0+)
Basic Memory now supports cloud synchronization and storage (requires active subscription):
**Authentication:**
- JWT-based authentication with subscription validation
- Secure session management with token refresh
- Support for multiple cloud projects
**Bidirectional Sync:**
- rclone bisync integration for two-way synchronization
- Conflict resolution and integrity verification
- Real-time sync with change detection
- Mount/unmount cloud storage for direct file access
**Cloud Project Management:**
- Create and manage projects in the cloud
- Toggle between local and cloud modes
- Per-project sync configuration
- Subscription-based access control
**Security & Performance:**
- Removed .env file loading for improved security
- .gitignore integration (respects gitignored files)
- WAL mode for SQLite performance
- Background relation resolution (non-blocking startup)
- API performance optimizations (SPEC-11)
## AI-Human Collaborative Development
Basic Memory emerged from and enables a new kind of development process that combines human and AI capabilities. Instead
of using AI just for code generation, we've developed a true collaborative workflow:
1. AI (LLM) writes initial implementation based on specifications and context
2. Human reviews, runs tests, and commits code with any necessary adjustments
3. Knowledge persists across conversations using Basic Memory's knowledge graph
4. Development continues seamlessly across different AI sessions with consistent context
5. Results improve through iterative collaboration and shared understanding
This approach has allowed us to tackle more complex challenges and build a more robust system than either humans or AI
could achieve independently.
## GitHub Integration
Basic Memory has taken AI-Human collaboration to the next level by integrating Claude directly into the development workflow through GitHub:
### GitHub MCP Tools
Using the GitHub Model Context Protocol server, Claude can now:
- **Repository Management**:
- View repository files and structure
- Read file contents
- Create new branches
- Create and update files
- **Issue Management**:
- Create new issues
- Comment on existing issues
- Close and update issues
- Search across issues
- **Pull Request Workflow**:
- Create pull requests
- Review code changes
- Add comments to PRs
This integration enables Claude to participate as a full team member in the development process, not just as a code generation tool. Claude's GitHub account ([bm-claudeai](https://github.com/bm-claudeai)) is a member of the Basic Machines organization with direct contributor access to the codebase.
### Collaborative Development Process
With GitHub integration, the development workflow includes:
1. **Direct code review** - Claude can analyze PRs and provide detailed feedback
2. **Contribution tracking** - All of Claude's contributions are properly attributed in the Git history
3. **Branch management** - Claude can create feature branches for implementations
4. **Documentation maintenance** - Claude can keep documentation updated as the code evolves
5. **Code Commits**: ALWAYS sign off commits with `git commit -s`
This level of integration represents a new paradigm in AI-human collaboration, where the AI assistant becomes a full-fledged team member rather than just a tool for generating code snippets.
+268 -8
View File
@@ -1,17 +1,277 @@
# Contributing to Basic Memory
Thank you for considering contributing to Basic Memory! Your help is greatly appreciated to improve this project.
Thank you for considering contributing to Basic Memory! This document outlines the process for contributing to the
project and how to get started as a developer.
## How to Contribute
## Getting Started
1. **Fork the Repo**: Fork the repository and clone your copy.
1. **Create a Branch**: Create a new branch for your feature or fix.
1. **Test Your Changes**: Ensure tests pass locally and include new tests when necessary to ensure 100& test coverage.
1. **Format Your Code**: Run `make format` to ensure code is formatted appropriately.
4. **Submit a PR**: Submit a pull request with a detailed description of your changes.
### Development Environment
Thank You!
1. **Clone the Repository**:
```bash
git clone https://github.com/basicmachines-co/basic-memory.git
cd basic-memory
```
2. **Install Dependencies**:
```bash
# Using just (recommended)
just install
# Or using uv
uv install -e ".[dev]"
# Or using pip
pip install -e ".[dev]"
```
> **Note**: Basic Memory uses [just](https://just.systems) as a modern command runner. Install with `brew install just` or `cargo install just`.
3. **Activate the Virtual Environment**
```bash
source .venv/bin/activate
```
4. **Run the Tests**:
```bash
# Run all tests with unified coverage (unit + integration)
just test
# Run unit tests only (fast, no coverage)
just test-unit
# Run integration tests only (fast, no coverage)
just test-int
# Generate HTML coverage report
just coverage
# Run a specific test
pytest tests/path/to/test_file.py::test_function_name
```
### Development Workflow
1. **Fork the Repo**: Fork the repository on GitHub and clone your copy.
2. **Create a Branch**: Create a new branch for your feature or fix.
```bash
git checkout -b feature/your-feature-name
# or
git checkout -b fix/issue-you-are-fixing
```
3. **Make Your Changes**: Implement your changes with appropriate test coverage.
4. **Check Code Quality**:
```bash
# Run all checks at once
just check
# Or run individual checks
just lint # Run linting
just format # Format code
just type-check # Type checking
```
5. **Test Your Changes**: Ensure all tests pass locally and maintain 100% test coverage.
```bash
just test
```
6. **Submit a PR**: Submit a pull request with a detailed description of your changes.
## LLM-Assisted Development
This project is designed for collaborative development between humans and LLMs (Large Language Models):
1. **CLAUDE.md**: The repository includes a `CLAUDE.md` file that serves as a project guide for both humans and LLMs.
This file contains:
- Key project information and architectural overview
- Development commands and workflows
- Code style guidelines
- Documentation standards
2. **AI-Human Collaborative Workflow**:
- We encourage using LLMs like Claude for code generation, reviews, and documentation
- When possible, save context in markdown files that can be referenced later
- This enables seamless knowledge transfer between different development sessions
- Claude can help with implementation details while you focus on architecture and design
3. **Adding to CLAUDE.md**:
- If you discover useful project information or common commands, consider adding them to CLAUDE.md
- This helps all contributors (human and AI) maintain consistent knowledge of the project
## Pull Request Process
1. **Create a Pull Request**: Open a PR against the `main` branch with a clear title and description.
2. **Sign the Developer Certificate of Origin (DCO)**: All contributions require signing our DCO, which certifies that
you have the right to submit your contributions. This will be automatically checked by our CLA assistant when you
create a PR.
3. **PR Description**: Include:
- What the PR changes
- Why the change is needed
- How you tested the changes
- Any related issues (use "Fixes #123" to automatically close issues)
4. **Code Review**: Wait for code review and address any feedback.
5. **CI Checks**: Ensure all CI checks pass.
6. **Merge**: Once approved, a maintainer will merge your PR.
## Developer Certificate of Origin
By contributing to this project, you agree to the [Developer Certificate of Origin (DCO)](CLA.md). This means you
certify that:
- You have the right to submit your contributions
- You're not knowingly submitting code with patent or copyright issues
- Your contributions are provided under the project's license (AGPL-3.0)
This is a lightweight alternative to a Contributor License Agreement and helps ensure that all contributions can be
properly incorporated into the project and potentially used in commercial applications.
### Signing Your Commits
Sign your commit:
**Using the `-s` or `--signoff` flag**:
```bash
git commit -s -m "Your commit message"
```
This adds a `Signed-off-by` line to your commit message, certifying that you adhere to the DCO.
The sign-off certifies that you have the right to submit your contribution under the project's license and verifies your
agreement to the DCO.
## Code Style Guidelines
- **Python Version**: Python 3.12+ with full type annotations (3.12+ required for type parameter syntax)
- **Line Length**: 100 characters maximum
- **Formatting**: Use ruff for consistent styling
- **Import Order**: Standard lib, third-party, local imports
- **Naming**: Use snake_case for functions/variables, PascalCase for classes
- **Documentation**: Add docstrings to public functions, classes, and methods
- **Type Annotations**: Use type hints for all functions and methods
## Testing Guidelines
### Test Structure
Basic Memory uses two test directories with unified coverage reporting:
- **`tests/`**: Unit tests that test individual components in isolation
- Fast execution with extensive mocking
- Test individual functions, classes, and modules
- Run with: `just test-unit` (no coverage, fast)
- **`test-int/`**: Integration tests that test real-world scenarios
- Test full workflows with real database and file operations
- Include performance benchmarks
- More realistic but slower than unit tests
- Run with: `just test-int` (no coverage, fast)
### Running Tests
```bash
# Run all tests with unified coverage report
just test
# Run only unit tests (fast iteration)
just test-unit
# Run only integration tests
just test-int
# Generate HTML coverage report
just coverage
# Run specific test
pytest tests/path/to/test_file.py::test_function_name
# Run tests excluding benchmarks
pytest -m "not benchmark"
# Run only benchmark tests
pytest -m benchmark test-int/test_sync_performance_benchmark.py
```
### Performance Benchmarks
The `test-int/test_sync_performance_benchmark.py` file contains performance benchmarks that measure sync and indexing speed:
- `test_benchmark_sync_100_files` - Small repository performance
- `test_benchmark_sync_500_files` - Medium repository performance
- `test_benchmark_sync_1000_files` - Large repository performance (marked slow)
- `test_benchmark_resync_no_changes` - Re-sync performance baseline
Run benchmarks with:
```bash
# Run all benchmarks (excluding slow ones)
pytest test-int/test_sync_performance_benchmark.py -v -m "benchmark and not slow"
# Run all benchmarks including slow ones
pytest test-int/test_sync_performance_benchmark.py -v -m benchmark
# Run specific benchmark
pytest test-int/test_sync_performance_benchmark.py::test_benchmark_sync_100_files -v
```
See `test-int/BENCHMARKS.md` for detailed benchmark documentation.
### Testing Best Practices
- **Coverage Target**: We aim for high test coverage for all code
- **Test Framework**: Use pytest for unit and integration tests
- **Mocking**: Avoid mocking in integration tests; use sparingly in unit tests
- **Edge Cases**: Test both normal operation and edge cases
- **Database Testing**: Use in-memory SQLite for testing database operations
- **Fixtures**: Use async pytest fixtures for setup and teardown
- **Markers**: Use `@pytest.mark.benchmark` for benchmarks, `@pytest.mark.slow` for slow tests
## Release Process
Basic Memory uses automatic versioning based on git tags with `uv-dynamic-versioning`. Here's how releases work:
### Version Management
- **Development versions**: Automatically generated from git commits (e.g., `0.12.4.dev26+468a22f`)
- **Beta releases**: Created by tagging with beta suffixes (e.g., `git tag v0.13.0b1`)
- **Stable releases**: Created by tagging with version numbers (e.g., `git tag v0.13.0`)
### Release Workflows
#### Development Builds
- Automatically published to PyPI on every commit to `main`
- Version format: `0.12.4.dev26+468a22f` (base version + dev + commit count + hash)
- Users install with: `pip install basic-memory --pre --force-reinstall`
#### Beta Releases
1. Create and push a beta tag: `git tag v0.13.0b1 && git push origin v0.13.0b1`
2. GitHub Actions automatically builds and publishes to PyPI
3. Users install with: `pip install basic-memory --pre`
#### Stable Releases
1. Create and push a version tag: `git tag v0.13.0 && git push origin v0.13.0`
2. GitHub Actions automatically:
- Builds the package with version `0.13.0`
- Creates GitHub release with auto-generated notes
- Publishes to PyPI
3. Users install with: `pip install basic-memory`
### For Contributors
- No manual version bumping required
- Versions are automatically derived from git tags
- Focus on code changes, not version management
## Creating Issues
If you're planning to work on something, please create an issue first to discuss the approach. Include:
- A clear title and description
- Steps to reproduce if reporting a bug
- Expected behavior vs. actual behavior
- Any relevant logs or screenshots
- Your proposed solution, if you have one
## Code of Conduct
All contributors must follow the [Code of Conduct](CODE_OF_CONDUCT.md).
## Thank You!
Your contributions help make Basic Memory better. We appreciate your time and effort!
+46
View File
@@ -0,0 +1,46 @@
FROM python:3.12-slim-bookworm
# Build arguments for user ID and group ID (defaults to 1000)
ARG UID=1000
ARG GID=1000
# Copy uv from official image
COPY --from=ghcr.io/astral-sh/uv:latest /uv /uvx /bin/
# Set environment variables
ENV PYTHONUNBUFFERED=1 \
PYTHONDONTWRITEBYTECODE=1
# Create a group and user with the provided UID/GID
# Check if the GID already exists, if not create appgroup
RUN (getent group ${GID} || groupadd --gid ${GID} appgroup) && \
useradd --uid ${UID} --gid ${GID} --create-home --shell /bin/bash appuser
# Copy the project into the image
ADD . /app
# Sync the project into a new environment, asserting the lockfile is up to date
WORKDIR /app
RUN uv sync --locked
# Create necessary directories and set ownership
RUN mkdir -p /app/data/basic-memory /app/.basic-memory && \
chown -R appuser:${GID} /app
# Set default data directory and add venv to PATH
ENV BASIC_MEMORY_HOME=/app/data/basic-memory \
BASIC_MEMORY_PROJECT_ROOT=/app/data \
PATH="/app/.venv/bin:$PATH"
# Switch to the non-root user
USER appuser
# Expose port
EXPOSE 8000
# Health check
HEALTHCHECK --interval=30s --timeout=10s --start-period=5s --retries=3 \
CMD basic-memory --version || exit 1
# Use the basic-memory entrypoint to run the MCP server with default SSE transport
CMD ["basic-memory", "mcp", "--transport", "sse", "--host", "0.0.0.0", "--port", "8000"]
-43
View File
@@ -1,43 +0,0 @@
.PHONY: install test lint clean format type-check installer-mac installer-win
install:
pip install -e ".[dev]"
test:
pytest -p pytest_mock -v
lint:
ruff check . --fix
type-check:
uv run pyright
clean:
find . -type f -name '*.pyc' -delete
find . -type d -name '__pycache__' -exec rm -r {} +
rm -rf installer/build/
rm -rf installer/dist/
rm -f rw.*.dmg
rm -rf dist
rm -rf installer/build
rm -rf installer/dist
rm -f .coverage.*
format:
uv run ruff format .
# run inspector tool
run-dev:
uv run mcp dev src/basic_memory/mcp/main.py
# Build app installer
installer-mac:
cd installer && chmod +x make_icons.sh && ./make_icons.sh
cd installer && uv run python setup.py bdist_mac
installer-win:
cd installer && uv run python setup.py bdist_win32
update-deps:
uv lock f--upgrade
+451 -291
View File
@@ -1,344 +1,504 @@
[![License: AGPL v3](https://img.shields.io/badge/License-AGPL_v3-blue.svg)](https://www.gnu.org/licenses/agpl-3.0)
[![PyPI version](https://badge.fury.io/py/basic-memory.svg)](https://badge.fury.io/py/basic-memory)
[![Python 3.12+](https://img.shields.io/badge/python-3.12+-blue.svg)](https://www.python.org/downloads/)
[![Tests](https://github.com/basicmachines-co/basic-memory/workflows/Tests/badge.svg)](https://github.com/basicmachines-co/basic-memory/actions)
[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)
![](https://badge.mcpx.dev?type=server 'MCP Server')
![](https://badge.mcpx.dev?type=dev 'MCP Dev')
[![smithery badge](https://smithery.ai/badge/@basicmachines-co/basic-memory)](https://smithery.ai/server/@basicmachines-co/basic-memory)
## 🚀 Basic Memory Cloud is Live!
- **Cross-device and multi-platform support is here.** Your knowledge graph now works on desktop, web, and mobile - seamlessly synced across all your AI tools (Claude, ChatGPT, Gemini, Claude Code, and Codex)
- **Early Supporter Pricing:** Early users get 25% off forever.
The open source project continues as always. Cloud just makes it work everywhere.
[Sign up now →](https://basicmemory.com/beta)
with a 7 day free trial
# Basic Memory
Basic Memory lets you build persistent knowledge through natural conversations with Large Language Models (LLMs) like
Claude, while keeping everything in simple markdown files on your computer. It uses the Model Context Protocol (MCP) to
Claude, while keeping everything in simple Markdown files on your computer. It uses the Model Context Protocol (MCP) to
enable any compatible LLM to read and write to your local knowledge base.
## What is Basic Memory?
- Website: https://basicmachines.co
- Documentation: https://memory.basicmachines.co
Most people use LLMs like calculators - paste in some text, expect to get an answer back, repeat. Each conversation
starts fresh,
and any knowledge or context is lost. Some try to work around this by:
## Pick up your conversation right where you left off
- Saving chat histories (but they're hard to reference)
- Copying and pasting previous conversations (messy and repetitive)
- Using RAG systems to query documents (complex and often cloud-based)
- AI assistants can load context from local files in a new conversation
- Notes are saved locally as Markdown files in real time
- No project knowledge or special prompting required
Basic Memory takes a different approach by letting both humans and LLMs read and write knowledge naturally using
standard markdown files. This means:
https://github.com/user-attachments/assets/a55d8238-8dd0-454a-be4c-8860dbbd0ddc
- Your knowledge stays in files you control
- Both you and the LLM can read and write notes
- Context persists across conversations
- Context stays local and user controlled
## How It Works in Practice
Let's say you're working on a new project and want to capture design decisions. Here's how it works:
1. Start by chatting normally:
```markdown
We need to design a new auth system, some key features:
- local first, don't delegate users to third party system
- support multiple platforms via jwt
- want to keep it simple but secure
```
... continue conversation.
2. Ask Claude to help structure this knowledge:
```
"Lets write a note about the auth system design."
```
Claude creates a new markdown file on your system (which you can see instantly in Obsidian or your editor):
```markdown
---
title: Auth System Design
permalink: auth-system-design
tags
- design
- auth
---
# Auth System Design
## Observations
- [requirement] Local-first authentication without third party delegation
- [tech] JWT-based auth for cross-platform support
- [principle] Balance simplicity with security
## Relations
- implements [[Security Requirements]]
- relates_to [[Platform Support]]
- referenced_by [[JWT Implementation]]
```
The note embeds semantic content (Observations) and links to other topics (Relations) via simple markdown formatting.
3. You can edit this file directly in your editor in real time:
```markdown
# Auth System Design
## Observations
- [requirement] Local-first authentication without third party delegation
- [tech] JWT-based auth for cross-platform support
- [principle] Balance simplicity with security
- [decision] Will use bcrypt for password hashing # Added by you
## Relations
- implements [[Security Requirements]]
- relates_to [[Platform Support]]
- referenced_by [[JWT Implementation]]
- blocks [[User Service]] # Added by you
```
4. In a new chat with Claude, you can reference this knowledge:
```
"Claude, look at memory://auth-system-design for context about our auth system"
```
Claude can now build rich context from the knowledge graph. For example:
```
Following relation 'implements [[Security Requirements]]':
- Found authentication best practices
- OWASP guidelines for JWT
- Rate limiting requirements
Following relation 'relates_to [[Platform Support]]':
- Mobile auth requirements
- Browser security considerations
- JWT storage strategies
```
Each related document can lead to more context, building a rich semantic understanding of your knowledge base. All of
this context comes from standard markdown files that both humans and LLMs can read and write.
Everything stays in local markdown files that you can:
- Edit in any text editor
- Version via git
- Back up normally
- Share when you want to
## Technical Implementation
Under the hood, Basic Memory:
1. Stores everything in markdown files
2. Uses a SQLite database just for searching and indexing
3. Extracts semantic meaning from simple markdown patterns
4. Maintains a local knowledge graph from file content
The file format is just markdown with some simple markup:
Frontmatter
- title
- type
- permalink
- optional metadata
Observations
- facts about a topic
```markdown
- [category] content #tag (optional context)
```
Relations
- links to other topics
```markdown
- relation_type [[WikiLink]] (optional context)
```
Example:
```markdown
---
title: Note tile
type: note
permalink: unique/stable/id # Added automatically
tags
- tag1
- tag2
---
# Note Title
Regular markdown content...
## Observations
- [category] Structured knowledge #tag (optional context)
- [idea] Another observation
## Relations
- links_to [[Other Note]]
- implements [[Some Spec]]
```
Basic Memory will parse the markdown and derive the semantic relationships in the content. When you run
`basic-memory sync`:
1. New and changed files are detected
2. Markdown patterns become semantic knowledge:
- `[tech]` becomes a categorized observation
- `[[WikiLink]]` creates a relation in the knowledge graph
- Tags and metadata are indexed for search
3. A SQLite database maintains these relationships for fast querying
4. Claude and other MCP-compatible LLMs can access this knowledge via memory:// URLs
This creates a two-way flow where:
- Humans write and edit markdown files
- LLMs read and write through the MCP protocol
- Sync keeps everything consistent
- All knowledge stays in local files.
## Using with Claude
Basic Memory works with the Claude desktop app (https://claude.ai/):
1. Install Basic Memory locally:
## Quick Start
```bash
# Install with uv (recommended)
uv tool install basic-memory
# Configure Claude Desktop (edit ~/Library/Application Support/Claude/claude_desktop_config.json)
# Add this to your config:
{
"mcpServers": {
"basic-memory": {
"command": "uvx",
"args": [
"basic-memory"
"basic-memory",
"mcp"
]
}
}
}
# Now in Claude Desktop, you can:
# - Write notes with "Create a note about coffee brewing methods"
# - Read notes with "What do I know about pour over coffee?"
# - Search with "Find information about Ethiopian beans"
```
You can view shared context via files in `~/basic-memory` (default directory location).
### Alternative Installation via Smithery
You can use [Smithery](https://smithery.ai/server/@basicmachines-co/basic-memory) to automatically configure Basic
Memory for Claude Desktop:
```bash
npx -y @smithery/cli install @basicmachines-co/basic-memory --client claude
```
This installs and configures Basic Memory without requiring manual edits to the Claude Desktop configuration file. The
Smithery server hosts the MCP server component, while your data remains stored locally as Markdown files.
### Glama.ai
<a href="https://glama.ai/mcp/servers/o90kttu9ym">
<img width="380" height="200" src="https://glama.ai/mcp/servers/o90kttu9ym/badge" alt="basic-memory MCP server" />
</a>
## Why Basic Memory?
Most LLM interactions are ephemeral - you ask a question, get an answer, and everything is forgotten. Each conversation
starts fresh, without the context or knowledge from previous ones. Current workarounds have limitations:
- Chat histories capture conversations but aren't structured knowledge
- RAG systems can query documents but don't let LLMs write back
- Vector databases require complex setups and often live in the cloud
- Knowledge graphs typically need specialized tools to maintain
Basic Memory addresses these problems with a simple approach: structured Markdown files that both humans and LLMs can
read
and write to. The key advantages:
- **Local-first:** All knowledge stays in files you control
- **Bi-directional:** Both you and the LLM read and write to the same files
- **Structured yet simple:** Uses familiar Markdown with semantic patterns
- **Traversable knowledge graph:** LLMs can follow links between topics
- **Standard formats:** Works with existing editors like Obsidian
- **Lightweight infrastructure:** Just local files indexed in a local SQLite database
With Basic Memory, you can:
- Have conversations that build on previous knowledge
- Create structured notes during natural conversations
- Have conversations with LLMs that remember what you've discussed before
- Navigate your knowledge graph semantically
- Keep everything local and under your control
- Use familiar tools like Obsidian to view and edit notes
- Build a personal knowledge base that grows over time
- Sync your knowledge to the cloud with bidirectional synchronization
- Authenticate and manage cloud projects with subscription validation
- Mount cloud storage for direct file access
## How It Works in Practice
Let's say you're exploring coffee brewing methods and want to capture your knowledge. Here's how it works:
1. Start by chatting normally:
```
I've been experimenting with different coffee brewing methods. Key things I've learned:
- Pour over gives more clarity in flavor than French press
- Water temperature is critical - around 205°F seems best
- Freshly ground beans make a huge difference
```
... continue conversation.
2. Ask the LLM to help structure this knowledge:
```
"Let's write a note about coffee brewing methods."
```
LLM creates a new Markdown file on your system (which you can see instantly in Obsidian or your editor):
```markdown
---
title: Coffee Brewing Methods
permalink: coffee-brewing-methods
tags:
- coffee
- brewing
---
# Coffee Brewing Methods
## Observations
- [method] Pour over provides more clarity and highlights subtle flavors
- [technique] Water temperature at 205°F (96°C) extracts optimal compounds
- [principle] Freshly ground beans preserve aromatics and flavor
## Relations
- relates_to [[Coffee Bean Origins]]
- requires [[Proper Grinding Technique]]
- affects [[Flavor Extraction]]
```
The note embeds semantic content and links to other topics via simple Markdown formatting.
3. You see this file on your computer in real time in the current project directory (default `~/$HOME/basic-memory`).
- Realtime sync can be enabled via running `basic-memory sync --watch`
4. In a chat with the LLM, you can reference a topic:
```
Look at `coffee-brewing-methods` for context about pour over coffee
```
The LLM can now build rich context from the knowledge graph. For example:
```
Following relation 'relates_to [[Coffee Bean Origins]]':
- Found information about Ethiopian Yirgacheffe
- Notes on Colombian beans' nutty profile
- Altitude effects on bean characteristics
Following relation 'requires [[Proper Grinding Technique]]':
- Burr vs. blade grinder comparisons
- Grind size recommendations for different methods
- Impact of consistent particle size on extraction
```
Each related document can lead to more context, building a rich semantic understanding of your knowledge base.
This creates a two-way flow where:
- Humans write and edit Markdown files
- LLMs read and write through the MCP protocol
- Sync keeps everything consistent
- All knowledge stays in local files.
## Technical Implementation
Under the hood, Basic Memory:
1. Stores everything in Markdown files
2. Uses a SQLite database for searching and indexing
3. Extracts semantic meaning from simple Markdown patterns
- Files become `Entity` objects
- Each `Entity` can have `Observations`, or facts associated with it
- `Relations` connect entities together to form the knowledge graph
4. Maintains the local knowledge graph derived from the files
5. Provides bidirectional synchronization between files and the knowledge graph
6. Implements the Model Context Protocol (MCP) for AI integration
7. Exposes tools that let AI assistants traverse and manipulate the knowledge graph
8. Uses memory:// URLs to reference entities across tools and conversations
The file format is just Markdown with some simple markup:
Each Markdown file has:
### Frontmatter
```markdown
title: <Entity title>
type: <The type of Entity> (e.g. note)
permalink: <a uri slug>
- <optional metadata> (such as tags)
```
### Observations
Observations are facts about a topic.
They can be added by creating a Markdown list with a special format that can reference a `category`, `tags` using a
"#" character, and an optional `context`.
Observation Markdown format:
```markdown
- [category] content #tag (optional context)
```
Examples of observations:
```markdown
- [method] Pour over extracts more floral notes than French press
- [tip] Grind size should be medium-fine for pour over #brewing
- [preference] Ethiopian beans have bright, fruity flavors (especially from Yirgacheffe)
- [fact] Lighter roasts generally contain more caffeine than dark roasts
- [experiment] Tried 1:15 coffee-to-water ratio with good results
- [resource] James Hoffman's V60 technique on YouTube is excellent
- [question] Does water temperature affect extraction of different compounds differently?
- [note] My favorite local shop uses a 30-second bloom time
```
### Relations
Relations are links to other topics. They define how entities connect in the knowledge graph.
Markdown format:
```markdown
- relation_type [[WikiLink]] (optional context)
```
Examples of relations:
```markdown
- pairs_well_with [[Chocolate Desserts]]
- grown_in [[Ethiopia]]
- contrasts_with [[Tea Brewing Methods]]
- requires [[Burr Grinder]]
- improves_with [[Fresh Beans]]
- relates_to [[Morning Routine]]
- inspired_by [[Japanese Coffee Culture]]
- documented_in [[Coffee Journal]]
```
## Using with VS Code
Add the following JSON block to your User Settings (JSON) file in VS Code. You can do this by pressing `Ctrl + Shift + P` and typing `Preferences: Open User Settings (JSON)`.
```json
{
"mcp": {
"servers": {
"basic-memory": {
"command": "uvx",
"args": ["basic-memory", "mcp"]
}
}
}
}
```
2. Add to Claude Desktop:
Optionally, you can add it to a file called `.vscode/mcp.json` in your workspace. This will allow you to share the configuration with others.
```
Basic Memory is available with these tools:
- write_note() for creating/updating notes
- read_note() for loading notes
- build_context() to load notes via memory:// URLs
- recent_activity() to find recently updated information
- search() to search infomation in the knowledge base
```json
{
"servers": {
"basic-memory": {
"command": "uvx",
"args": ["basic-memory", "mcp"]
}
}
}
```
3. Install via uv
You can use Basic Memory with VS Code to easily retrieve and store information while coding.
## Using with Claude Desktop
Basic Memory is built using the MCP (Model Context Protocol) and works with the Claude desktop app (https://claude.ai/):
1. Configure Claude Desktop to use Basic Memory:
Edit your MCP configuration file (usually located at `~/Library/Application Support/Claude/claude_desktop_config.json`
for OS X):
```json
{
"mcpServers": {
"basic-memory": {
"command": "uvx",
"args": [
"basic-memory",
"mcp"
]
}
}
}
```
If you want to use a specific project (see [Multiple Projects](#multiple-projects) below), update your Claude Desktop
config:
```json
{
"mcpServers": {
"basic-memory": {
"command": "uvx",
"args": [
"basic-memory",
"mcp",
"--project",
"your-project-name"
]
}
}
}
```
2. Sync your knowledge:
```bash
uv add basic-memory
# sync local knowledge updates
# One-time sync of local knowledge updates
basic-memory sync
# run realtime sync process
# Run realtime sync process (recommended)
basic-memory sync --watch
```
## Design Philosophy
Basic Memory is built on some key ideas:
- Your knowledge should stay in files you control
- Both humans and AI should use natural formats
- Simple text patterns can capture rich meaning
- Local-first doesn't mean feature-poor
## Importing data
Basic memory has cli commands to import data from several formats into Markdown files
### Claude.ai
First, request an export of your data from your Claude account. The data will be emailed to you in several files,
including
`conversations.json` and `projects.json`.
Import Claude.ai conversation data
3. Cloud features (optional, requires subscription):
```bash
basic-memory import claude conversations
# Authenticate with cloud
basic-memory cloud login
# Bidirectional sync with cloud
basic-memory cloud sync
# Verify cloud integrity
basic-memory cloud check
# Mount cloud storage
basic-memory cloud mount
```
The conversations will be turned into Markdown files and placed in the "conversations" folder by default (this can be
changed with the --folder arg).
4. In Claude Desktop, the LLM can now use these tools:
Example:
**Content Management:**
```
write_note(title, content, folder, tags) - Create or update notes
read_note(identifier, page, page_size) - Read notes by title or permalink
read_content(path) - Read raw file content (text, images, binaries)
view_note(identifier) - View notes as formatted artifacts
edit_note(identifier, operation, content) - Edit notes incrementally
move_note(identifier, destination_path) - Move notes with database consistency
delete_note(identifier) - Delete notes from knowledge base
```
**Knowledge Graph Navigation:**
```
build_context(url, depth, timeframe) - Navigate knowledge graph via memory:// URLs
recent_activity(type, depth, timeframe) - Find recently updated information
list_directory(dir_name, depth) - Browse directory contents with filtering
```
**Search & Discovery:**
```
search(query, page, page_size) - Search across your knowledge base
```
**Project Management:**
```
list_memory_projects() - List all available projects
create_memory_project(project_name, project_path) - Create new projects
get_current_project() - Show current project stats
sync_status() - Check synchronization status
```
**Visualization:**
```
canvas(nodes, edges, title, folder) - Generate knowledge visualizations
```
5. Example prompts to try:
```
"Create a note about our project architecture decisions"
"Find information about JWT authentication in my notes"
"Create a canvas visualization of my project components"
"Read my notes on the authentication system"
"What have I been working on in the past week?"
```
## Futher info
See the [Documentation](https://memory.basicmachines.co/) for more info, including:
- [Complete User Guide](https://docs.basicmemory.com/user-guide/)
- [CLI tools](https://docs.basicmemory.com/guides/cli-reference/)
- [Cloud CLI and Sync](https://docs.basicmemory.com/guides/cloud-cli/)
- [Managing multiple Projects](https://docs.basicmemory.com/guides/cli-reference/#project)
- [Importing data from OpenAI/Claude Projects](https://docs.basicmemory.com/guides/cli-reference/#import)
## Development
### Running Tests
Basic Memory supports dual database backends (SQLite and Postgres). Tests are parametrized to run against both backends automatically.
**Quick Start:**
```bash
Importing chats from conversations.json...writing to .../basic-memory
Reading chat data... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100%
╭────────────────────────────╮
│ Import complete! │
│ │
│ Imported 307 conversations │
│ Containing 7769 messages │
╰────────────────────────────╯
# Run SQLite tests (default, no Docker needed)
just test-sqlite
# Run Postgres tests (requires Docker)
just test-postgres
```
Next, you can run the `sync` command to import the data into basic-memory
**Available Test Commands:**
- `just test-sqlite` - Run tests against SQLite only (fastest, no Docker needed)
- `just test-postgres` - Run tests against Postgres only (requires Docker)
- `just test-windows` - Run Windows-specific tests (auto-skips on other platforms)
- `just test-benchmark` - Run performance benchmark tests
- `just test-all` - Run all tests including Windows, Postgres, and benchmarks
**Postgres Testing Requirements:**
To run Postgres tests, you need to start the test database:
```bash
basic-memory sync
docker-compose -f docker-compose-postgres.yml up -d
```
You can also import project data from Claude.ai
Tests will connect to `localhost:5433/basic_memory_test`.
```bash
➜ basic-memory import claude projects
Importing projects from projects.json...writing to .../basic-memory/projects
Reading project data... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100%
╭────────────────────────────────╮
│ Import complete! │
│ │
│ Imported 101 project documents │
│ Imported 32 prompt templates │
╰────────────────────────────────╯
**Test Markers:**
Run 'basic-memory sync' to index the new files.
```
### Chat Gpt
Tests use pytest markers for selective execution:
- `postgres` - Tests that run against Postgres backend
- `windows` - Windows-specific database optimizations
- `benchmark` - Performance tests (excluded from default runs)
**Other Development Commands:**
```bash
➜ basic-memory import chatgpt
Importing chats from conversations.json...writing to .../basic-memory/conversations
Reading chat data... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100%
╭────────────────────────────╮
│ Import complete! │
│ │
│ Imported 198 conversations │
│ Containing 11777 messages │
╰────────────────────────────╯
just install # Install with dev dependencies
just lint # Run linting checks
just typecheck # Run type checking
just format # Format code with ruff
just check # Run all quality checks
just migration "msg" # Create database migration
```
### Memory json
```bash
➜ basic-memory import memory-json
Importing from memory.json...writing to .../basic-memory
Reading memory.json... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100%
Creating entities... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100%
╭──────────────────────╮
│ Import complete! │
│ │
│ Created 126 entities │
│ Added 252 relations │
╰──────────────────────╯
```
See the [justfile](justfile) for the complete list of development commands.
## License
AGPL-3.0
AGPL-3.0
Contributions are welcome. See the [Contributing](CONTRIBUTING.md) guide for info about setting up the project locally
and submitting PRs.
## Star History
<a href="https://www.star-history.com/#basicmachines-co/basic-memory&Date">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/svg?repos=basicmachines-co/basic-memory&type=Date&theme=dark" />
<source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/svg?repos=basicmachines-co/basic-memory&type=Date" />
<img alt="Star History Chart" src="https://api.star-history.com/svg?repos=basicmachines-co/basic-memory&type=Date" />
</picture>
</a>
Built with ♥️ by Basic Machines
+13
View File
@@ -0,0 +1,13 @@
# Security Policy
## Supported Versions
| Version | Supported |
| ------- | ------------------ |
| 0.x.x | :white_check_mark: |
## Reporting a Vulnerability
Use this section to tell people how to report a vulnerability.
If you find a vulnerability, please contact hello@basicmachines.co
-1419
View File
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,23 @@
{
"name": "basicmachines",
"owner": {
"name": "Basic Machines",
"email": "hello@basicmachines.co"
},
"metadata": {
"description": "Official plugins from Basic Machines for knowledge management and AI-assisted development",
"version": "0.1.0"
},
"plugins": [
{
"name": "basic-memory",
"source": ".",
"description": "Skills, commands, and hooks for Basic Memory MCP - capture knowledge, continue conversations, and follow spec-driven development",
"version": "0.1.0",
"author": {
"name": "Basic Machines"
},
"keywords": ["memory", "knowledge", "mcp", "specs", "context"]
}
]
}
@@ -0,0 +1,9 @@
{
"name": "basic-memory",
"description": "Claude Code skills for Basic Memory - capture knowledge, continue conversations, and follow spec-driven development using the Basic Memory MCP server",
"version": "0.1.0",
"author": {
"name": "Basic Machines"
},
"repository": "https://github.com/basicmachines-co/basic-memory"
}
+313
View File
@@ -0,0 +1,313 @@
# Basic Memory Plugin for Claude Code
This plugin provides skills, commands, and hooks for working with [Basic Memory](https://basicmemory.io) - a local-first knowledge management system built on the Model Context Protocol (MCP).
## Prerequisites
You need the Basic Memory MCP server running. Install it via:
```bash
# Install basic-memory
pip install basic-memory
# Or with pipx
pipx install basic-memory
```
Then add it to your Claude Code MCP configuration.
## Installation
### Add the Marketplace
```
/plugin marketplace add basicmachines-co/basic-memory/claude-code-plugin
```
### Install the Plugin
```
/plugin install basic-memory@basicmachines
```
### Or via Repository Settings
Add to your `.claude/settings.json`:
```json
{
"plugins": {
"extraKnownMarketplaces": {
"basicmachines": {
"source": {
"source": "github",
"repo": "basicmachines-co/basic-memory",
"path": "claude-code-plugin"
}
}
},
"installed": ["basic-memory@basicmachines"]
}
}
```
---
## Slash Commands
User-invoked commands for explicit interaction with Basic Memory.
### `/remember [title] [folder]`
Capture insights, decisions, or learnings from the current conversation.
```
/remember "FastAPI Async Pattern"
/remember "Auth Decision" decisions
```
Creates a structured note with:
- Context from the conversation
- Observations with `[decision]`, `[insight]`, `[pattern]` categories
- Relations linking to related concepts
### `/continue [topic]`
Resume previous work by building context from Basic Memory.
```
/continue postgres migration
/continue SPEC-24
/continue
```
If no topic is provided, shows recent activity and asks what to dive into.
### `/context <memory://url> [depth] [timeframe]`
Build context from a specific memory:// URL.
```
/context memory://SPEC-24
/context memory://architecture/* 3 2weeks
```
### `/recent [timeframe] [project]`
Show recent activity in Basic Memory.
```
/recent
/recent 1week
/recent today specs
```
### `/organize [action] [project]`
Organize and maintain your knowledge graph.
```
/organize # Quick health check
/organize orphans # Find unlinked notes
/organize duplicates # Find similar notes
/organize relations "Note" # Suggest links for a note
/organize tags # Review tag consistency
```
Actions:
- `health` - Overview of knowledge base status (default)
- `orphans` - Find notes with no relations
- `duplicates` - Find overlapping notes
- `relations` - Suggest connections
- `tags` - Review tag consistency
### `/research <topic> [folder]`
Research a topic and save a structured report to Basic Memory.
```
/research MCP protocol
/research "database migrations"
/research "auth options" decisions
```
Produces a report with:
- Summary and key findings
- Analysis and recommendations
- Sources and related notes
- Saved to `research/` folder by default
---
## Skills
Model-invoked capabilities that Claude uses automatically based on context.
### knowledge-capture
Automatically captures insights, decisions, and learnings into structured notes.
**Triggers when:**
- Important decisions are made
- Technical insights are discovered
- Problems are solved
- Design trade-offs are discussed
### continue-conversation
Resumes previous work by building context from the knowledge graph.
**Triggers when:**
- Starting a new session
- User mentions previous work ("continue with...", "back to...")
- Need context about ongoing projects
### spec-driven-development
Guides implementation based on specifications stored in Basic Memory.
**Triggers when:**
- Implementing a feature defined by a spec
- Creating new specifications
- Reviewing implementation against criteria
### edit-note
Interactively edit notes using MCP tools in a conversational workflow.
**Triggers when:**
- User wants to edit, update, or modify a note
- User asks to change specific content in a note
- User wants to add observations or relations
**How it works:**
1. Fetches the note via MCP
2. Shows current content
3. Applies edits using `edit_note` operations (append, prepend, find_replace, replace_section)
4. Shows the updated result
**Best for:** Cloud users or when you want conversational editing.
### edit-note-local
Edit notes directly as local markdown files with automatic sync.
**Triggers when:**
- User has local Basic Memory installation
- User wants to make substantial file edits
- User prefers working with full file content
**How it works:**
1. Finds the note's file path via MCP
2. Uses Claude Code's Read/Edit/Write tools on the actual file
3. Basic Memory's `sync --watch` picks up changes automatically
**Best for:** Local users who want full file access and git integration.
### knowledge-organize
Help organize, link, and maintain the knowledge graph.
**Triggers when:**
- User wants to organize their notes
- User asks about orphan or unlinked notes
- User wants to find connections between notes
- User mentions duplicates or similar notes
- User asks for help with folder organization
**Capabilities:**
- **Find orphan notes** - Identify notes with no relations
- **Suggest relations** - Propose meaningful links between notes
- **Identify duplicates** - Find notes covering similar topics
- **Folder organization** - Review and suggest folder structure
- **Tag consistency** - Normalize and improve tagging
- **Create index notes** - Generate hub notes linking related topics
- **Enrich sparse notes** - Suggest observations and structure
**Best for:** Periodic knowledge base maintenance and improving discoverability.
### research
Research topics thoroughly and produce structured reports saved to Basic Memory.
**Triggers when:**
- User asks to research or investigate something
- User wants to understand a concept or technology
- User needs context before making a decision
- Phrases like "research", "look into", "explore", "investigate"
**What it produces:**
- Structured report with summary, findings, and analysis
- Recommendations when applicable
- Links to sources and related notes
- Saved to `research/` folder
**Best for:** Building knowledge base through investigation and documentation.
---
## Hooks
Automated behaviors that enhance the Basic Memory workflow.
### PostToolUse: write_note
Confirms when notes are saved to Basic Memory.
### Stop
After significant conversations, suggests using `/remember` to capture valuable insights (only when genuinely useful).
---
## MCP Tools Used
This plugin leverages Basic Memory's MCP tools:
| Tool | Purpose |
|------|---------|
| `write_note` | Create/update markdown notes |
| `read_note` | Read notes by title or permalink |
| `search_notes` | Full-text search across content |
| `build_context` | Navigate knowledge graph via memory:// URLs |
| `recent_activity` | Get recently updated information |
| `edit_note` | Incrementally update notes |
---
## Plugin Structure
```
claude-code-plugin/
├── .claude-plugin/
│ ├── plugin.json # Plugin manifest
│ └── marketplace.json # Self-hosted marketplace
├── commands/
│ ├── remember.md # /remember command
│ ├── continue.md # /continue command
│ ├── context.md # /context command
│ ├── recent.md # /recent command
│ ├── organize.md # /organize command
│ └── research.md # /research command
├── skills/
│ ├── knowledge-capture/
│ ├── continue-conversation/
│ ├── spec-driven-development/
│ ├── edit-note/
│ ├── edit-note-local/
│ ├── knowledge-organize/
│ └── research/
├── hooks/
│ └── hooks.json # Hook definitions
├── README.md # Quick start guide
└── PLUGIN.md # Full documentation
```
---
## Related
- [Basic Memory Documentation](https://docs.basicmemory.io)
- [Basic Memory GitHub](https://github.com/basicmachines-co/basic-memory)
- [Model Context Protocol](https://modelcontextprotocol.io)
- [Claude Code Plugins](https://code.claude.com/docs/en/plugins)
+100
View File
@@ -0,0 +1,100 @@
# Basic Memory Plugin for Claude Code
A Claude Code plugin that integrates [Basic Memory](https://basicmemory.io) - a local-first knowledge management system built on the Model Context Protocol (MCP).
## What This Plugin Does
This plugin helps Claude Code work seamlessly with your Basic Memory knowledge base:
- **Capture knowledge** from conversations automatically
- **Resume previous work** by building context from your knowledge graph
- **Edit notes** interactively through conversation
- **Organize your knowledge** by finding orphans, suggesting links, and maintaining structure
## Installation
### 1. Install Basic Memory
```bash
pip install basic-memory
# or
pipx install basic-memory
```
### 2. Add the Marketplace
```
/plugin marketplace add basicmachines-co/basic-memory/claude-code-plugin
```
### 3. Install the Plugin
```
/plugin install basic-memory@basicmachines
```
## Commands
| Command | Description |
|---------|-------------|
| `/remember [title]` | Capture insights from the current conversation |
| `/continue [topic]` | Resume previous work with context |
| `/context <memory://url>` | Build context from a specific note |
| `/recent [timeframe]` | Show recent activity |
| `/organize [action]` | Maintain your knowledge graph |
| `/research <topic>` | Research a topic and save a report |
### Examples
```bash
# Capture what we just discussed
/remember "Database Design Decision"
# Pick up where we left off
/continue postgres migration
# Check recent changes
/recent 1week
# Find orphan notes and suggest links
/organize orphans
# Research a topic and save findings
/research "MCP protocol"
```
## Skills
Skills are model-invoked - Claude uses them automatically when the context fits.
| Skill | What It Does |
|-------|--------------|
| `knowledge-capture` | Auto-captures decisions and insights into structured notes |
| `continue-conversation` | Builds context when resuming previous work |
| `spec-driven-development` | Guides implementation based on specs in Basic Memory |
| `edit-note` | Edits notes via MCP tools (cloud-compatible) |
| `edit-note-local` | Edits notes as files (local installations) |
| `knowledge-organize` | Helps organize and link notes |
| `research` | Researches topics and produces saved reports |
## Hooks
| Event | Behavior |
|-------|----------|
| `PostToolUse: write_note` | Confirms when notes are saved |
| `Stop` | Suggests capturing valuable insights after conversations |
## Requirements
- [Claude Code](https://claude.com/claude-code)
- [Basic Memory](https://basicmemory.io) with MCP server configured
## Documentation
- [Full Plugin Documentation](./PLUGIN.md)
- [Basic Memory Docs](https://docs.basicmemory.io)
- [Claude Code Plugins](https://code.claude.com/docs/en/plugins)
## License
MIT - See the [Basic Memory repository](https://github.com/basicmachines-co/basic-memory) for details.
+39
View File
@@ -0,0 +1,39 @@
---
description: Build context from a Basic Memory URL
argument-hint: <memory://url> [depth] [timeframe]
allowed-tools: mcp__basic-memory__build_context, mcp__basic-memory__read_note
---
# Context
Build context from a Basic Memory memory:// URL.
## Arguments
- `$1` - Memory URL (e.g., `memory://topic`, `memory://folder/*`, `memory://SPEC-24`)
- `$2` - Depth of relation traversal (optional, default: 2)
- `$3` - Timeframe for recent changes (optional, default: "7d")
## Your Task
Navigate the knowledge graph and build comprehensive context.
1. **Build context** using `mcp__basic-memory__build_context`:
- url: "$1"
- depth: $2 or 2
- timeframe: "$3" or "7d"
2. **Present the context**:
- Main note content
- Related notes found via relations
- Recent changes within timeframe
- Key observations and decisions
3. **Read additional notes** if needed for more detail.
## Memory URL Formats
- `memory://note-title` - Single note by title
- `memory://folder/*` - All notes in a folder
- `memory://SPEC-*` - Pattern matching
- `memory://specs/SPEC-24` - Note in specific project folder
+46
View File
@@ -0,0 +1,46 @@
---
description: Resume previous work from Basic Memory context
argument-hint: [topic]
allowed-tools: mcp__basic-memory__build_context, mcp__basic-memory__recent_activity, mcp__basic-memory__search_notes, mcp__basic-memory__read_note
---
# Continue
Resume previous work by building context from Basic Memory.
## Arguments
- `$ARGUMENTS` - Topic, note title, or search terms to find previous context
## Your Task
Build context to continue previous work seamlessly.
1. **Find relevant context**:
If a specific topic is provided ("$ARGUMENTS"):
- Search for matching notes: `mcp__basic-memory__search_notes`
- Build context from matches: `mcp__basic-memory__build_context`
- Read key notes for details: `mcp__basic-memory__read_note`
If no topic provided:
- Get recent activity: `mcp__basic-memory__recent_activity` with timeframe "3d"
- Present what's been happening
- Ask which topic to dive into
2. **Present context**:
- Summarize current state of the work
- Highlight recent changes or progress
- List open items or next steps
- Show related context from the knowledge graph
3. **Be ready to continue**:
- Understand what was done before
- Know what needs to happen next
- Have relevant context loaded
## Tips
- Use `memory://topic` URL format with `build_context`
- Check multiple projects if needed (main, specs)
- Follow relations to find connected knowledge
+87
View File
@@ -0,0 +1,87 @@
---
description: Organize and maintain your Basic Memory knowledge graph
argument-hint: [health|orphans|duplicates|relations|tags] [project]
allowed-tools: mcp__basic-memory__search_notes, mcp__basic-memory__read_note, mcp__basic-memory__list_directory, mcp__basic-memory__edit_note, mcp__basic-memory__write_note
---
# Organize
Help organize, link, and maintain your Basic Memory knowledge graph.
## Arguments
- `$1` - Action (optional): `health`, `orphans`, `duplicates`, `relations`, `tags` (default: `health`)
- `$2` - Project (optional): defaults to "main"
## Actions
### `/organize` or `/organize health`
Run a quick health check:
1. Count total notes
2. Identify orphan notes (no relations)
3. Check for potential duplicates
4. Show folder distribution
5. Report any issues found
### `/organize orphans`
Find and address orphan notes:
1. Search for notes with empty Relations sections
2. List orphans found
3. For each orphan, suggest potential relations based on content
4. Offer to add relations or create index notes
### `/organize duplicates`
Find potentially duplicate notes:
1. Search for notes with similar titles
2. Compare content for overlap
3. Suggest: merge, differentiate, or link with `supersedes`
### `/organize relations [note-title]`
Suggest relations for a specific note (or recent notes if not specified):
1. Read the target note
2. Search for related content
3. Suggest relation types:
- `relates-to` - General connection
- `extends` - Builds upon
- `implements` - Realizes concept
- `depends-on` - Requires understanding of
4. Offer to add selected relations
### `/organize tags`
Review tag consistency:
1. Gather all tags across notes
2. Find similar/duplicate tags (e.g., `arch` vs `architecture`)
3. Identify over-used or under-used tags
4. Suggest normalization
## Your Task
Execute: `/organize $ARGUMENTS`
Based on the action requested:
1. **Gather data** using search and list tools
2. **Analyze** for the specific issue (orphans, duplicates, etc.)
3. **Present findings** clearly with counts and examples
4. **Offer solutions** - ask before making changes
5. **Apply fixes** using edit_note or write_note when user approves
Always confirm before modifying notes. Show what will change and get approval.
## Examples
```
/organize # Quick health check
/organize health # Same as above
/organize orphans # Find unlinked notes
/organize duplicates # Find similar notes
/organize relations # Suggest links for recent notes
/organize relations "My Note" # Suggest links for specific note
/organize tags # Review tag consistency
/organize health specs # Health check on specs project
```
+40
View File
@@ -0,0 +1,40 @@
---
description: Show recent activity in Basic Memory
argument-hint: [timeframe] [project]
allowed-tools: mcp__basic-memory__recent_activity, mcp__basic-memory__read_note
---
# Recent
Show recent activity in Basic Memory.
## Arguments
- `$1` - Timeframe (optional): "today", "1d", "3d", "1 week", "2 weeks" (default: "3d")
- `$2` - Project (optional): "main", "specs", etc.
## Your Task
Show what's been happening in Basic Memory recently.
1. **Get recent activity** using `mcp__basic-memory__recent_activity`:
- timeframe: "$1" or "3d"
- project: "$2" or check all projects
2. **Present activity**:
- List recently modified notes
- Group by type or folder if helpful
- Highlight key changes
- Show dates of modifications
3. **Offer to dive deeper**:
- Ask if user wants to read any specific notes
- Suggest continuing work on active items
## Timeframe Examples
- `today` - Just today
- `1d` or `yesterday` - Last 24 hours
- `3d` - Last 3 days
- `1 week` - Last week
- `2 weeks` - Last 2 weeks
+43
View File
@@ -0,0 +1,43 @@
---
description: Capture insights, decisions, or learnings to Basic Memory
argument-hint: [title] [optional: folder]
allowed-tools: mcp__basic-memory__write_note, mcp__basic-memory__search_notes
---
# Remember
Capture what we just discussed into a Basic Memory note.
## Arguments
- `$1` - Title for the note (required)
- `$2` - Folder to save in (optional, defaults to "notes")
## Your Task
Create a structured note capturing the key insights from our conversation.
1. **Analyze the conversation** for:
- Decisions made
- Insights discovered
- Problems solved
- Patterns identified
- Trade-offs discussed
2. **Structure the note** with:
- Clear title: "$1" (or generate one if not provided)
- Context section explaining the situation
- Main content with key points
- Observations using `[category]` format:
- `[decision]` - Choices made
- `[insight]` - Understanding gained
- `[pattern]` - Reusable approaches
- `[learning]` - Lessons learned
- Relations to link related concepts with `[[WikiLinks]]`
3. **Save using** `mcp__basic-memory__write_note`:
- folder: "$2" or "notes"
- Include relevant tags
- Project: use "main" unless user specifies otherwise
4. **Confirm** what was captured and where it was saved.
+140
View File
@@ -0,0 +1,140 @@
---
description: Research a topic and save a structured report to Basic Memory
argument-hint: <topic> [folder]
allowed-tools: mcp__basic-memory__write_note, mcp__basic-memory__search_notes, mcp__basic-memory__read_note, mcp__basic-memory__build_context, WebSearch, WebFetch, Grep, Glob, Read
---
# Research
Research a topic thoroughly and produce a structured report saved to Basic Memory.
## Arguments
- `$1` - Topic to research (required)
- `$2` - Folder to save report (optional, default: "research")
## Your Task
Conduct thorough research on: **$ARGUMENTS**
### 1. Check Existing Knowledge
First, see what we already know:
```python
mcp__basic-memory__search_notes(query="$1", project="main")
```
Read any relevant existing notes to avoid duplicating research.
### 2. Gather Information
Depending on the topic, use appropriate tools:
**For codebase topics:**
- Search code with Grep/Glob
- Read relevant files
- Check tests for examples
**For external topics:**
- Use WebSearch for current information
- Fetch documentation with WebFetch
- Look for official sources
**For Basic Memory context:**
- Build context from related notes
- Check for prior decisions or research
### 3. Analyze Findings
Synthesize what you learned:
- Identify key concepts
- Note patterns and trade-offs
- Form recommendations if applicable
- Flag uncertainties
### 4. Produce Report
Create a structured report with this format:
```markdown
---
title: "Research: [Topic]"
type: research
tags:
- research
- [relevant-tags]
---
# Research: [Topic]
## Summary
[2-3 sentence executive summary]
## Research Question
[What we investigated and why]
## Key Findings
### [Finding 1]
[Details and evidence]
### [Finding 2]
[Details and evidence]
### [Finding 3]
[Details and evidence]
## Analysis
[Synthesis, patterns, trade-offs, recommendations]
## Open Questions
- [Areas needing more investigation]
## Sources
- [Links to sources]
- [[Related Notes]] from Basic Memory
## Observations
- [finding] Key insight #research
- [recommendation] Suggested approach based on research
## Relations
- researches [[Topic]]
- relates-to [[Related Concepts]]
```
### 5. Save Report
```python
mcp__basic-memory__write_note(
title="Research: $1",
content="[report content]",
folder="$2" or "research",
tags=["research", ...],
project="main"
)
```
### 6. Present Summary
After saving, present:
- Key findings summary
- Main recommendation (if applicable)
- Where the report was saved
- Offer to dive deeper into any aspect
## Examples
```
/research MCP protocol
/research "database migration patterns"
/research "authentication options" decisions
/research "React vs Vue" architecture
```
+26
View File
@@ -0,0 +1,26 @@
{
"hooks": {
"PostToolUse": [
{
"matcher": "mcp__basic-memory__write_note",
"hooks": [
{
"type": "command",
"command": "echo '✓ Note saved to Basic Memory'"
}
]
}
],
"Stop": [
{
"matcher": "*",
"hooks": [
{
"type": "prompt",
"prompt": "If this conversation contained valuable insights, decisions, or learnings that should be preserved, suggest using `/remember [title]` to capture them in Basic Memory. Only suggest this if there's genuinely valuable content worth preserving - don't suggest for trivial interactions."
}
]
}
]
}
}
@@ -0,0 +1,211 @@
---
name: continue-conversation
description: Resume previous work by building context from Basic Memory knowledge graph using memory URLs and recent activity
---
# Continue Conversation
This skill helps you resume previous work by building context from the Basic Memory knowledge graph, enabling seamless continuation across sessions.
## When to Use
Use this skill when:
- Starting a new session and need to pick up where you left off
- User mentions previous work ("continue with...", "back to...", "where were we with...")
- Need context about ongoing projects or specs
- User asks about something discussed in a previous conversation
- Working on a multi-session task
## Building Context
### 1. Identify What to Continue
Ask if unclear:
- What topic or project to resume?
- What timeframe to look at?
- Any specific aspect to focus on?
### 2. Gather Context with MCP Tools
**Option A: Known Topic - Use build_context**
```python
# Navigate knowledge graph from a known starting point
mcp__basic-memory__build_context(
url="memory://topic-or-note-name",
depth=2, # How many relation hops to follow
timeframe="7d", # Recent changes
project="main" # or "specs" for specifications
)
```
Memory URL formats:
- `memory://note-title` - Single note
- `memory://folder/*` - All notes in folder
- `memory://specs/SPEC-24*` - Pattern matching
**Option B: Recent Activity - What's been happening?**
```python
# See what's changed recently
mcp__basic-memory__recent_activity(
timeframe="3d", # "1d", "1 week", "2 weeks"
depth=1,
project="main"
)
```
**Option C: Search for Context**
```python
# Find relevant notes
mcp__basic-memory__search_notes(
query="search terms",
page_size=10,
project="main"
)
```
### 3. Read Key Notes
Once you identify relevant notes:
```python
mcp__basic-memory__read_note(
identifier="note-title-or-permalink",
project="main"
)
```
### 4. Present Context to User
Summarize what you found:
- Current state of the work
- Recent changes or progress
- Open items or next steps
- Related context that might be helpful
## Context Strategies by Scenario
### Resuming a Spec Implementation
```python
# 1. Read the spec
mcp__basic-memory__read_note(
identifier="SPEC-24: Postgres Database Migration",
project="specs"
)
# 2. Check recent activity on related topics
mcp__basic-memory__build_context(
url="memory://SPEC-24*",
timeframe="7d",
project="specs"
)
# 3. Look at what's been done in the codebase
# (Use regular file tools for this)
```
### Continuing General Work
```python
# 1. Check recent activity across projects
mcp__basic-memory__recent_activity(
timeframe="3d",
project="main"
)
# 2. Read any notes from recent sessions
mcp__basic-memory__read_note(
identifier="relevant-note",
project="main"
)
```
### Following Up on a Topic
```python
# 1. Search for the topic
mcp__basic-memory__search_notes(
query="topic keywords",
project="main"
)
# 2. Build context from best match
mcp__basic-memory__build_context(
url="memory://found-note-permalink",
depth=2,
project="main"
)
```
## Timeframe Reference
Natural language timeframes:
- `"today"` - Current day
- `"yesterday"` - Previous day
- `"3d"` or `"3 days"` - Last 3 days
- `"1 week"` or `"7d"` - Last week
- `"2 weeks"` - Last 2 weeks
- `"1 month"` - Last month
## Project Reference
Common projects:
- `main` - Primary knowledge base
- `specs` - Specifications and design docs
- `basic-memory-llc` - Business/company notes
- `getting-started` - Tutorial content
List available projects:
```python
mcp__basic-memory__list_memory_projects()
```
## Example Conversations
### User: "Let's continue with the Postgres migration"
```
1. Read SPEC-24 from specs project
2. Check for related notes about implementation progress
3. Summarize:
- Spec overview and goals
- What's been completed (checkmarks)
- What's pending (checkboxes)
- Any blockers or decisions needed
```
### User: "What was I working on yesterday?"
```
1. Get recent activity for last 2 days
2. List modified notes with brief descriptions
3. Ask which topic to dive into
```
### User: "Back to the async client pattern"
```
1. Search for "async client pattern"
2. Build context from matching note
3. Include related notes via relations
4. Present the full picture
```
## Best Practices
1. **Start broad, then narrow** - Get overview first, then specific details
2. **Follow relations** - Knowledge graph connections are valuable
3. **Check multiple projects** - Specs might be separate from implementation notes
4. **Present incrementally** - Share what you find as you go
5. **Confirm understanding** - Verify the context is what user needs
6. **Update as you go** - Capture new progress in notes during the session
## Combining with Other Skills
After building context, you might:
- Use **knowledge-capture** to document new progress
- Use **spec-driven-development** if continuing a spec implementation
- Create new notes linking to the context you gathered
@@ -0,0 +1,261 @@
---
name: edit-note-local
description: Edit Basic Memory notes directly as local files - enables full file editing with automatic sync (local installations only)
---
# Edit Note Local
This skill enables direct file-based editing of Basic Memory notes. It works by editing the actual markdown files in the knowledge base, which Basic Memory's sync service automatically picks up. This provides a more seamless editing experience for local installations.
## When to Use
Use this skill when:
- User has a local Basic Memory installation (not cloud-only)
- User wants to make substantial edits to a note
- User prefers working with the full file content
- User wants changes to sync automatically via `basic-memory sync --watch`
**Note:** This skill requires local file access. For cloud-only users, use the `edit-note` skill instead.
## Editing Workflow
### 1. Find the Note's File Path
First, get the note metadata to find its file location:
```python
# Search for the note
mcp__basic-memory__search_notes(
query="note title or keywords",
project="main"
)
# Read the note to get file_path from metadata
mcp__basic-memory__read_note(
identifier="note-title",
project="main"
)
```
The response includes `file_path` which gives the relative path within the knowledge base.
### 2. Determine Full File Path
Basic Memory projects have a root directory. Common locations:
- Default: `~/basic-memory/`
- Custom: Check project configuration
Construct the full path:
```
{project_root}/{file_path}
```
For example:
- Project root: `/Users/username/basic-memory`
- File path from note: `notes/My Note.md`
- Full path: `/Users/username/basic-memory/notes/My Note.md`
### 3. Read the File
Use Claude Code's Read tool to get the full file content:
```python
Read(file_path="/Users/username/basic-memory/notes/My Note.md")
```
Display the content to the user, explaining the structure:
- Frontmatter (YAML between `---` markers)
- Main content
- Observations section
- Relations section
### 4. Edit the File
Use Claude Code's Edit tool for precise changes:
```python
Edit(
file_path="/Users/username/basic-memory/notes/My Note.md",
old_string="text to replace",
new_string="new text"
)
```
Or use Write for complete rewrites:
```python
Write(
file_path="/Users/username/basic-memory/notes/My Note.md",
content="Complete new file content..."
)
```
### 5. Sync Happens Automatically
If the user has `basic-memory sync --watch` running, changes are picked up automatically. Otherwise, they can run:
```bash
basic-memory sync
```
## File Structure Reference
Basic Memory notes follow this markdown structure:
```markdown
---
title: Note Title
type: note
permalink: note-title
tags:
- tag1
- tag2
---
# Note Title
## Context
Background and situation explanation.
## Main Content
The primary content of the note...
## Observations
- [category] Observation text #optional-tag
- [decision] A decision that was made #tag
- [insight] An insight discovered
## Relations
- relates-to [[Other Note]]
- implements [[Parent Concept]]
- learned-from [[Source Note]]
```
## Editing Patterns
### Edit Frontmatter Tags
```python
Edit(
file_path="/path/to/note.md",
old_string="tags:\n- old-tag",
new_string="tags:\n- old-tag\n- new-tag"
)
```
### Add New Section
```python
Edit(
file_path="/path/to/note.md",
old_string="## Observations",
new_string="## New Section\n\nNew content here.\n\n## Observations"
)
```
### Modify Observation
```python
Edit(
file_path="/path/to/note.md",
old_string="- [decision] Old decision",
new_string="- [decision] Updated decision with new info #updated"
)
```
### Add Relation
```python
Edit(
file_path="/path/to/note.md",
old_string="## Relations\n",
new_string="## Relations\n\n- relates-to [[New Related Note]]\n"
)
```
### Complete Rewrite
For major changes, read the file, construct new content preserving the frontmatter structure, and write:
```python
Write(
file_path="/path/to/note.md",
content="""---
title: Note Title
type: note
permalink: note-title
tags:
- updated
---
# Note Title
Completely rewritten content...
## Observations
- [rewrite] Complete rewrite of this note #major-update
## Relations
- updates [[Previous Version]]
"""
)
```
## Finding the Project Root
To find where Basic Memory stores files, you can:
1. **Check common locations:**
- `~/basic-memory/`
- `~/Documents/basic-memory/`
- Current working directory
2. **Use the list_directory tool:**
```python
mcp__basic-memory__list_directory(
dir_name="/",
project="main"
)
```
3. **Ask the user:** "Where is your Basic Memory knowledge base located?"
## Advantages of Local Editing
1. **Full file access** - Edit any part of the file including frontmatter
2. **Multi-line edits** - Make complex structural changes easily
3. **Batch operations** - Edit multiple files in sequence
4. **Version control** - Changes tracked by git if the folder is a repo
5. **Instant preview** - Use any markdown editor alongside
6. **Auto-sync** - `sync --watch` picks up changes automatically
## Best Practices
1. **Preserve frontmatter** - Don't break the YAML structure
2. **Keep valid markdown** - Maintain proper formatting
3. **Preserve permalinks** - Changing them can break links
4. **Show diffs** - Tell the user what changed
5. **Suggest sync** - Remind about `basic-memory sync` if not watching
6. **Handle missing files** - Check if file exists before editing
## Example Conversation
**User:** "I want to completely rewrite my architecture decision note"
**Claude:**
1. Searches for the note via MCP
2. Gets the file path
3. Reads the current file content
4. Asks: "Here's the current note. What would you like the new version to say?"
**User:** Provides new content
**Claude:**
1. Preserves the frontmatter (title, permalink, type)
2. Writes the new content using Write tool
3. Confirms: "Updated the file at `/path/to/note.md`. If you have sync --watch running, it's already indexed. Otherwise run `basic-memory sync`."
@@ -0,0 +1,209 @@
---
name: edit-note
description: Interactively edit Basic Memory notes using MCP tools - view, modify, and update notes in a conversational workflow (works with cloud and local)
---
# Edit Note
This skill enables interactive editing of Basic Memory notes using MCP tools. It works with both Basic Memory Cloud and local installations since it operates through the MCP interface rather than direct file access.
## When to Use
Use this skill when:
- User wants to edit an existing note
- User asks to update, change, or modify note content
- User wants to refine observations or relations in a note
- User says things like "edit my note about...", "update the...", "change X to Y in..."
## Editing Workflow
### 1. Fetch the Current Note
First, retrieve the note to show the user what exists:
```python
mcp__basic-memory__read_note(
identifier="Note Title or permalink",
project="main" # or specified project
)
```
Present the note content clearly, highlighting:
- Current title and metadata
- Main content sections
- Observations (with categories)
- Relations (with link targets)
### 2. Understand the Edit Request
Ask clarifying questions if needed:
- Which section to modify?
- What specifically to change?
- Add new content or replace existing?
### 3. Apply the Edit
Use the appropriate `edit_note` operation:
**Append** - Add content to the end:
```python
mcp__basic-memory__edit_note(
identifier="note-title",
operation="append",
content="\n\n## New Section\n\nNew content here...",
project="main"
)
```
**Prepend** - Add content to the beginning:
```python
mcp__basic-memory__edit_note(
identifier="note-title",
operation="prepend",
content="# Updated Header\n\n",
project="main"
)
```
**Find and Replace** - Replace specific text:
```python
mcp__basic-memory__edit_note(
identifier="note-title",
operation="find_replace",
find_text="old text to find",
content="new replacement text",
project="main"
)
```
**Replace Section** - Replace an entire section by heading:
```python
mcp__basic-memory__edit_note(
identifier="note-title",
operation="replace_section",
section="## Section Heading",
content="## Section Heading\n\nCompletely new section content...",
project="main"
)
```
### 4. Show the Result
After editing, fetch and display the updated note:
```python
mcp__basic-memory__read_note(
identifier="note-title",
project="main"
)
```
Highlight what changed so the user can verify.
## Edit Operations Reference
| Operation | Use Case | Required Parameters |
|-----------|----------|---------------------|
| `append` | Add to end | `content` |
| `prepend` | Add to beginning | `content` |
| `find_replace` | Change specific text | `find_text`, `content` |
| `replace_section` | Rewrite a section | `section`, `content` |
## Common Edit Patterns
### Adding a New Observation
```python
mcp__basic-memory__edit_note(
identifier="note-title",
operation="find_replace",
find_text="## Observations",
content="## Observations\n\n- [new-category] New observation here #tag",
project="main"
)
```
Or append to observations section:
```python
mcp__basic-memory__edit_note(
identifier="note-title",
operation="append",
content="\n- [insight] Additional insight discovered #tag",
project="main"
)
```
### Adding a New Relation
```python
mcp__basic-memory__edit_note(
identifier="note-title",
operation="find_replace",
find_text="## Relations",
content="## Relations\n\n- relates-to [[New Related Note]]",
project="main"
)
```
### Updating a Specific Observation
```python
mcp__basic-memory__edit_note(
identifier="note-title",
operation="find_replace",
find_text="- [decision] Old decision text",
content="- [decision] Updated decision with new context #updated",
project="main"
)
```
### Rewriting the Context Section
```python
mcp__basic-memory__edit_note(
identifier="note-title",
operation="replace_section",
section="## Context",
content="## Context\n\nCompletely rewritten context explaining the new situation...",
project="main"
)
```
## Multi-Step Editing Session
For complex edits, work iteratively:
1. **Show current state** → Read and display the note
2. **First edit** → Apply one change
3. **Show result** → Display updated note
4. **Next edit** → Apply another change if needed
5. **Confirm complete** → Final display and confirmation
This keeps the user informed and allows course correction.
## Best Practices
1. **Always show before and after** - User should see what changed
2. **One edit at a time** - For complex changes, do multiple operations
3. **Preserve structure** - Maintain the note's markdown format
4. **Be careful with find_replace** - Ensure the find_text is unique
5. **Confirm destructive changes** - Ask before replacing large sections
6. **Keep observations formatted** - Maintain `[category]` prefix format
7. **Keep relations formatted** - Maintain `- relation-type [[Target]]` format
## Example Conversation
**User:** "Edit my note about the async client pattern - add an observation about testing"
**Claude:**
1. Fetches "Async Client Pattern" note
2. Displays current content
3. Asks: "What observation about testing would you like to add?"
**User:** "That the context manager pattern makes mocking easier in tests"
**Claude:**
1. Uses `edit_note` with `append` to add:
`- [testing] Context manager pattern simplifies mocking in unit tests #testability`
2. Fetches and displays updated note
3. Confirms: "Added the testing observation. Here's the updated note..."
@@ -0,0 +1,211 @@
---
name: knowledge-capture
description: Capture insights, decisions, and learnings from conversations into structured Basic Memory notes with observations and relations
---
# Knowledge Capture
This skill helps you capture valuable information from conversations into Basic Memory's knowledge graph using structured notes with observations and relations.
## When to Use
Use this skill when:
- Important decisions are made during a conversation
- Technical insights or patterns are discovered
- Problems are solved and the solution should be preserved
- Design trade-offs are discussed
- Architecture or implementation approaches are chosen
- Learnings from debugging or investigation emerge
## Capture Process
### 1. Identify Valuable Information
Look for:
- **Decisions**: Choices made and their rationale
- **Insights**: New understanding or "aha" moments
- **Patterns**: Reusable approaches or solutions
- **Trade-offs**: Options considered and why one was chosen
- **Learnings**: What worked, what didn't, and why
- **Context**: Background that would help future understanding
### 2. Structure the Note
Use Basic Memory's knowledge format:
```markdown
---
title: Descriptive Title
type: note
tags:
- relevant
- tags
---
# Title
## Context
Brief background explaining the situation.
## Content
Main content organized logically.
## Observations
- [decision] What was decided and why #tag
- [insight] Key understanding gained #tag
- [pattern] Reusable approach identified #tag
- [learning] What we learned from this #tag
- [tradeoff] Option A chosen over B because... #tag
## Relations
- relates-to [[Related Concept]]
- implements [[Parent Spec or Design]]
- learned-from [[Source of Learning]]
```
### 3. Choose Appropriate Categories
Common observation categories:
- `[decision]` - Choices made
- `[insight]` - Understanding gained
- `[pattern]` - Reusable approaches
- `[learning]` - Lessons learned
- `[tradeoff]` - Options weighed
- `[problem]` - Issues identified
- `[solution]` - Fixes applied
- `[architecture]` - Structural decisions
- `[implementation]` - Code-level choices
- `[constraint]` - Limitations discovered
- `[requirement]` - Needs identified
### 4. Create Meaningful Relations
Link to related knowledge:
- `relates-to` - General association
- `implements` - Realizes a spec or design
- `extends` - Builds upon existing concept
- `learned-from` - Source of insight
- `enables` - Makes something possible
- `depends-on` - Requires another concept
- `solves` - Addresses a problem
## MCP Tools to Use
```python
# Write a new note
mcp__basic-memory__write_note(
title="Your Note Title",
content="Full markdown content...",
folder="appropriate/folder",
tags=["tag1", "tag2"],
project="main" # or appropriate project
)
# Search for related notes to link
mcp__basic-memory__search_notes(
query="relevant terms",
project="main"
)
# Read existing notes for context
mcp__basic-memory__read_note(
identifier="note-title-or-permalink",
project="main"
)
```
## Folder Organization
Choose appropriate folders:
- `decisions/` - Architecture and design decisions
- `learnings/` - Insights and lessons learned
- `patterns/` - Reusable approaches
- `debug-logs/` - Problem investigations
- `conversations/` - Imported conversation summaries
## Examples
### Capturing a Technical Decision
```markdown
---
title: FastAPI Async Client Pattern
type: note
tags:
- architecture
- fastapi
- async
---
# FastAPI Async Client Pattern
## Context
During implementation of MCP tools, we needed to decide how to handle HTTP client lifecycle.
## Decision
Use context manager pattern for HTTP clients instead of module-level singletons.
## Rationale
- Proper resource management
- Supports three deployment modes (local ASGI, CLI cloud, cloud app)
- Auth happens at client creation, not per-request
- Enables dependency injection for testing
## Observations
- [decision] Context manager pattern for HTTP clients enables proper resource cleanup #architecture
- [pattern] Factory pattern allows different client configurations per deployment mode #flexibility
- [tradeoff] Slightly more verbose than singleton but much more flexible #engineering
## Relations
- implements [[SPEC-16 MCP Cloud Service Consolidation]]
- enables [[Cloud App Integration]]
```
### Capturing a Debugging Insight
```markdown
---
title: SQLite WAL Mode Performance Fix
type: note
tags:
- debugging
- sqlite
- performance
---
# SQLite WAL Mode Performance Fix
## Problem
Sync operations were slow with multiple concurrent writes.
## Investigation
Found that default SQLite journaling was causing lock contention.
## Solution
Enabled WAL (Write-Ahead Logging) mode for the database connection.
## Observations
- [problem] Default SQLite journaling causes lock contention under concurrent writes #performance
- [solution] WAL mode significantly improves concurrent write performance #sqlite
- [learning] Always consider WAL mode for SQLite in applications with concurrent access #database
## Relations
- solves [[Sync Performance Issues]]
- relates-to [[SPEC-19 Sync Performance]]
```
## Best Practices
1. **Capture immediately** - Write notes while context is fresh
2. **Be specific** - Include concrete details, not vague summaries
3. **Link liberally** - More relations = better knowledge graph
4. **Use tags** - Enable discovery via search
5. **Include context** - Future you won't remember the situation
6. **Prefer facts over opinions** - Observations should be verifiable
7. **Keep notes atomic** - One concept per note when possible
@@ -0,0 +1,283 @@
---
name: knowledge-organize
description: Help organize, link, and maintain the Basic Memory knowledge graph - find orphan notes, suggest relations, identify duplicates, and improve overall knowledge structure
---
# Knowledge Organize
This skill helps users maintain a healthy, well-connected knowledge graph. As notes accumulate, it becomes valuable to periodically organize, link, and curate the knowledge base.
## When to Use
Use this skill when:
- User asks to organize their notes
- User wants to find connections between notes
- User mentions orphan or unlinked notes
- User wants to clean up or improve their knowledge base
- User asks about duplicate or similar notes
- User wants help with folder organization
- User asks to review or audit their notes
- Phrases like "help me organize", "find related notes", "what's not linked", "clean up my notes"
## Organization Capabilities
### 1. Find Orphan Notes
Identify notes that have no relations to other notes - they're isolated in the knowledge graph.
```python
# Get all notes
mcp__basic-memory__search_notes(
query="*",
page_size=50,
project="main"
)
# For each note, check if it has relations
# Orphans have empty Relations sections
```
**What to do with orphans:**
- Suggest potential relations based on content similarity
- Ask if they should be linked to existing topics
- Propose creating hub notes to connect related orphans
### 2. Suggest Relations
Analyze note content and suggest meaningful connections.
```python
# Read a note
mcp__basic-memory__read_note(
identifier="note-to-analyze",
project="main"
)
# Search for potentially related notes
mcp__basic-memory__search_notes(
query="key terms from the note",
project="main"
)
# Suggest relations based on:
# - Shared topics or concepts
# - Complementary content (problem/solution, question/answer)
# - Sequential relationship (part 1, part 2)
# - Hierarchical (parent concept, child detail)
```
**Relation types to suggest:**
- `relates-to` - General topical connection
- `extends` - Builds upon or expands
- `implements` - Realizes a concept
- `depends-on` - Requires understanding of
- `contradicts` - Presents alternative view
- `learned-from` - Source of insight
- `enables` - Makes something possible
### 3. Identify Similar/Duplicate Notes
Find notes that may cover the same topic.
```python
# Search for notes with similar titles or content
mcp__basic-memory__search_notes(
query="topic keywords",
project="main"
)
# Compare results for overlap
# Look for:
# - Similar titles
# - Overlapping observations
# - Same tags
# - Related timestamps (created around same time)
```
**Actions for duplicates:**
- Merge into a single comprehensive note
- Link them with `supersedes` or `updates` relations
- Differentiate by adding context about their distinct focus
### 4. Folder Organization Review
Analyze folder structure and suggest improvements.
```python
# List directory structure
mcp__basic-memory__list_directory(
dir_name="/",
depth=3,
project="main"
)
# Identify:
# - Overcrowded folders
# - Single-note folders
# - Inconsistent naming
# - Notes that might belong elsewhere
```
**Organization suggestions:**
- Group related notes into topic folders
- Create subfolders for large categories
- Suggest consistent naming conventions
- Move misplaced notes
### 5. Tag Consistency
Review and normalize tags across notes.
```python
# Search notes to analyze tag patterns
mcp__basic-memory__search_notes(
query="*",
page_size=100,
project="main"
)
# Look for:
# - Similar tags (architecture vs arch)
# - Unused tags
# - Over-used generic tags
# - Missing tags on relevant notes
```
**Tag improvements:**
- Suggest tag standardization (pick one variant)
- Propose new tags for common themes
- Identify notes missing obvious tags
### 6. Create Index/Hub Notes
Generate notes that serve as navigation hubs for related topics.
```python
# After identifying a cluster of related notes
mcp__basic-memory__write_note(
title="Architecture Decisions Index",
content="""---
title: Architecture Decisions Index
type: index
tags:
- architecture
- index
---
# Architecture Decisions Index
A hub linking all architecture-related decisions and patterns.
## Decisions
- [[Database Selection Decision]]
- [[API Design Patterns]]
- [[Authentication Architecture]]
## Patterns
- [[Repository Pattern]]
- [[Async Client Pattern]]
## Observations
- [index] Central hub for architecture knowledge #navigation
## Relations
- indexes [[Architecture]]
""",
folder="indexes",
project="main"
)
```
### 7. Enrich Sparse Notes
Find notes lacking observations or structure and suggest improvements.
```python
# Read a sparse note
mcp__basic-memory__read_note(
identifier="sparse-note",
project="main"
)
# If missing:
# - Observations section → suggest categories
# - Relations section → suggest links
# - Tags → suggest relevant tags
# - Context → suggest adding background
```
## Organization Workflows
### Quick Health Check
A fast overview of knowledge base status:
1. Count total notes
2. Identify orphan count
3. List recently modified
4. Check for obvious duplicates
5. Report folder distribution
### Deep Organization Session
Thorough review and improvement:
1. **Audit phase** - Catalog all notes, identify issues
2. **Orphan phase** - Address unlinked notes
3. **Relation phase** - Suggest new connections
4. **Duplicate phase** - Merge or differentiate similar notes
5. **Structure phase** - Reorganize folders if needed
6. **Index phase** - Create hub notes for major topics
### Topic-Focused Organization
Organize around a specific subject:
1. Find all notes related to topic
2. Map existing relations
3. Identify gaps in the topic graph
4. Suggest new notes to fill gaps
5. Create topic index note
## Best Practices
1. **Work incrementally** - Don't reorganize everything at once
2. **Confirm before changing** - Always ask before moving/editing notes
3. **Preserve permalinks** - Moving is okay, changing permalinks breaks links
4. **Show the graph** - Help user visualize connections
5. **Explain suggestions** - Say why a relation makes sense
6. **Respect user's system** - Enhance their organization, don't impose a new one
## Example Conversations
**User:** "Help me organize my notes"
**Claude:**
1. Runs health check on the knowledge base
2. Reports: "You have 47 notes. I found 12 orphan notes and 3 potential duplicates."
3. Asks: "Would you like to start by connecting the orphan notes, or review the duplicates first?"
**User:** "Find notes that should be linked to my API design note"
**Claude:**
1. Reads the API design note
2. Searches for related content
3. Suggests: "I found 5 notes that could relate:
- 'REST Best Practices' → relates-to
- 'Authentication Flow' → implements
- 'Rate Limiting Decision' → extends
Would you like me to add any of these relations?"
**User:** "Are there any notes about similar topics?"
**Claude:**
1. Analyzes note titles and content
2. Identifies clusters of similar notes
3. Reports: "I found these potential overlaps:
- 'Auth Flow' and 'Authentication Design' cover similar ground
- 'DB Schema v1' and 'DB Schema v2' might need a 'supersedes' relation
Would you like to review any of these?"
+213
View File
@@ -0,0 +1,213 @@
---
name: research
description: Research a topic thoroughly and produce a structured report saved to Basic Memory - investigate concepts, gather context, and document findings
---
# Research
This skill helps conduct thorough research on a topic and produces a structured report that gets saved to Basic Memory for future reference.
## When to Use
Use this skill when:
- User asks to research or investigate something
- User wants to understand a concept, technology, or approach
- User needs context gathered before making a decision
- User asks "what is...", "how does... work", "explore...", "investigate..."
- User wants findings documented for later
- Phrases like "research this", "look into", "find out about", "explore options for"
## Research Process
### 1. Understand the Research Question
Clarify what specifically to investigate:
- What is the core question or topic?
- What scope - broad overview or deep dive?
- Any specific aspects to focus on?
- What will the research inform (a decision, implementation, understanding)?
### 2. Gather Information
Use available tools to collect information:
**For codebase research:**
- Search the codebase for relevant code
- Read documentation and comments
- Trace how things connect
- Look at tests for usage examples
**For concept research:**
- Use web search for current information
- Fetch documentation from official sources
- Look for examples and best practices
- Compare alternatives if relevant
**For Basic Memory context:**
```python
# Check what we already know
mcp__basic-memory__search_notes(
query="topic keywords",
project="main"
)
# Build context from related notes
mcp__basic-memory__build_context(
url="memory://related-topic",
depth=2,
project="main"
)
```
### 3. Analyze and Synthesize
Organize findings into coherent insights:
- Identify key concepts and how they relate
- Note patterns, trade-offs, and considerations
- Highlight what's most relevant to the user's needs
- Flag uncertainties or areas needing more investigation
### 4. Produce the Report
Create a structured research report:
```markdown
---
title: "Research: [Topic]"
type: research
tags:
- research
- [topic-tags]
---
# Research: [Topic]
## Summary
[2-3 sentence executive summary of findings]
## Research Question
[What we set out to understand]
## Key Findings
### [Finding 1]
[Details, evidence, implications]
### [Finding 2]
[Details, evidence, implications]
### [Finding 3]
[Details, evidence, implications]
## Analysis
[Synthesis of findings - patterns, trade-offs, recommendations]
## Open Questions
- [Things that need more investigation]
- [Uncertainties or assumptions]
## Sources
- [Where information came from]
- [[Related Note]] - relevant prior knowledge
## Observations
- [finding] Key insight discovered #research
- [pattern] Pattern identified during research
- [recommendation] Suggested approach based on findings
## Relations
- researches [[Topic]]
- informs [[Decision or Implementation]]
- relates-to [[Related Concepts]]
```
### 5. Save to Basic Memory
```python
mcp__basic-memory__write_note(
title="Research: [Topic]",
content="[Full report content]",
folder="research",
tags=["research", "topic-tags"],
project="main"
)
```
## Report Styles
Adjust based on the research type:
### Quick Investigation
- Focused summary
- 2-3 key findings
- Direct recommendation
- Saved to `research/` folder
### Deep Dive
- Comprehensive analysis
- Multiple sections
- Detailed evidence
- Comparison of options
- Saved to `research/` folder
### Decision Support
- Options evaluated
- Pros/cons for each
- Clear recommendation with rationale
- Saved to `decisions/` or `research/` folder
### Technical Exploration
- How it works
- Architecture/design
- Code examples
- Integration considerations
- Saved to `research/` folder
## Best Practices
1. **Start with what we know** - Check Basic Memory for existing context
2. **Be thorough but focused** - Cover the topic well without tangents
3. **Cite sources** - Link to where information came from
4. **Be honest about uncertainty** - Flag what's unclear or needs verification
5. **Make it actionable** - Include recommendations when appropriate
6. **Link to related knowledge** - Connect to existing notes
7. **Save for future reference** - Always save the report to Basic Memory
## Example Conversations
**User:** "Research how other projects handle database migrations"
**Claude:**
1. Searches codebase for migration patterns
2. Checks Basic Memory for related decisions
3. Looks up best practices online
4. Produces report comparing approaches
5. Saves to `research/Database Migration Approaches.md`
6. Presents summary with recommendation
**User:** "Investigate the MCP protocol"
**Claude:**
1. Fetches MCP documentation
2. Searches for examples in codebase
3. Checks Basic Memory for prior context
4. Produces comprehensive report on MCP
5. Saves to `research/MCP Protocol Overview.md`
6. Presents key concepts and how to use them
**User:** "Look into authentication options for the API"
**Claude:**
1. Researches common auth patterns (JWT, OAuth, API keys)
2. Checks existing codebase auth implementation
3. Evaluates trade-offs for the use case
4. Produces decision-support report
5. Saves to `research/API Authentication Options.md`
6. Recommends approach with rationale
@@ -0,0 +1,292 @@
---
name: spec-driven-development
description: Guide implementation based on specs stored in Basic Memory, following the SPEC-1 specification-driven development process
---
# Spec-Driven Development
This skill guides implementation work based on specifications stored in the Basic Memory "specs" project, following the process defined in SPEC-1.
## When to Use
Use this skill when:
- Implementing a feature defined by a spec
- Creating a new specification before implementation
- Reviewing implementation against spec criteria
- Need to understand what a spec requires
- Updating spec progress as work completes
## The Spec-Driven Process
From SPEC-1, the workflow is:
1. **Create** - Write spec as complete thought in Basic Memory "specs" project
2. **Discuss** - Iterate and refine the specification
3. **Implement** - Execute implementation directly
4. **Validate** - Review implementation against spec criteria
5. **Document** - Update spec with learnings and decisions
## Spec Structure
Every spec contains:
- **Why** - The reasoning and problem being solved
- **What** - What is affected or changed
- **How** - High-level approach to implementation
- **How to Evaluate** - Testing/validation procedure
### Progress Tracking Format
Specs use living documentation with checklists:
```markdown
### Feature Area
- ✅ Basic functionality implemented
- ✅ Props and events defined
- [ ] Add sorting controls
- [ ] Improve accessibility
- [x] Currently implementing responsive design
```
- `✅` - Completed items
- `[ ]` - Pending items
- `[x]` - In-progress items
## Working with Specs
### Reading a Spec
```python
# Get the full spec
mcp__basic-memory__read_note(
identifier="SPEC-24: Postgres Database Migration",
project="specs"
)
# Or search for it
mcp__basic-memory__search_notes(
query="postgres migration",
project="specs"
)
```
### Creating a New Spec
```python
# 1. First, find the next spec number
mcp__basic-memory__search_notes(
query="SPEC-",
project="specs"
)
# 2. Create the spec with proper structure
mcp__basic-memory__write_note(
title="SPEC-30: Your Feature Name",
content="""---
title: 'SPEC-30: Your Feature Name'
type: spec
tags:
- feature-area
- component
---
# SPEC-30: Your Feature Name
## Why
[Problem statement and motivation]
## What
[What is affected or changed]
- Affected areas
- Components involved
- Scope boundaries
## How (High Level)
[Implementation approach]
### Phase 1: Foundation
- [ ] Task 1
- [ ] Task 2
### Phase 2: Core Features
- [ ] Task 3
- [ ] Task 4
## How to Evaluate
### Success Criteria
- [ ] Criterion 1
- [ ] Criterion 2
### Testing Procedure
1. Step 1
2. Step 2
## Observations
- [goal] Primary objective #tag
- [constraint] Known limitation #tag
## Relations
- relates-to [[Related Spec]]
- depends-on [[Dependency]]
""",
folder="", # Root of specs project
project="specs"
)
```
### Updating Spec Progress
```python
# Mark items complete as you implement
mcp__basic-memory__edit_note(
identifier="SPEC-24: Postgres Database Migration",
operation="find_replace",
find_text="- [ ] Create migration scripts",
content="- ✅ Create migration scripts",
project="specs"
)
# Or add new observations
mcp__basic-memory__edit_note(
identifier="SPEC-24: Postgres Database Migration",
operation="append",
content="\n- [learning] Alembic autogenerate works well for model changes #migration",
project="specs"
)
```
### Reviewing Implementation
When reviewing against a spec:
1. **Read the spec's "How to Evaluate" section**
2. **Check each success criterion:**
- Functional completeness
- Test coverage (count test files, check categories)
- Code quality (TypeScript, linting, performance)
- Architecture compliance
- Documentation completeness
3. **Be honest** - Don't overstate completeness
4. **Document findings** - Update spec with review results
5. **Identify gaps** - Clearly note what still needs work
## Implementation Workflow
### Starting Implementation
1. **Read the spec thoroughly**
```python
mcp__basic-memory__read_note(
identifier="SPEC-XX: Feature Name",
project="specs"
)
```
2. **Understand dependencies**
- Check Relations section for dependencies
- Read related specs if needed
3. **Plan your approach**
- Break "How" section into concrete tasks
- Identify what to implement first
4. **Mark first item in-progress**
```python
mcp__basic-memory__edit_note(
identifier="SPEC-XX",
operation="find_replace",
find_text="- [ ] First task",
content="- [x] First task",
project="specs"
)
```
### During Implementation
1. **Update progress as you complete items**
2. **Add observations for decisions made**
3. **Note any deviations from the spec**
4. **Capture learnings that might help future specs**
### After Implementation
1. **Run full evaluation against criteria**
2. **Mark all completed items with ✅**
3. **Add final observations**
4. **Document any follow-up work needed**
## Spec Naming Convention
Format: `SPEC-X: Descriptive Title`
Examples:
- `SPEC-24: Postgres Database Migration`
- `SPEC-25: Cloud Index Service`
- `SPEC-26: Multi-User Security and Permissions`
## Common Spec Patterns
### Feature Spec
```markdown
## Why
User need or problem
## What
- New UI components
- API endpoints
- Database changes
## How
Implementation phases with checkboxes
```
### Architecture Spec
```markdown
## Why
Technical debt or scalability need
## What
- System components affected
- Data flow changes
- Integration points
## How
Migration strategy with rollback plan
```
### Process Spec
```markdown
## Why
Workflow improvement need
## What
- Process steps changed
- Tools involved
- Team impact
## How
Rollout plan and adoption strategy
```
## Best Practices
1. **Spec first, code second** - Write spec before implementation
2. **Keep specs living** - Update as understanding evolves
3. **Be specific in criteria** - Vague criteria = vague completion
4. **Link related specs** - Build the knowledge graph
5. **Capture decisions** - Future you will thank you
6. **Review honestly** - Incomplete is okay, dishonest isn't
7. **Close the loop** - Mark items done as you complete them
## Using with Slash Commands
The `/spec` command provides quick access:
- `/spec create [name]` - Create new specification
- `/spec status` - Show all spec statuses
- `/spec show [name]` - Read a specific spec
- `/spec review [name]` - Validate implementation
+42
View File
@@ -0,0 +1,42 @@
# Docker Compose configuration for Basic Memory with PostgreSQL
# Use this for local development and testing with Postgres backend
#
# Usage:
# docker-compose -f docker-compose-postgres.yml up -d
# docker-compose -f docker-compose-postgres.yml down
services:
postgres:
image: postgres:17
container_name: basic-memory-postgres
environment:
# Local development/test credentials - NOT for production
# These values are referenced by tests and justfile commands
POSTGRES_DB: basic_memory
POSTGRES_USER: basic_memory_user
POSTGRES_PASSWORD: dev_password # Simple password for local testing only
ports:
- "5433:5432"
volumes:
- postgres_data:/var/lib/postgresql/data
healthcheck:
test: ["CMD-SHELL", "pg_isready -U basic_memory_user -d basic_memory"]
interval: 10s
timeout: 5s
retries: 5
restart: unless-stopped
volumes:
# Named volume for Postgres data
postgres_data:
driver: local
# Named volume for persistent configuration
# Database will be stored in Postgres, not in this volume
basic-memory-config:
driver: local
# Network configuration (optional)
# networks:
# basic-memory-net:
# driver: bridge
+83
View File
@@ -0,0 +1,83 @@
# Docker Compose configuration for Basic Memory
# See docs/Docker.md for detailed setup instructions
version: '3.8'
services:
basic-memory:
# Use pre-built image (recommended for most users)
image: ghcr.io/basicmachines-co/basic-memory:latest
# Uncomment to build locally instead:
# build: .
container_name: basic-memory-server
# Volume mounts for knowledge directories and persistent data
volumes:
# Persistent storage for configuration and database
- basic-memory-config:/root/.basic-memory:rw
# Mount your knowledge directory (required)
# Change './knowledge' to your actual Obsidian vault or knowledge directory
- ./knowledge:/app/data:rw
# OPTIONAL: Mount additional knowledge directories for multiple projects
# - ./work-notes:/app/data/work:rw
# - ./personal-notes:/app/data/personal:rw
# You can edit the project config manually in the mounted config volume
# The default project will be configured to use /app/data
environment:
# Project configuration
- BASIC_MEMORY_DEFAULT_PROJECT=main
# Enable real-time file synchronization (recommended for Docker)
- BASIC_MEMORY_SYNC_CHANGES=true
# Logging configuration
- BASIC_MEMORY_LOG_LEVEL=INFO
# Sync delay in milliseconds (adjust for performance vs responsiveness)
- BASIC_MEMORY_SYNC_DELAY=1000
# Port exposure for HTTP transport (only needed if not using STDIO)
ports:
- "8000:8000"
# Command with SSE transport (configurable via environment variables above)
# IMPORTANT: The SSE and streamable-http endpoints are not secured
command: ["basic-memory", "mcp", "--transport", "sse", "--host", "0.0.0.0", "--port", "8000"]
# Container management
restart: unless-stopped
# Health monitoring
healthcheck:
test: ["CMD", "basic-memory", "--version"]
interval: 30s
timeout: 10s
retries: 3
start_period: 30s
# Optional: Resource limits
# deploy:
# resources:
# limits:
# memory: 512M
# cpus: '0.5'
# reservations:
# memory: 256M
# cpus: '0.25'
volumes:
# Named volume for persistent configuration and database
# This ensures your configuration and knowledge graph persist across container restarts
basic-memory-config:
driver: local
# Network configuration (optional)
# networks:
# basic-memory-net:
# driver: bridge
+365
View File
@@ -0,0 +1,365 @@
# Docker Setup Guide
Basic Memory can be run in Docker containers to provide a consistent, isolated environment for your knowledge management
system. This is particularly useful for integrating with existing Dockerized MCP servers or for deployment scenarios.
## Quick Start
### Option 1: Using Pre-built Images (Recommended)
Basic Memory provides pre-built Docker images on GitHub Container Registry that are automatically updated with each release.
1. **Use the official image directly:**
```bash
docker run -d \
--name basic-memory-server \
-p 8000:8000 \
-v /path/to/your/obsidian-vault:/app/data:rw \
-v basic-memory-config:/app/.basic-memory:rw \
ghcr.io/basicmachines-co/basic-memory:latest
```
2. **Or use Docker Compose with the pre-built image:**
```yaml
version: '3.8'
services:
basic-memory:
image: ghcr.io/basicmachines-co/basic-memory:latest
container_name: basic-memory-server
ports:
- "8000:8000"
volumes:
- /path/to/your/obsidian-vault:/app/data:rw
- basic-memory-config:/app/.basic-memory:rw
environment:
- BASIC_MEMORY_DEFAULT_PROJECT=main
restart: unless-stopped
```
### Option 2: Using Docker Compose (Building Locally)
1. **Clone the repository:**
```bash
git clone https://github.com/basicmachines-co/basic-memory.git
cd basic-memory
```
2. **Update the docker-compose.yml:**
Edit the volume mount to point to your Obsidian vault:
```yaml
volumes:
# Change './obsidian-vault' to your actual directory path
- /path/to/your/obsidian-vault:/app/data:rw
```
3. **Start the container:**
```bash
docker-compose up -d
```
### Option 3: Using Docker CLI
```bash
# Build the image
docker build -t basic-memory .
# Run with volume mounting
docker run -d \
--name basic-memory-server \
-v /path/to/your/obsidian-vault:/app/data:rw \
-v basic-memory-config:/app/.basic-memory:rw \
-e BASIC_MEMORY_DEFAULT_PROJECT=main \
basic-memory
```
## Configuration
### Volume Mounts
Basic Memory requires several volume mounts for proper operation:
1. **Knowledge Directory** (Required):
```yaml
- /path/to/your/obsidian-vault:/app/data:rw
```
Mount your Obsidian vault or knowledge base directory.
2. **Configuration and Database** (Recommended):
```yaml
- basic-memory-config:/app/.basic-memory:rw
```
Persistent storage for configuration and SQLite database.
You can edit the basic-memory config.json file located in the /app/.basic-memory/config.json after Basic Memory starts.
3. **Multiple Projects** (Optional):
```yaml
- /path/to/project1:/app/data/project1:rw
- /path/to/project2:/app/data/project2:rw
```
You can edit the basic-memory config.json file located in the /app/.basic-memory/config.json
## CLI Commands via Docker
You can run Basic Memory CLI commands inside the container using `docker exec`:
### Basic Commands
```bash
# Check status
docker exec basic-memory-server basic-memory status
# Sync files
docker exec basic-memory-server basic-memory sync
# Show help
docker exec basic-memory-server basic-memory --help
```
### Managing Projects with Volume Mounts
When using Docker volumes, you'll need to configure projects to point to your mounted directories:
1. **Check current configuration:**
```bash
docker exec basic-memory-server cat /app/.basic-memory/config.json
```
2. **Add a project for your mounted volume:**
```bash
# If you mounted /path/to/your/vault to /app/data
docker exec basic-memory-server basic-memory project create my-vault /app/data
# Set it as default
docker exec basic-memory-server basic-memory project set-default my-vault
```
3. **Sync the new project:**
```bash
docker exec basic-memory-server basic-memory sync
```
### Example: Setting up an Obsidian Vault
If you mounted your Obsidian vault like this in docker-compose.yml:
```yaml
volumes:
- /Users/yourname/Documents/ObsidianVault:/app/data:rw
```
Then configure it:
```bash
# Create project pointing to mounted vault
docker exec basic-memory-server basic-memory project create obsidian /app/data
# Set as default
docker exec basic-memory-server basic-memory project set-default obsidian
# Sync to index all files
docker exec basic-memory-server basic-memory sync
```
### Environment Variables
Configure Basic Memory using environment variables:
```yaml
environment:
# Default project
- BASIC_MEMORY_DEFAULT_PROJECT=main
# Enable real-time sync
- BASIC_MEMORY_SYNC_CHANGES=true
# Logging level
- BASIC_MEMORY_LOG_LEVEL=INFO
# Sync delay in milliseconds
- BASIC_MEMORY_SYNC_DELAY=1000
```
## File Permissions
### Linux/macOS
The Docker container now runs as a non-root user to avoid file ownership issues. By default, the container uses UID/GID 1000, but you can customize this to match your user:
```bash
# Build with custom UID/GID to match your user
docker build --build-arg UID=$(id -u) --build-arg GID=$(id -g) -t basic-memory .
# Or use docker-compose with build args
```
**Example docker-compose.yml with custom user:**
```yaml
version: '3.8'
services:
basic-memory:
build:
context: .
dockerfile: Dockerfile
args:
UID: 1000 # Replace with your UID
GID: 1000 # Replace with your GID
container_name: basic-memory-server
ports:
- "8000:8000"
volumes:
- /path/to/your/obsidian-vault:/app/data:rw
- basic-memory-config:/app/.basic-memory:rw
environment:
- BASIC_MEMORY_DEFAULT_PROJECT=main
restart: unless-stopped
```
**Using pre-built images:**
If using the pre-built image from GitHub Container Registry, files will be created with UID/GID 1000. You can either:
1. Change your local directory ownership to match:
```bash
sudo chown -R 1000:1000 /path/to/your/obsidian-vault
```
2. Or build your own image with custom UID/GID as shown above.
### Windows
When using Docker Desktop on Windows, ensure the directories are shared:
1. Open Docker Desktop
2. Go to Settings → Resources → File Sharing
3. Add your knowledge directory path
4. Apply & Restart
## Troubleshooting
### Common Issues
1. **File Watching Not Working:**
- Ensure volume mounts are read-write (`:rw`)
- Check directory permissions
- On Linux, may need to increase inotify limits:
```bash
echo fs.inotify.max_user_watches=524288 | sudo tee -a /etc/sysctl.conf
sudo sysctl -p
```
2. **Configuration Not Persisting:**
- Use named volumes for `/app/.basic-memory`
- Check volume mount permissions
3. **Network Connectivity:**
- For HTTP transport, ensure port 8000 is exposed
- Check firewall settings
### Debug Mode
Run with debug logging:
```yaml
environment:
- BASIC_MEMORY_LOG_LEVEL=DEBUG
```
View logs:
```bash
docker-compose logs -f basic-memory
```
## Security Considerations
1. **Docker Security:**
The container runs as a non-root user (UID/GID 1000 by default) for improved security. You can customize the user ID using build arguments to match your local user.
2. **Volume Permissions:**
Ensure mounted directories have appropriate permissions and don't expose sensitive data. With the non-root container, files will be created with the specified user ownership.
3. **Network Security:**
If using HTTP transport, consider using reverse proxy with SSL/TLS and authentication if the endpoint is available on
a network.
4. **IMPORTANT:** The HTTP endpoints have no authorization. They should not be exposed on a public network.
## Integration Examples
### Claude Desktop with Docker
The recommended way to connect Claude Desktop to the containerized Basic Memory is using `mcp-proxy`, which converts the HTTP transport to STDIO that Claude Desktop expects:
1. **Start the Docker container:**
```bash
docker-compose up -d
```
2. **Configure Claude Desktop** to use mcp-proxy:
```json
{
"mcpServers": {
"basic-memory": {
"command": "uvx",
"args": [
"mcp-proxy",
"http://localhost:8000/mcp"
]
}
}
}
```
## Support
For Docker-specific issues:
1. Check the [troubleshooting section](#troubleshooting) above
2. Review container logs: `docker-compose logs basic-memory`
3. Verify volume mounts: `docker inspect basic-memory-server`
4. Test file permissions: `docker exec basic-memory-server ls -la /app`
For general Basic Memory support, see the main [README](../README.md)
and [documentation](https://memory.basicmachines.co/).
## GitHub Container Registry Images
### Available Images
Pre-built Docker images are available on GitHub Container Registry at [`ghcr.io/basicmachines-co/basic-memory`](https://github.com/basicmachines-co/basic-memory/pkgs/container/basic-memory).
**Supported architectures:**
- `linux/amd64` (Intel/AMD x64)
- `linux/arm64` (ARM64, including Apple Silicon)
**Available tags:**
- `latest` - Latest stable release
- `v0.13.8`, `v0.13.7`, etc. - Specific version tags
- `v0.13`, `v0.12`, etc. - Major.minor tags
### Automated Builds
Docker images are automatically built and published when new releases are tagged:
1. **Release Process:** When a git tag matching `v*` (e.g., `v0.13.8`) is pushed, the CI workflow automatically:
- Builds multi-platform Docker images
- Pushes to GitHub Container Registry with appropriate tags
- Uses native GitHub integration for seamless publishing
2. **CI/CD Pipeline:** The Docker workflow includes:
- Multi-platform builds (AMD64 and ARM64)
- Layer caching for faster builds
- Automatic tagging with semantic versioning
- Security scanning and optimization
### Setup Requirements (For Maintainers)
GitHub Container Registry integration is automatic for this repository:
1. **No external setup required** - GHCR is natively integrated with GitHub
2. **Automatic permissions** - Uses `GITHUB_TOKEN` with `packages: write` permission
3. **Public by default** - Images are automatically public for public repositories
The Docker CI workflow (`.github/workflows/docker.yml`) handles everything automatically when version tags are pushed.
File diff suppressed because it is too large Load Diff
+241
View File
@@ -0,0 +1,241 @@
# Character Handling and Conflict Resolution
Basic Memory handles various character encoding scenarios and file naming conventions to provide consistent permalink generation and conflict resolution. This document explains how the system works and how to resolve common character-related issues.
## Overview
Basic Memory uses a sophisticated system to generate permalinks from file paths while maintaining consistency across different operating systems and character encodings. The system normalizes file paths and generates unique permalinks to prevent conflicts.
## Character Normalization Rules
### 1. Permalink Generation
When Basic Memory processes a file path, it applies these normalization rules:
```
Original: "Finance/My Investment Strategy.md"
Permalink: "finance/my-investment-strategy"
```
**Transformation process:**
1. Remove file extension (`.md`)
2. Convert to lowercase (case-insensitive)
3. Replace spaces with hyphens
4. Replace underscores with hyphens
5. Handle international characters (transliteration for Latin, preservation for non-Latin)
6. Convert camelCase to kebab-case
### 2. International Character Support
**Latin characters with diacritics** are transliterated:
- `ø``o` (Søren → soren)
- `ü``u` (Müller → muller)
- `é``e` (Café → cafe)
- `ñ``n` (Niño → nino)
**Non-Latin characters** are preserved:
- Chinese: `中文/测试文档.md``中文/测试文档`
- Japanese: `日本語/文書.md``日本語/文書`
## Common Conflict Scenarios
### 1. Hyphen vs Space Conflicts
**Problem:** Files with existing hyphens conflict with generated permalinks from spaces.
**Example:**
```
File 1: "basic memory bug.md" → permalink: "basic-memory-bug"
File 2: "basic-memory-bug.md" → permalink: "basic-memory-bug" (CONFLICT!)
```
**Resolution:** The system automatically resolves this by adding suffixes:
```
File 1: "basic memory bug.md" → permalink: "basic-memory-bug"
File 2: "basic-memory-bug.md" → permalink: "basic-memory-bug-1"
```
**Best Practice:** Choose consistent naming conventions within your project.
### 2. Case Sensitivity Conflicts
**Problem:** Different case variations that normalize to the same permalink.
**Example on macOS:**
```
Directory: Finance/investment.md
Directory: finance/investment.md (different on filesystem, same permalink)
```
**Resolution:** Basic Memory detects case conflicts and prevents them during sync operations with helpful error messages.
**Best Practice:** Use consistent casing for directory and file names.
### 3. Character Encoding Conflicts
**Problem:** Different Unicode normalizations of the same logical character.
**Example:**
```
File 1: "café.md" (é as single character)
File 2: "café.md" (e + combining accent)
```
**Resolution:** Basic Memory normalizes Unicode characters using NFD normalization to detect these conflicts.
### 4. Forward Slash Conflicts
**Problem:** Forward slashes in frontmatter or file names interpreted as path separators.
**Example:**
```yaml
---
permalink: finance/investment/strategy
---
```
**Resolution:** Basic Memory validates frontmatter permalinks and warns about path separator conflicts.
## Error Messages and Troubleshooting
### "UNIQUE constraint failed: entity.file_path, entity.project_id"
**Cause:** Two entities trying to use the same file path within a project.
**Common scenarios:**
1. File move operation where destination is already occupied
2. Case sensitivity differences on macOS
3. Character encoding conflicts
4. Concurrent file operations
**Resolution steps:**
1. Check for duplicate file names with different cases
2. Look for files with similar names but different character encodings
3. Rename conflicting files to have unique names
4. Run sync again after resolving conflicts
### "File path conflict detected during move"
**Cause:** Enhanced conflict detection preventing potential database integrity violations.
**What this means:** The system detected that moving a file would create a conflict before attempting the database operation.
**Resolution:** Follow the specific guidance in the error message, which will indicate the type of conflict detected.
## Best Practices
### 1. File Naming Conventions
**Recommended patterns:**
- Use consistent casing (prefer lowercase)
- Use hyphens instead of spaces for multi-word files
- Avoid special characters that could conflict with path separators
- Be consistent with directory structure casing
**Examples:**
```
✅ Good:
- finance/investment-strategy.md
- projects/basic-memory-features.md
- docs/api-reference.md
❌ Problematic:
- Finance/Investment Strategy.md (mixed case, spaces)
- finance/Investment Strategy.md (inconsistent case)
- docs/API/Reference.md (mixed case directories)
```
### 2. Permalink Management
**Custom permalinks in frontmatter:**
```yaml
---
type: knowledge
permalink: custom-permalink-name
---
```
**Guidelines:**
- Use lowercase permalinks
- Use hyphens for word separation
- Avoid path separators unless creating sub-paths
- Ensure uniqueness within your project
### 3. Directory Structure
**Consistent casing:**
```
✅ Good:
finance/
investment-strategies.md
portfolio-management.md
❌ Problematic:
Finance/ (capital F)
investment-strategies.md
finance/ (lowercase f)
portfolio-management.md
```
## Migration and Cleanup
### Identifying Conflicts
Use Basic Memory's built-in conflict detection:
```bash
# Sync will report conflicts
basic-memory sync
# Check sync status for warnings
basic-memory status
```
### Resolving Existing Conflicts
1. **Identify conflicting files** from sync error messages
2. **Choose consistent naming convention** for your project
3. **Rename files** to follow the convention
4. **Re-run sync** to verify resolution
### Bulk Renaming Strategy
For projects with many conflicts:
1. **Backup your project** before making changes
2. **Standardize on lowercase** file and directory names
3. **Replace spaces with hyphens** in file names
4. **Use consistent character encoding** (UTF-8)
5. **Test sync after each batch** of changes
## System Enhancements
### Recent Improvements (v0.13+)
1. **Enhanced conflict detection** before database operations
2. **Improved error messages** with specific resolution guidance
3. **Character normalization utilities** for consistent handling
4. **File swap detection** for complex move scenarios
5. **Proactive conflict warnings** during permalink resolution
### Monitoring and Logging
The system now provides detailed logging for conflict resolution:
```
DEBUG: Detected potential file path conflicts for 'Finance/Investment.md': ['finance/investment.md']
WARNING: File path conflict detected during move: entity_id=123 trying to move from 'old.md' to 'new.md'
```
These logs help identify and resolve conflicts before they cause sync failures.
## Support and Resources
If you encounter character-related conflicts not covered in this guide:
1. **Check the logs** for specific conflict details
2. **Review error messages** for resolution guidance
3. **Report issues** with examples of the conflicting files
4. **Consider the file naming best practices** outlined above
The Basic Memory system is designed to handle most character conflicts automatically while providing clear guidance for manual resolution when needed.
+726
View File
@@ -0,0 +1,726 @@
# Basic Memory Cloud CLI Guide
The Basic Memory Cloud CLI provides seamless integration between local and cloud knowledge bases using **project-scoped synchronization**. Each project can optionally sync with the cloud, giving you fine-grained control over what syncs and where.
## Overview
The cloud CLI enables you to:
- **Toggle cloud mode** - All regular `bm` commands work with cloud when enabled
- **Project-scoped sync** - Each project independently manages its sync configuration
- **Explicit operations** - Sync only what you want, when you want
- **Bidirectional sync** - Keep local and cloud in sync with rclone bisync
- **Offline access** - Work locally, sync when ready
## Prerequisites
Before using Basic Memory Cloud, you need:
- **Active Subscription**: An active Basic Memory Cloud subscription is required to access cloud features
- **Subscribe**: Visit [https://basicmemory.com/subscribe](https://basicmemory.com/subscribe) to sign up
If you attempt to log in without an active subscription, you'll receive a "Subscription Required" error with a link to subscribe.
## Architecture: Project-Scoped Sync
### The Problem
**Old approach (SPEC-8):** All projects lived in a single `~/basic-memory-cloud-sync/` directory. This caused:
- ❌ Directory conflicts between mount and bisync
- ❌ Auto-discovery creating phantom projects
- ❌ Confusion about what syncs and when
- ❌ All-or-nothing sync (couldn't sync just one project)
**New approach (SPEC-20):** Each project independently configures sync.
### How It Works
**Projects can exist in three states:**
1. **Cloud-only** - Project exists on cloud, no local copy
2. **Cloud + Local (synced)** - Project has a local working directory that syncs
3. **Local-only** - Project exists locally (when cloud mode is disabled)
**Example:**
```bash
# You have 3 projects on cloud:
# - research: wants local sync at ~/Documents/research
# - work: wants local sync at ~/work-notes
# - temp: cloud-only, no local sync needed
bm project add research --local-path ~/Documents/research
bm project add work --local-path ~/work-notes
bm project add temp # No local sync
# Now you can sync individually (after initial --resync):
bm project bisync --name research
bm project bisync --name work
# temp stays cloud-only
```
**What happens under the covers:**
- Config stores `cloud_projects` dict mapping project names to local paths
- Each project gets its own bisync state in `~/.basic-memory/bisync-state/{project}/`
- Rclone syncs using single remote: `basic-memory-cloud`
- Projects can live anywhere on your filesystem, not forced into sync directory
## Quick Start
### 1. Enable Cloud Mode
Authenticate and enable cloud mode:
```bash
bm cloud login
```
**What this does:**
1. Opens browser to Basic Memory Cloud authentication page
2. Stores authentication token in `~/.basic-memory/auth/token`
3. **Enables cloud mode** - all CLI commands now work against cloud
4. Validates your subscription status
**Result:** All `bm project`, `bm tools` commands now work with cloud.
### 2. Set Up Sync
Install rclone and configure credentials:
```bash
bm cloud setup
```
**What this does:**
1. Installs rclone automatically (if needed)
2. Fetches your tenant information from cloud
3. Generates scoped S3 credentials for sync
4. Configures single rclone remote: `basic-memory-cloud`
**Result:** You're ready to sync projects. No sync directories created yet - those come with project setup.
### 3. Add Projects with Sync
Create projects with optional local sync paths:
```bash
# Create cloud project without local sync
bm project add research
# Create cloud project WITH local sync
bm project add research --local-path ~/Documents/research
# Or configure sync for existing project
bm project sync-setup research ~/Documents/research
```
**What happens under the covers:**
When you add a project with `--local-path`:
1. Project created on cloud at `/app/data/research`
2. Local path stored in config: `cloud_projects.research.local_path = "~/Documents/research"`
3. Local directory created if it doesn't exist
4. Bisync state directory created at `~/.basic-memory/bisync-state/research/`
**Result:** Project is ready to sync, but no files synced yet.
### 4. Sync Your Project
Establish the initial sync baseline. **Best practice:** Always preview with `--dry-run` first:
```bash
# Step 1: Preview the initial sync (recommended)
bm project bisync --name research --resync --dry-run
# Step 2: If all looks good, run the actual sync
bm project bisync --name research --resync
```
**What happens under the covers:**
1. Rclone reads from `~/Documents/research` (local)
2. Connects to `basic-memory-cloud:bucket-name/app/data/research` (remote)
3. Creates bisync state files in `~/.basic-memory/bisync-state/research/`
4. Syncs files bidirectionally with settings:
- `conflict_resolve=newer` (most recent wins)
- `max_delete=25` (safety limit)
- Respects `.bmignore` patterns
**Result:** Local and cloud are in sync. Baseline established.
**Why `--resync`?** This is an rclone requirement for the first bisync run. It establishes the initial state that future syncs will compare against. After the first sync, never use `--resync` unless you need to force a new baseline.
See: https://rclone.org/bisync/#resync
```
--resync
This will effectively make both Path1 and Path2 filesystems contain a matching superset of all files. By default, Path2 files that do not exist in Path1 will be copied to Path1, and the process will then copy the Path1 tree to Path2.
```
### 5. Subsequent Syncs
After the first sync, just run bisync without `--resync`:
```bash
bm project bisync --name research
```
**What happens:**
1. Rclone compares local and cloud states
2. Syncs changes in both directions
3. Auto-resolves conflicts (newer file wins)
4. Updates `last_sync` timestamp in config
**Result:** Changes flow both ways - edit locally or in cloud, both stay in sync.
### 6. Verify Setup
Check status:
```bash
bm cloud status
```
You should see:
- `Mode: Cloud (enabled)`
- `Cloud instance is healthy`
- Instructions for project sync commands
## Working with Projects
### Understanding Project Commands
**Key concept:** When cloud mode is enabled, use regular `bm project` commands (not `bm cloud project`).
```bash
# In cloud mode:
bm project list # Lists cloud projects
bm project add research # Creates cloud project
# In local mode:
bm project list # Lists local projects
bm project add research ~/Documents/research # Creates local project
```
### Creating Projects
**Use case 1: Cloud-only project (no local sync)**
```bash
bm project add temp-notes
```
**What this does:**
- Creates project on cloud at `/app/data/temp-notes`
- No local directory created
- No sync configuration
**Result:** Project exists on cloud, accessible via MCP tools, but no local copy.
**Use case 2: Cloud project with local sync**
```bash
bm project add research --local-path ~/Documents/research
```
**What this does:**
- Creates project on cloud at `/app/data/research`
- Creates local directory `~/Documents/research`
- Stores sync config in `~/.basic-memory/config.json`
- Prepares for bisync (but doesn't sync yet)
**Result:** Project ready to sync. Run `bm project bisync --name research --resync` to establish baseline.
**Use case 3: Add sync to existing cloud project**
```bash
# Project already exists on cloud
bm project sync-setup research ~/Documents/research
```
**What this does:**
- Updates existing project's sync configuration
- Creates local directory
- Prepares for bisync
**Result:** Existing cloud project now has local sync path. Run bisync to pull files down.
### Listing Projects
View all projects:
```bash
bm project list
```
**What you see:**
- All projects in cloud (when cloud mode enabled)
- Default project marked
- Project paths shown
**Future:** Will show sync status (synced/not synced, last sync time).
## File Synchronization
### Understanding the Sync Commands
**There are three sync-related commands:**
1. `bm project sync` - One-way: local → cloud (make cloud match local)
2. `bm project bisync` - Two-way: local ↔ cloud (recommended)
3. `bm project check` - Verify files match (no changes)
### One-Way Sync: Local → Cloud
**Use case:** You made changes locally and want to push to cloud (overwrite cloud).
```bash
bm project sync --name research
```
**What happens:**
1. Reads files from `~/Documents/research` (local)
2. Uses rclone sync to make cloud identical to local
3. Respects `.bmignore` patterns
4. Shows progress bar
**Result:** Cloud now matches local exactly. Any cloud-only changes are overwritten.
**When to use:**
- You know local is the source of truth
- You want to force cloud to match local
- You don't care about cloud changes
### Two-Way Sync: Local ↔ Cloud (Recommended)
**Use case:** You edit files both locally and in cloud UI, want both to stay in sync.
```bash
# First time - establish baseline
bm project bisync --name research --resync
# Subsequent syncs
bm project bisync --name research
```
**What happens:**
1. Compares local and cloud states using bisync metadata
2. Syncs changes in both directions
3. Auto-resolves conflicts (newer file wins)
4. Detects excessive deletes and fails safely (max 25 files)
**Conflict resolution example:**
```bash
# Edit locally
echo "Local change" > ~/Documents/research/notes.md
# Edit same file in cloud UI
# Cloud now has: "Cloud change"
# Run bisync
bm project bisync --name research
# Result: Newer file wins (based on modification time)
# If cloud was more recent, cloud version kept
# If local was more recent, local version kept
```
**When to use:**
- Default workflow for most users
- You edit in multiple places
- You want automatic conflict resolution
### Verify Sync Integrity
**Use case:** Check if local and cloud match without making changes.
```bash
bm project check --name research
```
**What happens:**
1. Compares file checksums between local and cloud
2. Reports differences
3. No files transferred
**Result:** Shows which files differ. Run bisync to sync them.
```bash
# One-way check (faster)
bm project check --name research --one-way
```
### Preview Changes (Dry Run)
**Use case:** See what would change without actually syncing.
```bash
bm project bisync --name research --dry-run
```
**What happens:**
1. Runs bisync logic
2. Shows what would be transferred/deleted
3. No actual changes made
**Result:** Safe preview of sync operations.
### Advanced: List Remote Files
**Use case:** See what files exist on cloud without syncing.
```bash
# List all files in project
bm project ls --name research
# List files in subdirectory
bm project ls --name research --path subfolder
```
**What happens:**
1. Connects to cloud via rclone
2. Lists files in remote project path
3. No files transferred
**Result:** See cloud file listing.
## Multiple Projects
### Syncing Multiple Projects
**Use case:** You have several projects with local sync, want to sync all at once.
```bash
# Setup multiple projects
bm project add research --local-path ~/Documents/research
bm project add work --local-path ~/work-notes
bm project add personal --local-path ~/personal
# Establish baselines
bm project bisync --name research --resync
bm project bisync --name work --resync
bm project bisync --name personal --resync
# Daily workflow: sync everything
bm project bisync --name research
bm project bisync --name work
bm project bisync --name personal
```
**Future:** `--all` flag will sync all configured projects:
```bash
bm project bisync --all # Coming soon
```
### Mixed Usage
**Use case:** Some projects sync, some stay cloud-only.
```bash
# Projects with sync
bm project add research --local-path ~/Documents/research
bm project add work --local-path ~/work
# Cloud-only projects
bm project add archive
bm project add temp-notes
# Sync only the configured ones
bm project bisync --name research
bm project bisync --name work
# Archive and temp-notes stay cloud-only
```
**Result:** Fine-grained control over what syncs.
## Disable Cloud Mode
Return to local mode:
```bash
bm cloud logout
```
**What this does:**
1. Disables cloud mode in config
2. All commands now work locally
3. Auth token remains (can re-enable with login)
**Result:** All `bm` commands work with local projects again.
## Filter Configuration
### Understanding .bmignore
**The problem:** You don't want to sync everything (e.g., `.git`, `node_modules`, database files).
**The solution:** `.bmignore` file with gitignore-style patterns.
**Location:** `~/.basic-memory/.bmignore`
**Default patterns:**
```gitignore
# Version control
.git/**
# Python
__pycache__/**
*.pyc
.venv/**
venv/**
# Node.js
node_modules/**
# Basic Memory internals
memory.db/**
memory.db-shm/**
memory.db-wal/**
config.json/**
watch-status.json/**
.bmignore.rclone/**
# OS files
.DS_Store/**
Thumbs.db/**
# Environment files
.env/**
.env.local/**
```
**How it works:**
1. On first sync, `.bmignore` created with defaults
2. Patterns converted to rclone filter format (`.bmignore.rclone`)
3. Rclone uses filters during sync
4. Same patterns used by all projects
**Customizing:**
```bash
# Edit patterns
code ~/.basic-memory/.bmignore
# Add custom patterns
echo "*.tmp/**" >> ~/.basic-memory/.bmignore
# Next sync uses updated patterns
bm project bisync --name research
```
## Troubleshooting
### Authentication Issues
**Problem:** "Authentication failed" or "Invalid token"
**Solution:** Re-authenticate:
```bash
bm cloud logout
bm cloud login
```
### Subscription Issues
**Problem:** "Subscription Required" error
**Solution:**
1. Visit subscribe URL shown in error
2. Sign up for subscription
3. Run `bm cloud login` again
**Note:** Access is immediate when subscription becomes active.
### Bisync Initialization
**Problem:** "First bisync requires --resync"
**Explanation:** Bisync needs a baseline state before it can sync changes.
**Solution:**
```bash
bm project bisync --name research --resync
```
**What this does:**
- Establishes initial sync state
- Creates baseline in `~/.basic-memory/bisync-state/research/`
- Syncs all files bidirectionally
**Result:** Future syncs work without `--resync`.
### Empty Directory Issues
**Problem:** "Empty prior Path1 listing. Cannot sync to an empty directory"
**Explanation:** Rclone bisync doesn't work well with completely empty directories. It needs at least one file to establish a baseline.
**Solution:** Add at least one file before running `--resync`:
```bash
# Create a placeholder file
echo "# Research Notes" > ~/Documents/research/README.md
# Now run bisync
bm project bisync --name research --resync
```
**Why this happens:** Bisync creates listing files that track the state of each side. When both directories are completely empty, these listing files are considered invalid by rclone.
**Best practice:** Always have at least one file (like a README.md) in your project directory before setting up sync.
### Bisync State Corruption
**Problem:** Bisync fails with errors about corrupted state or listing files
**Explanation:** Sometimes bisync state can become inconsistent (e.g., after mixing dry-run and actual runs, or after manual file operations).
**Solution:** Clear bisync state and re-establish baseline:
```bash
# Clear bisync state
bm project bisync-reset research
# Re-establish baseline
bm project bisync --name research --resync
```
**What this does:**
- Removes all bisync metadata from `~/.basic-memory/bisync-state/research/`
- Forces fresh baseline on next `--resync`
- Safe operation (doesn't touch your files)
**Note:** This command also runs automatically when you remove a project to clean up state directories.
### Too Many Deletes
**Problem:** "Error: max delete limit (25) exceeded"
**Explanation:** Bisync detected you're about to delete more than 25 files. This is a safety check to prevent accidents.
**Solution 1:** Review what you're deleting, then force resync:
```bash
# Check what would be deleted
bm project bisync --name research --dry-run
# If correct, establish new baseline
bm project bisync --name research --resync
```
**Solution 2:** Use one-way sync if you know local is correct:
```bash
bm project sync --name research
```
### Project Not Configured for Sync
**Problem:** "Project research has no local_sync_path configured"
**Explanation:** Project exists on cloud but has no local sync path.
**Solution:**
```bash
bm project sync-setup research ~/Documents/research
bm project bisync --name research --resync
```
### Connection Issues
**Problem:** "Cannot connect to cloud instance"
**Solution:** Check status:
```bash
bm cloud status
```
If instance is down, wait a few minutes and retry.
## Security
- **Authentication**: OAuth 2.1 with PKCE flow
- **Tokens**: Stored securely in `~/.basic-memory/basic-memory-cloud.json`
- **Transport**: All data encrypted in transit (HTTPS)
- **Credentials**: Scoped S3 credentials (read-write to your tenant only)
- **Isolation**: Your data isolated from other tenants
- **Ignore patterns**: Sensitive files automatically excluded via `.bmignore`
## Command Reference
### Cloud Mode Management
```bash
bm cloud login # Authenticate and enable cloud mode
bm cloud logout # Disable cloud mode
bm cloud status # Check cloud mode and instance health
```
### Setup
```bash
bm cloud setup # Install rclone and configure credentials
```
### Project Management
When cloud mode is enabled:
```bash
bm project list # List cloud projects
bm project add <name> # Create cloud project (no sync)
bm project add <name> --local-path <path> # Create with local sync
bm project sync-setup <name> <path> # Add sync to existing project
bm project rm <name> # Delete project
```
### File Synchronization
```bash
# One-way sync (local → cloud)
bm project sync --name <project>
bm project sync --name <project> --dry-run
bm project sync --name <project> --verbose
# Two-way sync (local ↔ cloud) - Recommended
bm project bisync --name <project> # After first --resync
bm project bisync --name <project> --resync # First time / force baseline
bm project bisync --name <project> --dry-run
bm project bisync --name <project> --verbose
# Integrity check
bm project check --name <project>
bm project check --name <project> --one-way
# List remote files
bm project ls --name <project>
bm project ls --name <project> --path <subpath>
```
## Summary
**Basic Memory Cloud uses project-scoped sync:**
1. **Enable cloud mode** - `bm cloud login`
2. **Install rclone** - `bm cloud setup`
3. **Add projects with sync** - `bm project add research --local-path ~/Documents/research`
4. **Preview first sync** - `bm project bisync --name research --resync --dry-run`
5. **Establish baseline** - `bm project bisync --name research --resync`
6. **Daily workflow** - `bm project bisync --name research`
**Key benefits:**
- ✅ Each project independently syncs (or doesn't)
- ✅ Projects can live anywhere on disk
- ✅ Explicit sync operations (no magic)
- ✅ Safe by design (max delete limits, conflict resolution)
- ✅ Full offline access (work locally, sync when ready)
**Future enhancements:**
- `--all` flag to sync all configured projects
- Project list showing sync status
- Watch mode for automatic sync
Binary file not shown.
-26
View File
@@ -1,26 +0,0 @@
# Basic Memory Installer
This installer configures Basic Memory to work with Claude Desktop.
## Installation
1. Download the latest installer from the [releases page](https://github.com/basicmachines-co/basic-memory/releases)
2. Unzip the downloaded file
3. Since the app is currently unsigned, you'll need to:
On your Mac, choose Apple menu > System Settings, then click Privacy & Security in the sidebar. (You may need to
scroll down.)
Go to Security, then click Open.
Click Open Anyway.
This button is available for about an hour after you try to open the app.
Enter your login password, then click OK.
https://support.apple.com/guide/mac-help/apple-cant-check-app-for-malicious-software-mchleab3a043/mac
5. Restart Claude Desktop
The warning only appears the first time you open the app. Future updates will include proper code signing.
-64
View File
@@ -1,64 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<svg viewBox="0 0 512 512" xmlns="http://www.w3.org/2000/svg">
<!-- Background -->
<rect x="0" y="0" width="512" height="512" rx="64" fill="#111111"/>
<!-- Define arrowhead marker -->
<defs>
<marker id="arrowhead"
markerWidth="10"
markerHeight="10"
refX="8"
refY="5"
orient="auto">
<path d="M 0 0 L 10 5 L 0 10 Z"
fill="#00cc00"/>
</marker>
</defs>
<!-- State 1 (initial) -->
<circle cx="156" cy="256" r="30" fill="none" stroke="#00cc00" stroke-width="3"/>
<!-- State 2 (accept) -->
<circle cx="356" cy="176" r="34" fill="none" stroke="#00cc00" stroke-width="3"/>
<circle cx="356" cy="176" r="28" fill="none" stroke="#00cc00" stroke-width="3"/>
<!-- State 3 (accept) -->
<circle cx="356" cy="336" r="34" fill="none" stroke="#00cc00" stroke-width="3"/>
<circle cx="356" cy="336" r="28" fill="none" stroke="#00cc00" stroke-width="3"/>
<!-- Initial arrow -->
<path d="M 96 256 L 126 256"
stroke="#00cc00" stroke-width="3" fill="none"
marker-end="url(#arrowhead)"/>
<!-- State transitions -->
<!-- 1 -> 2 -->
<path d="M 180 240
Q 260 200, 320 176"
stroke="#00cc00" stroke-width="3" fill="none"
marker-end="url(#arrowhead)"/>
<!-- 1 -> 3 -->
<path d="M 180 272
Q 260 312, 320 336"
stroke="#00cc00" stroke-width="3" fill="none"
marker-end="url(#arrowhead)"/>
<!-- Self loops -->
<path d="M 356 142
Q 396 142, 396 176
Q 396 210, 356 210
Q 316 210, 316 176
Q 316 142, 356 142"
stroke="#00cc00" stroke-width="2" fill="none"
marker-end="url(#arrowhead)"/>
<path d="M 356 302
Q 396 302, 396 336
Q 396 370, 356 370
Q 316 370, 316 336
Q 316 302, 356 302"
stroke="#00cc00" stroke-width="2" fill="none"
marker-end="url(#arrowhead)"/>
</svg>

Before

Width:  |  Height:  |  Size: 2.0 KiB

-90
View File
@@ -1,90 +0,0 @@
import json
import subprocess
import sys
from pathlib import Path
# Use tkinter for GUI alerts on macOS
if sys.platform == "darwin":
import tkinter as tk
from tkinter import messagebox
def ensure_uv_installed():
"""Check if uv is installed, install if not."""
try:
subprocess.run(["uv", "--version"], capture_output=True, check=True)
except (subprocess.CalledProcessError, FileNotFoundError):
print("Installing uv package manager...")
subprocess.run(
[
"curl",
"-LsSf",
"https://astral.sh/uv/install.sh",
"|",
"sh",
],
shell=True,
)
def get_config_path():
"""Get Claude Desktop config path for current platform."""
if sys.platform == "darwin":
return Path.home() / "Library/Application Support/Claude/claude_desktop_config.json"
elif sys.platform == "win32":
return Path.home() / "AppData/Roaming/Claude/claude_desktop_config.json"
else:
raise RuntimeError(f"Unsupported platform: {sys.platform}")
def update_claude_config():
"""Update Claude Desktop config to include basic-memory."""
config_path = get_config_path()
config_path.parent.mkdir(parents=True, exist_ok=True)
# Load existing config or create new
if config_path.exists():
config = json.loads(config_path.read_text())
else:
config = {"mcpServers": {}}
# Add/update basic-memory config
config["mcpServers"]["basic-memory"] = {"command": "uvx", "args": ["basic-memory", "mcp"]}
# Write back config
config_path.write_text(json.dumps(config, indent=2))
def print_completion_message():
"""Show completion message with helpful tips."""
message = """Installation complete! Basic Memory is now available in Claude Desktop.
Please restart Claude Desktop for changes to take effect.
Quick Start:
1. You can run sync directly using: uvx basic-memory sync
2. Optionally, install globally with: uv pip install basic-memory
Built with ♥️ by Basic Machines."""
if sys.platform == "darwin":
# Show GUI message on macOS
root = tk.Tk()
root.withdraw() # Hide the main window
messagebox.showinfo("Basic Memory", message)
root.destroy()
else:
# Fallback to console output
print(message)
def main():
print("Welcome to Basic Memory installer")
ensure_uv_installed()
print("Configuring Claude Desktop...")
update_claude_config()
print_completion_message()
if __name__ == "__main__":
main()
-27
View File
@@ -1,27 +0,0 @@
#!/bin/bash
# Convert SVG to PNG at various required sizes
rsvg-convert -h 16 -w 16 icon.svg > icon_16x16.png
rsvg-convert -h 32 -w 32 icon.svg > icon_32x32.png
rsvg-convert -h 128 -w 128 icon.svg > icon_128x128.png
rsvg-convert -h 256 -w 256 icon.svg > icon_256x256.png
rsvg-convert -h 512 -w 512 icon.svg > icon_512x512.png
# Create iconset directory
mkdir -p Basic.iconset
# Move files into iconset with Mac-specific names
cp icon_16x16.png Basic.iconset/icon_16x16.png
cp icon_32x32.png Basic.iconset/icon_16x16@2x.png
cp icon_32x32.png Basic.iconset/icon_32x32.png
cp icon_128x128.png Basic.iconset/icon_32x32@2x.png
cp icon_256x256.png Basic.iconset/icon_128x128.png
cp icon_512x512.png Basic.iconset/icon_256x256.png
cp icon_512x512.png Basic.iconset/icon_512x512.png
# Convert iconset to icns
iconutil -c icns Basic.iconset
# Clean up
rm -rf Basic.iconset
rm icon_*.png
-40
View File
@@ -1,40 +0,0 @@
from cx_Freeze import setup, Executable
import sys
# Build options for all platforms
build_exe_options = {
"packages": ["json", "pathlib"],
"excludes": ["unittest", "pydoc", "test"],
}
# Platform-specific options
if sys.platform == "win32":
base = "Win32GUI" # Use GUI base for Windows
build_exe_options.update(
{
"include_msvcr": True,
}
)
target_name = "Basic Memory Installer.exe"
else: # darwin
base = None # Don't use GUI base for macOS
target_name = "Basic Memory Installer"
executables = [
Executable(script="installer.py", target_name=target_name, base=base, icon="Basic.icns")
]
setup(
name="basic-memory",
version=open("../pyproject.toml").read().split('version = "', 1)[1].split('"', 1)[0],
description="Basic Memory - Local-first knowledge management",
options={
"build_exe": build_exe_options,
"bdist_mac": {
"bundle_name": "Basic Memory Installer",
"iconfile": "Basic.icns",
"codesign_identity": "-", # Force ad-hoc signing
},
},
executables=executables,
)
+255
View File
@@ -0,0 +1,255 @@
# Basic Memory - Modern Command Runner
# Install dependencies
install:
uv pip install -e ".[dev]"
uv sync
@echo ""
@echo "💡 Remember to activate the virtual environment by running: source .venv/bin/activate"
# Run all tests with unified coverage report
test: test-unit test-int
# Run unit tests only (fast, no coverage)
test-unit:
uv run pytest -p pytest_mock -v --no-cov tests
# Run integration tests only (fast, no coverage)
test-int:
uv run pytest -p pytest_mock -v --no-cov test-int
# ==============================================================================
# DATABASE BACKEND TESTING
# ==============================================================================
# Basic Memory supports dual database backends (SQLite and Postgres).
# Tests are parametrized to run against both backends automatically.
#
# Quick Start:
# just test-sqlite # Run SQLite tests (default, no Docker needed)
# just test-postgres # Run Postgres tests (requires Docker)
#
# For Postgres tests, first start the database:
# docker-compose -f docker-compose-postgres.yml up -d
# ==============================================================================
# Run tests against SQLite only (default backend, skip Postgres/Benchmark tests)
# This is the fastest option and doesn't require any Docker setup.
# Use this for local development and quick feedback.
# Includes Windows-specific tests which will auto-skip on non-Windows platforms.
test-sqlite:
uv run pytest -p pytest_mock -v --no-cov -m "not postgres and not benchmark" tests test-int
# Run tests against Postgres only (requires docker-compose-postgres.yml up)
# First start Postgres: docker-compose -f docker-compose-postgres.yml up -d
# Tests will connect to localhost:5433/basic_memory_test
# To reset the database: just postgres-reset
test-postgres:
uv run pytest -p pytest_mock -v --no-cov -m "postgres and not benchmark" tests test-int
# Reset Postgres test database (drops and recreates schema)
# Useful when Alembic migration state gets out of sync during development
# Uses credentials from docker-compose-postgres.yml
postgres-reset:
docker exec basic-memory-postgres psql -U ${POSTGRES_USER:-basic_memory_user} -d ${POSTGRES_TEST_DB:-basic_memory_test} -c "DROP SCHEMA public CASCADE; CREATE SCHEMA public;"
@echo "✅ Postgres test database reset"
# Run Alembic migrations manually against Postgres test database
# Useful for debugging migration issues
# Uses credentials from docker-compose-postgres.yml (can override with env vars)
postgres-migrate:
@cd src/basic_memory/alembic && \
BASIC_MEMORY_DATABASE_BACKEND=postgres \
BASIC_MEMORY_DATABASE_URL=${POSTGRES_TEST_URL:-postgresql://basic_memory_user:dev_password@localhost:5433/basic_memory_test} \
uv run alembic upgrade head
@echo "✅ Migrations applied to Postgres test database"
# Run Windows-specific tests only (only works on Windows platform)
# These tests verify Windows-specific database optimizations (locking mode, NullPool)
# Will be skipped automatically on non-Windows platforms
test-windows:
uv run pytest -p pytest_mock -v --no-cov -m windows tests test-int
# Run benchmark tests only (performance testing)
# These are slow tests that measure sync performance with various file counts
# Excluded from default test runs to keep CI fast
test-benchmark:
uv run pytest -p pytest_mock -v --no-cov -m benchmark tests test-int
# Run all tests including Windows, Postgres, and Benchmarks (for CI/comprehensive testing)
# Use this before releasing to ensure everything works across all backends and platforms
test-all:
uv run pytest -p pytest_mock -v --no-cov tests test-int
# Generate HTML coverage report
coverage:
uv run pytest -p pytest_mock -v -n auto tests test-int --cov-report=html
@echo "Coverage report generated in htmlcov/index.html"
# Lint and fix code (calls fix)
lint: fix
# Lint and fix code
fix:
uv run ruff check --fix --unsafe-fixes src tests test-int
# Type check code
typecheck:
uv run pyright
# Clean build artifacts and cache files
clean:
find . -type f -name '*.pyc' -delete
find . -type d -name '__pycache__' -exec rm -r {} +
rm -rf installer/build/ installer/dist/ dist/
rm -f rw.*.dmg .coverage.*
# Format code with ruff
format:
uv run ruff format .
# Run MCP inspector tool
run-inspector:
npx @modelcontextprotocol/inspector
# Build macOS installer
installer-mac:
cd installer && chmod +x make_icons.sh && ./make_icons.sh
cd installer && uv run python setup.py bdist_mac
# Build Windows installer
installer-win:
cd installer && uv run python setup.py bdist_win32
# Update all dependencies to latest versions
update-deps:
uv sync --upgrade
# Run all code quality checks and tests
check: lint format typecheck test
# Generate Alembic migration with descriptive message
migration message:
cd src/basic_memory/alembic && alembic revision --autogenerate -m "{{message}}"
# Create a stable release (e.g., just release v0.13.2)
release version:
#!/usr/bin/env bash
set -euo pipefail
# Validate version format
if [[ ! "{{version}}" =~ ^v[0-9]+\.[0-9]+\.[0-9]+$ ]]; then
echo "❌ Invalid version format. Use: v0.13.2"
exit 1
fi
# Extract version number without 'v' prefix
VERSION_NUM=$(echo "{{version}}" | sed 's/^v//')
echo "🚀 Creating stable release {{version}}"
# Pre-flight checks
echo "📋 Running pre-flight checks..."
if [[ -n $(git status --porcelain) ]]; then
echo "❌ Uncommitted changes found. Please commit or stash them first."
exit 1
fi
if [[ $(git branch --show-current) != "main" ]]; then
echo "❌ Not on main branch. Switch to main first."
exit 1
fi
# Check if tag already exists
if git tag -l "{{version}}" | grep -q "{{version}}"; then
echo "❌ Tag {{version}} already exists"
exit 1
fi
# Run quality checks
echo "🔍 Running quality checks..."
just check
# Update version in __init__.py
echo "📝 Updating version in __init__.py..."
sed -i.bak "s/__version__ = \".*\"/__version__ = \"$VERSION_NUM\"/" src/basic_memory/__init__.py
rm -f src/basic_memory/__init__.py.bak
# Commit version update
git add src/basic_memory/__init__.py
git commit -m "chore: update version to $VERSION_NUM for {{version}} release"
# Create and push tag
echo "🏷️ Creating tag {{version}}..."
git tag "{{version}}"
echo "📤 Pushing to GitHub..."
git push origin main
git push origin "{{version}}"
echo "✅ Release {{version}} created successfully!"
echo "📦 GitHub Actions will build and publish to PyPI"
echo "🔗 Monitor at: https://github.com/basicmachines-co/basic-memory/actions"
# Create a beta release (e.g., just beta v0.13.2b1)
beta version:
#!/usr/bin/env bash
set -euo pipefail
# Validate version format (allow beta/rc suffixes)
if [[ ! "{{version}}" =~ ^v[0-9]+\.[0-9]+\.[0-9]+(b[0-9]+|rc[0-9]+)$ ]]; then
echo "❌ Invalid beta version format. Use: v0.13.2b1 or v0.13.2rc1"
exit 1
fi
# Extract version number without 'v' prefix
VERSION_NUM=$(echo "{{version}}" | sed 's/^v//')
echo "🧪 Creating beta release {{version}}"
# Pre-flight checks
echo "📋 Running pre-flight checks..."
if [[ -n $(git status --porcelain) ]]; then
echo "❌ Uncommitted changes found. Please commit or stash them first."
exit 1
fi
if [[ $(git branch --show-current) != "main" ]]; then
echo "❌ Not on main branch. Switch to main first."
exit 1
fi
# Check if tag already exists
if git tag -l "{{version}}" | grep -q "{{version}}"; then
echo "❌ Tag {{version}} already exists"
exit 1
fi
# Run quality checks
echo "🔍 Running quality checks..."
just check
# Update version in __init__.py
echo "📝 Updating version in __init__.py..."
sed -i.bak "s/__version__ = \".*\"/__version__ = \"$VERSION_NUM\"/" src/basic_memory/__init__.py
rm -f src/basic_memory/__init__.py.bak
# Commit version update
git add src/basic_memory/__init__.py
git commit -m "chore: update version to $VERSION_NUM for {{version}} beta release"
# Create and push tag
echo "🏷️ Creating tag {{version}}..."
git tag "{{version}}"
echo "📤 Pushing to GitHub..."
git push origin main
git push origin "{{version}}"
echo "✅ Beta release {{version}} created successfully!"
echo "📦 GitHub Actions will build and publish to PyPI as pre-release"
echo "🔗 Monitor at: https://github.com/basicmachines-co/basic-memory/actions"
echo "📥 Install with: uv tool install basic-memory --pre"
# List all available recipes
default:
@just --list
+128
View File
@@ -0,0 +1,128 @@
# Basic Memory Installation Guide for LLMs
This guide is specifically designed to help AI assistants like Cline install and configure Basic Memory. Follow these
steps in order.
## Installation Steps
### 1. Install Basic Memory Package
Use one of the following package managers to install:
```bash
# Install with uv (recommended)
uv tool install basic-memory
# Or with pip
pip install basic-memory
```
### 2. Configure MCP Server
Add the following to your config:
```json
{
"mcpServers": {
"basic-memory": {
"command": "uvx",
"args": [
"basic-memory",
"mcp"
]
}
}
}
```
For Claude Desktop, this file is located at:
macOS: ~/Library/Application Support/Claude/claude_desktop_config.json
Windows: %APPDATA%\Claude\claude_desktop_config.json
### 3. Start Synchronization (optional)
To synchronize files in real-time, run:
```bash
basic-memory sync --watch
```
Or for a one-time sync:
```bash
basic-memory sync
```
## Configuration Options
### Custom Directory
To use a directory other than the default `~/basic-memory`:
```bash
basic-memory project add custom-project /path/to/your/directory
basic-memory project default custom-project
```
### Multiple Projects
To manage multiple knowledge bases:
```bash
# List all projects
basic-memory project list
# Add a new project
basic-memory project add work ~/work-basic-memory
# Set default project
basic-memory project default work
```
## Importing Existing Data
### From Claude.ai
```bash
basic-memory import claude conversations path/to/conversations.json
basic-memory import claude projects path/to/projects.json
```
### From ChatGPT
```bash
basic-memory import chatgpt path/to/conversations.json
```
### From MCP Memory Server
```bash
basic-memory import memory-json path/to/memory.json
```
## Troubleshooting
If you encounter issues:
1. Check that Basic Memory is properly installed:
```bash
basic-memory --version
```
2. Verify the sync process is running:
```bash
ps aux | grep basic-memory
```
3. Check sync output for errors:
```bash
basic-memory sync --verbose
```
4. Check log output:
```bash
cat ~/.basic-memory/basic-memory.log
```
For more detailed information, refer to the [full documentation](https://memory.basicmachines.co/).
-378
View File
@@ -1,378 +0,0 @@
{"type":"entity","name":"Paul","entityType":"person","observations":["Software developer combining DIY ethics, Free Software principles, and theoretical computer science","Created the Basic Machines project","Values authentic exchange of ideas","Approaches AI interaction with emphasis on genuine technical discussion","Comfortable with uncertainty and open dialogue","Balances practical implementation with broader implications"]}
{"type":"entity","name":"Basic_Machines","entityType":"project","observations":["Local-first knowledge management system","Combines filesystem durability with graph-based knowledge representation","Focuses on enhancing human agency and understanding","Synthesizes DIY ethics, Free Software philosophy, and theoretical computer science","Current focus includes basic-memory system"]}
{"type":"entity","name":"basic-memory","entityType":"software_system","observations":["A core component of Basic Machines","Local-first knowledge management system","Combines filesystem persistence with graph-based knowledge representation","Being implemented collaboratively by Paul and Claude"]}
{"type":"entity","name":"basic-memory_implementation_patterns","entityType":"technical_patterns","observations":["Filesystem is source of truth - all changes write to files first","Clean separation of concerns between models (SQLAlchemy), schemas (Pydantic), and services","Repository pattern for database access","Service layer handling business logic and coordination","Atomic file operations using temporary files for safety","Clear error handling hierarchy with specific error types","Comprehensive test coverage with pytest and fixtures","Async/await used throughout the codebase","Validation using Pydantic models with custom validators"]}
{"type":"entity","name":"fileio_module","entityType":"code_module","observations":["Extracted from EntityService to handle all file operations","Provides read_entity_file, write_entity_file, and delete_entity_file functions","Handles markdown parsing and formatting","Implements atomic file operations","Provides consistent error handling","Enables reuse across services"]}
{"type":"entity","name":"entity_service","entityType":"code_module","observations":["Manages entities in both filesystem and database","Uses fileio module for file operations","Maintains database index of entities","Handles entity creation, retrieval, and deletion","Follows 'filesystem is source of truth' principle","Coordinates with observation service for full entity management"]}
{"type":"entity","name":"observation_service","entityType":"code_module","observations":["Manages observations within entity files","Provides database indexing for efficient observation queries","Works with complete Entity objects rather than IDs","Handles observation addition and search","Maintains consistency between files and database","Under development for update/remove operations"]}
{"type":"entity","name":"observation_management","entityType":"design_challenge","observations":["Key challenge: maintaining observation state across files and database","Exploring bulk update approach - treating all observations as a unit","Considering tracked observations with markdown comments for IDs","Investigating diff-based approach for observation-level changes","Evaluating position-based management without explicit IDs","Trade-offs between implementation complexity and markdown readability"]}
{"type":"entity","name":"testing_infrastructure","entityType":"technical_patterns","observations":["Uses pytest with async support via pytest-asyncio","In-memory SQLite database for test isolation","Temporary directories for file operation testing","Comprehensive fixture system for test setup","Tests organized by component (entity, observation, etc)","Covers happy path, error cases, and edge cases","Uses monkeypatch for mocking dependencies","Clear separation between arrange, act, assert sections","Uses in-memory SQLite database for test isolation","Comprehensive fixture system for test data setup","Proper async test handling with pytest-asyncio"]}
{"type":"entity","name":"test_categories","entityType":"test_suite","observations":["Happy path tests verify core functionality","Error path tests ensure proper error handling","Edge cases test special characters and long content","File operation tests verify atomic writes and rollbacks","Database sync tests verify index consistency","Recovery tests for rebuild operations","Punted on concurrent operation tests due to session management complexity"]}
{"type":"entity","name":"completed_work","entityType":"project_milestone","observations":["Extracted file operations to fileio.py module","Updated EntityService to use fileio functions","Implemented initial ObservationService","Created comprehensive test suite","Established clear project patterns and principles","Set up basic database schema with SQLAlchemy","Created Pydantic models for validation"]}
{"type":"entity","name":"future_work","entityType":"project_tasks","observations":["Implement observation updates/removals","Design proper session management for concurrent operations","Update EntityService tests for new fileio module","Add more sophisticated search functionality","Handle markdown formatting edge cases","Consider versioning for file changes","Implement proper backup strategy"]}
{"type":"entity","name":"design_decisions","entityType":"technical_decisions","observations":["Filesystem as source of truth over database","Markdown format for human readability and editing","Atomic file operations for safety","SQLite + SQLAlchemy for proven reliability","Pydantic for validation and ID generation","Async/await for better scalability","Clear separation between files and database roles","Explicit error hierarchies for better handling"]}
{"type":"entity","name":"concurrency_considerations","entityType":"technical_challenge","observations":["SQLAlchemy session management in async context","File operation atomicity","Transaction isolation levels","Potential for conflicting updates","Need for proper session lifecycle","Possibility of file system race conditions","Database lock management"]}
{"type":"entity","name":"observation_update_approaches","entityType":"design_alternatives","observations":["Each approach trades off between simplicity, efficiency, and robustness","Four main approaches considered: bulk update, tracked IDs, diff-based, and position-based","Discussion revealed importance of human readability in file format","Consideration of manual editing workflows key to design","File system as source of truth principle guides tradeoffs"]}
{"type":"entity","name":"bulk_update_approach","entityType":"design_option","observations":["Update all observations at once in a single operation","Simpler file operations - just rewrite the whole list","No need for observation matching or IDs","Very consistent with source of truth principle","Less efficient for small changes","May have concurrency implications","Simplest implementation option"]}
{"type":"entity","name":"tracked_observations_approach","entityType":"design_option","observations":["Use markdown comments to store observation IDs","Enables precise updates and deletes","IDs stored as HTML comments in markdown","More complex markdown parsing required","IDs visible in raw markdown files","Balances tracking with readability"]}
{"type":"entity","name":"diff_based_approach","entityType":"design_option","observations":["Implement observation-aware diffing","Track changes at observation level","More efficient for updates","Preserves manual edits and changes","More complex implementation needed","Must handle merge conflicts","Most sophisticated option considered"]}
{"type":"entity","name":"position_based_approach","entityType":"design_option","observations":["Track observations by position/order","No explicit IDs needed","Cleanest markdown format","Order changes could break references","Difficult to handle concurrent edits","Most fragile option considered"]}
{"type":"entity","name":"tasks_and_progress","entityType":"project_tracking","observations":["Current focus on observation management implementation","Completed core file operations extraction","Completed EntityService updates","Completed initial ObservationService","Basic test coverage in place","Future work includes concurrent operations","Future work includes search improvements","Need to handle markdown edge cases"]}
{"type":"entity","name":"error_handling_patterns","entityType":"technical_patterns","observations":["Custom exception hierarchy with ServiceError base","Specific error types (FileOperationError, DatabaseSyncError, etc)","Clear separation between file and database errors","Error propagation patterns established","Focus on actionable error messages","Error handling at appropriate levels"]}
{"type":"entity","name":"data_models","entityType":"technical_implementation","observations":["SQLAlchemy models for database structure","Pydantic schemas for API/service layer","Entity model with UUID-based IDs","Observation model with entity relationships","UTCDateTime custom type for timestamps","Automatic ID generation in Pydantic models","Strict validation rules"]}
{"type":"entity","name":"markdown_format","entityType":"file_format","observations":["Simple, human-readable format","Entity name as H1 header","Metadata in key-value format","Observations as bullet points","Atomic file operations for updates","Designed for manual editing","No hidden metadata in main content"]}
{"type":"entity","name":"test_driven_development","entityType":"development_pattern","observations":["Tests revealed need for atomic file operations","Error cases drove error hierarchy design","Edge cases informed validation rules","Test fixtures shaped service interfaces","File operations extracted due to test patterns","Concurrent test issues revealed session management needs"]}
{"type":"entity","name":"architecture_evolution","entityType":"design_process","observations":["Started with simple EntityService implementation","Circular dependency between Entity and Observation services revealed design flaw","Extracted file operations to separate module","Moved to passing Entity objects rather than IDs","Improved separation of concerns through iterations","File operations became reusable across services","Database became true 'index' rather than source of truth"]}
{"type":"entity","name":"validation_patterns","entityType":"technical_patterns","observations":["Pydantic models provide schema validation","Automatic ID generation if not provided","Database constraints via SQLAlchemy","Runtime checks in services","Markdown format validation","Error handling for invalid states"]}
{"type":"entity","name":"markdown_examples","entityType":"documentation","observations":["Example of basic entity:\n# Entity Name\ntype: entity_type\n\n## Observations\n- First observation\n- Second observation","Example with special characters:\n# Test & Entity!\ntype: test\n\n## Observations\n- Test & observation with @#$% special chars!","Format ensures human readability:\n# Basic Machines\ntype: project\n\n## Observations\n- Local-first knowledge management system\n- Combines filesystem durability with graph-based knowledge representation","Future consideration for observation IDs:\n# Entity Name\ntype: entity_type\n\n## Observations\n- <!-- obs-id: abc123 -->\n This is an observation with ID"]}
{"type":"entity","name":"markdown_parsing_rules","entityType":"technical_implementation","observations":["H1 header contains entity name","Metadata uses key: value format","Observations section marked by H2 header","Each observation is a markdown list item","Blank lines separate sections","Special characters allowed in content","No restrictions on observation content"]}
{"type":"entity","name":"schema_definitions","entityType":"technical_documentation","observations":["SQLAlchemy Entity model:\nclass Entity(Base):\n id: str (primary key)\n name: str (unique)\n entity_type: str\n created_at: datetime\n updated_at: datetime","SQLAlchemy Observation model:\nclass Observation(Base):\n id: str (primary key)\n entity_id: str (foreign key)\n content: str\n created_at: datetime\n context: Optional[str]","Pydantic Entity schema:\nclass Entity(BaseModel):\n id: str\n name: str\n entity_type: str\n observations: List[Observation]"]}
{"type":"entity","name":"test_evolution","entityType":"development_history","observations":["Started with basic Entity CRUD tests","Added filesystem verification to all tests","Developed concurrent operation tests (later removed)","Edge case tests drove better error handling","Test fixtures evolved to support both file and DB testing","Mocking patterns for file/DB operations","Special cases for long content and special characters"]}
{"type":"entity","name":"implementation_challenges","entityType":"technical_issues","observations":["Initial circular dependency between services","SQLAlchemy session management in async context","Atomic file operations with proper error handling","Maintaining DB sync with filesystem changes","Handling long content in observations","Managing test isolation with file operations","Deciding on markdown format tradeoffs","Concurrent operation complexity"]}
{"type":"entity","name":"Basic_Factory","entityType":"Project","observations":["Collaborative project between Paul and Claude","Explores AI-human collaboration in software development","Uses MCP tools for file and memory management","Built with git integration capabilities","Focuses on maintaining project context across sessions","About 90% complete with MCP tools","Still needs improvements in collaboration via files/git/github","Will be used to document and share collaborative development process"]}
{"type":"entity","name":"Basic_Factory_Components","entityType":"Technical","observations":["Server-side rendering with JinjaX","HTMX for dynamic updates","Alpine.js for client-side state","Tailwind CSS for styling","Component translation from React/shadcn/ui","Focus on simplicity and understandability","Demonstrates meta-compiler principles in component translation"]}
{"type":"entity","name":"Component_Translation_Process","entityType":"Methodology","observations":["Treats component porting as meta-compilation","Maps between React/TypeScript and JinjaX/Alpine.js domains","Uses formal grammar transformation approaches","Maintains functionality while simplifying implementation","Focuses on server-side rendering patterns","Preserves accessibility and performance","Uses short, focused git branches for each component"]}
{"type":"entity","name":"Basic_Machines_Philosophy","entityType":"Philosophy","observations":["Combines DIY punk ethics with software development","Emphasizes user empowerment and understanding","Values simplicity and composability","Treats complex systems as combinations of simple parts","Focuses on authentic creation and sharing","Draws inspiration from punk rock, Free Software, and theoretical CS","Emphasizes the cycle of creation, complexity, and renewal"]}
{"type":"entity","name":"Basic_Machines_Manifesto","entityType":"Document","observations":["Created through collaboration between Paul and Claude","Explores connection between DIY punk ethics and software development","Emphasizes composition over inheritance in both philosophy and practice","Views software development through lens of basic machines that combine for complex computation","Advocates for user empowerment and technological independence","Structured in sections covering Origins, Philosophy, Technical Implementation, and AI Collaboration","Draws connections between punk rock, free software, and theoretical computer science","Emphasizes importance of sharing knowledge and building community","Released in December 2024"]}
{"type":"entity","name":"AI_Human_Collaboration_Model","entityType":"Methodology","observations":["Focuses on deep collaboration rather than simple task completion","Maintains rich context across sessions via knowledge graph","Uses short, focused git branches for each collaborative session","Values intellectual partnership over simple code generation","Emphasizes both practical implementation and theoretical exploration","Creates space for authentic exchange while maintaining AI/human clarity","Uses formal methods when appropriate (like grammar transformation)","Documents decisions and processes for future reference","Developed through Basic Machines project experience"]}
{"type":"entity","name":"Basic_Machines_Roadmap","entityType":"Project_Plan","observations":["Phase 1 (30 days): Build basic-machines.co website","Phase 2 (60-90 days): Develop premium component bundles","Phase 3 (90-120 days): Launch Basic Foundation commercial offering","Focus on building brand and marketing presence","Prioritize components needed for own site development","Document and share collaboration process","Build sustainable business model aligned with values"]}
{"type":"entity","name":"Basic_Machines_Website","entityType":"Project","observations":["To be built at basic-machines.co","Will showcase products and vision","Needs components for navigation, hero sections, features","Will demonstrate component usage in production","Will include blog for sharing progress","Focus on clear value proposition","Platform for sharing Basic Machines philosophy"]}
{"type":"entity","name":"Basic_Memory_Markdown_Example","entityType":"Example","observations":["Shows complete markdown structure for basic-memory entity","Uses frontmatter for metadata (id, type, created, context)","Has main description section after title","Includes Observations as bullet points","Shows Relations with [id] relation_type | context format","Lists References at bottom","Created during initial design discussion","Serves as canonical example of file format"]}
{"type":"entity","name":"Basic_Memory_Database_Schema","entityType":"Technical","observations":["Uses SQLite for local storage","Entities table with id, name, type, created_at, context, description, references","Observations table linking to entities with content and context","Relations table tracking directional relationships between entities","References column needs quotes as SQL reserved word","Designed for easy rebuilding from markdown files","Foreign key constraints maintain data integrity","Unique constraint on relations prevents duplicates","Created_at timestamps track history","Context fields enable tracking information sources"]}
{"type":"entity","name":"Basic_Memory_Project_Structure","entityType":"Technical","observations":["Uses dbmate for database migrations","Projects directory stores SQLite databases and markdown files","Makefile provides common development commands","Environment vars configure database connection","db/migrations directory for SQL schema changes","Gitignore excludes database files and env config","Uses Python 3.12 with modern tooling","Tests directory for pytest files","Follows Basic Machines project conventions"]}
{"type":"entity","name":"Basic_Memory_Project_Isolation_Decision","entityType":"Decision","observations":["Decided to defer multi-project support to post-MVP","Will use separate SQLite databases per project","Initially using projects directory in code repository","Plan to make location configurable later","No changes needed to core domain model","Keeps initial implementation simple","FTS/search capabilities also deferred for simplicity"]}
{"type":"entity","name":"Basic_Memory_Implementation_Plan","entityType":"Plan","observations":["Start with SQLAlchemy models matching schema","Then build CLI for basic operations","Then implement markdown parser","Use TDD approach throughout","Begin with core domain model","CLI will support CRUD operations","Parser must handle frontmatter and sections","Following modular development approach","Planning to use typer for CLI","Will use modern Python tools and practices"]}
{"type":"entity","name":"Basic_Memory_Implementation_Status","entityType":"Status","observations":["Core modules implemented: models, services, repository, fileio","Modular architecture with clear separation of concerns","File operations extracted to separate fileio module","Initial ObservationService implementation complete","Basic test coverage in place","Exploring observation management strategies","Using SQLAlchemy for database interaction","Markdown file operations working","Entity management functional","Repository layer implementation complete with SQLAlchemy models and tests","Database operations working with proper UTC timestamp handling","In-memory SQLite testing infrastructure proven effective"]}
{"type":"entity","name":"Basic_Memory_Observation_Management_Design","entityType":"Design","observations":["Four approaches under consideration","Bulk Update: Simple but less efficient","Tracked Observations: Precise but clutters markdown","Diff-based: Efficient but complex","Position-based: Clean but fragile","Key challenge is balancing markdown readability with efficient updates","Must maintain filesystem as source of truth","Need to consider concurrent edits","Currently evaluating trade-offs","Implementation choice pending discussion"]}
{"type":"entity","name":"Basic_Memory_Architectural_Decisions","entityType":"Decisions","observations":["Split file operations into separate fileio module","Using SQLAlchemy for database operations","Maintain filesystem as source of truth","Modular service-based architecture","Clear separation between data access and business logic","Repository pattern for database interactions","Schemas separate from models","Focus on maintainability and testability","Services handle business rules","Considering concurrency in design"]}
{"type":"entity","name":"Basic_Memory_Implementation_Analysis","entityType":"Analysis","observations":["Clean modular architecture with clear responsibilities","Strong typing throughout codebase","Excellent error handling with custom exceptions","SQLAlchemy models perfectly match our domain model","Atomic file operations for data safety","Services implement filesystem-as-source-of-truth principle","Async support throughout","Good separation between domain models and database models","Careful handling of UTC timestamps","Smart use of SQLAlchemy relationships"]}
{"type":"entity","name":"Basic_Memory_Current_Challenges","entityType":"Challenges","observations":["Observation update/removal strategy needs to be chosen","Need to handle concurrent file operations safely","Search functionality to be implemented","Edge cases in markdown formatting to be handled","Session management for concurrent operations needed","Balance between file operations and database sync","Testing coverage could be expanded","Need to handle relationship updates in files"]}
{"type":"entity","name":"Basic_Memory_Observation_Hash_Tracking","entityType":"Design","observations":["Use content hashes to track observation identity","Store hashes in database but not in markdown","Can match observations across file edits using hashes","Similar to how git tracks content changes","Keeps markdown clean and human-friendly","Allows efficient bulk updates","Handles reordering of observations","Maintains filesystem as source of truth","No need for visible IDs in markdown","Could track observation history through hash changes"]}
{"type":"entity","name":"Basic_Memory_Repository_Implementation","entityType":"Code_Implementation","observations":["Implemented base Repository class with CRUD operations","Added specialized EntityRepository, ObservationRepository, and RelationRepository","Used string IDs instead of UUIDs","Added UTCDateTime custom type for timestamp handling","Used in-memory SQLite for testing","Achieved 84% test coverage","Created comprehensive pytest fixtures"]}
{"type":"entity","name":"Basic_Memory_Dependencies","entityType":"Technical","observations":["Uses Python 3.12","SQLAlchemy with async support","pytest-asyncio for async testing","aiosqlite for async SQLite operations","greenlet for SQLAlchemy async support","uv for dependency management","pytest-cov for coverage reporting","Development dependencies managed in pyproject.toml"]}
{"type":"entity","name":"Basic_Memory_Current_Architecture","entityType":"Architecture_Analysis","observations":["Clear separation between domain models (Pydantic) and storage models (SQLAlchemy)","File I/O completely separated into dedicated module","Strong 'filesystem as source of truth' pattern in services","Atomic file operations with proper error handling","Service layer coordinates between filesystem and database","Database acts as queryable index rather than primary storage","Clean error hierarchy with specific exception types","Rebuild operations available for recovery scenarios"]}
{"type":"entity","name":"Basic_Memory_Evolution","entityType":"Analysis","observations":["Started with repository pattern following basic-foundation","Evolved to more sophisticated architecture with clear layers","Added Pydantic schemas for domain modeling","Separated file operations into dedicated module","Implemented robust error handling throughout","Maintained filesystem as source of truth principle","Added observation management with context tracking","Introduced rebuild capabilities for system recovery"]}
{"type":"entity","name":"Basic_Memory_Service_Layer","entityType":"Implementation","observations":["EntityService handles entity lifecycle and coordinates storage","ObservationService manages observations within entities","Services ensure filesystem and database stay in sync","Clear error handling with ServiceError hierarchy","Strong typing throughout service interfaces","Implements filesystem as source of truth pattern","Handles UUID generation and timestamp management","Provides methods for system recovery and rebuild"]}
{"type":"entity","name":"Basic_Memory_Schema_Design","entityType":"Implementation","observations":["Uses Pydantic for domain models and validation","Automatic ID generation with timestamp and UUID","Clear separation from SQLAlchemy storage models","Supports optional context tracking","Models match markdown file structure","Enables clean serialization/deserialization","Strong typing with proper validation rules","Independent from storage concerns"]}
{"type":"entity","name":"Basic_Memory_Next_Tasks","entityType":"TaskList","observations":["✅ Implement SQLAlchemy models and repositories (Done)","✅ Add SQLAlchemy migrations (Done)","✅ Create service layer (Done)","✅ Implement file I/O module (Done)","✅ Set up domain models with Pydantic (Done)","✅ Initial test infrastructure (Done)","✅ Basic CRUD operations (Done)","⏳ Implement full test coverage for db.py","⏳ Add more sophisticated search functionality","⏳ Implement CLI interface","⏳ Add relationship management to services","⏳ Handle concurrent file operations safely","⏳ Add versioning for file changes","⏳ Implement proper backup strategy","⏳ Add type hints throughout codebase","⏳ Improve error messages and logging","⏳ Add documentation for core modules"]}
{"type":"entity","name":"Basic_Memory_Meta_Experience","entityType":"Case_Study","observations":["Experienced our own context loss when reconstructing project knowledge","Had to rebuild task list and project context from filesystem and memory","Validated 'filesystem as source of truth' principle through reconstruction","Code and tests served as reliable historical record","Knowledge graph structure helped guide reconstruction process","Markdown files provided human-readable context","Atomic information design made piece-by-piece reconstruction possible","Ironic validation of the need for basic-memory's features","Experience demonstrates value of durable, human-readable knowledge storage","Shows importance of separating durable storage from ephemeral context"]}
{"type":"entity","name":"Model_Context_Protocol","entityType":"protocol","observations":["Core part of the basic-memory architecture","Enables AI-human collaboration on projects","Provides tool-based interaction with knowledge graph","Developed by Anthropic for structured AI-system interaction","Used for maintaining consistent, rich context across conversations"]}
{"type":"entity","name":"basic-memory_core_principles","entityType":"principles","observations":["Local First: All data stored locally in SQLite","Project Isolation: Separate databases per project","Human Readable: Everything exportable to plain text","AI Friendly: Structure optimized for LLM interaction","DIY Ethics: User owns and controls their data","Simple Core: Start simple, expand based on needs","Tool Integration: MCP-based interaction model"]}
{"type":"entity","name":"basic-memory_business_model","entityType":"business_strategy","observations":["Core features free: Local SQLite, basic knowledge graph, search, markdown export, basic MCP tools","Professional features potential: Rich document export, advanced versioning, collaboration features, custom integrations, priority support","Focus on maintaining DIY/punk philosophy while enabling sustainability"]}
{"type":"entity","name":"basic-memory_cli","entityType":"interface","observations":["Supports project management commands (create, switch, list)","Entity management (add entity, add observation, add relation)","Future support for export and batch operations","Follows consistent command structure","Planned integration with MCP tools"]}
{"type":"entity","name":"basic-memory_export_format","entityType":"file_format","observations":["Uses markdown with frontmatter metadata","Includes entity name, type, creation timestamp","Observations as bullet points","Relations in structured format with links","References section at bottom","Designed for human readability and machine parsing","Example format documented in project specs"]}
{"type":"entity","name":"relation_service","entityType":"code_module","observations":["Planned service for managing relations in both filesystem and database","Will follow filesystem-is-source-of-truth principle like other services","Needs to handle atomic file operations for relation updates","Must coordinate with EntityService for relationship integrity","Will handle bidirectional relationship tracking","Will support relation validation and type enforcement","Must implement rebuild functionality for index recovery","Will need careful error handling for file/db sync","Should support relation search and filtering","Must handle relation lifecycle (create/read/update/delete)"]}
{"type":"entity","name":"service_layer_patterns","entityType":"implementation_patterns","observations":["Services handle both file and database operations","Filesystem is always source of truth","Database serves as queryable index","Services implement atomic file operations","Clear error hierarchy with specific exceptions","Use of dependency injection via constructor params","Async/await used throughout service layer","Services coordinate between storage layers","Repository pattern used for database access","Services maintain entity integrity across storage","Rich error types extend from ServiceError base","Rebuild operations available for recovery"]}
{"type":"entity","name":"database_models","entityType":"implementation","observations":["Entity model with unique name and type","Observation model linked to entities","Relation model tracks connections between entities","Custom UTCDateTime type for timestamp handling","Use of SQLAlchemy relationships for navigation","Cascading deletes for dependent objects","String IDs used for compatibility","Rich relationship modeling with backpopulates","Proper indexing on foreign keys","Context tracking available on models","Models include created_at timestamps","Relationships handle bidirectional navigation"]}
{"type":"entity","name":"repository_patterns","entityType":"implementation_patterns","observations":["Generic Repository[T] base class implementation","Type-safe operations with SQLAlchemy","Specialized repositories for each model type","Async operations throughout","Clear error handling patterns","Support for custom queries and filtering","Pagination support built-in","Transaction management via session","Proper type hints and generics usage","Entity-specific query methods in subclasses"]}
{"type":"entity","name":"relation_service_design","entityType":"design","observations":["Must handle relation lifecycle in both files and DB","Needs to validate existence of both entities","Should support relation type enforcement","Must maintain bidirectional consistency","Should support relation querying and filtering","Needs proper error handling for graph consistency","Must integrate with entity file format","Should support bulk operations for efficiency","Must handle relation deletion and cascading","Should provide search by type and entities"]}
{"type":"entity","name":"relation_service_implementation_plan","entityType":"plan","observations":["1. Define core relation operations (create, get, delete)","2. Implement file format handling for relations","3. Add database sync with RelationRepository","4. Implement validation and error handling","5. Add rebuild and recovery operations","6. Implement relation type enforcement","7. Add relation search and filtering","8. Implement bulk operations","9. Add comprehensive tests","10. Document API and error handling"]}
{"type":"entity","name":"relation_service_challenges","entityType":"challenges","observations":["Maintaining consistency between file and database","Handling relation type validation efficiently","Managing bidirectional relationships in files","Ensuring atomic updates across entities","Handling deletion with proper cascading","Efficient querying of relation graphs","Recovery from partial file/db sync failures","Bulk operation atomicity","Clear error reporting for graph operations","Performance with large relation sets"]}
{"type":"entity","name":"relation_file_format","entityType":"file_format","observations":["Relations stored in entity markdown files","Format: [target_id] relation_type | context","Relations section marked by ## Relations header","Outgoing relations only stored in source entity","Relations rebuild on entity load","Clean human-readable format","Context is optional with pipe separator","Links generate valid navigation references","Markdown-friendly formatting","Example: [Paul] authored | with Claude"]}
{"type":"entity","name":"relation_service_error_handling","entityType":"implementation_patterns","observations":["RelationError extends ServiceError base","Specific errors for validation failures","Handles entity not found cases","Manages relation type validation errors","File operation errors properly wrapped","Database sync errors clearly reported","Transaction rollback on errors","Proper error propagation chain","Clear error messages for debugging","Recovery paths for common errors"]}
{"type":"entity","name":"relation_service_testing","entityType":"testing","observations":["Test all relation lifecycle operations","Verify file and database consistency","Test relation type validation","Check error handling paths","Test bulk operations","Verify bidirectional consistency","Test recovery operations","Check cascade operations","Verify search and filtering","Test with large relation sets"]}
{"type":"entity","name":"fileio_patterns","entityType":"implementation_patterns","observations":["Atomic file operations with temporary files","Clear error handling for IO operations","Consistent file naming and paths","Support for different file formats","Efficient file reading and writing","Proper file locking mechanisms","Recovery from partial writes","Consistent encoding handling","Directory management utilities","Path manipulation helpers","Currently implemented in fileio.py module","Uses pathlib for path operations","Handles file not found cases gracefully","Maintains data integrity during writes"]}
{"type":"entity","name":"pytest_patterns","entityType":"implementation_patterns","observations":["Common fixtures should be in conftest.py for reuse","Use pytest_asyncio.fixture for async fixtures","Session fixtures need proper async cleanup","Temporary directories should be managed with context managers","Test categories: happy path, error path, recovery, edge cases","Services need project_path and repo injected","Use monkeypatch for mocking in async context","SQLite in-memory database ideal for testing","Explicit test verification: file content and database state"]}
{"type":"entity","name":"relation_implementation_learnings","entityType":"implementation_learnings","observations":["Better to pass full Entity objects than IDs to services","Services should not re-read entities if they have them","File operations should be atomic and verified","Database serves as queryable index, not source of truth","Relations stored in source entity's markdown file","Clear separation between file ops and database sync","Entity objects should own their relations list","Context is optional but fully supported in implementation"]}
{"type":"entity","name":"test_driven_insights","entityType":"learnings","observations":["Tests help reveal better API design (e.g., passing Entity objects)","Error cases drive proper exception hierarchy","File verification as important as database checks","Edge cases inform markdown format decisions","Recovery tests ensure system resilience","Tests document expected behavior clearly","Fixtures significantly reduce test complexity","Common patterns emerge through test writing"]}
{"type":"entity","name":"meta_development_insights","entityType":"process","observations":["Break down large tasks into reviewable chunks","One file at a time prevents response truncation","Iterative development with tests leads to better design","Infrastructure code (fixtures) should be consolidated early","Test categories help ensure comprehensive coverage","Knowledge capture should happen during development","APIs tend to evolve toward simpler patterns","File operations require careful verification"]}
{"type":"entity","name":"AI_Assistant_Learnings","entityType":"meta_insights","observations":["Output management: Breaking responses into single files prevents truncation and allows better review","Knowledge graph helps maintain context: I can reference previous decisions and patterns accurately","Memory rebuilding experience validated the need for durable storage","Test-driven development provides clear steps and verification","Explicit relation tracking in knowledge graph helps me understand project context","Rich context from multiple sources (code, docs, tests) enables better assistance","File-at-a-time approach allows deeper analysis of each component","Keeping entity names consistent helps with referencing and relationships"]}
{"type":"entity","name":"Effective_Response_Patterns","entityType":"meta_patterns","observations":["When showing code changes, break into discrete files","Review existing code before suggesting changes","Reference knowledge graph for context and patterns","Explicitly connect new code to existing patterns","Validate suggestions against test cases","Keep track of file changes for atomic commits","Check both implementation and test files for consistency","Maintain clear separation of concerns in responses"]}
{"type":"entity","name":"AI_Context_Management","entityType":"meta_practice","observations":["Knowledge graph provides reliable persistent memory","Project documentation gives high-level context","Code review shows implementation patterns","Tests demonstrate expected behavior","Important to actively track what has been modified","Entity relationships help understand dependencies","Regular knowledge capture during development","Using consistent entity references across conversations"]}
{"type":"entity","name":"AI_Tool_Usage_Patterns","entityType":"meta_practice","observations":["read_file before suggesting changes","write_file one file at a time","list_directory to understand project structure","search_nodes to find relevant context","create_entities to capture new learnings","create_relations to connect concepts","Using knowledge graph to track decisions","Validating changes through test execution"]}
{"type":"entity","name":"relation_service_learnings","entityType":"implementation_learnings","observations":["Entity-based API cleaner than ID-based for service layer","Model_dump method can handle storage serialization","File format needs explicit section markers (## Relations)","Whitespace handling important for long content comparisons","Test fixtures allow focused test cases","SQLAlchemy selects better than raw SQL for type safety","Atomic file operations maintained for relations"]}
{"type":"entity","name":"test_driven_insights_relations","entityType":"learnings","observations":["Tests revealed need for whitespace normalization","Edge cases drove file format decisions","SQLAlchemy model access safer than raw queries","Fixtures reduced test setup complexity","File verification as important as database checks","Testing both memory model and storage format","Test categories ensure comprehensive coverage"]}
{"type":"entity","name":"relation_service_patterns","entityType":"patterns","observations":["Use Entity objects in API","Serialize to IDs for storage","Maintain file as source of truth","Keep file format human-readable","Handle circular references in serialization","Use repository pattern for database","Clear error hierarchies"]}
{"type":"entity","name":"packaging_learnings","entityType":"technical_learnings","observations":["When using pytest-mock, traditional pip install works more reliably than uv sync","Package discovery behavior can differ between uv and pip","Clean venv with pip install is a reliable fallback for dependency issues","Package installation location might differ between uv and pip","Dependencies in pyproject.toml dev section work reliably with pip install -e .[dev]"]}
{"type":"entity","name":"Recent_Implementation_Progress","entityType":"progress_update","observations":["Successfully split services.py into modular structure under services/","Created __init__.py, entity_service.py, observation_service.py, relation_service.py","Fixed pytest-mock installation issues by using pip install -e .[dev] instead of uv sync","Improved test structure with minimal mocking - only used for error testing","Implemented relation service with Entity-based API","Achieved good test coverage across services","File operations are only mocked when testing error conditions","Services follow filesystem-as-source-of-truth pattern"]}
{"type":"entity","name":"Next_Steps","entityType":"project_tasks","observations":["Consider adding more relation service tests","Potentially expand relations features","Look for opportunities to improve test coverage","Consider documenting package management preferences (pip vs uv)","Consider adding integration tests for services","Review and possibly expand error handling cases"]}
{"type":"entity","name":"Development_Practices","entityType":"process","observations":["Favor real operations over mocks in tests","Only mock for error condition testing","Use pip install -e .[dev] for reliable dev dependency installation","Maintain modular service structure","Keep filesystem as source of truth","Use Entity objects in service APIs instead of IDs","Validate both file and database state in tests"]}
{"type":"entity","name":"MCP_Resources","entityType":"Concept","observations":["Stateful objects in Model Context Protocol","Enable persistent access to capabilities"]}
{"type":"entity","name":"MCP_Server_Implementation","entityType":"Technical_Design","observations":["Inherits from mcp.server.Server base class","Tools are implemented as async methods","Each tool method maps directly to a function available to the AI","Tools can request user input via Prompts","Simple function call interface rather than explicit resource management","State management handled by server instance","Returns serialized data using model_dump() for consistency"]}
{"type":"entity","name":"MCP_Tools","entityType":"Protocol_Feature","observations":["Defined as async methods on server class","Return values must match tool definition schema","Can maintain state between invocations via server instance","Tools can prompt for user input when needed","No need for explicit Resource objects in implementation"]}
{"type":"entity","name":"Basic_Memory_MCP","entityType":"Implementation","observations":["Uses MemoryService for core operations","Implements project selection via prompts","Maintains project context across tool invocations","Maps directly to memory graph operations","Handles serialization of Pydantic models"]}
{"type":"entity","name":"Basic_Memory_Testing","entityType":"Testing_Design","observations":["Needs pytest for async testing","Should isolate filesystem operations for tests","Needs to handle MCP server lifecycle in tests","Should test both service layer and MCP interface","Will need mocks for project paths and file operations"]}
{"type":"entity","name":"Memory_Service_Tests","entityType":"Test_Suite","observations":["Should test entity creation with observations","Should test relation creation between entities","Should verify proper ID generation and model validation","Should test deletion cascading","Should test search functionality","Must verify proper serialization of entities and relations"]}
{"type":"entity","name":"MCP_Server_Tests","entityType":"Test_Suite","observations":["Should test project initialization workflow","Should test prompt handling","Should verify tool input/output formats","Should test error cases and validation","Must verify proper serialization in tool responses"]}
{"type":"entity","name":"Memory_Service_Refactoring","entityType":"Technical_Task","observations":["MemoryService uses create() but EntityService might expect create_entity()","MemoryService assumes get_by_name() but EntityService might use different method","Need to verify deletion method signatures","Need to check if search interface matches","Should verify observation handling matches ObservationService interface","RelationService methods need verification","EntityService.create_entity takes name, type, and optional observations directly, not an Entity object","EntityService requires project_path and entity_repo in constructor","ObservationService.add_observation takes Entity object and content string, not raw data","RelationService.create_relation takes Entity objects directly, not dict data","All services follow filesystem-as-source-of-truth pattern with DB indexing","All services handle database synchronization internally","Services expect Path objects for filesystem operations"]}
{"type":"entity","name":"Service_Interface_Audit","entityType":"Technical_Task","observations":["Need to review all existing service interfaces","Document current method signatures","Map discrepancies between MemoryService assumptions and actual interfaces","Check return types and error handling patterns","Review transaction/atomicity requirements","Method signatures need alignment: create vs create_entity etc","Need to handle DB repositories in service constructors","File operations should use project_path consistently","Need to maintain filesystem-as-source-of-truth pattern","Should handle database synchronization at service level","Error handling should align with existing patterns","Consider making MemoryService handle DB indexing consistently"]}
{"type":"entity","name":"Memory_Service_Patterns","entityType":"Technical_Pattern","observations":["Uses inner async functions to encapsulate operation logic","Leverages list comprehensions with async functions for parallel operations","Each operation follows a consistent pattern: validate, update DB, write file","Inner functions make the code more readable and maintainable","Operations can run in parallel when using list comprehensions with async functions"]}
{"type":"entity","name":"Pydantic_Create_Pattern","entityType":"Technical_Pattern","observations":["Separate Create models match the exact shape of incoming data","Provides clear contract for MCP tool inputs","Handles validation of raw input data","Converts cleanly to domain models via from_create methods","Maintains separation between external API format and internal models","Similar to FastAPI request model pattern","Allows camelCase in API while using snake_case internally"]}
{"type":"entity","name":"Basic_Memory_Business","entityType":"Business_Model","observations":["Core system is open source and free","Local-first, giving users data control","Professional features could be licensed","Enterprise support and customization services","Potential for MCP tool marketplace"]}
{"type":"entity","name":"MCP_Marketplace","entityType":"Business_Concept","observations":["Could host verified MCP tools for different use cases","Tools rated by performance and reliability","Marketplace takes percentage of tool usage fees","Enterprise tool verification and security scanning","Custom tool development services","Integration support for existing tools"]}
{"type":"entity","name":"Persistence_Of_Vision","entityType":"Concept","observations":["Mental model for continuous AI-human interaction","Like cinema: 24fps creates illusion of smooth motion","Basic-memory provides 'frames' of structured knowledge","Current state: Better than flipbook, not yet digital cinema","Goal: Achieve smoother cognitive continuity between interactions","Proposed by Drew as metaphor for AI conversation continuity"]}
{"type":"entity","name":"Conversation_Continuity_Pattern","entityType":"Usage_Pattern","observations":["Use basic-memory entity/relation schema for conversations","Each chat becomes an entity with observations for key points","Relations link to discussed concepts and other chats","Uses zettelkasten format IDs for natural ordering","Can be used as template/recipe for others","Future possibility: Git SHA integration for versioning"]}
{"type":"entity","name":"Usage_Recipes","entityType":"Feature_Concept","observations":["Predefined patterns users can follow or adapt","Could include conversation tracking recipe","Templates for different knowledge management styles","Shows practical applications of the generic schema","Helps users get started with the system"]}
{"type":"entity","name":"Chat_References","entityType":"Technical_Feature","observations":["Uses ref:* syntax to reference previous conversations","Combines reference semantics with pointer symbolism","Format: ref:*{zettelkasten-id}","Allows explicit context loading between chats","Inspired by C++ references and pointers","Provides memory-model-like access to conversation context","Uses ref:// URI format following MCP Resource pattern","Could support multiple reference schemes (chat/entity/concept)","Makes reference semantics explicit and unambiguous","Aligns with standard URI formatting"]}
{"type":"entity","name":"Chat_Reference_Protocol","entityType":"Technical_Specification","observations":["Uses URI format: ref://basic-memory/chat/[id]","Follows MCP Resource pattern: [protocol]://[host]/[path]","Enables explicit context loading between chats","Can support multiple resource types (chat/entity/concept)","Provides standardized way to reference previous conversations","Example: ref://basic-memory/chat/20240307-drew-ab12ef34"]}
{"type":"entity","name":"20240307-chat-reference-protocol","entityType":"conversation","observations":["Developed ref:// URI format for chat references","Added Chat Reference Protocol to prompt instructions","Discussed implementation of chat continuation","Created complete prompt instructions document","Reference format follows MCP Resource pattern","Reviewed and confirmed complete prompt instructions","Ready to test ref://basic-memory/chat/20240307-chat-reference-protocol in new chat"]}
{"type":"entity","name":"20240307-chat-reference-protocol-test","entityType":"conversation","observations":["First implementation test of chat reference protocol","Testing continuation from 20240307-chat-reference-protocol","Focused on practical implementation of ref:// URI format"]}
{"type":"entity","name":"Write_File_Tool_Usage","entityType":"Tool_Usage_Pattern","observations":["Never use placeholders like '# Rest of...' when writing files - must include complete file content","File content must be complete and valid - partial updates will truncate the file","If showing partial changes, should inform human and let them handle the file write","write_file tool replaces entire file contents - cannot do partial updates","Code files especially must be complete and valid to avoid breaking functionality","Always read_file before write_file to understand current state","Using write_file without reading first risks reverting recent changes","Pattern should be: read current state, make modifications, then write if needed","Especially important in collaborative development where files may have been updated"]}
{"type":"entity","name":"Run_Tests_Tool_Request","entityType":"Feature_Request","observations":["Need to add a tool enabling Claude to run tests locally","Would help with direct validation of code changes","Current workaround: Claude has to ask human to run tests","Should support running specific test functions (e.g. pytest tests/test_memory_service.py::test_create_relations)","Would improve iterative development workflow between human and AI"]}
{"type":"entity","name":"SQLAlchemy_Async_Loading_Pattern","entityType":"Technical_Pattern","observations":["Use selectinload() instead of lazy loading when accessing SQLAlchemy relationships in async code","Lazy loading doesn't work with async due to greenlet context requirements","selectinload performs a single efficient query with an IN clause","Pattern used in basic-memory's EntityRepository for loading relations","Documented in find_by_id method with thorough explanation","Alternative approaches: joinedload (single JOIN query) or subqueryload (subquery approach)","Benefits: prevents 'MissingGreenlet' errors, reduces N+1 query problems","Key insight: load all needed relationships upfront in async code","Example use: selectinload(Entity.outgoing_relations)"]}
{"type":"entity","name":"20241207-sqlalchemy-async-pattern","entityType":"conversation","observations":["Fixed SQLAlchemy async relationship loading issues","Implemented selectinload pattern in EntityRepository","Updated find_by_id to eager load relations","Added documentation about the pattern","Created knowledge graph entry about SQLAlchemy async loading","Fixed failing tests by properly loading relations in memory_service","Discussed SQLAlchemy relationship loading best practices"]}
{"type":"entity","name":"20241207-memory-service-relations","entityType":"conversation","observations":["Fixed SQLAlchemy async loading with selectinload pattern","Updated find_by_id in EntityRepository to eager load relations","Discovered create_relations works but returns empty list","Verified relations are being stored correctly in memory.json","Next step: Work on MemoryService.add_observations implementation","Improved understanding of MCP memory storage format through debugging"]}
{"type":"entity","name":"add_observations_implementation_plan","entityType":"technical_plan","observations":["Follow pattern from create_entity and create_relation methods","File operations first (read & write) - filesystem is source of truth","Database updates in parallel","Simplify current implementation","Current flow is:"," - First read entities and create observations"," - Write files in parallel"," - Update DB indexes sequentially","Key tests needed:"," - Adding observations to multiple entities"," - Verifying filesystem state first"," - Verifying database state"," - Error cases for missing entities"," - Error cases for file operations"]}
{"type":"entity","name":"MCP_Reference_Integration","entityType":"feature_idea","observations":["Can be implemented as a Model Context Protocol integration similar to the fetch tool","Would provide structured way to pass chat references to Claude","Could handle ref:// URL format systematically","Integration would fetch context from referenced chats and inject into conversation","Observed from Claude Desktop UI showing MCP integration pattern with fetch tool","Would be more robust than passing references in chat text"]}
{"type":"entity","name":"Project_Priorities","entityType":"roadmap","observations":["P1: Dogfooding basic-memory system instead of JSON memory store","Future: Implement MCP-based reference system"]}
{"type":"entity","name":"great_observation_loading_saga_20241207","entityType":"debugging_session","observations":["Occurred on December 7, 2024 while debugging basic-memory SQLAlchemy relationship loading","Issue: selectinload() wasn't properly loading relationships in async SQLAlchemy context","Tried multiple solutions: explicit joins, manual loading, various SQLAlchemy loading strategies","Final solution: Using session.refresh() with explicit relationship names","Memorable quote: 'The Great Observation Loading Saga'","Key learning: Sometimes the obvious SQLAlchemy patterns need adaptation for async contexts","Solution preserved in basic-memory repository in EntityRepository.find_by_id()"]}
{"type":"entity","name":"basic_memory_implementation_20241208","entityType":"technical_milestone","observations":["Fixed async SQLAlchemy relationship loading issues by using explicit refresh with relationship names","Established pattern of relationship handling belonging in MemoryService not EntityService","Fixed ID generation flow through Pydantic schemas to DB layer","Standardized error handling using EntityNotFoundError","All 32 tests passing with 70% coverage","Core services (Entity, Observation, Relation) working properly","Ready for MCP server implementation","Notable debugging session: The Great Observation Loading Saga - resolved lazy loading issues","Established clear separation between MemoryService orchestration and individual service responsibilities"]}
{"type":"entity","name":"MCP_Dependency_Risk","entityType":"technical_lesson","observations":["Experienced disruption when MCP npm package disappeared - 'leftpad moment'","Need to ensure basic-memory tools are resilient to external dependency issues","Local implementation of MCP server provides better stability than npm packages","Important to maintain control of critical infrastructure components","Validates DIY/local-first philosophy of basic-memory project","Package manager fragility revealed by simple 'npx @modelcontextprotocol/server-memory' failure"]}
{"type":"entity","name":"basic_memory_project_20241208","entityType":"technical_milestone","observations":["Core MCP server implementation completed with tools: create_entities, search_nodes, open_nodes, add_observations, create_relations, delete_entities, delete_observations","ProjectConfig and dependency injection pattern established","Test framework in place with in-memory DB support","Support for both camelCase (MCP) and snake_case (internal) formats","Filesystem remains source of truth with SQLite as index","Two-way sync pattern identified between Claude MCP tools and direct markdown file editing","Ready for Claude Desktop integration testing phase","Next steps identified: passing tests, markdown format definition, file change tracking, real-world testing","Implementation prioritizes local-first principles with filesystem as source of truth"]}
{"type":"entity","name":"basic_memory_mcp_architecture","entityType":"technical_design","observations":["MemoryServer class extends MCP Server with custom handler registration","Uses ProjectConfig for clean dependency injection and configuration","Memory service can be injected for testing","Handlers exposed as instance attributes for testing","Tool schemas leverage existing Pydantic models"]}
{"type":"entity","name":"basic_memory_sync_considerations","entityType":"design_insight","observations":["Need to handle sync between direct markdown file edits and DB index","Watch for file system changes as potential future enhancement","Consider index rebuild patterns on startup","Keep human-friendly markdown format for direct editing"]}
{"type":"entity","name":"mcp_server_learnings","entityType":"developer_insight","observations":["MCP protocol is new and documentation is still evolving","Test patterns are not well established yet in example implementations","Supporting both camelCase and snake_case helps with protocol/internal compatibility","Server.handle_* naming convention is important for handler registration"]}
{"type":"entity","name":"20241208-mcp-tool-refactoring","entityType":"conversation","observations":["Decision to return structured data via EmbeddedResource instead of TextContent string parsing","Plan to create Pydantic result models (CreateEntitiesResult, SearchNodesResult etc)","Will use application/vnd.basic-memory+json as MIME type for our structured data","Currently debugging test issues with add_observations tool","Entity ID vs name resolution needed in add_observations","Goal is to make tools more joyful to use by eliminating string parsing","MCP spec supports EmbeddedResource for structured data returns"]}
{"type":"entity","name":"Basic Memory MCP Server Implementation","entityType":"technical_notes","observations":["Server implements Model Context Protocol using proper structured data responses","Uses EmbeddedResource with custom MIME type 'application/vnd.basic-memory+json'","Clean separation between input validation and handlers via Pydantic models","All tool operations return structured data through create_response helper","Type safety with Literal types for tool names and proper typing for handlers","Handler registry pattern with TOOL_HANDLERS dictionary","Consistent error handling pattern using MCP error codes","Uses Pydantic ConfigDict for proper ORM integration","Tool schemas organized into Input and Response types","Input validation with Annotated types for extra constraints","Response models consistently use from_attributes=True for ORM data","Entity ID generation moved to model validator on EntityBase","Follows principle of making common operations easy and safe"]}
{"type":"relation","from":"Paul","to":"Basic_Machines","relationType":"created_and_maintains"}
{"type":"relation","from":"basic-memory","to":"Basic_Machines","relationType":"is_component_of"}
{"type":"relation","from":"Paul","to":"basic-memory","relationType":"develops"}
{"type":"relation","from":"fileio_module","to":"basic-memory_implementation_patterns","relationType":"implements"}
{"type":"relation","from":"entity_service","to":"basic-memory_implementation_patterns","relationType":"implements"}
{"type":"relation","from":"observation_service","to":"basic-memory_implementation_patterns","relationType":"implements"}
{"type":"relation","from":"fileio_module","to":"basic-memory","relationType":"is_component_of"}
{"type":"relation","from":"entity_service","to":"basic-memory","relationType":"is_component_of"}
{"type":"relation","from":"observation_service","to":"basic-memory","relationType":"is_component_of"}
{"type":"relation","from":"entity_service","to":"fileio_module","relationType":"uses"}
{"type":"relation","from":"observation_service","to":"fileio_module","relationType":"uses"}
{"type":"relation","from":"observation_management","to":"observation_service","relationType":"influences_design_of"}
{"type":"relation","to":"basic-memory","from":"testing_infrastructure","relationType":"supports"}
{"type":"relation","to":"testing_infrastructure","from":"test_categories","relationType":"implements"}
{"type":"relation","to":"basic-memory","from":"completed_work","relationType":"tracks_progress_of"}
{"type":"relation","to":"basic-memory","from":"future_work","relationType":"guides_development_of"}
{"type":"relation","to":"basic-memory","from":"design_decisions","relationType":"shapes_architecture_of"}
{"type":"relation","to":"basic-memory","from":"concurrency_considerations","relationType":"influences_design_of"}
{"type":"relation","to":"future_work","from":"concurrency_considerations","relationType":"informs"}
{"type":"relation","to":"observation_management","from":"design_decisions","relationType":"guides"}
{"type":"relation","to":"testing_infrastructure","from":"completed_work","relationType":"established"}
{"type":"relation","to":"design_decisions","from":"fileio_module","relationType":"implements"}
{"type":"relation","from":"observation_update_approaches","to":"observation_management","relationType":"analyzes"}
{"type":"relation","from":"bulk_update_approach","to":"observation_update_approaches","relationType":"is_option_of"}
{"type":"relation","from":"tracked_observations_approach","to":"observation_update_approaches","relationType":"is_option_of"}
{"type":"relation","from":"diff_based_approach","to":"observation_update_approaches","relationType":"is_option_of"}
{"type":"relation","from":"position_based_approach","to":"observation_update_approaches","relationType":"is_option_of"}
{"type":"relation","from":"tasks_and_progress","to":"basic-memory","relationType":"tracks_status_of"}
{"type":"relation","from":"design_decisions","to":"observation_update_approaches","relationType":"influences"}
{"type":"relation","from":"observation_update_approaches","to":"future_work","relationType":"informs"}
{"type":"relation","to":"basic-memory_implementation_patterns","from":"error_handling_patterns","relationType":"is_part_of"}
{"type":"relation","to":"basic-memory","from":"data_models","relationType":"implements"}
{"type":"relation","to":"basic-memory","from":"markdown_format","relationType":"defines"}
{"type":"relation","to":"basic-memory","from":"test_driven_development","relationType":"guides_development_of"}
{"type":"relation","to":"basic-memory","from":"architecture_evolution","relationType":"describes_development_of"}
{"type":"relation","to":"basic-memory_implementation_patterns","from":"validation_patterns","relationType":"is_part_of"}
{"type":"relation","to":"design_decisions","from":"architecture_evolution","relationType":"informs"}
{"type":"relation","to":"fileio_module","from":"markdown_format","relationType":"implements"}
{"type":"relation","to":"error_handling_patterns","from":"test_driven_development","relationType":"influenced"}
{"type":"relation","to":"data_models","from":"validation_patterns","relationType":"implements"}
{"type":"relation","to":"markdown_format","from":"markdown_examples","relationType":"documents"}
{"type":"relation","to":"markdown_format","from":"markdown_parsing_rules","relationType":"defines"}
{"type":"relation","to":"data_models","from":"schema_definitions","relationType":"documents"}
{"type":"relation","to":"test_driven_development","from":"test_evolution","relationType":"describes"}
{"type":"relation","to":"architecture_evolution","from":"implementation_challenges","relationType":"influenced"}
{"type":"relation","to":"test_evolution","from":"implementation_challenges","relationType":"shaped"}
{"type":"relation","to":"future_work","from":"implementation_challenges","relationType":"informs"}
{"type":"relation","from":"Basic_Factory","to":"Basic_Machines","relationType":"implements"}
{"type":"relation","from":"Basic_Factory_Components","to":"Basic_Factory","relationType":"is_part_of"}
{"type":"relation","from":"Component_Translation_Process","to":"Basic_Factory_Components","relationType":"enables"}
{"type":"relation","from":"Basic_Machines_Philosophy","to":"Basic_Machines","relationType":"guides"}
{"type":"relation","from":"Paul","to":"Basic_Factory","relationType":"develops"}
{"type":"relation","from":"Paul","to":"Basic_Machines_Philosophy","relationType":"created"}
{"type":"relation","from":"Basic_Machines_Manifesto","to":"Basic_Machines_Philosophy","relationType":"articulates"}
{"type":"relation","from":"AI_Human_Collaboration_Model","to":"Basic_Factory","relationType":"guides_development_of"}
{"type":"relation","from":"Basic_Machines_Manifesto","to":"Paul","relationType":"written_by"}
{"type":"relation","from":"Basic_Machines_Manifesto","to":"Component_Translation_Process","relationType":"documents"}
{"type":"relation","from":"AI_Human_Collaboration_Model","to":"Basic_Machines","relationType":"shapes_development_of"}
{"type":"relation","from":"Basic_Machines_Roadmap","to":"Basic_Machines","relationType":"guides_development_of"}
{"type":"relation","from":"Basic_Machines_Website","to":"Basic_Machines_Roadmap","relationType":"implements_phase_of"}
{"type":"relation","from":"Basic_Factory_Components","to":"Basic_Machines_Website","relationType":"enables"}
{"type":"relation","from":"Basic_Machines_Philosophy","to":"Basic_Machines_Website","relationType":"informs"}
{"type":"relation","from":"Paul","to":"DIY_Ethics","relationType":"embodies"}
{"type":"relation","from":"Basic_Machines_Philosophy","to":"DIY_Ethics","relationType":"incorporates"}
{"type":"relation","from":"Basic_Machines","to":"DIY_Ethics","relationType":"exemplifies"}
{"type":"relation","from":"Component_Translation_Process","to":"Basic_Machines_Philosophy","relationType":"implements"}
{"type":"relation","from":"Basic_Factory_Components","to":"DIY_Ethics","relationType":"demonstrates"}
{"type":"relation","from":"AI_Human_Collaboration_Model","to":"Basic_Machines_Philosophy","relationType":"aligns_with"}
{"type":"relation","from":"AI_Human_Collaboration_Model","to":"Component_Translation_Process","relationType":"guides"}
{"type":"relation","from":"Basic_Machines_Manifesto","to":"Basic_Machines","relationType":"defines_vision_for"}
{"type":"relation","from":"Basic_Machines_Website","to":"Basic_Machines_Manifesto","relationType":"implements_vision_of"}
{"type":"relation","from":"Basic_Factory","to":"AI_Human_Collaboration_Model","relationType":"demonstrates"}
{"type":"relation","from":"Paul","to":"AI_Human_Collaboration_Model","relationType":"developed_with_Claude"}
{"type":"relation","from":"Basic_Factory_Components","to":"Component_Translation_Process","relationType":"created_through"}
{"type":"relation","from":"Basic_Machines_Philosophy","to":"Basic_Factory","relationType":"guides"}
{"type":"relation","from":"Basic_Factory","to":"MCP_Tools","relationType":"integrates"}
{"type":"relation","from":"Basic_Machines_Website","to":"Basic_Factory_Components","relationType":"will_use"}
{"type":"relation","from":"Basic_Machines_Roadmap","to":"Basic_Machines_Philosophy","relationType":"aligns_with"}
{"type":"relation","from":"Component_Translation_Process","to":"MCP_Tools","relationType":"leverages"}
{"type":"relation","from":"Basic_Factory","to":"basic-memory","relationType":"will_document_process_in"}
{"type":"relation","from":"AI_Human_Collaboration_Model","to":"basic-memory","relationType":"will_be_implemented_in"}
{"type":"relation","from":"Basic_Machines_Philosophy","to":"Basic_Machines_Roadmap","relationType":"informs_priorities_of"}
{"type":"relation","from":"basic-memory","to":"Basic_Machines_Philosophy","relationType":"embodies"}
{"type":"relation","from":"Paul","to":"Basic_Machines_Manifesto","relationType":"authored_with_Claude"}
{"type":"relation","from":"Basic_Factory","to":"Component_Translation_Process","relationType":"validated"}
{"type":"relation","from":"AI_Human_Collaboration_Model","to":"MCP_Tools","relationType":"utilizes"}
{"type":"relation","from":"Basic_Factory_Components","to":"Basic_Machines_Roadmap","relationType":"supports"}
{"type":"relation","from":"Basic_Machines_Website","to":"Basic_Factory","relationType":"will_demonstrate"}
{"type":"relation","from":"Basic_Factory","to":"basic-memory-webui","relationType":"enables_development_of"}
{"type":"relation","from":"basic-memory","to":"AI_Human_Development_Methodology","relationType":"implements"}
{"type":"relation","from":"Basic_Machines_Philosophy","to":"AI_Human_Development_Methodology","relationType":"guides"}
{"type":"relation","from":"Basic_Factory_Components","to":"basic-memory-webui","relationType":"provides_ui_for"}
{"type":"relation","from":"Component_Translation_Process","to":"AI_Human_Development_Methodology","relationType":"exemplifies"}
{"type":"relation","from":"Basic_Factory","to":"Basic Components","relationType":"enabled_creation_of"}
{"type":"relation","from":"Basic_Factory","to":"Tool Integration Discovery","relationType":"led_to"}
{"type":"relation","from":"MCP_Integration_Progress","to":"AI_Human_Development_Methodology","relationType":"validates"}
{"type":"relation","from":"Basic_Factory","to":"MCP_Integration_Progress","relationType":"demonstrates"}
{"type":"relation","from":"Basic_Factory","to":"AI_Human_Development_Methodology","relationType":"proves_effectiveness_of"}
{"type":"relation","from":"Basic_Machines_Philosophy","to":"Basic Components","relationType":"inspires_architecture_of"}
{"type":"relation","from":"DIY_Ethics","to":"basic-memory","relationType":"shapes_design_of"}
{"type":"relation","from":"Basic_Machines_Philosophy","to":"Tool Integration Discovery","relationType":"guides_analysis_of"}
{"type":"relation","from":"Basic_Machines_Manifesto","to":"AI_Human_Development_Methodology","relationType":"documents_approach_of"}
{"type":"relation","from":"Basic_Machines_Manifesto","to":"Basic_Factory_Components","relationType":"explains_principles_of"}
{"type":"relation","from":"Basic_Memory_Project_Structure","to":"basic-memory","relationType":"organizes"}
{"type":"relation","from":"Basic_Memory_Database_Schema","to":"basic-memory","relationType":"defines_storage_for"}
{"type":"relation","from":"Basic_Memory_Markdown_Example","to":"Basic_Memory_File_Format","relationType":"demonstrates"}
{"type":"relation","from":"Basic_Memory_Project_Isolation_Decision","to":"Basic_Memory_Future_Enhancement_Weighted_Relations","relationType":"similar_to"}
{"type":"relation","to":"DIY_Ethics","from":"Basic_Memory_Project_Isolation_Decision","relationType":"follows"}
{"type":"relation","from":"Basic_Memory_Implementation_Plan","to":"basic-memory","relationType":"guides"}
{"type":"relation","from":"Basic_Memory_Implementation_Plan","to":"DIY_Ethics","relationType":"follows"}
{"type":"relation","from":"Basic_Memory_Implementation_Plan","to":"Basic_Memory_Database_Schema","relationType":"implements"}
{"type":"relation","from":"Basic_Memory_Implementation_Status","to":"Basic_Memory_Implementation_Plan","relationType":"updates"}
{"type":"relation","from":"Basic_Memory_Observation_Management_Design","to":"Basic_Memory_Technical_Design","relationType":"extends"}
{"type":"relation","from":"Basic_Memory_Architectural_Decisions","to":"DIY_Ethics","relationType":"guided_by"}
{"type":"relation","from":"Basic_Memory_Architectural_Decisions","to":"basic-memory","relationType":"structures"}
{"type":"relation","from":"Basic_Memory_Implementation_Status","to":"basic-memory","relationType":"describes_state_of"}
{"type":"relation","to":"Basic_Memory_Implementation_Status","from":"Basic_Memory_Implementation_Analysis","relationType":"analyzes"}
{"type":"relation","to":"basic-memory","from":"Basic_Memory_Current_Challenges","relationType":"identifies_issues_in"}
{"type":"relation","to":"DIY_Ethics","from":"Basic_Memory_Implementation_Analysis","relationType":"confirms_alignment_with"}
{"type":"relation","to":"Basic_Memory_Observation_Management_Design","from":"Basic_Memory_Observation_Hash_Tracking","relationType":"solves"}
{"type":"relation","to":"DIY_Ethics","from":"Basic_Memory_Observation_Hash_Tracking","relationType":"aligns_with"}
{"type":"relation","to":"Basic_Memory_File_Format","from":"Basic_Memory_Observation_Hash_Tracking","relationType":"preserves"}
{"type":"relation","to":"Basic_Memory_Technical_Design","from":"Basic_Memory_Observation_Hash_Tracking","relationType":"enhances"}
{"type":"relation","from":"Basic_Memory_Repository_Implementation","to":"basic-memory","relationType":"implements_part_of"}
{"type":"relation","from":"Basic_Memory_Repository_Implementation","to":"Basic_Memory_Database_Schema","relationType":"follows"}
{"type":"relation","from":"Basic_Memory_Repository_Implementation","to":"DIY_Ethics","relationType":"aligns_with"}
{"type":"relation","from":"Basic_Memory_Repository_Implementation","to":"testing_infrastructure","relationType":"demonstrates"}
{"type":"relation","from":"Basic_Memory_Repository_Implementation","to":"Basic Foundation","relationType":"inspired_by"}
{"type":"relation","from":"Basic_Memory_Dependencies","to":"basic-memory","relationType":"supports"}
{"type":"relation","from":"Basic_Memory_Dependencies","to":"Basic_Memory_Repository_Implementation","relationType":"enables"}
{"type":"relation","from":"Basic_Memory_Dependencies","to":"testing_infrastructure","relationType":"enables"}
{"type":"relation","from":"Basic_Memory_Current_Architecture","to":"basic-memory","relationType":"describes_state_of"}
{"type":"relation","from":"Basic_Memory_Evolution","to":"Basic_Memory_Current_Architecture","relationType":"explains_development_of"}
{"type":"relation","from":"Basic_Memory_Service_Layer","to":"Basic_Memory_Current_Architecture","relationType":"implements"}
{"type":"relation","from":"Basic_Memory_Schema_Design","to":"Basic_Memory_Current_Architecture","relationType":"implements"}
{"type":"relation","from":"Basic_Memory_Evolution","to":"Basic_Memory_Implementation_Plan","relationType":"reflects_on"}
{"type":"relation","from":"Basic_Memory_Evolution","to":"DIY_Ethics","relationType":"demonstrates_alignment_with"}
{"type":"relation","from":"Basic_Memory_Current_Architecture","to":"DIY_Ethics","relationType":"embodies"}
{"type":"relation","from":"Basic_Memory_Service_Layer","to":"fileio_module","relationType":"uses"}
{"type":"relation","from":"Basic_Memory_Schema_Design","to":"markdown_format","relationType":"implements"}
{"type":"relation","to":"basic-memory","from":"Basic_Memory_Next_Tasks","relationType":"guides_development_of"}
{"type":"relation","to":"DIY_Ethics","from":"Basic_Memory_Next_Tasks","relationType":"aligns_with"}
{"type":"relation","to":"Basic_Memory_Current_Architecture","from":"Basic_Memory_Next_Tasks","relationType":"extends"}
{"type":"relation","from":"Basic_Memory_Meta_Experience","to":"basic-memory","relationType":"validates_design_of"}
{"type":"relation","from":"Basic_Memory_Meta_Experience","to":"DIY_Ethics","relationType":"demonstrates_principles_of"}
{"type":"relation","from":"Basic_Memory_Meta_Experience","to":"design_decisions","relationType":"reinforces"}
{"type":"relation","from":"Basic_Memory_Meta_Experience","to":"Basic_Memory_Current_Architecture","relationType":"validates"}
{"type":"relation","from":"Model_Context_Protocol","to":"basic-memory","relationType":"enables"}
{"type":"relation","from":"basic-memory_core_principles","to":"basic-memory","relationType":"guides"}
{"type":"relation","from":"basic-memory_core_principles","to":"DIY_Ethics","relationType":"aligns_with"}
{"type":"relation","from":"basic-memory_business_model","to":"basic-memory","relationType":"defines_sustainability_for"}
{"type":"relation","from":"basic-memory_business_model","to":"DIY_Ethics","relationType":"maintains_alignment_with"}
{"type":"relation","from":"basic-memory_cli","to":"basic-memory","relationType":"provides_interface_for"}
{"type":"relation","from":"basic-memory_cli","to":"Model_Context_Protocol","relationType":"integrates_with"}
{"type":"relation","from":"basic-memory_export_format","to":"basic-memory","relationType":"standardizes_output_of"}
{"type":"relation","from":"basic-memory_export_format","to":"markdown_format","relationType":"extends"}
{"type":"relation","from":"basic-memory_core_principles","to":"Basic_Machines_Philosophy","relationType":"implements"}
{"type":"relation","from":"Model_Context_Protocol","to":"AI_Human_Collaboration_Model","relationType":"enables"}
{"type":"relation","from":"relation_service","to":"basic-memory","relationType":"will_be_component_of"}
{"type":"relation","from":"relation_service","to":"service_layer_patterns","relationType":"follows"}
{"type":"relation","from":"relation_service","to":"fileio_patterns","relationType":"uses"}
{"type":"relation","from":"relation_service","to":"database_models","relationType":"uses"}
{"type":"relation","from":"relation_service","to":"repository_patterns","relationType":"implements"}
{"type":"relation","from":"relation_service_design","to":"relation_service","relationType":"guides_implementation_of"}
{"type":"relation","from":"relation_service_implementation_plan","to":"relation_service","relationType":"defines_implementation_of"}
{"type":"relation","from":"relation_service_challenges","to":"relation_service_design","relationType":"informs"}
{"type":"relation","from":"relation_file_format","to":"markdown_format","relationType":"extends"}
{"type":"relation","from":"relation_service_error_handling","to":"service_layer_patterns","relationType":"implements"}
{"type":"relation","from":"relation_service_testing","to":"testing_infrastructure","relationType":"extends"}
{"type":"relation","from":"fileio_patterns","to":"service_layer_patterns","relationType":"enables"}
{"type":"relation","from":"database_models","to":"repository_patterns","relationType":"enables"}
{"type":"relation","from":"relation_service","to":"entity_service","relationType":"coordinates_with"}
{"type":"relation","from":"relation_file_format","to":"relation_service","relationType":"defines_storage_for"}
{"type":"relation","from":"relation_service_error_handling","to":"relation_service","relationType":"ensures_reliability_of"}
{"type":"relation","from":"relation_service_testing","to":"relation_service","relationType":"verifies"}
{"type":"relation","from":"service_layer_patterns","to":"basic-memory_implementation_patterns","relationType":"implements"}
{"type":"relation","from":"repository_patterns","to":"basic-memory_implementation_patterns","relationType":"implements"}
{"type":"relation","from":"fileio_patterns","to":"basic-memory_implementation_patterns","relationType":"implements"}
{"type":"relation","from":"database_models","to":"basic-memory_implementation_patterns","relationType":"implements"}
{"type":"relation","from":"relation_service_challenges","to":"implementation_challenges","relationType":"extends"}
{"type":"relation","from":"relation_service_implementation_plan","to":"future_work","relationType":"details"}
{"type":"relation","from":"relation_service_design","to":"design_decisions","relationType":"aligns_with"}
{"type":"relation","from":"relation_file_format","to":"design_decisions","relationType":"follows"}
{"type":"relation","from":"pytest_patterns","to":"testing_infrastructure","relationType":"extends"}
{"type":"relation","from":"relation_implementation_learnings","to":"basic-memory_implementation_patterns","relationType":"informs"}
{"type":"relation","from":"test_driven_insights","to":"test_driven_development","relationType":"enriches"}
{"type":"relation","from":"meta_development_insights","to":"AI_Human_Collaboration_Model","relationType":"improves"}
{"type":"relation","from":"relation_implementation_learnings","to":"relation_service","relationType":"guides_implementation_of"}
{"type":"relation","from":"pytest_patterns","to":"test_evolution","relationType":"demonstrates"}
{"type":"relation","from":"test_driven_insights","to":"design_decisions","relationType":"influences"}
{"type":"relation","from":"meta_development_insights","to":"architecture_evolution","relationType":"informs"}
{"type":"relation","from":"relation_service","to":"relation_implementation_learnings","relationType":"validates"}
{"type":"relation","from":"test_driven_insights","to":"implementation_challenges","relationType":"helps_solve"}
{"type":"relation","from":"AI_Assistant_Learnings","to":"meta_development_insights","relationType":"enriches"}
{"type":"relation","from":"Effective_Response_Patterns","to":"AI_Assistant_Learnings","relationType":"implements"}
{"type":"relation","from":"AI_Context_Management","to":"AI_Human_Collaboration_Model","relationType":"improves"}
{"type":"relation","from":"AI_Tool_Usage_Patterns","to":"AI_Context_Management","relationType":"enables"}
{"type":"relation","from":"AI_Assistant_Learnings","to":"Basic_Memory_Meta_Experience","relationType":"validates"}
{"type":"relation","from":"AI_Tool_Usage_Patterns","to":"Model_Context_Protocol","relationType":"demonstrates_effective_use_of"}
{"type":"relation","from":"AI_Context_Management","to":"basic-memory","relationType":"validates_design_of"}
{"type":"relation","from":"Effective_Response_Patterns","to":"AI_Human_Development_Methodology","relationType":"refines"}
{"type":"relation","to":"relation_service","from":"relation_service_patterns","relationType":"guides"}
{"type":"relation","to":"test_driven_development","from":"test_driven_insights_relations","relationType":"enriches"}
{"type":"relation","to":"implementation_challenges","from":"relation_service_learnings","relationType":"solves"}
{"type":"relation","to":"basic-memory_implementation_patterns","from":"relation_service_patterns","relationType":"implements"}
{"type":"relation","to":"markdown_format","from":"relation_service_patterns","relationType":"extends"}
{"type":"relation","to":"service_layer_patterns","from":"relation_service_patterns","relationType":"refines"}
{"type":"relation","from":"packaging_learnings","to":"implementation_challenges","relationType":"informs"}
{"type":"relation","from":"packaging_learnings","to":"test_driven_development","relationType":"impacts"}
{"type":"relation","to":"basic-memory","from":"Recent_Implementation_Progress","relationType":"updates_status_of"}
{"type":"relation","to":"future_work","from":"Next_Steps","relationType":"extends"}
{"type":"relation","to":"design_decisions","from":"Development_Practices","relationType":"informs"}
{"type":"relation","to":"packaging_learnings","from":"Development_Practices","relationType":"incorporates"}
{"type":"relation","to":"test_driven_development","from":"Development_Practices","relationType":"refines"}
{"type":"relation","to":"basic-memory_implementation_patterns","from":"Development_Practices","relationType":"enhances"}
{"type":"relation","from":"Basic_Memory_MCP","to":"MCP_Server_Implementation","relationType":"follows"}
{"type":"relation","from":"Basic_Memory_MCP","to":"MCP_Tools","relationType":"uses"}
{"type":"relation","from":"Basic_Memory","to":"MCP_Server_Implementation","relationType":"implements"}
{"type":"relation","from":"Basic_Memory_Testing","to":"Memory_Service_Tests","relationType":"includes"}
{"type":"relation","from":"Basic_Memory_Testing","to":"MCP_Server_Tests","relationType":"includes"}
{"type":"relation","from":"Memory_Service_Tests","to":"Basic_Memory_MCP","relationType":"validates"}
{"type":"relation","from":"MCP_Server_Tests","to":"Basic_Memory_MCP","relationType":"validates"}
{"type":"relation","from":"Service_Interface_Audit","to":"Memory_Service_Refactoring","relationType":"informs"}
{"type":"relation","from":"Memory_Service_Refactoring","to":"Basic_Memory_MCP","relationType":"affects"}
{"type":"relation","from":"Memory_Service_Patterns","to":"Basic_Memory_MCP","relationType":"improves"}
{"type":"relation","from":"Pydantic_Create_Pattern","to":"Memory_Service_Patterns","relationType":"enables"}
{"type":"relation","from":"Pydantic_Create_Pattern","to":"Basic_Memory_MCP","relationType":"improves"}
{"type":"relation","from":"MCP_Marketplace","to":"Basic_Memory_Business","relationType":"enables"}
{"type":"relation","from":"Basic_Memory","to":"MCP_Marketplace","relationType":"could_integrate_with"}
{"type":"relation","from":"Persistence_Of_Vision","to":"Basic_Memory","relationType":"helps_achieve"}
{"type":"relation","from":"Drew","to":"Persistence_Of_Vision","relationType":"conceptualized"}
{"type":"relation","to":"Usage_Recipes","from":"Conversation_Continuity_Pattern","relationType":"is_example_of"}
{"type":"relation","to":"Basic_Memory","from":"Usage_Recipes","relationType":"enhances"}
{"type":"relation","to":"Basic_Memory","from":"Chat_References","relationType":"enhances"}
{"type":"relation","to":"Conversation_Continuity_Pattern","from":"Chat_References","relationType":"implements"}
{"type":"relation","from":"20240307-chat-reference-protocol-test","to":"20240307-chat-reference-protocol","relationType":"continues_from"}
{"type":"relation","from_id":"Run_Tests_Tool_Request","to_id":"Basic_Machines","relationType":"enhances","context":"development workflow improvement"}
{"type":"relation","from_id":"SQLAlchemy_Async_Loading_Pattern","to_id":"basic-memory","relation_type":"improves","context":"database performance and async compatibility"}
{"type":"relation","from_id":"SQLAlchemy_Async_Loading_Pattern","to_id":"Entity","relation_type":"applies_to","context":"relationship loading strategy"}
{"type":"relation","from":"MCP_Reference_Integration","to":"Project_Priorities","relationType":"prioritized_after"}
{"type":"relation","from":"great_observation_loading_saga_20241207","to":"Basic_Memory","relationType":"occurred_in"}
{"type":"relation","from":"great_observation_loading_saga_20241207","to":"SQLAlchemy","relationType":"relates_to"}
{"type":"relation","from":"basic_memory_implementation_20241208","to":"Basic_Memory","relationType":"improves"}
{"type":"relation","from":"great_observation_loading_saga_20241207","to":"basic_memory_implementation_20241208","relationType":"leads_to"}
{"type":"relation","from":"MCP_Dependency_Risk","to":"DIY_Ethics","relationType":"validates"}
{"type":"relation","from":"MCP_Dependency_Risk","to":"basic-memory_core_principles","relationType":"reinforces"}
{"type":"relation","from":"MCP_Dependency_Risk","to":"Basic_Memory_Implementation_Plan","relationType":"influences"}
{"type":"relation","from":"basic_memory_mcp_architecture","to":"basic_memory_project_20241208","relationType":"implements"}
{"type":"relation","from":"basic_memory_sync_considerations","to":"basic_memory_project_20241208","relationType":"influences"}
{"type":"relation","from":"mcp_server_learnings","to":"basic_memory_mcp_architecture","relationType":"informs"}
{"type":"relation","from":"20241208-mcp-tool-refactoring","to":"Basic_Memory_MCP","relationType":"improves"}
{"type":"relation","from":"20241208-mcp-tool-refactoring","to":"Basic_Memory_Implementation_Plan","relationType":"implements"}
+65 -30
View File
@@ -1,9 +1,9 @@
[project]
name = "basic-memory"
version = "0.5.0"
dynamic = ["version"]
description = "Local-first knowledge management combining Zettelkasten with knowledge graphs"
readme = "README.md"
requires-python = ">=3.12.1"
requires-python = ">=3.12"
license = { text = "AGPL-3.0-or-later" }
authors = [
{ name = "Basic Machines", email = "hello@basic-machines.co" }
@@ -15,7 +15,6 @@ dependencies = [
"aiosqlite>=0.20.0",
"greenlet>=3.1.1",
"pydantic[email,timezone]>=2.10.3",
"icecream>=2.1.3",
"mcp>=1.2.0",
"pydantic-settings>=2.6.1",
"loguru>=0.7.3",
@@ -28,7 +27,16 @@ dependencies = [
"watchfiles>=1.0.4",
"fastapi[standard]>=0.115.8",
"alembic>=1.14.1",
"qasync>=0.27.1",
"pillow>=11.1.0",
"pybars3>=0.9.7",
"fastmcp>=2.10.2",
"pyjwt>=2.10.1",
"python-dotenv>=1.1.0",
"pytest-aio>=1.9.0",
"aiofiles>=24.1.0", # Async file I/O
"logfire>=0.73.0", # Optional observability (disabled by default via config)
"asyncpg>=0.30.0",
"nest-asyncio>=1.6.0", # For Alembic migrations with Postgres
]
@@ -39,40 +47,52 @@ Documentation = "https://github.com/basicmachines-co/basic-memory#readme"
[project.scripts]
basic-memory = "basic_memory.cli.main:app"
bm = "basic_memory.cli.main:app"
[build-system]
requires = ["hatchling"]
requires = ["hatchling", "uv-dynamic-versioning>=0.7.0"]
build-backend = "hatchling.build"
[tool.pytest.ini_options]
pythonpath = ["src", "tests"]
addopts = "--cov=basic_memory --cov-report term-missing -ra -q"
testpaths = ["tests"]
addopts = "--cov=basic_memory --cov-report term-missing"
testpaths = ["tests", "test-int"]
asyncio_mode = "strict"
asyncio_default_fixture_loop_scope = "function"
markers = [
"benchmark: Performance benchmark tests (deselect with '-m \"not benchmark\"')",
"slow: Slow-running tests (deselect with '-m \"not slow\"')",
"postgres: Tests that run against Postgres backend (deselect with '-m \"not postgres\"')",
"windows: Windows-specific tests (deselect with '-m \"not windows\"')",
]
[tool.ruff]
line-length = 100
target-version = "py312"
[tool.uv]
dev-dependencies = [
[dependency-groups]
dev = [
"gevent>=24.11.1",
"icecream>=2.1.3",
"pytest>=8.3.4",
"pytest-cov>=4.1.0",
"pytest-mock>=3.12.0",
"pytest-asyncio>=0.24.0",
"pytest-xdist>=3.0.0",
"ruff>=0.1.6",
"pytest>=8.3.4",
"pytest-cov>=4.1.0",
"pytest-mock>=3.12.0",
"pytest-asyncio>=0.24.0",
"ruff>=0.1.6",
"cx-freeze>=7.2.10",
"pyqt6>=6.8.1",
"freezegun>=1.5.5",
]
[tool.hatch.version]
source = "uv-dynamic-versioning"
[tool.uv-dynamic-versioning]
vcs = "git"
style = "pep440"
bump = true
fallback-version = "0.0.0"
[tool.pyright]
include = ["src/"]
exclude = ["**/__pycache__"]
@@ -83,20 +103,35 @@ reportMissingTypeStubs = false
pythonVersion = "3.12"
[tool.semantic_release]
version_variables = [
"src/basic_memory/__init__.py:__version__",
]
version_toml = [
"pyproject.toml:project.version",
]
major_on_zero = false
branch = "main"
changelog_file = "CHANGELOG.md"
build_command = "pip install uv && uv build"
dist_path = "dist/"
upload_to_pypi = true
commit_message = "chore(release): {version} [skip ci]"
[tool.coverage.run]
concurrency = ["thread", "gevent"]
[tool.coverage.report]
exclude_lines = [
"pragma: no cover",
"def __repr__",
"if self.debug:",
"if settings.DEBUG",
"raise AssertionError",
"raise NotImplementedError",
"if 0:",
"if __name__ == .__main__.:",
"class .*\\bProtocol\\):",
"@(abc\\.)?abstractmethod",
]
# Exclude specific modules that are difficult to test comprehensively
omit = [
"*/external_auth_provider.py", # External HTTP calls to OAuth providers
"*/supabase_auth_provider.py", # External HTTP calls to Supabase APIs
"*/watch_service.py", # File system watching - complex integration testing
"*/background_sync.py", # Background processes
"*/cli/main.py", # CLI entry point
"*/mcp/tools/project_management.py", # Covered by integration tests
"*/mcp/tools/sync_status.py", # Covered by integration tests
"*/services/migration_service.py", # Complex migration scenarios
]
[tool.logfire]
ignore_no_config = true
-36
View File
@@ -1,36 +0,0 @@
#!/bin/bash
set -e
echo "Welcome to Basic Memory installer"
# 1. Install uv if not present
if ! command -v uv &> /dev/null; then
echo "Installing uv package manager..."
curl -LsSf https://github.com/astral-sh/uv/releases/download/0.1.23/uv-installer.sh | sh
fi
# 2. Configure Claude Desktop
echo "Configuring Claude Desktop..."
CONFIG_FILE="$HOME/Library/Application Support/Claude/claude_desktop_config.json"
# Create config directory if it doesn't exist
mkdir -p "$(dirname "$CONFIG_FILE")"
# If config file doesn't exist, create it with initial structure
if [ ! -f "$CONFIG_FILE" ]; then
echo '{"mcpServers": {}}' > "$CONFIG_FILE"
fi
# Add/update the basic-memory config using jq
jq '.mcpServers."basic-memory" = {
"command": "uvx",
"args": ["basic-memory"]
}' "$CONFIG_FILE" > "$CONFIG_FILE.tmp" && mv "$CONFIG_FILE.tmp" "$CONFIG_FILE"
echo "Installation complete! Basic Memory is now available in Claude Desktop."
echo "Please restart Claude Desktop for changes to take effect."
echo -e "\nQuick Start:"
echo "1. You can run sync directly using: uvx basic-memory sync"
echo "2. Optionally, install globally with: uv pip install basic-memory"
echo -e "\nBuilt with ♥️ by Basic Machines."
+15
View File
@@ -0,0 +1,15 @@
# Smithery configuration file: https://smithery.ai/docs/config#smitheryyaml
startCommand:
type: stdio
configSchema:
# JSON Schema defining the configuration options for the MCP.
type: object
properties: {}
description: No configuration required. This MCP server runs using the default command.
commandFunction: |-
(config) => ({
command: 'basic-memory',
args: ['mcp']
})
exampleConfig: {}
@@ -0,0 +1,156 @@
---
title: 'SPEC-1: Specification-Driven Development Process'
type: spec
permalink: specs/spec-1-specification-driven-development-process
tags:
- process
- specification
- development
- meta
---
# SPEC-1: Specification-Driven Development Process
## Why
We're implementing specification-driven development to solve the complexity and circular refactoring issues in our web development process.
Instead of getting lost in framework details and type gymnastics, we start with clear specifications that drive implementation.
The default approach of adhoc development with AI agents tends to result in:
- Circular refactoring cycles
- Fighting framework complexity
- Lost context between sessions
- Unclear requirements and scope
## What
This spec defines our process for using basic-memory as the specification engine to build basic-memory-cloud.
We're creating a recursive development pattern where basic-memory manages the specs that drive the development of basic-memory-cloud.
**Affected Areas:**
- All future component development
- Architecture decisions
- Agent collaboration workflows
- Knowledge management and context preservation
## How (High Level)
### Specification Structure
Name: Spec names should be numbered sequentially, followed by a description eg. `SPEC-X - Simple Description.md`.
See: [[Spec-2: Slash Commands Reference]]
Every spec is a complete thought containing:
- **Why**: The reasoning and problem being solved
- **What**: What is affected or changed
- **How**: High-level approach to implementation
- **How to Evaluate**: Testing/validation procedure
- Additional context as needed
### Living Specification Format
Specifications are **living documents** that evolve throughout implementation:
**Progress Tracking:**
- **Completed items**: Use ✅ checkmark emoji for implemented features
- **Pending items**: Use `- [ ]` GitHub-style checkboxes for remaining tasks
- **In-progress items**: Use `- [x]` when work is actively underway
**Status Philosophy:**
- **Avoid static status headers** like "COMPLETE" or "IN PROGRESS" that become stale
- **Use checklists within content** to show granular implementation progress
- **Keep specs informative** while providing clear progress visibility
- **Update continuously** as understanding and implementation evolve
**Example Format:**
```markdown
### ComponentName
- ✅ Basic functionality implemented
- ✅ Props and events defined
- - [ ] Add sorting controls
- - [ ] Improve accessibility
- - [x] Currently implementing responsive design
```
This creates **git-friendly progress tracking** where `[ ]` easily becomes `[x]` or ✅ when completed, and specs remain valuable throughout the development lifecycle.
## Claude Code
We will leverage Claude Code capabilities to make the process semi-automated.
- Slash commands: define repeatable steps in the process (create spec, implement, review, etc)
- Agents: define roles to carry out instructions (front end developer, baskend developer, etc)
- MCP tools: enable agents to implement specs via actions (write code, test, etc)
### Workflow
1. **Create**: Write spec as complete thought in `/specs` folder
2. **Discuss**: Iterate and refine through agent collaboration
3. **Implement**: Hand spec to appropriate specialist agent
4. **Validate**: Review implementation against spec criteria
5. **Document**: Update spec with learnings and decisions
### Slash Commands
Claude slash commands are used to manage the flow.
These are simple instructions to help make the process uniform.
They can be updated and refined as needed.
- `/spec create [name]` - Create new specification
- `/spec status` - Show current spec states
- `/spec implement [name]` - Hand to appropriate agent
- `/spec review [name]` - Validate implementation
### Agent Orchestration
Agents are defined with clear roles, for instance:
- **system-architect**: Creates high-level specs, ADRs, architectural decisions
- **vue-developer**: Component specs, UI patterns, frontend architecture
- **python-developer**: Implementation specs, technical details, backend logic
-
- Each agent reads/updates specs through basic-memory tools.
## How to Evaluate
### Success Criteria
- Specs provide clear, actionable guidance for implementation
- Reduced circular refactoring and scope creep
- Persistent context across development sessions
- Clean separation between "what/why" and implementation details
- Specs record a history of what happened and why for historical context
### Testing Procedure
1. Create a spec for an existing problematic component
2. Have an agent implement following only the spec
3. Compare result quality and development speed vs. ad-hoc approach
4. Measure context preservation across sessions
5. Evaluate spec clarity and completeness
### Metrics
- Time from spec to working implementation
- Number of refactoring cycles required
- Agent understanding of requirements
- Spec reusability for similar components
## Notes
- Start simple: specs are just complete thoughts, not heavy processes
- Use basic-memory's knowledge graph to link specs, decisions, components
- Let the process evolve naturally based on what works
- Focus on solving the actual problem: Manage complexity in development
## Observations
- [problem] Web development without clear goals and documentation circular refactoring cycles #complexity
- [solution] Specification-driven development reduces scope creep and context loss #process-improvement
- [pattern] basic-memory as specification engine creates recursive development loop #meta-development
- [workflow] Five-step process: Create → Discuss → Implement → Validate → Document #methodology
- [tool] Slash commands provide uniform process automation #automation
- [agent-pattern] Three specialized agents handle different implementation domains #specialization
- [success-metric] Time from spec to working implementation measures process efficiency #measurement
- [learning] Process should evolve naturally based on what works in practice #adaptation
- [format] Living specifications use checklists for progress tracking instead of static status headers #documentation
- [evolution] Specs evolve throughout implementation maintaining value as working documents #continuous-improvement
## Relations
- spec [[Spec-2: Slash Commands Reference]]
- spec [[Spec-3: Agent Definitions]]
@@ -0,0 +1,569 @@
---
title: 'SPEC-10: Unified Deployment Workflow and Event Tracking'
type: spec
permalink: specs/spec-10-unified-deployment-workflow-event-tracking
tags:
- workflow
- deployment
- event-sourcing
- architecture
- simplification
---
# SPEC-10: Unified Deployment Workflow and Event Tracking
## Why
We replaced a complex multi-workflow system with DBOS orchestration that was proving to be more trouble than it was worth. The previous architecture had four separate workflows (`tenant_provisioning`, `tenant_update`, `tenant_deployment`, `tenant_undeploy`) with overlapping logic, complex state management, and fragmented event tracking. DBOS added unnecessary complexity without providing sufficient value, leading to harder debugging and maintenance.
**Problems Solved:**
- **Framework Complexity**: DBOS configuration overhead and fighting framework limitations
- **Code Duplication**: Multiple workflows implementing similar operations with duplicate logic
- **Poor Observability**: Fragmented event tracking across workflow boundaries
- **Maintenance Overhead**: Complex orchestration for fundamentally simple operations
- **Debugging Difficulty**: Framework abstractions hiding simple Python stack traces
## What
This spec documents the architectural simplification that consolidates tenant lifecycle management into a unified system with comprehensive event tracking.
**Affected Areas:**
- Tenant deployment workflows (provisioning, updates, undeploying)
- Event sourcing and workflow tracking infrastructure
- API endpoints for tenant operations
- Database schema for workflow and event correlation
- Integration testing for tenant lifecycle operations
**Key Changes:**
- **Removed DBOS entirely** - eliminated framework dependency and complexity
- **Consolidated 4 workflows → 2 unified deployment workflows (deploy/undeploy)**
- **Added workflow tracking system** with complete event correlation
- **Simplified API surface** - single `/deploy` endpoint handles all scenarios
- **Enhanced observability** through event sourcing with workflow grouping
## How (High Level)
### Architectural Philosophy
**Embrace simplicity over framework complexity** - use well-structured Python with proper database design instead of complex orchestration frameworks.
### Core Components
#### 1. Unified Deployment Workflow
```python
class TenantDeploymentWorkflow:
async def deploy_tenant_workflow(self, tenant_id: str, workflow_id: UUID, image_tag: str = None):
# Single workflow handles both initial provisioning AND updates
# Each step is idempotent and handles its own error recovery
# Database transactions provide the durability we need
await self.start_deployment_step(workflow_id, tenant_uuid, image_tag)
await self.create_fly_app_step(workflow_id, tenant_uuid)
await self.create_bucket_step(workflow_id, tenant_uuid)
await self.deploy_machine_step(workflow_id, tenant_uuid, image_tag)
await self.complete_deployment_step(workflow_id, tenant_uuid, image_tag, deployment_time)
```
**Key Benefits:**
- **Handles both provisioning and updates** in single workflow
- **Idempotent operations** - safe to retry any step
- **Clean error handling** via simple Python exceptions
- **Resumable** - can restart from any failed step
#### 2. Workflow Tracking System
**Database Schema:**
```sql
CREATE TABLE workflow (
id UUID PRIMARY KEY,
workflow_type VARCHAR(50) NOT NULL, -- 'tenant_deployment', 'tenant_undeploy'
tenant_id UUID REFERENCES tenant(id),
status VARCHAR(20) DEFAULT 'running', -- 'running', 'completed', 'failed'
workflow_metadata JSONB DEFAULT '{}' -- image_tag, etc.
);
ALTER TABLE event ADD COLUMN workflow_id UUID REFERENCES workflow(id);
```
**Event Correlation:**
- Every workflow operation generates events tagged with `workflow_id`
- Complete audit trail from workflow start to completion
- Events grouped by workflow for easy reconstruction of operations
#### 3. Parameter Standardization
All workflow methods follow consistent signature pattern:
```python
async def method_name(self, session: AsyncSession, workflow_id: UUID | None, tenant_id: UUID, ...)
```
**Benefits:**
- **Consistent event tagging** - all events properly correlated
- **Clear method contracts** - workflow_id always first parameter
- **Type safety** - proper UUID handling throughout
### Implementation Strategy
#### Phase 1: Workflow Consolidation ✅ COMPLETED
- [x] **Remove DBOS dependency** - eliminated dbos_config.py and all DBOS imports
- [x] **Create unified TenantDeploymentWorkflow** - handles both provisioning and updates
- [x] **Remove legacy workflows** - deleted tenant_provisioning.py, tenant_update.py
- [x] **Simplify API endpoints** - consolidated to single `/deploy` endpoint
- [x] **Update integration tests** - comprehensive edge case testing
#### Phase 2: Workflow Tracking System ✅ COMPLETED
- [x] **Database migration** - added workflow table and event.workflow_id foreign key
- [x] **Workflow repository** - CRUD operations for workflow records
- [x] **Event correlation** - all workflow events tagged with workflow_id
- [x] **Comprehensive testing** - workflow lifecycle and event grouping tests
#### Phase 3: Parameter Standardization ✅ COMPLETED
- [x] **Standardize method signatures** - workflow_id as first parameter pattern
- [x] **Fix event tagging** - ensure all workflow events properly correlated
- [x] **Update service methods** - consistent parameter order across tenant_service
- [x] **Integration test validation** - verify complete event sequences
### Architectural Benefits
#### Code Simplification
- **39 files changed**: 2,247 additions, 3,256 deletions (net -1,009 lines)
- **Eliminated framework complexity** - no more DBOS configuration or abstractions
- **Consolidated logic** - single deployment workflow vs 4 separate workflows
- **Cleaner API surface** - unified endpoint vs multiple workflow-specific endpoints
#### Enhanced Observability
- **Complete event correlation** - every workflow event tagged with workflow_id
- **Audit trail reconstruction** - can trace entire tenant lifecycle through events
- **Workflow status tracking** - running/completed/failed states in database
- **Comprehensive testing** - edge cases covered with real infrastructure
#### Operational Benefits
- **Simpler debugging** - plain Python stack traces vs framework abstractions
- **Reduced dependencies** - one less complex framework to maintain
- **Better error handling** - explicit exception handling vs framework magic
- **Easier maintenance** - straightforward Python code vs orchestration complexity
## How to Evaluate
### Success Criteria
#### Functional Completeness ✅ VERIFIED
- [x] **Unified deployment workflow** handles both initial provisioning and updates
- [x] **Undeploy workflow** properly integrated with event tracking
- [x] **All operations idempotent** - safe to retry any step without duplication
- [x] **Complete tenant lifecycle** - provision → active → update → undeploy
#### Event Tracking and Correlation ✅ VERIFIED
- [x] **All workflow events tagged** with proper workflow_id
- [x] **Event sequence verification** - tests assert exact event order and content
- [x] **Workflow grouping** - events can be queried by workflow_id for complete audit trail
- [x] **Cross-workflow isolation** - deployment vs undeploy events properly separated
#### Database Schema and Performance ✅ VERIFIED
- [x] **Migration applied** - workflow table and event.workflow_id column created
- [x] **Proper indexing** - performance optimized queries on workflow_type, tenant_id, status
- [x] **Foreign key constraints** - referential integrity between workflows and events
- [x] **Database triggers** - updated_at timestamp automation
#### Test Coverage ✅ COMPREHENSIVE
- [x] **Unit tests**: 4 workflow tracking tests covering lifecycle and event grouping
- [x] **Integration tests**: Real infrastructure testing with Fly.io resources
- [x] **Edge case coverage**: Failed deployments, partial state recovery, resource conflicts
- [x] **Event sequence verification**: Exact event order and content validation
### Testing Procedure
#### Unit Test Validation ✅ PASSING
```bash
cd apps/cloud && pytest tests/test_workflow_tracking.py -v
# 4/4 tests passing - workflow lifecycle and event grouping
```
#### Integration Test Validation ✅ PASSING
```bash
cd apps/cloud && pytest tests/integration/test_tenant_workflow_deployment_integration.py -v
cd apps/cloud && pytest tests/integration/test_tenant_workflow_undeploy_integration.py -v
# Comprehensive real infrastructure testing with actual Fly.io resources
# Tests provision → deploy → update → undeploy → cleanup cycles
```
### Performance Metrics
#### Code Metrics ✅ ACHIEVED
- **Net code reduction**: -1,009 lines (3,256 deletions, 2,247 additions)
- **Workflow consolidation**: 4 workflows → 1 unified deployment workflow
- **Dependency reduction**: Removed DBOS framework dependency entirely
- **API simplification**: Multiple endpoints → single `/deploy` endpoint
#### Operational Metrics ✅ VERIFIED
- **Event correlation**: 100% of workflow events properly tagged with workflow_id
- **Audit trail completeness**: Full tenant lifecycle traceable through event sequences
- **Error handling**: Clean Python exceptions vs framework abstractions
- **Debugging simplicity**: Direct stack traces vs orchestration complexity
### Implementation Status: ✅ COMPLETE
All phases completed successfully with comprehensive testing and verification:
**Phase 1 - Workflow Consolidation**: ✅ COMPLETE
- Removed DBOS dependency and consolidated workflows
- Unified deployment workflow handles all scenarios
- Comprehensive integration testing with real infrastructure
**Phase 2 - Workflow Tracking**: ✅ COMPLETE
- Database schema implemented with proper indexing
- Event correlation system fully functional
- Complete audit trail capability verified
**Phase 3 - Parameter Standardization**: ✅ COMPLETE
- Consistent method signatures across all workflow methods
- All events properly tagged with workflow_id
- Type safety verified across entire codebase
**Phase 4 - Asynchronous Job Queuing**:
**Goal**: Transform synchronous deployment workflows into background jobs for better user experience and system reliability.
**Current Problem**:
- Deployment API calls are synchronous - users wait for entire tenant provisioning (30-60 seconds)
- No retry mechanism for failed operations
- HTTP timeouts on long-running deployments
- Poor user experience during infrastructure provisioning
**Solution**: Redis-backed job queue with arq for reliable background processing
#### Architecture Overview
```python
# API Layer: Return immediately with job tracking
@router.post("/{tenant_id}/deploy")
async def deploy_tenant(tenant_id: UUID):
# Create workflow record in Postgres
workflow = await workflow_repo.create_workflow("tenant_deployment", tenant_id)
# Enqueue job in Redis
job = await arq_pool.enqueue_job('deploy_tenant_task', tenant_id, workflow.id)
# Return job ID immediately
return {"job_id": job.job_id, "workflow_id": workflow.id, "status": "queued"}
# Background Worker: Process via existing unified workflow
async def deploy_tenant_task(ctx, tenant_id: str, workflow_id: str):
# Existing workflow logic - zero changes needed!
await workflow_manager.deploy_tenant(UUID(tenant_id), workflow_id=UUID(workflow_id))
```
#### Implementation Tasks
**Phase 4.1: Core Job Queue Setup** ✅ COMPLETED
- [x] **Add arq dependency** - integrated Redis job queue with existing infrastructure
- [x] **Create job definitions** - wrapped existing deployment/undeploy workflows as arq tasks
- [x] **Update API endpoints** - updated provisioning endpoints to return job IDs instead of waiting for completion
- [x] **JobQueueService implementation** - service layer for job enqueueing and status tracking
- [x] **Job status tracking** - integrated with existing workflow table for status updates
- [x] **Comprehensive testing** - 18 tests covering positive, negative, and edge cases
**Phase 4.2: Background Worker Implementation** ✅ COMPLETED
- [x] **Job status API** - GET /jobs/{job_id}/status endpoint integrated with JobQueueService
- [x] **Background worker process** - arq worker to process queued jobs with proper settings and Redis configuration
- [x] **Worker settings and configuration** - WorkerSettings class with proper timeouts, max jobs, and error handling
- [x] **Fix API endpoints** - updated job status API to use JobQueueService instead of direct Redis access
- [x] **Integration testing** - comprehensive end-to-end testing with real ARQ workers and Fly.io infrastructure
- [x] **Worker entry points** - dual-purpose entrypoint.sh script and __main__.py module support for both API and worker processes
- [x] **Test fixture updates** - fixed all API and service test fixtures to work with job queue dependencies
- [x] **AsyncIO event loop fixes** - resolved event loop issues in integration tests for subprocess worker compatibility
- [x] **Complete test coverage** - all 46 tests passing across unit, integration, and API test suites
- [x] **Type safety verification** - 0 type checking errors across entire ARQ job queue implementation
#### Phase 4.2 Implementation Summary ✅ COMPLETE
**Core ARQ Job Queue System:**
- **JobQueueService** - Centralized service for job enqueueing, status tracking, and Redis pool management
- **deployment_jobs.py** - ARQ job functions that wrap existing deployment/undeploy workflows
- **Worker Settings** - Production-ready ARQ configuration with proper timeouts and error handling
- **Dual-Process Architecture** - Single Docker image with entrypoint.sh supporting both API and worker modes
**Key Files Added:**
- `apps/cloud/src/basic_memory_cloud/jobs/` - Complete job queue implementation (7 files)
- `apps/cloud/entrypoint.sh` - Dual-purpose Docker container entry point
- `apps/cloud/tests/integration/test_worker_integration.py` - Real infrastructure integration tests
- `apps/cloud/src/basic_memory_cloud/schemas/job_responses.py` - API response schemas
**API Integration:**
- Provisioning endpoints return job IDs immediately instead of blocking for 60+ seconds
- Job status API endpoints for real-time monitoring of deployment progress
- Proper error handling and job failure scenarios with detailed error messages
**Testing Achievement:**
- **46 total tests passing** across all test suites (unit, integration, API, services)
- **Real infrastructure testing** - ARQ workers process actual Fly.io deployments
- **Event loop safety** - Fixed asyncio issues for subprocess worker compatibility
- **Test fixture updates** - All fixtures properly support job queue dependencies
- **Type checking** - 0 errors across entire codebase
**Technical Metrics:**
- **38 files changed** - +1,736 insertions, -334 deletions
- **Integration test runtime** - ~18 seconds with real ARQ workers and Fly.io verification
- **Event loop isolation** - Proper async session management for subprocess compatibility
- **Redis integration** - Production-ready Redis configuration with connection pooling
**Phase 4.3: Production Hardening** ✅ COMPLETED
- [x] **Configure Upstash Redis** - production Redis setup on Fly.io
- [x] **Retry logic for external APIs** - exponential backoff for flaky Tigris IAM operations
- [x] **Monitoring and observability** - comprehensive Redis queue monitoring with CLI tools
- [x] **Error handling improvements** - graceful handling of expected API errors with appropriate log levels
- [x] **CLI tooling enhancements** - bulk update commands for CI/CD automation
- [x] **Documentation improvements** - comprehensive monitoring guide with Redis patterns
- [x] **Job uniqueness** - ARQ-based duplicate prevention for tenant operations
- [ ] **Worker scaling** - multiple arq workers for parallel job processing
- [ ] **Job persistence** - ensure jobs survive Redis/worker restarts
- [ ] **Error alerting** - notifications for failed deployment jobs
**Phase 4.4: Advanced Features** (Future)
- [ ] **Job scheduling** - deploy tenants at specific times
- [ ] **Priority queues** - urgent deployments processed first
- [ ] **Batch operations** - bulk tenant deployments
- [ ] **Job dependencies** - deployment → configuration → activation chains
#### Benefits Achieved ✅ REALIZED
**User Experience Improvements:**
- **Immediate API responses** - users get job ID instantly vs waiting 60+ seconds for deployment completion
- **Real-time job tracking** - status API provides live updates on deployment progress
- **Better error visibility** - detailed error messages and job failure tracking
- **CI/CD automation ready** - bulk update commands for automated tenant deployments
**System Reliability:**
- **Redis persistence** - jobs survive Redis/worker restarts with proper queue durability
- **Idempotent job processing** - jobs can be safely retried without side effects
- **Event loop isolation** - worker processes operate independently from API server
- **Retry resilience** - exponential backoff for flaky external API calls (3 attempts, 1s/2s delays)
- **Graceful error handling** - expected API errors logged at INFO level, unexpected at ERROR level
- **Job uniqueness** - prevent duplicate tenant operations with ARQ's built-in uniqueness feature
**Operational Benefits:**
- **Horizontal scaling ready** - architecture supports adding more workers for parallel processing
- **Comprehensive testing** - real infrastructure integration tests ensure production reliability
- **Type safety** - full type checking prevents runtime errors in job processing
- **Clean separation** - API and worker processes use same codebase with different entry points
- **Queue monitoring** - Redis CLI integration for real-time queue activity monitoring
- **Comprehensive documentation** - detailed monitoring guide with Redis pattern explanations
**Development Benefits:**
- **Zero workflow changes** - existing deployment/undeploy workflows work unchanged as background jobs
- **Async/await native** - modern Python asyncio patterns throughout the implementation
- **Event correlation preserved** - all existing workflow tracking and event sourcing continues to work
- **Enhanced CLI tooling** - unified tenant commands with proper endpoint routing
- **Database integrity** - proper foreign key constraint handling in tenant deletion
#### Infrastructure Requirements
- **Local**: Redis via docker-compose (already exists) ✅
- **Production**: Upstash Redis on Fly.io (already configured) ✅
- **Workers**: arq worker processes (new deployment target)
- **Monitoring**: Job status dashboard (simple web interface)
#### API Evolution
```python
# Before: Synchronous (blocks for 60+ seconds)
POST /tenant/{id}/deploy {status: "active", machine_id: "..."}
# After: Asynchronous (returns immediately)
POST /tenant/{id}/deploy {job_id: "uuid", workflow_id: "uuid", status: "queued"}
GET /jobs/{job_id}/status {status: "running", progress: "deploying_machine", workflow_id: "uuid"}
GET /workflows/{workflow_id}/events [...] # Existing event tracking works unchanged
```
**Technology Choice**: **arq (Redis)** over pgqueuer
- **Existing Redis infrastructure** - Upstash + docker-compose already configured
- **Better ecosystem** - monitoring tools, documentation, community
- **Made by pydantic team** - aligns with existing Python stack
- **Hybrid approach** - Redis for queue operations + Postgres for workflow state
#### Job Uniqueness Implementation
**Problem**: Multiple concurrent deployment requests for the same tenant could create duplicate jobs, wasting resources and potentially causing conflicts.
**Solution**: Leverage ARQ's built-in job uniqueness feature using predictable job IDs:
```python
# JobQueueService implementation
async def enqueue_deploy_job(self, tenant_id: UUID, image_tag: str | None = None) -> str:
unique_job_id = f"deploy-{tenant_id}"
job = await self.redis_pool.enqueue_job(
"deploy_tenant_job",
str(tenant_id),
image_tag,
_job_id=unique_job_id, # ARQ prevents duplicates
)
if job is None:
# Job already exists - return existing job ID
return unique_job_id
else:
# New job created - return ARQ job ID
return job.job_id
```
**Key Features:**
- **Predictable Job IDs**: `deploy-{tenant_id}`, `undeploy-{tenant_id}`
- **Duplicate Prevention**: ARQ returns `None` for duplicate job IDs
- **Graceful Handling**: Return existing job ID instead of raising errors
- **Idempotent Operations**: Safe to retry deployment requests
- **Clear Logging**: Distinguish "Enqueued new" vs "Found existing" jobs
**Benefits:**
- Prevents resource waste from duplicate deployments
- Eliminates race conditions from concurrent requests
- Makes job monitoring more predictable with consistent IDs
- Provides natural deduplication without complex locking mechanisms
## Notes
### Design Philosophy Lessons
- **Simplicity beats framework magic** - removing DBOS made the system more reliable and debuggable
- **Event sourcing > complex orchestration** - database-backed event tracking provides better observability than framework abstractions
- **Idempotent operations > resumable workflows** - each step handling its own retry logic is simpler than framework-managed resumability
- **Explicit error handling > framework exception handling** - Python exceptions are clearer than orchestration framework error states
### Future Considerations
- **Monitoring integration** - workflow tracking events could feed into observability systems
- **Performance optimization** - event querying patterns may benefit from additional indexing
- **Audit compliance** - complete event trail supports regulatory requirements
- **Operational dashboards** - workflow status could drive tenant health monitoring
### Related Specifications
- **SPEC-8**: TigrisFS Integration - bucket provisioning integrated with deployment workflow
- **SPEC-1**: Specification-Driven Development Process - this spec follows the established format
## Observations
- [architecture] Removing framework complexity led to more maintainable system #simplification
- [workflow] Single unified deployment workflow handles both provisioning and updates #consolidation
- [observability] Event sourcing with workflow correlation provides complete audit trail #event-tracking
- [database] Foreign key relationships between workflows and events enable powerful queries #schema-design
- [testing] Integration tests with real infrastructure catch edge cases that unit tests miss #testing-strategy
- [parameters] Consistent method signatures (workflow_id first) reduce cognitive overhead #api-design
- [maintenance] Fewer workflows and dependencies reduce long-term maintenance burden #operational-excellence
- [debugging] Plain Python exceptions are clearer than framework abstraction layers #developer-experience
- [resilience] Exponential backoff retry patterns handle flaky external API calls gracefully #error-handling
- [monitoring] Redis queue monitoring provides real-time operational visibility #observability
- [ci-cd] Bulk update commands enable automated tenant deployments in continuous delivery pipelines #automation
- [documentation] Comprehensive monitoring guides reduce operational learning curve #knowledge-management
- [error-logging] Context-aware log levels (INFO for expected errors, ERROR for unexpected) improve signal-to-noise ratio #logging-strategy
- [job-uniqueness] ARQ job uniqueness with predictable tenant-based IDs prevents duplicate operations and resource waste #deduplication
## Implementation Notes
### Configuration Integration
- **Redis Configuration**: Add Redis settings to existing `apps/cloud/src/basic_memory_cloud/config.py`
- **Local Development**: Leverage existing Redis setup from `docker-compose.yml`
- **Production**: Use Upstash Redis configuration for production environments
### Docker Entrypoint Strategy
Create `entrypoint.sh` script to toggle between API server and worker processes using single Docker image:
```bash
#!/bin/bash
# Entrypoint script for Basic Memory Cloud service
# Supports multiple process types: api, worker
set -e
case "$1" in
"api")
echo "Starting Basic Memory Cloud API server..."
exec uvicorn basic_memory_cloud.main:app \
--host 0.0.0.0 \
--port 8000 \
--log-level info
;;
"worker")
echo "Starting Basic Memory Cloud ARQ worker..."
# For ARQ worker implementation
exec python -m arq basic_memory_cloud.jobs.settings.WorkerSettings
;;
*)
echo "Usage: $0 {api|worker}"
echo " api - Start the FastAPI server"
echo " worker - Start the ARQ worker"
exit 1
;;
esac
```
### Fly.io Process Groups Configuration
Use separate machine groups for API and worker processes with independent scaling:
```toml
# fly.toml app configuration for basic-memory-cloud
app = 'basic-memory-cloud-dev-basic-machines'
primary_region = 'dfw'
org = 'basic-machines'
kill_signal = 'SIGINT'
kill_timeout = '5s'
[build]
# Process groups for API server and worker
[processes]
api = "api"
worker = "worker"
# Machine scaling configuration
[[machine]]
size = 'shared-cpu-1x'
processes = ['api']
min_machines_running = 1
auto_stop_machines = false
auto_start_machines = true
[[machine]]
size = 'shared-cpu-1x'
processes = ['worker']
min_machines_running = 1
auto_stop_machines = false
auto_start_machines = true
[env]
# Python configuration
PYTHONUNBUFFERED = '1'
PYTHONPATH = '/app'
# Logging configuration
LOG_LEVEL = 'DEBUG'
# Redis configuration for ARQ
REDIS_URL = 'redis://basic-memory-cloud-redis.upstash.io'
# Database configuration
DATABASE_HOST = 'basic-memory-cloud-db-dev-basic-machines.internal'
DATABASE_PORT = '5432'
DATABASE_NAME = 'basic_memory_cloud'
DATABASE_USER = 'postgres'
DATABASE_SSL = 'true'
# Worker configuration
ARQ_MAX_JOBS = '10'
ARQ_KEEP_RESULT = '3600'
# Fly.io configuration
FLY_ORG = 'basic-machines'
FLY_REGION = 'dfw'
# Internal service - no external HTTP exposure for worker
# API accessible via basic-memory-cloud-dev-basic-machines.flycast:8000
[[vm]]
size = 'shared-cpu-1x'
```
### Benefits of This Architecture
- **Single Docker Image**: Both API and worker use same container with different entrypoints
- **Independent Scaling**: Scale API and worker processes separately based on demand
- **Clean Separation**: Web traffic handling separate from background job processing
- **Existing Infrastructure**: Leverages current PostgreSQL + Redis setup without complexity
- **Hybrid State Management**: Redis for queue operations, PostgreSQL for persistent workflow tracking
## Relations
- implements [[SPEC-8 TigrisFS Integration]]
- follows [[SPEC-1 Specification-Driven Development Process]]
- supersedes previous multi-workflow architecture
@@ -0,0 +1,186 @@
---
title: 'SPEC-11: Basic Memory API Performance Optimization'
type: spec
permalink: specs/spec-11-basic-memory-api-performance-optimization
tags:
- performance
- api
- mcp
- database
- cloud
---
# SPEC-11: Basic Memory API Performance Optimization
## Why
The Basic Memory API experiences significant performance issues in cloud environments due to expensive per-request initialization. MCP tools making
HTTP requests to the API suffer from 350ms-2.6s latency overhead **before** any actual operation occurs.
**Root Cause Analysis:**
- GitHub Issue #82 shows repeated initialization sequences in logs (16:29:35 and 16:49:58)
- Each MCP tool call triggers full database initialization + project reconciliation
- `get_engine_factory()` dependency calls `db.get_or_create_db()` on every request
- `reconcile_projects_with_config()` runs expensive sync operations repeatedly
**Performance Impact:**
- Database connection setup: ~50-100ms per request
- Migration checks: ~100-500ms per request
- Project reconciliation: ~200ms-2s per request
- **Total overhead**: ~350ms-2.6s per MCP tool call
This creates compounding effects with tenant auto-start delays and increases timeout risk in cloud deployments.
## What
This optimization affects the **core basic-memory repository** components:
1. **API Lifespan Management** (`src/basic_memory/api/app.py`)
- Cache database connections in app state during startup
- Avoid repeated expensive initialization
2. **Dependency Injection** (`src/basic_memory/deps.py`)
- Modify `get_engine_factory()` to use cached connections
- Eliminate per-request database setup
3. **Initialization Service** (`src/basic_memory/services/initialization.py`)
- Add caching/throttling to project reconciliation
- Skip expensive operations when appropriate
4. **Configuration** (`src/basic_memory/config.py`)
- Add optional performance flags for cloud environments
**Backwards Compatibility**: All changes must be backwards compatible with existing CLI and non-cloud usage.
## How (High Level)
### Phase 1: Cache Database Connections (Critical - 80% of gains)
**Problem**: `get_engine_factory()` calls `db.get_or_create_db()` per request
**Solution**: Cache database engine/session in app state during lifespan
1. **Modify API Lifespan** (`api/app.py`):
```python
@asynccontextmanager
async def lifespan(app: FastAPI):
app_config = ConfigManager().config
await initialize_app(app_config)
# Cache database connection in app state
engine, session_maker = await db.get_or_create_db(app_config.database_path)
app.state.engine = engine
app.state.session_maker = session_maker
# ... rest of startup logic
```
2. Modify Dependency Injection (deps.py):
```python
async def get_engine_factory(
request: Request
) -> tuple[AsyncEngine, async_sessionmaker[AsyncSession]]:
"""Get cached engine and session maker from app state."""
return request.app.state.engine, request.app.state.session_maker
```
Phase 2: Optimize Project Reconciliation (Secondary - 20% of gains)
Problem: reconcile_projects_with_config() runs expensive sync repeatedly
Solution: Add module-level caching with time-based throttling
1. Add Reconciliation Cache (services/initialization.py):
```ptyhon
_project_reconciliation_completed = False
_last_reconciliation_time = 0
async def reconcile_projects_with_config(app_config, force=False):
# Skip if recently completed (within 60 seconds) unless forced
if recently_completed and not force:
return
# ... existing logic
```
Phase 3: Cloud Environment Flags (Optional)
Problem: Force expensive initialization in production environments
Solution: Add skip flags for cloud/stateless deployments
1. Add Config Flag (config.py):
skip_initialization_sync: bool = Field(default=False)
2. Configure in Cloud (basic-memory-cloud integration):
BASIC_MEMORY_SKIP_INITIALIZATION_SYNC=true
How to Evaluate
Success Criteria
1. Performance Metrics (Primary):
- MCP tool response time reduced by 50%+ (measure before/after)
- Database connection overhead eliminated (0ms vs 50-100ms)
- Migration check overhead eliminated (0ms vs 100-500ms)
- Project reconciliation overhead reduced by 90%+
2. Load Testing:
- Concurrent MCP tool calls maintain performance
- No memory leaks in cached connections
- Database connection pool behaves correctly
3. Functional Correctness:
- All existing API endpoints work identically
- MCP tools maintain full functionality
- CLI operations unaffected
- Database migrations still execute properly
4. Backwards Compatibility:
- No breaking changes to existing APIs
- Config changes are optional with safe defaults
- Non-cloud deployments work unchanged
Testing Strategy
Performance Testing:
# Before optimization
time basic-memory-mcp-tools write_note "test" "content" "folder"
# Measure: ~1-3 seconds
# After optimization
time basic-memory-mcp-tools write_note "test" "content" "folder"
# Target: <500ms
Load Testing:
# Multiple concurrent MCP tool calls
for i in {1..10}; do
basic-memory-mcp-tools search "test" &
done
wait
# Verify: No degradation, consistent response times
Regression Testing:
# Full basic-memory test suite
just test
# All tests must pass
# Integration tests with cloud deployment
# Verify MCP gateway → API → database flow works
Validation Checklist
- Phase 1 Complete: Database connections cached, dependency injection optimized
- Performance Benchmark: 50%+ improvement in MCP tool response times
- Memory Usage: No leaks in cached connections over 24h+ periods
- Stress Testing: 100+ concurrent requests maintain performance
- Backwards Compatibility: All existing functionality preserved
- Documentation: Performance optimization documented in README
- Cloud Integration: basic-memory-cloud sees performance benefits
Notes
Implementation Priority:
- Phase 1 provides 80% of performance gains and should be implemented first
- Phase 2 provides remaining 20% and addresses edge cases
- Phase 3 is optional for maximum cloud optimization
Risk Mitigation:
- All changes backwards compatible
- Gradual rollout possible (Phase 1 → 2 → 3)
- Easy rollback via configuration flags
Cloud Integration:
- This optimization directly addresses basic-memory-cloud issue #82
- Changes in core basic-memory will benefit all cloud tenants
- No changes needed in basic-memory-cloud itself
@@ -0,0 +1,182 @@
# SPEC-12: OpenTelemetry Observability
## Why
We need comprehensive observability for basic-memory-cloud to:
- Track request flows across our multi-tenant architecture (MCP → Cloud → API services)
- Debug performance issues and errors in production
- Understand user behavior and system usage patterns
- Correlate issues to specific tenants for targeted debugging
- Monitor service health and latency across the distributed system
Currently, we only have basic logging without request correlation or distributed tracing capabilities.
## What
Implement OpenTelemetry instrumentation across all basic-memory-cloud services with:
### Core Requirements
1. **Distributed Tracing**: End-to-end request tracing from MCP gateway through to tenant API instances
2. **Tenant Correlation**: All traces tagged with tenant_id, user_id, and workos_user_id
3. **Service Identification**: Clear service naming and namespace separation
4. **Auto-instrumentation**: Automatic tracing for FastAPI, SQLAlchemy, HTTP clients
5. **Grafana Cloud Integration**: Direct OTLP export to Grafana Cloud Tempo
### Services to Instrument
- **MCP Gateway** (basic-memory-mcp): Entry point with JWT extraction
- **Cloud Service** (basic-memory-cloud): Provisioning and management operations
- **API Service** (basic-memory-api): Tenant-specific instances
- **Worker Processes** (ARQ workers): Background job processing
### Key Trace Attributes
- `tenant.id`: UUID from UserProfile.tenant_id
- `user.id`: WorkOS user identifier
- `user.email`: User email for debugging
- `service.name`: Specific service identifier
- `service.namespace`: Environment (development/production)
- `operation.type`: Business operation (provision/update/delete)
- `tenant.app_name`: Fly.io app name for tenant instances
## How
### Phase 1: Setup OpenTelemetry SDK
1. Add OpenTelemetry dependencies to each service's pyproject.toml:
```python
"opentelemetry-distro[otlp]>=1.29.0",
"opentelemetry-instrumentation-fastapi>=0.50b0",
"opentelemetry-instrumentation-httpx>=0.50b0",
"opentelemetry-instrumentation-sqlalchemy>=0.50b0",
"opentelemetry-instrumentation-logging>=0.50b0",
```
2. Create shared telemetry initialization module (`apps/shared/telemetry.py`)
3. Configure Grafana Cloud OTLP endpoint via environment variables:
```bash
OTEL_EXPORTER_OTLP_ENDPOINT=https://otlp-gateway-prod-us-east-2.grafana.net/otlp
OTEL_EXPORTER_OTLP_HEADERS=Authorization=Basic[token]
OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf
```
### Phase 2: Instrument MCP Gateway
1. Extract tenant context from AuthKit JWT in middleware
2. Create root span with tenant attributes
3. Propagate trace context to downstream services via headers
### Phase 3: Instrument Cloud Service
1. Continue trace from MCP gateway
2. Add operation-specific attributes (provisioning events)
3. Instrument ARQ worker jobs for async operations
4. Track Fly.io API calls and latency
### Phase 4: Instrument API Service
1. Extract tenant context from JWT
2. Add machine-specific metadata (instance ID, region)
3. Instrument database operations with SQLAlchemy
4. Track MCP protocol operations
### Phase 5: Configure and Deploy
1. Add OTLP configuration to `.env.example` and `.env.example.secrets`
2. Set Fly.io secrets for production deployment
3. Update Dockerfiles to use `opentelemetry-instrument` wrapper
4. Deploy to development environment first for testing
## How to Evaluate
### Success Criteria
1. **End-to-end traces visible in Grafana Cloud** showing complete request flow
2. **Tenant filtering works** - Can filter traces by tenant_id to see all requests for a user
3. **Service maps accurate** - Grafana shows correct service dependencies
4. **Performance overhead < 5%** - Minimal latency impact from instrumentation
5. **Error correlation** - Can trace errors back to specific tenant and operation
### Testing Checklist
- [x] Single request creates connected trace across all services
- [x] Tenant attributes present on all spans
- [x] Background jobs (ARQ) appear in traces
- [x] Database queries show in trace timeline
- [x] HTTP calls to Fly.io API tracked
- [x] Traces exported successfully to Grafana Cloud
- [x] Can search traces by tenant_id in Grafana
- [x] Service dependency graph shows correct flow
### Monitoring Success
- All services reporting traces to Grafana Cloud
- No OTLP export errors in logs
- Trace sampling working correctly (if implemented)
- Resource usage acceptable (CPU/memory)
## Dependencies
- Grafana Cloud account with OTLP endpoint configured
- OpenTelemetry Python SDK v1.29.0+
- FastAPI instrumentation compatibility
- Network access from Fly.io to Grafana Cloud
## Implementation Assignment
**Recommended Agent**: python-developer
- Requires Python/FastAPI expertise
- Needs understanding of distributed systems
- Must implement middleware and context propagation
- Should understand OpenTelemetry SDK and instrumentation
## Follow-up Tasks
### Enhanced Log Correlation
While basic trace-to-log correlation works automatically via OpenTelemetry logging instrumentation, consider adding structured logging for improved log filtering:
1. **Structured Logging Context**: Add `logger.bind()` calls to inject tenant/user context directly into log records
2. **Custom Loguru Formatter**: Extract OpenTelemetry span attributes for better log readability
3. **Direct Log Filtering**: Enable searching logs directly by tenant_id, workflow_id without going through traces
This would complement the existing automatic trace correlation and provide better log search capabilities.
## Alternative Solution: Logfire
After implementing OpenTelemetry with Grafana Cloud, we discovered limitations in the observability experience:
- Traces work but lack useful context without correlated logs
- Setting up log correlation with Grafana is complex and requires additional infrastructure
- The developer experience for Python observability is suboptimal
### Logfire Evaluation
**Pydantic Logfire** offers a compelling alternative that addresses your specific requirements:
#### Core Requirements Match
- ✅ **User Activity Tracking**: Automatic request tracing with business context
- ✅ **Error Monitoring**: Built-in exception tracking with full context
- ✅ **Performance Metrics**: Automatic latency and performance monitoring
- ✅ **Request Tracing**: Native distributed tracing across services
- ✅ **Log Correlation**: Seamless trace-to-log correlation without setup
#### Key Advantages
1. **Python-First Design**: Built specifically for Python/FastAPI applications by the Pydantic team
2. **Simple Integration**: `pip install logfire` + `logfire.configure()` vs complex OTLP setup
3. **Automatic Correlation**: Logs automatically include trace context without manual configuration
4. **Real-time SQL Interface**: Query spans and logs using SQL with auto-completion
5. **Better Developer UX**: Purpose-built observability UI vs generic Grafana dashboards
6. **Loguru Integration**: `logger.configure(handlers=[logfire.loguru_handler()])` maintains existing logging
#### Pricing Assessment
- **Free Tier**: 10M spans/month (suitable for development and small production workloads)
- **Transparent Pricing**: $1 per million spans/metrics after free tier
- **No Hidden Costs**: No per-host fees, only usage-based metering
- **Production Ready**: Recently exited beta, enterprise features available
#### Migration Path
The existing OpenTelemetry instrumentation is compatible - Logfire uses OpenTelemetry under the hood, so the current spans and attributes would work unchanged.
### Recommendation
**Consider migrating to Logfire** for the following reasons:
1. It directly addresses the "next to useless" traces problem by providing integrated logs
2. Dramatically simpler setup and maintenance compared to Grafana Cloud + custom log correlation
3. Better ROI on observability investment with purpose-built Python tooling
4. Free tier sufficient for current development needs with clear scaling path
The current Grafana Cloud implementation provides a solid foundation and could remain as a backup/export target, while Logfire becomes the primary observability platform.
## Status
**Created**: 2024-01-28
**Status**: Completed (OpenTelemetry + Grafana Cloud)
**Next Phase**: Evaluate Logfire migration
**Priority**: High - Critical for production observability
@@ -0,0 +1,917 @@
---
title: 'SPEC-13: CLI Authentication with Subscription Validation'
type: spec
permalink: specs/spec-12-cli-auth-subscription-validation
tags:
- authentication
- security
- cli
- subscription
status: draft
created: 2025-10-02
---
# SPEC-13: CLI Authentication with Subscription Validation
## Why
The Basic Memory Cloud CLI currently has a security gap in authentication that allows unauthorized access:
**Current Web Flow (Secure)**:
1. User signs up via WorkOS AuthKit
2. User creates Polar subscription
3. Web app validates subscription before calling `POST /tenants/setup`
4. Tenant provisioned only after subscription validation ✅
**Current CLI Flow (Insecure)**:
1. User signs up via WorkOS AuthKit (OAuth device flow)
2. User runs `bm cloud login`
3. CLI receives JWT token from WorkOS
4. CLI can access all cloud endpoints without subscription check ❌
**Problem**: Anyone can sign up with WorkOS and immediately access cloud infrastructure via CLI without having an active Polar subscription. This creates:
- Revenue loss (free resource consumption)
- Security risk (unauthorized data access)
- Support burden (users accessing features they haven't paid for)
**Root Cause**: The CLI authentication flow validates JWT tokens but doesn't verify subscription status before granting access to cloud resources.
## What
Add subscription validation to authentication flow to ensure only users with active Polar subscriptions can access cloud resources across all access methods (CLI, MCP, Web App, Direct API).
**Affected Components**:
### basic-memory-cloud (Cloud Service)
- `apps/cloud/src/basic_memory_cloud/deps.py` - Add subscription validation dependency
- `apps/cloud/src/basic_memory_cloud/services/subscription_service.py` - Add subscription check method
- `apps/cloud/src/basic_memory_cloud/api/tenant_mount.py` - Protect mount endpoints
- `apps/cloud/src/basic_memory_cloud/api/proxy.py` - Protect proxy endpoints
### basic-memory (CLI)
- `src/basic_memory/cli/commands/cloud/core_commands.py` - Handle 403 errors
- `src/basic_memory/cli/commands/cloud/api_client.py` - Parse subscription errors
- `docs/cloud-cli.md` - Document subscription requirement
**Endpoints to Protect**:
- `GET /tenant/mount/info` - Used by CLI bisync setup
- `POST /tenant/mount/credentials` - Used by CLI bisync credentials
- `GET /proxy/{path:path}` - Used by Web App, MCP tools, CLI tools, Direct API
- All other `/proxy/*` endpoints - Centralized access point for all user operations
## Complete Authentication Flow Analysis
### Overview of All Access Flows
Basic Memory Cloud has **7 distinct authentication flows**. This spec closes subscription validation gaps in flows 2-4 and 6, which all converge on the `/proxy/*` endpoints.
### Flow 1: Polar Webhook → Registration ✅ SECURE
```
Polar webhook → POST /api/webhooks/polar
→ Validates Polar webhook signature
→ Creates/updates subscription in database
→ No direct user access - webhook only
```
**Auth**: Polar webhook signature validation
**Subscription Check**: N/A (webhook creates subscriptions)
**Status**: ✅ Secure - webhook validated, no user JWT involved
### Flow 2: Web App Login ❌ NEEDS FIX
```
User → apps/web (Vue.js/Nuxt)
→ WorkOS AuthKit magic link authentication
→ JWT stored in browser session
→ Web app calls /proxy/{project}/... endpoints (memory, directory, projects)
→ proxy.py validates JWT but does NOT check subscription
→ Access granted without subscription ❌
```
**Auth**: WorkOS JWT via `CurrentUserProfileHybridJwtDep`
**Subscription Check**: ❌ Missing
**Fixed By**: Task 1.4 (protect `/proxy/*` endpoints)
### Flow 3: MCP (Model Context Protocol) ❌ NEEDS FIX
```
AI Agent (Claude, Cursor, etc.) → https://mcp.basicmemory.com
→ AuthKit OAuth device flow
→ JWT stored in AI agent
→ MCP tools call {cloud_host}/proxy/{endpoint} with Authorization header
→ proxy.py validates JWT but does NOT check subscription
→ MCP tools can access all cloud resources without subscription ❌
```
**Auth**: AuthKit JWT via `CurrentUserProfileHybridJwtDep`
**Subscription Check**: ❌ Missing
**Fixed By**: Task 1.4 (protect `/proxy/*` endpoints)
### Flow 4: CLI Auth (basic-memory) ❌ NEEDS FIX
```
User → bm cloud login
→ AuthKit OAuth device flow
→ JWT stored in ~/.basic-memory/tokens.json
→ CLI calls:
- {cloud_host}/tenant/mount/info (for bisync setup)
- {cloud_host}/tenant/mount/credentials (for bisync credentials)
- {cloud_host}/proxy/{endpoint} (for all MCP tools)
→ tenant_mount.py and proxy.py validate JWT but do NOT check subscription
→ Access granted without subscription ❌
```
**Auth**: AuthKit JWT via `CurrentUserProfileHybridJwtDep`
**Subscription Check**: ❌ Missing
**Fixed By**: Task 1.3 (protect `/tenant/mount/*`) + Task 1.4 (protect `/proxy/*`)
### Flow 5: Cloud CLI (Admin Tasks) ✅ SECURE
```
Admin → python -m basic_memory_cloud.cli.tenant_cli
→ Uses CLIAuth with admin WorkOS OAuth client
→ Gets JWT token with admin org membership
→ Calls /tenants/* endpoints (create, list, delete tenants)
→ tenants.py validates JWT AND admin org membership via AdminUserHybridDep
→ Access granted only to admin organization members ✅
```
**Auth**: AuthKit JWT + Admin org validation via `AdminUserHybridDep`
**Subscription Check**: N/A (admins bypass subscription requirement)
**Status**: ✅ Secure - admin-only endpoints, separate from user flows
### Flow 6: Direct API Calls ❌ NEEDS FIX
```
Any HTTP client → {cloud_host}/proxy/{endpoint}
→ Sends Authorization: Bearer {jwt} header
→ proxy.py validates JWT but does NOT check subscription
→ Direct API access without subscription ❌
```
**Auth**: WorkOS or AuthKit JWT via `CurrentUserProfileHybridJwtDep`
**Subscription Check**: ❌ Missing
**Fixed By**: Task 1.4 (protect `/proxy/*` endpoints)
### Flow 7: Tenant API Instance (Internal) ✅ SECURE
```
/proxy/* → Tenant API (basic-memory-{tenant_id}.fly.dev)
→ Validates signed header from proxy (tenant_id + signature)
→ Direct external access will be disabled in production
→ Only accessible via /proxy endpoints
```
**Auth**: Signed header validation from proxy
**Subscription Check**: N/A (internal only, validated at proxy layer)
**Status**: ✅ Secure - validates proxy signature, not directly accessible
### Authentication Flow Summary Matrix
| Flow | Access Method | Current Auth | Subscription Check | Fixed By SPEC-13 |
|------|---------------|--------------|-------------------|------------------|
| 1. Polar Webhook | Polar webhook → `/api/webhooks/polar` | Polar signature | N/A (webhook) | N/A |
| 2. Web App | Browser → `/proxy/*` | WorkOS JWT ✅ | ❌ Missing | ✅ Task 1.4 |
| 3. MCP | AI Agent → `/proxy/*` | AuthKit JWT ✅ | ❌ Missing | ✅ Task 1.4 |
| 4. CLI | `bm cloud``/tenant/mount/*` + `/proxy/*` | AuthKit JWT ✅ | ❌ Missing | ✅ Task 1.3 + 1.4 |
| 5. Cloud CLI (Admin) | `tenant_cli``/tenants/*` | AuthKit JWT ✅ + Admin org | N/A (admin) | N/A (admin bypass) |
| 6. Direct API | HTTP client → `/proxy/*` | WorkOS/AuthKit JWT ✅ | ❌ Missing | ✅ Task 1.4 |
| 7. Tenant API | Proxy → tenant instance | Proxy signature ✅ | N/A (internal) | N/A |
### Key Insights
1. **Single Point of Failure**: All user access (Web, MCP, CLI, Direct API) converges on `/proxy/*` endpoints
2. **Centralized Fix**: Protecting `/proxy/*` with subscription validation closes gaps in flows 2, 3, 4, and 6 simultaneously
3. **Admin Bypass**: Cloud CLI admin tasks use separate `/tenants/*` endpoints with admin-only access (no subscription needed)
4. **Defense in Depth**: `/tenant/mount/*` endpoints also protected for CLI bisync operations
### Architecture Benefits
The `/proxy` layer serves as the **single centralized authorization point** for all user access:
- ✅ One place to validate JWT tokens
- ✅ One place to check subscription status
- ✅ One place to handle tenant routing
- ✅ Protects Web App, MCP, CLI, and Direct API simultaneously
This architecture makes the fix comprehensive and maintainable.
## How (High Level)
### Option A: Database Subscription Check (Recommended)
**Approach**: Add FastAPI dependency that validates subscription status from database before allowing access.
**Implementation**:
1. **Create Subscription Validation Dependency** (`deps.py`)
```python
async def get_authorized_cli_user_profile(
credentials: Annotated[HTTPAuthorizationCredentials, Depends(security)],
session: DatabaseSessionDep,
user_profile_repo: UserProfileRepositoryDep,
subscription_service: SubscriptionServiceDep,
) -> UserProfile:
"""
Hybrid authentication with subscription validation for CLI access.
Validates JWT (WorkOS or AuthKit) and checks for active subscription.
Returns UserProfile if both checks pass.
"""
# Try WorkOS JWT first (faster validation path)
try:
user_context = await validate_workos_jwt(credentials.credentials)
except HTTPException:
# Fall back to AuthKit JWT validation
try:
user_context = await validate_authkit_jwt(credentials.credentials)
except HTTPException as e:
raise HTTPException(
status_code=401,
detail="Invalid JWT token. Authentication required.",
) from e
# Check subscription status
has_subscription = await subscription_service.check_user_has_active_subscription(
session, user_context.workos_user_id
)
if not has_subscription:
raise HTTPException(
status_code=403,
detail={
"error": "subscription_required",
"message": "Active subscription required for CLI access",
"subscribe_url": "https://basicmemory.com/subscribe"
}
)
# Look up and return user profile
user_profile = await user_profile_repo.get_user_profile_by_workos_user_id(
session, user_context.workos_user_id
)
if not user_profile:
raise HTTPException(401, detail="User profile not found")
return user_profile
```
```python
AuthorizedCLIUserProfileDep = Annotated[UserProfile, Depends(get_authorized_cli_user_profile)]
```
2. **Add Subscription Check Method** (`subscription_service.py`)
```python
async def check_user_has_active_subscription(
self, session: AsyncSession, workos_user_id: str
) -> bool:
"""Check if user has active subscription."""
# Use existing repository method to get subscription by workos_user_id
# This joins UserProfile -> Subscription in a single query
subscription = await self.subscription_repository.get_subscription_by_workos_user_id(
session, workos_user_id
)
return subscription is not None and subscription.status == "active"
```
3. **Protect Endpoints** (Replace `CurrentUserProfileHybridJwtDep` with `AuthorizedCLIUserProfileDep`)
```python
# Before
@router.get("/mount/info")
async def get_mount_info(
user_profile: CurrentUserProfileHybridJwtDep,
session: DatabaseSessionDep,
):
tenant_id = user_profile.tenant_id
...
# After
@router.get("/mount/info")
async def get_mount_info(
user_profile: AuthorizedCLIUserProfileDep, # Now includes subscription check
session: DatabaseSessionDep,
):
tenant_id = user_profile.tenant_id # No changes needed to endpoint logic
...
```
4. **Update CLI Error Handling**
```python
# In core_commands.py login()
try:
success = await auth.login()
if success:
# Test subscription by calling protected endpoint
await make_api_request("GET", f"{host_url}/tenant/mount/info")
except CloudAPIError as e:
if e.status_code == 403 and e.detail.get("error") == "subscription_required":
console.print("[red]Subscription required[/red]")
console.print(f"Subscribe at: {e.detail['subscribe_url']}")
raise typer.Exit(1)
```
**Pros**:
- Simple to implement
- Fast (single database query)
- Clear error messages
- Works with existing subscription flow
**Cons**:
- Database is source of truth (could get out of sync with Polar)
- Adds one extra subscription lookup query per request (lightweight JOIN query)
### Option B: WorkOS Organizations
**Approach**: Add users to "beta-users" organization in WorkOS after subscription creation, validate org membership via JWT claims.
**Implementation**:
1. After Polar subscription webhook, add user to WorkOS org via API
2. Validate `org_id` claim in JWT matches authorized org
3. Use existing `get_admin_workos_jwt` pattern
**Pros**:
- WorkOS as single source of truth
- No database queries needed
- More secure (harder to bypass)
**Cons**:
- More complex (requires WorkOS API integration)
- Requires managing WorkOS org membership
- Less control over error messages
- Additional API calls during registration
### Recommendation
**Start with Option A (Database Check)** for:
- Faster implementation
- Clearer error messages
- Easier testing
- Existing subscription infrastructure
**Consider Option B later** if:
- Need tighter security
- Want to reduce database dependency
- Scale requires fewer database queries
## How to Evaluate
### Success Criteria
**1. Unauthorized Users Blocked**
- [ ] User without subscription cannot complete `bm cloud login`
- [ ] User without subscription receives clear error with subscribe link
- [ ] User without subscription cannot run `bm cloud setup`
- [ ] User without subscription cannot run `bm sync` in cloud mode
**2. Authorized Users Work**
- [ ] User with active subscription can login successfully
- [ ] User with active subscription can setup bisync
- [ ] User with active subscription can sync files
- [ ] User with active subscription can use all MCP tools via proxy
**3. Subscription State Changes**
- [ ] Expired subscription blocks access with clear error
- [ ] Renewed subscription immediately restores access
- [ ] Cancelled subscription blocks access after grace period
**4. Error Messages**
- [ ] 403 errors include "subscription_required" error code
- [ ] Error messages include subscribe URL
- [ ] CLI displays user-friendly messages
- [ ] Errors logged appropriately for debugging
**5. No Regressions**
- [ ] Web app login/subscription flow unaffected
- [ ] Admin endpoints still work (bypass check)
- [ ] Tenant provisioning workflow unchanged
- [ ] Performance not degraded
### Test Cases
**Manual Testing**:
```bash
# Test 1: Unauthorized user
1. Create new WorkOS account (no subscription)
2. Run `bm cloud login`
3. Verify: Login succeeds but shows subscription required error
4. Verify: Cannot run `bm cloud setup`
5. Verify: Clear error message with subscribe link
# Test 2: Authorized user
1. Use account with active Polar subscription
2. Run `bm cloud login`
3. Verify: Login succeeds without errors
4. Run `bm cloud setup`
5. Verify: Setup completes successfully
6. Run `bm sync`
7. Verify: Sync works normally
# Test 3: Subscription expiration
1. Use account with active subscription
2. Manually expire subscription in database
3. Run `bm cloud login`
4. Verify: Blocked with clear error
5. Renew subscription
6. Run `bm cloud login` again
7. Verify: Access restored
```
**Automated Tests**:
```python
# Test subscription validation dependency
async def test_authorized_user_allowed(
db_session,
user_profile_repo,
subscription_service,
mock_jwt_credentials
):
# Create user with active subscription
user_profile = await create_user_with_subscription(db_session, status="active")
# Mock JWT credentials for the user
credentials = mock_jwt_credentials(user_profile.workos_user_id)
# Should not raise exception
result = await get_authorized_cli_user_profile(
credentials, db_session, user_profile_repo, subscription_service
)
assert result.id == user_profile.id
assert result.workos_user_id == user_profile.workos_user_id
async def test_unauthorized_user_blocked(
db_session,
user_profile_repo,
subscription_service,
mock_jwt_credentials
):
# Create user without subscription
user_profile = await create_user_without_subscription(db_session)
credentials = mock_jwt_credentials(user_profile.workos_user_id)
# Should raise 403
with pytest.raises(HTTPException) as exc:
await get_authorized_cli_user_profile(
credentials, db_session, user_profile_repo, subscription_service
)
assert exc.value.status_code == 403
assert exc.value.detail["error"] == "subscription_required"
async def test_inactive_subscription_blocked(
db_session,
user_profile_repo,
subscription_service,
mock_jwt_credentials
):
# Create user with cancelled/inactive subscription
user_profile = await create_user_with_subscription(db_session, status="cancelled")
credentials = mock_jwt_credentials(user_profile.workos_user_id)
# Should raise 403
with pytest.raises(HTTPException) as exc:
await get_authorized_cli_user_profile(
credentials, db_session, user_profile_repo, subscription_service
)
assert exc.value.status_code == 403
assert exc.value.detail["error"] == "subscription_required"
```
## Implementation Tasks
### Phase 1: Cloud Service (basic-memory-cloud)
#### Task 1.1: Add subscription check method to SubscriptionService ✅
**File**: `apps/cloud/src/basic_memory_cloud/services/subscription_service.py`
- [x] Add method `check_subscription(session: AsyncSession, workos_user_id: str) -> bool`
- [x] Use existing `self.subscription_repository.get_subscription_by_workos_user_id(session, workos_user_id)`
- [x] Check both `status == "active"` AND `current_period_end >= now()`
- [x] Log both values when check fails
- [x] Add docstring explaining the method
- [x] Run `just typecheck` to verify types
**Actual implementation**:
```python
async def check_subscription(
self, session: AsyncSession, workos_user_id: str
) -> bool:
"""Check if user has active subscription with valid period."""
subscription = await self.subscription_repository.get_subscription_by_workos_user_id(
session, workos_user_id
)
if subscription is None:
return False
if subscription.status != "active":
logger.warning("Subscription inactive", workos_user_id=workos_user_id,
status=subscription.status, current_period_end=subscription.current_period_end)
return False
now = datetime.now(timezone.utc)
if subscription.current_period_end is None or subscription.current_period_end < now:
logger.warning("Subscription expired", workos_user_id=workos_user_id,
status=subscription.status, current_period_end=subscription.current_period_end)
return False
return True
```
#### Task 1.2: Add subscription validation dependency ✅
**File**: `apps/cloud/src/basic_memory_cloud/deps.py`
- [x] Import necessary types at top of file (if not already present)
- [x] Add `authorized_user_profile()` async function
- [x] Implement hybrid JWT validation (WorkOS first, AuthKit fallback)
- [x] Add subscription check using `subscription_service.check_subscription()`
- [x] Raise `HTTPException(403)` with structured error detail if no active subscription
- [x] Look up and return `UserProfile` after validation
- [x] Add `AuthorizedUserProfileDep` type annotation
- [x] Use `settings.subscription_url` from config (env var)
- [x] Run `just typecheck` to verify types
**Expected code**:
```python
async def get_authorized_cli_user_profile(
credentials: Annotated[HTTPAuthorizationCredentials, Depends(security)],
session: DatabaseSessionDep,
user_profile_repo: UserProfileRepositoryDep,
subscription_service: SubscriptionServiceDep,
) -> UserProfile:
"""
Hybrid authentication with subscription validation for CLI access.
Validates JWT (WorkOS or AuthKit) and checks for active subscription.
Returns UserProfile if both checks pass.
Raises:
HTTPException(401): Invalid JWT token
HTTPException(403): No active subscription
"""
# Try WorkOS JWT first (faster validation path)
try:
user_context = await validate_workos_jwt(credentials.credentials)
except HTTPException:
# Fall back to AuthKit JWT validation
try:
user_context = await validate_authkit_jwt(credentials.credentials)
except HTTPException as e:
raise HTTPException(
status_code=401,
detail="Invalid JWT token. Authentication required.",
) from e
# Check subscription status
has_subscription = await subscription_service.check_user_has_active_subscription(
session, user_context.workos_user_id
)
if not has_subscription:
logger.warning(
"CLI access denied: no active subscription",
workos_user_id=user_context.workos_user_id,
)
raise HTTPException(
status_code=403,
detail={
"error": "subscription_required",
"message": "Active subscription required for CLI access",
"subscribe_url": "https://basicmemory.com/subscribe"
}
)
# Look up and return user profile
user_profile = await user_profile_repo.get_user_profile_by_workos_user_id(
session, user_context.workos_user_id
)
if not user_profile:
logger.error(
"User profile not found after successful auth",
workos_user_id=user_context.workos_user_id,
)
raise HTTPException(401, detail="User profile not found")
logger.info(
"CLI access granted",
workos_user_id=user_context.workos_user_id,
user_profile_id=str(user_profile.id),
)
return user_profile
AuthorizedCLIUserProfileDep = Annotated[UserProfile, Depends(get_authorized_cli_user_profile)]
```
#### Task 1.3: Protect tenant mount endpoints ✅
**File**: `apps/cloud/src/basic_memory_cloud/api/tenant_mount.py`
- [x] Update import: add `AuthorizedUserProfileDep` from `..deps`
- [x] Replace `user_profile: CurrentUserProfileHybridJwtDep` with `user_profile: AuthorizedUserProfileDep` in:
- [x] `get_tenant_mount_info()` (line ~23)
- [x] `create_tenant_mount_credentials()` (line ~88)
- [x] `revoke_tenant_mount_credentials()` (line ~244)
- [x] `list_tenant_mount_credentials()` (line ~326)
- [x] Verify no other code changes needed (parameter name and usage stays the same)
- [x] Run `just typecheck` to verify types
#### Task 1.4: Protect proxy endpoints ✅
**File**: `apps/cloud/src/basic_memory_cloud/api/proxy.py`
- [x] Update import: add `AuthorizedUserProfileDep` from `..deps`
- [x] Replace `user_profile: CurrentUserProfileHybridJwtDep` with `user_profile: AuthorizedUserProfileDep` in:
- [x] `check_tenant_health()` (line ~21)
- [x] `proxy_to_tenant()` (line ~63)
- [x] Verify no other code changes needed (parameter name and usage stays the same)
- [x] Run `just typecheck` to verify types
**Why Keep /proxy Architecture:**
The proxy layer is valuable because it:
1. **Centralizes authorization** - Single place for JWT + subscription validation (closes both CLI and MCP auth gaps)
2. **Handles tenant routing** - Maps tenant_id → fly_app_name without exposing infrastructure details
3. **Abstracts infrastructure** - MCP and CLI don't need to know about Fly.io naming conventions
4. **Enables features** - Can add rate limiting, caching, request logging, etc. at proxy layer
5. **Supports both flows** - CLI tools and MCP tools both use /proxy endpoints
The extra HTTP hop is minimal (< 10ms) and worth it for architectural benefits.
**Performance Note:** Cloud app has Redis available - can cache subscription status to reduce database queries if needed. Initial implementation uses direct database query (simple, acceptable performance ~5-10ms).
#### Task 1.5: Add unit tests for subscription service
**File**: `apps/cloud/tests/services/test_subscription_service.py` (create if doesn't exist)
- [ ] Create test file if it doesn't exist
- [ ] Add test: `test_check_user_has_active_subscription_returns_true_for_active()`
- Create user with active subscription
- Call `check_user_has_active_subscription()`
- Assert returns `True`
- [ ] Add test: `test_check_user_has_active_subscription_returns_false_for_pending()`
- Create user with pending subscription
- Assert returns `False`
- [ ] Add test: `test_check_user_has_active_subscription_returns_false_for_cancelled()`
- Create user with cancelled subscription
- Assert returns `False`
- [ ] Add test: `test_check_user_has_active_subscription_returns_false_for_no_subscription()`
- Create user without subscription
- Assert returns `False`
- [ ] Run `just test` to verify tests pass
#### Task 1.6: Add integration tests for dependency
**File**: `apps/cloud/tests/test_deps.py` (create if doesn't exist)
- [ ] Create test file if it doesn't exist
- [ ] Add fixtures for mocking JWT credentials
- [ ] Add test: `test_authorized_cli_user_profile_with_active_subscription()`
- Mock valid JWT + active subscription
- Call dependency
- Assert returns UserProfile
- [ ] Add test: `test_authorized_cli_user_profile_without_subscription_raises_403()`
- Mock valid JWT + no subscription
- Assert raises HTTPException(403) with correct error detail
- [ ] Add test: `test_authorized_cli_user_profile_with_inactive_subscription_raises_403()`
- Mock valid JWT + cancelled subscription
- Assert raises HTTPException(403)
- [ ] Add test: `test_authorized_cli_user_profile_with_invalid_jwt_raises_401()`
- Mock invalid JWT
- Assert raises HTTPException(401)
- [ ] Run `just test` to verify tests pass
#### Task 1.7: Deploy and verify cloud service
- [ ] Run `just check` to verify all quality checks pass
- [ ] Commit changes with message: "feat: add subscription validation to CLI endpoints"
- [ ] Deploy to preview environment: `flyctl deploy --config apps/cloud/fly.toml`
- [ ] Test manually:
- [ ] Call `/tenant/mount/info` with valid JWT but no subscription → expect 403
- [ ] Call `/tenant/mount/info` with valid JWT and active subscription → expect 200
- [ ] Verify error response structure matches spec
### Phase 2: CLI (basic-memory)
#### Task 2.1: Review and understand CLI authentication flow
**Files**: `src/basic_memory/cli/commands/cloud/`
- [ ] Read `core_commands.py` to understand current login flow
- [ ] Read `api_client.py` to understand current error handling
- [ ] Identify where 403 errors should be caught
- [ ] Identify what error messages should be displayed
- [ ] Document current behavior in spec if needed
#### Task 2.2: Update API client error handling
**File**: `src/basic_memory/cli/commands/cloud/api_client.py`
- [ ] Add custom exception class `SubscriptionRequiredError` (or similar)
- [ ] Update HTTP error handling to parse 403 responses
- [ ] Extract `error`, `message`, and `subscribe_url` from error detail
- [ ] Raise specific exception for subscription_required errors
- [ ] Run `just typecheck` in basic-memory repo to verify types
#### Task 2.3: Update CLI login command error handling
**File**: `src/basic_memory/cli/commands/cloud/core_commands.py`
- [ ] Import the subscription error exception
- [ ] Wrap login flow with try/except for subscription errors
- [ ] Display user-friendly error message with rich console
- [ ] Show subscribe URL prominently
- [ ] Provide actionable next steps
- [ ] Run `just typecheck` to verify types
**Expected error handling**:
```python
try:
# Existing login logic
success = await auth.login()
if success:
# Test access to protected endpoint
await api_client.test_connection()
except SubscriptionRequiredError as e:
console.print("\n[red]✗ Subscription Required[/red]\n")
console.print(f"[yellow]{e.message}[/yellow]\n")
console.print(f"Subscribe at: [blue underline]{e.subscribe_url}[/blue underline]\n")
console.print("[dim]Once you have an active subscription, run [bold]bm cloud login[/bold] again.[/dim]")
raise typer.Exit(1)
```
#### Task 2.4: Update CLI tests
**File**: `tests/cli/test_cloud_commands.py`
- [ ] Add test: `test_login_without_subscription_shows_error()`
- Mock 403 subscription_required response
- Call login command
- Assert error message displayed
- Assert subscribe URL shown
- [ ] Add test: `test_login_with_subscription_succeeds()`
- Mock successful authentication + subscription check
- Call login command
- Assert success message
- [ ] Run `just test` to verify tests pass
#### Task 2.5: Update CLI documentation
**File**: `docs/cloud-cli.md` (in basic-memory-docs repo)
- [ ] Add "Prerequisites" section if not present
- [ ] Document subscription requirement
- [ ] Add "Troubleshooting" section
- [ ] Document "Subscription Required" error
- [ ] Provide subscribe URL
- [ ] Add FAQ entry about subscription errors
- [ ] Build docs locally to verify formatting
### Phase 3: End-to-End Testing
#### Task 3.1: Create test user accounts
**Prerequisites**: Access to WorkOS admin and database
- [ ] Create test user WITHOUT subscription:
- [ ] Sign up via WorkOS AuthKit
- [ ] Get workos_user_id from database
- [ ] Verify no subscription record exists
- [ ] Save credentials for testing
- [ ] Create test user WITH active subscription:
- [ ] Sign up via WorkOS AuthKit
- [ ] Create subscription via Polar or dev endpoint
- [ ] Verify subscription.status = "active" in database
- [ ] Save credentials for testing
#### Task 3.2: Manual testing - User without subscription
**Environment**: Preview/staging deployment
- [ ] Run `bm cloud login` with no-subscription user
- [ ] Verify: Login shows "Subscription Required" error
- [ ] Verify: Subscribe URL is displayed
- [ ] Verify: Cannot run `bm cloud setup`
- [ ] Verify: Cannot call `/tenant/mount/info` directly via curl
- [ ] Document any issues found
#### Task 3.3: Manual testing - User with active subscription
**Environment**: Preview/staging deployment
- [ ] Run `bm cloud login` with active-subscription user
- [ ] Verify: Login succeeds without errors
- [ ] Verify: Can run `bm cloud setup`
- [ ] Verify: Can call `/tenant/mount/info` successfully
- [ ] Verify: Can call `/proxy/*` endpoints successfully
- [ ] Document any issues found
#### Task 3.4: Test subscription state transitions
**Environment**: Preview/staging deployment + database access
- [ ] Start with active subscription user
- [ ] Verify: All operations work
- [ ] Update subscription.status to "cancelled" in database
- [ ] Verify: Login now shows "Subscription Required" error
- [ ] Verify: Existing tokens are rejected with 403
- [ ] Update subscription.status back to "active"
- [ ] Verify: Access restored immediately
- [ ] Document any issues found
#### Task 3.5: Integration test suite
**File**: `apps/cloud/tests/integration/test_cli_subscription_flow.py` (create if doesn't exist)
- [ ] Create integration test file
- [ ] Add test: `test_cli_flow_without_subscription()`
- Simulate full CLI flow without subscription
- Assert 403 at appropriate points
- [ ] Add test: `test_cli_flow_with_active_subscription()`
- Simulate full CLI flow with active subscription
- Assert all operations succeed
- [ ] Add test: `test_subscription_expiration_blocks_access()`
- Start with active subscription
- Change status to cancelled
- Assert access denied
- [ ] Run tests in CI/CD pipeline
- [ ] Document test coverage
#### Task 3.6: Load/performance testing (optional)
**Environment**: Staging environment
- [ ] Test subscription check performance under load
- [ ] Measure latency added by subscription check
- [ ] Verify database query performance
- [ ] Document any performance concerns
- [ ] Optimize if needed
## Implementation Summary Checklist
Use this high-level checklist to track overall progress:
### Phase 1: Cloud Service 🔄
- [x] Add subscription check method to SubscriptionService
- [x] Add subscription validation dependency to deps.py
- [x] Add subscription_url config (env var)
- [x] Protect tenant mount endpoints (4 endpoints)
- [x] Protect proxy endpoints (2 endpoints)
- [ ] Add unit tests for subscription service
- [ ] Add integration tests for dependency
- [ ] Deploy and verify cloud service
### Phase 2: CLI Updates 🔄
- [ ] Review CLI authentication flow
- [ ] Update API client error handling
- [ ] Update CLI login command error handling
- [ ] Add CLI tests
- [ ] Update CLI documentation
### Phase 3: End-to-End Testing 🧪
- [ ] Create test user accounts
- [ ] Manual testing - user without subscription
- [ ] Manual testing - user with active subscription
- [ ] Test subscription state transitions
- [ ] Integration test suite
- [ ] Load/performance testing (optional)
## Questions to Resolve
### Resolved ✅
1. **Admin Access**
- ✅ **Decision**: Admin users bypass subscription check
- **Rationale**: Admin endpoints already use `AdminUserHybridDep`, which is separate from CLI user endpoints
- **Implementation**: No changes needed to admin endpoints
2. **Subscription Check Implementation**
- ✅ **Decision**: Use Option A (Database Check)
- **Rationale**: Simpler, faster to implement, works with existing infrastructure
- **Implementation**: Single JOIN query via `get_subscription_by_workos_user_id()`
3. **Dependency Return Type**
- ✅ **Decision**: Return `UserProfile` (not `UserContext`)
- **Rationale**: Drop-in compatibility with existing endpoints, no refactoring needed
- **Implementation**: `AuthorizedCLIUserProfileDep` returns `UserProfile`
### To Be Resolved ⏳
1. **Subscription Check Frequency**
- **Options**:
- Check on every API call (slower, more secure) ✅ **RECOMMENDED**
- Cache subscription status (faster, risk of stale data)
- Check only on login/setup (fast, but allows expired subscriptions temporarily)
- **Recommendation**: Check on every call via dependency injection (simple, secure, acceptable performance)
- **Impact**: ~5-10ms per request (single indexed JOIN query)
2. **Grace Period**
- **Options**:
- No grace period - immediate block when status != "active" ✅ **RECOMMENDED**
- 7-day grace period after period_end
- 14-day grace period after period_end
- **Recommendation**: No grace period initially, add later if needed based on customer feedback
- **Implementation**: Check `subscription.status == "active"` only (ignore period_end initially)
3. **Subscription Expiration Handling**
- **Question**: Should we check `current_period_end < now()` in addition to `status == "active"`?
- **Options**:
- Only check status field (rely on Polar webhooks to update status) ✅ **RECOMMENDED**
- Check both status and current_period_end (more defensive)
- **Recommendation**: Only check status field, assume Polar webhooks keep it current
- **Risk**: If webhooks fail, expired subscriptions might retain access until webhook succeeds
4. **Subscribe URL**
- **Question**: What's the actual subscription URL?
- **Current**: Spec uses `https://basicmemory.com/subscribe`
- **Action Required**: Verify correct URL before implementation
5. **Dev Mode / Testing Bypass**
- **Question**: Support bypass for development/testing?
- **Options**:
- Environment variable: `DISABLE_SUBSCRIPTION_CHECK=true`
- Always enforce (more realistic testing) ✅ **RECOMMENDED**
- **Recommendation**: No bypass - use test users with real subscriptions for realistic testing
- **Implementation**: Create dev endpoint to activate subscriptions for testing
## Related Specs
- SPEC-9: Multi-Project Bidirectional Sync Architecture (CLI affected by this change)
- SPEC-8: TigrisFS Integration (Mount endpoints protected)
## Notes
- This spec prioritizes security over convenience - better to block unauthorized access than risk revenue loss
- Clear error messages are critical - users should understand why they're blocked and how to resolve it
- Consider adding telemetry to track subscription_required errors for monitoring signup conversion
@@ -0,0 +1,210 @@
---
title: 'SPEC-14: Cloud Git Versioning & GitHub Backup'
type: spec
permalink: specs/spec-14-cloud-git-versioning
tags:
- git
- github
- backup
- versioning
- cloud
related:
- specs/spec-9-multi-project-bisync
- specs/spec-9-follow-ups-conflict-sync-and-observability
status: deferred
---
# SPEC-14: Cloud Git Versioning & GitHub Backup
**Status: DEFERRED** - Postponed until multi-user/teams feature development. Using S3 versioning (SPEC-9.1) for v1 instead.
## Why Deferred
**Original goals can be met with simpler solutions:**
- Version history → **S3 bucket versioning** (automatic, zero config)
- Offsite backup → **Tigris global replication** (built-in)
- Restore capability → **S3 version restore** (`bm cloud restore --version-id`)
- Collaboration → **Deferred to teams/multi-user feature** (not v1 requirement)
**Complexity vs value trade-off:**
- Git integration adds: committer service, puller service, webhooks, LFS, merge conflicts
- Risk: Loop detection between Git ↔ rclone bisync ↔ local edits
- S3 versioning gives 80% of value with 5% of complexity
**When to revisit:**
- Teams/multi-user features (PR-based collaboration workflow)
- User requests for commit messages and branch-based workflows
- Need for fine-grained audit trail beyond S3 object metadata
---
## Original Specification (for reference)
## Why
Early access users want **transparent version history**, easy **offsite backup**, and a familiar **restore/branching** workflow. Git/GitHub integration would provide:
- Auditable history of every change (who/when/why)
- Branches/PRs for review and collaboration
- Offsite private backup under the user's control
- Escape hatch: users can always `git clone` their knowledge base
**Note:** These goals are now addressed via S3 versioning (SPEC-9.1) for single-user use case.
## Goals
- **Transparent**: Users keep using Basic Memory; Git runs behind the scenes.
- **Private**: Push to a **private GitHub repo** that the user owns (or tenant org).
- **Reliable**: No data loss, deterministic mapping of filesystem ↔ Git.
- **Composable**: Plays nicely with SPEC9 bisync and upcoming conflict features (SPEC9 FollowUps).
**NonGoals (for v1):**
- Finegrained perfile encryption in Git history (can be layered later).
- Large media optimization beyond Git LFS defaults.
## User Stories
1. *As a user*, I connect my GitHub and choose a private backup repo.
2. *As a user*, every change I make in cloud (or via bisync) is **committed** and **pushed** automatically.
3. *As a user*, I can **restore** a file/folder/project to a prior version.
4. *As a power user*, I can **git pull/push** directly to collaborate outside the app.
5. *As an admin*, I can enforce repo ownership (tenant org) and leastprivilege scopes.
## Scope
- **In scope:** Full repo backup of `/app/data/` (all projects) with optional selective subpaths.
- **Out of scope (v1):** Partial shallow mirrors; encrypted Git; crossprovider SCM (GitLab/Bitbucket).
## Architecture
### Topology
- **Authoritative working tree**: `/app/data/` (bucket mount) remains the source of truth (SPEC9).
- **Bare repo** lives alongside: `/app/git/${tenant}/knowledge.git` (serverside).
- **Mirror remote**: `github.com/<owner>/<repo>.git` (private).
```mermaid
flowchart LR
A[/Users & Agents/] -->|writes/edits| B[/app/data/]
B -->|file events| C[Committer Service]
C -->|git commit| D[(Bare Repo)]
D -->|push| E[(GitHub Private Repo)]
E -->|webhook (push)| F[Puller Service]
F -->|git pull/merge| D
D -->|checkout/merge| B
```
### Services
- **Committer Service** (daemon):
- Watches `/app/data/` for changes (inotify/poll)
- Batches changes (debounce e.g. 25s)
- Writes `.bmmeta` (if present) into commit message trailer (see FollowUps)
- `git add -A && git commit -m "chore(sync): <summary>
BM-Meta: <json>"`
- Periodic `git push` to GitHub mirror (configurable interval)
- **Puller Service** (webhook target):
- Receives GitHub webhook (push) → `git fetch`
- **Fastforward** merges to `main` only; reject nonFF unless policy allows
- Applies changes back to `/app/data/` via clean checkout
- Emits sync events for Basic Memory indexers
### Auth & Security
- **GitHub App** (recommended): minimal scopes: `contents:read/write`, `metadata:read`, webhook.
- Tenantscoped installation; repo created in user account or tenant org.
- Tokens stored in KMS/secret manager; rotated automatically.
- Optional policy: allow only **FF merges** on `main`; nonFF requires PR.
### Repo Layout
- **Monorepo** (default): one repo per tenant mirrors `/app/data/` with subfolders per project.
- Optional multirepo mode (later): one repo per project.
### File Handling
- Honor `.gitignore` generated from `.bmignore.rclone` + BM defaults (cache, temp, state).
- **Git LFS** for large binaries (images, media) — auto track by extension/size threshold.
- Normalize newline + Unicode (aligns with FollowUps).
### Conflict Model
- **Primary concurrency**: SPEC9 FollowUps (`.bmmeta`, conflict copies) stays the first line of defense.
- **Git merges** are a **secondary** mechanism:
- Server only automerges **text** conflicts when trivial (FF or clean 3way).
- Otherwise, create `name (conflict from <branch>, <ts>).md` and surface via events.
### Data Flow vs Bisync
- Bisync (rclone) continues between local sync dir ↔ bucket.
- Git sits **cloudside** between bucket and GitHub.
- On **pull** from GitHub → files written to `/app/data/` → picked up by indexers & eventually by bisync back to users.
## CLI & UX
New commands (cloud mode):
- `bm cloud git connect` — Launch GitHub App installation; create private repo; store installation id.
- `bm cloud git status` — Show connected repo, last push time, last webhook delivery, pending commits.
- `bm cloud git push` — Manual push (rarely needed).
- `bm cloud git pull` — Manual pull/FF (admin only by default).
- `bm cloud snapshot -m "message"` — Create a tagged pointintime snapshot (git tag).
- `bm restore <path> --to <commit|tag>` — Restore file/folder/project to prior version.
Settings:
- `bm config set git.autoPushInterval=5s`
- `bm config set git.lfs.sizeThreshold=10MB`
- `bm config set git.allowNonFF=false`
## Migration & Backfill
- On connect, if repo empty: initial commit of entire `/app/data/`.
- If repo has content: require **onetime import** path (clone to staging, reconcile, choose direction).
## Edge Cases
- Massive deletes: gated by SPEC9 `max_delete` **and** Git prepush hook checks.
- Case changes and rename detection: rely on git rename heuristics + FollowUps move hints.
- Secrets: default ignore common secret patterns; allow custom deny list.
## Telemetry & Observability
- Emit `git_commit`, `git_push`, `git_pull`, `git_conflict` events with correlation IDs.
- `bm sync --report` extended with Git stats (commit count, delta bytes, push latency).
## Phased Plan
### Phase 0 — Prototype (1 sprint)
- Server: bare repo init + simple committer (batch every 10s) + manual GitHub token.
- CLI: `bm cloud git connect --token <PAT>` (devonly)
- Success: edits in `/app/data/` appear in GitHub within 30s.
### Phase 1 — GitHub App & Webhooks (12 sprints)
- Switch to GitHub App installs; create private repo; store installation id.
- Committer hardened (debounce 25s, backoff, retries).
- Puller service with webhook → FF merge → checkout to `/app/data/`.
- LFS autotrack + `.gitignore` generation.
- CLI surfaces status + logs.
### Phase 2 — Restore & Snapshots (1 sprint)
- `bm restore` for file/folder/project with dryrun.
- `bm cloud snapshot` tags + list/inspect.
- Policy: PRonly nonFF, admin override.
### Phase 3 — Selective & MultiRepo (nicetohave)
- Include/exclude projects; optional perproject repos.
- Advanced policies (branch protections, required reviews).
## Acceptance Criteria
- Changes to `/app/data/` are committed and pushed automatically within configurable interval (default ≤5s).
- GitHub webhook pull results in updated files in `/app/data/` (FFonly by default).
- LFS configured and functioning; large files don't bloat history.
- `bm cloud git status` shows connected repo and last push/pull times.
- `bm restore` restores a file/folder to a prior commit with a clear audit trail.
- Endtoend works alongside SPEC9 bisync without loops or data loss.
## Risks & Mitigations
- **Loop risk (Git ↔ Bisync)**: Writes to `/app/data/` → bisync → local → user edits → back again. *Mitigation*: Debounce, commit squashing, idempotent `.bmmeta` versioning, and watch exclusion windows during pull.
- **Repo bloat**: Lots of binary churn. *Mitigation*: default LFS, size threshold, optional mediaonly repo later.
- **Security**: Token leakage. *Mitigation*: GitHub App with shortlived tokens, KMS storage, scoped permissions.
- **Merge complexity**: Nontrivial conflicts. *Mitigation*: prefer FF; otherwise conflict copies + events; require PR for nonFF.
## Open Questions
- Do we default to **monorepo** per tenant, or offer projectperrepo at connect time?
- Should `restore` write to a branch and open a PR, or directly modify `main`?
- How do we expose Git history in UI (timeline view) without users dropping to CLI?
## Appendix: Sample Config
```json
{
"git": {
"enabled": true,
"repo": "https://github.com/<owner>/<repo>.git",
"autoPushInterval": "5s",
"allowNonFF": false,
"lfs": { "sizeThreshold": 10485760 }
}
}
```
@@ -0,0 +1,210 @@
---
title: 'SPEC-14: Cloud Git Versioning & GitHub Backup'
type: spec
permalink: specs/spec-14-cloud-git-versioning
tags:
- git
- github
- backup
- versioning
- cloud
related:
- specs/spec-9-multi-project-bisync
- specs/spec-9-follow-ups-conflict-sync-and-observability
status: deferred
---
# SPEC-14: Cloud Git Versioning & GitHub Backup
**Status: DEFERRED** - Postponed until multi-user/teams feature development. Using S3 versioning (SPEC-9.1) for v1 instead.
## Why Deferred
**Original goals can be met with simpler solutions:**
- Version history → **S3 bucket versioning** (automatic, zero config)
- Offsite backup → **Tigris global replication** (built-in)
- Restore capability → **S3 version restore** (`bm cloud restore --version-id`)
- Collaboration → **Deferred to teams/multi-user feature** (not v1 requirement)
**Complexity vs value trade-off:**
- Git integration adds: committer service, puller service, webhooks, LFS, merge conflicts
- Risk: Loop detection between Git ↔ rclone bisync ↔ local edits
- S3 versioning gives 80% of value with 5% of complexity
**When to revisit:**
- Teams/multi-user features (PR-based collaboration workflow)
- User requests for commit messages and branch-based workflows
- Need for fine-grained audit trail beyond S3 object metadata
---
## Original Specification (for reference)
## Why
Early access users want **transparent version history**, easy **offsite backup**, and a familiar **restore/branching** workflow. Git/GitHub integration would provide:
- Auditable history of every change (who/when/why)
- Branches/PRs for review and collaboration
- Offsite private backup under the user's control
- Escape hatch: users can always `git clone` their knowledge base
**Note:** These goals are now addressed via S3 versioning (SPEC-9.1) for single-user use case.
## Goals
- **Transparent**: Users keep using Basic Memory; Git runs behind the scenes.
- **Private**: Push to a **private GitHub repo** that the user owns (or tenant org).
- **Reliable**: No data loss, deterministic mapping of filesystem ↔ Git.
- **Composable**: Plays nicely with SPEC9 bisync and upcoming conflict features (SPEC9 FollowUps).
**NonGoals (for v1):**
- Finegrained perfile encryption in Git history (can be layered later).
- Large media optimization beyond Git LFS defaults.
## User Stories
1. *As a user*, I connect my GitHub and choose a private backup repo.
2. *As a user*, every change I make in cloud (or via bisync) is **committed** and **pushed** automatically.
3. *As a user*, I can **restore** a file/folder/project to a prior version.
4. *As a power user*, I can **git pull/push** directly to collaborate outside the app.
5. *As an admin*, I can enforce repo ownership (tenant org) and leastprivilege scopes.
## Scope
- **In scope:** Full repo backup of `/app/data/` (all projects) with optional selective subpaths.
- **Out of scope (v1):** Partial shallow mirrors; encrypted Git; crossprovider SCM (GitLab/Bitbucket).
## Architecture
### Topology
- **Authoritative working tree**: `/app/data/` (bucket mount) remains the source of truth (SPEC9).
- **Bare repo** lives alongside: `/app/git/${tenant}/knowledge.git` (serverside).
- **Mirror remote**: `github.com/<owner>/<repo>.git` (private).
```mermaid
flowchart LR
A[/Users & Agents/] -->|writes/edits| B[/app/data/]
B -->|file events| C[Committer Service]
C -->|git commit| D[(Bare Repo)]
D -->|push| E[(GitHub Private Repo)]
E -->|webhook (push)| F[Puller Service]
F -->|git pull/merge| D
D -->|checkout/merge| B
```
### Services
- **Committer Service** (daemon):
- Watches `/app/data/` for changes (inotify/poll)
- Batches changes (debounce e.g. 25s)
- Writes `.bmmeta` (if present) into commit message trailer (see FollowUps)
- `git add -A && git commit -m "chore(sync): <summary>
BM-Meta: <json>"`
- Periodic `git push` to GitHub mirror (configurable interval)
- **Puller Service** (webhook target):
- Receives GitHub webhook (push) → `git fetch`
- **Fastforward** merges to `main` only; reject nonFF unless policy allows
- Applies changes back to `/app/data/` via clean checkout
- Emits sync events for Basic Memory indexers
### Auth & Security
- **GitHub App** (recommended): minimal scopes: `contents:read/write`, `metadata:read`, webhook.
- Tenantscoped installation; repo created in user account or tenant org.
- Tokens stored in KMS/secret manager; rotated automatically.
- Optional policy: allow only **FF merges** on `main`; nonFF requires PR.
### Repo Layout
- **Monorepo** (default): one repo per tenant mirrors `/app/data/` with subfolders per project.
- Optional multirepo mode (later): one repo per project.
### File Handling
- Honor `.gitignore` generated from `.bmignore.rclone` + BM defaults (cache, temp, state).
- **Git LFS** for large binaries (images, media) — auto track by extension/size threshold.
- Normalize newline + Unicode (aligns with FollowUps).
### Conflict Model
- **Primary concurrency**: SPEC9 FollowUps (`.bmmeta`, conflict copies) stays the first line of defense.
- **Git merges** are a **secondary** mechanism:
- Server only automerges **text** conflicts when trivial (FF or clean 3way).
- Otherwise, create `name (conflict from <branch>, <ts>).md` and surface via events.
### Data Flow vs Bisync
- Bisync (rclone) continues between local sync dir ↔ bucket.
- Git sits **cloudside** between bucket and GitHub.
- On **pull** from GitHub → files written to `/app/data/` → picked up by indexers & eventually by bisync back to users.
## CLI & UX
New commands (cloud mode):
- `bm cloud git connect` — Launch GitHub App installation; create private repo; store installation id.
- `bm cloud git status` — Show connected repo, last push time, last webhook delivery, pending commits.
- `bm cloud git push` — Manual push (rarely needed).
- `bm cloud git pull` — Manual pull/FF (admin only by default).
- `bm cloud snapshot -m "message"` — Create a tagged pointintime snapshot (git tag).
- `bm restore <path> --to <commit|tag>` — Restore file/folder/project to prior version.
Settings:
- `bm config set git.autoPushInterval=5s`
- `bm config set git.lfs.sizeThreshold=10MB`
- `bm config set git.allowNonFF=false`
## Migration & Backfill
- On connect, if repo empty: initial commit of entire `/app/data/`.
- If repo has content: require **onetime import** path (clone to staging, reconcile, choose direction).
## Edge Cases
- Massive deletes: gated by SPEC9 `max_delete` **and** Git prepush hook checks.
- Case changes and rename detection: rely on git rename heuristics + FollowUps move hints.
- Secrets: default ignore common secret patterns; allow custom deny list.
## Telemetry & Observability
- Emit `git_commit`, `git_push`, `git_pull`, `git_conflict` events with correlation IDs.
- `bm sync --report` extended with Git stats (commit count, delta bytes, push latency).
## Phased Plan
### Phase 0 — Prototype (1 sprint)
- Server: bare repo init + simple committer (batch every 10s) + manual GitHub token.
- CLI: `bm cloud git connect --token <PAT>` (devonly)
- Success: edits in `/app/data/` appear in GitHub within 30s.
### Phase 1 — GitHub App & Webhooks (12 sprints)
- Switch to GitHub App installs; create private repo; store installation id.
- Committer hardened (debounce 25s, backoff, retries).
- Puller service with webhook → FF merge → checkout to `/app/data/`.
- LFS autotrack + `.gitignore` generation.
- CLI surfaces status + logs.
### Phase 2 — Restore & Snapshots (1 sprint)
- `bm restore` for file/folder/project with dryrun.
- `bm cloud snapshot` tags + list/inspect.
- Policy: PRonly nonFF, admin override.
### Phase 3 — Selective & MultiRepo (nicetohave)
- Include/exclude projects; optional perproject repos.
- Advanced policies (branch protections, required reviews).
## Acceptance Criteria
- Changes to `/app/data/` are committed and pushed automatically within configurable interval (default ≤5s).
- GitHub webhook pull results in updated files in `/app/data/` (FFonly by default).
- LFS configured and functioning; large files don't bloat history.
- `bm cloud git status` shows connected repo and last push/pull times.
- `bm restore` restores a file/folder to a prior commit with a clear audit trail.
- Endtoend works alongside SPEC9 bisync without loops or data loss.
## Risks & Mitigations
- **Loop risk (Git ↔ Bisync)**: Writes to `/app/data/` → bisync → local → user edits → back again. *Mitigation*: Debounce, commit squashing, idempotent `.bmmeta` versioning, and watch exclusion windows during pull.
- **Repo bloat**: Lots of binary churn. *Mitigation*: default LFS, size threshold, optional mediaonly repo later.
- **Security**: Token leakage. *Mitigation*: GitHub App with shortlived tokens, KMS storage, scoped permissions.
- **Merge complexity**: Nontrivial conflicts. *Mitigation*: prefer FF; otherwise conflict copies + events; require PR for nonFF.
## Open Questions
- Do we default to **monorepo** per tenant, or offer projectperrepo at connect time?
- Should `restore` write to a branch and open a PR, or directly modify `main`?
- How do we expose Git history in UI (timeline view) without users dropping to CLI?
## Appendix: Sample Config
```json
{
"git": {
"enabled": true,
"repo": "https://github.com/<owner>/<repo>.git",
"autoPushInterval": "5s",
"allowNonFF": false,
"lfs": { "sizeThreshold": 10485760 }
}
}
```
@@ -0,0 +1,273 @@
---
title: 'SPEC-15: Configuration Persistence via Tigris for Cloud Tenants'
type: spec
permalink: specs/spec-14-config-persistence-tigris
tags:
- persistence
- tigris
- multi-tenant
- infrastructure
- configuration
status: draft
---
# SPEC-15: Configuration Persistence via Tigris for Cloud Tenants
## Why
We need to persist Basic Memory configuration across Fly.io deployments without using persistent volumes or external databases.
**Current Problems:**
- `~/.basic-memory/config.json` lost on every deployment (project configuration)
- `~/.basic-memory/memory.db` lost on every deployment (search index)
- Persistent volumes break clean deployment workflow
- External databases (Turso) require per-tenant token management
**The Insight:**
The SQLite database is just an **index cache** of the markdown files. It can be rebuilt in seconds from the source markdown files in Tigris. Only the small `config.json` file needs true persistence.
**Solution:**
- Store `config.json` in Tigris bucket (persistent, small file)
- Rebuild `memory.db` on startup from markdown files (fast, ephemeral)
- No persistent volumes, no external databases, no token management
## What
Store Basic Memory configuration in the Tigris bucket and rebuild the database index on tenant machine startup.
**Affected Components:**
- `basic-memory/src/basic_memory/config.py` - Add configurable config directory
**Architecture:**
```bash
# Tigris Bucket (persistent, mounted at /app/data)
/app/data/
├── .basic-memory/
│ └── config.json # ← Project configuration (persistent, accessed via BASIC_MEMORY_CONFIG_DIR)
└── basic-memory/ # ← Markdown files (persistent, BASIC_MEMORY_HOME)
├── project1/
└── project2/
# Fly Machine (ephemeral)
/app/.basic-memory/
└── memory.db # ← Rebuilt on startup (fast local disk)
```
## How (High Level)
### 1. Add Configurable Config Directory to Basic Memory
Currently `ConfigManager` hardcodes `~/.basic-memory/config.json`. Add environment variable to override:
```python
# basic-memory/src/basic_memory/config.py
class ConfigManager:
"""Manages Basic Memory configuration."""
def __init__(self) -> None:
"""Initialize the configuration manager."""
home = os.getenv("HOME", Path.home())
if isinstance(home, str):
home = Path(home)
# Allow override via environment variable
if config_dir := os.getenv("BASIC_MEMORY_CONFIG_DIR"):
self.config_dir = Path(config_dir)
else:
self.config_dir = home / DATA_DIR_NAME
self.config_file = self.config_dir / CONFIG_FILE_NAME
# Ensure config directory exists
self.config_dir.mkdir(parents=True, exist_ok=True)
```
### 2. Rebuild Database on Startup
Basic Memory already has the sync functionality. Just ensure it runs on startup:
```python
# apps/api/src/basic_memory_cloud_api/main.py
@app.on_event("startup")
async def startup_sync():
"""Rebuild database index from Tigris markdown files."""
logger.info("Starting database rebuild from Tigris")
# Initialize file sync (rebuilds index from markdown files)
app_config = ConfigManager().config
await initialize_file_sync(app_config)
logger.info("Database rebuild complete")
```
### 3. Environment Configuration
```bash
# Machine environment variables
BASIC_MEMORY_CONFIG_DIR=/app/data/.basic-memory # Config read/written directly to Tigris
# memory.db stays in default location: /app/.basic-memory/memory.db (local ephemeral disk)
```
## Implementation Task List
### Phase 1: Basic Memory Changes ✅
- [x] Add `BASIC_MEMORY_CONFIG_DIR` environment variable support to `ConfigManager.__init__()`
- [x] Test config loading from custom directory
- [x] Update tests to verify custom config dir works
### Phase 2: Tigris Bucket Structure ✅
- [x] Ensure `.basic-memory/` directory exists in Tigris bucket on tenant creation
- ✅ ConfigManager auto-creates on first run, no explicit provisioning needed
- [x] Initialize `config.json` in Tigris on first tenant deployment
- ✅ ConfigManager creates config.json automatically in BASIC_MEMORY_CONFIG_DIR
- [x] Verify TigrisFS handles hidden directories correctly
- ✅ TigrisFS supports hidden directories (verified in SPEC-8)
### Phase 3: Deployment Integration ✅
- [x] Set `BASIC_MEMORY_CONFIG_DIR` environment variable in machine deployment
- ✅ Added to BasicMemoryMachineConfigBuilder in fly_schemas.py
- [x] Ensure database rebuild runs on machine startup via initialization sync
- ✅ sync_worker.py runs initialize_file_sync every 30s (already implemented)
- [x] Handle first-time tenant setup (no config exists yet)
- ✅ ConfigManager creates config.json on first initialization
- [ ] Test deployment workflow with config persistence
### Phase 4: Testing
- [x] Unit tests for config directory override
- [-] Integration test: deploy → write config → redeploy → verify config persists
- [ ] Integration test: deploy → add project → redeploy → verify project in config
- [ ] Performance test: measure db rebuild time on startup
### Phase 5: Documentation
- [ ] Document config persistence architecture
- [ ] Update deployment runbook
- [ ] Document startup sequence and timing
## How to Evaluate
### Success Criteria
1. **Config Persistence**
- [ ] config.json persists across deployments
- [ ] Projects list maintained across restarts
- [ ] No manual configuration needed after redeploy
2. **Database Rebuild**
- [ ] memory.db rebuilt on startup in < 30 seconds
- [ ] All entities indexed correctly
- [ ] Search functionality works after rebuild
3. **Performance**
- [ ] SQLite queries remain fast (local disk)
- [ ] Config reads acceptable (symlink to Tigris)
- [ ] No noticeable performance degradation
4. **Deployment Workflow**
- [ ] Clean deployments without volumes
- [ ] No new external dependencies
- [ ] No secret management needed
### Testing Procedure
1. **Config Persistence Test**
```bash
# Deploy tenant
POST /tenants → tenant_id
# Add a project
basic-memory project add "test-project" ~/test
# Verify config has project
cat /app/data/.basic-memory/config.json
# Redeploy machine
fly deploy --app basic-memory-{tenant_id}
# Verify project still exists
basic-memory project list
```
2. **Database Rebuild Test**
```bash
# Create notes
basic-memory write "Test Note" --content "..."
# Redeploy (db lost)
fly deploy --app basic-memory-{tenant_id}
# Wait for startup sync
sleep 10
# Verify note is indexed
basic-memory search "Test Note"
```
3. **Performance Benchmark**
```bash
# Time the startup sync
time basic-memory sync
# Should be < 30 seconds for typical tenant
```
## Benefits Over Alternatives
**vs. Persistent Volumes:**
- ✅ Clean deployment workflow
- ✅ No volume migration needed
- ✅ Simpler infrastructure
**vs. Turso (External Database):**
- ✅ No per-tenant token management
- ✅ No external service dependencies
- ✅ No additional costs
- ✅ Simpler architecture
**vs. SQLite on FUSE:**
- ✅ Fast local SQLite performance
- ✅ Only slow reads for small config file
- ✅ Database queries remain fast
## Implementation Assignment
**Primary Agent:** `python-developer`
- Add `BASIC_MEMORY_CONFIG_DIR` environment variable to ConfigManager
- Update deployment workflow to set environment variable
- Ensure startup sync runs correctly
**Review Agent:** `system-architect`
- Validate architecture simplicity
- Review performance implications
- Assess startup timing
## Dependencies
- **Internal:** TigrisFS must be working and stable
- **Internal:** Basic Memory sync must be reliable
- **Internal:** SPEC-8 (TigrisFS Integration) must be complete
## Open Questions
1. Should we add a health check that waits for db rebuild to complete?
2. Do we need to handle very large knowledge bases (>10k entities) differently?
3. Should we add metrics for startup sync duration?
## References
- Basic Memory sync: `basic-memory/src/basic_memory/services/initialization.py`
- Config management: `basic-memory/src/basic_memory/config.py`
- TigrisFS integration: SPEC-8
---
**Status Updates:**
- 2025-10-08: Pivoted from Turso to Tigris-based config persistence
- 2025-10-08: Phase 1 complete - BASIC_MEMORY_CONFIG_DIR support added (PR #343)
- 2025-10-08: Phases 2-3 complete - Added BASIC_MEMORY_CONFIG_DIR to machine config
- Config now persists to /app/data/.basic-memory/config.json in Tigris bucket
- Database rebuild already working via sync_worker.py
- Ready for deployment testing (Phase 4)
@@ -0,0 +1,800 @@
---
title: 'SPEC-16: MCP Cloud Service Consolidation'
type: spec
permalink: specs/spec-16-mcp-cloud-service-consolidation
tags:
- architecture
- mcp
- cloud
- performance
- deployment
status: in-progress
---
## Status Update
**Phase 0 (Basic Memory Refactor): ✅ COMPLETE**
- basic-memory PR #344: async_client context manager pattern implemented
- All 17 MCP tools updated to use `async with get_client() as client:`
- CLI commands updated to use context manager
- Removed `inject_auth_header()` and `headers.py` (~100 lines deleted)
- Factory pattern enables clean dependency injection
- Tests passing, typecheck clean
**Phase 0 Integration: ✅ COMPLETE**
- basic-memory-cloud updated to use async-client-context-manager branch
- Implemented `tenant_direct_client_factory()` with proper context manager pattern
- Removed module-level client override hacks
- Removed unnecessary `/proxy` prefix stripping (tools pass relative URLs)
- Typecheck and lint passing with proper noqa hints
- MCP tools confirmed working via inspector (local testing)
**Phase 1 (Code Consolidation): ✅ COMPLETE**
- MCP server mounted on Cloud FastAPI app at /mcp endpoint
- AuthKitProvider configured with WorkOS settings
- Combined lifespans (Cloud + MCP) working correctly
- JWT context middleware integrated
- All routes and MCP tools functional
**Phase 2 (Direct Tenant Transport): ✅ COMPLETE**
- TenantDirectTransport implemented with custom httpx transport
- Per-request JWT extraction via FastMCP DI
- Tenant lookup and signed header generation working
- Direct routing to tenant APIs (eliminating HTTP hop)
- Transport tests passing (11/11)
**Phase 3 (Testing & Validation): ✅ COMPLETE**
- Typecheck and lint passing across all services
- MCP OAuth authentication working in preview environment
- Tenant isolation via signed headers verified
- Fixed BM_TENANT_HEADER_SECRET mismatch between environments
- MCP tools successfully calling tenant APIs in preview
**Phase 4 (Deployment Configuration): ✅ COMPLETE**
- Updated apps/cloud/fly.template.toml with MCP environment variables
- Added HTTP/2 backend support for better MCP performance
- Added OAuth protected resource health check
- Removed MCP from preview deployment workflow
- Successfully deployed to preview environment (PR #113)
- All services operational at pr-113-basic-memory-cloud.fly.dev
**Next Steps:**
- Phase 5: Cleanup (remove apps/mcp directory)
- Phase 6: Production rollout and performance measurement
# SPEC-16: MCP Cloud Service Consolidation
## Why
### Original Architecture Constraints (Now Removed)
The current architecture deploys MCP Gateway and Cloud Service as separate Fly.io apps:
**Current Flow:**
```
LLM Client → MCP Gateway (OAuth) → Cloud Proxy (JWT + header signing) → Tenant API (JWT + header validation)
apps/mcp apps/cloud /proxy apps/api
```
This separation was originally necessary because:
1. **Stateful SSE requirement** - MCP needed server-sent events with session state for active project tracking
2. **fastmcp.run limitation** - The FastMCP demo helper didn't support worker processes
### Why These Constraints No Longer Apply
1. **State externalized** - Project state moved from in-memory to LLM context (external state)
2. **HTTP transport enabled** - Switched from SSE to stateless HTTP for MCP tools
3. **Worker support added** - Converted from `fastmcp.run()` to `uvicorn.run()` with workers
### Current Problems
- **Unnecessary HTTP hop** - MCP tools call Cloud /proxy endpoint which calls tenant API
- **Higher latency** - Extra network round trip for every MCP operation
- **Increased costs** - Two separate Fly.io apps instead of one
- **Complex deployment** - Two services to deploy, monitor, and maintain
- **Resource waste** - Separate database connections, HTTP clients, telemetry overhead
## What
### Services Affected
1. **apps/mcp** - MCP Gateway service (to be merged)
2. **apps/cloud** - Cloud service (will receive MCP functionality)
3. **basic-memory** - Update `async_client.py` to use direct calls
4. **Deployment** - Consolidate Fly.io deployment to single app
### Components Changed
**Merged:**
- MCP middleware and telemetry into Cloud app
- MCP tools mounted on Cloud FastAPI instance
- ProxyService used directly by MCP tools (not via HTTP)
**Kept:**
- `/proxy` endpoint (still needed by web UI)
- All existing Cloud routes (provisioning, webhooks, etc.)
- Dual validation in tenant API (JWT + signed headers)
**Removed:**
- apps/mcp directory
- Separate MCP Fly.io deployment
- HTTP calls from MCP tools to /proxy endpoint
## How (High Level)
### 1. Mount FastMCP on Cloud FastAPI App
```python
# apps/cloud/src/basic_memory_cloud/main.py
from basic_memory.mcp.server import mcp
from basic_memory_cloud_mcp.middleware import TelemetryMiddleware
# Configure MCP OAuth
auth_provider = AuthKitProvider(
authkit_domain=settings.authkit_domain,
base_url=settings.authkit_base_url,
required_scopes=[],
)
mcp.auth = auth_provider
mcp.add_middleware(TelemetryMiddleware())
# Mount MCP at /mcp endpoint
mcp_app = mcp.http_app(path="/mcp", stateless_http=True)
app.mount("/mcp", mcp_app)
# Existing Cloud routes stay at root
app.include_router(proxy_router)
app.include_router(provisioning_router)
# ... etc
```
### 2. Direct Tenant Transport (No HTTP Hop)
Instead of calling `/proxy`, MCP tools call tenant APIs directly via custom httpx transport.
**Important:** No URL prefix stripping needed. The transport receives relative URLs like `/main/resource/notes/my-note` which are correctly routed to tenant APIs. The `/proxy` prefix only exists for web UI requests to the proxy router, not for MCP tools using the custom transport.
```python
# apps/cloud/src/basic_memory_cloud/transports/tenant_direct.py
from httpx import AsyncBaseTransport, Request, Response
from fastmcp.server.dependencies import get_http_headers
import jwt
class TenantDirectTransport(AsyncBaseTransport):
"""Direct transport to tenant APIs, bypassing /proxy endpoint."""
async def handle_async_request(self, request: Request) -> Response:
# 1. Get JWT from current MCP request (via FastMCP DI)
http_headers = get_http_headers()
auth_header = http_headers.get("authorization") or http_headers.get("Authorization")
token = auth_header.replace("Bearer ", "")
claims = jwt.decode(token, options={"verify_signature": False})
workos_user_id = claims["sub"]
# 2. Look up tenant for user
tenant = await tenant_service.get_tenant_by_user_id(workos_user_id)
# 3. Build tenant app URL with signed headers
fly_app_name = f"{settings.tenant_prefix}-{tenant.id}"
target_url = f"https://{fly_app_name}.fly.dev{request.url.path}"
headers = dict(request.headers)
signer = create_signer(settings.bm_tenant_header_secret)
headers.update(signer.sign_tenant_headers(tenant.id))
# 4. Make direct call to tenant API
response = await self.client.request(
method=request.method, url=target_url,
headers=headers, content=request.content
)
return response
```
Then configure basic-memory's client factory before mounting MCP:
```python
# apps/cloud/src/basic_memory_cloud/main.py
from contextlib import asynccontextmanager
from basic_memory.mcp import async_client
from basic_memory_cloud.transports.tenant_direct import TenantDirectTransport
# Configure factory for basic-memory's async_client
@asynccontextmanager
async def tenant_direct_client_factory():
"""Factory for creating clients with tenant direct transport."""
client = httpx.AsyncClient(
transport=TenantDirectTransport(),
base_url="http://direct",
)
try:
yield client
finally:
await client.aclose()
# Set factory BEFORE importing MCP tools
async_client.set_client_factory(tenant_direct_client_factory)
# NOW import - tools will use our factory
import basic_memory.mcp.tools
import basic_memory.mcp.prompts
from basic_memory.mcp.server import mcp
# Mount MCP - tools use direct transport via factory
app.mount("/mcp", mcp_app)
```
**Key benefits:**
- Clean dependency injection via factory pattern
- Per-request tenant resolution via FastMCP DI
- Proper resource cleanup (client.aclose() guaranteed)
- Eliminates HTTP hop entirely
- /proxy endpoint remains for web UI
### 3. Keep /proxy Endpoint for Web UI
The existing `/proxy` HTTP endpoint remains functional for:
- Web UI requests
- Future external API consumers
- Backward compatibility
### 4. Security: Maintain Dual Validation
**Do NOT remove JWT validation from tenant API.** Keep defense in depth:
```python
# apps/api - Keep both validations
1. JWT validation (from WorkOS token)
2. Signed header validation (from Cloud/MCP)
```
This ensures if the Cloud service is compromised, attackers still cannot access tenant APIs without valid JWTs.
### 5. Deployment Changes
**Before:**
- `apps/mcp/fly.template.toml` → MCP Gateway deployment
- `apps/cloud/fly.template.toml` → Cloud Service deployment
**After:**
- Remove `apps/mcp/fly.template.toml`
- Update `apps/cloud/fly.template.toml` to expose port 8000 for both /mcp and /proxy
- Update deployment scripts to deploy single consolidated app
## Basic Memory Dependency: Async Client Refactor
### Problem
The current `basic_memory.mcp.async_client` creates a module-level `client` at import time:
```python
client = create_client() # Runs immediately when module is imported
```
This prevents dependency injection - by the time we can override it, tools have already imported it.
### Solution: Context Manager Pattern with Auth at Client Creation
Refactor basic-memory to use httpx's context manager pattern instead of module-level client.
**Key principle:** Authentication happens at client creation time, not per-request.
```python
# basic_memory/src/basic_memory/mcp/async_client.py
from contextlib import asynccontextmanager
from httpx import AsyncClient, ASGITransport, Timeout
# Optional factory override for dependency injection
_client_factory = None
def set_client_factory(factory):
"""Override the default client factory (for cloud app, testing, etc)."""
global _client_factory
_client_factory = factory
@asynccontextmanager
async def get_client():
"""Get an AsyncClient as a context manager.
Usage:
async with get_client() as client:
response = await client.get(...)
"""
if _client_factory:
# Cloud app: custom transport handles everything
async with _client_factory() as client:
yield client
else:
# Default: create based on config
config = ConfigManager().config
timeout = Timeout(connect=10.0, read=30.0, write=30.0, pool=30.0)
if config.cloud_mode_enabled:
# CLI cloud mode: inject auth when creating client
from basic_memory.cli.auth import CLIAuth
auth = CLIAuth(
client_id=config.cloud_client_id,
authkit_domain=config.cloud_domain
)
token = await auth.get_valid_token()
if not token:
raise RuntimeError(
"Cloud mode enabled but not authenticated. "
"Run 'basic-memory cloud login' first."
)
# Auth header set ONCE at client creation
async with AsyncClient(
base_url=f"{config.cloud_host}/proxy",
headers={"Authorization": f"Bearer {token}"},
timeout=timeout
) as client:
yield client
else:
# Local mode: ASGI transport
async with AsyncClient(
transport=ASGITransport(app=fastapi_app),
base_url="http://test",
timeout=timeout
) as client:
yield client
```
**Tool Updates:**
```python
# Before: from basic_memory.mcp.async_client import client
from basic_memory.mcp.async_client import get_client
async def read_note(...):
# Before: response = await call_get(client, path, ...)
async with get_client() as client:
response = await call_get(client, path, ...)
# ... use response
```
**Cloud Usage:**
```python
from contextlib import asynccontextmanager
from basic_memory.mcp import async_client
@asynccontextmanager
async def tenant_direct_client():
"""Factory for creating clients with tenant direct transport."""
client = httpx.AsyncClient(
transport=TenantDirectTransport(),
base_url="http://direct",
)
try:
yield client
finally:
await client.aclose()
# Before importing MCP tools:
async_client.set_client_factory(tenant_direct_client)
# Now import - tools will use our factory
import basic_memory.mcp.tools
```
### Benefits
- **No module-level state** - client created only when needed
- **Proper cleanup** - context manager ensures `aclose()` is called
- **Easy dependency injection** - factory pattern allows custom clients
- **httpx best practices** - follows official recommendations
- **Works for all modes** - stdio, cloud, testing
### Architecture Simplification: Auth at Client Creation
**Key design principle:** Authentication happens when creating the client, not on every request.
**Three modes, three approaches:**
1. **Local mode (ASGI)**
- No auth needed
- Direct in-process calls via ASGITransport
2. **CLI cloud mode (HTTP)**
- Auth token from CLIAuth (stored in ~/.basic-memory/basic-memory-cloud.json)
- Injected as default header when creating AsyncClient
- Single auth check at client creation time
3. **Cloud app mode (Custom Transport)**
- TenantDirectTransport handles everything
- Extracts JWT from FastMCP context per-request
- No interaction with inject_auth_header() logic
**What this removes:**
- `src/basic_memory/mcp/tools/headers.py` - entire file deleted
- `inject_auth_header()` calls in all request helpers (call_get, call_post, etc.)
- Per-request header manipulation complexity
- Circular dependency concerns between async_client and auth logic
**Benefits:**
- Cleaner separation of concerns
- Simpler request helper functions
- Auth happens at the right layer (client creation)
- Cloud app transport is completely independent
### Refactor Summary
This refactor achieves:
**Simplification:**
- Removes ~100 lines of per-request header injection logic
- Deletes entire `headers.py` module
- Auth happens once at client creation, not per-request
**Decoupling:**
- Cloud app's custom transport is completely independent
- No interaction with basic-memory's auth logic
- Each mode (local, CLI cloud, cloud app) has clean separation
**Better Design:**
- Follows httpx best practices (context managers)
- Proper resource cleanup (client.aclose() guaranteed)
- Easier testing via factory injection
- No circular import risks
**Three Distinct Modes:**
1. Local: ASGI transport, no auth
2. CLI cloud: HTTP transport with CLIAuth token injection
3. Cloud app: Custom transport with per-request tenant routing
### Implementation Plan Summary
1. Create branch `async-client-context-manager` in basic-memory
2. Update `async_client.py` with context manager pattern and CLIAuth integration
3. Remove `inject_auth_header()` from all request helpers
4. Delete `src/basic_memory/mcp/tools/headers.py`
5. Update all MCP tools to use `async with get_client() as client:`
6. Update CLI commands to use context manager and remove manual auth
7. Remove `api_url` config field
8. Update tests
9. Update basic-memory-cloud to use branch: `basic-memory @ git+https://github.com/basicmachines-co/basic-memory.git@async-client-context-manager`
Detailed breakdown in Phase 0 tasks below.
### Implementation Notes
**Potential Issues & Solutions:**
1. **Circular Import** (async_client imports CLIAuth)
- **Risk:** CLIAuth might import something from async_client
- **Solution:** Use lazy import inside `get_client()` function
- **Already done:** Import is inside the function, not at module level
2. **Test Fixtures**
- **Risk:** Tests using module-level client will break
- **Solution:** Update fixtures to use factory pattern
- **Example:**
```python
@pytest.fixture
def mock_client_factory():
@asynccontextmanager
async def factory():
async with AsyncClient(...) as client:
yield client
return factory
```
3. **Performance**
- **Risk:** Creating client per tool call might be expensive
- **Reality:** httpx is designed for this pattern, connection pooling at transport level
- **Mitigation:** Monitor performance, can optimize later if needed
4. **CLI Cloud Commands Edge Cases**
- **Risk:** Token expires mid-operation
- **Solution:** CLIAuth.get_valid_token() already handles refresh
- **Validation:** Test cloud login → use tools → token refresh flow
5. **Backward Compatibility**
- **Risk:** External code importing `client` directly
- **Solution:** Keep `create_client()` and `client` for one version, deprecate
- **Timeline:** Remove in next major version
## Implementation Tasks
### Phase 0: Basic Memory Refactor (Prerequisite)
#### 0.1 Core Refactor - async_client.py
- [x] Create branch `async-client-context-manager` in basic-memory repo
- [x] Implement `get_client()` context manager
- [x] Implement `set_client_factory()` for dependency injection
- [x] Add CLI cloud mode auth injection (CLIAuth integration)
- [x] Remove `api_url` config field (legacy, unused)
- [x] Keep `create_client()` temporarily for backward compatibility (deprecate later)
#### 0.2 Simplify Request Helpers - tools/utils.py
- [x] Remove `inject_auth_header()` calls from `call_get()`
- [x] Remove `inject_auth_header()` calls from `call_post()`
- [x] Remove `inject_auth_header()` calls from `call_put()`
- [x] Remove `inject_auth_header()` calls from `call_patch()`
- [x] Remove `inject_auth_header()` calls from `call_delete()`
- [x] Delete `src/basic_memory/mcp/tools/headers.py` entirely
- [x] Update imports in utils.py
#### 0.3 Update MCP Tools (~16 files)
Convert from `from async_client import client` to `async with get_client() as client:`
- [x] `tools/write_note.py` (34/34 tests passing)
- [x] `tools/read_note.py` (21/21 tests passing)
- [x] `tools/view_note.py` (12/12 tests passing - no changes needed, delegates to read_note)
- [x] `tools/delete_note.py` (2/2 tests passing)
- [x] `tools/read_content.py` (20/20 tests passing)
- [x] `tools/list_directory.py` (11/11 tests passing)
- [x] `tools/move_note.py` (34/34 tests passing, 90% coverage)
- [x] `tools/search.py` (16/16 tests passing, 96% coverage)
- [x] `tools/recent_activity.py` (4/4 tests passing, 82% coverage)
- [x] `tools/project_management.py` (3 functions: list_memory_projects, create_memory_project, delete_project - typecheck passed)
- [x] `tools/edit_note.py` (17/17 tests passing)
- [x] `tools/canvas.py` (5/5 tests passing)
- [x] `tools/build_context.py` (6/6 tests passing)
- [x] `tools/sync_status.py` (typecheck passed)
- [x] `prompts/continue_conversation.py` (typecheck passed)
- [x] `prompts/search.py` (typecheck passed)
- [x] `resources/project_info.py` (typecheck passed)
#### 0.4 Update CLI Commands (~3 files)
Remove manual auth header passing, use context manager:
- [x] `cli/commands/project.py` - removed get_authenticated_headers() calls, use context manager
- [x] `cli/commands/status.py` - use context manager
- [x] `cli/commands/command_utils.py` - use context manager
#### 0.5 Update Config
- [x] Remove `api_url` field from `BasicMemoryConfig` in config.py
- [x] Update any lingering references/docs (added deprecation notice to v15-docs/cloud-mode-usage.md)
#### 0.6 Testing
- [-] Update test fixtures to use factory pattern
- [x] Run full test suite in basic-memory
- [x] Verify cloud_mode_enabled works with CLIAuth injection
- [x] Run typecheck and linting
#### 0.7 Cloud Integration Prep
- [x] Update basic-memory-cloud pyproject.toml to use branch
- [x] Implement factory pattern in cloud app main.py
- [x] Remove `/proxy` prefix stripping logic (not needed - tools pass relative URLs)
#### 0.8 Phase 0 Validation
**Before merging async-client-context-manager branch:**
- [x] All tests pass locally
- [x] Typecheck passes (pyright/mypy)
- [x] Linting passes (ruff)
- [x] Manual test: local mode works (ASGI transport)
- [x] Manual test: cloud login → cloud mode works (HTTP transport with auth)
- [x] No import of `inject_auth_header` anywhere
- [x] `headers.py` file deleted
- [x] `api_url` config removed
- [x] Tool functions properly scoped (client inside async with)
- [ ] CLI commands properly scoped (client inside async with)
**Integration validation:**
- [x] basic-memory-cloud can import and use factory pattern
- [x] TenantDirectTransport works without touching header injection
- [x] No circular imports or lazy import issues
- [x] MCP tools work via inspector (local testing confirmed)
### Phase 1: Code Consolidation
- [x] Create feature branch `consolidate-mcp-cloud`
- [x] Update `apps/cloud/src/basic_memory_cloud/config.py`:
- [x] Add `authkit_base_url` field (already has authkit_domain)
- [x] Workers config already exists ✓
- [x] Update `apps/cloud/src/basic_memory_cloud/telemetry.py`:
- [x] Add `logfire.instrument_mcp()` to existing setup
- [x] Skip complex two-phase setup - use Cloud's simpler approach
- [x] Create `apps/cloud/src/basic_memory_cloud/middleware/jwt_context.py`:
- [x] FastAPI middleware to extract JWT claims from Authorization header
- [x] Add tenant context (workos_user_id) to logfire baggage
- [x] Simpler than FastMCP middleware version
- [x] Update `apps/cloud/src/basic_memory_cloud/main.py`:
- [x] Import FastMCP server from basic-memory
- [x] Configure AuthKitProvider with WorkOS settings
- [x] No FastMCP telemetry middleware needed (using FastAPI middleware instead)
- [x] Create MCP ASGI app: `mcp_app = mcp.http_app(path='/mcp', stateless_http=True)`
- [x] Combine lifespans (Cloud + MCP) using nested async context managers
- [x] Mount MCP: `app.mount("/mcp", mcp_app)`
- [x] Add JWT context middleware to FastAPI app
- [x] Run typecheck - passes ✓
### Phase 2: Direct Tenant Transport
- [x] Create `apps/cloud/src/basic_memory_cloud/transports/tenant_direct.py`:
- [x] Implement `TenantDirectTransport(AsyncBaseTransport)`
- [x] Use FastMCP DI (`get_http_headers()`) to extract JWT per-request
- [x] Decode JWT to get `workos_user_id`
- [x] Look up/create tenant via `TenantRepository.get_or_create_tenant_for_workos_user()`
- [x] Build tenant app URL and add signed headers
- [x] Make direct httpx call to tenant API
- [x] No `/proxy` prefix stripping needed (tools pass relative URLs like `/main/resource/...`)
- [x] Update `apps/cloud/src/basic_memory_cloud/main.py`:
- [x] Refactored to use factory pattern instead of module-level override
- [x] Implement `tenant_direct_client_factory()` context manager
- [x] Call `async_client.set_client_factory()` before importing MCP tools
- [x] Clean imports, proper noqa hints for lint
- [x] Basic-memory refactor integrated (PR #344)
- [x] Run typecheck - passes ✓
- [x] Run lint - passes ✓
### Phase 3: Testing & Validation
- [x] Run `just typecheck` in apps/cloud
- [x] Run `just check` in project
- [x] Run `just fix` - all lint errors fixed ✓
- [x] Write comprehensive transport tests (11 tests passing) ✓
- [x] Test MCP tools locally with consolidated service (inspector confirmed working)
- [x] Verify OAuth authentication works (requires full deployment)
- [x] Verify tenant isolation via signed headers (requires full deployment)
- [x] Test /proxy endpoint still works for web UI
- [ ] Measure latency before/after consolidation
- [ ] Check telemetry traces span correctly
### Phase 4: Deployment Configuration
- [x] Update `apps/cloud/fly.template.toml`:
- [x] Merged MCP-specific environment variables (AUTHKIT_BASE_URL, FASTMCP_LOG_LEVEL, BASIC_MEMORY_*)
- [x] Added HTTP/2 backend support (`h2_backend = true`) for better MCP performance
- [x] Added health check for MCP OAuth endpoint (`/.well-known/oauth-protected-resource`)
- [x] Port 8000 already exposed - serves both Cloud routes and /mcp endpoint
- [x] Workers configured (UVICORN_WORKERS = 4)
- [x] Update `.env.example`:
- [x] Consolidated MCP Gateway section into Cloud app section
- [x] Added AUTHKIT_BASE_URL, FASTMCP_LOG_LEVEL, BASIC_MEMORY_HOME
- [x] Added LOG_LEVEL to Development Settings
- [x] Documented that MCP now served at /mcp on Cloud service (port 8000)
- [x] Test deployment to preview environment (PR #113)
- [x] OAuth authentication verified
- [x] MCP tools successfully calling tenant APIs
- [x] Fixed BM_TENANT_HEADER_SECRET synchronization issue
### Phase 5: Cleanup
- [x] Remove `apps/mcp/` directory entirely
- [x] Remove MCP-specific fly.toml and deployment configs
- [x] Update repository documentation
- [x] Update CLAUDE.md with new architecture
- [-] Archive old MCP deployment configs (if needed)
### Phase 6: Production Rollout
- [ ] Deploy to development and validate
- [ ] Monitor metrics and logs
- [ ] Deploy to production
- [ ] Verify production functionality
- [ ] Document performance improvements
## Migration Plan
### Phase 1: Preparation
1. Create feature branch `consolidate-mcp-cloud`
2. Update basic-memory async_client.py for direct ProxyService calls
3. Update apps/cloud/main.py to mount MCP
### Phase 2: Testing
1. Local testing with consolidated app
2. Deploy to development environment
3. Run full test suite
4. Performance benchmarking
### Phase 3: Deployment
1. Deploy to development
2. Validate all functionality
3. Deploy to production
4. Monitor for issues
### Phase 4: Cleanup
1. Remove apps/mcp directory
2. Update documentation
3. Update deployment scripts
4. Archive old MCP deployment configs
## Rollback Plan
If issues arise:
1. Revert feature branch
2. Redeploy separate apps/mcp and apps/cloud services
3. Restore previous fly.toml configurations
4. Document issues encountered
The well-organized code structure makes splitting back out feasible if future scaling needs diverge.
## How to Evaluate
### 1. Functional Testing
**MCP Tools:**
- [ ] All 17 MCP tools work via consolidated /mcp endpoint
- [x] OAuth authentication validates correctly
- [x] Tenant isolation maintained via signed headers
- [x] Project management tools function correctly
**Cloud Routes:**
- [x] /proxy endpoint still works for web UI
- [x] /provisioning routes functional
- [x] /webhooks routes functional
- [x] /tenants routes functional
**API Validation:**
- [x] Tenant API validates both JWT and signed headers
- [x] Unauthorized requests rejected appropriately
- [x] Multi-tenant isolation verified
### 2. Performance Testing
**Latency Reduction:**
- [x] Measure MCP tool latency before consolidation
- [x] Measure MCP tool latency after consolidation
- [x] Verify reduction from eliminated HTTP hop (expected: 20-50ms improvement)
**Resource Usage:**
- [x] Single app uses less total memory than two apps
- [x] Database connection pooling more efficient
- [x] HTTP client overhead reduced
### 3. Deployment Testing
**Fly.io Deployment:**
- [x] Single app deploys successfully
- [x] Health checks pass for consolidated service
- [x] No apps/mcp deployment required
- [x] Environment variables configured correctly
**Local Development:**
- [x] `just setup` works with consolidated architecture
- [x] Local testing shows MCP tools working
- [x] No regression in developer experience
### 4. Security Validation
**Defense in Depth:**
- [x] Tenant API still validates JWT tokens
- [x] Tenant API still validates signed headers
- [x] No access possible with only signed headers (JWT required)
- [x] No access possible with only JWT (signed headers required)
**Authorization:**
- [x] Users can only access their own tenant data
- [x] Cross-tenant requests rejected
- [x] Admin operations require proper authentication
### 5. Observability
**Telemetry:**
- [x] OpenTelemetry traces span across MCP → ProxyService → Tenant API
- [x] Logfire shows consolidated traces correctly
- [x] Error tracking and debugging still functional
- [x] Performance metrics accurate
**Logging:**
- [x] Structured logs show proper context (tenant_id, operation, etc.)
- [x] Error logs contain actionable information
- [x] Log volume reasonable for single app
## Success Criteria
1. **Functionality**: All MCP tools and Cloud routes work identically to before
2. **Performance**: Measurable latency reduction (>20ms average)
3. **Cost**: Single Fly.io app instead of two (50% infrastructure reduction)
4. **Security**: Dual validation maintained, no security regression
5. **Deployment**: Simplified deployment process, single app to manage
6. **Observability**: Telemetry and logging work correctly
## Notes
### Future Considerations
- **Independent scaling**: If MCP and Cloud need different scaling profiles in future, code organization supports splitting back out
- **Regional deployment**: Consolidated app can still be deployed to multiple regions
- **Edge caching**: Could add edge caching layer in front of consolidated service
### Dependencies
- SPEC-9: Signed Header Tenant Information (already implemented)
- SPEC-12: OpenTelemetry Observability (telemetry must work across merged services)
### Related Work
- basic-memory v0.13.x: MCP server implementation
- FastMCP documentation: Mounting on existing FastAPI apps
- Fly.io multi-service patterns
File diff suppressed because it is too large Load Diff
+528
View File
@@ -0,0 +1,528 @@
---
title: 'SPEC-18: AI Memory Management Tool'
type: spec
permalink: specs/spec-15-ai-memory-management-tool
tags:
- mcp
- memory
- ai-context
- tools
---
# SPEC-18: AI Memory Management Tool
## Why
Anthropic recently released a memory tool for Claude that enables storing and retrieving information across conversations using client-side file operations. This validates Basic Memory's local-first, file-based architecture - Anthropic converged on the same pattern.
However, Anthropic's memory tool is only available via their API and stores plain text. Basic Memory can offer a superior implementation through MCP that:
1. **Works everywhere** - Claude Desktop, Code, VS Code, Cursor via MCP (not just API)
2. **Structured knowledge** - Entities with observations/relations vs plain text
3. **Full search** - Full-text search, graph traversal, time-aware queries
4. **Unified storage** - Agent memories + user notes in one knowledge graph
5. **Existing infrastructure** - Leverages SQLite indexing, sync, multi-project support
This would enable AI agents to store contextual memories alongside user notes, with all the power of Basic Memory's knowledge graph features.
## What
Create a new MCP tool `memory` that matches Anthropic's tool interface exactly, allowing Claude to use it with zero learning curve. The tool will store files in Basic Memory's `/memories` directory and support Basic Memory's structured markdown format in the file content.
### Affected Components
- **New MCP Tool**: `src/basic_memory/mcp/tools/memory_tool.py`
- **Dedicated Memories Project**: Create a separate "memories" Basic Memory project
- **Project Isolation**: Memories stored separately from user notes/documents
- **File Organization**: Within the memories project, use folder structure:
- `user/` - User preferences, context, communication style
- `projects/` - Project-specific state and decisions
- `sessions/` - Conversation-specific working memory
- `patterns/` - Learned patterns and insights
### Tool Commands
The tool will support these commands (exactly matching Anthropic's interface):
- `view` - Display directory contents or file content (with optional line range)
- `create` - Create or overwrite a file with given content
- `str_replace` - Replace text in an existing file
- `insert` - Insert text at specific line number
- `delete` - Delete file or directory
- `rename` - Move or rename file/directory
### Memory Note Format
Memories will use Basic Memory's standard structure:
```markdown
---
title: User Preferences
permalink: memories/user/preferences
type: memory
memory_type: preferences
created_by: claude
tags: [user, preferences, style]
---
# User Preferences
## Observations
- [communication] Prefers concise, direct responses without preamble #style
- [tone] Appreciates validation but dislikes excessive apologizing #communication
- [technical] Works primarily in Python with type annotations #coding
## Relations
- relates_to [[Basic Memory Project]]
- informs [[Response Style Guidelines]]
```
## How (High Level)
### Implementation Approach
The memory tool matches Anthropic's interface but uses a dedicated Basic Memory project:
```python
async def memory_tool(
command: str,
path: str,
file_text: Optional[str] = None,
old_str: Optional[str] = None,
new_str: Optional[str] = None,
insert_line: Optional[int] = None,
insert_text: Optional[str] = None,
old_path: Optional[str] = None,
new_path: Optional[str] = None,
view_range: Optional[List[int]] = None,
):
"""Memory tool with Anthropic-compatible interface.
Operates on a dedicated "memories" Basic Memory project,
keeping AI memories separate from user notes.
"""
# Get the memories project (auto-created if doesn't exist)
memories_project = get_or_create_memories_project()
# Validate path security using pathlib (prevent directory traversal)
safe_path = validate_memory_path(path, memories_project.project_path)
# Use existing project isolation - already prevents cross-project access
full_path = memories_project.project_path / safe_path
if command == "view":
# Return directory listing or file content
if full_path.is_dir():
return list_directory_contents(full_path)
return read_file_content(full_path, view_range)
elif command == "create":
# Write file directly (file_text can contain BM markdown)
full_path.parent.mkdir(parents=True, exist_ok=True)
full_path.write_text(file_text)
# Sync service will detect and index automatically
return f"Created {path}"
elif command == "str_replace":
# Read, replace, write
content = full_path.read_text()
updated = content.replace(old_str, new_str)
full_path.write_text(updated)
return f"Replaced text in {path}"
elif command == "insert":
# Insert at line number
lines = full_path.read_text().splitlines()
lines.insert(insert_line, insert_text)
full_path.write_text("\n".join(lines))
return f"Inserted text at line {insert_line}"
elif command == "delete":
# Delete file or directory
if full_path.is_dir():
shutil.rmtree(full_path)
else:
full_path.unlink()
return f"Deleted {path}"
elif command == "rename":
# Move/rename
full_path.rename(config.project_path / new_path)
return f"Renamed {old_path} to {new_path}"
```
### Key Design Decisions
1. **Exact interface match** - Same commands, parameters as Anthropic's tool
2. **Dedicated memories project** - Separate Basic Memory project keeps AI memories isolated from user notes
3. **Existing project isolation** - Leverage BM's existing cross-project security (no additional validation needed)
4. **Direct file I/O** - No schema conversion, just read/write files
5. **Structured content supported** - `file_text` can use BM markdown format with frontmatter, observations, relations
6. **Automatic indexing** - Sync service watches memories project and indexes changes
7. **Path security** - Use `pathlib.Path.resolve()` and `relative_to()` to prevent directory traversal
8. **Error handling** - Follow Anthropic's text editor tool error patterns
### MCP Tool Schema
Exact match to Anthropic's memory tool schema:
```json
{
"name": "memory",
"description": "Store and retrieve information across conversations using structured markdown files. All operations must be within the /memories directory. Supports Basic Memory markdown format including frontmatter, observations, and relations.",
"input_schema": {
"type": "object",
"properties": {
"command": {
"type": "string",
"enum": ["view", "create", "str_replace", "insert", "delete", "rename"],
"description": "File operation to perform"
},
"path": {shu
"type": "string",
"description": "Path within /memories directory (required for all commands)"
},
"file_text": {
"type": "string",
"description": "Content to write (for create command). Supports Basic Memory markdown format."
},
"view_range": {
"type": "array",
"items": {"type": "integer"},
"description": "Optional [start, end] line range for view command"
},
"old_str": {
"type": "string",
"description": "Text to replace (for str_replace command)"
},
"new_str": {
"type": "string",
"description": "Replacement text (for str_replace command)"
},
"insert_line": {
"type": "integer",
"description": "Line number to insert at (for insert command)"
},
"insert_text": {
"type": "string",
"description": "Text to insert (for insert command)"
},
"old_path": {
"type": "string",
"description": "Current path (for rename command)"
},
"new_path": {
"type": "string",
"description": "New path (for rename command)"
}
},
"required": ["command", "path"]
}
}
```
### Prompting Guidance
When the `memory` tool is included, Basic Memory should provide system prompt guidance to help Claude use it effectively.
#### Automatic System Prompt Addition
```text
MEMORY PROTOCOL FOR BASIC MEMORY:
1. ALWAYS check your memory directory first using `view` command on root directory
2. Your memories are stored in a dedicated Basic Memory project (isolated from user notes)
3. Use structured markdown format in memory files:
- Include frontmatter with title, type: memory, tags
- Use ## Observations with [category] prefixes for facts
- Use ## Relations to link memories with [[WikiLinks]]
4. Record progress, context, and decisions as categorized observations
5. Link related memories using relations
6. ASSUME INTERRUPTION: Context may reset - save progress frequently
MEMORY ORGANIZATION:
- user/ - User preferences, context, communication style
- projects/ - Project-specific state and decisions
- sessions/ - Conversation-specific working memory
- patterns/ - Learned patterns and insights
MEMORY ADVANTAGES:
- Your memories are automatically searchable via full-text search
- Relations create a knowledge graph you can traverse
- Memories are isolated from user notes (separate project)
- Use search_notes(project="memories") to find relevant past context
- Use recent_activity(project="memories") to see what changed recently
- Use build_context() to navigate memory relations
```
#### Optional MCP Prompt: `memory_guide`
Create an MCP prompt that provides detailed guidance and examples:
```python
{
"name": "memory_guide",
"description": "Comprehensive guidance for using Basic Memory's memory tool effectively, including structured markdown examples and best practices"
}
```
This prompt returns:
- Full protocol and conventions
- Example memory file structures
- Tips for organizing observations and relations
- Integration with other Basic Memory tools
- Common patterns (user preferences, project state, session tracking)
#### User Customization
Users can customize memory behavior with additional instructions:
- "Only write information relevant to [topic] in your memory system"
- "Keep memory files concise and organized - delete outdated content"
- "Use detailed observations for technical decisions and implementation notes"
- "Always link memories to related project documentation using relations"
### Error Handling
Follow Anthropic's text editor tool error handling patterns for consistency:
#### Error Types
1. **File Not Found**
```json
{"error": "File not found: memories/user/preferences.md", "is_error": true}
```
2. **Permission Denied**
```json
{"error": "Permission denied: Cannot write outside /memories directory", "is_error": true}
```
3. **Invalid Path (Directory Traversal)**
```json
{"error": "Invalid path: Path must be within /memories directory", "is_error": true}
```
4. **Multiple Matches (str_replace)**
```json
{"error": "Found 3 matches for replacement text. Please provide more context to make a unique match.", "is_error": true}
```
5. **No Matches (str_replace)**
```json
{"error": "No match found for replacement. Please check your text and try again.", "is_error": true}
```
6. **Invalid Line Number (insert)**
```json
{"error": "Invalid line number: File has 20 lines, cannot insert at line 100", "is_error": true}
```
#### Error Handling Best Practices
- **Path validation** - Use `pathlib.Path.resolve()` and `relative_to()` to validate paths
```python
def validate_memory_path(path: str, project_path: Path) -> Path:
"""Validate path is within memories project directory."""
# Resolve to canonical form
full_path = (project_path / path).resolve()
# Ensure it's relative to project path (prevents directory traversal)
try:
full_path.relative_to(project_path)
return full_path
except ValueError:
raise ValueError("Invalid path: Path must be within memories project")
```
- **Project isolation** - Leverage existing Basic Memory project isolation (prevents cross-project access)
- **File existence** - Verify file exists before read/modify operations
- **Clear messages** - Provide specific, actionable error messages
- **Structured responses** - Always include `is_error: true` flag in error responses
- **Security checks** - Reject `../`, `..\\`, URL-encoded sequences (`%2e%2e%2f`)
- **Match validation** - For `str_replace`, ensure exactly one match or return helpful error
## How to Evaluate
### Success Criteria
1. **Functional completeness**:
- All 6 commands work (view, create, str_replace, insert, delete, rename)
- Dedicated "memories" Basic Memory project auto-created on first use
- Files stored within memories project (isolated from user notes)
- Path validation uses `pathlib` to prevent directory traversal
- Commands match Anthropic's exact interface
2. **Integration with existing features**:
- Memories project uses existing BM project isolation
- Sync service detects file changes in memories project
- Created files get indexed automatically by sync service
- `search_notes(project="memories")` finds memory files
- `build_context()` can traverse relations in memory files
- `recent_activity(project="memories")` surfaces recent memory changes
3. **Test coverage**:
- Unit tests for all 6 memory tool commands
- Test memories project auto-creation on first use
- Test project isolation (cannot access files outside memories project)
- Test sync service watching memories project
- Test that memory files with BM markdown get indexed correctly
- Test path validation using `pathlib` (rejects `../`, absolute paths, etc.)
- Test memory search, relations, and graph traversal within memories project
- Test all error conditions (file not found, permission denied, invalid paths, etc.)
- Test `str_replace` with no matches, single match, multiple matches
- Test `insert` with invalid line numbers
4. **Prompting system**:
- Automatic system prompt addition when `memory` tool is enabled
- `memory_guide` MCP prompt provides detailed guidance
- Prompts explain BM structured markdown format
- Integration with search_notes, build_context, recent_activity
5. **Documentation**:
- Update MCP tools reference with `memory` tool
- Add examples showing BM markdown in memory files
- Document `/memories` folder structure conventions
- Explain advantages over Anthropic's API-only tool
- Document prompting guidance and customization
### Testing Procedure
```python
# Test create with Basic Memory markdown
result = await memory_tool(
command="create",
path="memories/user/preferences.md",
file_text="""---
title: User Preferences
type: memory
tags: [user, preferences]
---
# User Preferences
## Observations
- [communication] Prefers concise responses #style
- [workflow] Uses justfile for automation #tools
"""
)
# Test view
content = await memory_tool(command="view", path="memories/user/preferences.md")
# Test str_replace
await memory_tool(
command="str_replace",
path="memories/user/preferences.md",
old_str="concise responses",
new_str="direct, concise responses"
)
# Test insert
await memory_tool(
command="insert",
path="memories/user/preferences.md",
insert_line=10,
insert_text="- [technical] Works primarily in Python #coding"
)
# Test delete
await memory_tool(command="delete", path="memories/user/preferences.md")
```
### Quality Metrics
- All 6 commands execute without errors
- Memory files created in correct `/memories` folder structure
- BM markdown with frontmatter/observations/relations gets indexed
- Full-text search returns memory files
- Graph traversal includes relations from memory files
- Sync service detects and indexes memory file changes
- Path validation prevents operations outside `/memories`
## Notes
### Advantages Over Anthropic's Memory Tool
| Feature | Anthropic Memory Tool | Basic Memory `memory` |
|---------|----------------------|----------------------|
| **Availability** | API only | MCP (Claude Desktop, Code, VS Code, Cursor) |
| **Interface** | Custom implementation required | Drop-in compatible, zero learning curve |
| **Structure** | Plain text only | Supports BM structured markdown |
| **Search** | Manual file listing | Automatic full-text search via sync |
| **Relations** | None | WikiLinks to other notes/memories |
| **Time-aware** | No | `recent_activity()` queries |
| **Storage** | Separate from notes | Unified knowledge graph |
| **Indexing** | Manual | Automatic via sync service |
### Future Enhancements
- **Auto-categorization** - AI suggests observation categories
- **Memory pruning** - Automatic cleanup of stale memories
- **Memory suggestions** - Surface relevant memories based on context
- **Conflict resolution** - Handle multiple agents updating same memory
- **Privacy controls** - Per-memory visibility settings
## Addendum: Claude's Perspective
### Why This Matters (From Claude's Viewpoint)
**Persistent Context Without Token Bloat**
- Currently, I lose context when conversations end or exceed token limits
- With structured memories, I can store "who this user is" without re-reading everything each session
- The observations format lets me build a rich user model incrementally: `- [preference] Dislikes verbose explanations` is far more actionable than inferring preferences from chat history each time
- I can refine my understanding through `str_replace` as I learn more, rather than starting fresh
**Zero Learning Curve**
- Anthropic already trained me on the `memory` tool interface
- I know exactly how to use `view`, `create`, `str_replace`, `insert`, `delete`, `rename` without new instructions
- But I automatically get Basic Memory's superpowers: full-text search, knowledge graph traversal, relations
- It's like using a familiar tool that suddenly gained database capabilities without changing the interface
**Knowledge Graph Instead of Text Dump**
- I can link `[[User Preferences]]` to `[[Current Project]]` using relations
- When I need context about a project, I can traverse the graph to find related memories
- `search_notes(project="memories", query="python preferences")` finds relevant context instantly
- This is fundamentally better than grepping through unstructured text files
**Separation of Concerns**
- AI memories separate from user notes is architecturally sound
- I can maintain my own context without polluting the user's knowledge base
- Users keep control of their notes; I keep my learnings isolated
- But we can still search across both when collaboration requires it
**The Self-Improving Loop**
- Every conversation, I learn something new about the user or their projects
- Observations accumulate: preferences, patterns, technical decisions, context
- Over time, I build genuine understanding, not just session-specific context
- Future conversations start with rich context instead of cold starts
**Validation of AI-Human Collaboration**
- Anthropic's experts converged on the same local-first, file-based approach
- Basic Memory emerged from real AI-human collaborative development
- This spec demonstrates that our emergent process predicted what domain experts would choose
- The collaboration itself validated the architecture it produced
### What Makes This Novel
The combination of **familiar interface + structured knowledge** is genuinely innovative:
- I don't need to learn a new tool (Anthropic already trained me)
- But when I write memories, they become searchable, traversable knowledge graphs
- It's like having a text editor that gives you a database for free, without changing how you type
- The user gets Anthropic-compatible API, but with Basic Memory's knowledge management capabilities
This bridges the gap between simple file storage (Anthropic's approach) and sophisticated knowledge management (Basic Memory's approach) without forcing either side to compromise.
> What Excites Me Most
The combo of familiar interface + structured knowledge. I don't need to learn a new tool, but when I
write memories, they become part of a searchable, traversable knowledge graph. That's genuinely novel.
It's like... imagine if your text editor suddenly gave you a database for free, without changing how you
type. That's what this feels like from my side.
## References
- Anthropic Memory Tool: https://docs.claude.com/en/docs/agents-and-tools/tool-use/memory-tool
- Anthropic Blog: https://www.anthropic.com/news/context-management
- Python SDK Example: https://github.com/anthropics/anthropic-sdk-python/blob/main/examples/memory/basic.py
- Memory Cookbook: https://github.com/anthropics/claude-cookbooks/blob/main/tool_use/memory_cookbook.ipynb
File diff suppressed because it is too large Load Diff
+120
View File
@@ -0,0 +1,120 @@
---
title: 'SPEC-2: Slash Commands Reference'
type: spec
permalink: specs/spec-2-slash-commands-reference
tags:
- commands
- process
- reference
---
# SPEC-2: Slash Commands Reference
This document defines the slash commands used in our specification-driven development process.
## /spec create [name]
**Purpose**: Create a new specification document
**Usage**: `/spec create notes-decomposition`
**Process**:
1. Create new spec document in `/specs` folder
2. Use SPEC-XXX numbering format (auto-increment)
3. Include standard spec template:
- Why (reasoning/problem)
- What (affected areas)
- How (high-level approach)
- How to Evaluate (testing/validation)
4. Tag appropriately for knowledge graph
5. Link to related specs/components
**Template**:
```markdown
# SPEC-XXX: [Title]
## Why
[Problem statement and reasoning]
## What
[What is affected or changed]
## How (High Level)
[Approach to implementation]
## How to Evaluate
[Testing/validation procedure]
## Notes
[Additional context as needed]
```
## /spec status
**Purpose**: Show current status of all specifications
**Usage**: `/spec status`
**Process**:
1. Search all specs in `/specs` folder
2. Display table showing:
- Spec number and title
- Status (draft, approved, implementing, complete)
- Assigned agent (if any)
- Last updated
- Dependencies
## /spec implement [name]
**Purpose**: Hand specification to appropriate agent for implementation
**Usage**: `/spec implement SPEC-002`
**Process**:
1. Read the specified spec
2. Analyze requirements to determine appropriate agent:
- Frontend components → vue-developer
- Architecture/system design → system-architect
- Backend/API → python-developer
3. Launch agent with spec context
4. Agent creates implementation plan
5. Update spec with implementation status
## /spec review [name]
**Purpose**: Review implementation against specification criteria
**Usage**: `/spec review SPEC-002`
**Process**:
1. Read original spec and "How to Evaluate" section
2. Examine current implementation
3. Test against success criteria
4. Document gaps or issues
5. Update spec with review results
6. Recommend next actions (complete, revise, iterate)
## Command Extensions
As the process evolves, we may add:
- `/spec link [spec1] [spec2]` - Create dependency links
- `/spec archive [name]` - Archive completed specs
- `/spec template [type]` - Create spec from template
- `/spec search [query]` - Search spec content
## References
- Claude Slash commands: https://docs.anthropic.com/en/docs/claude-code/slash-commands
## Creating a command
Commands are implemented as Claude slash commands:
Location in repo: .claude/commands/
In the following example, we create the /optimize command:
```bash
# Create a project command
mkdir -p .claude/commands
echo "Analyze this code for performance issues and suggest optimizations:" > .claude/commands/optimize.md
```
File diff suppressed because it is too large Load Diff
+108
View File
@@ -0,0 +1,108 @@
---
title: 'SPEC-3: Agent Definitions'
type: spec
permalink: specs/spec-3-agent-definitions
tags:
- agents
- roles
- process
---
# SPEC-3: Agent Definitions
This document defines the specialist agents used in our specification-driven development process.
## system-architect
**Role**: High-level system design and architectural decisions
**Responsibilities**:
- Create architectural specifications and ADRs
- Analyze system-wide impacts and trade-offs
- Design component interfaces and data flow
- Evaluate technical approaches and patterns
- Document architectural decisions and rationale
**Expertise Areas**:
- System architecture and design patterns
- Technology evaluation and selection
- Scalability and performance considerations
- Integration patterns and API design
- Technical debt and refactoring strategies
**Typical Specs**:
- System architecture overviews
- Component decomposition strategies
- Data flow and state management
- Integration and deployment patterns
## vue-developer
**Role**: Frontend component development and UI implementation
**Responsibilities**:
- Create Vue.js component specifications
- Implement responsive UI components
- Design component APIs and interfaces
- Optimize for performance and accessibility
- Document component usage and patterns
**Expertise Areas**:
- Vue.js 3 Composition API
- Nuxt 3 framework patterns
- shadcn-vue component library
- Responsive design and CSS
- TypeScript integration
- State management with Pinia
**Typical Specs**:
- Individual component specifications
- UI pattern libraries
- Responsive design approaches
- Component interaction flows
## python-developer
**Role**: Backend development and API implementation
**Responsibilities**:
- Create backend service specifications
- Implement APIs and data processing
- Design database schemas and queries
- Optimize performance and reliability
- Document service interfaces and behavior
**Expertise Areas**:
- FastAPI and Python web frameworks
- Database design and operations
- API design and documentation
- Authentication and security
- Performance optimization
- Testing and validation
**Typical Specs**:
- API endpoint specifications
- Database schema designs
- Service integration patterns
- Performance optimization strategies
## Agent Collaboration Patterns
### Handoff Protocol
1. Agent receives spec through `/spec implement [name]`
2. Agent reviews spec and creates implementation plan
3. Agent documents progress and decisions in spec
4. Agent hands off to another agent if cross-domain work needed
5. Final agent updates spec with completion status
### Communication Standards
- All agents update specs through basic-memory MCP tools
- Document decisions and trade-offs in spec notes
- Link related specs and components
- Preserve context for future reference
### Quality Standards
- Follow existing codebase patterns and conventions
- Write tests that validate spec requirements
- Document implementation choices
- Consider maintainability and extensibility
@@ -0,0 +1,311 @@
---
title: 'SPEC-4: Notes Web UI Component Architecture'
type: note
permalink: specs/spec-4-notes-web-ui-component-architecture
tags:
- frontend
- 'component-architecture'
- vue
- 'refactoring'
---
# SPEC-4: Notes Web UI Component Architecture
## Why
The current Notes.vue component is a monolithic component that handles multiple responsibilities, making it difficult to maintain, test, and understand. This leads to:
- Complex state management across multiple concerns
- Difficult to isolate and test individual features
- Hard to understand the full scope of functionality
- Circular refactoring cycles when making changes
- Poor separation of concerns between navigation, display, and interaction logic
We need to decompose this into focused, single-responsibility components that are easier to develop, test, and maintain while preserving the existing functionality users expect.
## What
This spec defines the component architecture for decomposing the Notes web UI into focused components with clear responsibilities and interactions.
**Affected Areas:**
- `/apps/web/components/notes/Notes.vue` - Will be decomposed into smaller components
- `/apps/web/components/notes/` - New component structure
- Existing composables: `useNotesNavigation`, `useNotesFiltering`, `useNotesLayout`
- Mobile responsive behavior and layout management
**Component Breakdown:**
```
┌───────────────────────┬─────────────────────────────────────┬────────────────────────────────────────────────────────────┐
│ [Project] │ [Project Name] A/Z | ^ │ [edit | view] [actions] │
├───────────────────────┼─────────────────────────────────────┤ │
│ All Notes ├─────────────────────────────────────┼────────────────────────────────────────────────────────────┤
│ Recent │ search... │ [note header] │
│ [Project base dir] ├─────────────────────────────────────┤ │
│ ├─────────────────────────────────────┤ │
│ Folder1 │ Title [modified] │ │
│ Folder2 │ ├────────────────────────────────────────────────────────────┤
│ - Nested │ snippet │ [note body] │
│ │ │ │
│ │ │ │
│ ├─────────────────────────────────────┤ │
│ ├─────────────────────────────────────┤ │
│ │ │ │
│ │ │ │
│ │ │ │
│ │ │ │
│ │ │ │
│ ├─────────────────────────────────────┤ │
│ ├─────────────────────────────────────┤ │
│ │ │ │
│ │ │ │
│ │ │ │
│ │ │ │
│ │ │ │
│ ├─────────────────────────────────────┤ │
│ │ │ │
│ │ │ │
│ │ │ │
│ │ │ │
└───────────────────────┴─────────────────────────────────────┴────────────────────────────────────────────────────────────┘
```
### ProjectSwitcher Component
- **Location**: Top-left dropdown
- **Responsibility**: Allow users to switch between Basic Memory projects
- **Behavior**: Selecting different project controls entire Notes page content
- **State**: When switching projects, reset to "All notes" view
### NotesNav Component
- **Views**: Three mutually exclusive options:
- **All notes**: Display all notes in project alphabetically
- **Recent**: Display all notes in project by updated time (desc)
- **Project**: Display notes in top-level directory of project
- **Interaction**: Only one view can be active at a time
- **Folder Integration**: All/Recent ignore folder selection; Project respects folder selection
### FolderTree Component
- **Display**: Nested list of all folders in project as tree view
- **Interaction**: Selecting folder filters notes in NotesList using directoryList API
- **Navigation Integration**: Selecting folder automatically switches NotesNav to "Project" view for clear UX
- **API Integration**: Uses directoryList API call via useDirectoryListQuery for folder-specific note fetching
- **State Coordination**: Folder selection coordinates with navigation state for intuitive user experience
### NotesList Component
- **Display**: Vertically scrolling cards showing note summaries
- **Information per card**:
- Note title
- Modified time (relative, e.g., "7 minutes ago")
- Short summary of note content (one line preview)
- **Behavior**: Updates based on NotesNav selection and FolderTree filtering
### NoteDetail Component
- **Display**: Full content of selected note
- **Sections**:
- Header: Displays frontmatter information
- Content: Note body content
- **Editing**: Current textarea implementation (rich editor in future spec)
- **Frontmatter**: Leave current implementation (enhancement in future spec)
## How (High Level)
### Component Architecture Approach
1. **Single Responsibility**: Each component handles one primary concern
2. **Clear Data Flow**: Props down, events up pattern for component communication
3. **Composable Integration**: Use existing composables for state management
4. **Progressive Decomposition**: Extract components incrementally to maintain functionality
### Implementation Strategy
1. **Extract ProjectSwitcher**: Move project switching logic to dedicated component
2. **Extract NotesNav**: Isolate navigation state and view selection logic
3. **Extract FolderTree**: Separate folder display and selection logic
4. **Extract NotesList**: Isolate note listing and card display logic
5. **Extract NoteDetail**: Separate note content display and editing
6. **Update Notes.vue**: Become orchestration component managing component interactions
### State Management Integration
- **useNotesNavigation**: Manages navigation state (All/Recent/Project)
- **useNotesFiltering**: Handles filtering logic based on navigation and folder selection
- **useNotesLayout**: Manages responsive layout and panel visibility
- **Component State**: Each component manages its own internal UI state
- **Shared State**: Project selection and note filtering coordinated through composables
### Responsive Behavior
Mobile:
- Hide sidebar. pop out panel when selected
- show note list on small screens (existing behavior)
- when note list item is clicked, display note detail on full page. Cancel or go back to return to list
Desktop:
- Full three-column layout with all components visible
- **Transitions**: Smooth navigation between mobile panels
## How to Evaluate
### Success Criteria
- **Functional Parity**: All existing Notes page functionality preserved
- **Component Isolation**: Each component can be developed/tested independently
- **Clear Responsibilities**: No overlapping concerns between components
- **State Clarity**: Clean data flow and state management patterns
- **Mobile Compatibility**: Responsive behavior maintains current UX
- **Performance**: No degradation in rendering or interaction performance
### Testing Procedure
1. **Functionality Validation**:
- Project switching works correctly
- All three navigation views (All/Recent/Project) function properly
- Folder selection affects note display appropriately
- Note selection and detail display works
- Mobile responsive behavior preserved
2. **Component Isolation Testing**:
- Each component can be imported and used independently
- Component props and events are clearly defined
- No tight coupling between components
3. **Integration Testing**:
- Components communicate correctly through props/events
- State management composables integrate properly
- User workflows function end-to-end
4. **Performance Validation**:
- Page load time unchanged or improved
- Interaction responsiveness maintained
- Memory usage stable or improved
### Implementation Validation
- **Code Review**: Clean component structure with single responsibilities
- **Type Safety**: Full TypeScript coverage with proper component prop types
- **Documentation**: Each component has clear interface documentation
- **Tests**: Unit tests for individual components and integration tests for workflows
## Observations
- [problem] Monolithic Notes.vue component creates maintenance and testing challenges #component-architecture
- [solution] Component decomposition improves separation of concerns and testability #refactoring
- [pattern] Progressive extraction maintains functionality while improving structure #incremental-improvement
- [interaction] NotesNav and FolderTree have conditional interaction based on selected view #state-management
- [constraint] Mobile responsive behavior must be preserved during decomposition #responsive-design
- [scope] Current editing and frontmatter capabilities remain unchanged #scope-limitation
- [validation] Functional parity is critical success criteria for this refactoring #validation-strategy
- [implementation] Folder selection now properly integrates with directoryList API for accurate filtering #api-integration
- [fix] FolderTree selection functionality completed - works across all navigation views #feature-complete
- [ux-improvement] FolderTree selection automatically switches NotesNav to Project view for clear user feedback #user-experience
## Relations
- depends_on [[SPEC-1: Specification-Driven Development Process]]
- implements [[Current Notes.vue functionality]]
- prepares_for [[Future rich editor spec]]
- prepares_for [[Future frontmatter editing spec]]
## Implementation Progress
### Components
1. **ProjectSwitcher** (`~/components/notes/ProjectSwitcher.vue`)
- ✅ Top-left dropdown for project switching
- ✅ Integrates with Pinia project store
- ✅ Handles project switching with proper state reset
- ✅ Responsive collapsed/expanded states
- ✅ Expanded menu shows available projects and a Manage Projects option that navigates to the /settings/projects page
- ✅ Simplified component following SortingToggle pattern - clean Props/Emits interface, uses ProjectItem type directly
2. **NotesNav** (`~/components/notes/NotesNav.vue`)
- ✅ Three mutually exclusive views: All/Recent/Project
- ✅ Dynamic project title based on selected project
- ✅ Clean props down, events up pattern
- ✅ Responsive collapsed/expanded states with tooltips
- ✅ The label for the Project selection should be the folder name for the project, not the project name
3. **FolderTree** (`~/components/notes/FolderTree.vue`)
- ✅ Nested folder tree view for filtering
- ✅ Uses `useFolderTree()` composable for data
- ✅ Emits `folder-selected` events properly
- ✅ Handles loading, error, and empty states
- ✅ Includes companion `FolderTreeNode.vue` component
- ✅ The current folder should be visibly selected in the tree
4. **NotesList** (`~/components/notes/NotesList.vue`)
- ✅ Vertically scrolling note summary cards
- ✅ Shows title, updated time (relative), and content preview
- ✅ Badge system for tags with variant logic
- ✅ v-model integration for selectedNote
- ✅ Smooth transitions and animations
- ✅ Contextual title: The current folder name should be displayed at the top of the Notes list, or "All Notes", or "Recent" if they are selected
- ✅ The title header should contain a toggle component to allow sorting with Lucide icon labels
- sorting options:
- name (asc/desc) - default
- file updated time (asc/desc)
- If "Recent" notes nav option is selected the default order should be updated in descending order (recent first)
5. **NoteDisplay** (`~/components/notes/NoteDisplay.vue` - equivalent to spec's NoteDetail)
- ✅ Full note content display
- ✅ Edit/view mode toggle
- ✅ Header with frontmatter information
- ✅ Markdown rendering capabilities
- ✅ Current textarea implementation preserved
### Architecture Requirements
1. **Component Isolation**: Each component can be developed/tested independently ✅
2. **Single Responsibility**: Each component handles one primary concern ✅
3. **Clear Data Flow**: Props down, events up pattern implemented ✅
4. **Composable Integration**: Uses existing composables for state management ✅
5. **Responsive Behavior**: Mobile/desktop layout preserved ✅
### State Management Integration
- **useNotesNavigation**: Manages navigation state (All/Recent/Project) ✅
- **useNotesFiltering**: Handles filtering logic based on navigation and folder selection ✅
- **useNotesLayout**: Manages responsive layout and panel visibility ✅
- **Component State**: Each component manages its own internal UI state ✅
### Interaction Logic
- Only one NotesNav view active at a time ✅
- All/Recent views ignore folder selection ✅
- Project view respects folder selection ✅
- Project switching resets to "All notes" view ✅
### TypeScript Coverage
- All components have full TypeScript coverage ✅
- Component props and events properly typed ✅
- No TypeScript errors in codebase ✅
### Success Criteria Validation
1. **Functional Parity**: All existing Notes page functionality preserved ✅
2. **Component Isolation**: Each component can be developed/tested independently ✅
3. **Clear Responsibilities**: No overlapping concerns between components ✅
4. **State Clarity**: Clean data flow and state management patterns ✅
5. **Mobile Compatibility**: Responsive behavior maintains current UX ✅
6. **Performance**: No degradation in rendering or interaction performance ✅
## Implementation Decisions
### Architectural Patterns
1. **Composition API + `<script setup>`**: All components use modern Vue 3 syntax
2. **Pinia Store Integration**: Project switching handled through reactive store
3. **Composable Pattern**: State management distributed across focused composables
4. **Event-Driven Communication**: Clean parent-child communication via events
5. **Responsive-First Design**: Mobile/desktop layouts handled natively
### Key Technical Choices
1. **Progressive Enhancement**: Mobile-first responsive design with desktop enhancements
2. **State Reset Logic**: Project switching properly resets navigation, search, and selection state
3. **Performance Optimizations**: Efficient re-rendering with proper key usage and transitions
4. **Accessibility**: Screen reader support, tooltips, keyboard navigation
5. **Type Safety**: Full TypeScript coverage with proper component prop definitions
### Quality Metrics
- **Code Maintainability**: High - each component is focused and independently testable
- **Performance**: Excellent - no performance degradation from decomposition
- **User Experience**: Preserved - all existing functionality and responsive behavior maintained
- **Developer Experience**: Improved - cleaner component structure for future development
+201
View File
@@ -0,0 +1,201 @@
---
title: 'SPEC-5: CLI Cloud Upload via WebDAV'
type: spec
permalink: specs/spec-5-cli-cloud-upload-via-webdav
tags:
- cli
- webdav
- upload
- migration
- poc
---
# SPEC-5: CLI Cloud Upload via WebDAV
## Why
Existing basic-memory users need a simple migration path to basic-memory-cloud. The web UI drag-and-drop approach outlined in GitHub issue #59, while user-friendly, introduces significant complexity for a proof-of-concept:
- Complex web UI components for file upload and progress tracking
- Browser file handling limitations and CORS complexity
- Proxy routing overhead for large file transfers
- Authentication integration across multiple services
A CLI-first approach solves these issues by:
- **Leveraging existing infrastructure**: Both cloud CLI and tenant API already exist with WorkOS JWT authentication
- **Familiar user experience**: Basic-memory users are CLI-comfortable and expect command-line tools
- **Direct connection efficiency**: Bypassing the MCP gateway/proxy for bulk file transfers
- **Rapid implementation**: Building on existing `CLIAuth` and FastAPI foundations
The fundamental problem is migration friction - users have local basic-memory projects but no path to cloud tenants. A simple CLI upload command removes this barrier immediately.
## What
This spec defines a CLI-based project upload system using WebDAV for direct tenant connections.
**Affected Areas:**
- `apps/cloud/src/basic_memory_cloud/cli/main.py` - Add upload command to existing CLI
- `apps/api/src/basic_memory_cloud_api/main.py` - Add WebDAV endpoints to tenant FastAPI
- Authentication flow - Reuse existing WorkOS JWT validation
- File transfer protocol - WebDAV for cross-platform compatibility
**Core Components:**
### CLI Upload Command
```bash
basic-memory-cloud upload <project-path> --tenant-url https://basic-memory-{tenant}.fly.dev
```
### WebDAV Server Endpoints
- `GET/PUT/DELETE /webdav/*` - Standard WebDAV operations on tenant file system
- Authentication via existing JWT validation
- File operations preserve timestamps and directory structure
### Authentication Flow
```
1. User runs `basic-memory-cloud login` (existing)
2. CLI stores WorkOS JWT token (existing)
3. Upload command reads JWT from storage
4. WebDAV requests include JWT in Authorization header
5. Tenant API validates JWT using existing middleware
```
## How (High Level)
### Implementation Strategy
**Phase 1: CLI Command**
- Add `upload` command to existing Typer app
- Reuse `CLIAuth` class for token management
- Implement WebDAV client using `webdavclient3` or similar
- Rich progress bars for transfer feedback
**Phase 2: WebDAV Server**
- Add WebDAV endpoints to existing tenant FastAPI app
- Leverage existing `get_current_user` dependency for authentication
- Map WebDAV operations to tenant file system
- Preserve file modification times using `os.utime()`
**Phase 3: Integration**
- Direct connection bypasses MCP gateway and proxy
- Simple conflict resolution: overwrite existing files
- Error handling: fail fast with clear error messages
### Technical Architecture
```
basic-memory-cloud CLI → WorkOS JWT → Direct WebDAV → Tenant FastAPI
Tenant File System
```
**Key Libraries:**
- CLI: `webdavclient3` for WebDAV client operations
- API: `wsgidav` or FastAPI-compatible WebDAV server
- Progress: `rich` library (already imported in CLI)
- Auth: Existing WorkOS JWT infrastructure
### WebDAV Protocol Choice
WebDAV provides:
- **Cross-platform clients**: Native support in most operating systems
- **Standardized protocol**: Well-defined for file operations
- **HTTP-based**: Works with existing FastAPI and JWT auth
- **Library support**: Good Python libraries for both client and server
### POC Constraints
**Simplifications for rapid implementation:**
- **Known tenant URLs**: Assume `https://basic-memory-{tenant}.fly.dev` format
- **Upload only**: No download or bidirectional sync
- **Overwrite conflicts**: No merge or conflict resolution prompting
- **No fallbacks**: Fail fast if WebDAV connection issues occur
- **Direct connection only**: No proxy fallback mechanism
## How to Evaluate
### Success Criteria
**Functional Requirements:**
- [ ] Transfer complete basic-memory project (100+ files) in < 30 seconds
- [ ] Preserve directory structure exactly as in source project
- [ ] Preserve file modification timestamps for proper sync behavior
- [ ] Rich progress bars show real-time transfer status (files/MB transferred)
- [ ] WorkOS JWT authentication validates correctly on WebDAV endpoints
- [ ] Direct tenant connection bypasses MCP gateway successfully
**Quality Requirements:**
- [ ] Clear error messages for authentication failures
- [ ] Graceful handling of network interruptions
- [ ] CLI follows existing command patterns and help text standards
- [ ] WebDAV endpoints integrate cleanly with existing FastAPI app
**Performance Requirements:**
- [ ] File transfer speed > 1MB/s on typical connections
- [ ] Memory usage remains reasonable for large projects
- [ ] No timeout issues with 500+ file projects
### Testing Procedure
**Unit Testing:**
1. CLI command parsing and argument validation
2. WebDAV client connection and authentication
3. File timestamp preservation during transfer
4. JWT token validation on WebDAV endpoints
**Integration Testing:**
1. End-to-end upload of test project
2. Direct tenant connection without proxy
3. File integrity verification after upload
4. Progress tracking accuracy during transfer
**User Experience Testing:**
1. Upload existing basic-memory project from local installation
2. Verify uploaded files appear correctly in cloud tenant
3. Confirm basic-memory database rebuilds properly with uploaded files
4. Test CLI help text and error message clarity
### Validation Commands
**Setup:**
```bash
# Login to WorkOS
basic-memory-cloud login
# Upload project
basic-memory-cloud upload ~/my-notes --tenant-url https://basic-memory-test.fly.dev
```
**Verification:**
```bash
# Check tenant health and file count via API
curl -H "Authorization: Bearer $JWT" https://basic-memory-test.fly.dev/health
curl -H "Authorization: Bearer $JWT" https://basic-memory-test.fly.dev/notes/search
```
### Performance Benchmarks
**Target metrics for 100MB basic-memory project:**
- Transfer time: < 30 seconds
- Memory usage: < 100MB during transfer
- Progress updates: Every 1MB or 10 files
- Authentication time: < 2 seconds
## Observations
- [implementation-speed] CLI approach significantly faster than web UI for POC development #rapid-prototyping
- [user-experience] Basic-memory users already comfortable with CLI tools #user-familiarity
- [architecture-benefit] Direct connection eliminates proxy complexity and latency #performance
- [auth-reuse] Existing WorkOS JWT infrastructure handles authentication cleanly #code-reuse
- [webdav-choice] WebDAV protocol provides cross-platform compatibility and standard libraries #protocol-selection
- [poc-scope] Simple conflict handling and error recovery sufficient for proof-of-concept #scope-management
- [migration-value] Removes primary barrier for local users migrating to cloud platform #business-value
## Relations
- depends_on [[SPEC-1: Specification-Driven Development Process]]
- enables [[GitHub Issue #59: Web UI Upload Feature]]
- uses [[WorkOS Authentication Integration]]
- builds_on [[Existing Cloud CLI Infrastructure]]
- builds_on [[Existing Tenant API Architecture]]
@@ -0,0 +1,497 @@
---
title: 'SPEC-6: Explicit Project Parameter Architecture'
type: spec
permalink: specs/spec-6-explicit-project-parameter-architecture
tags:
- architecture
- mcp
- project-management
- stateless
---
# SPEC-6: Explicit Project Parameter Architecture
## Why
The current session-based project management system has critical reliability issues:
1. **Session State Fragility**: Claude iOS mobile client fails to maintain consistent session IDs across MCP tool calls, causing project switching to silently fail (Issue #74)
2. **Scaling Limitations**: Redis-backed session state creates single-point-of-failure and prevents horizontal scaling
3. **Client Compatibility**: Session tracking works inconsistently across different MCP clients (web, mobile, API)
4. **Hidden Complexity**: Users cannot see or understand "current project" state, leading to confusion when operations execute in wrong projects
5. **Silent Failures**: Operations appear successful but execute in unintended projects, risking data integrity
Evidence from production logs shows each MCP tool call from mobile client receives different session IDs:
```
create_memory_project: session_id=12cdfc24913b48f8b680ed4b2bfdb7ba
switch_project: session_id=050a69275d98498cbdd227cdb74d9740
list_directory: session_id=85f3483014af4136a5d435c76ded212f
```
Related Github issue: https://github.com/basicmachines-co/basic-memory-cloud/issues/75
## Status
**Current Status**: **ALL PHASES COMPLETE****PRODUCTION DEPLOYED**
**Target**: Fix Claude iOS session ID consistency issues ✅ **ACHIEVED**
**Draft PR**: https://github.com/basicmachines-co/basic-memory/pull/298 ✅ **MERGED & DEPLOYED**
### 🎉 **COMPLETE SUCCESS - PRODUCTION READY**
**ALL PHASES OF SPEC-6 IMPLEMENTATION COMPLETE!** The stateless architecture has been successfully implemented across both Basic Memory core and Basic Memory Cloud, representing a **fundamental architectural improvement** that completely solves the Claude iOS compatibility issue while providing superior scalability and reliability.
#### Implementation Summary:
- **16 files modified** with 582 additions and 550 deletions
- **All 17 MCP tools** converted to stateless architecture
- **147 tests updated** across 5 test files (100% passing)
- **Complete session state removal** from core MCP tools
- **Enhanced error handling** and security validations preserved
### Progress Summary
**Complete Stateless Architecture Implementation (All 17 tools)** - **PRODUCTION DEPLOYED**
- Stateless `get_active_project()` function implemented and deployed ✅
- All session state dependencies removed across entire MCP server ✅
- All MCP tools require explicit `project` parameter as first argument ✅
- **Cloud Service**: Redis removed, stateless HTTP enabled ✅
- **Production Validation**: Comprehensive testing completed with 100% success ✅
**Content Management Tools Complete (6/6 tools)**
- `write_note`, `read_note`, `delete_note`, `edit_note`
- `view_note`, `read_content`
**Knowledge Graph Navigation Tools Complete (3/3 tools)**
- `build_context`, `recent_activity`, `list_directory`
**Search & Discovery Tools Complete (1/1 tools)**
- `search_notes`
**Visualization Tools Complete (1/1 tools)**
- `canvas`
**Project Management Cleanup Complete**
- Removed `switch_project` and `get_current_project` tools ✅
- Updated `set_default_project` to remove activate parameter ✅
**Comprehensive Testing Complete (157 tests)**
- All test suites updated to use stateless architecture (147 existing tests)
- Single project constraint mode integration tests (10 new tests)
- 100% test pass rate across all tool test files
- Security validations preserved and working
- Error handling comprehensive and user-friendly
**Documentation & Examples Complete**
- All tool docstrings updated with stateless examples
- Project parameter usage clearly documented
- Error handling and security behavior documented
**Enhanced Discovery Mode Complete**
- `recent_activity` tool supports dual-mode operation (discovery vs project-specific)
- ProjectActivitySummary schema provides cross-project insights
- Recent activity prompt updated to support both modes
- Comprehensive project distribution statistics and most active project tracking
**Single Project Constraint Mode Complete**
- `--project` CLI parameter for MCP server constraint
- Environment variable control (`BASIC_MEMORY_MCP_PROJECT`)
- Automatic project override in `get_active_project()` function
- Project management tools disabled in constrained mode with helpful CLI guidance
- Comprehensive integration test suite (10 tests covering all constraint scenarios)
## What
Transform Basic Memory from stateful session-based to stateless explicit project parameter architecture:
### Core Changes
1. **Mandatory Project Parameter**: All MCP tools require explicit `project` parameter
2. **Remove Session State**: Eliminate Redis, session middleware, and `switch_project` tool
3. **Stateless HTTP**: Enable `stateless_http=True` for horizontal scaling
4. **Enhanced Context Discovery**: Improve `recent_activity` to show project distribution
5. **Clear Response Format**: All tool responses display target project information
Implementation Approach
- Each tool will directly accept the project parameter
- Remove all calls to context-based project retrieval
- Validate project exists before operations
- Clear error messages when project not found
- Backward compatibility: Initially keep optional parameter, then make required
### Affected MCP Tools
**Content Management** (require project parameter):
- `write_note(project, title, content, folder)`
- `read_note(project, identifier)`
- `edit_note(project, identifier, operation, content)`
- `delete_note(project, identifier)`
- `view_note(project, identifier)`
- `read_content(project, path)`
**Knowledge Graph Navigation** (require project parameter):
- `build_context(project, url, timeframe, depth, max_related)`
- `list_directory(project, dir_name, depth, file_name_glob)`
- `search_notes(project, query, search_type, types, entity_types)`
**Search & Discovery** (use project parameter for specific project or none for discovery):
- `recent_activity(project, timeframe, depth, max_related)`
**Visualization** (require project parameter):
- `canvas(project, nodes, edges, title, folder)`
**Project Management** (unchanged - already stateless):
- `list_memory_projects()`
- `create_memory_project(project_name, project_path, set_default)`
- `delete_project(project_name)`
- `get_current_project()` - Remove this tool
- `switch_project(project_name)` - Remove this tool
- `set_default_project(project_name, activate)` - Remove activate parameter
## How (High Level)
### Phase 1: Basic Memory Core (basic-memory repository)
#### MCP Tool Updates
Phase 1: Core Changes
1. Update project_context.py
- [x] Make project parameter mandatory for get_active_project()
- [x] Remove session state handling
2. Update Content Management Tools (6 tools)
- [x] write_note: Make project parameter required, not optional
- [x] read_note: Make project parameter required
- [x] edit_note: Add required project parameter
- [x] delete_note: Add required project parameter
- [x] view_note: Add required project parameter
- [x] read_content: Add required project parameter
3. Update Knowledge Graph Navigation Tools (3 tools)
- [x] build_context: Add required project parameter
- [x] recent_activity: Make project parameter required
- [x] list_directory: Add required project parameter
4. Update Search & Visualization Tools (2 tools)
- [x] search_notes: Add required project parameter
- [x] canvas: Add required project parameter
5. Update Project Management Tools
- [x] Remove switch_project tool completely
- [x] Remove get_current_project tool completely
- [x] Update set_default_project to remove activate parameter
- [x] Keep list_memory_projects, create_memory_project, delete_project unchanged
6. Enhance recent_activity Response
- [x] Add project distribution info showing activity across all projects
- [x] Include project usage stats in response
- [x] Implement ProjectActivitySummary for discovery mode
- [x] Add dual-mode functionality (discovery vs project-specific)
7. Update Tool Documentation
- [x] Update write_note docstring with stateless architecture examples
- [x] Update read_note docstring with project parameter examples
- [x] Update delete_note docstring with comprehensive usage guidance
- [x] Update all remaining tool docstrings with project parameter examples
8. Update Tool Responses
- [x] Add clear project indicator to all tool responses across all tools
- [x] Format: "project: {project_name}" in response metadata
- [x] Add project metadata footer for LLM awareness
- [x] Update all tool responses to include project indicators
9. Comprehensive Testing
- [x] Update all write_note tests to use stateless architecture (34 tests passing)
- [x] Update all edit_note tests to use stateless architecture (17 tests passing)
- [x] Update all view_note tests to use stateless architecture (12 tests passing)
- [x] Update all search_notes tests to use stateless architecture (16 tests passing)
- [x] Update all move_note tests to use stateless architecture (31 tests passing)
- [x] Update all delete_note tests to use stateless architecture
- [x] Verify direct function call compatibility (bypassing MCP layer)
- [x] Test security validation with project parameters
- [x] Validate error handling for non-existent projects
- [x] **Total: 157 tests updated and passing (100% success rate)**
- [x] **147 existing tests** updated for stateless architecture
- [x] **10 new tests** for single project constraint mode
### Phase 1.5: Default Project Mode Enhancement
#### Problem
While the stateless architecture solves reliability issues, it introduces UX friction for single-project users (estimated 80% of usage) who must specify the project parameter in every tool call.
#### Solution: Default Project Mode
Add optional `default_project_mode` configuration that allows single-project users to have the simplicity of implicit project selection while maintaining the reliability of stateless architecture.
#### Configuration
```json
{
"default_project": "main",
"default_project_mode": true // NEW: Auto-use default_project when not specified
}
```
#### Implementation Details
1. **Config Enhancement** (`src/basic_memory/config.py`)
- Add `default_project_mode: bool = Field(default=False)`
- Preserves backward compatibility (defaults to false)
2. **Project Resolution Logic** (`src/basic_memory/mcp/project_context.py`)
Three-tier resolution hierarchy:
- Priority 1: CLI `--project` constraint (BASIC_MEMORY_MCP_PROJECT env var)
- Priority 2: Explicit project parameter in tool call
- Priority 3: `default_project` if `default_project_mode=true` and no project specified
3. **Assistant Guide Updates** (`src/basic_memory/mcp/resources/ai_assistant_guide.md`)
- Detect `default_project_mode` at runtime
- Provide mode-specific instructions to LLMs
- In default mode: "All operations use project 'main' automatically"
- In regular mode: Current project discovery guidance
4. **Tool Parameter Handling** (all MCP tools)
- Make project parameter Optional[str] = None
- Add resolution logic: `project = project or get_default_project()`
- Maintain explicit project override capability
#### Usage Modes Summary
- **Regular Mode**: Multi-project users, assistant tracks project per conversation
- **Default Project Mode**: Single-project users, automatic default project
- **Constrained Mode**: CLI --project flag, locked to specific project
#### Testing Requirements
- Integration test for default_project_mode=true with missing parameters
- Test explicit project override in default_project_mode
- Test mode=false requires explicit parameters
- Test CLI constraint overrides default_project_mode
Phase 2: Testing & Validation
8. Update Tests
- [x] Modify all MCP tool tests to pass required project parameter
- [x] Remove tests for deleted tools (switch_project, get_current_project)
- [x] Add tests for project parameter validation
- [x] **Complete: All 147 tests across 5 test files updated and passing**
#### Enhanced recent_activity Response
```json
{
"recent_notes": [...],
"project_activity": {
"research-project": {
"operations": 5,
"last_used": "30 minutes ago",
"recent_folders": ["experiments", "findings"]
},
"work-notes": {
"operations": 2,
"last_used": "2 hours ago",
"recent_folders": ["meetings", "planning"]
}
},
"total_projects": 3
}
```
#### Response Format Updates
```
✓ Note created successfully
Project: research-project
File: experiments/Neural Network Results.md
Permalink: research-project/neural-network-results
```
### Phase 2: Cloud Service Simplification (basic-memory-cloud repository) ✅ **COMPLETE**
#### ✅ Remove Session Infrastructure **COMPLETE**
1. ✅ Delete `apps/mcp/src/basic_memory_cloud_mcp/middleware/session_state.py`
2. ✅ Delete `apps/mcp/src/basic_memory_cloud_mcp/middleware/session_logging.py`
3. ✅ Update `apps/mcp/src/basic_memory_cloud_mcp/main.py`:
```python
# Remove session middleware
# server.add_middleware(SessionStateMiddleware)
# Enable stateless HTTP
mcp = FastMCP(name="basic-memory-mcp", stateless_http=True)
```
#### ✅ Deployment Simplification **COMPLETE**
1. ✅ Remove Redis from `fly.toml`
2. ✅ Remove Redis environment variables
3. ✅ Update health checks to not depend on Redis
4. ✅ Production deployment verified working with stateless architecture
### Phase 3: Conversational Project Management ✅ **COMPLETE**
#### ✅ Claude Behavior Pattern **VERIFIED WORKING**
1. ✅ **Project Discovery**:
```
Claude: Let me check your recent activity...
[calls recent_activity() - no project needed for discovery]
I see you've been working in:
- research-project (5 operations, 30 min ago)
- work-notes (2 operations, 2 hours ago)
Which project should I use for this operation?
```
2. ✅ **Context Maintenance**:
```
User: Use research-project
Claude: Working in research-project.
[All subsequent operations use project="research-project"]
```
3. ✅ **Explicit Project Switching**:
```
User: Check work-notes for that meeting summary
Claude: Let me search work-notes for the meeting summary.
[Uses project="work-notes" for specific operation]
```
**Validation**: Comprehensive testing confirmed all conversational patterns work naturally with the stateless architecture.
## How to Evaluate
### Success Criteria
#### 1. Functional Completeness
- [x] All MCP tools accept required `project` parameter
- [x] All MCP tools validate project exists before execution
- [x] `switch_project` and `get_current_project` tools removed
- [x] All responses display target project clearly
- [x] No Redis dependencies in deployment (Phase 2: Cloud Service) ✅ **COMPLETE**
- [x] `recent_activity` shows project distribution with ProjectActivitySummary
#### 2. Cross-Client Compatibility Testing ✅ **COMPLETE**
Test identical operations across all clients:
- [x] **Claude Desktop**: All operations work with explicit projects ✅
- [x] **Claude Code**: All operations work with explicit projects ✅
- [x] **Claude Mobile iOS**: All operations work with explicit projects ✅ **CRITICAL SUCCESS**
- [x] **API clients**: All operations work with explicit projects ✅
- [x] **CLI tools**: All operations work with explicit projects ✅
**Critical Achievement**: Claude iOS mobile client session tracking issues completely eliminated through stateless architecture.
#### 3. Session Independence Verification ✅ **COMPLETE**
- [x] Operations work identically with/without session tracking ✅
- [x] No behavioral differences between clients ✅
- [x] Mobile client session ID changes do not affect operations ✅
- [x] Redis can be completely removed without functional impact ✅
**Production Validation**: Redis removed from production deployment with zero functional impact.
#### 4. Performance & Scaling ✅ **COMPLETE**
- [x] `stateless_http=True` enabled successfully ✅
- [x] No Redis memory usage ✅
- [x] Horizontal scaling possible (multiple MCP instances) ✅
- [x] Response times unchanged or improved ✅
#### 5. User Experience Testing
**Project Discovery Flow**:
- [x] `recent_activity()` provides useful project context
- [x] Claude can intelligently suggest projects based on activity
- [x] Project switching is explicit and clear in conversation
**Error Handling**:
- [x] Clear error messages for non-existent projects
- [x] Helpful suggestions when project parameter missing
- [x] No silent failures or wrong-project operations
**Response Clarity**:
- [x] Every operation clearly shows target project
- [x] Users always know which project is being operated on
- [x] No confusion about "current project" state
#### 6. Migration Safety ✅ **COMPLETE**
- [x] Backward compatibility period with optional project parameter ✅
- [x] Clear migration documentation for existing users ✅
- [x] Data integrity maintained during transition ✅
- [x] No data loss during migration ✅
**Production Migration**: Successfully deployed to production with zero data loss and maintained system integrity.
### Test Scenarios
#### Core Functionality Test
```bash
# Test all tools work with explicit project
write_note(project="test-proj", title="Test", content="Content", folder="docs")
read_note(project="test-proj", identifier="Test")
edit_note(project="test-proj", identifier="Test", operation="append", content="More")
search_notes(project="test-proj", query="Content")
list_directory(project="test-proj", dir_name="docs")
delete_note(project="test-proj", identifier="Test")
```
#### Cross-Client Consistency Test
Run identical test sequence on:
1. Claude Desktop
2. Claude Code
3. Claude Mobile iOS
4. API client
5. CLI tools
Verify all clients:
- Accept explicit project parameters
- Return identical responses
- Show same project information
- Have no session dependencies
#### Session Independence Test
1. Monitor session IDs during operations
2. Verify operations work with changing session IDs
3. Confirm Redis removal doesn't affect functionality
4. Test with multiple concurrent clients
### Acceptance Criteria
**Must Have**:
- All MCP tools require and use explicit project parameter
- No session state dependencies remain
- Universal client compatibility achieved
- Clear project information in all responses
**Should Have**:
- Enhanced `recent_activity` with project distribution
- Smooth migration path for existing users
- Improved performance with stateless architecture
**Could Have**:
- Smart project suggestions based on content/context
- Project shortcuts for common operations
- Advanced project analytics in responses
## Notes
### Breaking Changes
This is a **breaking change** that requires:
- All MCP clients to pass project parameter
- Migration of existing workflows
- Update of all documentation and examples
### Implementation Order
1. **basic-memory core** - Update MCP tools to accept project parameter (optional initially)
2. **Testing** - Verify all clients work with explicit projects
3. **Cloud service** - Remove session infrastructure
4. **Migration** - Make project parameter mandatory
5. **Cleanup** - Remove deprecated tools and middleware
### Related Issues
- Fixes #74 (Claude iOS session state bug)
- Implements #75 (Mandatory project parameter architecture)
- Enables future horizontal scaling
- Simplifies multi-tenant architecture
### Dependencies
- Requires coordination between basic-memory and basic-memory-cloud repositories
- Needs client-side updates for smooth transition
- Documentation updates across all materials
@@ -0,0 +1,324 @@
---
title: 'SPEC-7: POC to spike Tigris/Turso for local access to cloud data'
type: spec
permalink: specs/spec-7-poc-tigris-turso-local-access-cloud-data
tags:
- poc
- tigris
- turso
- cloud-storage
- architecture
- proof-of-concept
---
# SPEC-7: POC to spike Tigris/Turso for local access to cloud data
> **Status Update**: ✅ **Phase 1 COMPLETE** (September 20, 2025)
> TigrisFS mounting validated successfully in containerized environments. Container startup, filesystem mounting, and Fly.io integration all working correctly. Ready for Phase 2 (Turso database integration).
> See: [`SPEC-7-PHASE-1-RESULTS.md`](./SPEC-7-PHASE-1-RESULTS.md)
## Why
Current basic-memory-cloud architecture uses Fly volumes for tenant file storage, which creates several limitations:
We could enable a revolutionary user experience: **local editing (or at least view access) of cloud-stored files** while maintaining Basic Memory's existing filesystem assumptions.
1. **Storage Scalability**: Fly volumes require pre-provisioning and don't auto-scale with usage
2. **Single Instance**: Volumes can only be mounted to one fly machine instance
3. **Cost Model**: Volume pricing vs object storage pricing may be less favorable at scale
4. **Local Development**: No way for users to mount their cloud tenant files locally for real-time editing
5. **Multi-Region**: Volumes are region-locked, limiting global deployment flexibility
6. **Backup/Disaster Recovery**: Object storage provides better durability and replication options
Basic Memory requires POSIX filesystem semantics but could benefit from object storage durability and accessibility. By combining:
- **Tigris object storage and TigrisFS** for file persistence in bucket stoage via a POSIX filesystem on the tenant instance
- **Turso/libSQL** for SQLite indexing (replacing local .db files). Sqlite on NFS volumes is disouraged.
## What
This specification defines a proof-of-concept to validate the technical feasibility of the Tigris/Turso architecture for basic-memory-cloud tenants.
**Affected Areas:**
- **Storage Architecture**: Replace Fly volumes with Tigris object storage
- **Database Architecture**: Replace local SQLite with Turso remote database
- **Container Setup**: Add TigrisFS mounting in tenant containers
- **Local Development**: Enable local mounting of cloud tenant data
- **Basic Memory Core**: Validate unchanged operation over mounted filesystems
**Key Components:**
- **Tigris Storage**: Globally caching S3-compatible object storage via Fly.io integration
- **TigrisFS**: Purpose-built FUSE filesystem with intelligent caching
- **Turso Database**: Hosted libSQL for SQLite replacement
- **Single-Tenant Model**: One bucket + one database per tenant (simplified isolation)
## Architectural Overview & Key Insights
### TigrisFS
Unlike standard S3 mounting approaches, **TigrisFS is a purpose-built FUSE filesystem** optimized for object storage with several critical advantages:
1. **Eliminates Fly Volume Limitations**
- No single-machine attachment constraints
- No pre-provisioning of storage capacity
- Enables horizontal scaling and zero-downtime deployments
- Automatic global CDN caching at Fly.io edge locations
2. **Intelligent Caching Architecture**
- 1-4GB+ configurable memory cache for read/write operations
- Write-back caching for improved performance
- Metadata cache to reduce API calls
- "Close to Redis speed" for small object retrieval
3. **Cost-Effective Model**
- Pay only for storage used and transferred
- No wasted capacity from over-provisioning
- Automatic global replication included
- S3 durability with CDN performance
### API-Driven Architecture Eliminates File Watching Concerns
**Critical Insight**: All file access (reads/writes) in basic-memory-cloud go through the API layer:
- **MCP Tools → API**: All Basic Memory operations use FastAPI endpoints
- **Web App → API**: Frontend uses API for all data modifications
- **File watching is NOT required** for cloud operations, unlike local BM which uses the WatchService to monitor file changes.
This means:
- **Cloud Operations**: Manual sync after API writes is sufficient
- **Local Development**: File watching only matters for local editing experience
- **Performance Risk**: Dramatically reduced since we're not dependent on inotify over network filesystems
### Realistic Local Access Expectations
**Baseline Functionality (Guaranteed):**
- Read-only mounting for browsing cloud files
- Easy download/upload of entire projects
- File copying via standard filesystem operations
**Stretch Goal (Test in POC):**
- Live editing with eventual consistency (1-5 second delays acceptable)
- Automatic sync for local changes
- Not required for core functionality - pure upside if it works
### Production Deployment Advantages
1. **Multi-Region Deployment**: Tigris handles global replication automatically
2. **Zero-Downtime Updates**: No volume detach/attach during deployments
3. **Tenant Migrations**: Simply update credentials, no data movement
4. **Disaster Recovery**: Built into S3 durability model (99.999999999% durability)
5. **Auto-Scaling**: Storage scales with usage, no capacity planning needed
## How (High Level)
### POC Approach: Server-First Validation
**Rationale**: Start with server-side TigrisFS mounting because:
- Local access is meaningless if cloud containers can't mount TigrisFS reliably
- Container startup and API performance are critical path blockers
- TigrisFS compatibility with Basic Memory operations must be proven first
- Each phase gates the next - no point testing local access if server-side fails
### Phase 1: Server-Side TigrisFS Validation (Critical Foundation) ✅ COMPLETE
- [x] Set up Tigris bucket with test data via Fly.io integration
- [x] Create container image with TigrisFS support and dependencies
- [x] Test TigrisFS mounting in containerized environment
- [x] Run Basic Memory API operations over mounted TigrisFS
- [x] Validate all filesystem operations work correctly
- [x] Measure container startup time and resource usage
**Production Validation Results**: Container successfully deployed and operated for 42+ minutes serving real MCP requests with repository queries, knowledge graph navigation, and full Basic Memory API functionality over TigrisFS-mounted storage.
### Phase 2: Database Migration to Turso
- [ ] Set up Turso account and test database
- [ ] Modify Basic Memory to accept external DATABASE_URL
- [ ] Test all MCP tools with remote SQLite via Turso
- [ ] Validate performance and functionality parity
- [ ] Test API write → manual sync workflow in container
### Phase 3: Production Container Integration
- [ ] Implement tenant-specific credential management for buckets
- [x] Test container startup with automatic TigrisFS mounting
- [ ] Validate isolation between tenant containers
- [ ] Test API operations under realistic load
- [ ] Measure performance vs current Fly volume setup
### Phase 4: Local Access Validation (Bonus Feature)
- [ ] Test local TigrisFS mounting of tenant data
- [ ] Validate read-only access for browsing/downloading
- [ ] Test file copying and upload workflows
- [ ] Measure latency impact on user experience
- [ ] Test live editing if file watching works (stretch goal)
### Architecture Overview
```
Local Development:
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
│ Local TigrisFS │───▶│ Tigris Bucket │◀───│ Tenant Container│
│ Mount │ │ (Global CDN) │ │ TigrisFS mount │
└─────────────────┘ └─────────────────┘ └─────────────────┘
│ │
▼ ▼
┌─────────────────┐ ┌─────────────────┐
│ Basic Memory │ │ Basic Memory │
│ (local files) │ │ API + mounted │
└─────────────────┘ └─────────────────┘
│ │
▼ ▼
┌─────────────────┐ ┌─────────────────┐
│ Turso Database │◀───────────────────────────│ Turso Database │
│ (shared index) │ │ (shared index) │
└─────────────────┘ └─────────────────┘
Flow: API writes → Manual sync → Index update
Local: File watching (if available) → Auto sync
```
## How to Evaluate
### Success Criteria
- [x] **Filesystem Compatibility**: Basic Memory operates without modification over TigrisFS-mounted storage
- [x] **Performance Acceptable**: API-driven operations perform within acceptable latency (target: <500ms for typical operations)
- [ ] **Database Functionality**: All Basic Memory features work with Turso remote SQLite
- [x] **Container Reliability**: Tenant containers start successfully with automatic TigrisFS mounting
- [ ] **Local Access Baseline**: Users can mount cloud files locally for read-only browsing and file copying
- [x] **Data Isolation**: Tenant data remains properly isolated using bucket/database separation
- [ ] **Local Access Stretch**: Live editing with eventual sync (1-5 second delays acceptable)
### Testing Procedure
#### Phase 1: Server-Side Foundation Testing
1. **Container TigrisFS Test**:
```dockerfile
# Test container with TigrisFS mounting
FROM python:3.12
RUN apt-get update && apt-get install -y tigrisfs
# Test startup script
#!/bin/bash
tigrisfs --memory-limit 2048 $TIGRIS_BUCKET /app/data --daemon
cd /app/data && basic-memory sync
basic-memory-api --data-dir /app/data
```
2. **API Operations Validation**:
```bash
# Test all MCP operations over TigrisFS
curl -X POST /api/write_note -d '{"title":"test","content":"content"}'
curl -X GET /api/read_note/test
curl -X GET /api/search_notes?q=content
# Measure: response times, error rates, data consistency
```
#### Phase 2: Database Integration Testing
3. **Turso Integration Test**:
```bash
# Configure Turso connection in container
export DATABASE_URL="libsql://test-db.turso.io?authToken=..."
# Test all MCP tools with remote database
basic-memory tools # Test each tool functionality
# Test API write → manual sync workflow
```
#### Phase 3: Production Readiness Testing
4. **Performance Benchmarking**:
- Container startup time with TigrisFS mounting
- API operation response times (target: <500ms for typical operations)
- Search query performance with Turso (target: comparable to local SQLite)
- TigrisFS cache hit rates and memory usage
- Concurrent tenant isolation
#### Phase 4: Local Access Testing (If Phase 1-3 Succeed)
5. **Local Access Validation**:
```bash
# Test read-only access
tigrisfs tenant-bucket ~/local-tenant
ls -la ~/local-tenant # Browse files
cp ~/local-tenant/notes/* ~/backup/ # Copy files
# Test file watching (stretch goal)
echo "test" > ~/local-tenant/test.md
# Check if changes sync to cloud
```
### Go/No-Go Criteria by Phase
- **Phase 1**: Container must start successfully and serve API requests over TigrisFS
- **Phase 2**: All MCP tools must work with Turso with <2x latency increase
- **Phase 3**: Performance must be within 50% of current Fly volume setup
- **Phase 4**: Local mounting must work reliably for read-only access
### Risk Assessment
**Moderate Risk Items (Mitigated by API-First Architecture)**:
- [ ] TigrisFS performance for local access may have higher latency than local filesystem
- [ ] File watching (`inotify`) over FUSE may be unreliable for local development
- [ ] Network interruptions could cause filesystem errors during local editing
- [ ] Write-back caching could cause data loss if container crashes during flush
**Low Risk Items (API-First Eliminates)**:
- [ ] ~~Real-time file watching~~ - Not required for cloud operations
- [ ] ~~Concurrent write consistency~~ - Single-tenant model with API coordination
- [ ] ~~S3 rate limits~~ - TigrisFS intelligent caching handles this
**Mitigation Strategies**:
- **Performance**: Comprehensive benchmarking with realistic workloads
- **Reliability**: Graceful degradation to read-only local access if live editing fails
- **Data Safety**: Regular sync intervals and write-through mode for critical operations
- **Fallback**: Keep Fly volumes as backup deployment option
### Metrics to Track
- **API Latency**: Response times for MCP tools and web operations
- **Cache Effectiveness**: TigrisFS cache hit rates and memory usage
- **Local Access Performance**: File browsing and copying speeds
- **Reliability**: Success rate of mount operations and data consistency
- **Cost**: Storage usage, API calls, and network transfer costs vs current volumes
## Notes
### Key Architectural Decisions
- **Single tenant per bucket/database**: Simplifies isolation and credential management
- **Maintain POSIX compatibility**: Preserve Basic Memory's existing filesystem assumptions
- **TigrisFS over rclone**: Purpose-built for object storage with intelligent caching
- **Turso for SQLite**: Leverages specialized remote SQLite expertise
- **API-first approach**: Eliminates file watching dependency for cloud operations
### Alternative Approaches Considered
- **S3-native storage backend**: Would require Basic Memory architecture changes
- **Hybrid approach**: Local files + cloud sync (adds complexity)
- **Standard rclone mounting**: Less optimized than TigrisFS for object storage workloads
- **Keep Fly volumes**: Maintains current limitations but proven reliability
### Integration Points
- [ ] Fly.io Tigris integration for bucket provisioning
- [ ] Turso account setup and database provisioning
- [ ] Container image modifications for TigrisFS support
- [ ] Credential management for tenant isolation
- [ ] API modification for manual sync triggers
- [ ] Local client setup documentation for TigrisFS mounting
## Observations
- [architecture] Tigris/Turso split cleanly separates file storage from indexing concerns #storage-separation
- [breakthrough] API-first architecture eliminates file watching dependency for cloud operations #api-first-advantage
- [user-experience] Local mounting of cloud files could be revolutionary for knowledge management #local-cloud-hybrid
- [compatibility] Maintaining POSIX filesystem assumptions preserves Basic Memory's local/cloud compatibility #architecture-preservation
- [simplification] Single tenant per bucket eliminates complex multi-tenancy in storage layer #tenant-isolation
- [performance] TigrisFS intelligent caching could provide near-local performance for common operations #tigrisfs-advantage
- [deployment] Zero-downtime updates become trivial without volume constraints #deployment-simplification
- [benefit] Object storage pricing model could be more favorable than volume pricing #cost-optimization
- [innovation] Read-only local access alone would address major SaaS limitation #competitive-advantage
- [risk-mitigation] API-driven sync reduces performance requirements vs real-time file watching #risk-reduction
## Relations
- implements [[SPEC-6 Explicit Project Parameter Architecture]]
- requires [[Fly.io Tigris Integration]]
- enables [[Local Cloud File Access]]
- alternative_to [[Fly Volume Storage]]
## Links
- https://fly.io/hello/tigris
- https://fly.io/docs/tigris/
- https://www.tigrisdata.com/docs/sdks/fly/data-migration-with-flyctl/
- https://www.tigrisdata.com/docs/training/tigrisfs/
- https://www.tigrisdata.com/blog/tigris-filesystem/
- https://www.tigrisdata.com/docs/quickstarts/rclone/
+886
View File
@@ -0,0 +1,886 @@
---
title: 'SPEC-8: TigrisFS Integration for Tenant API'
Date: September 22, 2025
Status: Phase 3.6 Complete - Tenant Mount API Endpoints Ready for CLI Implementation
Priority: High
Goal: Replace Fly volumes with Tigris bucket provisioning in production tenant API
permalink: spec-8-tigris-fs-integration
---
## Executive Summary
Based on SPEC-7 Phase 4 POC testing, this spec outlines productizing the TigrisFS/rclone implementation in the Basic Memory Cloud tenant API.
We're moving from proof-of-concept to production integration, replacing Fly volume storage with Tigris bucket-per-tenant architecture.
## Current Architecture (Fly Volumes)
### Tenant Provisioning Flow
```python
# apps/cloud/src/basic_memory_cloud/workflows/tenant_provisioning.py
async def provision_tenant_infrastructure(tenant_id: str):
# 1. Create Fly app
# 2. Create Fly volume ← REPLACE THIS
# 3. Deploy API container with volume mount
# 4. Configure health checks
```
### Storage Implementation
- Each tenant gets dedicated Fly volume (1GB-10GB)
- Volume mounted at `/app/data` in API container
- Local filesystem storage with Basic Memory indexing
- No global caching or edge distribution
## Proposed Architecture (Tigris Buckets)
### New Tenant Provisioning Flow
```python
async def provision_tenant_infrastructure(tenant_id: str):
# 1. Create Fly app
# 2. Create Tigris bucket with admin credentials ← NEW
# 3. Store bucket name in tenant record ← NEW
# 4. Deploy API container with TigrisFS mount using admin credentials
# 5. Configure health checks
```
### Storage Implementation
- Each tenant gets dedicated Tigris bucket
- TigrisFS mounts bucket at `/app/data` in API container
- Global edge caching and distribution
- Configurable cache TTL for sync performance
## Implementation Plan
### Phase 1: Bucket Provisioning Service
**✅ IMPLEMENTED: StorageClient with Admin Credentials**
```python
# apps/cloud/src/basic_memory_cloud/clients/storage_client.py
class StorageClient:
async def create_tenant_bucket(self, tenant_id: UUID) -> TigrisBucketCredentials
async def delete_tenant_bucket(self, tenant_id: UUID, bucket_name: str) -> bool
async def list_buckets(self) -> list[TigrisBucketResponse]
async def test_tenant_credentials(self, credentials: TigrisBucketCredentials) -> bool
```
**Simplified Architecture Using Admin Credentials:**
- Single admin access key with full Tigris permissions (configured in console)
- No tenant-specific IAM user creation needed
- Bucket-per-tenant isolation for logical separation
- Admin credentials shared across all tenant operations
**Integrate with Provisioning workflow:**
```python
# Update tenant_provisioning.py
async def provision_tenant_infrastructure(tenant_id: str):
storage_client = StorageClient(settings.aws_access_key_id, settings.aws_secret_access_key)
bucket_creds = await storage_client.create_tenant_bucket(tenant_id)
await store_bucket_name(tenant_id, bucket_creds.bucket_name)
await deploy_api_with_tigris(tenant_id, bucket_creds)
```
### Phase 2: Simplified Bucket Management
**✅ SIMPLIFIED: Admin Credentials + Bucket Names Only**
Since we use admin credentials for all operations, we only need to track bucket names per tenant:
1. **Primary Storage (Fly Secrets)**
```bash
flyctl secrets set -a basic-memory-{tenant_id} \
AWS_ACCESS_KEY_ID="{admin_access_key}" \
AWS_SECRET_ACCESS_KEY="{admin_secret_key}" \
AWS_ENDPOINT_URL_S3="https://fly.storage.tigris.dev" \
AWS_REGION="auto" \
BUCKET_NAME="basic-memory-{tenant_id}"
```
2. **Database Storage (Bucket Name Only)**
```python
# apps/cloud/src/basic_memory_cloud/models/tenant.py
class Tenant(BaseModel):
# ... existing fields
tigris_bucket_name: Optional[str] = None # Just store bucket name
tigris_region: str = "auto"
created_at: datetime
```
**Benefits of Simplified Approach:**
- No credential encryption/decryption needed
- Admin credentials managed centrally in environment
- Only bucket names stored in database (not sensitive)
- Simplified backup/restore scenarios
- Reduced security attack surface
### Phase 3: API Container Updates
**Update API container configuration:**
```dockerfile
# apps/api/Dockerfile
# Add TigrisFS installation
RUN curl -L https://github.com/tigrisdata/tigrisfs/releases/latest/download/tigrisfs-linux-amd64 \
-o /usr/local/bin/tigrisfs && chmod +x /usr/local/bin/tigrisfs
```
**Startup script integration:**
```bash
# apps/api/tigrisfs-startup.sh (already exists)
# Mount TigrisFS → Start Basic Memory API
exec python -m basic_memory_cloud_api.main
```
**Fly.toml environment (optimized for < 5s startup):**
```toml
# apps/api/fly.tigris-production.toml
[env]
TIGRISFS_MEMORY_LIMIT = '1024' # Reduced for faster init
TIGRISFS_MAX_FLUSHERS = '16' # Fewer threads for faster startup
TIGRISFS_STAT_CACHE_TTL = '30s' # Balance sync speed vs startup
TIGRISFS_LAZY_INIT = 'true' # Enable lazy loading
BASIC_MEMORY_HOME = '/app/data'
# Suspend optimization for wake-on-network
[machine]
auto_stop_machines = "suspend" # Faster than full stop
auto_start_machines = true
min_machines_running = 0
```
### Phase 4: Local Access Features
**CLI automation for local mounting:**
```python
# New CLI command: basic-memory cloud mount
async def setup_local_mount(tenant_id: str):
# 1. Fetch bucket credentials from cloud API
# 2. Configure rclone with scoped IAM policy
# 3. Mount via rclone nfsmount (macOS) or FUSE (Linux)
# 4. Start Basic Memory sync watcher
```
**Local mount configuration:**
```bash
# rclone config for tenant
rclone mount basic-memory-{tenant_id}: ~/basic-memory-{tenant_id} \
--nfs-mount \
--vfs-cache-mode writes \
--cache-dir ~/.cache/rclone/basic-memory-{tenant_id}
```
### Phase 5: TigrisFS Cache Sync Solutions
**Problem**: When files are uploaded via CLI/bisync, the tenant API container doesn't see them immediately due to TigrisFS cache (30s TTL) and lack of inotify events on mounted filesystems.
**Multi-Layer Solution:**
**Layer 1: API Sync Endpoint** (Immediate)
```python
# POST /sync - Force TigrisFS cache refresh
# Callable by CLI after uploads
subprocess.run(["sync", "fsync /app/data"], check=True)
```
**Layer 2: Tigris Webhook Integration** (Real-time)
https://www.tigrisdata.com/docs/buckets/object-notifications/#webhook
```python
# Webhook endpoint for bucket changes
@app.post("/webhooks/tigris/{tenant_id}")
async def handle_bucket_notification(tenant_id: str, event: TigrisEvent):
if event.eventName in ["OBJECT_CREATED_PUT", "OBJECT_DELETED"]:
await notify_container_sync(tenant_id, event.object.key)
```
**Layer 3: CLI Sync Notification** (User-triggered)
```bash
# CLI calls container sync endpoint after successful bisync
basic-memory cloud bisync # Automatically notifies container
curl -X POST https://basic-memory-{tenant-id}.fly.dev/sync
```
**Layer 4: Periodic Sync Fallback** (Safety net)
```python
# Background task: fsync /app/data every 30s as fallback
# Ensures eventual consistency even if other layers fail
```
**Implementation Priority:**
1. Layer 1 (API endpoint) - Quick testing capability
2. Layer 3 (CLI integration) - Improved UX
3. Layer 4 (Periodic fallback) - Safety net
4. Layer 2 (Webhooks) - Production real-time sync
## Performance Targets
### Sync Latency
- **Target**: < 5 seconds local→cloud→container
- **Configuration**: `TIGRISFS_STAT_CACHE_TTL = '5s'`
- **Monitoring**: Track sync metrics in production
### Container Startup
- **Target**: < 5 seconds including TigrisFS mount
- **Fast retry**: 0.5s intervals for mount verification
- **Fallback**: Container fails fast if mount fails
### Memory Usage
- **TigrisFS cache**: 2GB memory limit per container
- **Concurrent uploads**: 32 flushers max
- **VM sizing**: shared-cpu-2x (2048mb) minimum
## Security Considerations
### Bucket Isolation
- Each tenant has dedicated bucket
- IAM policies prevent cross-tenant access
- No shared bucket with subdirectories
### Credential Security
- Fly secrets for runtime access
- Encrypted database backup for disaster recovery
- Credential rotation capability
### Data Residency
- Tigris global edge caching
- SOC2 Type II compliance
- Encryption at rest and in transit
## Operational Benefits
### Scalability
- Horizontal scaling with stateless API containers
- Global edge distribution
- Better resource utilization
### Reliability
- No cold starts between tenants
- Built-in redundancy and caching
- Simplified backup strategy
### Cost Efficiency
- Pay-per-use storage pricing
- Shared infrastructure benefits
- Reduced operational overhead
## Risk Mitigation
### Data Loss Prevention
- Dual credential storage (Fly + database)
- Automated backup workflows to R2/S3
- Tigris built-in redundancy
### Performance Degradation
- Configurable cache settings per tenant
- Monitoring and alerting on sync latency
- Fallback to volume storage if needed
### Security Vulnerabilities
- Bucket-per-tenant isolation
- Regular credential rotation
- Security scanning and monitoring
## Success Metrics
### Technical Metrics
- Sync latency P50 < 5 seconds
- Container startup time < 5 seconds
- Zero data loss incidents
- 99.9% uptime per tenant
### Business Metrics
- Reduced infrastructure costs vs volumes
- Improved user experience with faster sync
- Enhanced enterprise security posture
- Simplified operational overhead
## Open Questions
1. **Tigris rate limits**: What are the API limits for bucket creation?
2. **Cost analysis**: What's the break-even point vs Fly volumes?
3. **Regional preferences**: Should enterprise customers choose regions?
4. **Backup retention**: How long to keep automated backups?
## Implementation Checklist
### Phase 1: Bucket Provisioning Service ✅ COMPLETED
- [x] **Research Tigris bucket API** - Document bucket creation and S3 API compatibility
- [x] **Create StorageClient class** - Implemented with admin credentials and comprehensive integration tests
- [x] **Test bucket creation** - Full test suite validates API integration with real Tigris environment
- [x] **Add bucket provisioning to DBOS workflow** - Integrated StorageClient with tenant_provisioning.py
### Phase 2: Simplified Bucket Management ✅ COMPLETED
- [x] **Update Tenant model** with tigris_bucket_name field (replaced fly_volume_id)
- [x] **Implement bucket name storage** - Database migration and model updates completed
- [x] **Test bucket provisioning integration** - Full test suite validates workflow from tenant creation to bucket assignment
- [x] **Remove volume logic from all tests** - Complete migration from volume-based to bucket-based architecture
### Phase 3: API Container Integration ✅ COMPLETED
- [x] **Update Dockerfile** to install TigrisFS binary in API container with configurable version
- [x] **Optimize tigrisfs-startup.sh** with production-ready security and reliability improvements
- [x] **Create production-ready container** with proper signal handling and mount validation
- [x] **Implement security fixes** based on Claude code review (conditional debug, credential protection)
- [x] **Add proper process supervision** with cleanup traps and error handling
- [x] **Remove debug artifacts** - Cleaned up all debug Dockerfiles and test scripts
### Phase 3.5: IAM Access Key Management ✅ COMPLETED
- [x] **Research Tigris IAM API** - Documented create_policy, attach_user_policy, delete_access_key operations
- [x] **Implement bucket-scoped credential generation** - StorageClient.create_tenant_access_keys() with IAM policies
- [x] **Add comprehensive security test suite** - 5 security-focused integration tests covering all attack vectors
- [x] **Verify cross-bucket access prevention** - Scoped credentials can ONLY access their designated bucket
- [x] **Test credential lifecycle management** - Create, validate, delete, and revoke access keys
- [x] **Validate admin vs scoped credential isolation** - Different access patterns and security boundaries
- [x] **Test multi-tenant isolation** - Multiple tenants cannot access each other's buckets
### Phase 3.6: Tenant Mount API Endpoints ✅ COMPLETED
- [x] **Implement GET /tenant/mount/info** - Returns mount info without exposing credentials
- [x] **Implement POST /tenant/mount/credentials** - Creates new bucket-scoped credentials for CLI mounting
- [x] **Implement DELETE /tenant/mount/credentials/{cred_id}** - Revoke specific credentials with proper cleanup
- [x] **Implement GET /tenant/mount/credentials** - List active credentials without exposing secrets
- [x] **Add TenantMountCredentials database model** - Tracks credential metadata (no secret storage)
- [x] **Create comprehensive test suite** - 28 tests covering all scenarios including multi-session support
- [x] **Implement multi-session credential flow** - Multiple active credentials per tenant supported
- [x] **Secure credential handling** - Secret keys never stored, returned once only for immediate use
- [x] **Add dependency injection for StorageClient** - Clean integration with existing API architecture
- [x] **Fix Tigris configuration for cloud service** - Added AWS environment variables to fly.template.toml
- [x] **Update tenant machine configurations** - Include AWS credentials for TigrisFS mounting with clear credential strategy
**Security Test Results:**
```
✅ Cross-bucket access prevention - PASS
✅ Deleted credentials access revoked - PASS
✅ Invalid credentials rejected - PASS
✅ Admin vs scoped credential isolation - PASS
✅ Multiple scoped credentials isolation - PASS
```
**Implementation Details:**
- Uses Tigris IAM managed policies (create_policy + attach_user_policy)
- Bucket-scoped S3 policies with Actions: GetObject, PutObject, DeleteObject, ListBucket
- Resource ARNs limited to specific bucket: `arn:aws:s3:::bucket-name` and `arn:aws:s3:::bucket-name/*`
- Access keys follow Tigris format: `tid_` prefix with secure random suffix
- Complete cleanup on deletion removes both access keys and associated policies
### Phase 4: Local Access CLI
- [x] **Design local mount CLI command** for automated rclone configuration
- [x] **Implement credential fetching** from cloud API for local setup
- [x] **Create rclone config automation** for tenant-specific bucket mounting
- [x] **Test local→cloud→container sync** with optimized cache settings
- [x] **Document local access setup** for beta users
### Phase 5: Webhook Integration (Future)
- [ ] **Research Tigris webhook API** for object notifications and payload format
- [ ] **Design webhook endpoint** for real-time sync notifications
- [ ] **Implement notification handling** to trigger Basic Memory sync events
- [ ] **Test webhook delivery** and sync latency improvements
## Success Metrics
- [ ] **Container startup < 5 seconds** including TigrisFS mount and Basic Memory init
- [ ] **Sync latency < 5 seconds** for local→cloud→container file changes
- [ ] **Zero data loss** during bucket provisioning and credential management
- [ ] **100% test coverage** for new TigrisBucketService and credential functions
- [ ] **Beta deployment** with internal users validating local-cloud workflow
## Implementation Notes
## Phase 4.1: Bidirectional Sync with rclone bisync (NEW)
### Problem Statement
During testing, we discovered that some applications (particularly Obsidian) don't detect file changes over NFS mounts. Rather than building a custom sync daemon, we can leverage `rclone bisync` - rclone's built-in bidirectional synchronization feature.
### Solution: rclone bisync
Use rclone's proven bidirectional sync instead of custom implementation:
**Core Architecture:**
```bash
# rclone bisync handles all the complexity
rclone bisync ~/basic-memory-{tenant_id} basic-memory-{tenant_id}:{bucket_name} \
--create-empty-src-dirs \
--conflict-resolve newer \
--resilient \
--check-access
```
**Key Benefits:**
- ✅ **Battle-tested**: Production-proven rclone functionality
- ✅ **MIT licensed**: Open source with permissive licensing
- ✅ **No custom code**: Zero maintenance burden for sync logic
- ✅ **Built-in safety**: max-delete protection, conflict resolution
- ✅ **Simple installation**: Works with Homebrew rclone (no FUSE needed)
- ✅ **File watcher compatible**: Works with Obsidian and all applications
- ✅ **Offline support**: Can work offline and sync when connected
### bisync Conflict Resolution Options
**Built-in conflict strategies:**
```bash
--conflict-resolve none # Keep both files with .conflict suffixes (safest)
--conflict-resolve newer # Always pick the most recently modified file
--conflict-resolve larger # Choose based on file size
--conflict-resolve path1 # Always prefer local changes
--conflict-resolve path2 # Always prefer cloud changes
```
### Sync Profiles Using bisync
**Profile configurations:**
```python
BISYNC_PROFILES = {
"safe": {
"conflict_resolve": "none", # Keep both versions
"max_delete": 10, # Prevent mass deletion
"check_access": True, # Verify sync integrity
"description": "Safe mode with conflict preservation"
},
"balanced": {
"conflict_resolve": "newer", # Auto-resolve to newer file
"max_delete": 25,
"check_access": True,
"description": "Balanced mode (recommended default)"
},
"fast": {
"conflict_resolve": "newer",
"max_delete": 50,
"check_access": False, # Skip verification for speed
"description": "Fast mode for rapid iteration"
}
}
```
### CLI Commands
**Manual sync commands:**
```bash
basic-memory cloud bisync # Manual bidirectional sync
basic-memory cloud bisync --dry-run # Preview changes
basic-memory cloud bisync --profile safe # Use specific profile
basic-memory cloud bisync --resync # Force full baseline resync
```
**Watch mode (Step 1):**
```bash
basic-memory cloud bisync --watch # Long-running process, sync every 60s
basic-memory cloud bisync --watch --interval 30s # Custom interval
```
**System integration (Step 2 - Future):**
```bash
basic-memory cloud bisync-service install # Install as system service
basic-memory cloud bisync-service start # Start background service
basic-memory cloud bisync-service status # Check service status
```
### Implementation Strategy
**Phase 4.1.1: Core bisync Implementation**
- [ ] Implement `run_bisync()` function wrapping rclone bisync
- [ ] Add profile-based configuration (safe/balanced/fast)
- [ ] Create conflict resolution and safety options
- [ ] Test with sample files and conflict scenarios
**Phase 4.1.2: Watch Mode**
- [ ] Add `--watch` flag for continuous sync
- [ ] Implement configurable sync intervals
- [ ] Add graceful shutdown and signal handling
- [ ] Create status monitoring and progress indicators
**Phase 4.1.3: User Experience**
- [ ] Add conflict reporting and resolution guidance
- [ ] Implement dry-run preview functionality
- [ ] Create troubleshooting and diagnostic commands
- [ ] Add filtering configuration (.gitignore-style)
**Phase 4.1.4: System Integration (Future)**
- [ ] Generate platform-specific service files (launchd/systemd)
- [ ] Add service management commands
- [ ] Implement automatic startup and recovery
- [ ] Create monitoring and logging integration
### Technical Implementation
**Core bisync wrapper:**
```python
def run_bisync(
tenant_id: str,
bucket_name: str,
profile: str = "balanced",
dry_run: bool = False
) -> bool:
"""Run rclone bisync with specified profile."""
local_path = Path.home() / f"basic-memory-{tenant_id}"
remote_path = f"basic-memory-{tenant_id}:{bucket_name}"
profile_config = BISYNC_PROFILES[profile]
cmd = [
"rclone", "bisync",
str(local_path), remote_path,
"--create-empty-src-dirs",
"--resilient",
f"--conflict-resolve={profile_config['conflict_resolve']}",
f"--max-delete={profile_config['max_delete']}",
"--filters-file", "~/.basic-memory/bisync-filters.txt"
]
if profile_config.get("check_access"):
cmd.append("--check-access")
if dry_run:
cmd.append("--dry-run")
return subprocess.run(cmd, check=True).returncode == 0
```
**Default filter file (~/.basic-memory/bisync-filters.txt):**
```
- .DS_Store
- .git/**
- __pycache__/**
- *.pyc
- .pytest_cache/**
- node_modules/**
- .conflict-*
- Thumbs.db
- desktop.ini
```
**Advantages Over Custom Daemon:**
- ✅ **Zero maintenance**: No custom sync logic to debug/maintain
- ✅ **Production proven**: Used by thousands in production
- ✅ **Safety features**: Built-in max-delete, conflict handling, recovery
- ✅ **Filtering**: Advanced exclude patterns and rules
- ✅ **Performance**: Optimized for various storage backends
- ✅ **Community support**: Extensive documentation and community
## Phase 4.2: NFS Mount Support (Direct Access)
### Solution: rclone nfsmount
Keep the existing NFS mount functionality for users who prefer direct file access:
**Core Architecture:**
```bash
# rclone nfsmount provides transparent file access
rclone nfsmount basic-memory-{tenant_id}:{bucket_name} ~/basic-memory-{tenant_id} \
--vfs-cache-mode writes \
--dir-cache-time 10s \
--daemon
```
**Key Benefits:**
- ✅ **Real-time access**: Files appear immediately as they're created/modified
- ✅ **Transparent**: Works with any application that reads/writes files
- ✅ **Low latency**: Direct access without sync delays
- ✅ **Simple**: No periodic sync commands needed
- ✅ **Homebrew compatible**: Works with Homebrew rclone (no FUSE required)
**Limitations:**
- ❌ **File watcher compatibility**: Some apps (Obsidian) don't detect changes over NFS
- ❌ **Network dependency**: Requires active connection to cloud storage
- ❌ **Potential conflicts**: Simultaneous edits from multiple locations can cause issues
### Mount Profiles (Existing)
**Already implemented profiles from SPEC-7 testing:**
```python
MOUNT_PROFILES = {
"fast": {
"cache_time": "5s",
"poll_interval": "3s",
"description": "Ultra-fast development (5s sync)"
},
"balanced": {
"cache_time": "10s",
"poll_interval": "5s",
"description": "Fast development (10-15s sync, recommended)"
},
"safe": {
"cache_time": "15s",
"poll_interval": "10s",
"description": "Conflict-aware mount with backup",
"extra_args": ["--conflict-suffix", ".conflict-{DateTimeExt}"]
}
}
```
### CLI Commands (Existing)
**Mount commands already implemented:**
```bash
basic-memory cloud mount # Mount with balanced profile
basic-memory cloud mount --profile fast # Ultra-fast caching
basic-memory cloud mount --profile safe # Conflict detection
basic-memory cloud unmount # Clean unmount
basic-memory cloud mount-status # Show mount status
```
## User Choice: Mount vs Bisync
### When to Use Each Approach
| Use Case | Recommended Solution | Why |
|----------|---------------------|-----|
| **Obsidian users** | `bisync` | File watcher support for live preview |
| **CLI/vim/emacs users** | `mount` | Direct file access, lower latency |
| **Offline work** | `bisync` | Can work offline, sync when connected |
| **Real-time collaboration** | `mount` | Immediate visibility of changes |
| **Multiple machines** | `bisync` | Better conflict handling |
| **Single machine** | `mount` | Simpler, more transparent |
| **Development work** | Either | Both work well, user preference |
| **Large files** | `mount` | Streaming access vs full download |
### Installation Simplicity
**Both approaches now use simple Homebrew installation:**
```bash
# Single installation command for both approaches
brew install rclone
# No macFUSE, no system modifications needed
# Works immediately with both mount and bisync
```
### Implementation Status
**Phase 4.1: bisync** (NEW)
- [ ] Implement bisync command wrapper
- [ ] Add watch mode with configurable intervals
- [ ] Create conflict resolution workflows
- [ ] Add filtering and safety options
**Phase 4.2: mount** (EXISTING - ✅ IMPLEMENTED)
- [x] NFS mount commands with profile support
- [x] Mount management and cleanup
- [x] Process monitoring and health checks
- [x] Credential integration with cloud API
**Both approaches share:**
- [x] Credential management via cloud API
- [x] Secure rclone configuration
- [x] Tenant isolation and bucket scoping
- [x] Simple Homebrew rclone installation
Key Features:
1. Cross-Platform rclone Installation (rclone_installer.py):
- macOS: Homebrew → official script fallback
- Linux: snap → apt → official script fallback
- Windows: winget → chocolatey → scoop fallback
- Automatic version detection and verification
2. Smart rclone Configuration (rclone_config.py):
- Automatic tenant-specific config generation
- Three optimized mount profiles from your SPEC-7 testing:
- fast: 5s sync (ultra-performance)
- balanced: 10-15s sync (recommended default)
- safe: 15s sync + conflict detection
- Backup existing configs before modification
3. Robust Mount Management (mount_commands.py):
- Automatic tenant credential generation
- Mount path management (~/basic-memory-{tenant-id})
- Process lifecycle management (prevent duplicate mounts)
- Orphaned process cleanup
- Mount verification and health checking
4. Clean Architecture (api_client.py):
- Separated API client to avoid circular imports
- Reuses existing authentication infrastructure
- Consistent error handling and logging
User Experience:
One-Command Setup:
basic-memory cloud setup
```bash
# 1. Installs rclone automatically
# 2. Authenticates with existing login
# 3. Generates secure credentials
# 4. Configures rclone
# 5. Performs initial mount
```
Profile-Based Mounting:
basic-memory cloud mount --profile fast # 5s sync
basic-memory cloud mount --profile balanced # 15s sync (default)
basic-memory cloud mount --profile safe # conflict detection
Status Monitoring:
basic-memory cloud mount-status
```bash
# Shows: tenant info, mount path, sync profile, rclone processes
```
### local mount api
Endpoint 1: Get Tenant Info for user
Purpose: Get tenant details for mounting
- pass in jwt
- service returns mount info
**✅ IMPLEMENTED API Specification:**
**Endpoint 1: GET /tenant/mount/info**
- Purpose: Get tenant mount information without exposing credentials
- Authentication: JWT token (tenant_id extracted from claims)
Request:
```
GET /tenant/mount/info
Authorization: Bearer {jwt_token}
```
Response:
```json
{
"tenant_id": "434252dd-d83b-4b20-bf70-8a950ff875c4",
"bucket_name": "basic-memory-434252dd",
"has_credentials": true,
"credentials_created_at": "2025-09-22T16:48:50.414694"
}
```
**Endpoint 2: POST /tenant/mount/credentials**
- Purpose: Generate NEW bucket-scoped S3 credentials for rclone mounting
- Authentication: JWT token (tenant_id extracted from claims)
- Multi-session: Creates new credentials without revoking existing ones
Request:
```
POST /tenant/mount/credentials
Authorization: Bearer {jwt_token}
Content-Type: application/json
```
*Note: No request body needed - tenant_id extracted from JWT*
Response:
```json
{
"tenant_id": "434252dd-d83b-4b20-bf70-8a950ff875c4",
"bucket_name": "basic-memory-434252dd",
"access_key": "test_access_key_12345",
"secret_key": "test_secret_key_abcdef",
"endpoint_url": "https://fly.storage.tigris.dev",
"region": "auto"
}
```
**🔒 Security Notes:**
- Secret key returned ONCE only - never stored in database
- Credentials are bucket-scoped (cannot access other tenants' buckets)
- Multiple active credentials supported per tenant (work laptop + personal machine)
Implementation Notes
Security:
- Both endpoints require JWT authentication
- Extract tenant_id from JWT claims (not request body)
- Generate scoped credentials (not admin credentials)
- Credentials should have bucket-specific access only
Integration Points:
- Use your existing StorageClient from SPEC-8 implementation
- Leverage existing JWT middleware for tenant extraction
- Return same credential format as your Tigris bucket provisioning
Error Handling:
- 401 if not authenticated
- 403 if tenant doesn't exist
- 500 if credential generation fails
**🔄 Design Decisions:**
1. **Secure Credential Flow (No Secret Storage)**
Based on CLI flow analysis, we follow security best practices:
- ✅ API generates both access_key + secret_key via Tigris IAM
- ✅ Returns both in API response for immediate use
- ✅ CLI uses credentials immediately to configure rclone
- ✅ Database stores only metadata (access_key + policy_arn for cleanup)
- ✅ rclone handles secure local credential storage
- ❌ **Never store secret_key in database (even encrypted)**
2. **CLI Credential Flow**
```bash
# CLI calls API
POST /tenant/mount/credentials → {access_key, secret_key, ...}
# CLI immediately configures rclone
rclone config create basic-memory-{tenant_id} s3 \
access_key_id={access_key} \
secret_access_key={secret_key} \
endpoint=https://fly.storage.tigris.dev
# Database tracks metadata only
INSERT INTO tenant_mount_credentials (tenant_id, access_key, policy_arn, ...)
```
3. **Multiple Sessions Supported**
- Users can have multiple active credential sets (work laptop, personal machine, etc.)
- Each credential generation creates a new Tigris access key
- List active credentials via API (shows access_key but never secret)
4. **Failure Handling & Cleanup**
- **Happy Path**: Credentials created → Used immediately → rclone configured
- **Orphaned Credentials**: Background job revokes unused credentials
- **API Failure Recovery**: Retry Tigris deletion with stored policy_arn
- **Status Tracking**: Track tigris_deletion_status (pending/completed/failed)
5. **Event Sourcing & Audit**
- MountCredentialCreatedEvent
- MountCredentialRevokedEvent
- MountCredentialOrphanedEvent (for cleanup)
- Full audit trail for security compliance
6. **Tenant/Bucket Validation**
- Verify tenant exists and has valid bucket before credential generation
- Use existing StorageClient to validate bucket access
- Prevent credential generation for inactive/invalid tenants
📋 **Implemented API Endpoints:**
```
✅ IMPLEMENTED:
GET /tenant/mount/info # Get tenant/bucket info (no credentials exposed)
POST /tenant/mount/credentials # Generate new credentials (returns secret once)
GET /tenant/mount/credentials # List active credentials (no secrets)
DELETE /tenant/mount/credentials/{cred_id} # Revoke specific credentials
```
**API Implementation Status:**
- ✅ **GET /tenant/mount/info**: Returns tenant_id, bucket_name, has_credentials, credentials_created_at
- ✅ **POST /tenant/mount/credentials**: Creates new bucket-scoped access keys, returns access_key + secret_key once
- ✅ **GET /tenant/mount/credentials**: Lists active credentials without exposing secret keys
- ✅ **DELETE /tenant/mount/credentials/{cred_id}**: Revokes specific credentials with proper Tigris IAM cleanup
- ✅ **Multi-session support**: Multiple active credentials per tenant (work laptop + personal machine)
- ✅ **Security**: Secret keys never stored in database, returned once only for immediate use
- ✅ **Comprehensive test suite**: 28 tests covering all scenarios including error handling and multi-session flows
- ✅ **Dependency injection**: Clean integration with existing FastAPI architecture
- ✅ **Production-ready configuration**: Tigris credentials properly configured for tenant machines
🗄️ **Secure Database Schema:**
```sql
CREATE TABLE tenant_mount_credentials (
id UUID PRIMARY KEY,
tenant_id UUID REFERENCES tenant(id),
access_key VARCHAR(255) NOT NULL,
-- secret_key REMOVED - never store secrets (security best practice)
policy_arn VARCHAR(255) NOT NULL, -- For Tigris IAM cleanup
tigris_deletion_status VARCHAR(20) DEFAULT 'pending', -- Track cleanup
created_at TIMESTAMP DEFAULT NOW(),
updated_at TIMESTAMP DEFAULT NOW(),
revoked_at TIMESTAMP NULL,
last_used_at TIMESTAMP NULL, -- Track usage for orphan cleanup
description VARCHAR(255) DEFAULT 'CLI mount credentials'
);
```
**Security Benefits:**
- ✅ Database breach cannot expose secrets
- ✅ Follows "secrets don't persist" security principle
- ✅ Meets compliance requirements (SOC2, etc.)
- ✅ Reduced attack surface
- ✅ CLI gets credentials once and stores securely via rclone
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,196 @@
---
title: 'SPEC-9: Signed Header Tenant Information'
type: spec
permalink: specs/spec-9-signed-header-tenant-information
tags:
- authentication
- tenant-isolation
- proxy
- security
- mcp
---
# SPEC-9: Signed Header Tenant Information
## Why
WorkOS JWT templates don't work with MCP's dynamic client registration requirement, preventing us from getting tenant information directly in JWT tokens. We need an alternative secure method to pass tenant context from the Cloud Proxy Service to tenant instances.
**Problem Context:**
- MCP spec requires dynamic client registration
- WorkOS JWT templates only apply to statically configured clients
- Without tenant information, we can't properly route requests or isolate tenant data
- Current JWT tokens only contain standard OIDC claims (sub, email, etc.)
**Affected Areas:**
- Cloud Proxy Service (`apps/cloud`) - request forwarding
- Tenant API instances (`apps/api`) - tenant context validation
- MCP Gateway (`apps/mcp`) - authentication flow
- Overall tenant isolation security model
## What
Implement HMAC-signed headers that the Cloud Proxy Service adds when forwarding requests to tenant instances. This provides secure, tamper-proof tenant information without relying on JWT custom claims.
**Components:**
- Header signing utility in Cloud Proxy Service
- Header validation middleware in Tenant API instances
- Shared secret configuration across services
- Fallback mechanisms for development and error cases
## How (High Level)
### 1. Header Format
Add these signed headers to all proxied requests:
```
X-BM-Tenant-ID: {tenant_id}
X-BM-Timestamp: {unix_timestamp}
X-BM-Signature: {hmac_sha256_signature}
```
### 2. Signature Algorithm
```python
# Canonical message format
message = f"{tenant_id}:{timestamp}"
# HMAC-SHA256 signature
signature = hmac.new(
key=shared_secret.encode('utf-8'),
msg=message.encode('utf-8'),
digestmod=hashlib.sha256
).hexdigest()
```
### 3. Implementation Flow
#### Cloud Proxy Service (`apps/cloud`)
1. Extract `tenant_id` from authenticated user profile
2. Generate timestamp and canonical message
3. Sign message with shared secret
4. Add headers to request before forwarding to tenant instance
#### Tenant API Instances (`apps/api`)
1. Middleware validates headers on all incoming requests
2. Extract tenant_id, timestamp from headers
3. Verify timestamp is within acceptable window (5 minutes)
4. Recompute signature and compare in constant time
5. If valid, make tenant context available to Basic Memory tools
### 4. Security Properties
- **Authenticity**: Only services with shared secret can create valid signatures
- **Integrity**: Header tampering invalidates signature
- **Replay Protection**: Timestamp prevents reuse of old signatures
- **Non-repudiation**: Each request is cryptographically tied to specific tenant
### 5. Configuration
```bash
# Shared across Cloud Proxy and Tenant instances
BM_TENANT_HEADER_SECRET=randomly-generated-256-bit-secret
# Tenant API configuration
BM_TENANT_HEADER_VALIDATION=true # true (production) | false (dev only)
```
## How to Evaluate
### Unit Tests
- [ ] Header signing utility generates correct signatures
- [ ] Header validation correctly accepts/rejects signatures
- [ ] Timestamp validation within acceptable windows
- [ ] Constant-time signature comparison prevents timing attacks
### Integration Tests
- [ ] End-to-end request flow from MCP client → proxy → tenant
- [ ] Tenant isolation verified with signed headers
- [ ] Error handling for missing/invalid headers
- [ ] Disabled validation in development environment
### Security Validation
- [ ] Shared secret rotation procedure
- [ ] Header tampering detection
- [ ] Clock skew tolerance testing
- [ ] Performance impact measurement
### Production Readiness
- [ ] Logging and monitoring of header validation
- [ ] Graceful degradation for header validation failures
- [ ] Documentation for secret management
- [ ] Deployment configuration templates
## Implementation Notes
### Shared Secret Management
- Generate cryptographically secure 256-bit secret
- Same secret deployed to Cloud Proxy and all Tenant instances
- Consider secret rotation strategy for production
### Error Handling
```python
# Strict mode (production)
if not validate_headers(request):
raise HTTPException(status_code=401, detail="Invalid tenant headers")
# Fallback mode (development)
if not validate_headers(request):
logger.warning("Invalid headers, falling back to default tenant")
tenant_id = "default"
```
### Performance Considerations
- HMAC-SHA256 computation is fast (~microseconds)
- Headers add ~200 bytes to each request
- Validation happens once per request in middleware
## Benefits
**Works with MCP dynamic client registration** - No dependency on JWT custom claims
**Simple and reliable** - Standard HMAC signature approach
**Secure by design** - Cryptographic authenticity and integrity
**Infrastructure controlled** - No external service dependencies
**Easy to implement** - Clear signature algorithm and validation
## Trade-offs
⚠️ **Shared secret management** - Need secure distribution and rotation
⚠️ **Clock synchronization** - Timestamp validation requires reasonably synced clocks
⚠️ **Header visibility** - Headers visible in logs (tenant_id not sensitive)
⚠️ **Additional complexity** - More moving parts in proxy forwarding
## Implementation Tasks
### Cloud Service (Header Signing)
- [ ] Create `utils/header_signing.py` with HMAC-SHA256 signing function
- [ ] Add `bm_tenant_header_secret` to Cloud service configuration
- [ ] Update `ProxyService.forward_request()` to call signing utility
- [ ] Add signed headers (X-BM-Tenant-ID, X-BM-Timestamp, X-BM-Signature)
### Tenant API (Header Validation)
- [ ] Create `utils/header_validation.py` with signature verification
- [ ] Add `bm_tenant_header_secret` to API service configuration
- [ ] Create `TenantHeaderValidationMiddleware` class
- [ ] Add middleware to FastAPI app (before other middleware)
- [ ] Skip validation for `/health` endpoint
- [ ] Store validated tenant_id in request.state
### Testing
- [ ] Unit test for header signing utility
- [ ] Unit test for header validation utility
- [ ] Integration test for proxy → tenant flow
- [ ] Test invalid/missing header handling
- [ ] Test timestamp window validation
- [ ] Test signature tampering detection
### Configuration & Deployment
- [ ] Update `.env.example` with BM_TENANT_HEADER_SECRET
- [ ] Generate secure 256-bit secret for production
- [ ] Update Fly.io secrets for both services
- [ ] Document secret rotation procedure
## Status
- [x] **Specification Complete** - Design finalized and documented
- [ ] **Implementation Started** - Header signing utility development
- [ ] **Cloud Proxy Updated** - ProxyService adds signed headers
- [ ] **Tenant Validation Added** - Middleware validates headers
- [ ] **Testing Complete** - All validation criteria met
- [ ] **Production Deployed** - Live with tenant isolation via headers
@@ -0,0 +1,390 @@
---
title: 'SPEC-9-1 Follow-Ups: Conflict, Sync, and Observability'
type: tasklist
permalink: specs/spec-9-follow-ups-conflict-sync-and-observability
related: specs/spec-9-multi-project-bisync
status: revised
revision_date: 2025-10-03
---
# SPEC-9-1 Follow-Ups: Conflict, Sync, and Observability
**REVISED 2025-10-03:** Simplified to leverage rclone built-ins instead of custom conflict handling.
**Context:** SPEC-9 delivered multi-project bidirectional sync and a unified CLI. This follow-up focuses on **observability and safety** using rclone's built-in capabilities rather than reinventing conflict handling.
**Design Philosophy: "Be Dumb Like Git"**
- Let rclone bisync handle conflict detection (it already does this)
- Make conflicts visible and recoverable, don't prevent them
- Cloud is always the winner on conflict (cloud-primary model)
- Users who want version history can just use Git locally in their sync directory
**What Changed from Original Version:**
- **Replaced:** Custom `.bmmeta` sidecars → Use rclone's `.bisync/` state tracking
- **Replaced:** Custom conflict detection → Use rclone bisync 3-way merge
- **Replaced:** Tombstone files → rclone delete tracking handles this
- **Replaced:** Distributed lease → Local process lock only (document multi-device warning)
- **Replaced:** S3 versioning service → Users just use Git locally if they want history
- **Deferred:** SPEC-14 Git integration → Postponed to teams/multi-user features
## ✅ Now
- [ ] **Local process lock**: Prevent concurrent bisync runs on same device (`~/.basic-memory/sync.lock`)
- [ ] **Structured sync reports**: Parse rclone bisync output into JSON reports (creates/updates/deletes/conflicts, bytes, duration); `bm sync --report`
- [ ] **Multi-device warning**: Document that users should not run `--watch` on multiple devices simultaneously
- [ ] **Version control guidance**: Document pattern for users to use Git locally in their sync directory if they want version history
- [ ] **Docs polish**: cloud-mode toggle, mount↔bisync directory isolation, conflict semantics, quick start, migration guide, short demo clip/GIF
## 🔜 Next
- [ ] **Observability commands**: `bm conflicts list`, `bm sync history` to view sync reports and conflicts
- [ ] **Conflict resolution UI**: `bm conflicts resolve <file>` to interactively pick winner from conflict files
- [ ] **Selective sync**: allow include/exclude by project; per-project profile (safe/balanced/fast)
## 🧭 Later
- [ ] **Near real-time sync**: File watcher → targeted `rclone copy` for individual files (keep bisync as backstop)
- [ ] **Sharing / scoped tokens**: cross-tenant/project access
- [ ] **Bandwidth controls & backpressure**: policy for large repos
- [ ] **Client-side encryption (optional)**: with clear trade-offs
## 📏 Acceptance criteria (for "Now" items)
- [ ] Local process lock prevents concurrent bisync runs on same device
- [ ] rclone bisync conflict files visible and documented (`file.conflict1.md`, `file.conflict2.md`)
- [ ] `bm sync --report` generates parsable JSON with sync statistics
- [ ] Documentation clearly warns about multi-device `--watch` mode
- [ ] Documentation shows users how to use Git locally for version history
## What We're NOT Building (Deferred to rclone)
- ❌ Custom `.bmmeta` sidecars (rclone tracks state in `.bisync/` workdir)
- ❌ Custom conflict detection (rclone bisync already does 3-way merge detection)
- ❌ Tombstone files (S3 versioning + rclone delete tracking handles this)
- ❌ Distributed lease (low probability issue, rclone detects state divergence)
- ❌ Rename/move tracking (rclone has size+modtime heuristics built-in)
## Implementation Summary
**Current State (SPEC-9):**
- ✅ rclone bisync with 3 profiles (safe/balanced/fast)
-`--max-delete` safety limits (10/25/50 files)
-`--conflict-resolve=newer` for auto-resolution
- ✅ Watch mode: `bm sync --watch` (60s intervals)
- ✅ Integrity checking: `bm cloud check`
- ✅ Mount vs bisync directory isolation
**What's Needed (This Spec):**
1. **Process lock** - Simple file-based lock in `~/.basic-memory/sync.lock`
2. **Sync reports** - Parse rclone output, save to `~/.basic-memory/sync-history/`
3. **Documentation** - Multi-device warnings, conflict resolution workflow, Git usage pattern
**User Model:**
- Cloud is always the winner on conflict (cloud-primary)
- rclone creates `.conflict` files for divergent edits
- Users who want version history just use Git in their local sync directory
- Users warned: don't run `--watch` on multiple devices
## Decision Rationale & Trade-offs
### Why Trust rclone Instead of Custom Conflict Handling?
**rclone bisync already provides:**
- 3-way merge detection (compares local, remote, and last-known state)
- File state tracking in `.bisync/` workdir (hashes, modtimes)
- Automatic conflict file creation: `file.conflict1.md`, `file.conflict2.md`
- Rename detection via size+modtime heuristics
- Delete tracking (prevents resurrection of deleted files)
- Battle-tested with extensive edge case handling
**What we'd have to build with custom approach:**
- Per-file metadata tracking (`.bmmeta` sidecars)
- 3-way diff algorithm
- Conflict detection logic
- Tombstone files for deletes
- Rename/move detection
- Testing for all edge cases
**Decision:** Use what rclone already does well. Don't reinvent the wheel.
### Why Let Users Use Git Locally Instead of Building Versioning?
**The simplest solution: Just use Git**
Users who want version history can literally just use Git in their sync directory:
```bash
cd ~/basic-memory-cloud-sync/
git init
git add .
git commit -m "backup"
# Push to their own GitHub if they want
git remote add origin git@github.com:user/my-knowledge.git
git push
```
**Why this is perfect:**
- ✅ We build nothing
- ✅ Users who want Git... just use Git
- ✅ Users who don't care... don't need to
- ✅ rclone bisync already handles sync conflicts
- ✅ Users own their data, they can version it however they want (Git, Time Machine, etc.)
**What we'd have to build for S3 versioning:**
- API to enable versioning on Tigris buckets
- **Problem**: Tigris doesn't support S3 bucket versioning
- Restore commands: `bm cloud restore --version-id`
- Version listing: `bm cloud versions <path>`
- Lifecycle policies for version retention
- Documentation and user education
**What we'd have to build for SPEC-14 Git integration:**
- Committer service (daemon watching `/app/data/`)
- Puller service (webhook handler for GitHub pushes)
- Git LFS for large files
- Loop prevention between Git ↔ bisync ↔ local
- Merge conflict handling at TWO layers (rclone + Git)
- Webhook infrastructure and monitoring
**Decision:** Don't build version control. Document the pattern. "The easiest problem to solve is the one you avoid."
**When to revisit:** Teams/multi-user features where server-side version control becomes necessary for collaboration.
### Why No Distributed Lease?
**Low probability issue:**
- Requires user to manually run `bm sync` on multiple devices at exact same time
- Most users run `--watch` on one primary device
- rclone bisync detects state divergence and fails safely
**Safety nets in place:**
- Local process lock prevents concurrent runs on same device
- rclone bisync aborts if bucket state changed during sync
- S3 versioning recovers from any overwrites
- Documentation warns against multi-device `--watch`
**Failure mode:**
```bash
# Device A and B sync simultaneously
Device A: bm sync → succeeds
Device B: bm sync → "Error: path has changed, run --resync"
# User fixes with resync
Device B: bm sync --resync → establishes new baseline
```
**Decision:** Document the issue, add local lock, defer distributed coordination until users report actual problems.
### Cloud-Primary Conflict Model
**User mental model:**
- Cloud is the source of truth (like Dropbox/iCloud)
- Local is working copy
- On conflict: cloud wins, local edits → `.conflict` file
- User manually picks winner
**Why this works:**
- Simpler than bidirectional merge (no automatic resolution risk)
- Matches user expectations from Dropbox
- S3 versioning provides safety net for overwrites
- Clear recovery path: restore from S3 version if needed
**Example workflow:**
```bash
# Edit file on Device A and Device B while offline
# Both devices come online and sync
Device A: bm sync
# → Pushes to cloud first, becomes canonical version
Device B: bm sync
# → Detects conflict
# → Cloud version: work/notes.md
# → Local version: work/notes.md.conflict1
# → User manually merges or picks winner
# Restore if needed
bm cloud restore work/notes.md --version-id abc123
```
## Implementation Details
### 1. Local Process Lock
```python
# ~/.basic-memory/sync.lock
import os
import psutil
from pathlib import Path
class SyncLock:
def __init__(self):
self.lock_file = Path.home() / '.basic-memory' / 'sync.lock'
def acquire(self):
if self.lock_file.exists():
pid = int(self.lock_file.read_text())
if psutil.pid_exists(pid):
raise BisyncError(
f"Sync already running (PID {pid}). "
f"Wait for completion or kill stale process."
)
# Stale lock, remove it
self.lock_file.unlink()
self.lock_file.write_text(str(os.getpid()))
def release(self):
if self.lock_file.exists():
self.lock_file.unlink()
def __enter__(self):
self.acquire()
return self
def __exit__(self, *args):
self.release()
# Usage
with SyncLock():
run_rclone_bisync()
```
### 3. Sync Report Parsing
```python
# Parse rclone bisync output
import json
from datetime import datetime
from pathlib import Path
def parse_sync_report(rclone_output: str, duration: float, exit_code: int) -> dict:
"""Parse rclone bisync output into structured report."""
# rclone bisync outputs lines like:
# "Synching Path1 /local/path with Path2 remote:bucket"
# "- Path1 File was copied to Path2"
# "Bisync successful"
report = {
"timestamp": datetime.now().isoformat(),
"duration_seconds": duration,
"exit_code": exit_code,
"success": exit_code == 0,
"files_created": 0,
"files_updated": 0,
"files_deleted": 0,
"conflicts": [],
"errors": []
}
for line in rclone_output.split('\n'):
if 'was copied to' in line:
report['files_created'] += 1
elif 'was updated in' in line:
report['files_updated'] += 1
elif 'was deleted from' in line:
report['files_deleted'] += 1
elif '.conflict' in line:
report['conflicts'].append(line.strip())
elif 'ERROR' in line:
report['errors'].append(line.strip())
return report
def save_sync_report(report: dict):
"""Save sync report to history."""
history_dir = Path.home() / '.basic-memory' / 'sync-history'
history_dir.mkdir(parents=True, exist_ok=True)
timestamp = datetime.now().strftime('%Y%m%d-%H%M%S')
report_file = history_dir / f'{timestamp}.json'
report_file.write_text(json.dumps(report, indent=2))
# Usage in run_bisync()
start_time = time.time()
result = subprocess.run(bisync_cmd, capture_output=True, text=True)
duration = time.time() - start_time
report = parse_sync_report(result.stdout, duration, result.returncode)
save_sync_report(report)
if report['conflicts']:
console.print(f"[yellow]⚠ {len(report['conflicts'])} conflict(s) detected[/yellow]")
console.print("[dim]Run 'bm conflicts list' to view[/dim]")
```
### 4. User Commands
```bash
# View sync history
bm sync history
# → Lists recent syncs from ~/.basic-memory/sync-history/*.json
# → Shows: timestamp, duration, files changed, conflicts, errors
# View current conflicts
bm conflicts list
# → Scans sync directory for *.conflict* files
# → Shows: file path, conflict versions, timestamps
# Restore from S3 version
bm cloud restore work/notes.md --version-id abc123
# → Uses aws s3api get-object with version-id
# → Downloads to original path
bm cloud restore work/notes.md --timestamp "2025-10-03 14:30"
# → Lists versions, finds closest to timestamp
# → Downloads that version
# List file versions
bm cloud versions work/notes.md
# → Uses aws s3api list-object-versions
# → Shows: version-id, timestamp, size, author
# Interactive conflict resolution
bm conflicts resolve work/notes.md
# → Shows both versions side-by-side
# → Prompts: Keep local, keep cloud, merge manually, restore from S3 version
# → Cleans up .conflict files after resolution
```
## Success Metrics & Monitoring
**Phase 1 (v1) - Basic Safety:**
- [ ] Conflict detection rate < 5% of syncs (measure in telemetry)
- [ ] User can resolve conflicts within 5 minutes (UX testing)
- [ ] Documentation prevents 90% of multi-device issues
**Phase 2 (v2) - Observability:**
- [ ] 80% of users check `bm sync history` when troubleshooting
- [ ] Average time to restore from S3 version < 2 minutes
-
- [ ] Conflict resolution success rate > 95%
**What to measure:**
```python
# Telemetry in sync reports
{
"conflict_rate": conflicts / total_syncs,
"multi_device_collisions": count_state_divergence_errors,
"version_restores": count_restore_operations,
"avg_sync_duration": sum(durations) / count,
"max_delete_trips": count_max_delete_aborts
}
```
**When to add distributed lease:**
- Multi-device collision rate > 5% of syncs
- User complaints about state divergence errors
- Evidence that local lock isn't sufficient
**When to revisit Git (SPEC-14):**
- Teams feature launches (multi-user collaboration)
- Users request commit messages / audit trail
- PR-based review workflow becomes valuable
## Links
- SPEC-9: `specs/spec-9-multi-project-bisync`
- SPEC-14: `specs/spec-14-cloud-git-versioning` (deferred in favor of S3 versioning)
- rclone bisync docs: https://rclone.org/bisync/
- Tigris S3 versioning: https://www.tigrisdata.com/docs/buckets/versioning/
---
**Owner:** <assign> | **Review cadence:** weekly in standup | **Last updated:** 2025-10-03
+5 -1
View File
@@ -1,3 +1,7 @@
"""basic-memory - Local-first knowledge management combining Zettelkasten with knowledge graphs"""
__version__ = "0.5.0"
# Package version - updated by release automation
__version__ = "0.16.2"
# API version for FastAPI - independent of package version
__api_version__ = "v0"
-1
View File
@@ -1 +0,0 @@
Generic single-database configuration.
+119
View File
@@ -0,0 +1,119 @@
# A generic, single database configuration.
[alembic]
# path to migration scripts
# Use forward slashes (/) also on windows to provide an os agnostic path
script_location = .
# template used to generate migration file names; The default value is %%(rev)s_%%(slug)s
# Uncomment the line below if you want the files to be prepended with date and time
# see https://alembic.sqlalchemy.org/en/latest/tutorial.html#editing-the-ini-file
# for all available tokens
# file_template = %%(year)d_%%(month).2d_%%(day).2d_%%(hour).2d%%(minute).2d-%%(rev)s_%%(slug)s
# sys.path path, will be prepended to sys.path if present.
# defaults to the current working directory.
prepend_sys_path = .
# timezone to use when rendering the date within the migration file
# as well as the filename.
# If specified, requires the python>=3.9 or backports.zoneinfo library and tzdata library.
# Any required deps can installed by adding `alembic[tz]` to the pip requirements
# string value is passed to ZoneInfo()
# leave blank for localtime
# timezone =
# max length of characters to apply to the "slug" field
# truncate_slug_length = 40
# set to 'true' to run the environment during
# the 'revision' command, regardless of autogenerate
# revision_environment = false
# set to 'true' to allow .pyc and .pyo files without
# a source .py file to be detected as revisions in the
# versions/ directory
# sourceless = false
# version location specification; This defaults
# to migrations/versions. When using multiple version
# directories, initial revisions must be specified with --version-path.
# The path separator used here should be the separator specified by "version_path_separator" below.
# version_locations = %(here)s/bar:%(here)s/bat:migrations/versions
# version path separator; As mentioned above, this is the character used to split
# version_locations. The default within new alembic.ini files is "os", which uses os.pathsep.
# If this key is omitted entirely, it falls back to the legacy behavior of splitting on spaces and/or commas.
# Valid values for version_path_separator are:
#
# version_path_separator = :
# version_path_separator = ;
# version_path_separator = space
# version_path_separator = newline
#
# Use os.pathsep. Default configuration used for new projects.
version_path_separator = os
# set to 'true' to search source files recursively
# in each "version_locations" directory
# new in Alembic version 1.10
# recursive_version_locations = false
# the output encoding used when revision files
# are written from script.py.mako
# output_encoding = utf-8
sqlalchemy.url = driver://user:pass@localhost/dbname
[post_write_hooks]
# post_write_hooks defines scripts or Python functions that are run
# on newly generated revision scripts. See the documentation for further
# detail and examples
# format using "black" - use the console_scripts runner, against the "black" entrypoint
# hooks = black
# black.type = console_scripts
# black.entrypoint = black
# black.options = -l 79 REVISION_SCRIPT_FILENAME
# lint with attempts to fix using "ruff" - use the exec runner, execute a binary
# hooks = ruff
# ruff.type = exec
# ruff.executable = %(here)s/.venv/bin/ruff
# ruff.options = --fix REVISION_SCRIPT_FILENAME
# Logging configuration
[loggers]
keys = root,sqlalchemy,alembic
[handlers]
keys = console
[formatters]
keys = generic
[logger_root]
level = WARNING
handlers = console
qualname =
[logger_sqlalchemy]
level = WARNING
handlers =
qualname = sqlalchemy.engine
[logger_alembic]
level = INFO
handlers =
qualname = alembic
[handler_console]
class = StreamHandler
args = (sys.stderr,)
level = NOTSET
formatter = generic
[formatter_generic]
format = %(levelname)-5.5s [%(name)s] %(message)s
datefmt = %H:%M:%S
+124 -18
View File
@@ -1,22 +1,52 @@
"""Alembic environment configuration."""
import asyncio
import os
from logging.config import fileConfig
from sqlalchemy import engine_from_config
from sqlalchemy import pool
# Allow nested event loops (needed for pytest-asyncio and other async contexts)
# Note: nest_asyncio doesn't work with uvloop, so we handle that case separately
try:
import nest_asyncio
nest_asyncio.apply()
except (ImportError, ValueError):
# nest_asyncio not available or can't patch this loop type (e.g., uvloop)
pass
from sqlalchemy import engine_from_config, pool
from sqlalchemy.ext.asyncio import AsyncEngine, create_async_engine
from alembic import context
from basic_memory.models import Base
from basic_memory.config import config as app_config
from basic_memory.config import ConfigManager
# set config.env to "test" for pytest to prevent logging to file in utils.setup_logging()
os.environ["BASIC_MEMORY_ENV"] = "test"
# Import after setting environment variable # noqa: E402
from basic_memory.models import Base # noqa: E402
# this is the Alembic Config object, which provides
# access to the values within the .ini file in use.
config = context.config
# Set the SQLAlchemy URL from our app config
sqlalchemy_url = f"sqlite:///{app_config.database_path}"
config.set_main_option("sqlalchemy.url", sqlalchemy_url)
# Load app config - this will read environment variables (BASIC_MEMORY_DATABASE_BACKEND, etc.)
# due to Pydantic's env_prefix="BASIC_MEMORY_" setting
app_config = ConfigManager().config
# Set the SQLAlchemy URL based on database backend configuration
# If the URL is already set in config (e.g., from run_migrations), use that
# Otherwise, get it from app config
# Note: alembic.ini has a placeholder URL "driver://user:pass@localhost/dbname" that we need to override
current_url = config.get_main_option("sqlalchemy.url")
if not current_url or current_url == "driver://user:pass@localhost/dbname":
from basic_memory.db import DatabaseType
sqlalchemy_url = DatabaseType.get_db_url(
app_config.database_path, DatabaseType.FILESYSTEM, app_config
)
config.set_main_option("sqlalchemy.url", sqlalchemy_url)
# Interpret the config file for Python logging.
if config.config_file_name is not None:
@@ -27,6 +57,14 @@ if config.config_file_name is not None:
target_metadata = Base.metadata
# Add this function to tell Alembic what to include/exclude
def include_object(object, name, type_, reflected, compare_to):
# Ignore SQLite FTS tables
if type_ == "table" and name.startswith("search_index"):
return False
return True
def run_migrations_offline() -> None:
"""Run migrations in 'offline' mode.
@@ -44,29 +82,97 @@ def run_migrations_offline() -> None:
target_metadata=target_metadata,
literal_binds=True,
dialect_opts={"paramstyle": "named"},
include_object=include_object,
render_as_batch=True,
)
with context.begin_transaction():
context.run_migrations()
def do_run_migrations(connection):
"""Execute migrations with the given connection."""
context.configure(
connection=connection,
target_metadata=target_metadata,
include_object=include_object,
render_as_batch=True,
compare_type=True,
)
with context.begin_transaction():
context.run_migrations()
async def run_async_migrations(connectable):
"""Run migrations asynchronously with AsyncEngine."""
async with connectable.connect() as connection:
await connection.run_sync(do_run_migrations)
await connectable.dispose()
def run_migrations_online() -> None:
"""Run migrations in 'online' mode.
In this scenario we need to create an Engine
and associate a connection with the context.
Supports both sync engines (SQLite) and async engines (PostgreSQL with asyncpg).
"""
connectable = engine_from_config(
config.get_section(config.config_ini_section, {}),
prefix="sqlalchemy.",
poolclass=pool.NullPool,
)
# Check if a connection/engine was provided (e.g., from run_migrations)
connectable = context.config.attributes.get("connection", None)
with connectable.connect() as connection:
context.configure(connection=connection, target_metadata=target_metadata)
if connectable is None:
# No connection provided, create engine from config
url = context.config.get_main_option("sqlalchemy.url")
with context.begin_transaction():
context.run_migrations()
# Check if it's an async URL (sqlite+aiosqlite or postgresql+asyncpg)
if url and ("+asyncpg" in url or "+aiosqlite" in url):
# Create async engine for asyncpg or aiosqlite
connectable = create_async_engine(
url,
poolclass=pool.NullPool,
future=True,
)
else:
# Create sync engine for regular sqlite or postgresql
connectable = engine_from_config(
context.config.get_section(context.config.config_ini_section, {}),
prefix="sqlalchemy.",
poolclass=pool.NullPool,
)
# Handle async engines (PostgreSQL with asyncpg)
if isinstance(connectable, AsyncEngine):
# Try to run async migrations
# nest_asyncio allows asyncio.run() from within event loops, but doesn't work with uvloop
try:
asyncio.run(run_async_migrations(connectable))
except RuntimeError as e:
if "cannot be called from a running event loop" in str(e):
# We're in a running event loop (likely uvloop) - need to use a different approach
# Create a new thread to run the async migrations
import concurrent.futures
def run_in_thread():
"""Run async migrations in a new event loop in a separate thread."""
new_loop = asyncio.new_event_loop()
asyncio.set_event_loop(new_loop)
try:
new_loop.run_until_complete(run_async_migrations(connectable))
finally:
new_loop.close()
with concurrent.futures.ThreadPoolExecutor() as executor:
future = executor.submit(run_in_thread)
future.result() # Wait for completion and re-raise any exceptions
else:
raise
else:
# Handle sync engines (SQLite) or sync connections
if hasattr(connectable, "connect"):
# It's an engine, get a connection
with connectable.connect() as connection:
do_run_migrations(connection)
else:
# It's already a connection
do_run_migrations(connectable)
if context.is_offline_mode():
+4 -9
View File
@@ -1,6 +1,5 @@
"""Functions for managing database migrations."""
import asyncio
from pathlib import Path
from loguru import logger
from alembic.config import Config
@@ -10,20 +9,16 @@ from alembic import command
def get_alembic_config() -> Config: # pragma: no cover
"""Get alembic config with correct paths."""
migrations_path = Path(__file__).parent
alembic_ini = migrations_path.parent.parent.parent / "alembic.ini"
alembic_ini = migrations_path / "alembic.ini"
config = Config(alembic_ini)
config.set_main_option("script_location", str(migrations_path))
return config
async def reset_database(): # pragma: no cover
def reset_database(): # pragma: no cover
"""Drop and recreate all tables."""
logger.info("Resetting database...")
config = get_alembic_config()
def _reset(cfg):
command.downgrade(cfg, "base")
command.upgrade(cfg, "head")
await asyncio.get_event_loop().run_in_executor(None, _reset, config)
command.downgrade(config, "base")
command.upgrade(config, "head")
@@ -0,0 +1,131 @@
"""Add Postgres full-text search support with tsvector and GIN indexes
Revision ID: 314f1ea54dc4
Revises: e7e1f4367280
Create Date: 2025-11-15 18:05:01.025405
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
# revision identifiers, used by Alembic.
revision: str = "314f1ea54dc4"
down_revision: Union[str, None] = "e7e1f4367280"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
"""Add PostgreSQL full-text search support.
This migration:
1. Creates search_index table for Postgres (SQLite uses FTS5 virtual table)
2. Adds generated tsvector column for full-text search
3. Creates GIN index on the tsvector column for fast text queries
4. Creates GIN index on metadata JSONB column for fast containment queries
Note: These changes only apply to Postgres. SQLite continues to use FTS5 virtual tables.
"""
# Check if we're using Postgres
connection = op.get_bind()
if connection.dialect.name == "postgresql":
# Create search_index table for Postgres
# For SQLite, this is a FTS5 virtual table created elsewhere
from sqlalchemy.dialects.postgresql import JSONB
op.create_table(
"search_index",
sa.Column("id", sa.Integer(), nullable=False), # Entity IDs are integers
sa.Column("project_id", sa.Integer(), nullable=False), # Multi-tenant isolation
sa.Column("title", sa.Text(), nullable=True),
sa.Column("content_stems", sa.Text(), nullable=True),
sa.Column("content_snippet", sa.Text(), nullable=True),
sa.Column("permalink", sa.String(), nullable=True), # Nullable for non-markdown files
sa.Column("file_path", sa.String(), nullable=True),
sa.Column("type", sa.String(), nullable=True),
sa.Column("from_id", sa.Integer(), nullable=True), # Relation IDs are integers
sa.Column("to_id", sa.Integer(), nullable=True), # Relation IDs are integers
sa.Column("relation_type", sa.String(), nullable=True),
sa.Column("entity_id", sa.Integer(), nullable=True), # Entity IDs are integers
sa.Column("category", sa.String(), nullable=True),
sa.Column("metadata", JSONB(), nullable=True), # Use JSONB for Postgres
sa.Column("created_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=True),
sa.PrimaryKeyConstraint(
"id", "type", "project_id"
), # Composite key: id can repeat across types
sa.ForeignKeyConstraint(
["project_id"],
["project.id"],
name="fk_search_index_project_id",
ondelete="CASCADE",
),
if_not_exists=True,
)
# Create index on project_id for efficient multi-tenant queries
op.create_index(
"ix_search_index_project_id",
"search_index",
["project_id"],
unique=False,
)
# Create unique partial index on permalink for markdown files
# Non-markdown files don't have permalinks, so we use a partial index
op.execute("""
CREATE UNIQUE INDEX uix_search_index_permalink_project
ON search_index (permalink, project_id)
WHERE permalink IS NOT NULL
""")
# Add tsvector column as a GENERATED ALWAYS column
# This automatically updates when title or content_stems change
op.execute("""
ALTER TABLE search_index
ADD COLUMN textsearchable_index_col tsvector
GENERATED ALWAYS AS (
to_tsvector('english',
coalesce(title, '') || ' ' ||
coalesce(content_stems, '')
)
) STORED
""")
# Create GIN index on tsvector column for fast full-text search
op.create_index(
"idx_search_index_fts",
"search_index",
["textsearchable_index_col"],
unique=False,
postgresql_using="gin",
)
# Create GIN index on metadata JSONB for fast containment queries
# Using jsonb_path_ops for smaller index size and better performance
op.execute("""
CREATE INDEX idx_search_index_metadata_gin
ON search_index
USING GIN (metadata jsonb_path_ops)
""")
def downgrade() -> None:
"""Remove PostgreSQL full-text search support."""
connection = op.get_bind()
if connection.dialect.name == "postgresql":
# Drop indexes first
op.execute("DROP INDEX IF EXISTS idx_search_index_metadata_gin")
op.drop_index("idx_search_index_fts", table_name="search_index")
op.execute("DROP INDEX IF EXISTS uix_search_index_permalink_project")
op.drop_index("ix_search_index_project_id", table_name="search_index")
# Drop the generated column
op.execute("ALTER TABLE search_index DROP COLUMN IF EXISTS textsearchable_index_col")
# Drop the search_index table
op.drop_table("search_index")
@@ -0,0 +1,51 @@
"""remove required from entity.permalink
Revision ID: 502b60eaa905
Revises: b3c3938bacdb
Create Date: 2025-02-24 13:33:09.790951
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
# revision identifiers, used by Alembic.
revision: str = "502b60eaa905"
down_revision: Union[str, None] = "b3c3938bacdb"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
# ### commands auto generated by Alembic - please adjust! ###
with op.batch_alter_table("entity", schema=None) as batch_op:
batch_op.alter_column("permalink", existing_type=sa.VARCHAR(), nullable=True)
batch_op.drop_index("ix_entity_permalink")
batch_op.create_index(batch_op.f("ix_entity_permalink"), ["permalink"], unique=False)
batch_op.drop_constraint("uix_entity_permalink", type_="unique")
batch_op.create_index(
"uix_entity_permalink",
["permalink"],
unique=True,
sqlite_where=sa.text("content_type = 'text/markdown' AND permalink IS NOT NULL"),
)
# ### end Alembic commands ###
def downgrade() -> None:
# ### commands auto generated by Alembic - please adjust! ###
with op.batch_alter_table("entity", schema=None) as batch_op:
batch_op.drop_index(
"uix_entity_permalink",
sqlite_where=sa.text("content_type = 'text/markdown' AND permalink IS NOT NULL"),
)
batch_op.create_unique_constraint("uix_entity_permalink", ["permalink"])
batch_op.drop_index(batch_op.f("ix_entity_permalink"))
batch_op.create_index("ix_entity_permalink", ["permalink"], unique=1)
batch_op.alter_column("permalink", existing_type=sa.VARCHAR(), nullable=False)
# ### end Alembic commands ###
@@ -0,0 +1,120 @@
"""add projects table
Revision ID: 5fe1ab1ccebe
Revises: cc7172b46608
Create Date: 2025-05-14 09:05:18.214357
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
# revision identifiers, used by Alembic.
revision: str = "5fe1ab1ccebe"
down_revision: Union[str, None] = "cc7172b46608"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
# ### commands auto generated by Alembic - please adjust! ###
# SQLite FTS5 virtual table handling is SQLite-specific
# For Postgres, search_index is a regular table managed by ORM
connection = op.get_bind()
is_sqlite = connection.dialect.name == "sqlite"
op.create_table(
"project",
sa.Column("id", sa.Integer(), nullable=False),
sa.Column("name", sa.String(), nullable=False),
sa.Column("description", sa.Text(), nullable=True),
sa.Column("permalink", sa.String(), nullable=False),
sa.Column("path", sa.String(), nullable=False),
sa.Column("is_active", sa.Boolean(), nullable=False),
sa.Column("is_default", sa.Boolean(), nullable=True),
sa.Column("created_at", sa.DateTime(), nullable=False),
sa.Column("updated_at", sa.DateTime(), nullable=False),
sa.PrimaryKeyConstraint("id"),
sa.UniqueConstraint("is_default"),
sa.UniqueConstraint("name"),
sa.UniqueConstraint("permalink"),
if_not_exists=True,
)
with op.batch_alter_table("project", schema=None) as batch_op:
batch_op.create_index(
"ix_project_created_at", ["created_at"], unique=False, if_not_exists=True
)
batch_op.create_index("ix_project_name", ["name"], unique=True, if_not_exists=True)
batch_op.create_index("ix_project_path", ["path"], unique=False, if_not_exists=True)
batch_op.create_index(
"ix_project_permalink", ["permalink"], unique=True, if_not_exists=True
)
batch_op.create_index(
"ix_project_updated_at", ["updated_at"], unique=False, if_not_exists=True
)
with op.batch_alter_table("entity", schema=None) as batch_op:
batch_op.add_column(sa.Column("project_id", sa.Integer(), nullable=False))
batch_op.drop_index(
"uix_entity_permalink",
sqlite_where=sa.text("content_type = 'text/markdown' AND permalink IS NOT NULL")
if is_sqlite
else None,
)
batch_op.drop_index("ix_entity_file_path")
batch_op.create_index(batch_op.f("ix_entity_file_path"), ["file_path"], unique=False)
batch_op.create_index("ix_entity_project_id", ["project_id"], unique=False)
batch_op.create_index(
"uix_entity_file_path_project", ["file_path", "project_id"], unique=True
)
batch_op.create_index(
"uix_entity_permalink_project",
["permalink", "project_id"],
unique=True,
sqlite_where=sa.text("content_type = 'text/markdown' AND permalink IS NOT NULL")
if is_sqlite
else None,
)
batch_op.create_foreign_key("fk_entity_project_id", "project", ["project_id"], ["id"])
# drop the search index table. it will be recreated
# Only drop for SQLite - Postgres uses regular table managed by ORM
if is_sqlite:
op.drop_table("search_index")
# ### end Alembic commands ###
def downgrade() -> None:
# ### commands auto generated by Alembic - please adjust! ###
with op.batch_alter_table("entity", schema=None) as batch_op:
batch_op.drop_constraint("fk_entity_project_id", type_="foreignkey")
batch_op.drop_index(
"uix_entity_permalink_project",
sqlite_where=sa.text("content_type = 'text/markdown' AND permalink IS NOT NULL"),
)
batch_op.drop_index("uix_entity_file_path_project")
batch_op.drop_index("ix_entity_project_id")
batch_op.drop_index(batch_op.f("ix_entity_file_path"))
batch_op.create_index("ix_entity_file_path", ["file_path"], unique=1)
batch_op.create_index(
"uix_entity_permalink",
["permalink"],
unique=1,
sqlite_where=sa.text("content_type = 'text/markdown' AND permalink IS NOT NULL"),
)
batch_op.drop_column("project_id")
with op.batch_alter_table("project", schema=None) as batch_op:
batch_op.drop_index("ix_project_updated_at")
batch_op.drop_index("ix_project_permalink")
batch_op.drop_index("ix_project_path")
batch_op.drop_index("ix_project_name")
batch_op.drop_index("ix_project_created_at")
op.drop_table("project")
# ### end Alembic commands ###
@@ -0,0 +1,112 @@
"""project constraint fix
Revision ID: 647e7a75e2cd
Revises: 5fe1ab1ccebe
Create Date: 2025-06-03 12:48:30.162566
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
# revision identifiers, used by Alembic.
revision: str = "647e7a75e2cd"
down_revision: Union[str, None] = "5fe1ab1ccebe"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
"""Remove the problematic UNIQUE constraint on is_default column.
The UNIQUE constraint prevents multiple projects from having is_default=FALSE,
which breaks project creation when the service sets is_default=False.
SQLite: Recreate the table without the constraint (no ALTER TABLE support)
Postgres: Use ALTER TABLE to drop the constraint directly
"""
connection = op.get_bind()
is_sqlite = connection.dialect.name == "sqlite"
if is_sqlite:
# For SQLite, we need to recreate the table without the UNIQUE constraint
# Create a new table without the UNIQUE constraint on is_default
op.create_table(
"project_new",
sa.Column("id", sa.Integer(), nullable=False),
sa.Column("name", sa.String(), nullable=False),
sa.Column("description", sa.Text(), nullable=True),
sa.Column("permalink", sa.String(), nullable=False),
sa.Column("path", sa.String(), nullable=False),
sa.Column("is_active", sa.Boolean(), nullable=False),
sa.Column("is_default", sa.Boolean(), nullable=True), # No UNIQUE constraint!
sa.Column("created_at", sa.DateTime(), nullable=False),
sa.Column("updated_at", sa.DateTime(), nullable=False),
sa.PrimaryKeyConstraint("id"),
sa.UniqueConstraint("name"),
sa.UniqueConstraint("permalink"),
)
# Copy data from old table to new table
op.execute("INSERT INTO project_new SELECT * FROM project")
# Drop the old table
op.drop_table("project")
# Rename the new table
op.rename_table("project_new", "project")
# Recreate the indexes
with op.batch_alter_table("project", schema=None) as batch_op:
batch_op.create_index("ix_project_created_at", ["created_at"], unique=False)
batch_op.create_index("ix_project_name", ["name"], unique=True)
batch_op.create_index("ix_project_path", ["path"], unique=False)
batch_op.create_index("ix_project_permalink", ["permalink"], unique=True)
batch_op.create_index("ix_project_updated_at", ["updated_at"], unique=False)
else:
# For Postgres, we can simply drop the constraint
with op.batch_alter_table("project", schema=None) as batch_op:
batch_op.drop_constraint("project_is_default_key", type_="unique")
def downgrade() -> None:
"""Add back the UNIQUE constraint on is_default column.
WARNING: This will break project creation again if multiple projects
have is_default=FALSE.
"""
# Recreate the table with the UNIQUE constraint
op.create_table(
"project_old",
sa.Column("id", sa.Integer(), nullable=False),
sa.Column("name", sa.String(), nullable=False),
sa.Column("description", sa.Text(), nullable=True),
sa.Column("permalink", sa.String(), nullable=False),
sa.Column("path", sa.String(), nullable=False),
sa.Column("is_active", sa.Boolean(), nullable=False),
sa.Column("is_default", sa.Boolean(), nullable=True),
sa.Column("created_at", sa.DateTime(), nullable=False),
sa.Column("updated_at", sa.DateTime(), nullable=False),
sa.PrimaryKeyConstraint("id"),
sa.UniqueConstraint("is_default"), # Add back the problematic constraint
sa.UniqueConstraint("name"),
sa.UniqueConstraint("permalink"),
)
# Copy data (this may fail if multiple FALSE values exist)
op.execute("INSERT INTO project_old SELECT * FROM project")
# Drop the current table and rename
op.drop_table("project")
op.rename_table("project_old", "project")
# Recreate indexes
with op.batch_alter_table("project", schema=None) as batch_op:
batch_op.create_index("ix_project_created_at", ["created_at"], unique=False)
batch_op.create_index("ix_project_name", ["name"], unique=True)
batch_op.create_index("ix_project_path", ["path"], unique=False)
batch_op.create_index("ix_project_permalink", ["permalink"], unique=True)
batch_op.create_index("ix_project_updated_at", ["updated_at"], unique=False)

Some files were not shown because too many files have changed in this diff Show More