Commit Graph

948 Commits

Author SHA1 Message Date
phernandez 045e931b49 fix: use Path comparison for cross-platform rclone command test
- Compare Path objects instead of strings in test_project_sync_success
- Fixes Windows test failure where paths use backslashes
- Ensures test passes on Windows, Linux, and macOS

Signed-off-by: Claude <noreply@anthropic.com>
Signed-off-by: phernandez <paul@basicmachines.co>
2025-10-30 11:14:00 -05:00
phernandez d754cf9980 fix: use cross-platform path comparisons in Windows tests
- Use .as_posix() for path comparison (Windows uses backslashes)
- Use Path.is_absolute() instead of checking for '/' prefix
- Ensures tests pass on Windows, Linux, and macOS

Signed-off-by: Claude <noreply@anthropic.com>
Signed-off-by: phernandez <paul@basicmachines.co>
2025-10-30 10:10:36 -05:00
phernandez bc37ecf64c fix: handle project removal for cloud-only projects
**Problem:** Cloud projects that exist only in the database (not in
config.json) couldn't be removed. The error "Project 'name' not found"
occurred because config validation failed before database deletion.

**Root cause:**
- In cloud mode, projects can exist in database without config entries
- ProjectService.remove_project() called config_manager.remove_project()
  first, which raised ValueError if project wasn't in config
- This prevented removal of cloud-only projects

**Solution:**
1. Check database first for project existence (source of truth)
2. Validate default project status from both database and config
3. Try to remove from config, but catch ValueError if not found
4. Always remove from database if project exists there

**Additional fix:**
- Project list Default column now shows in local mode (always) and in
  cloud mode only if default_project_mode is enabled
- This fixes integration tests that expect to see default marker

Fixes cloud-only project deletion bug.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: phernandez <paul@basicmachines.co>
2025-10-28 17:49:06 -05:00
phernandez b049c5cbc3 test: fix rclone command tests for path normalization
Update test expectations to match new path normalization behavior:
- API returns normalized paths (without /app/data/ prefix)
- get_project_remote() strips /app/data/ if present
- Remote paths are now basic-memory-cloud:bucket/research
  instead of basic-memory-cloud:bucket/app/data/research

This reflects the fix for path doubling bug where files were
syncing to /app/data/app/data/project/.

All 22 rclone tests passing.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: phernandez <paul@basicmachines.co>
2025-10-28 17:41:51 -05:00
phernandez d749c7737f feat: SPEC-20 enhancements - cleanup, path normalization, and docs
This commit adds several critical improvements discovered during
manual testing of SPEC-20 project-scoped rclone sync:

**Critical Bug Fixes:**

1. Path Normalization (fixes path doubling bug)
   - API: Strip /app/data/ prefix in project_router.py
   - CLI: Defensive normalization in project.py
   - Rclone: Fix get_project_remote() path construction
   - Prevents files syncing to /app/data/app/data/project/

2. Rclone Flag Fix
   - Changed --filters-file to correct --filter-from flag
   - Fixes "unknown flag" error in sync and bisync

**Enhancements:**

3. Automatic Database Sync
   - POST to /{project}/project/sync after file operations
   - Keeps database in sync with files automatically
   - Skipped on --dry-run operations

4. Enhanced Project Removal
   - Clean up local sync directory (with --delete-notes)
   - Always remove bisync state directory
   - Always remove cloud_projects config entry
   - Informative messages about what was/wasn't deleted

5. Bisync State Reset Command
   - New: bm project bisync-reset <project>
   - Clears corrupted bisync metadata
   - Safe recovery tool for bisync issues

6. Improved Project List UI
   - Show Local Path column in cloud mode
   - Conditionally show/hide columns based on config
   - Prevent path truncation with no_wrap/overflow
   - Apply path normalization to display

**Documentation:**

7. Cloud CLI Documentation
   - Add troubleshooting: empty directory bisync issues
   - Add troubleshooting: bisync state corruption
   - Document bisync-reset command usage
   - Explain rclone bisync limitations

8. SPEC-20 Updates
   - Mark implementation complete
   - Document all enhancements in Implementation Notes
   - Update phase checklists with completed work
   - Add manual testing results

**Tests:**

9. Unit Tests for --local-path
   - Test config persistence with --local-path
   - Test no config without --local-path
   - Test tilde expansion
   - Test nested directory creation

All changes tested manually end-to-end.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: phernandez <paul@basicmachines.co>
2025-10-28 17:36:36 -05:00
phernandez db85186e37 fix: Remove obsolete tests for deleted sync functionality
Removed tests for:
- test_bisync_commands.py (tenant-wide bisync functions)
- test_cloud_utils.py (deprecated utilities)
- test_rclone_config.py (mount profiles)
- test_sync_commands_integration.py (removed sync command)

Fixed:
- Removed 'sync' import from commands/__init__.py

Tests now pass with SPEC-20 project-scoped architecture.

Signed-off-by: phernandez <paul@basicmachines.co>
2025-10-28 13:48:51 -05:00
phernandez da9c7028b7 docs(SPEC-20): Update cloud-cli.md for project-scoped sync
Complete rewrite to reflect SPEC-20 architecture:
- Removed outdated mount/bisync workflow confusion
- Added clear problem/solution explanations
- Documented project-scoped sync model
- Explained what happens under the covers for each command
- Added use case examples with concrete scenarios
- Updated all commands to project-scoped approach
- Removed references to deprecated features
- Added troubleshooting with explanations

Documentation now ready for user testing.

Signed-off-by: phernandez <paul@basicmachines.co>
2025-10-28 12:56:58 -05:00
phernandez daf5add9bb docs(SPEC-20): Mark Phase 5 cleanup as complete
All Phase 5 tasks completed:
- Removed mount_commands.py and all mount functionality
- Removed all tenant-wide bisync functions
- Removed bisync_config from config schema
- Simplified cloud setup command
- Removed top-level sync command
- All typecheck errors resolved

Signed-off-by: phernandez <paul@basicmachines.co>
2025-10-28 12:52:47 -05:00
phernandez 22d1a8b6c3 feat(SPEC-20): Phase 5 cleanup - Remove tenant-wide sync operations
- Removed mount_commands.py (mount functionality deprecated)
- Removed BISYNC_PROFILES and all tenant-wide bisync functions
- Removed bisync_config from config schema
- Simplified core_commands.py setup command (project-scoped workflow)
- Removed top-level sync command (confusing in cloud mode, automatic in local mode)
- Updated db.py to use run_sync() directly instead of sync command
- Cleaned up unused imports throughout

All operations now project-scoped via:
  bm project bisync --name <project>
  bm project sync --name <project>
  bm project check --name <project>

Signed-off-by: phernandez <paul@basicmachines.co>
2025-10-28 12:50:22 -05:00
phernandez c63bf1a332 docs: Mark SPEC-20 Phase 4 complete
All Phase 4 CLI integration tasks are complete:
- Added --local-path flag to project add command
- Added sync-setup, sync, bisync, check, ls commands
- Integrated with rclone_commands module
- All commands working with proper error handling

Optional tasks deferred for now:
- Update project list to show sync status
- Simplify cloud setup command
- Integration tests

Ready for Phase 5 cleanup.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: phernandez <paul@basicmachines.co>
2025-10-28 12:28:30 -05:00
phernandez 3b1cd8763b feat: Add project sync CLI commands (SPEC-20 Phase 4)
Added comprehensive CLI integration for project-scoped sync:

New commands:
- `bm project add --local-path PATH` - Add project with optional local sync
- `bm project sync-setup NAME PATH` - Configure sync for existing project
- `bm project sync --name NAME` - One-way sync (local → cloud)
- `bm project bisync --name NAME` - Two-way sync (local ↔ cloud)
- `bm project check --name NAME` - Verify file integrity
- `bm project ls --name NAME` - List remote files

Features:
- Cloud-only commands (check cloud_mode_enabled)
- Uses get_mount_info() for bucket name discovery
- Reads local_sync_path from config.cloud_projects
- Integrates with rclone_commands module
- Comprehensive error handling and user guidance
- Dry-run and verbose options for all sync commands

Command workflow:
1. bm project add research --local-path ~/docs
2. bm project bisync --name research --resync  # First time
3. bm project bisync --name research            # Subsequent syncs

All commands include helpful error messages guiding users through setup.

Related: SPEC-20 Phase 4

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: phernandez <paul@basicmachines.co>
2025-10-28 12:26:31 -05:00
phernandez ffe5aa46dc docs: Update SPEC-20 to mark Phase 2 and Phase 3 complete
Marked completed checklist items:

Phase 2 (Rclone Config Simplification):
- Updated configure_rclone_remote() to use basic-memory-cloud
- Removed add_tenant_to_rclone_config()
- Removed tenant_id from remote naming
- Tested rclone config generation
- Cleaned up deprecated import references

Phase 3 (Project-Scoped Rclone Commands):
- Created rclone_commands.py with full implementation
- Implemented all sync operations (sync, bisync, check, ls)
- Added helper functions and SyncProject dataclass
- Wrote 22 tests with 99% coverage
- Temporarily disabled mount commands

Both phases complete and tested. Ready for Phase 4.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: phernandez <paul@basicmachines.co>
2025-10-28 12:08:56 -05:00
phernandez 73065240fe feat: Add project-scoped rclone commands (SPEC-20 Phase 3)
Created new rclone_commands.py module with project-scoped sync operations:

New functionality:
- SyncProject dataclass for sync-enabled projects
- get_project_remote() - Build rclone remote paths
- project_sync() - One-way sync (local → cloud)
- project_bisync() - Two-way sync (local ↔ cloud)
- project_check() - Integrity verification
- project_ls() - List remote files
- Helper functions: get_bmignore_filter_path(), get_project_bisync_state(), bisync_initialized()

Features:
- Per-project bisync state tracking
- Balanced defaults (conflict_resolve=newer, max_delete=25)
- Automatic --resync requirement for first bisync
- Dry-run support for all operations
- Comprehensive error handling

Temporarily disabled mount commands in core_commands.py to allow tests to run.
Mount commands will be fully removed in Phase 5.

Tests: 22 functional tests with 99% coverage
Related: SPEC-20 Phase 3

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: phernandez <paul@basicmachines.co>
2025-10-28 11:55:09 -05:00
phernandez 685cccf708 refactor: Clean up deprecated rclone_config import references
Updated bisync_commands.py and core_commands.py to use simplified
configure_rclone_remote() function instead of deprecated
add_tenant_to_rclone_config(). Removed MOUNT_PROFILES import and
replaced dynamic help text with static string.

mount_commands.py still has errors but will be removed entirely in Phase 5.

Related to SPEC-20 Phase 2 cleanup.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: phernandez <paul@basicmachines.co>
2025-10-28 11:25:38 -05:00
phernandez 0887ad4f8a refactor(rclone): Clean up deprecated functions from rclone_config.py
Remove deprecated tenant-specific and mount-related functions:
- Removed add_tenant_to_rclone_config() (replaced by configure_rclone_remote)
- Removed remove_tenant_from_rclone_config()
- Removed all mount-related functions:
  - RcloneMountProfile class
  - MOUNT_PROFILES dict
  - get_default_mount_path()
  - build_mount_command()
  - is_path_mounted()
  - get_rclone_processes()
  - kill_rclone_process()
  - unmount_path()
  - cleanup_orphaned_rclone_processes()

Simplified module to only include:
- configure_rclone_remote() for single remote setup
- Core config management (load, save, backup)

Note: mount_commands.py still depends on removed code and will fail to import.
This will be fully resolved in Phase 5 when mount_commands.py is removed entirely.

Part of SPEC-20 Phase 5 (early cleanup).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: phernandez <paul@basicmachines.co>
2025-10-28 11:14:26 -05:00
phernandez 7c2b8b5a58 feat(rclone): Add simplified configure_rclone_remote() function (SPEC-20 Phase 2)
Add new configure_rclone_remote() that uses single remote name:
- Single remote: "basic-memory-cloud" (not tenant-specific)
- Simplifies from per-tenant remotes to one credential set per user
- Maintains backup_rclone_config() for safety
- Add comprehensive functional tests for rclone config
- Test remote configuration, updates, save/load operations

Old add_tenant_to_rclone_config() kept for backward compatibility
but will be removed in Phase 5.

Part of SPEC-20 Simplified Project-Scoped Rclone Sync.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: phernandez <paul@basicmachines.co>
2025-10-28 11:02:38 -05:00
phernandez 8bc550f613 feat(config): Add cloud_projects schema for project-scoped sync (SPEC-20 Phase 1)
Add CloudProjectConfig model and cloud_projects dict to BasicMemoryConfig:
- CloudProjectConfig tracks local_path, last_sync, bisync_initialized
- cloud_projects: dict[str, CloudProjectConfig] in config
- Fix datetime serialization with mode='json' in model_dump()
- Add comprehensive tests for cloud_projects functionality
- Backward compatible: old configs without cloud_projects load correctly

This enables project-scoped sync configuration tracked in config file
rather than filesystem discovery, preventing phantom projects.

Part of SPEC-20 Simplified Project-Scoped Rclone Sync.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: phernandez <paul@basicmachines.co>
2025-10-28 10:35:04 -05:00
phernandez a92741985a Add SPEC-20: Simplified Project-Scoped Rclone Sync
- Radical simplification of rclone integration
- Remove mount/bisync workflows complexity (6 profiles → 1 default)
- Introduce cloud_projects dict for sync configuration
- Project-scoped sync operations (explicit, no auto-discovery)
- Thin wrappers around rclone commands
- 74-task implementation checklist across 7 phases
- Supersedes SPEC-8 sync implementation

Signed-off-by: phernandez <paul@basicmachines.co>
2025-10-28 10:22:59 -05:00
Paul Hernandez a7d7cc5ee6 fix: Normalize YAML frontmatter types to prevent AttributeError (#236) (#402)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude <noreply@anthropic.com>
2025-10-27 09:19:50 -05:00
Paul Hernandez a7e696b039 Add free trial information to README
Added information about a 7-day free trial.

Signed-off-by: Paul Hernandez <60959+phernandez@users.noreply.github.com>
2025-10-24 10:05:11 -05:00
Paul Hernandez 8aaddb6d45 Add free trial information to README
Added information about a 7-day free trial to the README.

Signed-off-by: Paul Hernandez <60959+phernandez@users.noreply.github.com>
2025-10-24 10:04:41 -05:00
Paul Hernandez d7565312fc Announce Basic Memory Cloud launch in README
Added a section announcing the launch of Basic Memory Cloud with details on cross-device support and early supporter pricing.

Signed-off-by: Paul Hernandez <60959+phernandez@users.noreply.github.com>
2025-10-24 09:53:14 -05:00
Paul Hernandez c7e6eab02f feat: Add delete_notes parameter to remove project endpoint (#391)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude <noreply@anthropic.com>
2025-10-21 14:09:20 -05:00
Paul Hernandez bb8da31472 fix: Handle null, empty, and string 'None' title in markdown frontmatter (#387) (#389)
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Paul Hernandez <phernandez@users.noreply.github.com>
2025-10-21 09:29:19 -05:00
Paul Hernandez e78345ff25 feat: Streaming Foundation & Async I/O Consolidation (SPEC-19) (#384)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude <noreply@anthropic.com>
2025-10-21 09:03:59 -05:00
Paul Hernandez 32236cd247 fix: Handle YAML parsing errors gracefully in update_frontmatter (#378) (#379)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude <noreply@anthropic.com>
2025-10-16 20:17:58 -05:00
Paul Hernandez 4fd6d0c648 fix: Optimize sync memory usage to prevent OOM on large projects (#380)
Signed-off-by: Claude <noreply@anthropic.com>
Signed-off-by: phernandez <paul@basicmachines.co>
2025-10-16 20:17:34 -05:00
Paul Hernandez e6c8e3662c fix: preserve mtime webdav upload 376 (#377)
Signed-off-by: Claude <noreply@anthropic.com>
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude <noreply@anthropic.com>
2025-10-16 17:18:10 -05:00
Paul Hernandez 449b62d947 fix: Prevent deleted projects from being recreated by background sync (#193) (#370)
Signed-off-by: Claude <noreply@anthropic.com>
Signed-off-by: phernandez <paul@basicmachines.co>
2025-10-16 15:16:24 -05:00
Paul Hernandez b7497d7484 fix: Use filesystem timestamps for entity sync instead of database operation time (#138) (#369)
Signed-off-by: Claude <noreply@anthropic.com>
Signed-off-by: phernandez <paul@basicmachines.co>
2025-10-16 14:21:27 -05:00
Paul Hernandez d1431bdb1b fix: Handle YAML parsing errors and missing entity_type in markdown files (#368)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude <noreply@anthropic.com>
2025-10-16 13:18:58 -05:00
Paul Hernandez 171bef717f fix: Resolve UNIQUE constraint violation in entity upsert with observations (#187) (#367)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude <noreply@anthropic.com>
2025-10-16 12:30:56 -05:00
Paul Hernandez 729a5a3b8d fix: Terminate sync immediately when project is deleted (#366)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude <noreply@anthropic.com>
2025-10-16 11:07:12 -05:00
Paul Hernandez 434cdf24dd feat: Add circuit breaker for file sync failures (#364)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude <noreply@anthropic.com>
2025-10-16 09:47:48 -05:00
Paul Hernandez 7f9c1a97a4 feat: Add --verbose and --no-gitignore options to cloud upload (#362)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude <noreply@anthropic.com>
2025-10-15 20:03:36 -05:00
Paul Hernandez 53fb13b054 fix: Make project creation endpoint idempotent (#357)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude <noreply@anthropic.com>
2025-10-15 19:27:41 -05:00
Paul Hernandez bd6c8348b8 fix: Handle None text values in Claude conversations importer (#353)
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Paul Hernandez <phernandez@users.noreply.github.com>
2025-10-15 16:08:57 -05:00
phernandez 994c8b8e7e chore: update version to 0.15.2 for v0.15.2 release v0.15.2 2025-10-14 09:36:47 -05:00
phernandez a78e8c3ac5 style: Apply linter formatting changes 2025-10-14 09:34:10 -05:00
phernandez 53900c5baa fix: Project commands now respect cloud_mode at runtime
- Moved config evaluation from module load time to runtime
- Unified add_project command to handle both cloud and local modes
- Commands (default, sync-config, move) now check cloud_mode at runtime
- Fixes test failures where monkeypatch wasn't applied before command registration
2025-10-14 09:24:13 -05:00
phernandez 02c6de3387 docs: Add v0.15.2 changelog entry 2025-10-14 00:43:14 -05:00
phernandez 9ccf4b6b56 remove extra conole out from sync message after upload
Signed-off-by: phernandez <paul@basicmachines.co>
2025-10-14 00:38:02 -05:00
phernandez ba74ca7e18 fix: Update CloudProjectCreateResponse schema to match API response
The /proxy/projects/projects POST endpoint returns a ProjectStatusResponse
with fields: message, status, default, old_project, new_project.

Updated CloudProjectCreateResponse schema to match this format instead of
expecting name, path, message fields.

Also updated all related tests to use the correct response format:
- tests/cli/test_cloud_utils.py (3 tests)
- tests/cli/test_bisync_commands.py (1 test)

Fixes the validation error when creating cloud projects via upload command:
"bm cloud upload --project test --create-project specs"

Signed-off-by: Pablo Hernandez <pablo@basicmachines.co>
Signed-off-by: phernandez <paul@basicmachines.co>
2025-10-13 23:56:59 -05:00
Paul Hernandez 5258f45730 feat: Add WebDAV upload command for cloud projects (#356)
Signed-off-by: phernandez <paul@basicmachines.co>
Signed-off-by: Pablo Hernandez <pablo@basicmachines.co>
Co-authored-by: Claude <noreply@anthropic.com>
2025-10-13 23:20:54 -05:00
phernandez e773c002ce chore: update version to 0.15.1 for v0.15.1 release v0.15.1 2025-10-13 11:04:44 -05:00
phernandez e70ba944e7 docs: Add v0.15.1 changelog entry
Add comprehensive changelog for v0.15.1 release including:
- Performance improvements (43% faster sync, 10-100x faster directory ops)
- Bug fixes for cloud mode, project paths, and Claude Desktop compatibility
- New features: async client context manager, BASIC_MEMORY_PROJECT_ROOT
- Documentation updates and SPEC-15/16 additions

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-13 10:58:21 -05:00
phernandez e41579f971 Merge branch 'main' of github.com:basicmachines-co/basic-memory 2025-10-13 10:44:54 -05:00
Paul Hernandez 2b7008d997 fix: Update view_note and ChatGPT tools for Claude Desktop compatibility (#355)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude <noreply@anthropic.com>
2025-10-13 10:44:35 -05:00
phernandez 56e5cc072b Merge branch 'main' of github.com:basicmachines-co/basic-memory 2025-10-13 07:30:08 -05:00
Paul Hernandez c0538ad2dd perf: Optimize sync/indexing for 43% faster performance (#352)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude <noreply@anthropic.com>
2025-10-12 14:41:44 -05:00