basicmachines-co-basic-memory/v15-docs/cloud-bisync.md

Cloud Bidirectional Sync (SPEC-9)

Status: New Feature PR: #322 Requires: Active subscription, rclone installation

What's New

v0.15.0 introduces bidirectional cloud synchronization using rclone bisync. Your local files sync automatically with the cloud, enabling multi-device workflows, backups, and collaboration.

Quick Start

One-Time Setup

# Install and configure cloud sync
bm cloud bisync-setup

# What it does:
# 1. Installs rclone
# 2. Gets tenant credentials
# 3. Configures rclone remote
# 4. Creates sync directory
# 5. Performs initial sync

Regular Sync

# Recommended: Use standard sync command
bm sync                    # Syncs local → database
bm cloud bisync            # Syncs local ↔ cloud

# Or: Use watch mode (auto-sync every 60 seconds)
bm sync --watch

How Bidirectional Sync Works

Sync Architecture

Local Files          rclone bisync          Cloud Storage
~/basic-memory-      <─────────────>       s3://bucket/
cloud-sync/          (bidirectional)       tenant-id/
  ├── project-a/                              ├── project-a/
  ├── project-b/                              ├── project-b/
  └── notes/                                  └── notes/

Sync Profiles

Three profiles optimize for different use cases:

Profile	Conflicts	Max Deletes	Speed	Use Case
safe	Keep both versions	10	Slower	Preserve all changes, manual conflict resolution
balanced	Use newer file	25	Medium	Default - auto-resolve most conflicts
fast	Use newer file	50	Fastest	Rapid iteration, trust newer versions

Conflict Resolution

safe profile (--conflict-resolve=none):

• Conflicting files saved as file.conflict1, file.conflict2
• Manual resolution required
• No data loss

balanced/fast profiles (--conflict-resolve=newer):

• Automatically uses the newer file
• Faster syncs
• Good for single-user workflows

Commands

bm cloud bisync-setup

One-time setup for cloud sync.

bm cloud bisync-setup

# Optional: Custom sync directory
bm cloud bisync-setup --dir ~/my-sync-folder

What happens:

1. Checks for/installs rclone
2. Generates scoped S3 credentials
3. Configures rclone remote
4. Creates local sync directory
5. Performs initial baseline sync (--resync)

Configuration saved to:

• ~/.basic-memory/config.json - sync_dir path
• ~/.config/rclone/rclone.conf - remote credentials
• ~/.basic-memory/bisync-state/{tenant_id}/ - sync state

bm cloud bisync

Manual bidirectional sync.

# Basic sync (uses 'balanced' profile)
bm cloud bisync

# Choose sync profile
bm cloud bisync --profile safe
bm cloud bisync --profile balanced
bm cloud bisync --profile fast

# Dry run (preview changes)
bm cloud bisync --dry-run

# Force resync (rebuild baseline)
bm cloud bisync --resync

# Verbose output
bm cloud bisync --verbose

Auto-registration:

• Scans local directory for new projects
• Creates them on cloud before sync
• Ensures cloud knows about all local projects

bm sync (Recommended)

The standard sync command now handles both local and cloud:

# One command for everything
bm sync                    # Local sync + cloud sync
bm sync --watch            # Continuous sync every 60s

Sync Directory Structure

Default Layout

~/basic-memory-cloud-sync/     # Configurable via --dir
├── project-a/                 # Auto-created local projects
│   ├── notes/
│   ├── ideas/
│   └── .bmignore              # Respected during sync
├── project-b/
│   └── documents/
└── .basic-memory/             # Metadata (ignored in sync)

Important Paths

Path	Purpose
~/basic-memory-cloud-sync/	Default local sync directory
~/basic-memory-cloud/	Mount point (DO NOT use for bisync)
~/.basic-memory/bisync-state/{tenant_id}/	Sync state and history
~/.basic-memory/.bmignore	Patterns to exclude from sync

Critical: Bisync and mount must use different directories

File Filtering with .bmignore

Default Patterns

Basic Memory respects .bmignore patterns (gitignore format):

# ~/.basic-memory/.bmignore (default)
.git
.DS_Store
node_modules
*.tmp
.env
__pycache__
.pytest_cache
.ruff_cache
.vscode
.idea

How It Works

1. .bmignore patterns converted to rclone filter format
2. Auto-regenerated when .bmignore changes
3. Stored as ~/.basic-memory/.bmignore.rclone
4. Applied to all bisync operations

Custom Patterns

Edit ~/.basic-memory/.bmignore:

# Your custom patterns
.git
*.log
temp/
*.backup

Next sync will use updated filters.

Project Management

Auto-Registration

Bisync automatically registers new local projects:

# You create a new project locally
mkdir ~/basic-memory-cloud-sync/new-project
echo "# Hello" > ~/basic-memory-cloud-sync/new-project/README.md

# Next sync auto-creates on cloud
bm cloud bisync
# → "Found 1 new local project, creating on cloud..."
# → "✓ Created project: new-project"

Project Discovery

# List cloud projects
bm cloud status

# Shows:
# - Total projects
# - Last sync time
# - Storage used

Cloud Mode

To work with cloud projects via CLI:

# Set cloud API URL
export BASIC_MEMORY_API_URL=https://api.basicmemory.cloud

# Or in config.json:
{
  "api_url": "https://api.basicmemory.cloud"
}

# Now CLI tools work against cloud
bm sync --project new-project        # Syncs cloud project
bm tools continue-conversation --project new-project

Sync Workflow Examples

Daily Workflow

# Morning: Start watch mode
bm sync --watch &

# Work in your sync directory
cd ~/basic-memory-cloud-sync/work-notes
vim ideas.md

# Changes auto-sync every 60s
# Watch output shows sync progress

Multi-Device Workflow

Device A:

# Make changes
echo "# New Idea" > ~/basic-memory-cloud-sync/ideas/innovation.md

# Sync to cloud
bm cloud bisync
# → "✓ Sync completed - 1 file uploaded"

Device B:

# Pull changes from cloud
bm cloud bisync
# → "✓ Sync completed - 1 file downloaded"

# See the new file
cat ~/basic-memory-cloud-sync/ideas/innovation.md
# → "# New Idea"

Conflict Scenario

Using balanced profile (auto-resolve):

# Both devices edit same file
# Device A: Updated at 10:00 AM
# Device B: Updated at 10:05 AM

# Device A syncs
bm cloud bisync
# → "✓ Sync completed"

# Device B syncs
bm cloud bisync
# → "Resolving conflict: using newer version"
# → "✓ Sync completed"
# → Device B's version (10:05) wins

Using safe profile (manual resolution):

bm cloud bisync --profile safe
# → "Conflict detected: ideas.md"
# → "Saved as: ideas.md.conflict1 and ideas.md.conflict2"
# → "Please resolve manually"

# Review both versions
diff ideas.md.conflict1 ideas.md.conflict2

# Merge and cleanup
vim ideas.md  # Merge manually
rm ideas.md.conflict*

Monitoring and Status

Check Sync Status

bm cloud status

Shows:

Cloud Bisync Status
┏━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Property            ┃ Value                      ┃
┡━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ Status              │ ✓ Initialized              │
│ Local Directory     │ ~/basic-memory-cloud-sync  │
│ Remote              │ s3://bucket/tenant-id      │
│ Last Sync           │ 2 minutes ago              │
│ Total Projects      │ 5                          │
└─────────────────────┴────────────────────────────┘

Verify Integrity

bm cloud check

Compares local and cloud file hashes to detect:

• Corrupted files
• Missing files
• Sync drift

Troubleshooting

"First bisync requires --resync"

Problem: Initial sync not established

$ bm cloud bisync
Error: First bisync requires --resync to establish baseline

Solution:

bm cloud bisync --resync

"Cannot use mount directory for bisync"

Problem: Trying to use mounted directory for sync

$ bm cloud bisync --dir ~/basic-memory-cloud
Error: Cannot use ~/basic-memory-cloud for bisync - it's the mount directory!

Solution: Use different directory

bm cloud bisync --dir ~/basic-memory-cloud-sync

Sync Conflicts

Problem: Files modified on both sides

Safe profile (manual):

# Find conflict files
find ~/basic-memory-cloud-sync -name "*.conflict*"

# Review and merge
vimdiff file.conflict1 file.conflict2

# Keep desired version
mv file.conflict1 file
rm file.conflict2

Balanced profile (auto):

# Already resolved to newer version
# Check git history if needed
cd ~/basic-memory-cloud-sync
git log file.md

Deleted Too Many Files

Problem: Exceeds max_delete threshold

$ bm cloud bisync
Error: Deletion exceeds safety limit (26 > 25)

Solution: Review deletions, then force if intentional

# Preview what would be deleted
bm cloud bisync --dry-run

# If intentional, use higher threshold profile
bm cloud bisync --profile fast  # max_delete=50

# Or resync to establish new baseline
bm cloud bisync --resync

rclone Not Found

Problem: rclone not installed

$ bm cloud bisync
Error: rclone not found

Solution:

# Run setup again
bm cloud bisync-setup
# → Installs rclone automatically

Configuration

Bisync Config

Edit ~/.basic-memory/config.json:

{
  "bisync_config": {
    "sync_dir": "~/basic-memory-cloud-sync",
    "default_profile": "balanced",
    "auto_sync_interval": 60
  }
}

rclone Config

Located at ~/.config/rclone/rclone.conf:

[basic-memory-{tenant_id}]
type = s3
provider = AWS
env_auth = false
access_key_id = AKIA...
secret_access_key = ***
region = us-east-1
endpoint = https://fly.storage.tigris.dev

Security: This file contains credentials - keep private (mode 600)

Performance Tips

1. Use balanced profile: Best trade-off for most users
2. Enable watch mode: bm sync --watch for auto-sync
3. Optimize .bmignore: Exclude build artifacts and temp files
4. Batch changes: Group related edits before sync
5. Use fast profile: For rapid iteration on solo projects

Migration from WebDAV

If upgrading from v0.14.x WebDAV:

1. Backup existing setup

   cp -r ~/basic-memory ~/basic-memory.backup
   

2. Run bisync setup

   bm cloud bisync-setup
   

3. Copy projects to sync directory

   cp -r ~/basic-memory/* ~/basic-memory-cloud-sync/
   

4. Initial sync

   bm cloud bisync --resync
   

5. Remove old WebDAV config (if applicable)

Security

• Scoped credentials: S3 credentials only access your tenant
• Encrypted transport: All traffic over HTTPS/TLS
• No plain text secrets: Credentials stored securely in rclone config
• File permissions: Config files restricted to user (600)
• .bmignore: Prevents syncing sensitive files

See Also

• SPEC-9: Multi-Project Bidirectional Sync Architecture
• cloud-authentication.md - Required for cloud access
• cloud-mount.md - Alternative: mount cloud storage
• env-file-removal.md - Why .env files aren't synced
• gitignore-integration.md - File filtering patterns