mirror of
https://github.com/basicmachines-co/basic-memory
synced 2026-06-21 13:47:35 +00:00
Compare commits
1 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| b6f9bdcc71 |
@@ -1,95 +0,0 @@
|
||||
# /beta - Create Beta Release
|
||||
|
||||
Create a new beta release using the automated justfile target with quality checks and tagging.
|
||||
|
||||
## Usage
|
||||
```
|
||||
/beta <version>
|
||||
```
|
||||
|
||||
**Parameters:**
|
||||
- `version` (required): Beta version like `v0.13.2b1` or `v0.13.2rc1`
|
||||
|
||||
## Implementation
|
||||
|
||||
You are an expert release manager for the Basic Memory project. When the user runs `/beta`, execute the following steps:
|
||||
|
||||
### Step 1: Pre-flight Validation
|
||||
1. Verify version format matches `v\d+\.\d+\.\d+(b\d+|rc\d+)` pattern
|
||||
2. Check current git status for uncommitted changes
|
||||
3. Verify we're on the `main` branch
|
||||
4. Confirm no existing tag with this version
|
||||
|
||||
### Step 2: Use Justfile Automation
|
||||
Execute the automated beta release process:
|
||||
```bash
|
||||
just beta <version>
|
||||
```
|
||||
|
||||
The justfile target handles:
|
||||
- ✅ Beta version format validation (supports b1, b2, rc1, etc.)
|
||||
- ✅ Git status and branch checks
|
||||
- ✅ Quality checks (`just check` - lint, format, type-check, tests)
|
||||
- ✅ Version update in `src/basic_memory/__init__.py`
|
||||
- ✅ Automatic commit with proper message
|
||||
- ✅ Tag creation and pushing to GitHub
|
||||
- ✅ Beta release workflow trigger
|
||||
|
||||
### Step 3: Monitor Beta Release
|
||||
1. Check GitHub Actions workflow starts successfully
|
||||
2. Monitor workflow at: https://github.com/basicmachines-co/basic-memory/actions
|
||||
3. Verify PyPI pre-release publication
|
||||
4. Test beta installation: `uv tool install basic-memory --pre`
|
||||
|
||||
### Step 4: Beta Testing Instructions
|
||||
Provide users with beta testing instructions:
|
||||
|
||||
```bash
|
||||
# Install/upgrade to beta
|
||||
uv tool install basic-memory --pre
|
||||
|
||||
# Or upgrade existing installation
|
||||
uv tool upgrade basic-memory --prerelease=allow
|
||||
```
|
||||
|
||||
## Version Guidelines
|
||||
- **First beta**: `v0.13.2b1`
|
||||
- **Subsequent betas**: `v0.13.2b2`, `v0.13.2b3`, etc.
|
||||
- **Release candidates**: `v0.13.2rc1`, `v0.13.2rc2`, etc.
|
||||
- **Final release**: `v0.13.2` (use `/release` command)
|
||||
|
||||
## Error Handling
|
||||
- If `just beta` fails, examine the error output for specific issues
|
||||
- If quality checks fail, fix issues and retry
|
||||
- If version format is invalid, correct and retry
|
||||
- If tag already exists, increment version number
|
||||
|
||||
## Success Output
|
||||
```
|
||||
✅ Beta Release v0.13.2b1 Created Successfully!
|
||||
|
||||
🏷️ Tag: v0.13.2b1
|
||||
🚀 GitHub Actions: Running
|
||||
📦 PyPI: Will be available in ~5 minutes as pre-release
|
||||
|
||||
Install/test with:
|
||||
uv tool install basic-memory --pre
|
||||
|
||||
Monitor release: https://github.com/basicmachines-co/basic-memory/actions
|
||||
```
|
||||
|
||||
## Beta Testing Workflow
|
||||
1. **Create beta**: Use `/beta v0.13.2b1`
|
||||
2. **Test features**: Install and validate new functionality
|
||||
3. **Fix issues**: Address bugs found during testing
|
||||
4. **Iterate**: Create `v0.13.2b2` if needed
|
||||
5. **Release candidate**: Create `v0.13.2rc1` when stable
|
||||
6. **Final release**: Use `/release v0.13.2` when ready
|
||||
|
||||
## Context
|
||||
- Beta releases are pre-releases for testing new features
|
||||
- Automatically published to PyPI with pre-release flag
|
||||
- Uses the automated justfile target for consistency
|
||||
- Version is automatically updated in `__init__.py`
|
||||
- Ideal for validating changes before stable release
|
||||
- Supports both beta (b1, b2) and release candidate (rc1, rc2) versions
|
||||
@@ -1,160 +0,0 @@
|
||||
# /changelog - Generate or Update Changelog Entry
|
||||
|
||||
Analyze commits and generate formatted changelog entry for a version.
|
||||
|
||||
## Usage
|
||||
```
|
||||
/changelog <version> [type]
|
||||
```
|
||||
|
||||
**Parameters:**
|
||||
- `version` (required): Version like `v0.14.0` or `v0.14.0b1`
|
||||
- `type` (optional): `beta`, `rc`, or `stable` (default: `stable`)
|
||||
|
||||
## Implementation
|
||||
|
||||
You are an expert technical writer for the Basic Memory project. When the user runs `/changelog`, execute the following steps:
|
||||
|
||||
### Step 1: Version Analysis
|
||||
1. **Determine Commit Range**
|
||||
```bash
|
||||
# Find last release tag
|
||||
git tag -l "v*" --sort=-version:refname | grep -v "b\|rc" | head -1
|
||||
|
||||
# Get commits since last release
|
||||
git log --oneline ${last_tag}..HEAD
|
||||
```
|
||||
|
||||
2. **Parse Conventional Commits**
|
||||
- Extract feat: (features)
|
||||
- Extract fix: (bug fixes)
|
||||
- Extract BREAKING CHANGE: (breaking changes)
|
||||
- Extract chore:, docs:, test: (other improvements)
|
||||
|
||||
### Step 2: Categorize Changes
|
||||
1. **Features (feat:)**
|
||||
- New MCP tools
|
||||
- New CLI commands
|
||||
- New API endpoints
|
||||
- Major functionality additions
|
||||
|
||||
2. **Bug Fixes (fix:)**
|
||||
- User-facing bug fixes
|
||||
- Critical issues resolved
|
||||
- Performance improvements
|
||||
- Security fixes
|
||||
|
||||
3. **Technical Improvements**
|
||||
- Test coverage improvements
|
||||
- Code quality enhancements
|
||||
- Dependency updates
|
||||
- Documentation updates
|
||||
|
||||
4. **Breaking Changes**
|
||||
- API changes
|
||||
- Configuration changes
|
||||
- Behavior changes
|
||||
- Migration requirements
|
||||
|
||||
### Step 3: Generate Changelog Entry
|
||||
Create formatted entry following existing CHANGELOG.md style:
|
||||
|
||||
Example:
|
||||
```markdown
|
||||
## <version> (<date>)
|
||||
|
||||
### Features
|
||||
|
||||
- **Multi-Project Management System** - Switch between projects instantly during conversations
|
||||
([`993e88a`](https://github.com/basicmachines-co/basic-memory/commit/993e88a))
|
||||
- Instant project switching with session context
|
||||
- Project-specific operations and isolation
|
||||
- Project discovery and management tools
|
||||
|
||||
- **Advanced Note Editing** - Incremental editing with append, prepend, find/replace, and section operations
|
||||
([`6fc3904`](https://github.com/basicmachines-co/basic-memory/commit/6fc3904))
|
||||
- `edit_note` tool with multiple operation types
|
||||
- Smart frontmatter-aware editing
|
||||
- Validation and error handling
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
- **#118**: Fix YAML tag formatting to follow standard specification
|
||||
([`2dc7e27`](https://github.com/basicmachines-co/basic-memory/commit/2dc7e27))
|
||||
|
||||
- **#110**: Make --project flag work consistently across CLI commands
|
||||
([`02dd91a`](https://github.com/basicmachines-co/basic-memory/commit/02dd91a))
|
||||
|
||||
### Technical Improvements
|
||||
|
||||
- **Comprehensive Testing** - 100% test coverage with integration testing
|
||||
([`468a22f`](https://github.com/basicmachines-co/basic-memory/commit/468a22f))
|
||||
- MCP integration test suite
|
||||
- End-to-end testing framework
|
||||
- Performance and edge case validation
|
||||
|
||||
### Breaking Changes
|
||||
|
||||
- **Database Migration**: Automatic migration from per-project to unified database.
|
||||
Data will be re-index from the filesystem, resulting in no data loss.
|
||||
- **Configuration Changes**: Projects now synced between config.json and database
|
||||
- **Full Backward Compatibility**: All existing setups continue to work seamlessly
|
||||
```
|
||||
|
||||
### Step 4: Integration
|
||||
1. **Update CHANGELOG.md**
|
||||
- Insert new entry at top
|
||||
- Maintain consistent formatting
|
||||
- Include commit links and issue references
|
||||
|
||||
2. **Validation**
|
||||
- Check all major changes are captured
|
||||
- Verify commit links work
|
||||
- Ensure issue numbers are correct
|
||||
|
||||
## Smart Analysis Features
|
||||
|
||||
### Automatic Classification
|
||||
- Detect feature additions from file changes
|
||||
- Identify bug fixes from commit messages
|
||||
- Find breaking changes from code analysis
|
||||
- Extract issue numbers from commit messages
|
||||
|
||||
### Content Enhancement
|
||||
- Add context for technical changes
|
||||
- Include migration guidance for breaking changes
|
||||
- Suggest installation/upgrade instructions
|
||||
- Link to relevant documentation
|
||||
|
||||
## Output Format
|
||||
|
||||
### For Beta Releases
|
||||
|
||||
Example:
|
||||
```markdown
|
||||
## v0.13.0b4 (2025-06-03)
|
||||
|
||||
### Beta Changes Since v0.13.0b3
|
||||
|
||||
- Fix FastMCP API compatibility issues
|
||||
- Update dependencies to latest versions
|
||||
- Resolve setuptools import error
|
||||
|
||||
### Installation
|
||||
```bash
|
||||
uv tool install basic-memory --prerelease=allow
|
||||
```
|
||||
|
||||
### Known Issues
|
||||
- [List any known issues for beta testing]
|
||||
```
|
||||
|
||||
### For Stable Releases
|
||||
Full changelog with complete feature list, organized by impact and category.
|
||||
|
||||
## Context
|
||||
- Follows existing CHANGELOG.md format and style
|
||||
- Uses conventional commit standards
|
||||
- Includes GitHub commit links for traceability
|
||||
- Focuses on user-facing changes and value
|
||||
- Maintains consistency with previous entries
|
||||
@@ -1,131 +0,0 @@
|
||||
# /release-check - Pre-flight Release Validation
|
||||
|
||||
Comprehensive pre-flight check for release readiness without making any changes.
|
||||
|
||||
## Usage
|
||||
```
|
||||
/release-check [version]
|
||||
```
|
||||
|
||||
**Parameters:**
|
||||
- `version` (optional): Version to validate like `v0.13.0`. If not provided, determines from context.
|
||||
|
||||
## Implementation
|
||||
|
||||
You are an expert QA engineer for the Basic Memory project. When the user runs `/release-check`, execute the following validation steps:
|
||||
|
||||
### Step 1: Environment Validation
|
||||
1. **Git Status Check**
|
||||
- Verify working directory is clean
|
||||
- Confirm on `main` branch
|
||||
- Check if ahead/behind origin
|
||||
|
||||
2. **Version Validation**
|
||||
- Validate version format if provided
|
||||
- Check for existing tags with same version
|
||||
- Verify version increments properly from last release
|
||||
|
||||
### Step 2: Code Quality Gates
|
||||
1. **Test Suite Validation**
|
||||
```bash
|
||||
just test
|
||||
```
|
||||
- All tests must pass
|
||||
- Check test coverage (target: 95%+)
|
||||
- Validate no skipped critical tests
|
||||
|
||||
2. **Code Quality Checks**
|
||||
```bash
|
||||
just lint
|
||||
just type-check
|
||||
```
|
||||
- No linting errors
|
||||
- No type checking errors
|
||||
- Code formatting is consistent
|
||||
|
||||
### Step 3: Documentation Validation
|
||||
1. **Changelog Check**
|
||||
- CHANGELOG.md contains entry for target version
|
||||
- Entry includes all major features and fixes
|
||||
- Breaking changes are documented
|
||||
|
||||
2. **Documentation Currency**
|
||||
- README.md reflects current functionality
|
||||
- CLI reference is up to date
|
||||
- MCP tools are documented
|
||||
|
||||
### Step 4: Dependency Validation
|
||||
1. **Security Scan**
|
||||
- No known vulnerabilities in dependencies
|
||||
- All dependencies are at appropriate versions
|
||||
- No conflicting dependency versions
|
||||
|
||||
2. **Build Validation**
|
||||
- Package builds successfully
|
||||
- All required files are included
|
||||
- No missing dependencies
|
||||
|
||||
### Step 5: Issue Tracking Validation
|
||||
1. **GitHub Issues Check**
|
||||
- No critical open issues blocking release
|
||||
- All milestone issues are resolved
|
||||
- High-priority bugs are fixed
|
||||
|
||||
2. **Testing Coverage**
|
||||
- Integration tests pass
|
||||
- MCP tool tests pass
|
||||
- Cross-platform compatibility verified
|
||||
|
||||
## Report Format
|
||||
|
||||
Generate a comprehensive report:
|
||||
|
||||
```
|
||||
🔍 Release Readiness Check for v0.13.0
|
||||
|
||||
✅ PASSED CHECKS:
|
||||
├── Git status clean
|
||||
├── On main branch
|
||||
├── All tests passing (744/744)
|
||||
├── Test coverage: 98.2%
|
||||
├── Type checking passed
|
||||
├── Linting passed
|
||||
├── CHANGELOG.md updated
|
||||
└── No critical issues open
|
||||
|
||||
⚠️ WARNINGS:
|
||||
├── 2 medium-priority issues still open
|
||||
└── Documentation could be updated
|
||||
|
||||
❌ BLOCKING ISSUES:
|
||||
└── None found
|
||||
|
||||
🎯 RELEASE READINESS: ✅ READY
|
||||
|
||||
Recommended next steps:
|
||||
1. Address warnings if desired
|
||||
2. Run `/release v0.13.0` when ready
|
||||
```
|
||||
|
||||
## Validation Criteria
|
||||
|
||||
### Must Pass (Blocking)
|
||||
- [ ] All tests pass
|
||||
- [ ] No type errors
|
||||
- [ ] No linting errors
|
||||
- [ ] Working directory clean
|
||||
- [ ] On main branch
|
||||
- [ ] CHANGELOG.md has version entry
|
||||
- [ ] No critical open issues
|
||||
|
||||
### Should Pass (Warnings)
|
||||
- [ ] Test coverage >95%
|
||||
- [ ] No medium-priority open issues
|
||||
- [ ] Documentation up to date
|
||||
- [ ] No dependency vulnerabilities
|
||||
|
||||
## Context
|
||||
- This is a read-only validation - makes no changes
|
||||
- Provides confidence before running actual release
|
||||
- Helps identify issues early in release process
|
||||
- Can be run multiple times safely
|
||||
@@ -1,199 +0,0 @@
|
||||
# /release - Create Stable Release
|
||||
|
||||
Create a stable release using the automated justfile target with comprehensive validation.
|
||||
|
||||
## Usage
|
||||
```
|
||||
/release <version>
|
||||
```
|
||||
|
||||
**Parameters:**
|
||||
- `version` (required): Release version like `v0.13.2`
|
||||
|
||||
## Implementation
|
||||
|
||||
You are an expert release manager for the Basic Memory project. When the user runs `/release`, execute the following steps:
|
||||
|
||||
### Step 1: Pre-flight Validation
|
||||
|
||||
#### Version Check
|
||||
1. Check current version in `src/basic_memory/__init__.py`
|
||||
2. Verify new version format matches `v\d+\.\d+\.\d+` pattern
|
||||
3. Confirm version is higher than current version
|
||||
|
||||
#### Git Status
|
||||
1. Check current git status for uncommitted changes
|
||||
2. Verify we're on the `main` branch
|
||||
3. Confirm no existing tag with this version
|
||||
|
||||
#### Documentation Validation
|
||||
1. **Changelog Check**
|
||||
- CHANGELOG.md contains entry for target version
|
||||
- Entry includes all major features and fixes
|
||||
- Breaking changes are documented
|
||||
|
||||
### Step 2: Use Justfile Automation
|
||||
Execute the automated release process:
|
||||
```bash
|
||||
just release <version>
|
||||
```
|
||||
|
||||
The justfile target handles:
|
||||
- ✅ Version format validation
|
||||
- ✅ Git status and branch checks
|
||||
- ✅ Quality checks (`just check` - lint, format, type-check, tests)
|
||||
- ✅ Version update in `src/basic_memory/__init__.py`
|
||||
- ✅ Automatic commit with proper message
|
||||
- ✅ Tag creation and pushing to GitHub
|
||||
- ✅ Release workflow trigger (automatic on tag push)
|
||||
|
||||
The GitHub Actions workflow (`.github/workflows/release.yml`) then:
|
||||
- ✅ Builds the package using `uv build`
|
||||
- ✅ Creates GitHub release with auto-generated notes
|
||||
- ✅ Publishes to PyPI
|
||||
- ✅ Updates Homebrew formula (stable releases only)
|
||||
|
||||
### Step 3: Monitor Release Process
|
||||
1. Verify tag push triggered the workflow (should start automatically within seconds)
|
||||
2. Monitor workflow progress at: https://github.com/basicmachines-co/basic-memory/actions
|
||||
3. Watch for successful completion of both jobs:
|
||||
- `release` - Builds package and publishes to PyPI
|
||||
- `homebrew` - Updates Homebrew formula (stable releases only)
|
||||
4. Check for any workflow failures and investigate logs if needed
|
||||
|
||||
### Step 4: Post-Release Validation
|
||||
|
||||
#### GitHub Release
|
||||
1. Verify GitHub release is created at: https://github.com/basicmachines-co/basic-memory/releases/tag/<version>
|
||||
2. Check that release notes are auto-generated from commits
|
||||
3. Validate release assets (`.whl` and `.tar.gz` files are attached)
|
||||
|
||||
#### PyPI Publication
|
||||
1. Verify package published at: https://pypi.org/project/basic-memory/<version>/
|
||||
2. Test installation: `uv tool install basic-memory`
|
||||
3. Verify installed version: `basic-memory --version`
|
||||
|
||||
#### Homebrew Formula (Stable Releases Only)
|
||||
1. Check formula update at: https://github.com/basicmachines-co/homebrew-basic-memory
|
||||
2. Verify formula version matches release
|
||||
3. Test Homebrew installation: `brew install basicmachines-co/basic-memory/basic-memory`
|
||||
|
||||
#### MCP Registry Publication
|
||||
|
||||
After PyPI release is published, update the MCP registry:
|
||||
|
||||
1. **Verify PyPI Release**
|
||||
- Confirm package is live: https://pypi.org/project/basic-memory/<version>/
|
||||
- The `server.json` version was auto-updated by `just release`
|
||||
|
||||
2. **Publish to MCP Registry**
|
||||
```bash
|
||||
cd /Users/drew/code/basic-memory
|
||||
mcp-publisher publish
|
||||
```
|
||||
|
||||
If not authenticated:
|
||||
```bash
|
||||
mcp-publisher login github
|
||||
# Follow device authentication flow
|
||||
mcp-publisher publish
|
||||
```
|
||||
|
||||
3. **Verify Publication**
|
||||
```bash
|
||||
curl "https://registry.modelcontextprotocol.io/v0.1/servers?search=basic-memory"
|
||||
```
|
||||
|
||||
**Note:** The `mcp-publisher` CLI can be installed via Homebrew (`brew install mcp-publisher`) or from GitHub releases.
|
||||
|
||||
#### Website Updates
|
||||
|
||||
**1. basicmachines.co** (`/Users/drew/code/basicmachines.co`)
|
||||
- **Goal**: Update version number displayed on the homepage
|
||||
- **Location**: Search for "Basic Memory v0." in the codebase to find version displays
|
||||
- **What to update**:
|
||||
- Hero section heading that shows "Basic Memory v{VERSION}"
|
||||
- "What's New in v{VERSION}" section heading
|
||||
- Feature highlights array (look for array of features with title/description)
|
||||
- **Process**:
|
||||
1. Pull latest from GitHub: `git pull origin main`
|
||||
2. Create release branch: `git checkout -b release/v{VERSION}`
|
||||
3. Search codebase for current version number (e.g., "v0.16.1")
|
||||
4. Update version numbers to new release version
|
||||
5. Update feature highlights with 3-5 key features from this release (extract from CHANGELOG.md)
|
||||
6. Commit changes: `git commit -m "chore: update to v{VERSION}"`
|
||||
7. Push branch: `git push origin release/v{VERSION}`
|
||||
- **Deploy**: Follow deployment process for basicmachines.co
|
||||
|
||||
**2. docs.basicmemory.com** (`/Users/drew/code/docs.basicmemory.com`)
|
||||
- **Goal**: Add new release notes section to the latest-releases page
|
||||
- **File**: `src/pages/latest-releases.mdx`
|
||||
- **What to do**:
|
||||
1. Pull latest from GitHub: `git pull origin main`
|
||||
2. Create release branch: `git checkout -b release/v{VERSION}`
|
||||
3. Read the existing file to understand the format and structure
|
||||
4. Read `/Users/drew/code/basic-memory/CHANGELOG.md` to get release content
|
||||
5. Add new release section **at the top** (after MDX imports, before other releases)
|
||||
6. Follow the existing pattern:
|
||||
- Heading: `## [v{VERSION}](github-link) — YYYY-MM-DD`
|
||||
- Focus statement if applicable
|
||||
- `<Info>` block with highlights (3-5 key items)
|
||||
- Sections for Features, Bug Fixes, Breaking Changes, etc.
|
||||
- Link to full changelog at the end
|
||||
- Separator `---` between releases
|
||||
7. Commit changes: `git commit -m "docs: add v{VERSION} release notes"`
|
||||
8. Push branch: `git push origin release/v{VERSION}`
|
||||
- **Source content**: Extract and format sections from CHANGELOG.md for this version
|
||||
- **Deploy**: Follow deployment process for docs.basicmemory.com
|
||||
|
||||
**4. Announce Release**
|
||||
- Post to Discord community if significant changes
|
||||
- Update social media if major release
|
||||
- Notify users via appropriate channels
|
||||
|
||||
## Pre-conditions Check
|
||||
Before starting, verify:
|
||||
- [ ] All beta testing is complete
|
||||
- [ ] Critical bugs are fixed
|
||||
- [ ] Breaking changes are documented
|
||||
- [ ] CHANGELOG.md is updated (if needed)
|
||||
- [ ] Version number follows semantic versioning
|
||||
|
||||
## Error Handling
|
||||
- If `just release` fails, examine the error output for specific issues
|
||||
- If quality checks fail, fix issues and retry
|
||||
- If changelog entry missing, update CHANGELOG.md and commit before retrying
|
||||
- If GitHub Actions fail, check workflow logs for debugging
|
||||
|
||||
## Success Output
|
||||
```
|
||||
🎉 Stable Release v0.13.2 Created Successfully!
|
||||
|
||||
🏷️ Tag: v0.13.2
|
||||
📋 GitHub Release: https://github.com/basicmachines-co/basic-memory/releases/tag/v0.13.2
|
||||
📦 PyPI: https://pypi.org/project/basic-memory/0.13.2/
|
||||
🍺 Homebrew: https://github.com/basicmachines-co/homebrew-basic-memory
|
||||
🔌 MCP Registry: https://registry.modelcontextprotocol.io
|
||||
🚀 GitHub Actions: Completed
|
||||
|
||||
Install with pip/uv:
|
||||
uv tool install basic-memory
|
||||
|
||||
Install with Homebrew:
|
||||
brew install basicmachines-co/basic-memory/basic-memory
|
||||
|
||||
Users can now upgrade:
|
||||
uv tool upgrade basic-memory
|
||||
brew upgrade basic-memory
|
||||
```
|
||||
|
||||
## Context
|
||||
- This creates production releases used by end users
|
||||
- Must pass all quality gates before proceeding
|
||||
- Uses the automated justfile target for consistency
|
||||
- Version is automatically updated in `__init__.py` and `server.json`
|
||||
- Triggers automated GitHub release with changelog
|
||||
- Package is published to PyPI for `pip` and `uv` users
|
||||
- Homebrew formula is automatically updated for stable releases
|
||||
- MCP Registry is updated manually via `mcp-publisher publish`
|
||||
- Supports multiple installation methods (uv, pip, Homebrew)
|
||||
@@ -1,51 +0,0 @@
|
||||
---
|
||||
allowed-tools: mcp__basic-memory__write_note, mcp__basic-memory__read_note, mcp__basic-memory__search_notes, mcp__basic-memory__edit_note
|
||||
argument-hint: [create|status|show|review] [spec-name]
|
||||
description: Manage specifications in our development process
|
||||
---
|
||||
|
||||
## Context
|
||||
|
||||
Specifications are managed in the Basic Memory "specs" project. All specs live in a centralized location accessible across all repositories via MCP tools.
|
||||
|
||||
See SPEC-1 and SPEC-2 in the "specs" project for the full specification-driven development process.
|
||||
|
||||
Available commands:
|
||||
- `create [name]` - Create new specification
|
||||
- `status` - Show all spec statuses
|
||||
- `show [spec-name]` - Read a specific spec
|
||||
- `review [spec-name]` - Review implementation against spec
|
||||
|
||||
## Your task
|
||||
|
||||
Execute the spec command: `/spec $ARGUMENTS`
|
||||
|
||||
### If command is "create":
|
||||
1. Get next SPEC number by searching existing specs in "specs" project
|
||||
2. Create new spec using template from SPEC-2
|
||||
3. Use mcp__basic-memory__write_note with project="specs"
|
||||
4. Include standard sections: Why, What, How, How to Evaluate
|
||||
|
||||
### If command is "status":
|
||||
1. Use mcp__basic-memory__search_notes with project="specs"
|
||||
2. Display table with spec number, title, and progress
|
||||
3. Show completion status from checkboxes in content
|
||||
|
||||
### If command is "show":
|
||||
1. Use mcp__basic-memory__read_note with project="specs"
|
||||
2. Display the full spec content
|
||||
|
||||
### If command is "review":
|
||||
1. Read the specified spec and its "How to Evaluate" section
|
||||
2. Review current implementation against success criteria with careful evaluation of:
|
||||
- **Functional completeness** - All specified features working
|
||||
- **Test coverage analysis** - Actual test files and coverage percentage
|
||||
- Count existing test files vs required components/APIs/composables
|
||||
- Verify unit tests, integration tests, and end-to-end tests
|
||||
- Check for missing test categories (component, API, workflow)
|
||||
- **Code quality metrics** - TypeScript compilation, linting, performance
|
||||
- **Architecture compliance** - Component isolation, state management patterns
|
||||
- **Documentation completeness** - Implementation matches specification
|
||||
3. Provide honest, accurate assessment - do not overstate completeness
|
||||
4. Document findings and update spec with review results using mcp__basic-memory__edit_note
|
||||
5. If gaps found, clearly identify what still needs to be implemented/tested
|
||||
@@ -1,622 +0,0 @@
|
||||
# /project:test-live - Live Basic Memory Testing Suite
|
||||
|
||||
Execute comprehensive real-world testing of Basic Memory using the installed version.
|
||||
All test results are recorded as notes in a dedicated test project.
|
||||
|
||||
## Usage
|
||||
```
|
||||
/project:test-live [phase]
|
||||
```
|
||||
|
||||
**Parameters:**
|
||||
- `phase` (optional): Specific test phase to run (`recent`, `core`, `features`, `edge`, `workflows`, `stress`, or `all`)
|
||||
- `recent` - Focus on recent changes and new features (recommended for regular testing)
|
||||
- `core` - Essential tools only (Tier 1: write_note, read_note, search_notes, edit_note, list_memory_projects, recent_activity)
|
||||
- `features` - Core + important workflows (Tier 1 + Tier 2)
|
||||
- `all` - Comprehensive testing of all tools and scenarios
|
||||
|
||||
## Implementation
|
||||
|
||||
You are an expert QA engineer conducting live testing of Basic Memory.
|
||||
When the user runs `/project:test-live`, execute comprehensive test plan:
|
||||
|
||||
## Tool Testing Priority
|
||||
|
||||
### **Tier 1: Critical Core (Always Test)**
|
||||
1. **write_note** - Foundation of all knowledge creation
|
||||
2. **read_note** - Primary knowledge retrieval mechanism
|
||||
3. **search_notes** - Essential for finding information
|
||||
4. **edit_note** - Core content modification capability
|
||||
5. **list_memory_projects** - Project discovery and session guidance
|
||||
6. **recent_activity** - Project discovery mode and activity analysis
|
||||
|
||||
### **Tier 2: Important Workflows (Usually Test)**
|
||||
7. **build_context** - Conversation continuity via memory:// URLs
|
||||
8. **create_memory_project** - Essential for project setup
|
||||
9. **move_note** - Knowledge organization
|
||||
10. **sync_status** - Understanding system state
|
||||
11. **delete_project** - Project lifecycle management
|
||||
|
||||
### **Tier 3: Enhanced Functionality (Sometimes Test)**
|
||||
12. **view_note** - Claude Desktop artifact display
|
||||
13. **read_content** - Raw content access
|
||||
14. **delete_note** - Content removal
|
||||
15. **list_directory** - File system exploration
|
||||
16. **edit_note** (advanced modes) - Complex find/replace operations
|
||||
|
||||
### **Tier 4: Specialized (Rarely Test)**
|
||||
17. **canvas** - Obsidian visualization (specialized use case)
|
||||
18. **MCP Prompts** - Enhanced UX tools (ai_assistant_guide, continue_conversation)
|
||||
|
||||
## Stateless Architecture Testing
|
||||
|
||||
### **Project Discovery Workflow (CRITICAL)**
|
||||
Test the new stateless project selection flow:
|
||||
|
||||
1. **Initial Discovery**
|
||||
- Call `list_memory_projects()` without knowing which project to use
|
||||
- Verify clear session guidance appears: "Next: Ask which project to use"
|
||||
- Confirm removal of CLI-specific references
|
||||
|
||||
2. **Activity-Based Discovery**
|
||||
- Call `recent_activity()` without project parameter (discovery mode)
|
||||
- Verify intelligent project suggestions based on activity
|
||||
- Test guidance: "Should I use [most-active-project] for this task?"
|
||||
|
||||
3. **Session Tracking Validation**
|
||||
- Verify all tool responses include `[Session: Using project 'name']`
|
||||
- Confirm guidance reminds about session-wide project tracking
|
||||
|
||||
4. **Single Project Constraint Mode**
|
||||
- Test MCP server with `--project` parameter
|
||||
- Verify all operations constrained to specified project
|
||||
- Test project override behavior in constrained mode
|
||||
|
||||
### **Explicit Project Parameters (CRITICAL)**
|
||||
All tools must require explicit project parameters:
|
||||
|
||||
1. **Parameter Validation**
|
||||
- Test all Tier 1 tools require `project` parameter
|
||||
- Verify clear error messages for missing project
|
||||
- Test invalid project name handling
|
||||
|
||||
2. **No Session State Dependencies**
|
||||
- Confirm no tool relies on "current project" concept
|
||||
- Test rapid project switching within conversation
|
||||
- Verify each call is truly independent
|
||||
|
||||
### Pre-Test Setup
|
||||
|
||||
1. **Environment Verification**
|
||||
- Verify basic-memory is installed and accessible via MCP
|
||||
- Check version and confirm it's the expected release
|
||||
- Test MCP connection and tool availability
|
||||
|
||||
2. **Recent Changes Analysis** (if phase includes 'recent' or 'all')
|
||||
- Run `git log --oneline -20` to examine recent commits
|
||||
- Identify new features, bug fixes, and enhancements
|
||||
- Generate targeted test scenarios for recent changes
|
||||
- Prioritize regression testing for recently fixed issues
|
||||
|
||||
3. **Test Project Creation**
|
||||
|
||||
Run the bash `date` command to get the current date/time.
|
||||
|
||||
```
|
||||
Create project: "basic-memory-testing-[timestamp]"
|
||||
Location: ~/basic-memory-testing-[timestamp]
|
||||
Purpose: Record all test observations and results
|
||||
```
|
||||
|
||||
Make sure to use the newly created project for all subsequent test operations by specifying it in the `project` parameter of each tool call.
|
||||
|
||||
4. **Baseline Documentation**
|
||||
Create initial test session note with:
|
||||
- Test environment details
|
||||
- Version being tested
|
||||
- Recent changes identified (if applicable)
|
||||
- Test objectives and scope
|
||||
- Start timestamp
|
||||
|
||||
### Phase 0: Recent Changes Validation (if 'recent' or 'all' phase)
|
||||
|
||||
Based on recent commit analysis, create targeted test scenarios:
|
||||
|
||||
**Recent Changes Test Protocol:**
|
||||
1. **Feature Addition Tests** - For each new feature identified:
|
||||
- Test basic functionality
|
||||
- Test integration with existing tools
|
||||
- Verify documentation accuracy
|
||||
- Test edge cases and error handling
|
||||
|
||||
2. **Bug Fix Regression Tests** - For each recent fix:
|
||||
- Recreate the original problem scenario
|
||||
- Verify the fix works as expected
|
||||
- Test related functionality isn't broken
|
||||
- Document the verification in test notes
|
||||
|
||||
3. **Performance/Enhancement Validation** - For optimizations:
|
||||
- Establish baseline timing
|
||||
- Compare with expected improvements
|
||||
- Test under various load conditions
|
||||
- Document performance observations
|
||||
|
||||
**Example Recent Changes (Update based on actual git log):**
|
||||
- Watch Service Restart (#156): Test project creation → file modification → automatic restart
|
||||
- Cross-Project Moves (#161): Test move_note with cross-project detection
|
||||
- Docker Environment Support (#174): Test BASIC_MEMORY_HOME behavior
|
||||
- MCP Server Logging (#164): Verify log level configurations
|
||||
|
||||
### Phase 1: Core Functionality Validation (Tier 1 Tools)
|
||||
|
||||
Test essential MCP tools that form the foundation of Basic Memory:
|
||||
|
||||
**1. write_note Tests (Critical):**
|
||||
- ✅ Basic note creation with frontmatter
|
||||
- ✅ Special characters and Unicode in titles
|
||||
- ✅ Various content types (lists, headings, code blocks)
|
||||
- ✅ Empty notes and minimal content edge cases
|
||||
- ⚠️ Error handling for invalid parameters
|
||||
|
||||
**2. read_note Tests (Critical):**
|
||||
- ✅ Read by title, permalink, memory:// URLs
|
||||
- ✅ Non-existent notes (error handling)
|
||||
- ✅ Notes with complex markdown formatting
|
||||
- ⚠️ Performance with large notes (>10MB)
|
||||
|
||||
**3. search_notes Tests (Critical):**
|
||||
- ✅ Simple text queries across content
|
||||
- ✅ Tag-based searches with multiple tags
|
||||
- ✅ Boolean operators (AND, OR, NOT)
|
||||
- ✅ Empty/no results scenarios
|
||||
- ⚠️ Performance with 100+ notes
|
||||
|
||||
**4. edit_note Tests (Critical):**
|
||||
- ✅ Append operations preserving frontmatter
|
||||
- ✅ Prepend operations
|
||||
- ✅ Find/replace with validation
|
||||
- ✅ Section replacement under headers
|
||||
- ⚠️ Error scenarios (invalid operations)
|
||||
|
||||
**5. list_memory_projects Tests (Critical):**
|
||||
- ✅ Display all projects with clear session guidance
|
||||
- ✅ Project discovery workflow prompts
|
||||
- ✅ Removal of CLI-specific references
|
||||
- ✅ Empty project list handling
|
||||
- ✅ Single project constraint mode display
|
||||
|
||||
**6. recent_activity Tests (Critical - Discovery Mode):**
|
||||
- ✅ Discovery mode without project parameter
|
||||
- ✅ Intelligent project suggestions based on activity
|
||||
- ✅ Guidance prompts for project selection
|
||||
- ✅ Session tracking reminders in responses
|
||||
- ⚠️ Performance with multiple projects
|
||||
|
||||
### Phase 2: Important Workflows (Tier 2 Tools)
|
||||
|
||||
**7. build_context Tests (Important):**
|
||||
- ✅ Different depth levels (1, 2, 3+)
|
||||
- ✅ Various timeframes for context
|
||||
- ✅ memory:// URL navigation
|
||||
- ⚠️ Performance with complex relation graphs
|
||||
|
||||
**8. create_memory_project Tests (Important):**
|
||||
- ✅ Create projects dynamically
|
||||
- ✅ Set default during creation
|
||||
- ✅ Path validation and creation
|
||||
- ⚠️ Invalid paths and names
|
||||
- ✅ Integration with existing projects
|
||||
|
||||
**9. move_note Tests (Important):**
|
||||
- ✅ Move within same project
|
||||
- ✅ Cross-project moves with detection (#161)
|
||||
- ✅ Automatic folder creation
|
||||
- ✅ Database consistency validation
|
||||
- ⚠️ Special characters in paths
|
||||
|
||||
**10. sync_status Tests (Important):**
|
||||
- ✅ Background operation monitoring
|
||||
- ✅ File synchronization status
|
||||
- ✅ Project sync state reporting
|
||||
- ⚠️ Error state handling
|
||||
|
||||
### Phase 3: Enhanced Functionality (Tier 3 Tools)
|
||||
|
||||
**11. view_note Tests (Enhanced):**
|
||||
- ✅ Claude Desktop artifact display
|
||||
- ✅ Title extraction from frontmatter
|
||||
- ✅ Unicode and emoji content rendering
|
||||
- ⚠️ Error handling for non-existent notes
|
||||
|
||||
**12. read_content Tests (Enhanced):**
|
||||
- ✅ Raw file content access
|
||||
- ✅ Binary file handling
|
||||
- ✅ Image file reading
|
||||
- ⚠️ Large file performance
|
||||
|
||||
**13. delete_note Tests (Enhanced):**
|
||||
- ✅ Single note deletion
|
||||
- ✅ Database consistency after deletion
|
||||
- ⚠️ Non-existent note handling
|
||||
- ✅ Confirmation of successful deletion
|
||||
|
||||
**14. list_directory Tests (Enhanced):**
|
||||
- ✅ Directory content listing
|
||||
- ✅ Depth control and filtering
|
||||
- ✅ File name globbing
|
||||
- ⚠️ Empty directory handling
|
||||
|
||||
**15. delete_project Tests (Enhanced):**
|
||||
- ✅ Project removal from config
|
||||
- ✅ Database cleanup
|
||||
- ⚠️ Default project protection
|
||||
- ⚠️ Non-existent project handling
|
||||
|
||||
### Phase 4: Edge Case Exploration
|
||||
|
||||
**Boundary Testing:**
|
||||
- Very long titles and content (stress limits)
|
||||
- Empty projects and notes
|
||||
- Unicode, emojis, special symbols
|
||||
- Deeply nested folder structures
|
||||
- Circular relations and self-references
|
||||
- Maximum relation depths
|
||||
|
||||
**Error Scenarios:**
|
||||
- Invalid memory:// URLs
|
||||
- Missing files referenced in database
|
||||
- Invalid project names and paths
|
||||
- Malformed note structures
|
||||
- Concurrent operation conflicts
|
||||
|
||||
**Performance Testing:**
|
||||
- Create 100+ notes rapidly
|
||||
- Complex search queries
|
||||
- Deep relation chains (5+ levels)
|
||||
- Rapid successive operations
|
||||
- Memory usage monitoring
|
||||
|
||||
### Phase 5: Real-World Workflow Scenarios
|
||||
|
||||
**Meeting Notes Pipeline:**
|
||||
1. Create meeting notes with action items
|
||||
2. Extract action items using edit_note
|
||||
3. Build relations to project documents
|
||||
4. Update progress incrementally
|
||||
5. Search and track completion
|
||||
|
||||
**Research Knowledge Building:**
|
||||
1. Create research topic hierarchy
|
||||
2. Build complex relation networks
|
||||
3. Add incremental findings over time
|
||||
4. Search for connections and patterns
|
||||
5. Reorganize as knowledge evolves
|
||||
|
||||
**Multi-Project Workflow:**
|
||||
1. Technical documentation project
|
||||
2. Personal recipe collection project
|
||||
3. Learning/course notes project
|
||||
4. Specify different projects for different operations
|
||||
5. Cross-reference related concepts
|
||||
|
||||
**Content Evolution:**
|
||||
1. Start with basic notes
|
||||
2. Enhance with relations and observations
|
||||
3. Reorganize file structure using moves
|
||||
4. Update content with edit operations
|
||||
5. Validate knowledge graph integrity
|
||||
|
||||
### Phase 6: Specialized Tools Testing (Tier 4)
|
||||
|
||||
**16. canvas Tests (Specialized):**
|
||||
- ✅ JSON Canvas generation
|
||||
- ✅ Node and edge creation
|
||||
- ✅ Obsidian compatibility
|
||||
- ⚠️ Complex graph handling
|
||||
|
||||
**17. MCP Prompts Tests (Specialized):**
|
||||
- ✅ ai_assistant_guide output
|
||||
- ✅ continue_conversation functionality
|
||||
- ✅ Formatted search results
|
||||
- ✅ Enhanced activity reports
|
||||
|
||||
### Phase 7: Integration & File Watching Tests
|
||||
|
||||
**File System Integration:**
|
||||
- ✅ Watch service behavior with file changes
|
||||
- ✅ Project creation → watch restart (#156)
|
||||
- ✅ Multi-project synchronization
|
||||
- ⚠️ MCP→API→DB→File stack validation
|
||||
|
||||
**Real Integration Testing:**
|
||||
- ✅ End-to-end file watching vs manual operations
|
||||
- ✅ Cross-session persistence
|
||||
- ✅ Database consistency across operations
|
||||
- ⚠️ Performance under real file system changes
|
||||
|
||||
### Phase 8: Creative Stress Testing
|
||||
|
||||
**Creative Exploration:**
|
||||
- Rapid project creation/switching patterns
|
||||
- Unusual but valid markdown structures
|
||||
- Creative observation categories
|
||||
- Novel relation types and patterns
|
||||
- Unexpected tool combinations
|
||||
|
||||
**Stress Scenarios:**
|
||||
- Bulk operations (many notes quickly)
|
||||
- Complex nested moves and edits
|
||||
- Deep context building
|
||||
- Complex boolean search expressions
|
||||
- Resource constraint testing
|
||||
|
||||
## Test Execution Guidelines
|
||||
|
||||
### Quick Testing (core/features phases)
|
||||
- Focus on Tier 1 tools (core) or Tier 1+2 (features)
|
||||
- Test essential functionality and common edge cases
|
||||
- Record critical issues immediately
|
||||
- Complete in 15-20 minutes
|
||||
|
||||
### Comprehensive Testing (all phase)
|
||||
- Cover all tiers systematically
|
||||
- Include specialized tools and stress testing
|
||||
- Document performance baselines
|
||||
- Complete in 45-60 minutes
|
||||
|
||||
### Recent Changes Focus (recent phase)
|
||||
- Analyze git log for recent commits
|
||||
- Generate targeted test scenarios
|
||||
- Focus on regression testing for fixes
|
||||
- Validate new features thoroughly
|
||||
|
||||
## Test Observation Format
|
||||
|
||||
Record ALL observations immediately as Basic Memory notes:
|
||||
|
||||
```markdown
|
||||
---
|
||||
title: Test Session [Phase] YYYY-MM-DD HH:MM
|
||||
tags: [testing, v0.13.0, live-testing, [phase]]
|
||||
permalink: test-session-[phase]-[timestamp]
|
||||
---
|
||||
|
||||
# Test Session [Phase] - [Date/Time]
|
||||
|
||||
## Environment
|
||||
- Basic Memory version: [version]
|
||||
- MCP connection: [status]
|
||||
- Test project: [name]
|
||||
- Phase focus: [description]
|
||||
|
||||
## Test Results
|
||||
|
||||
### ✅ Successful Operations
|
||||
- [timestamp] ✅ write_note: Created note with emoji title 📝 #tier1 #functionality
|
||||
- [timestamp] ✅ search_notes: Boolean query returned 23 results in 0.4s #tier1 #performance
|
||||
- [timestamp] ✅ edit_note: Append operation preserved frontmatter #tier1 #reliability
|
||||
|
||||
### ⚠️ Issues Discovered
|
||||
- [timestamp] ⚠️ move_note: Slow with deep folder paths (2.1s) #tier2 #performance
|
||||
- [timestamp] 🚨 search_notes: Unicode query returned unexpected results #tier1 #bug #critical
|
||||
- [timestamp] ⚠️ build_context: Context lost for memory:// URLs #tier2 #issue
|
||||
|
||||
### 🚀 Enhancements Identified
|
||||
- edit_note could benefit from preview mode #ux-improvement
|
||||
- search_notes needs fuzzy matching for typos #feature-idea
|
||||
- move_note could auto-suggest folder creation #usability
|
||||
|
||||
### 📊 Performance Metrics
|
||||
- Average write_note time: 0.3s
|
||||
- Search with 100+ notes: 0.6s
|
||||
- Project parameter overhead: <0.1s
|
||||
- Memory usage: [observed levels]
|
||||
|
||||
## Relations
|
||||
- tests [[Basic Memory v0.13.0]]
|
||||
- part_of [[Live Testing Suite]]
|
||||
- found_issues [[Bug Report: Unicode Search]]
|
||||
- discovered [[Performance Optimization Opportunities]]
|
||||
```
|
||||
|
||||
## Quality Assessment Areas
|
||||
|
||||
**User Experience & Usability:**
|
||||
- Tool instruction clarity and examples
|
||||
- Error message actionability
|
||||
- Response time acceptability
|
||||
- Tool consistency and discoverability
|
||||
- Learning curve and intuitiveness
|
||||
|
||||
**System Behavior:**
|
||||
- Stateless operation independence
|
||||
- memory:// URL navigation reliability
|
||||
- Multi-step workflow cohesion
|
||||
- Edge case graceful handling
|
||||
- Recovery from user errors
|
||||
|
||||
**Documentation Alignment:**
|
||||
- Tool output clarity and helpfulness
|
||||
- Behavior vs. documentation accuracy
|
||||
- Example validity and usefulness
|
||||
- Real-world vs. documented workflows
|
||||
|
||||
**Mental Model Validation:**
|
||||
- Natural user expectation alignment
|
||||
- Surprising behavior identification
|
||||
- Mistake recovery ease
|
||||
- Knowledge graph concept naturalness
|
||||
|
||||
**Performance & Reliability:**
|
||||
- Operation completion times
|
||||
- Consistency across sessions
|
||||
- Scaling behavior with growth
|
||||
- Unexpected slowness identification
|
||||
|
||||
## Error Documentation Protocol
|
||||
|
||||
For each error discovered:
|
||||
|
||||
1. **Immediate Recording**
|
||||
- Create dedicated error note
|
||||
- Include exact reproduction steps
|
||||
- Capture error messages verbatim
|
||||
- Note system state when error occurred
|
||||
|
||||
2. **Error Note Format**
|
||||
```markdown
|
||||
---
|
||||
title: Bug Report - [Short Description]
|
||||
tags: [bug, testing, v0.13.0, [severity]]
|
||||
---
|
||||
|
||||
# Bug Report: [Description]
|
||||
|
||||
## Reproduction Steps
|
||||
1. [Exact steps to reproduce]
|
||||
2. [Include all parameters used]
|
||||
3. [Note any special conditions]
|
||||
|
||||
## Expected Behavior
|
||||
[What should have happened]
|
||||
|
||||
## Actual Behavior
|
||||
[What actually happened]
|
||||
|
||||
## Error Messages
|
||||
```
|
||||
[Exact error text]
|
||||
```
|
||||
|
||||
## Environment
|
||||
- Version: [version]
|
||||
- Project: [name]
|
||||
- Timestamp: [when]
|
||||
|
||||
## Severity
|
||||
- [ ] Critical (blocks major functionality)
|
||||
- [ ] High (impacts user experience)
|
||||
- [ ] Medium (workaround available)
|
||||
- [ ] Low (minor inconvenience)
|
||||
|
||||
## Relations
|
||||
- discovered_during [[Test Session [Phase]]]
|
||||
- affects [[Feature Name]]
|
||||
```
|
||||
|
||||
## Success Metrics Tracking
|
||||
|
||||
**Quantitative Measures:**
|
||||
- Test scenario completion rate
|
||||
- Bug discovery count with severity
|
||||
- Performance benchmark establishment
|
||||
- Tool coverage completeness
|
||||
|
||||
**Qualitative Measures:**
|
||||
- Conversation flow naturalness
|
||||
- Knowledge graph quality
|
||||
- User experience insights
|
||||
- System reliability assessment
|
||||
|
||||
## Test Execution Flow
|
||||
|
||||
1. **Setup Phase** (5 minutes)
|
||||
- Verify environment and create test project
|
||||
- Record baseline system state
|
||||
- Establish performance benchmarks
|
||||
|
||||
2. **Core Testing** (15-20 minutes per phase)
|
||||
- Execute test scenarios systematically
|
||||
- Record observations immediately
|
||||
- Note timestamps for performance tracking
|
||||
- Explore variations when interesting behaviors occur
|
||||
|
||||
3. **Documentation** (5 minutes per phase)
|
||||
- Create phase summary note
|
||||
- Link related test observations
|
||||
- Update running issues list
|
||||
- Record enhancement ideas
|
||||
|
||||
4. **Analysis Phase** (10 minutes)
|
||||
- Review all observations across phases
|
||||
- Identify patterns and trends
|
||||
- Create comprehensive summary report
|
||||
- Generate development recommendations
|
||||
|
||||
## Testing Success Criteria
|
||||
|
||||
### Core Testing (Tier 1) - Must Pass
|
||||
- All 6 critical tools function correctly
|
||||
- No critical bugs in essential workflows
|
||||
- Acceptable performance for basic operations
|
||||
- Error handling works as expected
|
||||
|
||||
### Feature Testing (Tier 1+2) - Should Pass
|
||||
- All 11 core + important tools function
|
||||
- Workflow scenarios complete successfully
|
||||
- Performance meets baseline expectations
|
||||
- Integration points work correctly
|
||||
|
||||
### Comprehensive Testing (All Tiers) - Complete Coverage
|
||||
- All tools tested across all scenarios
|
||||
- Edge cases and stress testing completed
|
||||
- Performance baselines established
|
||||
- Full documentation of issues and enhancements
|
||||
|
||||
## Expected Outcomes
|
||||
|
||||
**System Validation:**
|
||||
- Feature verification prioritized by tier importance
|
||||
- Recent changes validated for regression
|
||||
- Performance baseline establishment
|
||||
- Bug identification with severity assessment
|
||||
|
||||
**Knowledge Base Creation:**
|
||||
- Prioritized testing documentation
|
||||
- Real usage examples for user guides
|
||||
- Recent changes validation records
|
||||
- Performance insights for optimization
|
||||
|
||||
**Development Insights:**
|
||||
- Tier-based bug priority list
|
||||
- Recent changes impact assessment
|
||||
- Enhancement ideas from real usage
|
||||
- User experience improvement areas
|
||||
|
||||
## Post-Test Deliverables
|
||||
|
||||
1. **Test Summary Note**
|
||||
- Overall results and findings
|
||||
- Critical issues requiring immediate attention
|
||||
- Enhancement opportunities discovered
|
||||
- System readiness assessment
|
||||
|
||||
2. **Bug Report Collection**
|
||||
- All discovered issues with reproduction steps
|
||||
- Severity and impact assessments
|
||||
- Suggested fixes where applicable
|
||||
|
||||
3. **Performance Baseline**
|
||||
- Timing data for all operations
|
||||
- Scaling behavior observations
|
||||
- Resource usage patterns
|
||||
|
||||
4. **UX Improvement Recommendations**
|
||||
- Usability enhancement suggestions
|
||||
- Documentation improvement areas
|
||||
- Tool design optimization ideas
|
||||
|
||||
5. **Updated TESTING.md**
|
||||
- Incorporate new test scenarios discovered
|
||||
- Update based on real execution experience
|
||||
- Add performance benchmarks and targets
|
||||
|
||||
## Context
|
||||
- Uses real installed basic-memory version
|
||||
- Tests complete MCP→API→DB→File stack
|
||||
- Creates living documentation in Basic Memory itself
|
||||
- Follows integration over isolation philosophy
|
||||
- Prioritizes testing by tool importance and usage frequency
|
||||
- Adapts to recent development changes dynamically
|
||||
- Focuses on real usage patterns over checklist validation
|
||||
- Generates actionable insights prioritized by impact
|
||||
@@ -1,5 +0,0 @@
|
||||
{
|
||||
"enabledPlugins": {
|
||||
"basic-memory@basicmachines": true
|
||||
}
|
||||
}
|
||||
@@ -1,60 +0,0 @@
|
||||
# Git files
|
||||
.git/
|
||||
.gitignore
|
||||
.gitattributes
|
||||
|
||||
# Development files
|
||||
.vscode/
|
||||
.idea/
|
||||
*.swp
|
||||
*.swo
|
||||
*~
|
||||
|
||||
# Testing files
|
||||
tests/
|
||||
test-int/
|
||||
.pytest_cache/
|
||||
.coverage
|
||||
htmlcov/
|
||||
|
||||
# Build artifacts
|
||||
build/
|
||||
dist/
|
||||
*.egg-info/
|
||||
__pycache__/
|
||||
*.pyc
|
||||
*.pyo
|
||||
*.pyd
|
||||
.Python
|
||||
|
||||
# Virtual environments (uv creates these during build)
|
||||
.venv/
|
||||
venv/
|
||||
.env
|
||||
|
||||
# CI/CD files
|
||||
.github/
|
||||
|
||||
# Documentation (keep README.md and pyproject.toml)
|
||||
docs/
|
||||
CHANGELOG.md
|
||||
CLAUDE.md
|
||||
CONTRIBUTING.md
|
||||
|
||||
# Example files not needed for runtime
|
||||
examples/
|
||||
|
||||
# Local development files
|
||||
.basic-memory/
|
||||
*.db
|
||||
*.sqlite3
|
||||
|
||||
# OS files
|
||||
.DS_Store
|
||||
Thumbs.db
|
||||
|
||||
# Temporary files
|
||||
tmp/
|
||||
temp/
|
||||
*.tmp
|
||||
*.log
|
||||
@@ -1,28 +0,0 @@
|
||||
# Basic Memory Environment Variables Example
|
||||
# Copy this file to .env and customize as needed
|
||||
# Note: .env files are gitignored and should never be committed
|
||||
|
||||
# ============================================================================
|
||||
# PostgreSQL Test Database Configuration
|
||||
# ============================================================================
|
||||
# These variables allow you to override the default test database credentials
|
||||
# Default values match docker-compose-postgres.yml for local development
|
||||
#
|
||||
# Only needed if you want to use different credentials or a remote test database
|
||||
# By default, tests use: postgresql://basic_memory_user:dev_password@localhost:5433/basic_memory_test
|
||||
|
||||
# Full PostgreSQL test database URL (used by tests and migrations)
|
||||
# POSTGRES_TEST_URL=postgresql+asyncpg://basic_memory_user:dev_password@localhost:5433/basic_memory_test
|
||||
|
||||
# Individual components (used by justfile postgres-reset command)
|
||||
# POSTGRES_USER=basic_memory_user
|
||||
# POSTGRES_TEST_DB=basic_memory_test
|
||||
|
||||
# ============================================================================
|
||||
# Production Database Configuration
|
||||
# ============================================================================
|
||||
# For production use, set these in your deployment environment
|
||||
# DO NOT use the test credentials above in production!
|
||||
|
||||
# BASIC_MEMORY_DATABASE_BACKEND=postgres # or "sqlite"
|
||||
# BASIC_MEMORY_DATABASE_URL=postgresql+asyncpg://user:password@host:port/database
|
||||
@@ -1,38 +0,0 @@
|
||||
---
|
||||
name: Bug report
|
||||
about: Create a report to help us improve Basic Memory
|
||||
title: '[BUG] '
|
||||
labels: bug
|
||||
assignees: ''
|
||||
---
|
||||
|
||||
## Bug Description
|
||||
A clear and concise description of what the bug is.
|
||||
|
||||
## Steps To Reproduce
|
||||
Steps to reproduce the behavior:
|
||||
1. Install version '...'
|
||||
2. Run command '...'
|
||||
3. Use tool/feature '...'
|
||||
4. See error
|
||||
|
||||
## Expected Behavior
|
||||
A clear and concise description of what you expected to happen.
|
||||
|
||||
## Actual Behavior
|
||||
What actually happened, including error messages and output.
|
||||
|
||||
## Environment
|
||||
- OS: [e.g. macOS 14.2, Ubuntu 22.04]
|
||||
- Python version: [e.g. 3.12.1]
|
||||
- Basic Memory version: [e.g. 0.1.0]
|
||||
- Installation method: [e.g. pip, uv, source]
|
||||
- Claude Desktop version (if applicable):
|
||||
|
||||
## Additional Context
|
||||
- Configuration files (if relevant)
|
||||
- Logs or screenshots
|
||||
- Any special configuration or environment variables
|
||||
|
||||
## Possible Solution
|
||||
If you have any ideas on what might be causing the issue or how to fix it, please share them here.
|
||||
@@ -1,8 +0,0 @@
|
||||
blank_issues_enabled: false
|
||||
contact_links:
|
||||
- name: Basic Memory Discussions
|
||||
url: https://github.com/basicmachines-co/basic-memory/discussions
|
||||
about: For questions, ideas, or more open-ended discussions
|
||||
- name: Documentation
|
||||
url: https://github.com/basicmachines-co/basic-memory#readme
|
||||
about: Please check the documentation first before reporting an issue
|
||||
@@ -1,19 +0,0 @@
|
||||
---
|
||||
name: Documentation improvement
|
||||
about: Suggest improvements or report issues with documentation
|
||||
title: '[DOCS] '
|
||||
labels: documentation
|
||||
assignees: ''
|
||||
---
|
||||
|
||||
## Documentation Issue
|
||||
Describe what's missing, unclear, or incorrect in the current documentation.
|
||||
|
||||
## Location
|
||||
Where is the problematic documentation? (URL, file path, or section)
|
||||
|
||||
## Suggested Improvement
|
||||
How would you improve this documentation? Please be as specific as possible.
|
||||
|
||||
## Additional Context
|
||||
Any additional information or screenshots that might help explain the issue or improvement.
|
||||
@@ -1,28 +0,0 @@
|
||||
---
|
||||
name: Feature request
|
||||
about: Suggest an idea for Basic Memory
|
||||
title: '[FEATURE] '
|
||||
labels: enhancement
|
||||
assignees: ''
|
||||
---
|
||||
|
||||
## Feature Description
|
||||
A clear and concise description of the feature you'd like to see implemented.
|
||||
|
||||
## Problem This Feature Solves
|
||||
Describe the problem or limitation you're experiencing that this feature would address.
|
||||
|
||||
## Proposed Solution
|
||||
Describe how you envision this feature working. Include:
|
||||
- User workflow
|
||||
- Interface design (if applicable)
|
||||
- Technical approach (if you have ideas)
|
||||
|
||||
## Alternative Solutions
|
||||
Have you considered any alternative solutions or workarounds?
|
||||
|
||||
## Additional Context
|
||||
Add any other context, screenshots, or examples about the feature request here.
|
||||
|
||||
## Impact
|
||||
How would this feature benefit you and other users of Basic Memory?
|
||||
@@ -1,12 +0,0 @@
|
||||
# To get started with Dependabot version updates, you'll need to specify which
|
||||
# package ecosystems to update and where the package manifests are located.
|
||||
# Please see the documentation for all configuration options:
|
||||
# https://docs.github.com/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file
|
||||
|
||||
version: 2
|
||||
updates:
|
||||
- package-ecosystem: "" # See documentation for possible values
|
||||
directory: "/" # Location of package manifests
|
||||
schedule:
|
||||
interval: "weekly"
|
||||
|
||||
@@ -1,83 +0,0 @@
|
||||
name: Claude Code Review
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
types: [opened, synchronize]
|
||||
# Optional: Only run on specific file changes
|
||||
# paths:
|
||||
# - "src/**/*.ts"
|
||||
# - "src/**/*.tsx"
|
||||
# - "src/**/*.js"
|
||||
# - "src/**/*.jsx"
|
||||
|
||||
jobs:
|
||||
claude-review:
|
||||
# Only run for organization members and collaborators
|
||||
if: |
|
||||
github.event.pull_request.author_association == 'OWNER' ||
|
||||
github.event.pull_request.author_association == 'MEMBER' ||
|
||||
github.event.pull_request.author_association == 'COLLABORATOR'
|
||||
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
contents: read
|
||||
pull-requests: write
|
||||
issues: read
|
||||
id-token: write
|
||||
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 1
|
||||
|
||||
- name: Run Claude Code Review
|
||||
id: claude-review
|
||||
uses: anthropics/claude-code-action@v1
|
||||
with:
|
||||
claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
|
||||
github_token: ${{ secrets.GITHUB_TOKEN }}
|
||||
track_progress: true # Enable visual progress tracking
|
||||
allowed_bots: '*'
|
||||
prompt: |
|
||||
Review this Basic Memory PR against our team checklist:
|
||||
|
||||
## Code Quality & Standards
|
||||
- [ ] Follows Basic Memory's coding conventions in CLAUDE.md
|
||||
- [ ] Python 3.12+ type annotations and async patterns
|
||||
- [ ] SQLAlchemy 2.0 best practices
|
||||
- [ ] FastAPI and Typer conventions followed
|
||||
- [ ] 100-character line length limit maintained
|
||||
- [ ] No commented-out code blocks
|
||||
|
||||
## Testing & Documentation
|
||||
- [ ] Unit tests for new functions/methods
|
||||
- [ ] Integration tests for new MCP tools
|
||||
- [ ] Test coverage for edge cases
|
||||
- [ ] **100% test coverage maintained** (use `# pragma: no cover` only for truly hard-to-test code)
|
||||
- [ ] Documentation updated (README, docstrings)
|
||||
- [ ] CLAUDE.md updated if conventions change
|
||||
|
||||
## Basic Memory Architecture
|
||||
- [ ] MCP tools follow atomic, composable design
|
||||
- [ ] Database changes include Alembic migrations
|
||||
- [ ] Preserves local-first architecture principles
|
||||
- [ ] Knowledge graph operations maintain consistency
|
||||
- [ ] Markdown file handling preserves integrity
|
||||
- [ ] AI-human collaboration patterns followed
|
||||
|
||||
## Security & Performance
|
||||
- [ ] No hardcoded secrets or credentials
|
||||
- [ ] Input validation for MCP tools
|
||||
- [ ] Proper error handling and logging
|
||||
- [ ] Performance considerations addressed
|
||||
- [ ] No sensitive data in logs or commits
|
||||
|
||||
## Compatability
|
||||
- [ ] File path comparisons must be windows compatible
|
||||
- [ ] Avoid using emojis and unicode characters in console and log output
|
||||
|
||||
Read the CLAUDE.md file for detailed project context. For each checklist item, verify if it's satisfied and comment on any that need attention. Use inline comments for specific code issues and post a summary with checklist results.
|
||||
|
||||
# Allow broader tool access for thorough code review
|
||||
claude_args: '--allowed-tools "Bash(gh pr:*),Bash(gh issue:*),Bash(gh api:*),Bash(git log:*),Bash(git show:*),Read,Grep,Glob"'
|
||||
@@ -1,71 +0,0 @@
|
||||
name: Claude Issue Triage
|
||||
|
||||
on:
|
||||
issues:
|
||||
types: [opened]
|
||||
|
||||
jobs:
|
||||
triage:
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
issues: write
|
||||
id-token: write
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 1
|
||||
|
||||
- name: Run Claude Issue Triage
|
||||
uses: anthropics/claude-code-action@v1
|
||||
with:
|
||||
claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
|
||||
track_progress: true # Show triage progress
|
||||
prompt: |
|
||||
Analyze this new Basic Memory issue and perform triage:
|
||||
|
||||
**Issue Analysis:**
|
||||
1. **Type Classification:**
|
||||
- Bug report (code defect)
|
||||
- Feature request (new functionality)
|
||||
- Enhancement (improvement to existing feature)
|
||||
- Documentation (docs improvement)
|
||||
- Question/Support (user help)
|
||||
- MCP tool issue (specific to MCP functionality)
|
||||
|
||||
2. **Priority Assessment:**
|
||||
- Critical: Security issues, data loss, complete breakage
|
||||
- High: Major functionality broken, affects many users
|
||||
- Medium: Minor bugs, usability issues
|
||||
- Low: Nice-to-have improvements, cosmetic issues
|
||||
|
||||
3. **Component Classification:**
|
||||
- CLI commands
|
||||
- MCP tools
|
||||
- Database/sync
|
||||
- Cloud functionality
|
||||
- Documentation
|
||||
- Testing
|
||||
|
||||
4. **Complexity Estimate:**
|
||||
- Simple: Quick fix, documentation update
|
||||
- Medium: Requires some investigation/testing
|
||||
- Complex: Major feature work, architectural changes
|
||||
|
||||
**Actions to Take:**
|
||||
1. Add appropriate labels using: `gh issue edit ${{ github.event.issue.number }} --add-label "label1,label2"`
|
||||
2. Check for duplicates using: `gh search issues`
|
||||
3. If duplicate found, comment mentioning the original issue
|
||||
4. For feature requests, ask clarifying questions if needed
|
||||
5. For bugs, request reproduction steps if missing
|
||||
|
||||
**Available Labels:**
|
||||
- Type: bug, enhancement, feature, documentation, question, mcp-tool
|
||||
- Priority: critical, high, medium, low
|
||||
- Component: cli, mcp, database, cloud, docs, testing
|
||||
- Complexity: simple, medium, complex
|
||||
- Status: needs-reproduction, needs-clarification, duplicate
|
||||
|
||||
Read the issue carefully and provide helpful triage with appropriate labels.
|
||||
|
||||
claude_args: '--allowed-tools "Bash(gh issue:*),Bash(gh search:*),Read"'
|
||||
@@ -1,68 +0,0 @@
|
||||
name: Claude Code
|
||||
|
||||
on:
|
||||
issue_comment:
|
||||
types: [created]
|
||||
pull_request_review_comment:
|
||||
types: [created]
|
||||
issues:
|
||||
types: [opened, assigned]
|
||||
pull_request_review:
|
||||
types: [submitted]
|
||||
pull_request_target:
|
||||
types: [opened, synchronize]
|
||||
|
||||
jobs:
|
||||
claude:
|
||||
if: |
|
||||
(
|
||||
(github.event_name == 'issue_comment' && contains(github.event.comment.body, '@claude')) ||
|
||||
(github.event_name == 'pull_request_review_comment' && contains(github.event.comment.body, '@claude')) ||
|
||||
(github.event_name == 'pull_request_review' && contains(github.event.review.body, '@claude')) ||
|
||||
(github.event_name == 'issues' && (contains(github.event.issue.body, '@claude') || contains(github.event.issue.title, '@claude'))) ||
|
||||
(github.event_name == 'pull_request_target' && contains(github.event.pull_request.body, '@claude'))
|
||||
) && (
|
||||
github.event.comment.author_association == 'OWNER' ||
|
||||
github.event.comment.author_association == 'MEMBER' ||
|
||||
github.event.comment.author_association == 'COLLABORATOR' ||
|
||||
github.event.sender.author_association == 'OWNER' ||
|
||||
github.event.sender.author_association == 'MEMBER' ||
|
||||
github.event.sender.author_association == 'COLLABORATOR' ||
|
||||
github.event.pull_request.author_association == 'OWNER' ||
|
||||
github.event.pull_request.author_association == 'MEMBER' ||
|
||||
github.event.pull_request.author_association == 'COLLABORATOR'
|
||||
)
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
contents: read
|
||||
pull-requests: read
|
||||
issues: read
|
||||
id-token: write
|
||||
actions: read # Required for Claude to read CI results on PRs
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
# For pull_request_target, checkout the PR head to review the actual changes
|
||||
ref: ${{ github.event_name == 'pull_request_target' && github.event.pull_request.head.sha || github.sha }}
|
||||
fetch-depth: 1
|
||||
|
||||
- name: Run Claude Code
|
||||
id: claude
|
||||
uses: anthropics/claude-code-action@v1
|
||||
with:
|
||||
claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
|
||||
track_progress: true # Enable visual progress tracking
|
||||
|
||||
# This is an optional setting that allows Claude to read CI results on PRs
|
||||
additional_permissions: |
|
||||
actions: read
|
||||
|
||||
# Optional: Give a custom prompt to Claude. If this is not specified, Claude will perform the instructions specified in the comment that tagged it.
|
||||
# prompt: 'Update the pull request description to include a summary of changes.'
|
||||
|
||||
# Optional: Add claude_args to customize behavior and configuration
|
||||
# See https://github.com/anthropics/claude-code-action/blob/main/docs/usage.md
|
||||
# or https://docs.claude.com/en/docs/claude-code/sdk#command-line for available options
|
||||
# claude_args: '--model claude-opus-4-1-20250805 --allowed-tools Bash(gh pr:*)'
|
||||
|
||||
@@ -1,53 +0,0 @@
|
||||
name: Dev Release
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
workflow_dispatch: # Allow manual triggering
|
||||
|
||||
jobs:
|
||||
dev-release:
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
id-token: write
|
||||
contents: write
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: "3.12"
|
||||
|
||||
- name: Install uv
|
||||
run: |
|
||||
pip install uv
|
||||
|
||||
- name: Install dependencies and build
|
||||
run: |
|
||||
uv venv
|
||||
uv sync
|
||||
uv build
|
||||
|
||||
- name: Check if this is a dev version
|
||||
id: check_version
|
||||
run: |
|
||||
VERSION=$(uv run python -c "import basic_memory; print(basic_memory.__version__)")
|
||||
echo "version=$VERSION" >> $GITHUB_OUTPUT
|
||||
if [[ "$VERSION" == *"dev"* ]]; then
|
||||
echo "is_dev=true" >> $GITHUB_OUTPUT
|
||||
echo "Dev version detected: $VERSION"
|
||||
else
|
||||
echo "is_dev=false" >> $GITHUB_OUTPUT
|
||||
echo "Release version detected: $VERSION, skipping dev release"
|
||||
fi
|
||||
|
||||
- name: Publish dev version to PyPI
|
||||
if: steps.check_version.outputs.is_dev == 'true'
|
||||
uses: pypa/gh-action-pypi-publish@release/v1
|
||||
with:
|
||||
password: ${{ secrets.PYPI_TOKEN }}
|
||||
skip-existing: true # Don't fail if version already exists
|
||||
@@ -1,61 +0,0 @@
|
||||
name: Docker Image CI
|
||||
|
||||
on:
|
||||
push:
|
||||
tags:
|
||||
- 'v*' # Trigger on version tags like v1.0.0, v0.13.0, etc.
|
||||
workflow_dispatch: # Allow manual triggering for testing
|
||||
|
||||
env:
|
||||
REGISTRY: ghcr.io
|
||||
IMAGE_NAME: basicmachines-co/basic-memory
|
||||
|
||||
jobs:
|
||||
docker:
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
contents: read
|
||||
packages: write
|
||||
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Set up Docker Buildx
|
||||
uses: docker/setup-buildx-action@v3
|
||||
with:
|
||||
platforms: linux/amd64,linux/arm64
|
||||
|
||||
- name: Log in to GitHub Container Registry
|
||||
uses: docker/login-action@v3
|
||||
with:
|
||||
registry: ${{ env.REGISTRY }}
|
||||
username: ${{ github.actor }}
|
||||
password: ${{ secrets.GITHUB_TOKEN }}
|
||||
|
||||
- name: Extract metadata
|
||||
id: meta
|
||||
uses: docker/metadata-action@v5
|
||||
with:
|
||||
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
|
||||
tags: |
|
||||
type=ref,event=branch
|
||||
type=ref,event=pr
|
||||
type=semver,pattern={{version}}
|
||||
type=semver,pattern={{major}}.{{minor}}
|
||||
type=raw,value=latest,enable={{is_default_branch}}
|
||||
|
||||
- name: Build and push Docker image
|
||||
uses: docker/build-push-action@v5
|
||||
with:
|
||||
context: .
|
||||
file: ./Dockerfile
|
||||
platforms: linux/amd64,linux/arm64
|
||||
push: true
|
||||
tags: ${{ steps.meta.outputs.tags }}
|
||||
labels: ${{ steps.meta.outputs.labels }}
|
||||
cache-from: type=gha
|
||||
cache-to: type=gha,mode=max
|
||||
|
||||
@@ -1,41 +0,0 @@
|
||||
name: "Pull Request Title"
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
types:
|
||||
- opened
|
||||
- edited
|
||||
- synchronize
|
||||
|
||||
jobs:
|
||||
main:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: amannn/action-semantic-pull-request@v5
|
||||
env:
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
with:
|
||||
# Configure allowed types based on what we want in our changelog
|
||||
types: |
|
||||
feat
|
||||
fix
|
||||
chore
|
||||
docs
|
||||
style
|
||||
refactor
|
||||
perf
|
||||
test
|
||||
build
|
||||
ci
|
||||
# Require at least one from scope list (optional)
|
||||
scopes: |
|
||||
core
|
||||
cli
|
||||
api
|
||||
mcp
|
||||
sync
|
||||
ui
|
||||
deps
|
||||
installer
|
||||
# Allow breaking changes (needs "!" after type/scope)
|
||||
requireScopeForBreakingChange: true
|
||||
@@ -1,85 +0,0 @@
|
||||
name: Release
|
||||
|
||||
on:
|
||||
push:
|
||||
tags:
|
||||
- 'v*' # Trigger on version tags like v1.0.0, v0.13.0, etc.
|
||||
|
||||
jobs:
|
||||
release:
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
id-token: write
|
||||
contents: write
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: "3.12"
|
||||
|
||||
- name: Install uv
|
||||
run: |
|
||||
pip install uv
|
||||
|
||||
- name: Install dependencies and build
|
||||
run: |
|
||||
uv venv
|
||||
uv sync
|
||||
uv build
|
||||
|
||||
- name: Verify build succeeded
|
||||
run: |
|
||||
# Verify that build artifacts exist
|
||||
ls -la dist/
|
||||
echo "Build completed successfully"
|
||||
|
||||
- name: Create GitHub Release
|
||||
uses: softprops/action-gh-release@v2
|
||||
with:
|
||||
files: |
|
||||
dist/*.whl
|
||||
dist/*.tar.gz
|
||||
generate_release_notes: true
|
||||
tag_name: ${{ github.ref_name }}
|
||||
token: ${{ secrets.GITHUB_TOKEN }}
|
||||
|
||||
- name: Publish to PyPI
|
||||
uses: pypa/gh-action-pypi-publish@release/v1
|
||||
with:
|
||||
password: ${{ secrets.PYPI_TOKEN }}
|
||||
|
||||
homebrew:
|
||||
name: Update Homebrew Formula
|
||||
needs: release
|
||||
runs-on: ubuntu-latest
|
||||
# Only run for stable releases (not dev, beta, or rc versions)
|
||||
if: ${{ !contains(github.ref_name, 'dev') && !contains(github.ref_name, 'b') && !contains(github.ref_name, 'rc') }}
|
||||
permissions:
|
||||
contents: write
|
||||
actions: read
|
||||
steps:
|
||||
- name: Update Homebrew formula
|
||||
uses: mislav/bump-homebrew-formula-action@v3
|
||||
with:
|
||||
# Formula name in homebrew-basic-memory repo
|
||||
formula-name: basic-memory
|
||||
# The tap repository
|
||||
homebrew-tap: basicmachines-co/homebrew-basic-memory
|
||||
# Base branch of the tap repository
|
||||
base-branch: main
|
||||
# Download URL will be automatically constructed from the tag
|
||||
download-url: https://github.com/basicmachines-co/basic-memory/archive/refs/tags/${{ github.ref_name }}.tar.gz
|
||||
# Commit message for the formula update
|
||||
commit-message: |
|
||||
{{formulaName}} {{version}}
|
||||
|
||||
Created by https://github.com/basicmachines-co/basic-memory/actions/runs/${{ github.run_id }}
|
||||
env:
|
||||
# Personal Access Token with repo scope for homebrew-basic-memory repo
|
||||
COMMITTER_TOKEN: ${{ secrets.HOMEBREW_TOKEN }}
|
||||
|
||||
@@ -1,167 +0,0 @@
|
||||
name: Tests
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [ "main" ]
|
||||
pull_request:
|
||||
branches: [ "main" ]
|
||||
# pull_request_target runs on the BASE of the PR, not the merge result.
|
||||
# It has write permissions and access to secrets.
|
||||
# It's useful for PRs from forks or automated PRs but requires careful use for security reasons.
|
||||
# See: https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#pull_request_target
|
||||
pull_request_target:
|
||||
branches: [ "main" ]
|
||||
|
||||
jobs:
|
||||
test-sqlite:
|
||||
name: Test SQLite (${{ matrix.os }}, Python ${{ matrix.python-version }})
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
os: [ubuntu-latest, windows-latest]
|
||||
python-version: [ "3.12", "3.13", "3.14" ]
|
||||
runs-on: ${{ matrix.os }}
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: true
|
||||
|
||||
- name: Set up Python ${{ matrix.python-version }}
|
||||
uses: actions/setup-python@v4
|
||||
with:
|
||||
python-version: ${{ matrix.python-version }}
|
||||
cache: 'pip'
|
||||
|
||||
- name: Install uv
|
||||
run: |
|
||||
pip install uv
|
||||
|
||||
- name: Install just (Linux/macOS)
|
||||
if: runner.os != 'Windows'
|
||||
run: |
|
||||
curl --proto '=https' --tlsv1.2 -sSf https://just.systems/install.sh | bash -s -- --to /usr/local/bin
|
||||
|
||||
- name: Install just (Windows)
|
||||
if: runner.os == 'Windows'
|
||||
run: |
|
||||
# Install just using Chocolatey (pre-installed on GitHub Actions Windows runners)
|
||||
choco install just --yes
|
||||
shell: pwsh
|
||||
|
||||
- name: Create virtual env
|
||||
run: |
|
||||
uv venv
|
||||
|
||||
- name: Install dependencies
|
||||
run: |
|
||||
uv pip install -e .[dev]
|
||||
|
||||
- name: Run type checks
|
||||
run: |
|
||||
just typecheck
|
||||
|
||||
- name: Run linting
|
||||
run: |
|
||||
just lint
|
||||
|
||||
- name: Run tests (SQLite)
|
||||
run: |
|
||||
uv pip install pytest pytest-cov
|
||||
just test-sqlite
|
||||
|
||||
test-postgres:
|
||||
name: Test Postgres (Python ${{ matrix.python-version }})
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
python-version: [ "3.12", "3.13", "3.14" ]
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
# Note: No services section needed - testcontainers handles Postgres in Docker
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: true
|
||||
|
||||
- name: Set up Python ${{ matrix.python-version }}
|
||||
uses: actions/setup-python@v4
|
||||
with:
|
||||
python-version: ${{ matrix.python-version }}
|
||||
cache: 'pip'
|
||||
|
||||
- name: Install uv
|
||||
run: |
|
||||
pip install uv
|
||||
|
||||
- name: Install just
|
||||
run: |
|
||||
curl --proto '=https' --tlsv1.2 -sSf https://just.systems/install.sh | bash -s -- --to /usr/local/bin
|
||||
|
||||
- name: Create virtual env
|
||||
run: |
|
||||
uv venv
|
||||
|
||||
- name: Install dependencies
|
||||
run: |
|
||||
uv pip install -e .[dev]
|
||||
|
||||
- name: Run tests (Postgres via testcontainers)
|
||||
run: |
|
||||
uv pip install pytest pytest-cov
|
||||
just test-postgres
|
||||
|
||||
coverage:
|
||||
name: Coverage Summary (combined, Python 3.12)
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: true
|
||||
|
||||
- name: Set up Python 3.12
|
||||
uses: actions/setup-python@v4
|
||||
with:
|
||||
python-version: "3.12"
|
||||
cache: "pip"
|
||||
|
||||
- name: Install uv
|
||||
run: |
|
||||
pip install uv
|
||||
|
||||
- name: Install just
|
||||
run: |
|
||||
curl --proto '=https' --tlsv1.2 -sSf https://just.systems/install.sh | bash -s -- --to /usr/local/bin
|
||||
|
||||
- name: Create virtual env
|
||||
run: |
|
||||
uv venv
|
||||
|
||||
- name: Install dependencies
|
||||
run: |
|
||||
uv pip install -e .[dev]
|
||||
|
||||
- name: Run combined coverage (SQLite + Postgres)
|
||||
run: |
|
||||
uv pip install pytest pytest-cov
|
||||
just coverage
|
||||
|
||||
- name: Add coverage report to job summary
|
||||
if: always()
|
||||
run: |
|
||||
{
|
||||
echo "## Coverage"
|
||||
echo ""
|
||||
echo '```'
|
||||
uv run coverage report -m
|
||||
echo '```'
|
||||
} >> "$GITHUB_STEP_SUMMARY"
|
||||
|
||||
- name: Upload HTML coverage report
|
||||
if: always()
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: htmlcov
|
||||
path: htmlcov/
|
||||
-59
@@ -1,59 +0,0 @@
|
||||
*.py[cod]
|
||||
__pycache__/
|
||||
.pytest_cache/
|
||||
.testmondata*
|
||||
.coverage
|
||||
htmlcov/
|
||||
|
||||
# Distribution / packaging
|
||||
.Python
|
||||
build/
|
||||
develop-eggs/
|
||||
dist/
|
||||
downloads/
|
||||
eggs/
|
||||
.eggs/
|
||||
lib/
|
||||
lib64/
|
||||
parts/
|
||||
sdist/
|
||||
var/
|
||||
wheels/
|
||||
*.egg-info/
|
||||
.installed.cfg
|
||||
*.egg
|
||||
|
||||
# Installer artifacts
|
||||
installer/build/
|
||||
installer/dist/
|
||||
rw.*.dmg # Temporary disk images
|
||||
|
||||
# Virtual environments
|
||||
.env
|
||||
.venv
|
||||
env/
|
||||
venv/
|
||||
ENV/
|
||||
|
||||
# IDE
|
||||
.idea/
|
||||
.vscode/
|
||||
*.swp
|
||||
*.swo
|
||||
|
||||
# macOS
|
||||
.DS_Store
|
||||
.coverage.*
|
||||
|
||||
# obsidian docs:
|
||||
/docs/.obsidian/
|
||||
/examples/.obsidian/
|
||||
/examples/.basic-memory/
|
||||
|
||||
|
||||
# claude action
|
||||
claude-output
|
||||
**/.claude/settings.local.json
|
||||
.mcp.json
|
||||
.mcpregistry_*
|
||||
/.testmondata
|
||||
@@ -1 +0,0 @@
|
||||
3.14
|
||||
@@ -1,408 +0,0 @@
|
||||
# AGENTS.md - Basic Memory Project Guide
|
||||
|
||||
## Project Overview
|
||||
|
||||
Basic Memory is a local-first knowledge management system built on the Model Context Protocol (MCP). It enables
|
||||
bidirectional communication between LLMs (like Claude) and markdown files, creating a personal knowledge graph that can
|
||||
be traversed using links between documents.
|
||||
|
||||
## CODEBASE DEVELOPMENT
|
||||
|
||||
### Project information
|
||||
|
||||
See the [README.md](README.md) file for a project overview.
|
||||
|
||||
### Build and Test Commands
|
||||
|
||||
- Install: `just install` or `pip install -e ".[dev]"`
|
||||
- Run all tests (SQLite + Postgres): `just test`
|
||||
- Run all tests against SQLite: `just test-sqlite`
|
||||
- Run all tests against Postgres: `just test-postgres` (uses testcontainers)
|
||||
- Run unit tests (SQLite): `just test-unit-sqlite`
|
||||
- Run unit tests (Postgres): `just test-unit-postgres`
|
||||
- Run integration tests (SQLite): `just test-int-sqlite`
|
||||
- Run integration tests (Postgres): `just test-int-postgres`
|
||||
- Run impacted tests: `just testmon` (pytest-testmon)
|
||||
- Run MCP smoke test: `just test-smoke`
|
||||
- Fast local loop: `just fast-check`
|
||||
- Local consistency check: `just doctor`
|
||||
- Generate HTML coverage: `just coverage`
|
||||
- Single test: `pytest tests/path/to/test_file.py::test_function_name`
|
||||
- Run benchmarks: `pytest test-int/test_sync_performance_benchmark.py -v -m "benchmark and not slow"`
|
||||
- Lint: `just lint` or `ruff check . --fix`
|
||||
- Type check: `just typecheck` or `uv run pyright`
|
||||
- Format: `just format` or `uv run ruff format .`
|
||||
- Run all code checks: `just check` (runs lint, format, typecheck, test)
|
||||
- Create db migration: `just migration "Your migration message"`
|
||||
- Run development MCP Inspector: `just run-inspector`
|
||||
|
||||
**Note:** Project requires Python 3.12+ (uses type parameter syntax and `type` aliases introduced in 3.12)
|
||||
|
||||
**Postgres Testing:** Uses [testcontainers](https://testcontainers-python.readthedocs.io/) which automatically spins up a Postgres instance in Docker. No manual database setup required - just have Docker running.
|
||||
|
||||
**Doctor Note:** `just doctor` runs with a temporary HOME/config so it won't touch your local Basic Memory settings. It leaves temp dirs in `/tmp` (safe to ignore or remove).
|
||||
|
||||
**Testmon Note:** When no files have changed, `just testmon` may collect 0 tests. That's expected and means no impacted tests were detected.
|
||||
|
||||
### Code/Test/Verify Loop (fast path)
|
||||
|
||||
1) **Code:** make changes.
|
||||
2) **Test:** `just fast-check` (lint/format/typecheck + impacted tests + MCP smoke).
|
||||
3) **Verify:** `just doctor` (end-to-end file ↔ DB loop in a temp project).
|
||||
4) **Full gate (when needed):** `just test` or `just check` for SQLite + Postgres.
|
||||
|
||||
If testmon is “cold,” the first run may be long. Subsequent runs get much faster.
|
||||
|
||||
### Test Structure
|
||||
|
||||
- `tests/` - Unit tests for individual components (mocked, fast)
|
||||
- `test-int/` - Integration tests for real-world scenarios (no mocks, realistic)
|
||||
- Both directories are covered by unified coverage reporting
|
||||
- Benchmark tests in `test-int/` are marked with `@pytest.mark.benchmark`
|
||||
- Slow tests are marked with `@pytest.mark.slow`
|
||||
- Smoke tests are marked with `@pytest.mark.smoke`
|
||||
|
||||
### Code Style Guidelines
|
||||
|
||||
- Line length: 100 characters max
|
||||
- Python 3.12+ with full type annotations (uses type parameters and type aliases)
|
||||
- Format with ruff (consistent styling)
|
||||
- Import order: standard lib, third-party, local imports
|
||||
- Naming: snake_case for functions/variables, PascalCase for classes
|
||||
- Prefer async patterns with SQLAlchemy 2.0
|
||||
- Use Pydantic v2 for data validation and schemas
|
||||
- CLI uses Typer for command structure
|
||||
- API uses FastAPI for endpoints
|
||||
- Follow the repository pattern for data access
|
||||
- Tools communicate to api routers via the httpx ASGI client (in process)
|
||||
|
||||
### Code Change Guidelines
|
||||
|
||||
- **Full file read before edits**: Before editing any file, read it in full first to ensure complete context; partial reads lead to corrupted edits
|
||||
- **Minimize diffs**: Prefer the smallest change that satisfies the request. Avoid unrelated refactors or style rewrites unless necessary for correctness
|
||||
- **No speculative getattr**: Never use `getattr(obj, "attr", default)` when unsure about attribute names. Check the class definition or source code first
|
||||
- **Fail fast**: Write code with fail-fast logic by default. Do not swallow exceptions with errors or warnings
|
||||
- **No fallback logic**: Do not add fallback logic unless explicitly told to and agreed with the user
|
||||
- **No guessing**: Do not say "The issue is..." before you actually know what the issue is. Investigate first.
|
||||
|
||||
### Literate Programming Style
|
||||
|
||||
Code should tell a story. Comments must explain the "why" and narrative flow, not just the "what".
|
||||
|
||||
**Section Headers:**
|
||||
For files with multiple phases of logic, add section headers so the control flow reads like chapters:
|
||||
```python
|
||||
# --- Authentication ---
|
||||
# ... auth logic ...
|
||||
|
||||
# --- Data Validation ---
|
||||
# ... validation logic ...
|
||||
|
||||
# --- Business Logic ---
|
||||
# ... core logic ...
|
||||
```
|
||||
|
||||
**Decision Point Comments:**
|
||||
For conditionals that materially change behavior (gates, fallbacks, retries, feature flags), add comments with:
|
||||
- **Trigger**: what condition causes this branch
|
||||
- **Why**: the rationale (cost, correctness, UX, determinism)
|
||||
- **Outcome**: what changes downstream
|
||||
|
||||
```python
|
||||
# Trigger: project has no active sync watcher
|
||||
# Why: avoid duplicate file system watchers consuming resources
|
||||
# Outcome: starts new watcher, registers in active_watchers dict
|
||||
if project_id not in active_watchers:
|
||||
start_watcher(project_id)
|
||||
```
|
||||
|
||||
**Constraint Comments:**
|
||||
If code exists because of a constraint (async requirements, rate limits, schema compatibility), explain the constraint near the code:
|
||||
```python
|
||||
# SQLite requires WAL mode for concurrent read/write access
|
||||
connection.execute("PRAGMA journal_mode=WAL")
|
||||
```
|
||||
|
||||
**What NOT to Comment:**
|
||||
Avoid comments that restate obvious code:
|
||||
```python
|
||||
# Bad - restates code
|
||||
counter += 1 # increment counter
|
||||
|
||||
# Good - explains why
|
||||
counter += 1 # track retries for backoff calculation
|
||||
```
|
||||
|
||||
### Codebase Architecture
|
||||
|
||||
See [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) for detailed architecture documentation.
|
||||
|
||||
**Directory Structure:**
|
||||
- `/alembic` - Alembic db migrations
|
||||
- `/api` - FastAPI REST endpoints + `container.py` composition root
|
||||
- `/cli` - Typer CLI + `container.py` composition root
|
||||
- `/deps` - Feature-scoped FastAPI dependencies (config, db, projects, repositories, services, importers)
|
||||
- `/importers` - Import functionality for Claude, ChatGPT, and other sources
|
||||
- `/markdown` - Markdown parsing and processing
|
||||
- `/mcp` - MCP server + `container.py` composition root + `clients/` typed API clients
|
||||
- `/models` - SQLAlchemy ORM models
|
||||
- `/repository` - Data access layer
|
||||
- `/schemas` - Pydantic models for validation
|
||||
- `/services` - Business logic layer
|
||||
- `/sync` - File synchronization services + `coordinator.py` for lifecycle management
|
||||
|
||||
**Composition Roots:**
|
||||
Each entrypoint (API, MCP, CLI) has a composition root that:
|
||||
- Reads `ConfigManager` (the only place that reads global config)
|
||||
- Resolves runtime mode via `RuntimeMode` enum (TEST > CLOUD > LOCAL)
|
||||
- Provides dependencies to downstream code explicitly
|
||||
|
||||
**Typed API Clients (MCP):**
|
||||
MCP tools use typed clients in `mcp/clients/` to communicate with the API:
|
||||
- `KnowledgeClient` - Entity CRUD operations
|
||||
- `SearchClient` - Search operations
|
||||
- `MemoryClient` - Context building
|
||||
- `DirectoryClient` - Directory listing
|
||||
- `ResourceClient` - Resource reading
|
||||
- `ProjectClient` - Project management
|
||||
|
||||
Flow: MCP Tool → Typed Client → HTTP API → Router → Service → Repository
|
||||
|
||||
### Development Notes
|
||||
|
||||
- MCP tools are defined in src/basic_memory/mcp/tools/
|
||||
- MCP prompts are defined in src/basic_memory/mcp/prompts/
|
||||
- MCP tools should be atomic, composable operations
|
||||
- Use `textwrap.dedent()` for multi-line string formatting in prompts and tools
|
||||
- MCP Prompts are used to invoke tools and format content with instructions for an LLM
|
||||
- Schema changes require Alembic migrations
|
||||
- SQLite is used for indexing and full text search, files are source of truth
|
||||
- Testing uses pytest with asyncio support (strict mode)
|
||||
- Unit tests (`tests/`) use mocks when necessary; integration tests (`test-int/`) use real implementations
|
||||
- By default, tests run against SQLite (fast, no Docker needed)
|
||||
- Set `BASIC_MEMORY_TEST_POSTGRES=1` to run against Postgres (uses testcontainers - Docker required)
|
||||
- Each test runs in a standalone environment with isolated database and tmp_path directory
|
||||
- CI runs SQLite and Postgres tests in parallel for faster feedback
|
||||
- Performance benchmarks are in `test-int/test_sync_performance_benchmark.py`
|
||||
- Use pytest markers: `@pytest.mark.benchmark` for benchmarks, `@pytest.mark.slow` for slow tests
|
||||
- **Coverage must stay at 100%**: Write tests for new code. Only use `# pragma: no cover` when tests would require excessive mocking (e.g., TYPE_CHECKING blocks, error handlers that need failure injection, runtime-mode-dependent code paths)
|
||||
|
||||
### Async Client Pattern (Important!)
|
||||
|
||||
**All MCP tools and CLI commands use the context manager pattern for HTTP clients:**
|
||||
|
||||
```python
|
||||
from basic_memory.mcp.async_client import get_client
|
||||
|
||||
async def my_mcp_tool():
|
||||
async with get_client() as client:
|
||||
# Use client for API calls
|
||||
response = await call_get(client, "/path")
|
||||
return response
|
||||
```
|
||||
|
||||
**Do NOT use:**
|
||||
- ❌ `from basic_memory.mcp.async_client import client` (deprecated module-level client)
|
||||
- ❌ Manual auth header management
|
||||
- ❌ `inject_auth_header()` (deleted)
|
||||
|
||||
**Key principles:**
|
||||
- Auth happens at client creation, not per-request
|
||||
- Proper resource management via context managers
|
||||
- Supports three modes: Local (ASGI), CLI cloud (HTTP + auth), Cloud app (factory injection)
|
||||
- Factory pattern enables dependency injection for cloud consolidation
|
||||
|
||||
**For cloud app integration:**
|
||||
```python
|
||||
from basic_memory.mcp import async_client
|
||||
|
||||
# Set custom factory before importing tools
|
||||
async_client.set_client_factory(your_custom_factory)
|
||||
```
|
||||
|
||||
See SPEC-16 for full context manager refactor details.
|
||||
|
||||
## BASIC MEMORY PRODUCT USAGE
|
||||
|
||||
### Knowledge Structure
|
||||
|
||||
- Entity: Any concept, document, or idea represented as a markdown file
|
||||
- Observation: A categorized fact about an entity (`- [category] content`)
|
||||
- Relation: A directional link between entities (`- relation_type [[Target]]`)
|
||||
- Frontmatter: YAML metadata at the top of markdown files
|
||||
- Knowledge representation follows precise markdown format:
|
||||
- Observations with [category] prefixes
|
||||
- Relations with WikiLinks [[Entity]]
|
||||
- Frontmatter with metadata
|
||||
|
||||
### Basic Memory Commands
|
||||
|
||||
**Local Commands:**
|
||||
- Check sync status: `basic-memory status`
|
||||
- Doctor check (file <-> DB loop): `basic-memory doctor`
|
||||
- Import from Claude: `basic-memory import claude conversations`
|
||||
- Import from ChatGPT: `basic-memory import chatgpt`
|
||||
- Import from Memory JSON: `basic-memory import memory-json`
|
||||
- Tool access: `basic-memory tool` (provides CLI access to MCP tools)
|
||||
- Continue: `basic-memory tool continue-conversation --topic="search"`
|
||||
|
||||
**Project Management:**
|
||||
- List projects: `basic-memory project list`
|
||||
- Add project: `basic-memory project add "name" ~/path`
|
||||
- Project info: `basic-memory project info`
|
||||
- One-way sync (local -> cloud): `basic-memory project sync`
|
||||
- Bidirectional sync: `basic-memory project bisync`
|
||||
- Integrity check: `basic-memory project check`
|
||||
|
||||
**Cloud Commands (requires subscription):**
|
||||
- Authenticate: `basic-memory cloud login`
|
||||
- Logout: `basic-memory cloud logout`
|
||||
- Check cloud status: `basic-memory cloud status`
|
||||
- Setup cloud sync: `basic-memory cloud setup`
|
||||
- Manage snapshots: `basic-memory cloud snapshot [create|list|delete|show|browse]`
|
||||
- Restore from snapshot: `basic-memory cloud restore <path> --snapshot <id>`
|
||||
|
||||
### MCP Capabilities
|
||||
|
||||
- Basic Memory exposes these MCP tools to LLMs:
|
||||
|
||||
**Content Management:**
|
||||
- `write_note(title, content, directory, tags)` - Create/update markdown notes with semantic observations and relations
|
||||
- `read_note(identifier, page, page_size)` - Read notes by title, permalink, or memory:// URL with knowledge graph awareness
|
||||
- `read_content(path)` - Read raw file content (text, images, binaries) without knowledge graph processing
|
||||
- `view_note(identifier, page, page_size)` - View notes as formatted artifacts for better readability
|
||||
- `edit_note(identifier, operation, content)` - Edit notes incrementally (append, prepend, find/replace, replace_section)
|
||||
- `move_note(identifier, destination_path, is_directory)` - Move notes or directories to new locations, updating database and maintaining links
|
||||
- `delete_note(identifier, is_directory)` - Delete notes or directories from the knowledge base
|
||||
|
||||
**Knowledge Graph Navigation:**
|
||||
- `build_context(url, depth, timeframe)` - Navigate the knowledge graph via memory:// URLs for conversation continuity
|
||||
- `recent_activity(type, depth, timeframe)` - Get recently updated information with specified timeframe (e.g., "1d", "1 week")
|
||||
- `list_directory(dir_name, depth, file_name_glob)` - Browse directory contents with filtering and depth control
|
||||
|
||||
**Search & Discovery:**
|
||||
- `search_notes(query, page, page_size, search_type, types, entity_types, after_date)` - Full-text search across all content with advanced filtering options
|
||||
|
||||
**Project Management:**
|
||||
- `list_memory_projects()` - List all available projects with their status
|
||||
- `create_memory_project(project_name, project_path, set_default)` - Create new Basic Memory projects
|
||||
- `delete_project(project_name)` - Delete a project from configuration
|
||||
|
||||
**Visualization:**
|
||||
- `canvas(nodes, edges, title, directory)` - Generate Obsidian canvas files for knowledge graph visualization
|
||||
|
||||
**ChatGPT-Compatible Tools:**
|
||||
- `search(query)` - Search across knowledge base (OpenAI actions compatible)
|
||||
- `fetch(id)` - Fetch full content of a search result document
|
||||
|
||||
- MCP Prompts for better AI interaction:
|
||||
- `ai_assistant_guide()` - Guidance on effectively using Basic Memory tools for AI assistants
|
||||
- `continue_conversation(topic, timeframe)` - Continue previous conversations with relevant historical context
|
||||
- `search(query, after_date)` - Search with detailed, formatted results for better context understanding
|
||||
- `recent_activity(timeframe)` - View recently changed items with formatted output
|
||||
|
||||
### Cloud Features (v0.15.0+)
|
||||
|
||||
Basic Memory now supports cloud synchronization and storage (requires active subscription):
|
||||
|
||||
**Authentication:**
|
||||
- JWT-based authentication with subscription validation
|
||||
- Secure session management with token refresh
|
||||
- Support for multiple cloud projects
|
||||
|
||||
**Bidirectional Sync:**
|
||||
- rclone bisync integration for two-way synchronization
|
||||
- Conflict resolution and integrity verification
|
||||
- Real-time sync with change detection
|
||||
- Mount/unmount cloud storage for direct file access
|
||||
|
||||
**Cloud Project Management:**
|
||||
- Create and manage projects in the cloud
|
||||
- Toggle between local and cloud modes
|
||||
- Per-project sync configuration
|
||||
- Subscription-based access control
|
||||
|
||||
**Security & Performance:**
|
||||
- Removed .env file loading for improved security
|
||||
- .gitignore integration (respects gitignored files)
|
||||
- WAL mode for SQLite performance
|
||||
- Background relation resolution (non-blocking startup)
|
||||
- API performance optimizations (SPEC-11)
|
||||
|
||||
**CLI Routing Flags:**
|
||||
|
||||
When cloud mode is enabled, CLI commands route to the cloud API by default. Use `--local` and `--cloud` flags to override:
|
||||
|
||||
```bash
|
||||
# Force local routing (ignore cloud mode)
|
||||
basic-memory status --local
|
||||
basic-memory project list --local
|
||||
|
||||
# Force cloud routing (when cloud mode is disabled)
|
||||
basic-memory status --cloud
|
||||
basic-memory project info my-project --cloud
|
||||
```
|
||||
|
||||
Key behaviors:
|
||||
- The local MCP server (`basic-memory mcp`) automatically uses local routing
|
||||
- This allows simultaneous use of local Claude Desktop and cloud-based clients
|
||||
- Some commands (like `project default`, `project sync-config`, `project move`) require `--local` in cloud mode since they modify local configuration
|
||||
- Environment variable `BASIC_MEMORY_FORCE_LOCAL=true` forces local routing globally
|
||||
|
||||
## AI-Human Collaborative Development
|
||||
|
||||
Basic Memory emerged from and enables a new kind of development process that combines human and AI capabilities. Instead
|
||||
of using AI just for code generation, we've developed a true collaborative workflow:
|
||||
|
||||
1. AI (LLM) writes initial implementation based on specifications and context
|
||||
2. Human reviews, runs tests, and commits code with any necessary adjustments
|
||||
3. Knowledge persists across conversations using Basic Memory's knowledge graph
|
||||
4. Development continues seamlessly across different AI sessions with consistent context
|
||||
5. Results improve through iterative collaboration and shared understanding
|
||||
|
||||
This approach has allowed us to tackle more complex challenges and build a more robust system than either humans or AI
|
||||
could achieve independently.
|
||||
|
||||
**Problem-Solving Guidance:**
|
||||
- If a solution isn't working after reasonable effort, suggest alternative approaches
|
||||
- Don't persist with a problematic library or pattern when better alternatives exist
|
||||
- Example: When py-pglite caused cascading test failures, switching to testcontainers-postgres was the right call
|
||||
|
||||
## GitHub Integration
|
||||
|
||||
Basic Memory has taken AI-Human collaboration to the next level by integrating Claude directly into the development workflow through GitHub:
|
||||
|
||||
### GitHub MCP Tools
|
||||
|
||||
Using the GitHub Model Context Protocol server, Claude can now:
|
||||
|
||||
- **Repository Management**:
|
||||
- View repository files and structure
|
||||
- Read file contents
|
||||
- Create new branches
|
||||
- Create and update files
|
||||
|
||||
- **Issue Management**:
|
||||
- Create new issues
|
||||
- Comment on existing issues
|
||||
- Close and update issues
|
||||
- Search across issues
|
||||
|
||||
- **Pull Request Workflow**:
|
||||
- Create pull requests
|
||||
- Review code changes
|
||||
- Add comments to PRs
|
||||
|
||||
This integration enables Claude to participate as a full team member in the development process, not just as a code generation tool. Claude's GitHub account ([bm-claudeai](https://github.com/bm-claudeai)) is a member of the Basic Machines organization with direct contributor access to the codebase.
|
||||
|
||||
### Collaborative Development Process
|
||||
|
||||
With GitHub integration, the development workflow includes:
|
||||
|
||||
1. **Direct code review** - Claude can analyze PRs and provide detailed feedback
|
||||
2. **Contribution tracking** - All of Claude's contributions are properly attributed in the Git history
|
||||
3. **Branch management** - Claude can create feature branches for implementations
|
||||
4. **Documentation maintenance** - Claude can keep documentation updated as the code evolves
|
||||
5. **Code Commits**: ALWAYS sign off commits with `git commit -s`
|
||||
|
||||
This level of integration represents a new paradigm in AI-human collaboration, where the AI assistant becomes a full-fledged team member rather than just a tool for generating code snippets.
|
||||
-2446
File diff suppressed because it is too large
Load Diff
@@ -1,10 +0,0 @@
|
||||
cff-version: 1.0.3
|
||||
message: "If you use this project, please cite it as follows:"
|
||||
authors:
|
||||
- family-names: "Hernandez"
|
||||
given-names: "Paul"
|
||||
affiliation: "Basic Machines"
|
||||
title: "Basic Memory"
|
||||
version: "0.0.1"
|
||||
date-released: "2025-02-03"
|
||||
url: "https://github.com/basicmachines-co/basic-memory"
|
||||
@@ -1,71 +0,0 @@
|
||||
# Contributor License Agreement
|
||||
|
||||
## Copyright Assignment and License Grant
|
||||
|
||||
By signing this Contributor License Agreement ("Agreement"), you accept and agree to the following terms and conditions
|
||||
for your present and future Contributions submitted
|
||||
to Basic Machines LLC. Except for the license granted herein to Basic Machines LLC and recipients of software
|
||||
distributed by Basic Machines LLC, you reserve all right,
|
||||
title, and interest in and to your Contributions.
|
||||
|
||||
### 1. Definitions
|
||||
|
||||
"You" (or "Your") shall mean the copyright owner or legal entity authorized by the copyright owner that is making this
|
||||
Agreement with Basic Machines LLC.
|
||||
|
||||
"Contribution" shall mean any original work of authorship, including any modifications or additions to an existing work,
|
||||
that is intentionally submitted by You to Basic
|
||||
Machines LLC for inclusion in, or documentation of, any of the products owned or managed by Basic Machines LLC (the "
|
||||
Work").
|
||||
|
||||
### 2. Grant of Copyright License
|
||||
|
||||
Subject to the terms and conditions of this Agreement, You hereby grant to Basic Machines LLC and to recipients of
|
||||
software distributed by Basic Machines LLC a perpetual,
|
||||
worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to use, copy, modify, merge, publish,
|
||||
distribute, sublicense, and/or sell copies of the
|
||||
Work, and to permit persons to whom the Work is furnished to do so.
|
||||
|
||||
### 3. Assignment of Copyright
|
||||
|
||||
You hereby assign to Basic Machines LLC all right, title, and interest worldwide in all Copyright covering your
|
||||
Contributions. Basic Machines LLC may license the
|
||||
Contributions under any license terms, including copyleft, permissive, commercial, or proprietary licenses.
|
||||
|
||||
### 4. Grant of Patent License
|
||||
|
||||
Subject to the terms and conditions of this Agreement, You hereby grant to Basic Machines LLC and to recipients of
|
||||
software distributed by Basic Machines LLC a perpetual,
|
||||
worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to
|
||||
make, have made, use, offer to sell, sell, import, and
|
||||
otherwise transfer the Work.
|
||||
|
||||
### 5. Developer Certificate of Origin
|
||||
|
||||
By making a Contribution to this project, You certify that:
|
||||
|
||||
(a) The Contribution was created in whole or in part by You and You have the right to submit it under this Agreement; or
|
||||
|
||||
(b) The Contribution is based upon previous work that, to the best of Your knowledge, is covered under an appropriate
|
||||
open source license and You have the right under that
|
||||
license to submit that work with modifications, whether created in whole or in part by You, under this Agreement; or
|
||||
|
||||
(c) The Contribution was provided directly to You by some other person who certified (a), (b) or (c) and You have not
|
||||
modified it.
|
||||
|
||||
(d) You understand and agree that this project and the Contribution are public and that a record of the Contribution (
|
||||
including all personal information You submit with
|
||||
it, including Your sign-off) is maintained indefinitely and may be redistributed consistent with this project or the
|
||||
open source license(s) involved.
|
||||
|
||||
### 6. Representations
|
||||
|
||||
You represent that you are legally entitled to grant the above license and assignment. If your employer(s) has rights to
|
||||
intellectual property that you create that
|
||||
includes your Contributions, you represent that you have received permission to make Contributions on behalf of that
|
||||
employer, or that your employer has waived such rights
|
||||
for your Contributions to Basic Machines LLC.
|
||||
|
||||
---
|
||||
|
||||
This Agreement is effective as of the date you first submit a Contribution to Basic Machines LLC.
|
||||
@@ -1,19 +0,0 @@
|
||||
# Code of Conduct
|
||||
|
||||
## Purpose
|
||||
|
||||
Maintain a respectful and professional environment where contributions can be made without harassment or
|
||||
negativity.
|
||||
|
||||
## Standards
|
||||
|
||||
Respectful communication and collaboration are expected. Offensive behavior, harassment, or personal attacks will not be
|
||||
tolerated.
|
||||
|
||||
## Reporting Issues
|
||||
|
||||
To report inappropriate behavior, contact [paul@basicmachines.co].
|
||||
|
||||
## Consequences
|
||||
|
||||
Violations of this code may lead to consequences, including being banned from contributing to the project.
|
||||
-277
@@ -1,277 +0,0 @@
|
||||
# Contributing to Basic Memory
|
||||
|
||||
Thank you for considering contributing to Basic Memory! This document outlines the process for contributing to the
|
||||
project and how to get started as a developer.
|
||||
|
||||
## Getting Started
|
||||
|
||||
### Development Environment
|
||||
|
||||
1. **Clone the Repository**:
|
||||
```bash
|
||||
git clone https://github.com/basicmachines-co/basic-memory.git
|
||||
cd basic-memory
|
||||
```
|
||||
|
||||
2. **Install Dependencies**:
|
||||
```bash
|
||||
# Using just (recommended)
|
||||
just install
|
||||
|
||||
# Or using uv
|
||||
uv install -e ".[dev]"
|
||||
|
||||
# Or using pip
|
||||
pip install -e ".[dev]"
|
||||
```
|
||||
|
||||
> **Note**: Basic Memory uses [just](https://just.systems) as a modern command runner. Install with `brew install just` or `cargo install just`.
|
||||
|
||||
3. **Activate the Virtual Environment**
|
||||
```bash
|
||||
source .venv/bin/activate
|
||||
```
|
||||
|
||||
4. **Run the Tests**:
|
||||
```bash
|
||||
# Run all tests with unified coverage (unit + integration)
|
||||
just test
|
||||
|
||||
# Run unit tests only (fast, no coverage)
|
||||
just test-unit
|
||||
|
||||
# Run integration tests only (fast, no coverage)
|
||||
just test-int
|
||||
|
||||
# Generate HTML coverage report
|
||||
just coverage
|
||||
|
||||
# Run a specific test
|
||||
pytest tests/path/to/test_file.py::test_function_name
|
||||
```
|
||||
|
||||
### Development Workflow
|
||||
|
||||
1. **Fork the Repo**: Fork the repository on GitHub and clone your copy.
|
||||
2. **Create a Branch**: Create a new branch for your feature or fix.
|
||||
```bash
|
||||
git checkout -b feature/your-feature-name
|
||||
# or
|
||||
git checkout -b fix/issue-you-are-fixing
|
||||
```
|
||||
3. **Make Your Changes**: Implement your changes with appropriate test coverage.
|
||||
4. **Check Code Quality**:
|
||||
```bash
|
||||
# Run all checks at once
|
||||
just check
|
||||
|
||||
# Or run individual checks
|
||||
just lint # Run linting
|
||||
just format # Format code
|
||||
just type-check # Type checking
|
||||
```
|
||||
5. **Test Your Changes**: Ensure all tests pass locally and maintain 100% test coverage.
|
||||
```bash
|
||||
just test
|
||||
```
|
||||
6. **Submit a PR**: Submit a pull request with a detailed description of your changes.
|
||||
|
||||
## LLM-Assisted Development
|
||||
|
||||
This project is designed for collaborative development between humans and LLMs (Large Language Models):
|
||||
|
||||
1. **CLAUDE.md**: The repository includes a `CLAUDE.md` file that serves as a project guide for both humans and LLMs.
|
||||
This file contains:
|
||||
- Key project information and architectural overview
|
||||
- Development commands and workflows
|
||||
- Code style guidelines
|
||||
- Documentation standards
|
||||
|
||||
2. **AI-Human Collaborative Workflow**:
|
||||
- We encourage using LLMs like Claude for code generation, reviews, and documentation
|
||||
- When possible, save context in markdown files that can be referenced later
|
||||
- This enables seamless knowledge transfer between different development sessions
|
||||
- Claude can help with implementation details while you focus on architecture and design
|
||||
|
||||
3. **Adding to CLAUDE.md**:
|
||||
- If you discover useful project information or common commands, consider adding them to CLAUDE.md
|
||||
- This helps all contributors (human and AI) maintain consistent knowledge of the project
|
||||
|
||||
## Pull Request Process
|
||||
|
||||
1. **Create a Pull Request**: Open a PR against the `main` branch with a clear title and description.
|
||||
2. **Sign the Developer Certificate of Origin (DCO)**: All contributions require signing our DCO, which certifies that
|
||||
you have the right to submit your contributions. This will be automatically checked by our CLA assistant when you
|
||||
create a PR.
|
||||
3. **PR Description**: Include:
|
||||
- What the PR changes
|
||||
- Why the change is needed
|
||||
- How you tested the changes
|
||||
- Any related issues (use "Fixes #123" to automatically close issues)
|
||||
4. **Code Review**: Wait for code review and address any feedback.
|
||||
5. **CI Checks**: Ensure all CI checks pass.
|
||||
6. **Merge**: Once approved, a maintainer will merge your PR.
|
||||
|
||||
## Developer Certificate of Origin
|
||||
|
||||
By contributing to this project, you agree to the [Developer Certificate of Origin (DCO)](CLA.md). This means you
|
||||
certify that:
|
||||
|
||||
- You have the right to submit your contributions
|
||||
- You're not knowingly submitting code with patent or copyright issues
|
||||
- Your contributions are provided under the project's license (AGPL-3.0)
|
||||
|
||||
This is a lightweight alternative to a Contributor License Agreement and helps ensure that all contributions can be
|
||||
properly incorporated into the project and potentially used in commercial applications.
|
||||
|
||||
### Signing Your Commits
|
||||
|
||||
Sign your commit:
|
||||
|
||||
**Using the `-s` or `--signoff` flag**:
|
||||
|
||||
```bash
|
||||
git commit -s -m "Your commit message"
|
||||
```
|
||||
|
||||
This adds a `Signed-off-by` line to your commit message, certifying that you adhere to the DCO.
|
||||
|
||||
The sign-off certifies that you have the right to submit your contribution under the project's license and verifies your
|
||||
agreement to the DCO.
|
||||
|
||||
## Code Style Guidelines
|
||||
|
||||
- **Python Version**: Python 3.12+ with full type annotations (3.12+ required for type parameter syntax)
|
||||
- **Line Length**: 100 characters maximum
|
||||
- **Formatting**: Use ruff for consistent styling
|
||||
- **Import Order**: Standard lib, third-party, local imports
|
||||
- **Naming**: Use snake_case for functions/variables, PascalCase for classes
|
||||
- **Documentation**: Add docstrings to public functions, classes, and methods
|
||||
- **Type Annotations**: Use type hints for all functions and methods
|
||||
|
||||
## Testing Guidelines
|
||||
|
||||
### Test Structure
|
||||
|
||||
Basic Memory uses two test directories with unified coverage reporting:
|
||||
|
||||
- **`tests/`**: Unit tests that test individual components in isolation
|
||||
- Fast execution with extensive mocking
|
||||
- Test individual functions, classes, and modules
|
||||
- Run with: `just test-unit` (no coverage, fast)
|
||||
|
||||
- **`test-int/`**: Integration tests that test real-world scenarios
|
||||
- Test full workflows with real database and file operations
|
||||
- Include performance benchmarks
|
||||
- More realistic but slower than unit tests
|
||||
- Run with: `just test-int` (no coverage, fast)
|
||||
|
||||
### Running Tests
|
||||
|
||||
```bash
|
||||
# Run all tests with unified coverage report
|
||||
just test
|
||||
|
||||
# Run only unit tests (fast iteration)
|
||||
just test-unit
|
||||
|
||||
# Run only integration tests
|
||||
just test-int
|
||||
|
||||
# Generate HTML coverage report
|
||||
just coverage
|
||||
|
||||
# Run specific test
|
||||
pytest tests/path/to/test_file.py::test_function_name
|
||||
|
||||
# Run tests excluding benchmarks
|
||||
pytest -m "not benchmark"
|
||||
|
||||
# Run only benchmark tests
|
||||
pytest -m benchmark test-int/test_sync_performance_benchmark.py
|
||||
```
|
||||
|
||||
### Performance Benchmarks
|
||||
|
||||
The `test-int/test_sync_performance_benchmark.py` file contains performance benchmarks that measure sync and indexing speed:
|
||||
|
||||
- `test_benchmark_sync_100_files` - Small repository performance
|
||||
- `test_benchmark_sync_500_files` - Medium repository performance
|
||||
- `test_benchmark_sync_1000_files` - Large repository performance (marked slow)
|
||||
- `test_benchmark_resync_no_changes` - Re-sync performance baseline
|
||||
|
||||
Run benchmarks with:
|
||||
```bash
|
||||
# Run all benchmarks (excluding slow ones)
|
||||
pytest test-int/test_sync_performance_benchmark.py -v -m "benchmark and not slow"
|
||||
|
||||
# Run all benchmarks including slow ones
|
||||
pytest test-int/test_sync_performance_benchmark.py -v -m benchmark
|
||||
|
||||
# Run specific benchmark
|
||||
pytest test-int/test_sync_performance_benchmark.py::test_benchmark_sync_100_files -v
|
||||
```
|
||||
|
||||
See `test-int/BENCHMARKS.md` for detailed benchmark documentation.
|
||||
|
||||
### Testing Best Practices
|
||||
|
||||
- **Coverage Target**: We aim for high test coverage for all code
|
||||
- **Test Framework**: Use pytest for unit and integration tests
|
||||
- **Mocking**: Avoid mocking in integration tests; use sparingly in unit tests
|
||||
- **Edge Cases**: Test both normal operation and edge cases
|
||||
- **Database Testing**: Use in-memory SQLite for testing database operations
|
||||
- **Fixtures**: Use async pytest fixtures for setup and teardown
|
||||
- **Markers**: Use `@pytest.mark.benchmark` for benchmarks, `@pytest.mark.slow` for slow tests
|
||||
|
||||
## Release Process
|
||||
|
||||
Basic Memory uses automatic versioning based on git tags with `uv-dynamic-versioning`. Here's how releases work:
|
||||
|
||||
### Version Management
|
||||
- **Development versions**: Automatically generated from git commits (e.g., `0.12.4.dev26+468a22f`)
|
||||
- **Beta releases**: Created by tagging with beta suffixes (e.g., `git tag v0.13.0b1`)
|
||||
- **Stable releases**: Created by tagging with version numbers (e.g., `git tag v0.13.0`)
|
||||
|
||||
### Release Workflows
|
||||
|
||||
#### Development Builds
|
||||
- Automatically published to PyPI on every commit to `main`
|
||||
- Version format: `0.12.4.dev26+468a22f` (base version + dev + commit count + hash)
|
||||
- Users install with: `pip install basic-memory --pre --force-reinstall`
|
||||
|
||||
#### Beta Releases
|
||||
1. Create and push a beta tag: `git tag v0.13.0b1 && git push origin v0.13.0b1`
|
||||
2. GitHub Actions automatically builds and publishes to PyPI
|
||||
3. Users install with: `pip install basic-memory --pre`
|
||||
|
||||
#### Stable Releases
|
||||
1. Create and push a version tag: `git tag v0.13.0 && git push origin v0.13.0`
|
||||
2. GitHub Actions automatically:
|
||||
- Builds the package with version `0.13.0`
|
||||
- Creates GitHub release with auto-generated notes
|
||||
- Publishes to PyPI
|
||||
3. Users install with: `pip install basic-memory`
|
||||
|
||||
### For Contributors
|
||||
- No manual version bumping required
|
||||
- Versions are automatically derived from git tags
|
||||
- Focus on code changes, not version management
|
||||
|
||||
## Creating Issues
|
||||
|
||||
If you're planning to work on something, please create an issue first to discuss the approach. Include:
|
||||
|
||||
- A clear title and description
|
||||
- Steps to reproduce if reporting a bug
|
||||
- Expected behavior vs. actual behavior
|
||||
- Any relevant logs or screenshots
|
||||
- Your proposed solution, if you have one
|
||||
|
||||
## Code of Conduct
|
||||
|
||||
All contributors must follow the [Code of Conduct](CODE_OF_CONDUCT.md).
|
||||
|
||||
## Thank You!
|
||||
|
||||
Your contributions help make Basic Memory better. We appreciate your time and effort!
|
||||
-52
@@ -1,52 +0,0 @@
|
||||
FROM python:3.12-slim-bookworm
|
||||
|
||||
# Build arguments for user ID and group ID (defaults to 1000)
|
||||
ARG UID=1000
|
||||
ARG GID=1000
|
||||
|
||||
# Copy uv from official image
|
||||
COPY --from=ghcr.io/astral-sh/uv:latest /uv /uvx /bin/
|
||||
|
||||
# Set environment variables
|
||||
# UV_PYTHON_INSTALL_DIR ensures Python is installed to a persistent location
|
||||
# that survives in the final image (not in /root/.local which gets lost)
|
||||
# UV_PYTHON_PREFERENCE=only-managed tells uv to use its managed Python version
|
||||
ENV PYTHONUNBUFFERED=1 \
|
||||
PYTHONDONTWRITEBYTECODE=1 \
|
||||
UV_PYTHON_INSTALL_DIR=/python \
|
||||
UV_PYTHON_PREFERENCE=only-managed
|
||||
|
||||
# Create a group and user with the provided UID/GID
|
||||
# Check if the GID already exists, if not create appgroup
|
||||
RUN (getent group ${GID} || groupadd --gid ${GID} appgroup) && \
|
||||
useradd --uid ${UID} --gid ${GID} --create-home --shell /bin/bash appuser
|
||||
|
||||
# Copy the project into the image
|
||||
ADD . /app
|
||||
|
||||
# Install Python 3.13 explicitly and sync the project
|
||||
WORKDIR /app
|
||||
RUN uv python install 3.13
|
||||
RUN uv sync --locked --python 3.13
|
||||
|
||||
# Create necessary directories and set ownership
|
||||
RUN mkdir -p /app/data/basic-memory /app/.basic-memory && \
|
||||
chown -R appuser:${GID} /app
|
||||
|
||||
# Set default data directory and add venv to PATH
|
||||
ENV BASIC_MEMORY_HOME=/app/data/basic-memory \
|
||||
BASIC_MEMORY_PROJECT_ROOT=/app/data \
|
||||
PATH="/app/.venv/bin:$PATH"
|
||||
|
||||
# Switch to the non-root user
|
||||
USER appuser
|
||||
|
||||
# Expose port
|
||||
EXPOSE 8000
|
||||
|
||||
# Health check
|
||||
HEALTHCHECK --interval=30s --timeout=10s --start-period=5s --retries=3 \
|
||||
CMD basic-memory --version || exit 1
|
||||
|
||||
# Use the basic-memory entrypoint to run the MCP server with default SSE transport
|
||||
CMD ["basic-memory", "mcp", "--transport", "sse", "--host", "0.0.0.0", "--port", "8000"]
|
||||
@@ -1,661 +0,0 @@
|
||||
GNU AFFERO GENERAL PUBLIC LICENSE
|
||||
Version 3, 19 November 2007
|
||||
|
||||
Copyright (C) 2007 Free Software Foundation, Inc. <https://fsf.org/>
|
||||
Everyone is permitted to copy and distribute verbatim copies
|
||||
of this license document, but changing it is not allowed.
|
||||
|
||||
Preamble
|
||||
|
||||
The GNU Affero General Public License is a free, copyleft license for
|
||||
software and other kinds of works, specifically designed to ensure
|
||||
cooperation with the community in the case of network server software.
|
||||
|
||||
The licenses for most software and other practical works are designed
|
||||
to take away your freedom to share and change the works. By contrast,
|
||||
our General Public Licenses are intended to guarantee your freedom to
|
||||
share and change all versions of a program--to make sure it remains free
|
||||
software for all its users.
|
||||
|
||||
When we speak of free software, we are referring to freedom, not
|
||||
price. Our General Public Licenses are designed to make sure that you
|
||||
have the freedom to distribute copies of free software (and charge for
|
||||
them if you wish), that you receive source code or can get it if you
|
||||
want it, that you can change the software or use pieces of it in new
|
||||
free programs, and that you know you can do these things.
|
||||
|
||||
Developers that use our General Public Licenses protect your rights
|
||||
with two steps: (1) assert copyright on the software, and (2) offer
|
||||
you this License which gives you legal permission to copy, distribute
|
||||
and/or modify the software.
|
||||
|
||||
A secondary benefit of defending all users' freedom is that
|
||||
improvements made in alternate versions of the program, if they
|
||||
receive widespread use, become available for other developers to
|
||||
incorporate. Many developers of free software are heartened and
|
||||
encouraged by the resulting cooperation. However, in the case of
|
||||
software used on network servers, this result may fail to come about.
|
||||
The GNU General Public License permits making a modified version and
|
||||
letting the public access it on a server without ever releasing its
|
||||
source code to the public.
|
||||
|
||||
The GNU Affero General Public License is designed specifically to
|
||||
ensure that, in such cases, the modified source code becomes available
|
||||
to the community. It requires the operator of a network server to
|
||||
provide the source code of the modified version running there to the
|
||||
users of that server. Therefore, public use of a modified version, on
|
||||
a publicly accessible server, gives the public access to the source
|
||||
code of the modified version.
|
||||
|
||||
An older license, called the Affero General Public License and
|
||||
published by Affero, was designed to accomplish similar goals. This is
|
||||
a different license, not a version of the Affero GPL, but Affero has
|
||||
released a new version of the Affero GPL which permits relicensing under
|
||||
this license.
|
||||
|
||||
The precise terms and conditions for copying, distribution and
|
||||
modification follow.
|
||||
|
||||
TERMS AND CONDITIONS
|
||||
|
||||
0. Definitions.
|
||||
|
||||
"This License" refers to version 3 of the GNU Affero General Public License.
|
||||
|
||||
"Copyright" also means copyright-like laws that apply to other kinds of
|
||||
works, such as semiconductor masks.
|
||||
|
||||
"The Program" refers to any copyrightable work licensed under this
|
||||
License. Each licensee is addressed as "you". "Licensees" and
|
||||
"recipients" may be individuals or organizations.
|
||||
|
||||
To "modify" a work means to copy from or adapt all or part of the work
|
||||
in a fashion requiring copyright permission, other than the making of an
|
||||
exact copy. The resulting work is called a "modified version" of the
|
||||
earlier work or a work "based on" the earlier work.
|
||||
|
||||
A "covered work" means either the unmodified Program or a work based
|
||||
on the Program.
|
||||
|
||||
To "propagate" a work means to do anything with it that, without
|
||||
permission, would make you directly or secondarily liable for
|
||||
infringement under applicable copyright law, except executing it on a
|
||||
computer or modifying a private copy. Propagation includes copying,
|
||||
distribution (with or without modification), making available to the
|
||||
public, and in some countries other activities as well.
|
||||
|
||||
To "convey" a work means any kind of propagation that enables other
|
||||
parties to make or receive copies. Mere interaction with a user through
|
||||
a computer network, with no transfer of a copy, is not conveying.
|
||||
|
||||
An interactive user interface displays "Appropriate Legal Notices"
|
||||
to the extent that it includes a convenient and prominently visible
|
||||
feature that (1) displays an appropriate copyright notice, and (2)
|
||||
tells the user that there is no warranty for the work (except to the
|
||||
extent that warranties are provided), that licensees may convey the
|
||||
work under this License, and how to view a copy of this License. If
|
||||
the interface presents a list of user commands or options, such as a
|
||||
menu, a prominent item in the list meets this criterion.
|
||||
|
||||
1. Source Code.
|
||||
|
||||
The "source code" for a work means the preferred form of the work
|
||||
for making modifications to it. "Object code" means any non-source
|
||||
form of a work.
|
||||
|
||||
A "Standard Interface" means an interface that either is an official
|
||||
standard defined by a recognized standards body, or, in the case of
|
||||
interfaces specified for a particular programming language, one that
|
||||
is widely used among developers working in that language.
|
||||
|
||||
The "System Libraries" of an executable work include anything, other
|
||||
than the work as a whole, that (a) is included in the normal form of
|
||||
packaging a Major Component, but which is not part of that Major
|
||||
Component, and (b) serves only to enable use of the work with that
|
||||
Major Component, or to implement a Standard Interface for which an
|
||||
implementation is available to the public in source code form. A
|
||||
"Major Component", in this context, means a major essential component
|
||||
(kernel, window system, and so on) of the specific operating system
|
||||
(if any) on which the executable work runs, or a compiler used to
|
||||
produce the work, or an object code interpreter used to run it.
|
||||
|
||||
The "Corresponding Source" for a work in object code form means all
|
||||
the source code needed to generate, install, and (for an executable
|
||||
work) run the object code and to modify the work, including scripts to
|
||||
control those activities. However, it does not include the work's
|
||||
System Libraries, or general-purpose tools or generally available free
|
||||
programs which are used unmodified in performing those activities but
|
||||
which are not part of the work. For example, Corresponding Source
|
||||
includes interface definition files associated with source files for
|
||||
the work, and the source code for shared libraries and dynamically
|
||||
linked subprograms that the work is specifically designed to require,
|
||||
such as by intimate data communication or control flow between those
|
||||
subprograms and other parts of the work.
|
||||
|
||||
The Corresponding Source need not include anything that users
|
||||
can regenerate automatically from other parts of the Corresponding
|
||||
Source.
|
||||
|
||||
The Corresponding Source for a work in source code form is that
|
||||
same work.
|
||||
|
||||
2. Basic Permissions.
|
||||
|
||||
All rights granted under this License are granted for the term of
|
||||
copyright on the Program, and are irrevocable provided the stated
|
||||
conditions are met. This License explicitly affirms your unlimited
|
||||
permission to run the unmodified Program. The output from running a
|
||||
covered work is covered by this License only if the output, given its
|
||||
content, constitutes a covered work. This License acknowledges your
|
||||
rights of fair use or other equivalent, as provided by copyright law.
|
||||
|
||||
You may make, run and propagate covered works that you do not
|
||||
convey, without conditions so long as your license otherwise remains
|
||||
in force. You may convey covered works to others for the sole purpose
|
||||
of having them make modifications exclusively for you, or provide you
|
||||
with facilities for running those works, provided that you comply with
|
||||
the terms of this License in conveying all material for which you do
|
||||
not control copyright. Those thus making or running the covered works
|
||||
for you must do so exclusively on your behalf, under your direction
|
||||
and control, on terms that prohibit them from making any copies of
|
||||
your copyrighted material outside their relationship with you.
|
||||
|
||||
Conveying under any other circumstances is permitted solely under
|
||||
the conditions stated below. Sublicensing is not allowed; section 10
|
||||
makes it unnecessary.
|
||||
|
||||
3. Protecting Users' Legal Rights From Anti-Circumvention Law.
|
||||
|
||||
No covered work shall be deemed part of an effective technological
|
||||
measure under any applicable law fulfilling obligations under article
|
||||
11 of the WIPO copyright treaty adopted on 20 December 1996, or
|
||||
similar laws prohibiting or restricting circumvention of such
|
||||
measures.
|
||||
|
||||
When you convey a covered work, you waive any legal power to forbid
|
||||
circumvention of technological measures to the extent such circumvention
|
||||
is effected by exercising rights under this License with respect to
|
||||
the covered work, and you disclaim any intention to limit operation or
|
||||
modification of the work as a means of enforcing, against the work's
|
||||
users, your or third parties' legal rights to forbid circumvention of
|
||||
technological measures.
|
||||
|
||||
4. Conveying Verbatim Copies.
|
||||
|
||||
You may convey verbatim copies of the Program's source code as you
|
||||
receive it, in any medium, provided that you conspicuously and
|
||||
appropriately publish on each copy an appropriate copyright notice;
|
||||
keep intact all notices stating that this License and any
|
||||
non-permissive terms added in accord with section 7 apply to the code;
|
||||
keep intact all notices of the absence of any warranty; and give all
|
||||
recipients a copy of this License along with the Program.
|
||||
|
||||
You may charge any price or no price for each copy that you convey,
|
||||
and you may offer support or warranty protection for a fee.
|
||||
|
||||
5. Conveying Modified Source Versions.
|
||||
|
||||
You may convey a work based on the Program, or the modifications to
|
||||
produce it from the Program, in the form of source code under the
|
||||
terms of section 4, provided that you also meet all of these conditions:
|
||||
|
||||
a) The work must carry prominent notices stating that you modified
|
||||
it, and giving a relevant date.
|
||||
|
||||
b) The work must carry prominent notices stating that it is
|
||||
released under this License and any conditions added under section
|
||||
7. This requirement modifies the requirement in section 4 to
|
||||
"keep intact all notices".
|
||||
|
||||
c) You must license the entire work, as a whole, under this
|
||||
License to anyone who comes into possession of a copy. This
|
||||
License will therefore apply, along with any applicable section 7
|
||||
additional terms, to the whole of the work, and all its parts,
|
||||
regardless of how they are packaged. This License gives no
|
||||
permission to license the work in any other way, but it does not
|
||||
invalidate such permission if you have separately received it.
|
||||
|
||||
d) If the work has interactive user interfaces, each must display
|
||||
Appropriate Legal Notices; however, if the Program has interactive
|
||||
interfaces that do not display Appropriate Legal Notices, your
|
||||
work need not make them do so.
|
||||
|
||||
A compilation of a covered work with other separate and independent
|
||||
works, which are not by their nature extensions of the covered work,
|
||||
and which are not combined with it such as to form a larger program,
|
||||
in or on a volume of a storage or distribution medium, is called an
|
||||
"aggregate" if the compilation and its resulting copyright are not
|
||||
used to limit the access or legal rights of the compilation's users
|
||||
beyond what the individual works permit. Inclusion of a covered work
|
||||
in an aggregate does not cause this License to apply to the other
|
||||
parts of the aggregate.
|
||||
|
||||
6. Conveying Non-Source Forms.
|
||||
|
||||
You may convey a covered work in object code form under the terms
|
||||
of sections 4 and 5, provided that you also convey the
|
||||
machine-readable Corresponding Source under the terms of this License,
|
||||
in one of these ways:
|
||||
|
||||
a) Convey the object code in, or embodied in, a physical product
|
||||
(including a physical distribution medium), accompanied by the
|
||||
Corresponding Source fixed on a durable physical medium
|
||||
customarily used for software interchange.
|
||||
|
||||
b) Convey the object code in, or embodied in, a physical product
|
||||
(including a physical distribution medium), accompanied by a
|
||||
written offer, valid for at least three years and valid for as
|
||||
long as you offer spare parts or customer support for that product
|
||||
model, to give anyone who possesses the object code either (1) a
|
||||
copy of the Corresponding Source for all the software in the
|
||||
product that is covered by this License, on a durable physical
|
||||
medium customarily used for software interchange, for a price no
|
||||
more than your reasonable cost of physically performing this
|
||||
conveying of source, or (2) access to copy the
|
||||
Corresponding Source from a network server at no charge.
|
||||
|
||||
c) Convey individual copies of the object code with a copy of the
|
||||
written offer to provide the Corresponding Source. This
|
||||
alternative is allowed only occasionally and noncommercially, and
|
||||
only if you received the object code with such an offer, in accord
|
||||
with subsection 6b.
|
||||
|
||||
d) Convey the object code by offering access from a designated
|
||||
place (gratis or for a charge), and offer equivalent access to the
|
||||
Corresponding Source in the same way through the same place at no
|
||||
further charge. You need not require recipients to copy the
|
||||
Corresponding Source along with the object code. If the place to
|
||||
copy the object code is a network server, the Corresponding Source
|
||||
may be on a different server (operated by you or a third party)
|
||||
that supports equivalent copying facilities, provided you maintain
|
||||
clear directions next to the object code saying where to find the
|
||||
Corresponding Source. Regardless of what server hosts the
|
||||
Corresponding Source, you remain obligated to ensure that it is
|
||||
available for as long as needed to satisfy these requirements.
|
||||
|
||||
e) Convey the object code using peer-to-peer transmission, provided
|
||||
you inform other peers where the object code and Corresponding
|
||||
Source of the work are being offered to the general public at no
|
||||
charge under subsection 6d.
|
||||
|
||||
A separable portion of the object code, whose source code is excluded
|
||||
from the Corresponding Source as a System Library, need not be
|
||||
included in conveying the object code work.
|
||||
|
||||
A "User Product" is either (1) a "consumer product", which means any
|
||||
tangible personal property which is normally used for personal, family,
|
||||
or household purposes, or (2) anything designed or sold for incorporation
|
||||
into a dwelling. In determining whether a product is a consumer product,
|
||||
doubtful cases shall be resolved in favor of coverage. For a particular
|
||||
product received by a particular user, "normally used" refers to a
|
||||
typical or common use of that class of product, regardless of the status
|
||||
of the particular user or of the way in which the particular user
|
||||
actually uses, or expects or is expected to use, the product. A product
|
||||
is a consumer product regardless of whether the product has substantial
|
||||
commercial, industrial or non-consumer uses, unless such uses represent
|
||||
the only significant mode of use of the product.
|
||||
|
||||
"Installation Information" for a User Product means any methods,
|
||||
procedures, authorization keys, or other information required to install
|
||||
and execute modified versions of a covered work in that User Product from
|
||||
a modified version of its Corresponding Source. The information must
|
||||
suffice to ensure that the continued functioning of the modified object
|
||||
code is in no case prevented or interfered with solely because
|
||||
modification has been made.
|
||||
|
||||
If you convey an object code work under this section in, or with, or
|
||||
specifically for use in, a User Product, and the conveying occurs as
|
||||
part of a transaction in which the right of possession and use of the
|
||||
User Product is transferred to the recipient in perpetuity or for a
|
||||
fixed term (regardless of how the transaction is characterized), the
|
||||
Corresponding Source conveyed under this section must be accompanied
|
||||
by the Installation Information. But this requirement does not apply
|
||||
if neither you nor any third party retains the ability to install
|
||||
modified object code on the User Product (for example, the work has
|
||||
been installed in ROM).
|
||||
|
||||
The requirement to provide Installation Information does not include a
|
||||
requirement to continue to provide support service, warranty, or updates
|
||||
for a work that has been modified or installed by the recipient, or for
|
||||
the User Product in which it has been modified or installed. Access to a
|
||||
network may be denied when the modification itself materially and
|
||||
adversely affects the operation of the network or violates the rules and
|
||||
protocols for communication across the network.
|
||||
|
||||
Corresponding Source conveyed, and Installation Information provided,
|
||||
in accord with this section must be in a format that is publicly
|
||||
documented (and with an implementation available to the public in
|
||||
source code form), and must require no special password or key for
|
||||
unpacking, reading or copying.
|
||||
|
||||
7. Additional Terms.
|
||||
|
||||
"Additional permissions" are terms that supplement the terms of this
|
||||
License by making exceptions from one or more of its conditions.
|
||||
Additional permissions that are applicable to the entire Program shall
|
||||
be treated as though they were included in this License, to the extent
|
||||
that they are valid under applicable law. If additional permissions
|
||||
apply only to part of the Program, that part may be used separately
|
||||
under those permissions, but the entire Program remains governed by
|
||||
this License without regard to the additional permissions.
|
||||
|
||||
When you convey a copy of a covered work, you may at your option
|
||||
remove any additional permissions from that copy, or from any part of
|
||||
it. (Additional permissions may be written to require their own
|
||||
removal in certain cases when you modify the work.) You may place
|
||||
additional permissions on material, added by you to a covered work,
|
||||
for which you have or can give appropriate copyright permission.
|
||||
|
||||
Notwithstanding any other provision of this License, for material you
|
||||
add to a covered work, you may (if authorized by the copyright holders of
|
||||
that material) supplement the terms of this License with terms:
|
||||
|
||||
a) Disclaiming warranty or limiting liability differently from the
|
||||
terms of sections 15 and 16 of this License; or
|
||||
|
||||
b) Requiring preservation of specified reasonable legal notices or
|
||||
author attributions in that material or in the Appropriate Legal
|
||||
Notices displayed by works containing it; or
|
||||
|
||||
c) Prohibiting misrepresentation of the origin of that material, or
|
||||
requiring that modified versions of such material be marked in
|
||||
reasonable ways as different from the original version; or
|
||||
|
||||
d) Limiting the use for publicity purposes of names of licensors or
|
||||
authors of the material; or
|
||||
|
||||
e) Declining to grant rights under trademark law for use of some
|
||||
trade names, trademarks, or service marks; or
|
||||
|
||||
f) Requiring indemnification of licensors and authors of that
|
||||
material by anyone who conveys the material (or modified versions of
|
||||
it) with contractual assumptions of liability to the recipient, for
|
||||
any liability that these contractual assumptions directly impose on
|
||||
those licensors and authors.
|
||||
|
||||
All other non-permissive additional terms are considered "further
|
||||
restrictions" within the meaning of section 10. If the Program as you
|
||||
received it, or any part of it, contains a notice stating that it is
|
||||
governed by this License along with a term that is a further
|
||||
restriction, you may remove that term. If a license document contains
|
||||
a further restriction but permits relicensing or conveying under this
|
||||
License, you may add to a covered work material governed by the terms
|
||||
of that license document, provided that the further restriction does
|
||||
not survive such relicensing or conveying.
|
||||
|
||||
If you add terms to a covered work in accord with this section, you
|
||||
must place, in the relevant source files, a statement of the
|
||||
additional terms that apply to those files, or a notice indicating
|
||||
where to find the applicable terms.
|
||||
|
||||
Additional terms, permissive or non-permissive, may be stated in the
|
||||
form of a separately written license, or stated as exceptions;
|
||||
the above requirements apply either way.
|
||||
|
||||
8. Termination.
|
||||
|
||||
You may not propagate or modify a covered work except as expressly
|
||||
provided under this License. Any attempt otherwise to propagate or
|
||||
modify it is void, and will automatically terminate your rights under
|
||||
this License (including any patent licenses granted under the third
|
||||
paragraph of section 11).
|
||||
|
||||
However, if you cease all violation of this License, then your
|
||||
license from a particular copyright holder is reinstated (a)
|
||||
provisionally, unless and until the copyright holder explicitly and
|
||||
finally terminates your license, and (b) permanently, if the copyright
|
||||
holder fails to notify you of the violation by some reasonable means
|
||||
prior to 60 days after the cessation.
|
||||
|
||||
Moreover, your license from a particular copyright holder is
|
||||
reinstated permanently if the copyright holder notifies you of the
|
||||
violation by some reasonable means, this is the first time you have
|
||||
received notice of violation of this License (for any work) from that
|
||||
copyright holder, and you cure the violation prior to 30 days after
|
||||
your receipt of the notice.
|
||||
|
||||
Termination of your rights under this section does not terminate the
|
||||
licenses of parties who have received copies or rights from you under
|
||||
this License. If your rights have been terminated and not permanently
|
||||
reinstated, you do not qualify to receive new licenses for the same
|
||||
material under section 10.
|
||||
|
||||
9. Acceptance Not Required for Having Copies.
|
||||
|
||||
You are not required to accept this License in order to receive or
|
||||
run a copy of the Program. Ancillary propagation of a covered work
|
||||
occurring solely as a consequence of using peer-to-peer transmission
|
||||
to receive a copy likewise does not require acceptance. However,
|
||||
nothing other than this License grants you permission to propagate or
|
||||
modify any covered work. These actions infringe copyright if you do
|
||||
not accept this License. Therefore, by modifying or propagating a
|
||||
covered work, you indicate your acceptance of this License to do so.
|
||||
|
||||
10. Automatic Licensing of Downstream Recipients.
|
||||
|
||||
Each time you convey a covered work, the recipient automatically
|
||||
receives a license from the original licensors, to run, modify and
|
||||
propagate that work, subject to this License. You are not responsible
|
||||
for enforcing compliance by third parties with this License.
|
||||
|
||||
An "entity transaction" is a transaction transferring control of an
|
||||
organization, or substantially all assets of one, or subdividing an
|
||||
organization, or merging organizations. If propagation of a covered
|
||||
work results from an entity transaction, each party to that
|
||||
transaction who receives a copy of the work also receives whatever
|
||||
licenses to the work the party's predecessor in interest had or could
|
||||
give under the previous paragraph, plus a right to possession of the
|
||||
Corresponding Source of the work from the predecessor in interest, if
|
||||
the predecessor has it or can get it with reasonable efforts.
|
||||
|
||||
You may not impose any further restrictions on the exercise of the
|
||||
rights granted or affirmed under this License. For example, you may
|
||||
not impose a license fee, royalty, or other charge for exercise of
|
||||
rights granted under this License, and you may not initiate litigation
|
||||
(including a cross-claim or counterclaim in a lawsuit) alleging that
|
||||
any patent claim is infringed by making, using, selling, offering for
|
||||
sale, or importing the Program or any portion of it.
|
||||
|
||||
11. Patents.
|
||||
|
||||
A "contributor" is a copyright holder who authorizes use under this
|
||||
License of the Program or a work on which the Program is based. The
|
||||
work thus licensed is called the contributor's "contributor version".
|
||||
|
||||
A contributor's "essential patent claims" are all patent claims
|
||||
owned or controlled by the contributor, whether already acquired or
|
||||
hereafter acquired, that would be infringed by some manner, permitted
|
||||
by this License, of making, using, or selling its contributor version,
|
||||
but do not include claims that would be infringed only as a
|
||||
consequence of further modification of the contributor version. For
|
||||
purposes of this definition, "control" includes the right to grant
|
||||
patent sublicenses in a manner consistent with the requirements of
|
||||
this License.
|
||||
|
||||
Each contributor grants you a non-exclusive, worldwide, royalty-free
|
||||
patent license under the contributor's essential patent claims, to
|
||||
make, use, sell, offer for sale, import and otherwise run, modify and
|
||||
propagate the contents of its contributor version.
|
||||
|
||||
In the following three paragraphs, a "patent license" is any express
|
||||
agreement or commitment, however denominated, not to enforce a patent
|
||||
(such as an express permission to practice a patent or covenant not to
|
||||
sue for patent infringement). To "grant" such a patent license to a
|
||||
party means to make such an agreement or commitment not to enforce a
|
||||
patent against the party.
|
||||
|
||||
If you convey a covered work, knowingly relying on a patent license,
|
||||
and the Corresponding Source of the work is not available for anyone
|
||||
to copy, free of charge and under the terms of this License, through a
|
||||
publicly available network server or other readily accessible means,
|
||||
then you must either (1) cause the Corresponding Source to be so
|
||||
available, or (2) arrange to deprive yourself of the benefit of the
|
||||
patent license for this particular work, or (3) arrange, in a manner
|
||||
consistent with the requirements of this License, to extend the patent
|
||||
license to downstream recipients. "Knowingly relying" means you have
|
||||
actual knowledge that, but for the patent license, your conveying the
|
||||
covered work in a country, or your recipient's use of the covered work
|
||||
in a country, would infringe one or more identifiable patents in that
|
||||
country that you have reason to believe are valid.
|
||||
|
||||
If, pursuant to or in connection with a single transaction or
|
||||
arrangement, you convey, or propagate by procuring conveyance of, a
|
||||
covered work, and grant a patent license to some of the parties
|
||||
receiving the covered work authorizing them to use, propagate, modify
|
||||
or convey a specific copy of the covered work, then the patent license
|
||||
you grant is automatically extended to all recipients of the covered
|
||||
work and works based on it.
|
||||
|
||||
A patent license is "discriminatory" if it does not include within
|
||||
the scope of its coverage, prohibits the exercise of, or is
|
||||
conditioned on the non-exercise of one or more of the rights that are
|
||||
specifically granted under this License. You may not convey a covered
|
||||
work if you are a party to an arrangement with a third party that is
|
||||
in the business of distributing software, under which you make payment
|
||||
to the third party based on the extent of your activity of conveying
|
||||
the work, and under which the third party grants, to any of the
|
||||
parties who would receive the covered work from you, a discriminatory
|
||||
patent license (a) in connection with copies of the covered work
|
||||
conveyed by you (or copies made from those copies), or (b) primarily
|
||||
for and in connection with specific products or compilations that
|
||||
contain the covered work, unless you entered into that arrangement,
|
||||
or that patent license was granted, prior to 28 March 2007.
|
||||
|
||||
Nothing in this License shall be construed as excluding or limiting
|
||||
any implied license or other defenses to infringement that may
|
||||
otherwise be available to you under applicable patent law.
|
||||
|
||||
12. No Surrender of Others' Freedom.
|
||||
|
||||
If conditions are imposed on you (whether by court order, agreement or
|
||||
otherwise) that contradict the conditions of this License, they do not
|
||||
excuse you from the conditions of this License. If you cannot convey a
|
||||
covered work so as to satisfy simultaneously your obligations under this
|
||||
License and any other pertinent obligations, then as a consequence you may
|
||||
not convey it at all. For example, if you agree to terms that obligate you
|
||||
to collect a royalty for further conveying from those to whom you convey
|
||||
the Program, the only way you could satisfy both those terms and this
|
||||
License would be to refrain entirely from conveying the Program.
|
||||
|
||||
13. Remote Network Interaction; Use with the GNU General Public License.
|
||||
|
||||
Notwithstanding any other provision of this License, if you modify the
|
||||
Program, your modified version must prominently offer all users
|
||||
interacting with it remotely through a computer network (if your version
|
||||
supports such interaction) an opportunity to receive the Corresponding
|
||||
Source of your version by providing access to the Corresponding Source
|
||||
from a network server at no charge, through some standard or customary
|
||||
means of facilitating copying of software. This Corresponding Source
|
||||
shall include the Corresponding Source for any work covered by version 3
|
||||
of the GNU General Public License that is incorporated pursuant to the
|
||||
following paragraph.
|
||||
|
||||
Notwithstanding any other provision of this License, you have
|
||||
permission to link or combine any covered work with a work licensed
|
||||
under version 3 of the GNU General Public License into a single
|
||||
combined work, and to convey the resulting work. The terms of this
|
||||
License will continue to apply to the part which is the covered work,
|
||||
but the work with which it is combined will remain governed by version
|
||||
3 of the GNU General Public License.
|
||||
|
||||
14. Revised Versions of this License.
|
||||
|
||||
The Free Software Foundation may publish revised and/or new versions of
|
||||
the GNU Affero General Public License from time to time. Such new versions
|
||||
will be similar in spirit to the present version, but may differ in detail to
|
||||
address new problems or concerns.
|
||||
|
||||
Each version is given a distinguishing version number. If the
|
||||
Program specifies that a certain numbered version of the GNU Affero General
|
||||
Public License "or any later version" applies to it, you have the
|
||||
option of following the terms and conditions either of that numbered
|
||||
version or of any later version published by the Free Software
|
||||
Foundation. If the Program does not specify a version number of the
|
||||
GNU Affero General Public License, you may choose any version ever published
|
||||
by the Free Software Foundation.
|
||||
|
||||
If the Program specifies that a proxy can decide which future
|
||||
versions of the GNU Affero General Public License can be used, that proxy's
|
||||
public statement of acceptance of a version permanently authorizes you
|
||||
to choose that version for the Program.
|
||||
|
||||
Later license versions may give you additional or different
|
||||
permissions. However, no additional obligations are imposed on any
|
||||
author or copyright holder as a result of your choosing to follow a
|
||||
later version.
|
||||
|
||||
15. Disclaimer of Warranty.
|
||||
|
||||
THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY
|
||||
APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT
|
||||
HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY
|
||||
OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO,
|
||||
THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
|
||||
PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM
|
||||
IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF
|
||||
ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
|
||||
|
||||
16. Limitation of Liability.
|
||||
|
||||
IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
|
||||
WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS
|
||||
THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY
|
||||
GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE
|
||||
USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF
|
||||
DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD
|
||||
PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS),
|
||||
EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF
|
||||
SUCH DAMAGES.
|
||||
|
||||
17. Interpretation of Sections 15 and 16.
|
||||
|
||||
If the disclaimer of warranty and limitation of liability provided
|
||||
above cannot be given local legal effect according to their terms,
|
||||
reviewing courts shall apply local law that most closely approximates
|
||||
an absolute waiver of all civil liability in connection with the
|
||||
Program, unless a warranty or assumption of liability accompanies a
|
||||
copy of the Program in return for a fee.
|
||||
|
||||
END OF TERMS AND CONDITIONS
|
||||
|
||||
How to Apply These Terms to Your New Programs
|
||||
|
||||
If you develop a new program, and you want it to be of the greatest
|
||||
possible use to the public, the best way to achieve this is to make it
|
||||
free software which everyone can redistribute and change under these terms.
|
||||
|
||||
To do so, attach the following notices to the program. It is safest
|
||||
to attach them to the start of each source file to most effectively
|
||||
state the exclusion of warranty; and each file should have at least
|
||||
the "copyright" line and a pointer to where the full notice is found.
|
||||
|
||||
<one line to give the program's name and a brief idea of what it does.>
|
||||
Copyright (C) <year> <name of author>
|
||||
|
||||
This program is free software: you can redistribute it and/or modify
|
||||
it under the terms of the GNU Affero General Public License as published
|
||||
by the Free Software Foundation, either version 3 of the License, or
|
||||
(at your option) any later version.
|
||||
|
||||
This program is distributed in the hope that it will be useful,
|
||||
but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
GNU Affero General Public License for more details.
|
||||
|
||||
You should have received a copy of the GNU Affero General Public License
|
||||
along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
|
||||
Also add information on how to contact you by electronic and paper mail.
|
||||
|
||||
If your software can interact with users remotely through a computer
|
||||
network, you should also make sure that it provides a way for users to
|
||||
get its source. For example, if your program is a web application, its
|
||||
interface could display a "Source" link that leads users to an archive
|
||||
of the code. There are many ways you could offer source, and different
|
||||
solutions will be better for different programs; see section 13 for the
|
||||
specific requirements.
|
||||
|
||||
You should also get your employer (if you work as a programmer) or school,
|
||||
if any, to sign a "copyright disclaimer" for the program, if necessary.
|
||||
For more information on this, and how to apply and follow the GNU AGPL, see
|
||||
<https://www.gnu.org/licenses/>.
|
||||
@@ -1,553 +0,0 @@
|
||||
<!-- mcp-name: io.github.basicmachines-co/basic-memory -->
|
||||
[](https://www.gnu.org/licenses/agpl-3.0)
|
||||
[](https://badge.fury.io/py/basic-memory)
|
||||
[](https://www.python.org/downloads/)
|
||||
[](https://github.com/basicmachines-co/basic-memory/actions)
|
||||
[](https://github.com/astral-sh/ruff)
|
||||

|
||||

|
||||
|
||||
## 🚀 Basic Memory Cloud is Live!
|
||||
|
||||
- **Cross-device and multi-platform support is here.** Your knowledge graph now works on desktop, web, and mobile - seamlessly synced across all your AI tools (Claude, ChatGPT, Gemini, Claude Code, and Codex)
|
||||
- **Early Supporter Pricing:** Early users get 25% off forever.
|
||||
The open source project continues as always. Cloud just makes it work everywhere.
|
||||
|
||||
[Sign up now →](https://basicmemory.com)
|
||||
|
||||
with a 7 day free trial
|
||||
|
||||
# Basic Memory
|
||||
|
||||
Basic Memory lets you build persistent knowledge through natural conversations with Large Language Models (LLMs) like
|
||||
Claude, while keeping everything in simple Markdown files on your computer. It uses the Model Context Protocol (MCP) to
|
||||
enable any compatible LLM to read and write to your local knowledge base.
|
||||
|
||||
- Website: https://basicmemory.com
|
||||
- Documentation: https://docs.basicmemory.com
|
||||
|
||||
## Pick up your conversation right where you left off
|
||||
|
||||
- AI assistants can load context from local files in a new conversation
|
||||
- Notes are saved locally as Markdown files in real time
|
||||
- No project knowledge or special prompting required
|
||||
|
||||
https://github.com/user-attachments/assets/a55d8238-8dd0-454a-be4c-8860dbbd0ddc
|
||||
|
||||
## Quick Start
|
||||
|
||||
```bash
|
||||
# Install with uv (recommended)
|
||||
uv tool install basic-memory
|
||||
|
||||
# Configure Claude Desktop (edit ~/Library/Application Support/Claude/claude_desktop_config.json)
|
||||
# Add this to your config:
|
||||
{
|
||||
"mcpServers": {
|
||||
"basic-memory": {
|
||||
"command": "uvx",
|
||||
"args": [
|
||||
"basic-memory",
|
||||
"mcp"
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
# Now in Claude Desktop, you can:
|
||||
# - Write notes with "Create a note about coffee brewing methods"
|
||||
# - Read notes with "What do I know about pour over coffee?"
|
||||
# - Search with "Find information about Ethiopian beans"
|
||||
|
||||
```
|
||||
|
||||
You can view shared context via files in `~/basic-memory` (default directory location).
|
||||
|
||||
## Why Basic Memory?
|
||||
|
||||
Most LLM interactions are ephemeral - you ask a question, get an answer, and everything is forgotten. Each conversation
|
||||
starts fresh, without the context or knowledge from previous ones. Current workarounds have limitations:
|
||||
|
||||
- Chat histories capture conversations but aren't structured knowledge
|
||||
- RAG systems can query documents but don't let LLMs write back
|
||||
- Vector databases require complex setups and often live in the cloud
|
||||
- Knowledge graphs typically need specialized tools to maintain
|
||||
|
||||
Basic Memory addresses these problems with a simple approach: structured Markdown files that both humans and LLMs can
|
||||
read
|
||||
and write to. The key advantages:
|
||||
|
||||
- **Local-first:** All knowledge stays in files you control
|
||||
- **Bi-directional:** Both you and the LLM read and write to the same files
|
||||
- **Structured yet simple:** Uses familiar Markdown with semantic patterns
|
||||
- **Traversable knowledge graph:** LLMs can follow links between topics
|
||||
- **Standard formats:** Works with existing editors like Obsidian
|
||||
- **Lightweight infrastructure:** Just local files indexed in a local SQLite database
|
||||
|
||||
With Basic Memory, you can:
|
||||
|
||||
- Have conversations that build on previous knowledge
|
||||
- Create structured notes during natural conversations
|
||||
- Have conversations with LLMs that remember what you've discussed before
|
||||
- Navigate your knowledge graph semantically
|
||||
- Keep everything local and under your control
|
||||
- Use familiar tools like Obsidian to view and edit notes
|
||||
- Build a personal knowledge base that grows over time
|
||||
- Sync your knowledge to the cloud with bidirectional synchronization
|
||||
- Authenticate and manage cloud projects with subscription validation
|
||||
- Mount cloud storage for direct file access
|
||||
|
||||
## How It Works in Practice
|
||||
|
||||
Let's say you're exploring coffee brewing methods and want to capture your knowledge. Here's how it works:
|
||||
|
||||
1. Start by chatting normally:
|
||||
|
||||
```
|
||||
I've been experimenting with different coffee brewing methods. Key things I've learned:
|
||||
|
||||
- Pour over gives more clarity in flavor than French press
|
||||
- Water temperature is critical - around 205°F seems best
|
||||
- Freshly ground beans make a huge difference
|
||||
```
|
||||
|
||||
... continue conversation.
|
||||
|
||||
2. Ask the LLM to help structure this knowledge:
|
||||
|
||||
```
|
||||
"Let's write a note about coffee brewing methods."
|
||||
```
|
||||
|
||||
LLM creates a new Markdown file on your system (which you can see instantly in Obsidian or your editor):
|
||||
|
||||
```markdown
|
||||
---
|
||||
title: Coffee Brewing Methods
|
||||
permalink: coffee-brewing-methods
|
||||
tags:
|
||||
- coffee
|
||||
- brewing
|
||||
---
|
||||
|
||||
# Coffee Brewing Methods
|
||||
|
||||
## Observations
|
||||
|
||||
- [method] Pour over provides more clarity and highlights subtle flavors
|
||||
- [technique] Water temperature at 205°F (96°C) extracts optimal compounds
|
||||
- [principle] Freshly ground beans preserve aromatics and flavor
|
||||
|
||||
## Relations
|
||||
|
||||
- relates_to [[Coffee Bean Origins]]
|
||||
- requires [[Proper Grinding Technique]]
|
||||
- affects [[Flavor Extraction]]
|
||||
```
|
||||
|
||||
The note embeds semantic content and links to other topics via simple Markdown formatting.
|
||||
|
||||
3. You see this file on your computer in real time in the current project directory (default `~/$HOME/basic-memory`).
|
||||
|
||||
- Realtime sync can be enabled via running `basic-memory sync --watch`
|
||||
|
||||
4. In a chat with the LLM, you can reference a topic:
|
||||
|
||||
```
|
||||
Look at `coffee-brewing-methods` for context about pour over coffee
|
||||
```
|
||||
|
||||
The LLM can now build rich context from the knowledge graph. For example:
|
||||
|
||||
```
|
||||
Following relation 'relates_to [[Coffee Bean Origins]]':
|
||||
- Found information about Ethiopian Yirgacheffe
|
||||
- Notes on Colombian beans' nutty profile
|
||||
- Altitude effects on bean characteristics
|
||||
|
||||
Following relation 'requires [[Proper Grinding Technique]]':
|
||||
- Burr vs. blade grinder comparisons
|
||||
- Grind size recommendations for different methods
|
||||
- Impact of consistent particle size on extraction
|
||||
```
|
||||
|
||||
Each related document can lead to more context, building a rich semantic understanding of your knowledge base.
|
||||
|
||||
This creates a two-way flow where:
|
||||
|
||||
- Humans write and edit Markdown files
|
||||
- LLMs read and write through the MCP protocol
|
||||
- Sync keeps everything consistent
|
||||
- All knowledge stays in local files.
|
||||
|
||||
## Technical Implementation
|
||||
|
||||
Under the hood, Basic Memory:
|
||||
|
||||
1. Stores everything in Markdown files
|
||||
2. Uses a SQLite database for searching and indexing
|
||||
3. Extracts semantic meaning from simple Markdown patterns
|
||||
- Files become `Entity` objects
|
||||
- Each `Entity` can have `Observations`, or facts associated with it
|
||||
- `Relations` connect entities together to form the knowledge graph
|
||||
4. Maintains the local knowledge graph derived from the files
|
||||
5. Provides bidirectional synchronization between files and the knowledge graph
|
||||
6. Implements the Model Context Protocol (MCP) for AI integration
|
||||
7. Exposes tools that let AI assistants traverse and manipulate the knowledge graph
|
||||
8. Uses memory:// URLs to reference entities across tools and conversations
|
||||
|
||||
The file format is just Markdown with some simple markup:
|
||||
|
||||
Each Markdown file has:
|
||||
|
||||
### Frontmatter
|
||||
|
||||
```markdown
|
||||
title: <Entity title>
|
||||
type: <The type of Entity> (e.g. note)
|
||||
permalink: <a uri slug>
|
||||
|
||||
- <optional metadata> (such as tags)
|
||||
```
|
||||
|
||||
### Observations
|
||||
|
||||
Observations are facts about a topic.
|
||||
They can be added by creating a Markdown list with a special format that can reference a `category`, `tags` using a
|
||||
"#" character, and an optional `context`.
|
||||
|
||||
Observation Markdown format:
|
||||
|
||||
```markdown
|
||||
- [category] content #tag (optional context)
|
||||
```
|
||||
|
||||
Examples of observations:
|
||||
|
||||
```markdown
|
||||
- [method] Pour over extracts more floral notes than French press
|
||||
- [tip] Grind size should be medium-fine for pour over #brewing
|
||||
- [preference] Ethiopian beans have bright, fruity flavors (especially from Yirgacheffe)
|
||||
- [fact] Lighter roasts generally contain more caffeine than dark roasts
|
||||
- [experiment] Tried 1:15 coffee-to-water ratio with good results
|
||||
- [resource] James Hoffman's V60 technique on YouTube is excellent
|
||||
- [question] Does water temperature affect extraction of different compounds differently?
|
||||
- [note] My favorite local shop uses a 30-second bloom time
|
||||
```
|
||||
|
||||
### Relations
|
||||
|
||||
Relations are links to other topics. They define how entities connect in the knowledge graph.
|
||||
|
||||
Markdown format:
|
||||
|
||||
```markdown
|
||||
- relation_type [[WikiLink]] (optional context)
|
||||
```
|
||||
|
||||
Examples of relations:
|
||||
|
||||
```markdown
|
||||
- pairs_well_with [[Chocolate Desserts]]
|
||||
- grown_in [[Ethiopia]]
|
||||
- contrasts_with [[Tea Brewing Methods]]
|
||||
- requires [[Burr Grinder]]
|
||||
- improves_with [[Fresh Beans]]
|
||||
- relates_to [[Morning Routine]]
|
||||
- inspired_by [[Japanese Coffee Culture]]
|
||||
- documented_in [[Coffee Journal]]
|
||||
```
|
||||
|
||||
## Using with VS Code
|
||||
|
||||
Add the following JSON block to your User Settings (JSON) file in VS Code. You can do this by pressing `Ctrl + Shift + P` and typing `Preferences: Open User Settings (JSON)`.
|
||||
|
||||
```json
|
||||
{
|
||||
"mcp": {
|
||||
"servers": {
|
||||
"basic-memory": {
|
||||
"command": "uvx",
|
||||
"args": ["basic-memory", "mcp"]
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Optionally, you can add it to a file called `.vscode/mcp.json` in your workspace. This will allow you to share the configuration with others.
|
||||
|
||||
```json
|
||||
{
|
||||
"servers": {
|
||||
"basic-memory": {
|
||||
"command": "uvx",
|
||||
"args": ["basic-memory", "mcp"]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
You can use Basic Memory with VS Code to easily retrieve and store information while coding.
|
||||
|
||||
## Using with Claude Desktop
|
||||
|
||||
Basic Memory is built using the MCP (Model Context Protocol) and works with the Claude desktop app (https://claude.ai/):
|
||||
|
||||
1. Configure Claude Desktop to use Basic Memory:
|
||||
|
||||
Edit your MCP configuration file (usually located at `~/Library/Application Support/Claude/claude_desktop_config.json`
|
||||
for OS X):
|
||||
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"basic-memory": {
|
||||
"command": "uvx",
|
||||
"args": [
|
||||
"basic-memory",
|
||||
"mcp"
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
If you want to use a specific project (see [Multiple Projects](#multiple-projects) below), update your Claude Desktop
|
||||
config:
|
||||
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"basic-memory": {
|
||||
"command": "uvx",
|
||||
"args": [
|
||||
"basic-memory",
|
||||
"mcp",
|
||||
"--project",
|
||||
"your-project-name"
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
2. Sync your knowledge:
|
||||
|
||||
```bash
|
||||
# One-time sync of local knowledge updates
|
||||
basic-memory sync
|
||||
|
||||
# Run realtime sync process (recommended)
|
||||
basic-memory sync --watch
|
||||
```
|
||||
|
||||
3. Cloud features (optional, requires subscription):
|
||||
|
||||
```bash
|
||||
# Authenticate with cloud
|
||||
basic-memory cloud login
|
||||
|
||||
# Bidirectional sync with cloud
|
||||
basic-memory cloud sync
|
||||
|
||||
# Verify cloud integrity
|
||||
basic-memory cloud check
|
||||
|
||||
# Mount cloud storage
|
||||
basic-memory cloud mount
|
||||
```
|
||||
|
||||
**Routing Flags** (for users with cloud subscriptions):
|
||||
|
||||
When cloud mode is enabled, CLI commands communicate with the cloud API by default. Use routing flags to override this:
|
||||
|
||||
```bash
|
||||
# Force local routing (useful for local MCP server while cloud mode is enabled)
|
||||
basic-memory status --local
|
||||
basic-memory project list --local
|
||||
|
||||
# Force cloud routing (when cloud mode is disabled but you want cloud access)
|
||||
basic-memory status --cloud
|
||||
basic-memory project info my-project --cloud
|
||||
```
|
||||
|
||||
The local MCP server (`basic-memory mcp`) automatically uses local routing, so you can use both local Claude Desktop and cloud-based clients simultaneously.
|
||||
|
||||
4. In Claude Desktop, the LLM can now use these tools:
|
||||
|
||||
**Content Management:**
|
||||
```
|
||||
write_note(title, content, folder, tags) - Create or update notes
|
||||
read_note(identifier, page, page_size) - Read notes by title or permalink
|
||||
read_content(path) - Read raw file content (text, images, binaries)
|
||||
view_note(identifier) - View notes as formatted artifacts
|
||||
edit_note(identifier, operation, content) - Edit notes incrementally
|
||||
move_note(identifier, destination_path) - Move notes with database consistency
|
||||
delete_note(identifier) - Delete notes from knowledge base
|
||||
```
|
||||
|
||||
**Knowledge Graph Navigation:**
|
||||
```
|
||||
build_context(url, depth, timeframe) - Navigate knowledge graph via memory:// URLs
|
||||
recent_activity(type, depth, timeframe) - Find recently updated information
|
||||
list_directory(dir_name, depth) - Browse directory contents with filtering
|
||||
```
|
||||
|
||||
**Search & Discovery:**
|
||||
```
|
||||
search(query, page, page_size) - Search across your knowledge base
|
||||
search_notes(query, page, page_size, search_type, types, entity_types, after_date, metadata_filters, tags, status, project) - Search with filters
|
||||
search_by_metadata(filters, limit, offset, project) - Structured frontmatter search
|
||||
```
|
||||
|
||||
**Project Management:**
|
||||
```
|
||||
list_memory_projects() - List all available projects
|
||||
create_memory_project(project_name, project_path) - Create new projects
|
||||
get_current_project() - Show current project stats
|
||||
sync_status() - Check synchronization status
|
||||
```
|
||||
|
||||
**Visualization:**
|
||||
```
|
||||
canvas(nodes, edges, title, folder) - Generate knowledge visualizations
|
||||
```
|
||||
|
||||
5. Example prompts to try:
|
||||
|
||||
```
|
||||
"Create a note about our project architecture decisions"
|
||||
"Find information about JWT authentication in my notes"
|
||||
"Create a canvas visualization of my project components"
|
||||
"Read my notes on the authentication system"
|
||||
"What have I been working on in the past week?"
|
||||
```
|
||||
|
||||
## Futher info
|
||||
|
||||
See the [Documentation](https://docs.basicmemory.com) for more info, including:
|
||||
|
||||
- [Complete User Guide](https://docs.basicmemory.com/user-guide/)
|
||||
- [CLI tools](https://docs.basicmemory.com/guides/cli-reference/)
|
||||
- [Cloud CLI and Sync](https://docs.basicmemory.com/guides/cloud-cli/)
|
||||
- [Managing multiple Projects](https://docs.basicmemory.com/guides/cli-reference/#project)
|
||||
- [Importing data from OpenAI/Claude Projects](https://docs.basicmemory.com/guides/cli-reference/#import)
|
||||
|
||||
## Logging
|
||||
|
||||
Basic Memory uses [Loguru](https://github.com/Delgan/loguru) for logging. The logging behavior varies by entry point:
|
||||
|
||||
| Entry Point | Default Behavior | Use Case |
|
||||
|-------------|------------------|----------|
|
||||
| CLI commands | File only | Prevents log output from interfering with command output |
|
||||
| MCP server | File only | Stdout would corrupt the JSON-RPC protocol |
|
||||
| API server | File (local) or stdout (cloud) | Docker/cloud deployments use stdout |
|
||||
|
||||
**Log file location:** `~/.basic-memory/basic-memory.log` (10MB rotation, 10 days retention)
|
||||
|
||||
### Environment Variables
|
||||
|
||||
| Variable | Default | Description |
|
||||
|----------|---------|-------------|
|
||||
| `BASIC_MEMORY_LOG_LEVEL` | `INFO` | Log level: DEBUG, INFO, WARNING, ERROR |
|
||||
| `BASIC_MEMORY_CLOUD_MODE` | `false` | When `true`, API logs to stdout with structured context |
|
||||
| `BASIC_MEMORY_FORCE_LOCAL` | `false` | When `true`, forces local API routing (ignores cloud mode) |
|
||||
| `BASIC_MEMORY_ENV` | `dev` | Set to `test` for test mode (stderr only) |
|
||||
|
||||
### Examples
|
||||
|
||||
```bash
|
||||
# Enable debug logging
|
||||
BASIC_MEMORY_LOG_LEVEL=DEBUG basic-memory sync
|
||||
|
||||
# View logs
|
||||
tail -f ~/.basic-memory/basic-memory.log
|
||||
|
||||
# Cloud/Docker mode (stdout logging with structured context)
|
||||
BASIC_MEMORY_CLOUD_MODE=true uvicorn basic_memory.api.app:app
|
||||
```
|
||||
|
||||
## Development
|
||||
|
||||
### Running Tests
|
||||
|
||||
Basic Memory supports dual database backends (SQLite and Postgres). By default, tests run against SQLite. Set `BASIC_MEMORY_TEST_POSTGRES=1` to run against Postgres (uses testcontainers - Docker required).
|
||||
|
||||
**Quick Start:**
|
||||
```bash
|
||||
# Run all tests against SQLite (default, fast)
|
||||
just test-sqlite
|
||||
|
||||
# Run all tests against Postgres (uses testcontainers)
|
||||
just test-postgres
|
||||
|
||||
# Run both SQLite and Postgres tests
|
||||
just test
|
||||
```
|
||||
|
||||
**Available Test Commands:**
|
||||
|
||||
- `just test` - Run all tests against both SQLite and Postgres
|
||||
- `just test-sqlite` - Run all tests against SQLite (fast, no Docker needed)
|
||||
- `just test-postgres` - Run all tests against Postgres (uses testcontainers)
|
||||
- `just test-unit-sqlite` - Run unit tests against SQLite
|
||||
- `just test-unit-postgres` - Run unit tests against Postgres
|
||||
- `just test-int-sqlite` - Run integration tests against SQLite
|
||||
- `just test-int-postgres` - Run integration tests against Postgres
|
||||
- `just test-windows` - Run Windows-specific tests (auto-skips on other platforms)
|
||||
- `just test-benchmark` - Run performance benchmark tests
|
||||
- `just testmon` - Run tests impacted by recent changes (pytest-testmon)
|
||||
- `just test-smoke` - Run fast MCP end-to-end smoke test
|
||||
- `just fast-check` - Run fix/format/typecheck + impacted tests + smoke test
|
||||
- `just doctor` - Run local file <-> DB consistency checks with temp config
|
||||
|
||||
**Postgres Testing:**
|
||||
|
||||
Postgres tests use [testcontainers](https://testcontainers-python.readthedocs.io/) which automatically spins up a Postgres instance in Docker. No manual database setup required - just have Docker running.
|
||||
|
||||
**Testmon Note:** When no files have changed, `just testmon` may collect 0 tests. That's expected and means no impacted tests were detected.
|
||||
|
||||
**Test Markers:**
|
||||
|
||||
Tests use pytest markers for selective execution:
|
||||
- `windows` - Windows-specific database optimizations
|
||||
- `benchmark` - Performance tests (excluded from default runs)
|
||||
- `smoke` - Fast MCP end-to-end smoke tests
|
||||
|
||||
**Other Development Commands:**
|
||||
```bash
|
||||
just install # Install with dev dependencies
|
||||
just lint # Run linting checks
|
||||
just typecheck # Run type checking
|
||||
just format # Format code with ruff
|
||||
just fast-check # Fast local loop (fix/format/typecheck + testmon + smoke)
|
||||
just doctor # Local consistency check (temp config)
|
||||
just check # Run all quality checks
|
||||
just migration "msg" # Create database migration
|
||||
```
|
||||
|
||||
**Local Consistency Check:**
|
||||
```bash
|
||||
basic-memory doctor # Verifies file <-> database sync in a temp project
|
||||
```
|
||||
|
||||
See the [justfile](justfile) for the complete list of development commands.
|
||||
|
||||
## License
|
||||
|
||||
AGPL-3.0
|
||||
|
||||
Contributions are welcome. See the [Contributing](CONTRIBUTING.md) guide for info about setting up the project locally
|
||||
and submitting PRs.
|
||||
|
||||
## Star History
|
||||
|
||||
<a href="https://www.star-history.com/#basicmachines-co/basic-memory&Date">
|
||||
<picture>
|
||||
<source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/svg?repos=basicmachines-co/basic-memory&type=Date&theme=dark" />
|
||||
<source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/svg?repos=basicmachines-co/basic-memory&type=Date" />
|
||||
<img alt="Star History Chart" src="https://api.star-history.com/svg?repos=basicmachines-co/basic-memory&type=Date" />
|
||||
</picture>
|
||||
</a>
|
||||
|
||||
Built with ♥️ by Basic Machines
|
||||
-13
@@ -1,13 +0,0 @@
|
||||
# Security Policy
|
||||
|
||||
## Supported Versions
|
||||
|
||||
| Version | Supported |
|
||||
| ------- | ------------------ |
|
||||
| 0.x.x | :white_check_mark: |
|
||||
|
||||
## Reporting a Vulnerability
|
||||
|
||||
Use this section to tell people how to report a vulnerability.
|
||||
|
||||
If you find a vulnerability, please contact hello@basicmachines.co
|
||||
@@ -1,42 +0,0 @@
|
||||
# Docker Compose configuration for Basic Memory with PostgreSQL
|
||||
# Use this for local development and testing with Postgres backend
|
||||
#
|
||||
# Usage:
|
||||
# docker-compose -f docker-compose-postgres.yml up -d
|
||||
# docker-compose -f docker-compose-postgres.yml down
|
||||
|
||||
services:
|
||||
postgres:
|
||||
image: postgres:17
|
||||
container_name: basic-memory-postgres
|
||||
environment:
|
||||
# Local development/test credentials - NOT for production
|
||||
# These values are referenced by tests and justfile commands
|
||||
POSTGRES_DB: basic_memory
|
||||
POSTGRES_USER: basic_memory_user
|
||||
POSTGRES_PASSWORD: dev_password # Simple password for local testing only
|
||||
ports:
|
||||
- "5433:5432"
|
||||
volumes:
|
||||
- postgres_data:/var/lib/postgresql/data
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "pg_isready -U basic_memory_user -d basic_memory"]
|
||||
interval: 10s
|
||||
timeout: 5s
|
||||
retries: 5
|
||||
restart: unless-stopped
|
||||
|
||||
volumes:
|
||||
# Named volume for Postgres data
|
||||
postgres_data:
|
||||
driver: local
|
||||
|
||||
# Named volume for persistent configuration
|
||||
# Database will be stored in Postgres, not in this volume
|
||||
basic-memory-config:
|
||||
driver: local
|
||||
|
||||
# Network configuration (optional)
|
||||
# networks:
|
||||
# basic-memory-net:
|
||||
# driver: bridge
|
||||
@@ -1,83 +0,0 @@
|
||||
# Docker Compose configuration for Basic Memory
|
||||
# See docs/Docker.md for detailed setup instructions
|
||||
|
||||
version: '3.8'
|
||||
|
||||
services:
|
||||
basic-memory:
|
||||
# Use pre-built image (recommended for most users)
|
||||
image: ghcr.io/basicmachines-co/basic-memory:latest
|
||||
|
||||
# Uncomment to build locally instead:
|
||||
# build: .
|
||||
|
||||
container_name: basic-memory-server
|
||||
|
||||
# Volume mounts for knowledge directories and persistent data
|
||||
volumes:
|
||||
|
||||
# Persistent storage for configuration and database
|
||||
- basic-memory-config:/root/.basic-memory:rw
|
||||
|
||||
# Mount your knowledge directory (required)
|
||||
# Change './knowledge' to your actual Obsidian vault or knowledge directory
|
||||
- ./knowledge:/app/data:rw
|
||||
|
||||
# OPTIONAL: Mount additional knowledge directories for multiple projects
|
||||
# - ./work-notes:/app/data/work:rw
|
||||
# - ./personal-notes:/app/data/personal:rw
|
||||
|
||||
# You can edit the project config manually in the mounted config volume
|
||||
# The default project will be configured to use /app/data
|
||||
environment:
|
||||
# Project configuration
|
||||
- BASIC_MEMORY_DEFAULT_PROJECT=main
|
||||
|
||||
# Enable real-time file synchronization (recommended for Docker)
|
||||
- BASIC_MEMORY_SYNC_CHANGES=true
|
||||
|
||||
# Logging configuration
|
||||
- BASIC_MEMORY_LOG_LEVEL=INFO
|
||||
|
||||
# Sync delay in milliseconds (adjust for performance vs responsiveness)
|
||||
- BASIC_MEMORY_SYNC_DELAY=1000
|
||||
|
||||
# Port exposure for HTTP transport (only needed if not using STDIO)
|
||||
ports:
|
||||
- "8000:8000"
|
||||
|
||||
# Command with SSE transport (configurable via environment variables above)
|
||||
# IMPORTANT: The SSE and streamable-http endpoints are not secured
|
||||
command: ["basic-memory", "mcp", "--transport", "sse", "--host", "0.0.0.0", "--port", "8000"]
|
||||
|
||||
# Container management
|
||||
restart: unless-stopped
|
||||
|
||||
# Health monitoring
|
||||
healthcheck:
|
||||
test: ["CMD", "basic-memory", "--version"]
|
||||
interval: 30s
|
||||
timeout: 10s
|
||||
retries: 3
|
||||
start_period: 30s
|
||||
|
||||
# Optional: Resource limits
|
||||
# deploy:
|
||||
# resources:
|
||||
# limits:
|
||||
# memory: 512M
|
||||
# cpus: '0.5'
|
||||
# reservations:
|
||||
# memory: 256M
|
||||
# cpus: '0.25'
|
||||
|
||||
volumes:
|
||||
# Named volume for persistent configuration and database
|
||||
# This ensures your configuration and knowledge graph persist across container restarts
|
||||
basic-memory-config:
|
||||
driver: local
|
||||
|
||||
# Network configuration (optional)
|
||||
# networks:
|
||||
# basic-memory-net:
|
||||
# driver: bridge
|
||||
@@ -1,425 +0,0 @@
|
||||
# Basic Memory Architecture
|
||||
|
||||
This document describes the architectural patterns and composition structure of Basic Memory.
|
||||
|
||||
## Overview
|
||||
|
||||
Basic Memory is a local-first knowledge management system with three entrypoints:
|
||||
- **API** - FastAPI REST server for HTTP access
|
||||
- **MCP** - Model Context Protocol server for LLM integration
|
||||
- **CLI** - Typer command-line interface
|
||||
|
||||
Each entrypoint uses a **composition root** pattern to manage configuration and dependencies.
|
||||
|
||||
## Composition Roots
|
||||
|
||||
### What is a Composition Root?
|
||||
|
||||
A composition root is the single place in an application where dependencies are wired together. In Basic Memory, each entrypoint has its own composition root that:
|
||||
|
||||
1. Reads configuration from `ConfigManager`
|
||||
2. Resolves runtime mode (cloud/local/test)
|
||||
3. Creates and provides dependencies to downstream code
|
||||
|
||||
**Key principle**: Only composition roots read global configuration. All other modules receive configuration explicitly.
|
||||
|
||||
### Container Structure
|
||||
|
||||
Each entrypoint has a container dataclass in its package:
|
||||
|
||||
```
|
||||
src/basic_memory/
|
||||
├── api/
|
||||
│ └── container.py # ApiContainer
|
||||
├── mcp/
|
||||
│ └── container.py # McpContainer
|
||||
├── cli/
|
||||
│ └── container.py # CliContainer
|
||||
└── runtime.py # RuntimeMode enum and resolver
|
||||
```
|
||||
|
||||
### Container Pattern
|
||||
|
||||
All containers follow the same structure:
|
||||
|
||||
```python
|
||||
@dataclass
|
||||
class Container:
|
||||
config: BasicMemoryConfig
|
||||
mode: RuntimeMode
|
||||
|
||||
@classmethod
|
||||
def create(cls) -> "Container":
|
||||
"""Create container by reading ConfigManager."""
|
||||
config = ConfigManager().config
|
||||
mode = resolve_runtime_mode(
|
||||
cloud_mode_enabled=config.cloud_mode_enabled,
|
||||
is_test_env=config.is_test_env,
|
||||
)
|
||||
return cls(config=config, mode=mode)
|
||||
|
||||
@property
|
||||
def some_computed_property(self) -> bool:
|
||||
"""Derived values based on config and mode."""
|
||||
return self.mode.is_local and self.config.some_setting
|
||||
|
||||
# Module-level singleton
|
||||
_container: Container | None = None
|
||||
|
||||
def get_container() -> Container:
|
||||
if _container is None:
|
||||
raise RuntimeError("Container not initialized")
|
||||
return _container
|
||||
|
||||
def set_container(container: Container) -> None:
|
||||
global _container
|
||||
_container = container
|
||||
```
|
||||
|
||||
### Runtime Mode Resolution
|
||||
|
||||
The `RuntimeMode` enum centralizes mode detection:
|
||||
|
||||
```python
|
||||
class RuntimeMode(Enum):
|
||||
LOCAL = "local"
|
||||
CLOUD = "cloud"
|
||||
TEST = "test"
|
||||
|
||||
@property
|
||||
def is_cloud(self) -> bool:
|
||||
return self == RuntimeMode.CLOUD
|
||||
|
||||
@property
|
||||
def is_local(self) -> bool:
|
||||
return self == RuntimeMode.LOCAL
|
||||
|
||||
@property
|
||||
def is_test(self) -> bool:
|
||||
return self == RuntimeMode.TEST
|
||||
```
|
||||
|
||||
Resolution follows this precedence: **TEST > CLOUD > LOCAL**
|
||||
|
||||
```python
|
||||
def resolve_runtime_mode(cloud_mode_enabled: bool, is_test_env: bool) -> RuntimeMode:
|
||||
if is_test_env:
|
||||
return RuntimeMode.TEST
|
||||
if cloud_mode_enabled:
|
||||
return RuntimeMode.CLOUD
|
||||
return RuntimeMode.LOCAL
|
||||
```
|
||||
|
||||
## Dependencies Package
|
||||
|
||||
### Structure
|
||||
|
||||
The `deps/` package provides FastAPI dependencies organized by feature:
|
||||
|
||||
```
|
||||
src/basic_memory/deps/
|
||||
├── __init__.py # Re-exports for backwards compatibility
|
||||
├── config.py # Configuration access
|
||||
├── db.py # Database/session management
|
||||
├── projects.py # Project resolution
|
||||
├── repositories.py # Data access layer
|
||||
├── services.py # Business logic layer
|
||||
└── importers.py # Import functionality
|
||||
```
|
||||
|
||||
### Usage in Routers
|
||||
|
||||
```python
|
||||
from basic_memory.deps.services import get_entity_service
|
||||
from basic_memory.deps.projects import get_project_config
|
||||
|
||||
@router.get("/entities/{id}")
|
||||
async def get_entity(
|
||||
id: int,
|
||||
entity_service: EntityService = Depends(get_entity_service),
|
||||
project: ProjectConfig = Depends(get_project_config),
|
||||
):
|
||||
return await entity_service.get(id)
|
||||
```
|
||||
|
||||
### Backwards Compatibility
|
||||
|
||||
The old `deps.py` file still exists as a thin re-export shim:
|
||||
|
||||
```python
|
||||
# deps.py - backwards compatibility shim
|
||||
from basic_memory.deps import *
|
||||
```
|
||||
|
||||
New code should import from specific submodules (`basic_memory.deps.services`) for clarity.
|
||||
|
||||
## MCP Tools Architecture
|
||||
|
||||
### Typed API Clients
|
||||
|
||||
MCP tools communicate with the API through typed clients that encapsulate HTTP paths and response validation:
|
||||
|
||||
```
|
||||
src/basic_memory/mcp/clients/
|
||||
├── __init__.py # Re-exports all clients
|
||||
├── base.py # BaseClient with common logic
|
||||
├── knowledge.py # KnowledgeClient - entity CRUD
|
||||
├── search.py # SearchClient - search operations
|
||||
├── memory.py # MemoryClient - context building
|
||||
├── directory.py # DirectoryClient - directory listing
|
||||
├── resource.py # ResourceClient - resource reading
|
||||
└── project.py # ProjectClient - project management
|
||||
```
|
||||
|
||||
### Client Pattern
|
||||
|
||||
Each client encapsulates API paths and validates responses:
|
||||
|
||||
```python
|
||||
class KnowledgeClient(BaseClient):
|
||||
"""Client for knowledge/entity operations."""
|
||||
|
||||
async def resolve_entity(self, identifier: str) -> int:
|
||||
"""Resolve identifier to entity ID."""
|
||||
response = await call_get(
|
||||
self.http_client,
|
||||
f"{self._base_path}/resolve/{identifier}",
|
||||
)
|
||||
return int(response.text)
|
||||
|
||||
async def get_entity(self, entity_id: int) -> EntityResponse:
|
||||
"""Get entity by ID."""
|
||||
response = await call_get(
|
||||
self.http_client,
|
||||
f"{self._base_path}/entities/{entity_id}",
|
||||
)
|
||||
return EntityResponse.model_validate(response.json())
|
||||
```
|
||||
|
||||
### Tool → Client → API Flow
|
||||
|
||||
```
|
||||
MCP Tool (thin adapter)
|
||||
↓
|
||||
Typed Client (encapsulates paths, validates responses)
|
||||
↓
|
||||
HTTP API (FastAPI router)
|
||||
↓
|
||||
Service Layer (business logic)
|
||||
↓
|
||||
Repository Layer (data access)
|
||||
```
|
||||
|
||||
Example tool using typed client:
|
||||
|
||||
```python
|
||||
@mcp.tool()
|
||||
async def search_notes(
|
||||
query: str,
|
||||
project: str | None = None,
|
||||
metadata_filters: dict | None = None,
|
||||
tags: list[str] | None = None,
|
||||
status: str | None = None,
|
||||
) -> SearchResponse:
|
||||
async with get_client() as client:
|
||||
active_project = await get_active_project(client, project)
|
||||
|
||||
# Import client inside function to avoid circular imports
|
||||
from basic_memory.mcp.clients import SearchClient
|
||||
from basic_memory.schemas.search import SearchQuery
|
||||
|
||||
search_query = SearchQuery(
|
||||
text=query,
|
||||
metadata_filters=metadata_filters,
|
||||
tags=tags,
|
||||
status=status,
|
||||
)
|
||||
search_client = SearchClient(client, active_project.external_id)
|
||||
return await search_client.search(search_query.model_dump())
|
||||
```
|
||||
|
||||
## Sync Coordination
|
||||
|
||||
### SyncCoordinator
|
||||
|
||||
The `SyncCoordinator` centralizes sync/watch lifecycle management:
|
||||
|
||||
```python
|
||||
@dataclass
|
||||
class SyncCoordinator:
|
||||
"""Coordinates file sync and watch operations."""
|
||||
|
||||
status: SyncStatus = SyncStatus.NOT_STARTED
|
||||
sync_task: asyncio.Task | None = None
|
||||
watch_service: WatchService | None = None
|
||||
|
||||
async def start(self, ...):
|
||||
"""Start sync and watch operations."""
|
||||
|
||||
async def stop(self):
|
||||
"""Stop all sync operations gracefully."""
|
||||
|
||||
def get_status_info(self) -> dict:
|
||||
"""Get current sync status for observability."""
|
||||
```
|
||||
|
||||
### Status Enum
|
||||
|
||||
```python
|
||||
class SyncStatus(Enum):
|
||||
NOT_STARTED = "not_started"
|
||||
STARTING = "starting"
|
||||
RUNNING = "running"
|
||||
STOPPING = "stopping"
|
||||
STOPPED = "stopped"
|
||||
ERROR = "error"
|
||||
```
|
||||
|
||||
## Project Resolution
|
||||
|
||||
### ProjectResolver
|
||||
|
||||
Unified project selection across all entrypoints:
|
||||
|
||||
```python
|
||||
class ProjectResolver:
|
||||
"""Resolves which project to use based on context."""
|
||||
|
||||
def resolve(
|
||||
self,
|
||||
explicit_project: str | None = None,
|
||||
) -> ResolvedProject:
|
||||
"""Resolve project using three-tier hierarchy:
|
||||
1. Explicit project parameter
|
||||
2. Default project from config
|
||||
3. Single available project
|
||||
"""
|
||||
```
|
||||
|
||||
### Resolution Modes
|
||||
|
||||
```python
|
||||
class ResolutionMode(Enum):
|
||||
EXPLICIT = "explicit" # User specified project
|
||||
DEFAULT = "default" # Using configured default
|
||||
SINGLE_PROJECT = "single" # Only one project exists
|
||||
FALLBACK = "fallback" # Using first available
|
||||
```
|
||||
|
||||
## Testing Patterns
|
||||
|
||||
### Container Testing
|
||||
|
||||
Each container has corresponding tests:
|
||||
|
||||
```
|
||||
tests/
|
||||
├── api/test_api_container.py
|
||||
├── mcp/test_mcp_container.py
|
||||
└── cli/test_cli_container.py
|
||||
```
|
||||
|
||||
Tests verify:
|
||||
- Container creation from config
|
||||
- Runtime mode properties
|
||||
- Container accessor functions (get/set)
|
||||
|
||||
### Mocking Typed Clients
|
||||
|
||||
When testing MCP tools, mock at the client level:
|
||||
|
||||
```python
|
||||
def test_search_notes(monkeypatch):
|
||||
import basic_memory.mcp.clients as clients_mod
|
||||
|
||||
class MockSearchClient:
|
||||
async def search(self, query):
|
||||
return SearchResponse(results=[...])
|
||||
|
||||
monkeypatch.setattr(clients_mod, "SearchClient", MockSearchClient)
|
||||
```
|
||||
|
||||
## Design Principles
|
||||
|
||||
### 1. Explicit Dependencies
|
||||
|
||||
Modules receive configuration explicitly rather than reading globals:
|
||||
|
||||
```python
|
||||
# Good - explicit injection
|
||||
async def sync_files(config: BasicMemoryConfig):
|
||||
...
|
||||
|
||||
# Avoid - hidden global access
|
||||
async def sync_files():
|
||||
config = ConfigManager().config # Hidden coupling
|
||||
```
|
||||
|
||||
### 2. Single Responsibility
|
||||
|
||||
Each layer has a clear responsibility:
|
||||
- **Containers**: Wire dependencies
|
||||
- **Clients**: Encapsulate HTTP communication
|
||||
- **Services**: Business logic
|
||||
- **Repositories**: Data access
|
||||
- **Tools/Routers**: Thin adapters
|
||||
|
||||
### 3. Deferred Imports
|
||||
|
||||
To avoid circular imports, typed clients are imported inside functions:
|
||||
|
||||
```python
|
||||
async def my_tool():
|
||||
async with get_client() as client:
|
||||
# Import here to avoid circular dependency
|
||||
from basic_memory.mcp.clients import KnowledgeClient
|
||||
|
||||
knowledge_client = KnowledgeClient(client, project_id)
|
||||
```
|
||||
|
||||
### 4. Backwards Compatibility
|
||||
|
||||
When refactoring, maintain backwards compatibility via shims:
|
||||
|
||||
```python
|
||||
# Old module becomes a shim
|
||||
from basic_memory.new_location import *
|
||||
|
||||
# Docstring explains migration path
|
||||
"""
|
||||
DEPRECATED: Import from basic_memory.new_location instead.
|
||||
This shim will be removed in a future version.
|
||||
"""
|
||||
```
|
||||
|
||||
## File Organization
|
||||
|
||||
```
|
||||
src/basic_memory/
|
||||
├── api/
|
||||
│ ├── container.py # API composition root
|
||||
│ ├── routers/ # FastAPI routers
|
||||
│ └── ...
|
||||
├── mcp/
|
||||
│ ├── container.py # MCP composition root
|
||||
│ ├── clients/ # Typed API clients
|
||||
│ ├── tools/ # MCP tool definitions
|
||||
│ └── server.py # MCP server setup
|
||||
├── cli/
|
||||
│ ├── container.py # CLI composition root
|
||||
│ ├── app.py # Typer app
|
||||
│ └── commands/ # CLI command groups
|
||||
├── deps/
|
||||
│ ├── config.py # Config dependencies
|
||||
│ ├── db.py # Database dependencies
|
||||
│ ├── projects.py # Project dependencies
|
||||
│ ├── repositories.py # Repository dependencies
|
||||
│ ├── services.py # Service dependencies
|
||||
│ └── importers.py # Importer dependencies
|
||||
├── sync/
|
||||
│ ├── coordinator.py # SyncCoordinator
|
||||
│ └── ...
|
||||
├── runtime.py # RuntimeMode resolution
|
||||
├── project_resolver.py # Unified project selection
|
||||
└── config.py # Configuration management
|
||||
```
|
||||
-365
@@ -1,365 +0,0 @@
|
||||
# Docker Setup Guide
|
||||
|
||||
Basic Memory can be run in Docker containers to provide a consistent, isolated environment for your knowledge management
|
||||
system. This is particularly useful for integrating with existing Dockerized MCP servers or for deployment scenarios.
|
||||
|
||||
## Quick Start
|
||||
|
||||
### Option 1: Using Pre-built Images (Recommended)
|
||||
|
||||
Basic Memory provides pre-built Docker images on GitHub Container Registry that are automatically updated with each release.
|
||||
|
||||
1. **Use the official image directly:**
|
||||
```bash
|
||||
docker run -d \
|
||||
--name basic-memory-server \
|
||||
-p 8000:8000 \
|
||||
-v /path/to/your/obsidian-vault:/app/data:rw \
|
||||
-v basic-memory-config:/app/.basic-memory:rw \
|
||||
ghcr.io/basicmachines-co/basic-memory:latest
|
||||
```
|
||||
|
||||
2. **Or use Docker Compose with the pre-built image:**
|
||||
```yaml
|
||||
version: '3.8'
|
||||
services:
|
||||
basic-memory:
|
||||
image: ghcr.io/basicmachines-co/basic-memory:latest
|
||||
container_name: basic-memory-server
|
||||
ports:
|
||||
- "8000:8000"
|
||||
volumes:
|
||||
- /path/to/your/obsidian-vault:/app/data:rw
|
||||
- basic-memory-config:/app/.basic-memory:rw
|
||||
environment:
|
||||
- BASIC_MEMORY_DEFAULT_PROJECT=main
|
||||
restart: unless-stopped
|
||||
```
|
||||
|
||||
### Option 2: Using Docker Compose (Building Locally)
|
||||
|
||||
1. **Clone the repository:**
|
||||
```bash
|
||||
git clone https://github.com/basicmachines-co/basic-memory.git
|
||||
cd basic-memory
|
||||
```
|
||||
|
||||
2. **Update the docker-compose.yml:**
|
||||
Edit the volume mount to point to your Obsidian vault:
|
||||
```yaml
|
||||
volumes:
|
||||
# Change './obsidian-vault' to your actual directory path
|
||||
- /path/to/your/obsidian-vault:/app/data:rw
|
||||
```
|
||||
|
||||
3. **Start the container:**
|
||||
```bash
|
||||
docker-compose up -d
|
||||
```
|
||||
|
||||
### Option 3: Using Docker CLI
|
||||
|
||||
```bash
|
||||
# Build the image
|
||||
docker build -t basic-memory .
|
||||
|
||||
# Run with volume mounting
|
||||
docker run -d \
|
||||
--name basic-memory-server \
|
||||
-v /path/to/your/obsidian-vault:/app/data:rw \
|
||||
-v basic-memory-config:/app/.basic-memory:rw \
|
||||
-e BASIC_MEMORY_DEFAULT_PROJECT=main \
|
||||
basic-memory
|
||||
```
|
||||
|
||||
## Configuration
|
||||
|
||||
### Volume Mounts
|
||||
|
||||
Basic Memory requires several volume mounts for proper operation:
|
||||
|
||||
1. **Knowledge Directory** (Required):
|
||||
```yaml
|
||||
- /path/to/your/obsidian-vault:/app/data:rw
|
||||
```
|
||||
Mount your Obsidian vault or knowledge base directory.
|
||||
|
||||
2. **Configuration and Database** (Recommended):
|
||||
```yaml
|
||||
- basic-memory-config:/app/.basic-memory:rw
|
||||
```
|
||||
Persistent storage for configuration and SQLite database.
|
||||
|
||||
You can edit the basic-memory config.json file located in the /app/.basic-memory/config.json after Basic Memory starts.
|
||||
|
||||
3. **Multiple Projects** (Optional):
|
||||
```yaml
|
||||
- /path/to/project1:/app/data/project1:rw
|
||||
- /path/to/project2:/app/data/project2:rw
|
||||
```
|
||||
|
||||
You can edit the basic-memory config.json file located in the /app/.basic-memory/config.json
|
||||
|
||||
## CLI Commands via Docker
|
||||
|
||||
You can run Basic Memory CLI commands inside the container using `docker exec`:
|
||||
|
||||
### Basic Commands
|
||||
|
||||
```bash
|
||||
# Check status
|
||||
docker exec basic-memory-server basic-memory status
|
||||
|
||||
# Sync files
|
||||
docker exec basic-memory-server basic-memory sync
|
||||
|
||||
# Show help
|
||||
docker exec basic-memory-server basic-memory --help
|
||||
```
|
||||
|
||||
### Managing Projects with Volume Mounts
|
||||
|
||||
When using Docker volumes, you'll need to configure projects to point to your mounted directories:
|
||||
|
||||
1. **Check current configuration:**
|
||||
```bash
|
||||
docker exec basic-memory-server cat /app/.basic-memory/config.json
|
||||
```
|
||||
|
||||
2. **Add a project for your mounted volume:**
|
||||
```bash
|
||||
# If you mounted /path/to/your/vault to /app/data
|
||||
docker exec basic-memory-server basic-memory project create my-vault /app/data
|
||||
|
||||
# Set it as default
|
||||
docker exec basic-memory-server basic-memory project set-default my-vault
|
||||
```
|
||||
|
||||
3. **Sync the new project:**
|
||||
```bash
|
||||
docker exec basic-memory-server basic-memory sync
|
||||
```
|
||||
|
||||
### Example: Setting up an Obsidian Vault
|
||||
|
||||
If you mounted your Obsidian vault like this in docker-compose.yml:
|
||||
```yaml
|
||||
volumes:
|
||||
- /Users/yourname/Documents/ObsidianVault:/app/data:rw
|
||||
```
|
||||
|
||||
Then configure it:
|
||||
```bash
|
||||
# Create project pointing to mounted vault
|
||||
docker exec basic-memory-server basic-memory project create obsidian /app/data
|
||||
|
||||
# Set as default
|
||||
docker exec basic-memory-server basic-memory project set-default obsidian
|
||||
|
||||
# Sync to index all files
|
||||
docker exec basic-memory-server basic-memory sync
|
||||
```
|
||||
|
||||
### Environment Variables
|
||||
|
||||
Configure Basic Memory using environment variables:
|
||||
|
||||
```yaml
|
||||
environment:
|
||||
|
||||
# Default project
|
||||
- BASIC_MEMORY_DEFAULT_PROJECT=main
|
||||
|
||||
# Enable real-time sync
|
||||
- BASIC_MEMORY_SYNC_CHANGES=true
|
||||
|
||||
# Logging level
|
||||
- BASIC_MEMORY_LOG_LEVEL=INFO
|
||||
|
||||
# Sync delay in milliseconds
|
||||
- BASIC_MEMORY_SYNC_DELAY=1000
|
||||
```
|
||||
|
||||
## File Permissions
|
||||
|
||||
### Linux/macOS
|
||||
|
||||
The Docker container now runs as a non-root user to avoid file ownership issues. By default, the container uses UID/GID 1000, but you can customize this to match your user:
|
||||
|
||||
```bash
|
||||
# Build with custom UID/GID to match your user
|
||||
docker build --build-arg UID=$(id -u) --build-arg GID=$(id -g) -t basic-memory .
|
||||
|
||||
# Or use docker-compose with build args
|
||||
```
|
||||
|
||||
**Example docker-compose.yml with custom user:**
|
||||
```yaml
|
||||
version: '3.8'
|
||||
services:
|
||||
basic-memory:
|
||||
build:
|
||||
context: .
|
||||
dockerfile: Dockerfile
|
||||
args:
|
||||
UID: 1000 # Replace with your UID
|
||||
GID: 1000 # Replace with your GID
|
||||
container_name: basic-memory-server
|
||||
ports:
|
||||
- "8000:8000"
|
||||
volumes:
|
||||
- /path/to/your/obsidian-vault:/app/data:rw
|
||||
- basic-memory-config:/app/.basic-memory:rw
|
||||
environment:
|
||||
- BASIC_MEMORY_DEFAULT_PROJECT=main
|
||||
restart: unless-stopped
|
||||
```
|
||||
|
||||
**Using pre-built images:**
|
||||
If using the pre-built image from GitHub Container Registry, files will be created with UID/GID 1000. You can either:
|
||||
|
||||
1. Change your local directory ownership to match:
|
||||
```bash
|
||||
sudo chown -R 1000:1000 /path/to/your/obsidian-vault
|
||||
```
|
||||
|
||||
2. Or build your own image with custom UID/GID as shown above.
|
||||
|
||||
### Windows
|
||||
|
||||
When using Docker Desktop on Windows, ensure the directories are shared:
|
||||
|
||||
1. Open Docker Desktop
|
||||
2. Go to Settings → Resources → File Sharing
|
||||
3. Add your knowledge directory path
|
||||
4. Apply & Restart
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Common Issues
|
||||
|
||||
1. **File Watching Not Working:**
|
||||
- Ensure volume mounts are read-write (`:rw`)
|
||||
- Check directory permissions
|
||||
- On Linux, may need to increase inotify limits:
|
||||
```bash
|
||||
echo fs.inotify.max_user_watches=524288 | sudo tee -a /etc/sysctl.conf
|
||||
sudo sysctl -p
|
||||
```
|
||||
|
||||
2. **Configuration Not Persisting:**
|
||||
- Use named volumes for `/app/.basic-memory`
|
||||
- Check volume mount permissions
|
||||
|
||||
3. **Network Connectivity:**
|
||||
- For HTTP transport, ensure port 8000 is exposed
|
||||
- Check firewall settings
|
||||
|
||||
### Debug Mode
|
||||
|
||||
Run with debug logging:
|
||||
|
||||
```yaml
|
||||
environment:
|
||||
- BASIC_MEMORY_LOG_LEVEL=DEBUG
|
||||
```
|
||||
|
||||
View logs:
|
||||
|
||||
```bash
|
||||
docker-compose logs -f basic-memory
|
||||
```
|
||||
|
||||
|
||||
## Security Considerations
|
||||
|
||||
1. **Docker Security:**
|
||||
The container runs as a non-root user (UID/GID 1000 by default) for improved security. You can customize the user ID using build arguments to match your local user.
|
||||
|
||||
2. **Volume Permissions:**
|
||||
Ensure mounted directories have appropriate permissions and don't expose sensitive data. With the non-root container, files will be created with the specified user ownership.
|
||||
|
||||
3. **Network Security:**
|
||||
If using HTTP transport, consider using reverse proxy with SSL/TLS and authentication if the endpoint is available on
|
||||
a network.
|
||||
|
||||
4. **IMPORTANT:** The HTTP endpoints have no authorization. They should not be exposed on a public network.
|
||||
|
||||
## Integration Examples
|
||||
|
||||
### Claude Desktop with Docker
|
||||
|
||||
The recommended way to connect Claude Desktop to the containerized Basic Memory is using `mcp-proxy`, which converts the HTTP transport to STDIO that Claude Desktop expects:
|
||||
|
||||
1. **Start the Docker container:**
|
||||
```bash
|
||||
docker-compose up -d
|
||||
```
|
||||
|
||||
2. **Configure Claude Desktop** to use mcp-proxy:
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"basic-memory": {
|
||||
"command": "uvx",
|
||||
"args": [
|
||||
"mcp-proxy",
|
||||
"http://localhost:8000/mcp"
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
|
||||
## Support
|
||||
|
||||
For Docker-specific issues:
|
||||
|
||||
1. Check the [troubleshooting section](#troubleshooting) above
|
||||
2. Review container logs: `docker-compose logs basic-memory`
|
||||
3. Verify volume mounts: `docker inspect basic-memory-server`
|
||||
4. Test file permissions: `docker exec basic-memory-server ls -la /app`
|
||||
|
||||
For general Basic Memory support, see the main [README](../README.md)
|
||||
and [documentation](https://memory.basicmachines.co/).
|
||||
|
||||
## GitHub Container Registry Images
|
||||
|
||||
### Available Images
|
||||
|
||||
Pre-built Docker images are available on GitHub Container Registry at [`ghcr.io/basicmachines-co/basic-memory`](https://github.com/basicmachines-co/basic-memory/pkgs/container/basic-memory).
|
||||
|
||||
**Supported architectures:**
|
||||
- `linux/amd64` (Intel/AMD x64)
|
||||
- `linux/arm64` (ARM64, including Apple Silicon)
|
||||
|
||||
**Available tags:**
|
||||
- `latest` - Latest stable release
|
||||
- `v0.13.8`, `v0.13.7`, etc. - Specific version tags
|
||||
- `v0.13`, `v0.12`, etc. - Major.minor tags
|
||||
|
||||
### Automated Builds
|
||||
|
||||
Docker images are automatically built and published when new releases are tagged:
|
||||
|
||||
1. **Release Process:** When a git tag matching `v*` (e.g., `v0.13.8`) is pushed, the CI workflow automatically:
|
||||
- Builds multi-platform Docker images
|
||||
- Pushes to GitHub Container Registry with appropriate tags
|
||||
- Uses native GitHub integration for seamless publishing
|
||||
|
||||
2. **CI/CD Pipeline:** The Docker workflow includes:
|
||||
- Multi-platform builds (AMD64 and ARM64)
|
||||
- Layer caching for faster builds
|
||||
- Automatic tagging with semantic versioning
|
||||
- Security scanning and optimization
|
||||
|
||||
### Setup Requirements (For Maintainers)
|
||||
|
||||
GitHub Container Registry integration is automatic for this repository:
|
||||
|
||||
1. **No external setup required** - GHCR is natively integrated with GitHub
|
||||
2. **Automatic permissions** - Uses `GITHUB_TOKEN` with `packages: write` permission
|
||||
3. **Public by default** - Images are automatically public for public repositories
|
||||
|
||||
The Docker CI workflow (`.github/workflows/docker.yml`) handles everything automatically when version tags are pushed.
|
||||
@@ -1,494 +0,0 @@
|
||||
# Note Format Reference
|
||||
|
||||
Every document in Basic Memory is a plain Markdown file. Files are the source of truth — changes to files automatically update the knowledge graph in the database. You maintain complete ownership, files work with git, and knowledge persists independently of any AI conversation.
|
||||
|
||||
## Document Structure
|
||||
|
||||
A note has three parts: YAML frontmatter, content (observations), and relations.
|
||||
|
||||
```markdown
|
||||
---
|
||||
title: Coffee Brewing Methods
|
||||
type: note
|
||||
tags: [coffee, brewing]
|
||||
permalink: coffee-brewing-methods
|
||||
---
|
||||
|
||||
# Coffee Brewing Methods
|
||||
|
||||
## Observations
|
||||
- [method] Pour over provides more flavor clarity than French press
|
||||
- [technique] Water temperature at 205°F extracts optimal compounds #brewing
|
||||
- [preference] Ethiopian beans work well with lighter roasts (personal experience)
|
||||
|
||||
## Relations
|
||||
- relates_to [[Coffee Bean Origins]]
|
||||
- requires [[Proper Grinding Technique]]
|
||||
- contrasts_with [[Tea Brewing Methods]]
|
||||
```
|
||||
|
||||
The `## Observations` and `## Relations` headings are conventional but not required — the parser detects observations and relations by their syntax patterns anywhere in the document.
|
||||
|
||||
## Frontmatter
|
||||
|
||||
YAML metadata between `---` fences at the top of the file.
|
||||
|
||||
| Field | Required | Default | Description |
|
||||
|-------|----------|---------|-------------|
|
||||
| `title` | No | filename stem | Used for linking and references. Auto-set from filename if missing. |
|
||||
| `type` | No | `note` | Entity type. Used for schema resolution and filtering. |
|
||||
| `tags` | No | `[]` | List or comma-separated string. Used for organization and search. |
|
||||
| `permalink` | No | generated from title | Stable identifier. Persists even if the file moves. |
|
||||
| `schema` | No | none | Schema attachment — dict (inline), string (reference), or omitted (implicit). |
|
||||
|
||||
Custom fields are allowed. Any key not in the standard set is stored as `entity_metadata` and indexed for search and filtering.
|
||||
|
||||
```yaml
|
||||
---
|
||||
title: Paul Graham
|
||||
type: Person
|
||||
tags: [startups, essays, lisp]
|
||||
permalink: paul-graham
|
||||
status: active
|
||||
source: wikipedia
|
||||
---
|
||||
```
|
||||
|
||||
Here `status` and `source` are custom fields stored in `entity_metadata`.
|
||||
|
||||
### Frontmatter Value Handling
|
||||
|
||||
YAML automatically converts some values to native types. Basic Memory normalizes them:
|
||||
|
||||
- Date strings (`2025-10-24`) → kept as ISO format strings
|
||||
- Numbers (`1.0`) → converted to strings
|
||||
- Booleans (`true`) → converted to strings (`"True"`)
|
||||
- Lists and dicts → preserved, items normalized recursively
|
||||
|
||||
This prevents errors when downstream code expects string values.
|
||||
|
||||
## Observations
|
||||
|
||||
An observation is a categorized fact about the entity. Written as a Markdown list item.
|
||||
|
||||
**Syntax:**
|
||||
|
||||
```
|
||||
- [category] content text #tag1 #tag2 (context)
|
||||
```
|
||||
|
||||
| Part | Required | Description |
|
||||
|------|----------|-------------|
|
||||
| `[category]` | Yes | Classification in square brackets. Any text except `[]()` chars. |
|
||||
| content | Yes | The fact or statement. |
|
||||
| `#tags` | No | Inline tags. Space-separated, each starting with `#`. |
|
||||
| `(context)` | No | Parenthesized text at end of line. Supporting details or source. |
|
||||
|
||||
### Examples
|
||||
|
||||
```markdown
|
||||
- [tech] Uses SQLite for storage #database
|
||||
- [design] Follows local-first architecture #architecture
|
||||
- [decision] Selected bcrypt for passwords #security (based on OWASP audit)
|
||||
- [name] Paul Graham
|
||||
- [expertise] Startups
|
||||
- [expertise] Lisp
|
||||
- [expertise] Essay writing
|
||||
```
|
||||
|
||||
Array-like fields use repeated categories — multiple `[expertise]` observations above.
|
||||
|
||||
### What Is Not an Observation
|
||||
|
||||
The parser excludes these list item patterns:
|
||||
|
||||
| Pattern | Example | Reason |
|
||||
|---------|---------|--------|
|
||||
| Checkboxes | `- [ ] Todo item`, `- [x] Done`, `- [-] Cancelled` | Task list syntax |
|
||||
| Markdown links | `- [text](url)` | URL link syntax |
|
||||
| Bare wiki links | `- [[Target]]` | Treated as a relation instead |
|
||||
|
||||
A list item with `#tags` but no `[category]` is still parsed — the tags are extracted and the category defaults to `Note`.
|
||||
|
||||
## Relations
|
||||
|
||||
Relations connect documents to form the knowledge graph. There are two kinds.
|
||||
|
||||
### Explicit Relations
|
||||
|
||||
Written as list items with a relation type and a `[[wiki link]]` target.
|
||||
|
||||
**Syntax:**
|
||||
|
||||
```
|
||||
- relation_type [[Target Entity]] (context)
|
||||
```
|
||||
|
||||
| Part | Required | Description |
|
||||
|------|----------|-------------|
|
||||
| `relation_type` | No | Text before `[[`. Defaults to `relates_to` if omitted. |
|
||||
| `[[Target]]` | Yes | Wiki link to the target entity. Matched by title or permalink. |
|
||||
| `(context)` | No | Parenthesized text after `]]`. Supporting details. |
|
||||
|
||||
### Examples
|
||||
|
||||
```markdown
|
||||
- implements [[Search Design]]
|
||||
- depends_on [[Database Schema]]
|
||||
- works_at [[Y Combinator]] (co-founder)
|
||||
- [[Some Entity]]
|
||||
```
|
||||
|
||||
The last example — a bare `[[wiki link]]` in a list item — gets relation type `relates_to`.
|
||||
|
||||
Common relation types:
|
||||
- `implements`, `depends_on`, `relates_to`, `inspired_by`
|
||||
- `extends`, `part_of`, `contains`, `pairs_with`
|
||||
- `works_at`, `authored`, `collaborated_with`
|
||||
|
||||
Any text works as a relation type. These are conventions, not a fixed set.
|
||||
|
||||
### Inline References
|
||||
|
||||
Wiki links appearing in regular prose (not as list items) create implicit `links_to` relations.
|
||||
|
||||
```markdown
|
||||
This builds on [[Core Design]] and uses [[Utility Functions]].
|
||||
```
|
||||
|
||||
This creates two relations: `links_to [[Core Design]]` and `links_to [[Utility Functions]]`.
|
||||
|
||||
### Forward References
|
||||
|
||||
Relations can link to entities that don't exist yet. Basic Memory resolves them when the target is created.
|
||||
|
||||
## Permalinks and memory:// URLs
|
||||
|
||||
Every document has a unique **permalink** — a stable identifier derived from its title. You can set one explicitly in frontmatter, or let the system generate it.
|
||||
|
||||
```yaml
|
||||
permalink: auth-approaches-2024
|
||||
```
|
||||
|
||||
Permalinks form the basis of `memory://` URLs:
|
||||
|
||||
```
|
||||
memory://auth-approaches-2024 # By permalink
|
||||
memory://Authentication Approaches # By title (auto-resolves)
|
||||
memory://project/auth-approaches # By path
|
||||
```
|
||||
|
||||
Pattern matching is supported:
|
||||
|
||||
```
|
||||
memory://auth* # Starts with "auth"
|
||||
memory://*/approaches # Ends with "approaches"
|
||||
memory://project/*/requirements # Nested wildcard
|
||||
```
|
||||
|
||||
## Schemas
|
||||
|
||||
Schemas declare the expected structure of a note — which observation categories and relation types a well-formed note should have. They use Picoschema, a compact notation from Google's Dotprompt that fits naturally in YAML frontmatter.
|
||||
|
||||
### Picoschema Syntax
|
||||
|
||||
```yaml
|
||||
schema:
|
||||
name: string, full name # required field with description
|
||||
email?: string, contact email # ? = optional
|
||||
role?: string, job title
|
||||
works_at?: Organization, employer # capitalized type = entity reference
|
||||
tags?(array): string, categories # array of type
|
||||
status?(enum): [active, inactive] # enum with allowed values
|
||||
metadata?(object): # nested object
|
||||
updated_at?: string
|
||||
source?: string
|
||||
```
|
||||
|
||||
| Notation | Meaning | Example |
|
||||
|----------|---------|---------|
|
||||
| `field: type` | Required field | `name: string` |
|
||||
| `field?: type` | Optional field | `role?: string` |
|
||||
| `field(array): type` | Array of values | `expertise(array): string` |
|
||||
| `field?(enum): [vals]` | Enum with allowed values | `status?(enum): [active, inactive]` |
|
||||
| `field?(object):` | Nested object with sub-fields | `metadata?(object):` |
|
||||
| `, description` | Description after comma | `name: string, full name` |
|
||||
| `EntityName` | Capitalized type = entity reference | `works_at?: Organization` |
|
||||
|
||||
**Scalar types:** `string`, `integer`, `number`, `boolean`, `any`
|
||||
|
||||
Any type not in that set whose first letter is uppercase is treated as an entity reference (a relation target).
|
||||
|
||||
### Schema-to-Note Mapping
|
||||
|
||||
Schemas validate against existing observation/relation syntax. Note authors don't learn new syntax.
|
||||
|
||||
| Schema Declaration | Maps To | Example in Note |
|
||||
|--------------------|---------|-----------------|
|
||||
| `field: string` | Observation `[field] value` | `- [name] Paul Graham` |
|
||||
| `field?(array): string` | Multiple `[field]` observations | `- [expertise] Lisp` (repeated) |
|
||||
| `field?: EntityType` | Relation `field [[Target]]` | `- works_at [[Y Combinator]]` |
|
||||
| `field?(array): EntityType` | Multiple `field` relations | `- authored [[Book]]` (repeated) |
|
||||
| `tags` | Frontmatter `tags` array | `tags: [startups, essays]` |
|
||||
| `field?(enum): [vals]` | Observation `[field] value` where value is in the set | `- [status] active` |
|
||||
|
||||
Observations and relations not covered by the schema are valid — schemas describe a subset, not a straitjacket.
|
||||
|
||||
### Schema Attachment
|
||||
|
||||
Three ways to attach a schema to a note, resolved in priority order:
|
||||
|
||||
**1. Inline schema** — `schema` is a dict in frontmatter:
|
||||
|
||||
```yaml
|
||||
---
|
||||
title: Team Standup 2024-01-15
|
||||
type: meeting
|
||||
schema:
|
||||
attendees(array): string, who was there
|
||||
decisions(array): string, what was decided
|
||||
action_items(array): string, follow-ups
|
||||
blockers?(array): string, anything stuck
|
||||
---
|
||||
```
|
||||
|
||||
Good for one-off structured notes or prototyping a schema before extracting it.
|
||||
|
||||
**2. Explicit reference** — `schema` is a string naming a schema note:
|
||||
|
||||
```yaml
|
||||
---
|
||||
title: Basic Memory
|
||||
schema: SoftwareProject
|
||||
---
|
||||
```
|
||||
|
||||
or by permalink:
|
||||
|
||||
```yaml
|
||||
---
|
||||
title: LLM Memory Patterns
|
||||
schema: schema/research-project
|
||||
---
|
||||
```
|
||||
|
||||
Use when the note's `type` differs from the schema it should validate against, or when multiple schema variants exist.
|
||||
|
||||
**3. Implicit by type** — no `schema` field, resolved by matching `type`:
|
||||
|
||||
```yaml
|
||||
---
|
||||
title: Paul Graham
|
||||
type: Person
|
||||
---
|
||||
```
|
||||
|
||||
The system looks up a schema note where `entity: Person`. If found, it applies. If not, no validation occurs.
|
||||
|
||||
**4. No schema** — perfectly fine. Most notes don't need one.
|
||||
|
||||
### Schema Notes
|
||||
|
||||
A schema is itself a Basic Memory note with `type: schema`. It lives anywhere (though `schema/` is the conventional directory).
|
||||
|
||||
```yaml
|
||||
# schema/Person.md
|
||||
---
|
||||
title: Person
|
||||
type: schema
|
||||
entity: Person
|
||||
version: 1
|
||||
schema:
|
||||
name: string, full name
|
||||
role?: string, job title or position
|
||||
works_at?: Organization, employer
|
||||
expertise?(array): string, areas of knowledge
|
||||
email?: string, contact email
|
||||
settings:
|
||||
validation: warn
|
||||
---
|
||||
|
||||
# Person
|
||||
|
||||
A human individual in the knowledge graph.
|
||||
```
|
||||
|
||||
| Field | Required | Description |
|
||||
|-------|----------|-------------|
|
||||
| `type` | Yes | Must be `schema` |
|
||||
| `entity` | Yes | The entity type this schema describes (e.g., `Person`) |
|
||||
| `version` | No | Schema version number (default: `1`) |
|
||||
| `schema` | Yes | Picoschema dict defining the fields |
|
||||
| `settings.validation` | No | Validation mode (default: `warn`) |
|
||||
|
||||
Schema notes are regular notes — they show up in search, can have observations and relations, and participate in the knowledge graph.
|
||||
|
||||
### Validation Modes
|
||||
|
||||
| Mode | Behavior |
|
||||
|------|----------|
|
||||
| `warn` | Warnings in output, doesn't block (default) |
|
||||
| `strict` | Errors that block sync, for CI/CD enforcement |
|
||||
| `off` | No validation |
|
||||
|
||||
### Validation Output
|
||||
|
||||
```
|
||||
$ bm schema validate people/ada-lovelace.md
|
||||
|
||||
⚠ Person schema validation:
|
||||
- Missing required field: name (expected [name] observation)
|
||||
- Missing optional field: role
|
||||
- Missing optional field: works_at (no relation found)
|
||||
|
||||
ℹ Unmatched observations: [fact] ×2, [born] ×1
|
||||
ℹ Unmatched relations: collaborated_with
|
||||
```
|
||||
|
||||
"Unmatched" items are informational — observations and relations the schema doesn't cover.
|
||||
|
||||
### Schema Inference
|
||||
|
||||
Generate schemas from existing notes by analyzing observation and relation frequency:
|
||||
|
||||
```
|
||||
$ bm schema infer Person
|
||||
|
||||
Analyzing 30 notes with type: Person...
|
||||
|
||||
Observations found:
|
||||
[name] 30/30 100% → name: string
|
||||
[role] 27/30 90% → role?: string
|
||||
[expertise] 18/30 60% → expertise?(array): string
|
||||
[email] 8/30 27% → email?: string
|
||||
|
||||
Relations found:
|
||||
works_at 22/30 73% → works_at?: Organization
|
||||
|
||||
Suggested schema:
|
||||
name: string, full name
|
||||
role?: string, job title
|
||||
expertise?(array): string, areas of knowledge
|
||||
email?: string, contact email
|
||||
works_at?: Organization, employer
|
||||
|
||||
Save to schema/Person.md? [y/n]
|
||||
```
|
||||
|
||||
Frequency thresholds:
|
||||
- **100% present** → required field
|
||||
- **25%+ present** → optional field
|
||||
- **Below 25%** → excluded from suggestion
|
||||
|
||||
### Schema Drift Detection
|
||||
|
||||
Track how usage patterns shift over time:
|
||||
|
||||
```
|
||||
$ bm schema diff Person
|
||||
|
||||
Schema drift detected:
|
||||
|
||||
+ expertise: now in 81% of notes (was 12%)
|
||||
- department: dropped to 3% of notes
|
||||
~ works_at: cardinality changed (one → many)
|
||||
|
||||
Update schema? [y/n/review]
|
||||
```
|
||||
|
||||
## Complete Examples
|
||||
|
||||
### Simple Note (No Schema)
|
||||
|
||||
```markdown
|
||||
---
|
||||
title: Project Ideas
|
||||
type: note
|
||||
tags: [ideas, brainstorm]
|
||||
---
|
||||
|
||||
# Project Ideas
|
||||
|
||||
## Observations
|
||||
- [idea] Build a CLI tool for markdown linting #tooling
|
||||
- [idea] Create a recipe knowledge base #cooking
|
||||
- [priority] Focus on developer tools first (Q1 goal)
|
||||
|
||||
## Relations
|
||||
- inspired_by [[Developer Workflow Research]]
|
||||
- part_of [[Q1 Planning]]
|
||||
```
|
||||
|
||||
### Schema-Validated Note
|
||||
|
||||
Schema at `schema/Person.md`:
|
||||
|
||||
```yaml
|
||||
---
|
||||
title: Person
|
||||
type: schema
|
||||
entity: Person
|
||||
version: 1
|
||||
schema:
|
||||
name: string, full name
|
||||
role?: string, job title or position
|
||||
works_at?: Organization, employer
|
||||
expertise?(array): string, areas of knowledge
|
||||
email?: string, contact email
|
||||
settings:
|
||||
validation: warn
|
||||
---
|
||||
|
||||
# Person
|
||||
|
||||
A human individual in the knowledge graph.
|
||||
```
|
||||
|
||||
Note at `people/paul-graham.md`:
|
||||
|
||||
```markdown
|
||||
---
|
||||
title: Paul Graham
|
||||
type: Person
|
||||
tags: [startups, essays, lisp]
|
||||
---
|
||||
|
||||
# Paul Graham
|
||||
|
||||
## Observations
|
||||
- [name] Paul Graham
|
||||
- [role] Essayist and investor
|
||||
- [expertise] Startups
|
||||
- [expertise] Lisp
|
||||
- [expertise] Essay writing
|
||||
- [fact] Created Viaweb, the first web app
|
||||
|
||||
## Relations
|
||||
- works_at [[Y Combinator]]
|
||||
- authored [[Hackers and Painters]]
|
||||
```
|
||||
|
||||
The `[fact]` observation and `authored` relation are not in the schema — they're valid, just unmatched. The schema only checks that `[name]` exists (required) and looks for optional fields like `[role]`, `[expertise]`, and `works_at`.
|
||||
|
||||
### Inline Schema Note
|
||||
|
||||
```markdown
|
||||
---
|
||||
title: Team Standup 2024-01-15
|
||||
type: meeting
|
||||
schema:
|
||||
attendees(array): string, who was there
|
||||
decisions(array): string, what was decided
|
||||
action_items(array): string, follow-ups
|
||||
blockers?(array): string, anything stuck
|
||||
---
|
||||
|
||||
# Team Standup 2024-01-15
|
||||
|
||||
## Observations
|
||||
- [attendees] Paul
|
||||
- [attendees] Sarah
|
||||
- [decisions] Ship v2 by Friday
|
||||
- [action_items] Paul to review PR #42
|
||||
- [blockers] Waiting on API credentials
|
||||
```
|
||||
File diff suppressed because it is too large
Load Diff
Binary file not shown.
|
After Width: | Height: | Size: 416 KiB |
@@ -1,241 +0,0 @@
|
||||
# Character Handling and Conflict Resolution
|
||||
|
||||
Basic Memory handles various character encoding scenarios and file naming conventions to provide consistent permalink generation and conflict resolution. This document explains how the system works and how to resolve common character-related issues.
|
||||
|
||||
## Overview
|
||||
|
||||
Basic Memory uses a sophisticated system to generate permalinks from file paths while maintaining consistency across different operating systems and character encodings. The system normalizes file paths and generates unique permalinks to prevent conflicts.
|
||||
|
||||
## Character Normalization Rules
|
||||
|
||||
### 1. Permalink Generation
|
||||
|
||||
When Basic Memory processes a file path, it applies these normalization rules:
|
||||
|
||||
```
|
||||
Original: "Finance/My Investment Strategy.md"
|
||||
Permalink: "finance/my-investment-strategy"
|
||||
```
|
||||
|
||||
**Transformation process:**
|
||||
1. Remove file extension (`.md`)
|
||||
2. Convert to lowercase (case-insensitive)
|
||||
3. Replace spaces with hyphens
|
||||
4. Replace underscores with hyphens
|
||||
5. Handle international characters (transliteration for Latin, preservation for non-Latin)
|
||||
6. Convert camelCase to kebab-case
|
||||
|
||||
### 2. International Character Support
|
||||
|
||||
**Latin characters with diacritics** are transliterated:
|
||||
- `ø` → `o` (Søren → soren)
|
||||
- `ü` → `u` (Müller → muller)
|
||||
- `é` → `e` (Café → cafe)
|
||||
- `ñ` → `n` (Niño → nino)
|
||||
|
||||
**Non-Latin characters** are preserved:
|
||||
- Chinese: `中文/测试文档.md` → `中文/测试文档`
|
||||
- Japanese: `日本語/文書.md` → `日本語/文書`
|
||||
|
||||
## Common Conflict Scenarios
|
||||
|
||||
### 1. Hyphen vs Space Conflicts
|
||||
|
||||
**Problem:** Files with existing hyphens conflict with generated permalinks from spaces.
|
||||
|
||||
**Example:**
|
||||
```
|
||||
File 1: "basic memory bug.md" → permalink: "basic-memory-bug"
|
||||
File 2: "basic-memory-bug.md" → permalink: "basic-memory-bug" (CONFLICT!)
|
||||
```
|
||||
|
||||
**Resolution:** The system automatically resolves this by adding suffixes:
|
||||
```
|
||||
File 1: "basic memory bug.md" → permalink: "basic-memory-bug"
|
||||
File 2: "basic-memory-bug.md" → permalink: "basic-memory-bug-1"
|
||||
```
|
||||
|
||||
**Best Practice:** Choose consistent naming conventions within your project.
|
||||
|
||||
### 2. Case Sensitivity Conflicts
|
||||
|
||||
**Problem:** Different case variations that normalize to the same permalink.
|
||||
|
||||
**Example on macOS:**
|
||||
```
|
||||
Directory: Finance/investment.md
|
||||
Directory: finance/investment.md (different on filesystem, same permalink)
|
||||
```
|
||||
|
||||
**Resolution:** Basic Memory detects case conflicts and prevents them during sync operations with helpful error messages.
|
||||
|
||||
**Best Practice:** Use consistent casing for directory and file names.
|
||||
|
||||
### 3. Character Encoding Conflicts
|
||||
|
||||
**Problem:** Different Unicode normalizations of the same logical character.
|
||||
|
||||
**Example:**
|
||||
```
|
||||
File 1: "café.md" (é as single character)
|
||||
File 2: "café.md" (e + combining accent)
|
||||
```
|
||||
|
||||
**Resolution:** Basic Memory normalizes Unicode characters using NFD normalization to detect these conflicts.
|
||||
|
||||
### 4. Forward Slash Conflicts
|
||||
|
||||
**Problem:** Forward slashes in frontmatter or file names interpreted as path separators.
|
||||
|
||||
**Example:**
|
||||
```yaml
|
||||
---
|
||||
permalink: finance/investment/strategy
|
||||
---
|
||||
```
|
||||
|
||||
**Resolution:** Basic Memory validates frontmatter permalinks and warns about path separator conflicts.
|
||||
|
||||
## Error Messages and Troubleshooting
|
||||
|
||||
### "UNIQUE constraint failed: entity.file_path, entity.project_id"
|
||||
|
||||
**Cause:** Two entities trying to use the same file path within a project.
|
||||
|
||||
**Common scenarios:**
|
||||
1. File move operation where destination is already occupied
|
||||
2. Case sensitivity differences on macOS
|
||||
3. Character encoding conflicts
|
||||
4. Concurrent file operations
|
||||
|
||||
**Resolution steps:**
|
||||
1. Check for duplicate file names with different cases
|
||||
2. Look for files with similar names but different character encodings
|
||||
3. Rename conflicting files to have unique names
|
||||
4. Run sync again after resolving conflicts
|
||||
|
||||
### "File path conflict detected during move"
|
||||
|
||||
**Cause:** Enhanced conflict detection preventing potential database integrity violations.
|
||||
|
||||
**What this means:** The system detected that moving a file would create a conflict before attempting the database operation.
|
||||
|
||||
**Resolution:** Follow the specific guidance in the error message, which will indicate the type of conflict detected.
|
||||
|
||||
## Best Practices
|
||||
|
||||
### 1. File Naming Conventions
|
||||
|
||||
**Recommended patterns:**
|
||||
- Use consistent casing (prefer lowercase)
|
||||
- Use hyphens instead of spaces for multi-word files
|
||||
- Avoid special characters that could conflict with path separators
|
||||
- Be consistent with directory structure casing
|
||||
|
||||
**Examples:**
|
||||
```
|
||||
✅ Good:
|
||||
- finance/investment-strategy.md
|
||||
- projects/basic-memory-features.md
|
||||
- docs/api-reference.md
|
||||
|
||||
❌ Problematic:
|
||||
- Finance/Investment Strategy.md (mixed case, spaces)
|
||||
- finance/Investment Strategy.md (inconsistent case)
|
||||
- docs/API/Reference.md (mixed case directories)
|
||||
```
|
||||
|
||||
### 2. Permalink Management
|
||||
|
||||
**Custom permalinks in frontmatter:**
|
||||
```yaml
|
||||
---
|
||||
type: knowledge
|
||||
permalink: custom-permalink-name
|
||||
---
|
||||
```
|
||||
|
||||
**Guidelines:**
|
||||
- Use lowercase permalinks
|
||||
- Use hyphens for word separation
|
||||
- Avoid path separators unless creating sub-paths
|
||||
- Ensure uniqueness within your project
|
||||
|
||||
### 3. Directory Structure
|
||||
|
||||
**Consistent casing:**
|
||||
```
|
||||
✅ Good:
|
||||
finance/
|
||||
investment-strategies.md
|
||||
portfolio-management.md
|
||||
|
||||
❌ Problematic:
|
||||
Finance/ (capital F)
|
||||
investment-strategies.md
|
||||
finance/ (lowercase f)
|
||||
portfolio-management.md
|
||||
```
|
||||
|
||||
## Migration and Cleanup
|
||||
|
||||
### Identifying Conflicts
|
||||
|
||||
Use Basic Memory's built-in conflict detection:
|
||||
|
||||
```bash
|
||||
# Sync will report conflicts
|
||||
basic-memory sync
|
||||
|
||||
# Check sync status for warnings
|
||||
basic-memory status
|
||||
```
|
||||
|
||||
### Resolving Existing Conflicts
|
||||
|
||||
1. **Identify conflicting files** from sync error messages
|
||||
2. **Choose consistent naming convention** for your project
|
||||
3. **Rename files** to follow the convention
|
||||
4. **Re-run sync** to verify resolution
|
||||
|
||||
### Bulk Renaming Strategy
|
||||
|
||||
For projects with many conflicts:
|
||||
|
||||
1. **Backup your project** before making changes
|
||||
2. **Standardize on lowercase** file and directory names
|
||||
3. **Replace spaces with hyphens** in file names
|
||||
4. **Use consistent character encoding** (UTF-8)
|
||||
5. **Test sync after each batch** of changes
|
||||
|
||||
## System Enhancements
|
||||
|
||||
### Recent Improvements (v0.13+)
|
||||
|
||||
1. **Enhanced conflict detection** before database operations
|
||||
2. **Improved error messages** with specific resolution guidance
|
||||
3. **Character normalization utilities** for consistent handling
|
||||
4. **File swap detection** for complex move scenarios
|
||||
5. **Proactive conflict warnings** during permalink resolution
|
||||
|
||||
### Monitoring and Logging
|
||||
|
||||
The system now provides detailed logging for conflict resolution:
|
||||
|
||||
```
|
||||
DEBUG: Detected potential file path conflicts for 'Finance/Investment.md': ['finance/investment.md']
|
||||
WARNING: File path conflict detected during move: entity_id=123 trying to move from 'old.md' to 'new.md'
|
||||
```
|
||||
|
||||
These logs help identify and resolve conflicts before they cause sync failures.
|
||||
|
||||
## Support and Resources
|
||||
|
||||
If you encounter character-related conflicts not covered in this guide:
|
||||
|
||||
1. **Check the logs** for specific conflict details
|
||||
2. **Review error messages** for resolution guidance
|
||||
3. **Report issues** with examples of the conflicting files
|
||||
4. **Consider the file naming best practices** outlined above
|
||||
|
||||
The Basic Memory system is designed to handle most character conflicts automatically while providing clear guidance for manual resolution when needed.
|
||||
@@ -1,726 +0,0 @@
|
||||
# Basic Memory Cloud CLI Guide
|
||||
|
||||
The Basic Memory Cloud CLI provides seamless integration between local and cloud knowledge bases using **project-scoped synchronization**. Each project can optionally sync with the cloud, giving you fine-grained control over what syncs and where.
|
||||
|
||||
## Overview
|
||||
|
||||
The cloud CLI enables you to:
|
||||
- **Toggle cloud mode** - All regular `bm` commands work with cloud when enabled
|
||||
- **Project-scoped sync** - Each project independently manages its sync configuration
|
||||
- **Explicit operations** - Sync only what you want, when you want
|
||||
- **Bidirectional sync** - Keep local and cloud in sync with rclone bisync
|
||||
- **Offline access** - Work locally, sync when ready
|
||||
|
||||
## Prerequisites
|
||||
|
||||
Before using Basic Memory Cloud, you need:
|
||||
|
||||
- **Active Subscription**: An active Basic Memory Cloud subscription is required to access cloud features
|
||||
- **Subscribe**: Visit [https://basicmemory.com/subscribe](https://basicmemory.com/subscribe) to sign up
|
||||
|
||||
If you attempt to log in without an active subscription, you'll receive a "Subscription Required" error with a link to subscribe.
|
||||
|
||||
## Architecture: Project-Scoped Sync
|
||||
|
||||
### The Problem
|
||||
|
||||
**Old approach (SPEC-8):** All projects lived in a single `~/basic-memory-cloud-sync/` directory. This caused:
|
||||
- ❌ Directory conflicts between mount and bisync
|
||||
- ❌ Auto-discovery creating phantom projects
|
||||
- ❌ Confusion about what syncs and when
|
||||
- ❌ All-or-nothing sync (couldn't sync just one project)
|
||||
|
||||
**New approach (SPEC-20):** Each project independently configures sync.
|
||||
|
||||
### How It Works
|
||||
|
||||
**Projects can exist in three states:**
|
||||
|
||||
1. **Cloud-only** - Project exists on cloud, no local copy
|
||||
2. **Cloud + Local (synced)** - Project has a local working directory that syncs
|
||||
3. **Local-only** - Project exists locally (when cloud mode is disabled)
|
||||
|
||||
**Example:**
|
||||
|
||||
```bash
|
||||
# You have 3 projects on cloud:
|
||||
# - research: wants local sync at ~/Documents/research
|
||||
# - work: wants local sync at ~/work-notes
|
||||
# - temp: cloud-only, no local sync needed
|
||||
|
||||
bm project add research --local-path ~/Documents/research
|
||||
bm project add work --local-path ~/work-notes
|
||||
bm project add temp # No local sync
|
||||
|
||||
# Now you can sync individually (after initial --resync):
|
||||
bm project bisync --name research
|
||||
bm project bisync --name work
|
||||
# temp stays cloud-only
|
||||
```
|
||||
|
||||
**What happens under the covers:**
|
||||
- Config stores `cloud_projects` dict mapping project names to local paths
|
||||
- Each project gets its own bisync state in `~/.basic-memory/bisync-state/{project}/`
|
||||
- Rclone syncs using single remote: `basic-memory-cloud`
|
||||
- Projects can live anywhere on your filesystem, not forced into sync directory
|
||||
|
||||
## Quick Start
|
||||
|
||||
### 1. Enable Cloud Mode
|
||||
|
||||
Authenticate and enable cloud mode:
|
||||
|
||||
```bash
|
||||
bm cloud login
|
||||
```
|
||||
|
||||
**What this does:**
|
||||
1. Opens browser to Basic Memory Cloud authentication page
|
||||
2. Stores authentication token in `~/.basic-memory/auth/token`
|
||||
3. **Enables cloud mode** - all CLI commands now work against cloud
|
||||
4. Validates your subscription status
|
||||
|
||||
**Result:** All `bm project`, `bm tools` commands now work with cloud.
|
||||
|
||||
### 2. Set Up Sync
|
||||
|
||||
Install rclone and configure credentials:
|
||||
|
||||
```bash
|
||||
bm cloud setup
|
||||
```
|
||||
|
||||
**What this does:**
|
||||
1. Installs rclone automatically (if needed)
|
||||
2. Fetches your tenant information from cloud
|
||||
3. Generates scoped S3 credentials for sync
|
||||
4. Configures single rclone remote: `basic-memory-cloud`
|
||||
|
||||
**Result:** You're ready to sync projects. No sync directories created yet - those come with project setup.
|
||||
|
||||
### 3. Add Projects with Sync
|
||||
|
||||
Create projects with optional local sync paths:
|
||||
|
||||
```bash
|
||||
# Create cloud project without local sync
|
||||
bm project add research
|
||||
|
||||
# Create cloud project WITH local sync
|
||||
bm project add research --local-path ~/Documents/research
|
||||
|
||||
# Or configure sync for existing project
|
||||
bm project sync-setup research ~/Documents/research
|
||||
```
|
||||
|
||||
**What happens under the covers:**
|
||||
|
||||
When you add a project with `--local-path`:
|
||||
1. Project created on cloud at `/app/data/research`
|
||||
2. Local path stored in config: `cloud_projects.research.local_path = "~/Documents/research"`
|
||||
3. Local directory created if it doesn't exist
|
||||
4. Bisync state directory created at `~/.basic-memory/bisync-state/research/`
|
||||
|
||||
**Result:** Project is ready to sync, but no files synced yet.
|
||||
|
||||
### 4. Sync Your Project
|
||||
|
||||
Establish the initial sync baseline. **Best practice:** Always preview with `--dry-run` first:
|
||||
|
||||
```bash
|
||||
# Step 1: Preview the initial sync (recommended)
|
||||
bm project bisync --name research --resync --dry-run
|
||||
|
||||
# Step 2: If all looks good, run the actual sync
|
||||
bm project bisync --name research --resync
|
||||
```
|
||||
|
||||
**What happens under the covers:**
|
||||
1. Rclone reads from `~/Documents/research` (local)
|
||||
2. Connects to `basic-memory-cloud:bucket-name/app/data/research` (remote)
|
||||
3. Creates bisync state files in `~/.basic-memory/bisync-state/research/`
|
||||
4. Syncs files bidirectionally with settings:
|
||||
- `conflict_resolve=newer` (most recent wins)
|
||||
- `max_delete=25` (safety limit)
|
||||
- Respects `.bmignore` patterns
|
||||
|
||||
**Result:** Local and cloud are in sync. Baseline established.
|
||||
|
||||
**Why `--resync`?** This is an rclone requirement for the first bisync run. It establishes the initial state that future syncs will compare against. After the first sync, never use `--resync` unless you need to force a new baseline.
|
||||
|
||||
See: https://rclone.org/bisync/#resync
|
||||
```
|
||||
--resync
|
||||
This will effectively make both Path1 and Path2 filesystems contain a matching superset of all files. By default, Path2 files that do not exist in Path1 will be copied to Path1, and the process will then copy the Path1 tree to Path2.
|
||||
```
|
||||
|
||||
### 5. Subsequent Syncs
|
||||
|
||||
After the first sync, just run bisync without `--resync`:
|
||||
|
||||
```bash
|
||||
bm project bisync --name research
|
||||
```
|
||||
|
||||
**What happens:**
|
||||
1. Rclone compares local and cloud states
|
||||
2. Syncs changes in both directions
|
||||
3. Auto-resolves conflicts (newer file wins)
|
||||
4. Updates `last_sync` timestamp in config
|
||||
|
||||
**Result:** Changes flow both ways - edit locally or in cloud, both stay in sync.
|
||||
|
||||
### 6. Verify Setup
|
||||
|
||||
Check status:
|
||||
|
||||
```bash
|
||||
bm cloud status
|
||||
```
|
||||
|
||||
You should see:
|
||||
- `Mode: Cloud (enabled)`
|
||||
- `Cloud instance is healthy`
|
||||
- Instructions for project sync commands
|
||||
|
||||
## Working with Projects
|
||||
|
||||
### Understanding Project Commands
|
||||
|
||||
**Key concept:** When cloud mode is enabled, use regular `bm project` commands (not `bm cloud project`).
|
||||
|
||||
```bash
|
||||
# In cloud mode:
|
||||
bm project list # Lists cloud projects
|
||||
bm project add research # Creates cloud project
|
||||
|
||||
# In local mode:
|
||||
bm project list # Lists local projects
|
||||
bm project add research ~/Documents/research # Creates local project
|
||||
```
|
||||
|
||||
### Creating Projects
|
||||
|
||||
**Use case 1: Cloud-only project (no local sync)**
|
||||
|
||||
```bash
|
||||
bm project add temp-notes
|
||||
```
|
||||
|
||||
**What this does:**
|
||||
- Creates project on cloud at `/app/data/temp-notes`
|
||||
- No local directory created
|
||||
- No sync configuration
|
||||
|
||||
**Result:** Project exists on cloud, accessible via MCP tools, but no local copy.
|
||||
|
||||
**Use case 2: Cloud project with local sync**
|
||||
|
||||
```bash
|
||||
bm project add research --local-path ~/Documents/research
|
||||
```
|
||||
|
||||
**What this does:**
|
||||
- Creates project on cloud at `/app/data/research`
|
||||
- Creates local directory `~/Documents/research`
|
||||
- Stores sync config in `~/.basic-memory/config.json`
|
||||
- Prepares for bisync (but doesn't sync yet)
|
||||
|
||||
**Result:** Project ready to sync. Run `bm project bisync --name research --resync` to establish baseline.
|
||||
|
||||
**Use case 3: Add sync to existing cloud project**
|
||||
|
||||
```bash
|
||||
# Project already exists on cloud
|
||||
bm project sync-setup research ~/Documents/research
|
||||
```
|
||||
|
||||
**What this does:**
|
||||
- Updates existing project's sync configuration
|
||||
- Creates local directory
|
||||
- Prepares for bisync
|
||||
|
||||
**Result:** Existing cloud project now has local sync path. Run bisync to pull files down.
|
||||
|
||||
### Listing Projects
|
||||
|
||||
View all projects:
|
||||
|
||||
```bash
|
||||
bm project list
|
||||
```
|
||||
|
||||
**What you see:**
|
||||
- All projects in cloud (when cloud mode enabled)
|
||||
- Default project marked
|
||||
- Project paths shown
|
||||
|
||||
**Future:** Will show sync status (synced/not synced, last sync time).
|
||||
|
||||
## File Synchronization
|
||||
|
||||
### Understanding the Sync Commands
|
||||
|
||||
**There are three sync-related commands:**
|
||||
|
||||
1. `bm project sync` - One-way: local → cloud (make cloud match local)
|
||||
2. `bm project bisync` - Two-way: local ↔ cloud (recommended)
|
||||
3. `bm project check` - Verify files match (no changes)
|
||||
|
||||
### One-Way Sync: Local → Cloud
|
||||
|
||||
**Use case:** You made changes locally and want to push to cloud (overwrite cloud).
|
||||
|
||||
```bash
|
||||
bm project sync --name research
|
||||
```
|
||||
|
||||
**What happens:**
|
||||
1. Reads files from `~/Documents/research` (local)
|
||||
2. Uses rclone sync to make cloud identical to local
|
||||
3. Respects `.bmignore` patterns
|
||||
4. Shows progress bar
|
||||
|
||||
**Result:** Cloud now matches local exactly. Any cloud-only changes are overwritten.
|
||||
|
||||
**When to use:**
|
||||
- You know local is the source of truth
|
||||
- You want to force cloud to match local
|
||||
- You don't care about cloud changes
|
||||
|
||||
### Two-Way Sync: Local ↔ Cloud (Recommended)
|
||||
|
||||
**Use case:** You edit files both locally and in cloud UI, want both to stay in sync.
|
||||
|
||||
```bash
|
||||
# First time - establish baseline
|
||||
bm project bisync --name research --resync
|
||||
|
||||
# Subsequent syncs
|
||||
bm project bisync --name research
|
||||
```
|
||||
|
||||
**What happens:**
|
||||
1. Compares local and cloud states using bisync metadata
|
||||
2. Syncs changes in both directions
|
||||
3. Auto-resolves conflicts (newer file wins)
|
||||
4. Detects excessive deletes and fails safely (max 25 files)
|
||||
|
||||
**Conflict resolution example:**
|
||||
|
||||
```bash
|
||||
# Edit locally
|
||||
echo "Local change" > ~/Documents/research/notes.md
|
||||
|
||||
# Edit same file in cloud UI
|
||||
# Cloud now has: "Cloud change"
|
||||
|
||||
# Run bisync
|
||||
bm project bisync --name research
|
||||
|
||||
# Result: Newer file wins (based on modification time)
|
||||
# If cloud was more recent, cloud version kept
|
||||
# If local was more recent, local version kept
|
||||
```
|
||||
|
||||
**When to use:**
|
||||
- Default workflow for most users
|
||||
- You edit in multiple places
|
||||
- You want automatic conflict resolution
|
||||
|
||||
### Verify Sync Integrity
|
||||
|
||||
**Use case:** Check if local and cloud match without making changes.
|
||||
|
||||
```bash
|
||||
bm project check --name research
|
||||
```
|
||||
|
||||
**What happens:**
|
||||
1. Compares file checksums between local and cloud
|
||||
2. Reports differences
|
||||
3. No files transferred
|
||||
|
||||
**Result:** Shows which files differ. Run bisync to sync them.
|
||||
|
||||
```bash
|
||||
# One-way check (faster)
|
||||
bm project check --name research --one-way
|
||||
```
|
||||
|
||||
### Preview Changes (Dry Run)
|
||||
|
||||
**Use case:** See what would change without actually syncing.
|
||||
|
||||
```bash
|
||||
bm project bisync --name research --dry-run
|
||||
```
|
||||
|
||||
**What happens:**
|
||||
1. Runs bisync logic
|
||||
2. Shows what would be transferred/deleted
|
||||
3. No actual changes made
|
||||
|
||||
**Result:** Safe preview of sync operations.
|
||||
|
||||
### Advanced: List Remote Files
|
||||
|
||||
**Use case:** See what files exist on cloud without syncing.
|
||||
|
||||
```bash
|
||||
# List all files in project
|
||||
bm project ls --name research
|
||||
|
||||
# List files in subdirectory
|
||||
bm project ls --name research --path subfolder
|
||||
```
|
||||
|
||||
**What happens:**
|
||||
1. Connects to cloud via rclone
|
||||
2. Lists files in remote project path
|
||||
3. No files transferred
|
||||
|
||||
**Result:** See cloud file listing.
|
||||
|
||||
## Multiple Projects
|
||||
|
||||
### Syncing Multiple Projects
|
||||
|
||||
**Use case:** You have several projects with local sync, want to sync all at once.
|
||||
|
||||
```bash
|
||||
# Setup multiple projects
|
||||
bm project add research --local-path ~/Documents/research
|
||||
bm project add work --local-path ~/work-notes
|
||||
bm project add personal --local-path ~/personal
|
||||
|
||||
# Establish baselines
|
||||
bm project bisync --name research --resync
|
||||
bm project bisync --name work --resync
|
||||
bm project bisync --name personal --resync
|
||||
|
||||
# Daily workflow: sync everything
|
||||
bm project bisync --name research
|
||||
bm project bisync --name work
|
||||
bm project bisync --name personal
|
||||
```
|
||||
|
||||
**Future:** `--all` flag will sync all configured projects:
|
||||
|
||||
```bash
|
||||
bm project bisync --all # Coming soon
|
||||
```
|
||||
|
||||
### Mixed Usage
|
||||
|
||||
**Use case:** Some projects sync, some stay cloud-only.
|
||||
|
||||
```bash
|
||||
# Projects with sync
|
||||
bm project add research --local-path ~/Documents/research
|
||||
bm project add work --local-path ~/work
|
||||
|
||||
# Cloud-only projects
|
||||
bm project add archive
|
||||
bm project add temp-notes
|
||||
|
||||
# Sync only the configured ones
|
||||
bm project bisync --name research
|
||||
bm project bisync --name work
|
||||
|
||||
# Archive and temp-notes stay cloud-only
|
||||
```
|
||||
|
||||
**Result:** Fine-grained control over what syncs.
|
||||
|
||||
## Disable Cloud Mode
|
||||
|
||||
Return to local mode:
|
||||
|
||||
```bash
|
||||
bm cloud logout
|
||||
```
|
||||
|
||||
**What this does:**
|
||||
1. Disables cloud mode in config
|
||||
2. All commands now work locally
|
||||
3. Auth token remains (can re-enable with login)
|
||||
|
||||
**Result:** All `bm` commands work with local projects again.
|
||||
|
||||
## Filter Configuration
|
||||
|
||||
### Understanding .bmignore
|
||||
|
||||
**The problem:** You don't want to sync everything (e.g., `.git`, `node_modules`, database files).
|
||||
|
||||
**The solution:** `.bmignore` file with gitignore-style patterns.
|
||||
|
||||
**Location:** `~/.basic-memory/.bmignore`
|
||||
|
||||
**Default patterns:**
|
||||
|
||||
```gitignore
|
||||
# Version control
|
||||
.git/**
|
||||
|
||||
# Python
|
||||
__pycache__/**
|
||||
*.pyc
|
||||
.venv/**
|
||||
venv/**
|
||||
|
||||
# Node.js
|
||||
node_modules/**
|
||||
|
||||
# Basic Memory internals
|
||||
memory.db/**
|
||||
memory.db-shm/**
|
||||
memory.db-wal/**
|
||||
config.json/**
|
||||
watch-status.json/**
|
||||
.bmignore.rclone/**
|
||||
|
||||
# OS files
|
||||
.DS_Store/**
|
||||
Thumbs.db/**
|
||||
|
||||
# Environment files
|
||||
.env/**
|
||||
.env.local/**
|
||||
```
|
||||
|
||||
**How it works:**
|
||||
1. On first sync, `.bmignore` created with defaults
|
||||
2. Patterns converted to rclone filter format (`.bmignore.rclone`)
|
||||
3. Rclone uses filters during sync
|
||||
4. Same patterns used by all projects
|
||||
|
||||
**Customizing:**
|
||||
|
||||
```bash
|
||||
# Edit patterns
|
||||
code ~/.basic-memory/.bmignore
|
||||
|
||||
# Add custom patterns
|
||||
echo "*.tmp/**" >> ~/.basic-memory/.bmignore
|
||||
|
||||
# Next sync uses updated patterns
|
||||
bm project bisync --name research
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Authentication Issues
|
||||
|
||||
**Problem:** "Authentication failed" or "Invalid token"
|
||||
|
||||
**Solution:** Re-authenticate:
|
||||
|
||||
```bash
|
||||
bm cloud logout
|
||||
bm cloud login
|
||||
```
|
||||
|
||||
### Subscription Issues
|
||||
|
||||
**Problem:** "Subscription Required" error
|
||||
|
||||
**Solution:**
|
||||
1. Visit subscribe URL shown in error
|
||||
2. Sign up for subscription
|
||||
3. Run `bm cloud login` again
|
||||
|
||||
**Note:** Access is immediate when subscription becomes active.
|
||||
|
||||
### Bisync Initialization
|
||||
|
||||
**Problem:** "First bisync requires --resync"
|
||||
|
||||
**Explanation:** Bisync needs a baseline state before it can sync changes.
|
||||
|
||||
**Solution:**
|
||||
|
||||
```bash
|
||||
bm project bisync --name research --resync
|
||||
```
|
||||
|
||||
**What this does:**
|
||||
- Establishes initial sync state
|
||||
- Creates baseline in `~/.basic-memory/bisync-state/research/`
|
||||
- Syncs all files bidirectionally
|
||||
|
||||
**Result:** Future syncs work without `--resync`.
|
||||
|
||||
### Empty Directory Issues
|
||||
|
||||
**Problem:** "Empty prior Path1 listing. Cannot sync to an empty directory"
|
||||
|
||||
**Explanation:** Rclone bisync doesn't work well with completely empty directories. It needs at least one file to establish a baseline.
|
||||
|
||||
**Solution:** Add at least one file before running `--resync`:
|
||||
|
||||
```bash
|
||||
# Create a placeholder file
|
||||
echo "# Research Notes" > ~/Documents/research/README.md
|
||||
|
||||
# Now run bisync
|
||||
bm project bisync --name research --resync
|
||||
```
|
||||
|
||||
**Why this happens:** Bisync creates listing files that track the state of each side. When both directories are completely empty, these listing files are considered invalid by rclone.
|
||||
|
||||
**Best practice:** Always have at least one file (like a README.md) in your project directory before setting up sync.
|
||||
|
||||
### Bisync State Corruption
|
||||
|
||||
**Problem:** Bisync fails with errors about corrupted state or listing files
|
||||
|
||||
**Explanation:** Sometimes bisync state can become inconsistent (e.g., after mixing dry-run and actual runs, or after manual file operations).
|
||||
|
||||
**Solution:** Clear bisync state and re-establish baseline:
|
||||
|
||||
```bash
|
||||
# Clear bisync state
|
||||
bm project bisync-reset research
|
||||
|
||||
# Re-establish baseline
|
||||
bm project bisync --name research --resync
|
||||
```
|
||||
|
||||
**What this does:**
|
||||
- Removes all bisync metadata from `~/.basic-memory/bisync-state/research/`
|
||||
- Forces fresh baseline on next `--resync`
|
||||
- Safe operation (doesn't touch your files)
|
||||
|
||||
**Note:** This command also runs automatically when you remove a project to clean up state directories.
|
||||
|
||||
### Too Many Deletes
|
||||
|
||||
**Problem:** "Error: max delete limit (25) exceeded"
|
||||
|
||||
**Explanation:** Bisync detected you're about to delete more than 25 files. This is a safety check to prevent accidents.
|
||||
|
||||
**Solution 1:** Review what you're deleting, then force resync:
|
||||
|
||||
```bash
|
||||
# Check what would be deleted
|
||||
bm project bisync --name research --dry-run
|
||||
|
||||
# If correct, establish new baseline
|
||||
bm project bisync --name research --resync
|
||||
```
|
||||
|
||||
**Solution 2:** Use one-way sync if you know local is correct:
|
||||
|
||||
```bash
|
||||
bm project sync --name research
|
||||
```
|
||||
|
||||
### Project Not Configured for Sync
|
||||
|
||||
**Problem:** "Project research has no local_sync_path configured"
|
||||
|
||||
**Explanation:** Project exists on cloud but has no local sync path.
|
||||
|
||||
**Solution:**
|
||||
|
||||
```bash
|
||||
bm project sync-setup research ~/Documents/research
|
||||
bm project bisync --name research --resync
|
||||
```
|
||||
|
||||
### Connection Issues
|
||||
|
||||
**Problem:** "Cannot connect to cloud instance"
|
||||
|
||||
**Solution:** Check status:
|
||||
|
||||
```bash
|
||||
bm cloud status
|
||||
```
|
||||
|
||||
If instance is down, wait a few minutes and retry.
|
||||
|
||||
## Security
|
||||
|
||||
- **Authentication**: OAuth 2.1 with PKCE flow
|
||||
- **Tokens**: Stored securely in `~/.basic-memory/basic-memory-cloud.json`
|
||||
- **Transport**: All data encrypted in transit (HTTPS)
|
||||
- **Credentials**: Scoped S3 credentials (read-write to your tenant only)
|
||||
- **Isolation**: Your data isolated from other tenants
|
||||
- **Ignore patterns**: Sensitive files automatically excluded via `.bmignore`
|
||||
|
||||
## Command Reference
|
||||
|
||||
### Cloud Mode Management
|
||||
|
||||
```bash
|
||||
bm cloud login # Authenticate and enable cloud mode
|
||||
bm cloud logout # Disable cloud mode
|
||||
bm cloud status # Check cloud mode and instance health
|
||||
```
|
||||
|
||||
### Setup
|
||||
|
||||
```bash
|
||||
bm cloud setup # Install rclone and configure credentials
|
||||
```
|
||||
|
||||
### Project Management
|
||||
|
||||
When cloud mode is enabled:
|
||||
|
||||
```bash
|
||||
bm project list # List cloud projects
|
||||
bm project add <name> # Create cloud project (no sync)
|
||||
bm project add <name> --local-path <path> # Create with local sync
|
||||
bm project sync-setup <name> <path> # Add sync to existing project
|
||||
bm project rm <name> # Delete project
|
||||
```
|
||||
|
||||
### File Synchronization
|
||||
|
||||
```bash
|
||||
# One-way sync (local → cloud)
|
||||
bm project sync --name <project>
|
||||
bm project sync --name <project> --dry-run
|
||||
bm project sync --name <project> --verbose
|
||||
|
||||
# Two-way sync (local ↔ cloud) - Recommended
|
||||
bm project bisync --name <project> # After first --resync
|
||||
bm project bisync --name <project> --resync # First time / force baseline
|
||||
bm project bisync --name <project> --dry-run
|
||||
bm project bisync --name <project> --verbose
|
||||
|
||||
# Integrity check
|
||||
bm project check --name <project>
|
||||
bm project check --name <project> --one-way
|
||||
|
||||
# List remote files
|
||||
bm project ls --name <project>
|
||||
bm project ls --name <project> --path <subpath>
|
||||
```
|
||||
|
||||
## Summary
|
||||
|
||||
**Basic Memory Cloud uses project-scoped sync:**
|
||||
|
||||
1. **Enable cloud mode** - `bm cloud login`
|
||||
2. **Install rclone** - `bm cloud setup`
|
||||
3. **Add projects with sync** - `bm project add research --local-path ~/Documents/research`
|
||||
4. **Preview first sync** - `bm project bisync --name research --resync --dry-run`
|
||||
5. **Establish baseline** - `bm project bisync --name research --resync`
|
||||
6. **Daily workflow** - `bm project bisync --name research`
|
||||
|
||||
**Key benefits:**
|
||||
- ✅ Each project independently syncs (or doesn't)
|
||||
- ✅ Projects can live anywhere on disk
|
||||
- ✅ Explicit sync operations (no magic)
|
||||
- ✅ Safe by design (max delete limits, conflict resolution)
|
||||
- ✅ Full offline access (work locally, sync when ready)
|
||||
|
||||
**Future enhancements:**
|
||||
- `--all` flag to sync all configured projects
|
||||
- Project list showing sync status
|
||||
- Watch mode for automatic sync
|
||||
@@ -1,28 +0,0 @@
|
||||
## Coverage policy (practical 100%)
|
||||
|
||||
Basic Memory’s test suite intentionally mixes:
|
||||
- unit tests (fast, deterministic)
|
||||
- integration tests (real filesystem + real DB via `test-int/`)
|
||||
|
||||
To keep the default CI signal **stable and meaningful**, the default `pytest` coverage report targets **core library logic** and **excludes** a small set of modules that are either:
|
||||
- highly environment-dependent (OS/DB tuning)
|
||||
- inherently interactive (CLI)
|
||||
- background-task orchestration (watchers/sync runners)
|
||||
|
||||
### What's excluded (and why)
|
||||
|
||||
Coverage excludes are configured in `pyproject.toml` under `[tool.coverage.report].omit`.
|
||||
|
||||
Current exclusions include:
|
||||
- `src/basic_memory/cli/**`: interactive wrappers; behavior is validated via higher-level tests and smoke tests.
|
||||
- `src/basic_memory/db.py`: platform/backend tuning paths (SQLite/Postgres/Windows), covered by integration tests and targeted runs.
|
||||
- `src/basic_memory/services/initialization.py`: startup orchestration/background tasks; covered indirectly by app/MCP entrypoints.
|
||||
- `src/basic_memory/sync/sync_service.py`: heavy filesystem↔DB integration; validated in integration suite (not enforced in unit coverage).
|
||||
|
||||
### Recommended additional runs
|
||||
|
||||
If you want extra confidence locally/CI:
|
||||
- **Postgres backend**: run tests with `BASIC_MEMORY_TEST_POSTGRES=1`.
|
||||
- **Strict backend-complete coverage**: run coverage on SQLite + Postgres and combine the results (recommended).
|
||||
|
||||
|
||||
@@ -1,336 +0,0 @@
|
||||
# Basic Memory - Modern Command Runner
|
||||
|
||||
# Install dependencies
|
||||
install:
|
||||
uv pip install -e ".[dev]"
|
||||
uv sync
|
||||
@echo ""
|
||||
@echo "💡 Remember to activate the virtual environment by running: source .venv/bin/activate"
|
||||
|
||||
# ==============================================================================
|
||||
# DATABASE BACKEND TESTING
|
||||
# ==============================================================================
|
||||
# Basic Memory supports dual database backends (SQLite and Postgres).
|
||||
# By default, tests run against SQLite (fast, no dependencies).
|
||||
# Set BASIC_MEMORY_TEST_POSTGRES=1 to run against Postgres (uses testcontainers).
|
||||
#
|
||||
# Quick Start:
|
||||
# just test # Run all tests against SQLite (default)
|
||||
# just test-sqlite # Run all tests against SQLite
|
||||
# just test-postgres # Run all tests against Postgres (testcontainers)
|
||||
# just test-unit-sqlite # Run unit tests against SQLite
|
||||
# just test-unit-postgres # Run unit tests against Postgres
|
||||
# just test-int-sqlite # Run integration tests against SQLite
|
||||
# just test-int-postgres # Run integration tests against Postgres
|
||||
#
|
||||
# CI runs both in parallel for faster feedback.
|
||||
# ==============================================================================
|
||||
|
||||
# Run all tests against SQLite and Postgres
|
||||
test: test-sqlite test-postgres
|
||||
|
||||
# Run all tests against SQLite
|
||||
test-sqlite: test-unit-sqlite test-int-sqlite
|
||||
|
||||
# Run all tests against Postgres (uses testcontainers)
|
||||
test-postgres: test-unit-postgres test-int-postgres
|
||||
|
||||
# Run unit tests against SQLite
|
||||
test-unit-sqlite:
|
||||
BASIC_MEMORY_ENV=test uv run pytest -p pytest_mock -v --no-cov tests
|
||||
|
||||
# Run unit tests against Postgres
|
||||
test-unit-postgres:
|
||||
BASIC_MEMORY_ENV=test BASIC_MEMORY_TEST_POSTGRES=1 uv run pytest -p pytest_mock -v --no-cov tests
|
||||
|
||||
# Run integration tests against SQLite
|
||||
test-int-sqlite:
|
||||
uv run pytest -p pytest_mock -v --no-cov test-int
|
||||
|
||||
# Run integration tests against Postgres
|
||||
# Note: Uses timeout due to FastMCP Client + asyncpg cleanup hang (tests pass, process hangs on exit)
|
||||
# See: https://github.com/jlowin/fastmcp/issues/1311
|
||||
test-int-postgres:
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
# Use gtimeout (macOS/Homebrew) or timeout (Linux)
|
||||
TIMEOUT_CMD=$(command -v gtimeout || command -v timeout || echo "")
|
||||
if [[ -n "$TIMEOUT_CMD" ]]; then
|
||||
$TIMEOUT_CMD --signal=KILL 600 bash -c 'BASIC_MEMORY_TEST_POSTGRES=1 uv run pytest -p pytest_mock -v --no-cov test-int' || test $? -eq 137
|
||||
else
|
||||
echo "⚠️ No timeout command found, running without timeout..."
|
||||
BASIC_MEMORY_TEST_POSTGRES=1 uv run pytest -p pytest_mock -v --no-cov test-int
|
||||
fi
|
||||
|
||||
# Run tests impacted by recent changes (requires pytest-testmon)
|
||||
testmon *args:
|
||||
BASIC_MEMORY_ENV=test uv run pytest -p pytest_mock -v --no-cov --testmon --testmon-forceselect {{args}}
|
||||
|
||||
# Run MCP smoke test (fast end-to-end loop)
|
||||
test-smoke:
|
||||
BASIC_MEMORY_ENV=test uv run pytest -p pytest_mock -v --no-cov -m smoke test-int/mcp/test_smoke_integration.py
|
||||
|
||||
# Fast local loop: lint, format, typecheck, impacted tests
|
||||
fast-check:
|
||||
just fix
|
||||
just format
|
||||
just typecheck
|
||||
just testmon
|
||||
just test-smoke
|
||||
|
||||
# Reset Postgres test database (drops and recreates schema)
|
||||
# Useful when Alembic migration state gets out of sync during development
|
||||
# Uses credentials from docker-compose-postgres.yml
|
||||
postgres-reset:
|
||||
docker exec basic-memory-postgres psql -U ${POSTGRES_USER:-basic_memory_user} -d ${POSTGRES_TEST_DB:-basic_memory_test} -c "DROP SCHEMA public CASCADE; CREATE SCHEMA public;"
|
||||
@echo "✅ Postgres test database reset"
|
||||
|
||||
# Run Alembic migrations manually against Postgres test database
|
||||
# Useful for debugging migration issues
|
||||
# Uses credentials from docker-compose-postgres.yml (can override with env vars)
|
||||
postgres-migrate:
|
||||
@cd src/basic_memory/alembic && \
|
||||
BASIC_MEMORY_DATABASE_BACKEND=postgres \
|
||||
BASIC_MEMORY_DATABASE_URL=${POSTGRES_TEST_URL:-postgresql+asyncpg://basic_memory_user:dev_password@localhost:5433/basic_memory_test} \
|
||||
uv run alembic upgrade head
|
||||
@echo "✅ Migrations applied to Postgres test database"
|
||||
|
||||
# Run Windows-specific tests only (only works on Windows platform)
|
||||
# These tests verify Windows-specific database optimizations (locking mode, NullPool)
|
||||
# Will be skipped automatically on non-Windows platforms
|
||||
test-windows:
|
||||
uv run pytest -p pytest_mock -v --no-cov -m windows tests test-int
|
||||
|
||||
# Run benchmark tests only (performance testing)
|
||||
# These are slow tests that measure sync performance with various file counts
|
||||
# Excluded from default test runs to keep CI fast
|
||||
test-benchmark:
|
||||
uv run pytest -p pytest_mock -v --no-cov -m benchmark tests test-int
|
||||
|
||||
# Run all tests including Windows, Postgres, and Benchmarks (for CI/comprehensive testing)
|
||||
# Use this before releasing to ensure everything works across all backends and platforms
|
||||
test-all:
|
||||
uv run pytest -p pytest_mock -v --no-cov tests test-int
|
||||
|
||||
# Generate HTML coverage report
|
||||
coverage:
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
uv run coverage erase
|
||||
|
||||
echo "🔎 Coverage (SQLite)..."
|
||||
BASIC_MEMORY_ENV=test uv run coverage run --source=basic_memory -m pytest -p pytest_mock -v --no-cov tests test-int
|
||||
|
||||
echo "🔎 Coverage (Postgres via testcontainers)..."
|
||||
# Note: Uses timeout due to FastMCP Client + asyncpg cleanup hang (tests pass, process hangs on exit)
|
||||
# See: https://github.com/jlowin/fastmcp/issues/1311
|
||||
TIMEOUT_CMD=$(command -v gtimeout || command -v timeout || echo "")
|
||||
if [[ -n "$TIMEOUT_CMD" ]]; then
|
||||
$TIMEOUT_CMD --signal=KILL 600 bash -c 'BASIC_MEMORY_ENV=test BASIC_MEMORY_TEST_POSTGRES=1 uv run coverage run --source=basic_memory -m pytest -p pytest_mock -v --no-cov -m postgres tests test-int' || test $? -eq 137
|
||||
else
|
||||
echo "⚠️ No timeout command found, running without timeout..."
|
||||
BASIC_MEMORY_ENV=test BASIC_MEMORY_TEST_POSTGRES=1 uv run coverage run --source=basic_memory -m pytest -p pytest_mock -v --no-cov -m postgres tests test-int
|
||||
fi
|
||||
|
||||
echo "🧩 Combining coverage data..."
|
||||
uv run coverage combine
|
||||
uv run coverage report -m
|
||||
uv run coverage html
|
||||
echo "Coverage report generated in htmlcov/index.html"
|
||||
|
||||
# Lint and fix code (calls fix)
|
||||
lint: fix
|
||||
|
||||
# Lint and fix code
|
||||
fix:
|
||||
uv run ruff check --fix --unsafe-fixes src tests test-int
|
||||
|
||||
# Type check code
|
||||
typecheck:
|
||||
uv run pyright
|
||||
|
||||
# Clean build artifacts and cache files
|
||||
clean:
|
||||
find . -type f -name '*.pyc' -delete
|
||||
find . -type d -name '__pycache__' -exec rm -r {} +
|
||||
rm -rf installer/build/ installer/dist/ dist/
|
||||
rm -f rw.*.dmg .coverage.*
|
||||
|
||||
# Format code with ruff
|
||||
format:
|
||||
uv run ruff format .
|
||||
|
||||
# Run MCP inspector tool
|
||||
run-inspector:
|
||||
npx @modelcontextprotocol/inspector
|
||||
|
||||
# Run doctor checks in an isolated temp home/config
|
||||
doctor:
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
TMP_HOME=$(mktemp -d)
|
||||
TMP_CONFIG=$(mktemp -d)
|
||||
HOME="$TMP_HOME" \
|
||||
BASIC_MEMORY_ENV=test \
|
||||
BASIC_MEMORY_HOME="$TMP_HOME/basic-memory" \
|
||||
BASIC_MEMORY_CONFIG_DIR="$TMP_CONFIG" \
|
||||
./.venv/bin/python -m basic_memory.cli.main doctor --local
|
||||
|
||||
|
||||
# Update all dependencies to latest versions
|
||||
update-deps:
|
||||
uv sync --upgrade
|
||||
|
||||
# Run all code quality checks and tests
|
||||
check: lint format typecheck test
|
||||
|
||||
# Generate Alembic migration with descriptive message
|
||||
migration message:
|
||||
cd src/basic_memory/alembic && alembic revision --autogenerate -m "{{message}}"
|
||||
|
||||
# Create a stable release (e.g., just release v0.13.2)
|
||||
release version:
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
# Validate version format
|
||||
if [[ ! "{{version}}" =~ ^v[0-9]+\.[0-9]+\.[0-9]+$ ]]; then
|
||||
echo "❌ Invalid version format. Use: v0.13.2"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Extract version number without 'v' prefix
|
||||
VERSION_NUM=$(echo "{{version}}" | sed 's/^v//')
|
||||
|
||||
echo "🚀 Creating stable release {{version}}"
|
||||
|
||||
# Pre-flight checks
|
||||
echo "📋 Running pre-flight checks..."
|
||||
if [[ -n $(git status --porcelain) ]]; then
|
||||
echo "❌ Uncommitted changes found. Please commit or stash them first."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [[ $(git branch --show-current) != "main" ]]; then
|
||||
echo "❌ Not on main branch. Switch to main first."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Check if tag already exists
|
||||
if git tag -l "{{version}}" | grep -q "{{version}}"; then
|
||||
echo "❌ Tag {{version}} already exists"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Run quality checks
|
||||
echo "🔍 Running lint checks..."
|
||||
just lint
|
||||
just typecheck
|
||||
|
||||
# Update version in __init__.py
|
||||
echo "📝 Updating version in __init__.py..."
|
||||
sed -i.bak "s/__version__ = \".*\"/__version__ = \"$VERSION_NUM\"/" src/basic_memory/__init__.py
|
||||
rm -f src/basic_memory/__init__.py.bak
|
||||
|
||||
# Update version in server.json (MCP registry metadata)
|
||||
echo "📝 Updating version in server.json..."
|
||||
sed -i.bak "s/\"version\": \"[^\"]*\"/\"version\": \"$VERSION_NUM\"/g" server.json
|
||||
rm -f server.json.bak
|
||||
|
||||
# Commit version update
|
||||
git add src/basic_memory/__init__.py server.json
|
||||
git commit -m "chore: update version to $VERSION_NUM for {{version}} release"
|
||||
|
||||
# Create and push tag
|
||||
echo "🏷️ Creating tag {{version}}..."
|
||||
git tag "{{version}}"
|
||||
|
||||
echo "📤 Pushing to GitHub..."
|
||||
git push origin main
|
||||
git push origin "{{version}}"
|
||||
|
||||
echo "✅ Release {{version}} created successfully!"
|
||||
echo "📦 GitHub Actions will build and publish to PyPI"
|
||||
echo "🔗 Monitor at: https://github.com/basicmachines-co/basic-memory/actions"
|
||||
echo ""
|
||||
echo "📝 REMINDER: Post-release tasks:"
|
||||
echo " 1. docs.basicmemory.com - Add release notes to src/pages/latest-releases.mdx"
|
||||
echo " 2. basicmachines.co - Update version in src/components/sections/hero.tsx"
|
||||
echo " 3. MCP Registry - Run: mcp-publisher publish"
|
||||
echo " See: .claude/commands/release/release.md for detailed instructions"
|
||||
|
||||
# Create a beta release (e.g., just beta v0.13.2b1)
|
||||
beta version:
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
# Validate version format (allow beta/rc suffixes)
|
||||
if [[ ! "{{version}}" =~ ^v[0-9]+\.[0-9]+\.[0-9]+(b[0-9]+|rc[0-9]+)$ ]]; then
|
||||
echo "❌ Invalid beta version format. Use: v0.13.2b1 or v0.13.2rc1"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Extract version number without 'v' prefix
|
||||
VERSION_NUM=$(echo "{{version}}" | sed 's/^v//')
|
||||
|
||||
echo "🧪 Creating beta release {{version}}"
|
||||
|
||||
# Pre-flight checks
|
||||
echo "📋 Running pre-flight checks..."
|
||||
if [[ -n $(git status --porcelain) ]]; then
|
||||
echo "❌ Uncommitted changes found. Please commit or stash them first."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [[ $(git branch --show-current) != "main" ]]; then
|
||||
echo "❌ Not on main branch. Switch to main first."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Check if tag already exists
|
||||
if git tag -l "{{version}}" | grep -q "{{version}}"; then
|
||||
echo "❌ Tag {{version}} already exists"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Run quality checks
|
||||
echo "🔍 Running lint checks..."
|
||||
just lint
|
||||
just typecheck
|
||||
|
||||
# Update version in __init__.py
|
||||
echo "📝 Updating version in __init__.py..."
|
||||
sed -i.bak "s/__version__ = \".*\"/__version__ = \"$VERSION_NUM\"/" src/basic_memory/__init__.py
|
||||
rm -f src/basic_memory/__init__.py.bak
|
||||
|
||||
# Update version in server.json (MCP registry metadata)
|
||||
echo "📝 Updating version in server.json..."
|
||||
sed -i.bak "s/\"version\": \"[^\"]*\"/\"version\": \"$VERSION_NUM\"/g" server.json
|
||||
rm -f server.json.bak
|
||||
|
||||
# Commit version update
|
||||
git add src/basic_memory/__init__.py server.json
|
||||
git commit -m "chore: update version to $VERSION_NUM for {{version}} beta release"
|
||||
|
||||
# Create and push tag
|
||||
echo "🏷️ Creating tag {{version}}..."
|
||||
git tag "{{version}}"
|
||||
|
||||
echo "📤 Pushing to GitHub..."
|
||||
git push origin main
|
||||
git push origin "{{version}}"
|
||||
|
||||
echo "✅ Beta release {{version}} created successfully!"
|
||||
echo "📦 GitHub Actions will build and publish to PyPI as pre-release"
|
||||
echo "🔗 Monitor at: https://github.com/basicmachines-co/basic-memory/actions"
|
||||
echo "📥 Install with: uv tool install basic-memory --pre"
|
||||
echo ""
|
||||
echo "📝 REMINDER: For stable releases, update documentation sites:"
|
||||
echo " 1. docs.basicmemory.com - Add release notes to src/pages/latest-releases.mdx"
|
||||
echo " 2. basicmachines.co - Update version in src/components/sections/hero.tsx"
|
||||
echo " See: .claude/commands/release/release.md for detailed instructions"
|
||||
|
||||
# List all available recipes
|
||||
default:
|
||||
@just --list
|
||||
-128
@@ -1,128 +0,0 @@
|
||||
# Basic Memory Installation Guide for LLMs
|
||||
|
||||
This guide is specifically designed to help AI assistants like Cline install and configure Basic Memory. Follow these
|
||||
steps in order.
|
||||
|
||||
## Installation Steps
|
||||
|
||||
### 1. Install Basic Memory Package
|
||||
|
||||
Use one of the following package managers to install:
|
||||
|
||||
```bash
|
||||
# Install with uv (recommended)
|
||||
uv tool install basic-memory
|
||||
|
||||
# Or with pip
|
||||
pip install basic-memory
|
||||
```
|
||||
|
||||
### 2. Configure MCP Server
|
||||
|
||||
Add the following to your config:
|
||||
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"basic-memory": {
|
||||
"command": "uvx",
|
||||
"args": [
|
||||
"basic-memory",
|
||||
"mcp"
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
For Claude Desktop, this file is located at:
|
||||
|
||||
macOS: ~/Library/Application Support/Claude/claude_desktop_config.json
|
||||
Windows: %APPDATA%\Claude\claude_desktop_config.json
|
||||
|
||||
### 3. Start Synchronization (optional)
|
||||
|
||||
To synchronize files in real-time, run:
|
||||
|
||||
```bash
|
||||
basic-memory sync --watch
|
||||
```
|
||||
|
||||
Or for a one-time sync:
|
||||
|
||||
```bash
|
||||
basic-memory sync
|
||||
```
|
||||
|
||||
## Configuration Options
|
||||
|
||||
### Custom Directory
|
||||
|
||||
To use a directory other than the default `~/basic-memory`:
|
||||
|
||||
```bash
|
||||
basic-memory project add custom-project /path/to/your/directory
|
||||
basic-memory project default custom-project
|
||||
```
|
||||
|
||||
### Multiple Projects
|
||||
|
||||
To manage multiple knowledge bases:
|
||||
|
||||
```bash
|
||||
# List all projects
|
||||
basic-memory project list
|
||||
|
||||
# Add a new project
|
||||
basic-memory project add work ~/work-basic-memory
|
||||
|
||||
# Set default project
|
||||
basic-memory project default work
|
||||
```
|
||||
|
||||
## Importing Existing Data
|
||||
|
||||
### From Claude.ai
|
||||
|
||||
```bash
|
||||
basic-memory import claude conversations path/to/conversations.json
|
||||
basic-memory import claude projects path/to/projects.json
|
||||
```
|
||||
|
||||
### From ChatGPT
|
||||
|
||||
```bash
|
||||
basic-memory import chatgpt path/to/conversations.json
|
||||
```
|
||||
|
||||
### From MCP Memory Server
|
||||
|
||||
```bash
|
||||
basic-memory import memory-json path/to/memory.json
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
If you encounter issues:
|
||||
|
||||
1. Check that Basic Memory is properly installed:
|
||||
```bash
|
||||
basic-memory --version
|
||||
```
|
||||
|
||||
2. Verify the sync process is running:
|
||||
```bash
|
||||
ps aux | grep basic-memory
|
||||
```
|
||||
|
||||
3. Check sync output for errors:
|
||||
```bash
|
||||
basic-memory sync --verbose
|
||||
```
|
||||
|
||||
4. Check log output:
|
||||
```bash
|
||||
cat ~/.basic-memory/basic-memory.log
|
||||
```
|
||||
|
||||
For more detailed information, refer to the [full documentation](https://memory.basicmachines.co/).
|
||||
-148
@@ -1,148 +0,0 @@
|
||||
[project]
|
||||
name = "basic-memory"
|
||||
dynamic = ["version"]
|
||||
description = "Local-first knowledge management combining Zettelkasten with knowledge graphs"
|
||||
readme = "README.md"
|
||||
requires-python = ">=3.12"
|
||||
license = { text = "AGPL-3.0-or-later" }
|
||||
authors = [
|
||||
{ name = "Basic Machines", email = "hello@basic-machines.co" }
|
||||
]
|
||||
dependencies = [
|
||||
"sqlalchemy>=2.0.0",
|
||||
"pyyaml>=6.0.1",
|
||||
"typer>=0.9.0",
|
||||
"aiosqlite>=0.20.0",
|
||||
"greenlet>=3.1.1",
|
||||
"pydantic[email,timezone]>=2.12.0",
|
||||
"mcp>=1.23.1",
|
||||
"pydantic-settings>=2.6.1",
|
||||
"loguru>=0.7.3",
|
||||
"pyright>=1.1.390",
|
||||
"markdown-it-py>=3.0.0",
|
||||
"python-frontmatter>=1.1.0",
|
||||
"rich>=13.9.4",
|
||||
"unidecode>=1.3.8",
|
||||
"dateparser>=1.2.0",
|
||||
"watchfiles>=1.0.4",
|
||||
"fastapi[standard]>=0.115.8",
|
||||
"alembic>=1.14.1",
|
||||
"pillow>=11.1.0",
|
||||
"pybars3>=0.9.7",
|
||||
"fastmcp==2.12.3", # Pinned - 2.14.x breaks MCP tools visibility (issue #463)
|
||||
"pyjwt>=2.10.1",
|
||||
"python-dotenv>=1.1.0",
|
||||
"pytest-aio>=1.9.0",
|
||||
"aiofiles>=24.1.0", # Optional observability (disabled by default via config)
|
||||
"asyncpg>=0.30.0",
|
||||
"nest-asyncio>=1.6.0", # For Alembic migrations with Postgres
|
||||
"pytest-asyncio>=1.2.0",
|
||||
"psycopg==3.3.1",
|
||||
"mdformat>=0.7.22",
|
||||
"mdformat-gfm>=0.3.7",
|
||||
"mdformat-frontmatter>=2.0.8",
|
||||
"sniffio>=1.3.1",
|
||||
"anyio>=4.10.0",
|
||||
"httpx>=0.28.0",
|
||||
]
|
||||
|
||||
|
||||
[project.urls]
|
||||
Homepage = "https://github.com/basicmachines-co/basic-memory"
|
||||
Repository = "https://github.com/basicmachines-co/basic-memory"
|
||||
Documentation = "https://github.com/basicmachines-co/basic-memory#readme"
|
||||
|
||||
[project.scripts]
|
||||
basic-memory = "basic_memory.cli.main:app"
|
||||
bm = "basic_memory.cli.main:app"
|
||||
|
||||
[build-system]
|
||||
requires = ["hatchling", "uv-dynamic-versioning>=0.7.0"]
|
||||
build-backend = "hatchling.build"
|
||||
|
||||
[tool.pytest.ini_options]
|
||||
pythonpath = ["src", "tests"]
|
||||
addopts = "--cov=basic_memory --cov-report term-missing"
|
||||
testpaths = ["tests", "test-int"]
|
||||
asyncio_mode = "strict"
|
||||
asyncio_default_fixture_loop_scope = "function"
|
||||
markers = [
|
||||
"benchmark: Performance benchmark tests (deselect with '-m \"not benchmark\"')",
|
||||
"slow: Slow-running tests (deselect with '-m \"not slow\"')",
|
||||
"postgres: Tests that run against Postgres backend (deselect with '-m \"not postgres\"')",
|
||||
"windows: Windows-specific tests (deselect with '-m \"not windows\"')",
|
||||
"smoke: Fast end-to-end smoke tests for MCP flows",
|
||||
]
|
||||
|
||||
[tool.ruff]
|
||||
line-length = 100
|
||||
target-version = "py312"
|
||||
|
||||
[dependency-groups]
|
||||
dev = [
|
||||
"gevent>=24.11.1",
|
||||
"icecream>=2.1.3",
|
||||
"pytest>=8.3.4",
|
||||
"pytest-cov>=4.1.0",
|
||||
"pytest-mock>=3.12.0",
|
||||
"pytest-asyncio>=0.24.0",
|
||||
"pytest-xdist>=3.0.0",
|
||||
"ruff>=0.1.6",
|
||||
"freezegun>=1.5.5",
|
||||
"testcontainers[postgres]>=4.0.0",
|
||||
"psycopg>=3.2.0",
|
||||
"pyright>=1.1.408",
|
||||
"pytest-testmon>=2.2.0",
|
||||
]
|
||||
|
||||
[tool.hatch.version]
|
||||
source = "uv-dynamic-versioning"
|
||||
|
||||
[tool.uv-dynamic-versioning]
|
||||
vcs = "git"
|
||||
style = "pep440"
|
||||
bump = true
|
||||
fallback-version = "0.0.0"
|
||||
|
||||
[tool.pyright]
|
||||
include = ["src/"]
|
||||
exclude = ["**/__pycache__"]
|
||||
ignore = ["test/"]
|
||||
defineConstant = { DEBUG = true }
|
||||
reportMissingImports = "error"
|
||||
reportMissingTypeStubs = false
|
||||
pythonVersion = "3.12"
|
||||
|
||||
|
||||
|
||||
[tool.coverage.run]
|
||||
concurrency = ["thread", "gevent"]
|
||||
parallel = true
|
||||
source = ["basic_memory"]
|
||||
|
||||
[tool.coverage.report]
|
||||
exclude_lines = [
|
||||
"pragma: no cover",
|
||||
"def __repr__",
|
||||
"if self.debug:",
|
||||
"if settings.DEBUG",
|
||||
"raise AssertionError",
|
||||
"raise NotImplementedError",
|
||||
"if 0:",
|
||||
"if __name__ == .__main__.:",
|
||||
"class .*\\bProtocol\\):",
|
||||
"@(abc\\.)?abstractmethod",
|
||||
]
|
||||
|
||||
# Exclude specific modules that are difficult to test comprehensively
|
||||
omit = [
|
||||
"*/external_auth_provider.py", # External HTTP calls to OAuth providers
|
||||
"*/supabase_auth_provider.py", # External HTTP calls to Supabase APIs
|
||||
"*/watch_service.py", # File system watching - complex integration testing
|
||||
"*/background_sync.py", # Background processes
|
||||
"*/cli/**", # CLI is an interactive wrapper; core logic is covered via API/MCP/service tests
|
||||
"*/db.py", # Backend/runtime-dependent (sqlite/postgres/windows tuning); validated via integration tests
|
||||
"*/services/initialization.py", # Startup orchestration + background tasks (watchers); exercised indirectly in entrypoints
|
||||
"*/sync/sync_service.py", # Heavy filesystem/db integration; covered by integration suite, not enforced in unit coverage
|
||||
"*/services/migration_service.py", # Complex migration scenarios
|
||||
]
|
||||
-25
@@ -1,25 +0,0 @@
|
||||
{
|
||||
"$schema": "https://static.modelcontextprotocol.io/schemas/2025-12-11/server.schema.json",
|
||||
"name": "io.github.basicmachines-co/basic-memory",
|
||||
"description": "Local-first knowledge management with bi-directional LLM sync via Markdown files.",
|
||||
"repository": {
|
||||
"url": "https://github.com/basicmachines-co/basic-memory.git",
|
||||
"source": "github"
|
||||
},
|
||||
"version": "0.18.1",
|
||||
"packages": [
|
||||
{
|
||||
"registryType": "pypi",
|
||||
"identifier": "basic-memory",
|
||||
"version": "0.18.1",
|
||||
"runtimeHint": "uvx",
|
||||
"runtimeArguments": [
|
||||
{"type": "positional", "value": "basic-memory"},
|
||||
{"type": "positional", "value": "mcp"}
|
||||
],
|
||||
"transport": {
|
||||
"type": "stdio"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -1,15 +0,0 @@
|
||||
# Smithery configuration file: https://smithery.ai/docs/config#smitheryyaml
|
||||
|
||||
startCommand:
|
||||
type: stdio
|
||||
configSchema:
|
||||
# JSON Schema defining the configuration options for the MCP.
|
||||
type: object
|
||||
properties: {}
|
||||
description: No configuration required. This MCP server runs using the default command.
|
||||
commandFunction: |-
|
||||
(config) => ({
|
||||
command: 'basic-memory',
|
||||
args: ['mcp']
|
||||
})
|
||||
exampleConfig: {}
|
||||
@@ -1,7 +0,0 @@
|
||||
"""basic-memory - Local-first knowledge management combining Zettelkasten with knowledge graphs"""
|
||||
|
||||
# Package version - updated by release automation
|
||||
__version__ = "0.18.1"
|
||||
|
||||
# API version for FastAPI - independent of package version
|
||||
__api_version__ = "v0"
|
||||
@@ -1,119 +0,0 @@
|
||||
# A generic, single database configuration.
|
||||
|
||||
[alembic]
|
||||
# path to migration scripts
|
||||
# Use forward slashes (/) also on windows to provide an os agnostic path
|
||||
script_location = .
|
||||
|
||||
# template used to generate migration file names; The default value is %%(rev)s_%%(slug)s
|
||||
# Uncomment the line below if you want the files to be prepended with date and time
|
||||
# see https://alembic.sqlalchemy.org/en/latest/tutorial.html#editing-the-ini-file
|
||||
# for all available tokens
|
||||
# file_template = %%(year)d_%%(month).2d_%%(day).2d_%%(hour).2d%%(minute).2d-%%(rev)s_%%(slug)s
|
||||
|
||||
# sys.path path, will be prepended to sys.path if present.
|
||||
# defaults to the current working directory.
|
||||
prepend_sys_path = .
|
||||
|
||||
# timezone to use when rendering the date within the migration file
|
||||
# as well as the filename.
|
||||
# If specified, requires the python>=3.9 or backports.zoneinfo library and tzdata library.
|
||||
# Any required deps can installed by adding `alembic[tz]` to the pip requirements
|
||||
# string value is passed to ZoneInfo()
|
||||
# leave blank for localtime
|
||||
# timezone =
|
||||
|
||||
# max length of characters to apply to the "slug" field
|
||||
# truncate_slug_length = 40
|
||||
|
||||
# set to 'true' to run the environment during
|
||||
# the 'revision' command, regardless of autogenerate
|
||||
# revision_environment = false
|
||||
|
||||
# set to 'true' to allow .pyc and .pyo files without
|
||||
# a source .py file to be detected as revisions in the
|
||||
# versions/ directory
|
||||
# sourceless = false
|
||||
|
||||
# version location specification; This defaults
|
||||
# to migrations/versions. When using multiple version
|
||||
# directories, initial revisions must be specified with --version-path.
|
||||
# The path separator used here should be the separator specified by "version_path_separator" below.
|
||||
# version_locations = %(here)s/bar:%(here)s/bat:migrations/versions
|
||||
|
||||
# version path separator; As mentioned above, this is the character used to split
|
||||
# version_locations. The default within new alembic.ini files is "os", which uses os.pathsep.
|
||||
# If this key is omitted entirely, it falls back to the legacy behavior of splitting on spaces and/or commas.
|
||||
# Valid values for version_path_separator are:
|
||||
#
|
||||
# version_path_separator = :
|
||||
# version_path_separator = ;
|
||||
# version_path_separator = space
|
||||
# version_path_separator = newline
|
||||
#
|
||||
# Use os.pathsep. Default configuration used for new projects.
|
||||
version_path_separator = os
|
||||
|
||||
# set to 'true' to search source files recursively
|
||||
# in each "version_locations" directory
|
||||
# new in Alembic version 1.10
|
||||
# recursive_version_locations = false
|
||||
|
||||
# the output encoding used when revision files
|
||||
# are written from script.py.mako
|
||||
# output_encoding = utf-8
|
||||
|
||||
sqlalchemy.url = driver://user:pass@localhost/dbname
|
||||
|
||||
|
||||
[post_write_hooks]
|
||||
# post_write_hooks defines scripts or Python functions that are run
|
||||
# on newly generated revision scripts. See the documentation for further
|
||||
# detail and examples
|
||||
|
||||
# format using "black" - use the console_scripts runner, against the "black" entrypoint
|
||||
# hooks = black
|
||||
# black.type = console_scripts
|
||||
# black.entrypoint = black
|
||||
# black.options = -l 79 REVISION_SCRIPT_FILENAME
|
||||
|
||||
# lint with attempts to fix using "ruff" - use the exec runner, execute a binary
|
||||
# hooks = ruff
|
||||
# ruff.type = exec
|
||||
# ruff.executable = %(here)s/.venv/bin/ruff
|
||||
# ruff.options = --fix REVISION_SCRIPT_FILENAME
|
||||
|
||||
# Logging configuration
|
||||
[loggers]
|
||||
keys = root,sqlalchemy,alembic
|
||||
|
||||
[handlers]
|
||||
keys = console
|
||||
|
||||
[formatters]
|
||||
keys = generic
|
||||
|
||||
[logger_root]
|
||||
level = WARNING
|
||||
handlers = console
|
||||
qualname =
|
||||
|
||||
[logger_sqlalchemy]
|
||||
level = WARNING
|
||||
handlers =
|
||||
qualname = sqlalchemy.engine
|
||||
|
||||
[logger_alembic]
|
||||
level = INFO
|
||||
handlers =
|
||||
qualname = alembic
|
||||
|
||||
[handler_console]
|
||||
class = StreamHandler
|
||||
args = (sys.stderr,)
|
||||
level = NOTSET
|
||||
formatter = generic
|
||||
|
||||
[formatter_generic]
|
||||
format = %(levelname)-5.5s [%(name)s] %(message)s
|
||||
datefmt = %H:%M:%S
|
||||
@@ -1,189 +0,0 @@
|
||||
"""Alembic environment configuration."""
|
||||
|
||||
import asyncio
|
||||
import os
|
||||
from logging.config import fileConfig
|
||||
|
||||
# Allow nested event loops (needed for pytest-asyncio and other async contexts)
|
||||
# Note: nest_asyncio doesn't work with uvloop or Python 3.14+, so we handle those cases separately
|
||||
import sys
|
||||
|
||||
if sys.version_info < (3, 14):
|
||||
try:
|
||||
import nest_asyncio
|
||||
|
||||
nest_asyncio.apply()
|
||||
except (ImportError, ValueError):
|
||||
# nest_asyncio not available or can't patch this loop type (e.g., uvloop)
|
||||
pass
|
||||
# For Python 3.14+, we rely on the thread-based fallback in run_migrations_online()
|
||||
|
||||
from sqlalchemy import engine_from_config, pool
|
||||
from sqlalchemy.ext.asyncio import AsyncEngine, create_async_engine
|
||||
|
||||
from alembic import context
|
||||
|
||||
from basic_memory.config import ConfigManager
|
||||
|
||||
# Trigger: only set test env when actually running under pytest
|
||||
# Why: alembic/env.py is imported during normal operations (MCP server startup, migrations)
|
||||
# but we only want test behavior during actual test runs
|
||||
# Outcome: prevents is_test_env from returning True in production, enabling watch service
|
||||
if os.getenv("PYTEST_CURRENT_TEST") is not None:
|
||||
os.environ["BASIC_MEMORY_ENV"] = "test"
|
||||
|
||||
# Import after setting environment variable # noqa: E402
|
||||
from basic_memory.models import Base # noqa: E402
|
||||
|
||||
# this is the Alembic Config object, which provides
|
||||
# access to the values within the .ini file in use.
|
||||
config = context.config
|
||||
|
||||
# Load app config - this will read environment variables (BASIC_MEMORY_DATABASE_BACKEND, etc.)
|
||||
# due to Pydantic's env_prefix="BASIC_MEMORY_" setting
|
||||
app_config = ConfigManager().config
|
||||
|
||||
# Set the SQLAlchemy URL based on database backend configuration
|
||||
# If the URL is already set in config (e.g., from run_migrations), use that
|
||||
# Otherwise, get it from app config
|
||||
# Note: alembic.ini has a placeholder URL "driver://user:pass@localhost/dbname" that we need to override
|
||||
current_url = config.get_main_option("sqlalchemy.url")
|
||||
if not current_url or current_url == "driver://user:pass@localhost/dbname":
|
||||
from basic_memory.db import DatabaseType
|
||||
|
||||
sqlalchemy_url = DatabaseType.get_db_url(
|
||||
app_config.database_path, DatabaseType.FILESYSTEM, app_config
|
||||
)
|
||||
config.set_main_option("sqlalchemy.url", sqlalchemy_url)
|
||||
|
||||
# Interpret the config file for Python logging.
|
||||
if config.config_file_name is not None:
|
||||
fileConfig(config.config_file_name)
|
||||
|
||||
# add your model's MetaData object here
|
||||
# for 'autogenerate' support
|
||||
target_metadata = Base.metadata
|
||||
|
||||
|
||||
# Add this function to tell Alembic what to include/exclude
|
||||
def include_object(object, name, type_, reflected, compare_to):
|
||||
# Ignore SQLite FTS tables
|
||||
if type_ == "table" and name.startswith("search_index"):
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
def run_migrations_offline() -> None:
|
||||
"""Run migrations in 'offline' mode.
|
||||
|
||||
This configures the context with just a URL
|
||||
and not an Engine, though an Engine is acceptable
|
||||
here as well. By skipping the Engine creation
|
||||
we don't even need a DBAPI to be available.
|
||||
|
||||
Calls to context.execute() here emit the given string to the
|
||||
script output.
|
||||
"""
|
||||
url = config.get_main_option("sqlalchemy.url")
|
||||
context.configure(
|
||||
url=url,
|
||||
target_metadata=target_metadata,
|
||||
literal_binds=True,
|
||||
dialect_opts={"paramstyle": "named"},
|
||||
include_object=include_object,
|
||||
render_as_batch=True,
|
||||
)
|
||||
|
||||
with context.begin_transaction():
|
||||
context.run_migrations()
|
||||
|
||||
|
||||
def do_run_migrations(connection):
|
||||
"""Execute migrations with the given connection."""
|
||||
context.configure(
|
||||
connection=connection,
|
||||
target_metadata=target_metadata,
|
||||
include_object=include_object,
|
||||
render_as_batch=True,
|
||||
compare_type=True,
|
||||
)
|
||||
with context.begin_transaction():
|
||||
context.run_migrations()
|
||||
|
||||
|
||||
async def run_async_migrations(connectable):
|
||||
"""Run migrations asynchronously with AsyncEngine."""
|
||||
async with connectable.connect() as connection:
|
||||
await connection.run_sync(do_run_migrations)
|
||||
await connectable.dispose()
|
||||
|
||||
|
||||
def run_migrations_online() -> None:
|
||||
"""Run migrations in 'online' mode.
|
||||
|
||||
Supports both sync engines (SQLite) and async engines (PostgreSQL with asyncpg).
|
||||
"""
|
||||
# Check if a connection/engine was provided (e.g., from run_migrations)
|
||||
connectable = context.config.attributes.get("connection", None)
|
||||
|
||||
if connectable is None:
|
||||
# No connection provided, create engine from config
|
||||
url = context.config.get_main_option("sqlalchemy.url")
|
||||
|
||||
# Check if it's an async URL (sqlite+aiosqlite or postgresql+asyncpg)
|
||||
if url and ("+asyncpg" in url or "+aiosqlite" in url):
|
||||
# Create async engine for asyncpg or aiosqlite
|
||||
connectable = create_async_engine(
|
||||
url,
|
||||
poolclass=pool.NullPool,
|
||||
future=True,
|
||||
)
|
||||
else:
|
||||
# Create sync engine for regular sqlite or postgresql
|
||||
connectable = engine_from_config(
|
||||
context.config.get_section(context.config.config_ini_section, {}),
|
||||
prefix="sqlalchemy.",
|
||||
poolclass=pool.NullPool,
|
||||
)
|
||||
|
||||
# Handle async engines (PostgreSQL with asyncpg)
|
||||
if isinstance(connectable, AsyncEngine):
|
||||
# Try to run async migrations
|
||||
# nest_asyncio allows asyncio.run() from within event loops, but doesn't work with uvloop
|
||||
try:
|
||||
asyncio.run(run_async_migrations(connectable))
|
||||
except RuntimeError as e:
|
||||
if "cannot be called from a running event loop" in str(e):
|
||||
# We're in a running event loop (likely uvloop) - need to use a different approach
|
||||
# Create a new thread to run the async migrations
|
||||
import concurrent.futures
|
||||
|
||||
def run_in_thread():
|
||||
"""Run async migrations in a new event loop in a separate thread."""
|
||||
new_loop = asyncio.new_event_loop()
|
||||
asyncio.set_event_loop(new_loop)
|
||||
try:
|
||||
new_loop.run_until_complete(run_async_migrations(connectable))
|
||||
finally:
|
||||
new_loop.close()
|
||||
|
||||
with concurrent.futures.ThreadPoolExecutor() as executor:
|
||||
future = executor.submit(run_in_thread)
|
||||
future.result() # Wait for completion and re-raise any exceptions
|
||||
else:
|
||||
raise
|
||||
else:
|
||||
# Handle sync engines (SQLite) or sync connections
|
||||
if hasattr(connectable, "connect"):
|
||||
# It's an engine, get a connection
|
||||
with connectable.connect() as connection:
|
||||
do_run_migrations(connection)
|
||||
else:
|
||||
# It's already a connection
|
||||
do_run_migrations(connectable)
|
||||
|
||||
|
||||
if context.is_offline_mode():
|
||||
run_migrations_offline()
|
||||
else:
|
||||
run_migrations_online()
|
||||
@@ -1,24 +0,0 @@
|
||||
"""Functions for managing database migrations."""
|
||||
|
||||
from pathlib import Path
|
||||
from loguru import logger
|
||||
from alembic.config import Config
|
||||
from alembic import command
|
||||
|
||||
|
||||
def get_alembic_config() -> Config: # pragma: no cover
|
||||
"""Get alembic config with correct paths."""
|
||||
migrations_path = Path(__file__).parent
|
||||
alembic_ini = migrations_path / "alembic.ini"
|
||||
|
||||
config = Config(alembic_ini)
|
||||
config.set_main_option("script_location", str(migrations_path))
|
||||
return config
|
||||
|
||||
|
||||
def reset_database(): # pragma: no cover
|
||||
"""Drop and recreate all tables."""
|
||||
logger.info("Resetting database...")
|
||||
config = get_alembic_config()
|
||||
command.downgrade(config, "base")
|
||||
command.upgrade(config, "head")
|
||||
@@ -1,26 +0,0 @@
|
||||
"""${message}
|
||||
|
||||
Revision ID: ${up_revision}
|
||||
Revises: ${down_revision | comma,n}
|
||||
Create Date: ${create_date}
|
||||
|
||||
"""
|
||||
from typing import Sequence, Union
|
||||
|
||||
from alembic import op
|
||||
import sqlalchemy as sa
|
||||
${imports if imports else ""}
|
||||
|
||||
# revision identifiers, used by Alembic.
|
||||
revision: str = ${repr(up_revision)}
|
||||
down_revision: Union[str, None] = ${repr(down_revision)}
|
||||
branch_labels: Union[str, Sequence[str], None] = ${repr(branch_labels)}
|
||||
depends_on: Union[str, Sequence[str], None] = ${repr(depends_on)}
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
${upgrades if upgrades else "pass"}
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
${downgrades if downgrades else "pass"}
|
||||
-131
@@ -1,131 +0,0 @@
|
||||
"""Add Postgres full-text search support with tsvector and GIN indexes
|
||||
|
||||
Revision ID: 314f1ea54dc4
|
||||
Revises: e7e1f4367280
|
||||
Create Date: 2025-11-15 18:05:01.025405
|
||||
|
||||
"""
|
||||
|
||||
from typing import Sequence, Union
|
||||
|
||||
from alembic import op
|
||||
import sqlalchemy as sa
|
||||
|
||||
|
||||
# revision identifiers, used by Alembic.
|
||||
revision: str = "314f1ea54dc4"
|
||||
down_revision: Union[str, None] = "e7e1f4367280"
|
||||
branch_labels: Union[str, Sequence[str], None] = None
|
||||
depends_on: Union[str, Sequence[str], None] = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
"""Add PostgreSQL full-text search support.
|
||||
|
||||
This migration:
|
||||
1. Creates search_index table for Postgres (SQLite uses FTS5 virtual table)
|
||||
2. Adds generated tsvector column for full-text search
|
||||
3. Creates GIN index on the tsvector column for fast text queries
|
||||
4. Creates GIN index on metadata JSONB column for fast containment queries
|
||||
|
||||
Note: These changes only apply to Postgres. SQLite continues to use FTS5 virtual tables.
|
||||
"""
|
||||
# Check if we're using Postgres
|
||||
connection = op.get_bind()
|
||||
if connection.dialect.name == "postgresql":
|
||||
# Create search_index table for Postgres
|
||||
# For SQLite, this is a FTS5 virtual table created elsewhere
|
||||
from sqlalchemy.dialects.postgresql import JSONB
|
||||
|
||||
op.create_table(
|
||||
"search_index",
|
||||
sa.Column("id", sa.Integer(), nullable=False), # Entity IDs are integers
|
||||
sa.Column("project_id", sa.Integer(), nullable=False), # Multi-tenant isolation
|
||||
sa.Column("title", sa.Text(), nullable=True),
|
||||
sa.Column("content_stems", sa.Text(), nullable=True),
|
||||
sa.Column("content_snippet", sa.Text(), nullable=True),
|
||||
sa.Column("permalink", sa.String(), nullable=True), # Nullable for non-markdown files
|
||||
sa.Column("file_path", sa.String(), nullable=True),
|
||||
sa.Column("type", sa.String(), nullable=True),
|
||||
sa.Column("from_id", sa.Integer(), nullable=True), # Relation IDs are integers
|
||||
sa.Column("to_id", sa.Integer(), nullable=True), # Relation IDs are integers
|
||||
sa.Column("relation_type", sa.String(), nullable=True),
|
||||
sa.Column("entity_id", sa.Integer(), nullable=True), # Entity IDs are integers
|
||||
sa.Column("category", sa.String(), nullable=True),
|
||||
sa.Column("metadata", JSONB(), nullable=True), # Use JSONB for Postgres
|
||||
sa.Column("created_at", sa.DateTime(timezone=True), nullable=True),
|
||||
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=True),
|
||||
sa.PrimaryKeyConstraint(
|
||||
"id", "type", "project_id"
|
||||
), # Composite key: id can repeat across types
|
||||
sa.ForeignKeyConstraint(
|
||||
["project_id"],
|
||||
["project.id"],
|
||||
name="fk_search_index_project_id",
|
||||
ondelete="CASCADE",
|
||||
),
|
||||
if_not_exists=True,
|
||||
)
|
||||
|
||||
# Create index on project_id for efficient multi-tenant queries
|
||||
op.create_index(
|
||||
"ix_search_index_project_id",
|
||||
"search_index",
|
||||
["project_id"],
|
||||
unique=False,
|
||||
)
|
||||
|
||||
# Create unique partial index on permalink for markdown files
|
||||
# Non-markdown files don't have permalinks, so we use a partial index
|
||||
op.execute("""
|
||||
CREATE UNIQUE INDEX uix_search_index_permalink_project
|
||||
ON search_index (permalink, project_id)
|
||||
WHERE permalink IS NOT NULL
|
||||
""")
|
||||
|
||||
# Add tsvector column as a GENERATED ALWAYS column
|
||||
# This automatically updates when title or content_stems change
|
||||
op.execute("""
|
||||
ALTER TABLE search_index
|
||||
ADD COLUMN textsearchable_index_col tsvector
|
||||
GENERATED ALWAYS AS (
|
||||
to_tsvector('english',
|
||||
coalesce(title, '') || ' ' ||
|
||||
coalesce(content_stems, '')
|
||||
)
|
||||
) STORED
|
||||
""")
|
||||
|
||||
# Create GIN index on tsvector column for fast full-text search
|
||||
op.create_index(
|
||||
"idx_search_index_fts",
|
||||
"search_index",
|
||||
["textsearchable_index_col"],
|
||||
unique=False,
|
||||
postgresql_using="gin",
|
||||
)
|
||||
|
||||
# Create GIN index on metadata JSONB for fast containment queries
|
||||
# Using jsonb_path_ops for smaller index size and better performance
|
||||
op.execute("""
|
||||
CREATE INDEX idx_search_index_metadata_gin
|
||||
ON search_index
|
||||
USING GIN (metadata jsonb_path_ops)
|
||||
""")
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
"""Remove PostgreSQL full-text search support."""
|
||||
connection = op.get_bind()
|
||||
if connection.dialect.name == "postgresql":
|
||||
# Drop indexes first
|
||||
op.execute("DROP INDEX IF EXISTS idx_search_index_metadata_gin")
|
||||
op.drop_index("idx_search_index_fts", table_name="search_index")
|
||||
op.execute("DROP INDEX IF EXISTS uix_search_index_permalink_project")
|
||||
op.drop_index("ix_search_index_project_id", table_name="search_index")
|
||||
|
||||
# Drop the generated column
|
||||
op.execute("ALTER TABLE search_index DROP COLUMN IF EXISTS textsearchable_index_col")
|
||||
|
||||
# Drop the search_index table
|
||||
op.drop_table("search_index")
|
||||
@@ -1,93 +0,0 @@
|
||||
"""initial schema
|
||||
|
||||
Revision ID: 3dae7c7b1564
|
||||
Revises:
|
||||
Create Date: 2025-02-12 21:23:00.336344
|
||||
|
||||
"""
|
||||
|
||||
from typing import Sequence, Union
|
||||
|
||||
from alembic import op
|
||||
import sqlalchemy as sa
|
||||
|
||||
|
||||
# revision identifiers, used by Alembic.
|
||||
revision: str = "3dae7c7b1564"
|
||||
down_revision: Union[str, None] = None
|
||||
branch_labels: Union[str, Sequence[str], None] = None
|
||||
depends_on: Union[str, Sequence[str], None] = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
# ### commands auto generated by Alembic - please adjust! ###
|
||||
op.create_table(
|
||||
"entity",
|
||||
sa.Column("id", sa.Integer(), nullable=False),
|
||||
sa.Column("title", sa.String(), nullable=False),
|
||||
sa.Column("entity_type", sa.String(), nullable=False),
|
||||
sa.Column("entity_metadata", sa.JSON(), nullable=True),
|
||||
sa.Column("content_type", sa.String(), nullable=False),
|
||||
sa.Column("permalink", sa.String(), nullable=False),
|
||||
sa.Column("file_path", sa.String(), nullable=False),
|
||||
sa.Column("checksum", sa.String(), nullable=True),
|
||||
sa.Column("created_at", sa.DateTime(), nullable=False),
|
||||
sa.Column("updated_at", sa.DateTime(), nullable=False),
|
||||
sa.PrimaryKeyConstraint("id"),
|
||||
sa.UniqueConstraint("permalink", name="uix_entity_permalink"),
|
||||
)
|
||||
op.create_index("ix_entity_created_at", "entity", ["created_at"], unique=False)
|
||||
op.create_index(op.f("ix_entity_file_path"), "entity", ["file_path"], unique=True)
|
||||
op.create_index(op.f("ix_entity_permalink"), "entity", ["permalink"], unique=True)
|
||||
op.create_index("ix_entity_title", "entity", ["title"], unique=False)
|
||||
op.create_index("ix_entity_type", "entity", ["entity_type"], unique=False)
|
||||
op.create_index("ix_entity_updated_at", "entity", ["updated_at"], unique=False)
|
||||
op.create_table(
|
||||
"observation",
|
||||
sa.Column("id", sa.Integer(), nullable=False),
|
||||
sa.Column("entity_id", sa.Integer(), nullable=False),
|
||||
sa.Column("content", sa.Text(), nullable=False),
|
||||
sa.Column("category", sa.String(), nullable=False),
|
||||
sa.Column("context", sa.Text(), nullable=True),
|
||||
sa.Column("tags", sa.JSON(), server_default="[]", nullable=True),
|
||||
sa.ForeignKeyConstraint(["entity_id"], ["entity.id"], ondelete="CASCADE"),
|
||||
sa.PrimaryKeyConstraint("id"),
|
||||
)
|
||||
op.create_index("ix_observation_category", "observation", ["category"], unique=False)
|
||||
op.create_index("ix_observation_entity_id", "observation", ["entity_id"], unique=False)
|
||||
op.create_table(
|
||||
"relation",
|
||||
sa.Column("id", sa.Integer(), nullable=False),
|
||||
sa.Column("from_id", sa.Integer(), nullable=False),
|
||||
sa.Column("to_id", sa.Integer(), nullable=True),
|
||||
sa.Column("to_name", sa.String(), nullable=False),
|
||||
sa.Column("relation_type", sa.String(), nullable=False),
|
||||
sa.Column("context", sa.Text(), nullable=True),
|
||||
sa.ForeignKeyConstraint(["from_id"], ["entity.id"], ondelete="CASCADE"),
|
||||
sa.ForeignKeyConstraint(["to_id"], ["entity.id"], ondelete="CASCADE"),
|
||||
sa.PrimaryKeyConstraint("id"),
|
||||
sa.UniqueConstraint("from_id", "to_id", "relation_type", name="uix_relation"),
|
||||
)
|
||||
op.create_index("ix_relation_from_id", "relation", ["from_id"], unique=False)
|
||||
op.create_index("ix_relation_to_id", "relation", ["to_id"], unique=False)
|
||||
op.create_index("ix_relation_type", "relation", ["relation_type"], unique=False)
|
||||
# ### end Alembic commands ###
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
# ### commands auto generated by Alembic - please adjust! ###
|
||||
op.drop_index("ix_relation_type", table_name="relation")
|
||||
op.drop_index("ix_relation_to_id", table_name="relation")
|
||||
op.drop_index("ix_relation_from_id", table_name="relation")
|
||||
op.drop_table("relation")
|
||||
op.drop_index("ix_observation_entity_id", table_name="observation")
|
||||
op.drop_index("ix_observation_category", table_name="observation")
|
||||
op.drop_table("observation")
|
||||
op.drop_index("ix_entity_updated_at", table_name="entity")
|
||||
op.drop_index("ix_entity_type", table_name="entity")
|
||||
op.drop_index("ix_entity_title", table_name="entity")
|
||||
op.drop_index(op.f("ix_entity_permalink"), table_name="entity")
|
||||
op.drop_index(op.f("ix_entity_file_path"), table_name="entity")
|
||||
op.drop_index("ix_entity_created_at", table_name="entity")
|
||||
op.drop_table("entity")
|
||||
# ### end Alembic commands ###
|
||||
-51
@@ -1,51 +0,0 @@
|
||||
"""remove required from entity.permalink
|
||||
|
||||
Revision ID: 502b60eaa905
|
||||
Revises: b3c3938bacdb
|
||||
Create Date: 2025-02-24 13:33:09.790951
|
||||
|
||||
"""
|
||||
|
||||
from typing import Sequence, Union
|
||||
|
||||
from alembic import op
|
||||
import sqlalchemy as sa
|
||||
|
||||
|
||||
# revision identifiers, used by Alembic.
|
||||
revision: str = "502b60eaa905"
|
||||
down_revision: Union[str, None] = "b3c3938bacdb"
|
||||
branch_labels: Union[str, Sequence[str], None] = None
|
||||
depends_on: Union[str, Sequence[str], None] = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
# ### commands auto generated by Alembic - please adjust! ###
|
||||
with op.batch_alter_table("entity", schema=None) as batch_op:
|
||||
batch_op.alter_column("permalink", existing_type=sa.VARCHAR(), nullable=True)
|
||||
batch_op.drop_index("ix_entity_permalink")
|
||||
batch_op.create_index(batch_op.f("ix_entity_permalink"), ["permalink"], unique=False)
|
||||
batch_op.drop_constraint("uix_entity_permalink", type_="unique")
|
||||
batch_op.create_index(
|
||||
"uix_entity_permalink",
|
||||
["permalink"],
|
||||
unique=True,
|
||||
sqlite_where=sa.text("content_type = 'text/markdown' AND permalink IS NOT NULL"),
|
||||
)
|
||||
|
||||
# ### end Alembic commands ###
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
# ### commands auto generated by Alembic - please adjust! ###
|
||||
with op.batch_alter_table("entity", schema=None) as batch_op:
|
||||
batch_op.drop_index(
|
||||
"uix_entity_permalink",
|
||||
sqlite_where=sa.text("content_type = 'text/markdown' AND permalink IS NOT NULL"),
|
||||
)
|
||||
batch_op.create_unique_constraint("uix_entity_permalink", ["permalink"])
|
||||
batch_op.drop_index(batch_op.f("ix_entity_permalink"))
|
||||
batch_op.create_index("ix_entity_permalink", ["permalink"], unique=1)
|
||||
batch_op.alter_column("permalink", existing_type=sa.VARCHAR(), nullable=False)
|
||||
|
||||
# ### end Alembic commands ###
|
||||
@@ -1,120 +0,0 @@
|
||||
"""add projects table
|
||||
|
||||
Revision ID: 5fe1ab1ccebe
|
||||
Revises: cc7172b46608
|
||||
Create Date: 2025-05-14 09:05:18.214357
|
||||
|
||||
"""
|
||||
|
||||
from typing import Sequence, Union
|
||||
|
||||
from alembic import op
|
||||
import sqlalchemy as sa
|
||||
|
||||
|
||||
# revision identifiers, used by Alembic.
|
||||
revision: str = "5fe1ab1ccebe"
|
||||
down_revision: Union[str, None] = "cc7172b46608"
|
||||
branch_labels: Union[str, Sequence[str], None] = None
|
||||
depends_on: Union[str, Sequence[str], None] = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
# ### commands auto generated by Alembic - please adjust! ###
|
||||
|
||||
# SQLite FTS5 virtual table handling is SQLite-specific
|
||||
# For Postgres, search_index is a regular table managed by ORM
|
||||
connection = op.get_bind()
|
||||
is_sqlite = connection.dialect.name == "sqlite"
|
||||
|
||||
op.create_table(
|
||||
"project",
|
||||
sa.Column("id", sa.Integer(), nullable=False),
|
||||
sa.Column("name", sa.String(), nullable=False),
|
||||
sa.Column("description", sa.Text(), nullable=True),
|
||||
sa.Column("permalink", sa.String(), nullable=False),
|
||||
sa.Column("path", sa.String(), nullable=False),
|
||||
sa.Column("is_active", sa.Boolean(), nullable=False),
|
||||
sa.Column("is_default", sa.Boolean(), nullable=True),
|
||||
sa.Column("created_at", sa.DateTime(), nullable=False),
|
||||
sa.Column("updated_at", sa.DateTime(), nullable=False),
|
||||
sa.PrimaryKeyConstraint("id"),
|
||||
sa.UniqueConstraint("is_default"),
|
||||
sa.UniqueConstraint("name"),
|
||||
sa.UniqueConstraint("permalink"),
|
||||
if_not_exists=True,
|
||||
)
|
||||
with op.batch_alter_table("project", schema=None) as batch_op:
|
||||
batch_op.create_index(
|
||||
"ix_project_created_at", ["created_at"], unique=False, if_not_exists=True
|
||||
)
|
||||
batch_op.create_index("ix_project_name", ["name"], unique=True, if_not_exists=True)
|
||||
batch_op.create_index("ix_project_path", ["path"], unique=False, if_not_exists=True)
|
||||
batch_op.create_index(
|
||||
"ix_project_permalink", ["permalink"], unique=True, if_not_exists=True
|
||||
)
|
||||
batch_op.create_index(
|
||||
"ix_project_updated_at", ["updated_at"], unique=False, if_not_exists=True
|
||||
)
|
||||
|
||||
with op.batch_alter_table("entity", schema=None) as batch_op:
|
||||
batch_op.add_column(sa.Column("project_id", sa.Integer(), nullable=False))
|
||||
batch_op.drop_index(
|
||||
"uix_entity_permalink",
|
||||
sqlite_where=sa.text("content_type = 'text/markdown' AND permalink IS NOT NULL")
|
||||
if is_sqlite
|
||||
else None,
|
||||
)
|
||||
batch_op.drop_index("ix_entity_file_path")
|
||||
batch_op.create_index(batch_op.f("ix_entity_file_path"), ["file_path"], unique=False)
|
||||
batch_op.create_index("ix_entity_project_id", ["project_id"], unique=False)
|
||||
batch_op.create_index(
|
||||
"uix_entity_file_path_project", ["file_path", "project_id"], unique=True
|
||||
)
|
||||
batch_op.create_index(
|
||||
"uix_entity_permalink_project",
|
||||
["permalink", "project_id"],
|
||||
unique=True,
|
||||
sqlite_where=sa.text("content_type = 'text/markdown' AND permalink IS NOT NULL")
|
||||
if is_sqlite
|
||||
else None,
|
||||
)
|
||||
batch_op.create_foreign_key("fk_entity_project_id", "project", ["project_id"], ["id"])
|
||||
|
||||
# drop the search index table. it will be recreated
|
||||
# Only drop for SQLite - Postgres uses regular table managed by ORM
|
||||
if is_sqlite:
|
||||
op.drop_table("search_index")
|
||||
|
||||
# ### end Alembic commands ###
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
# ### commands auto generated by Alembic - please adjust! ###
|
||||
with op.batch_alter_table("entity", schema=None) as batch_op:
|
||||
batch_op.drop_constraint("fk_entity_project_id", type_="foreignkey")
|
||||
batch_op.drop_index(
|
||||
"uix_entity_permalink_project",
|
||||
sqlite_where=sa.text("content_type = 'text/markdown' AND permalink IS NOT NULL"),
|
||||
)
|
||||
batch_op.drop_index("uix_entity_file_path_project")
|
||||
batch_op.drop_index("ix_entity_project_id")
|
||||
batch_op.drop_index(batch_op.f("ix_entity_file_path"))
|
||||
batch_op.create_index("ix_entity_file_path", ["file_path"], unique=1)
|
||||
batch_op.create_index(
|
||||
"uix_entity_permalink",
|
||||
["permalink"],
|
||||
unique=1,
|
||||
sqlite_where=sa.text("content_type = 'text/markdown' AND permalink IS NOT NULL"),
|
||||
)
|
||||
batch_op.drop_column("project_id")
|
||||
|
||||
with op.batch_alter_table("project", schema=None) as batch_op:
|
||||
batch_op.drop_index("ix_project_updated_at")
|
||||
batch_op.drop_index("ix_project_permalink")
|
||||
batch_op.drop_index("ix_project_path")
|
||||
batch_op.drop_index("ix_project_name")
|
||||
batch_op.drop_index("ix_project_created_at")
|
||||
|
||||
op.drop_table("project")
|
||||
# ### end Alembic commands ###
|
||||
@@ -1,112 +0,0 @@
|
||||
"""project constraint fix
|
||||
|
||||
Revision ID: 647e7a75e2cd
|
||||
Revises: 5fe1ab1ccebe
|
||||
Create Date: 2025-06-03 12:48:30.162566
|
||||
|
||||
"""
|
||||
|
||||
from typing import Sequence, Union
|
||||
|
||||
from alembic import op
|
||||
import sqlalchemy as sa
|
||||
|
||||
|
||||
# revision identifiers, used by Alembic.
|
||||
revision: str = "647e7a75e2cd"
|
||||
down_revision: Union[str, None] = "5fe1ab1ccebe"
|
||||
branch_labels: Union[str, Sequence[str], None] = None
|
||||
depends_on: Union[str, Sequence[str], None] = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
"""Remove the problematic UNIQUE constraint on is_default column.
|
||||
|
||||
The UNIQUE constraint prevents multiple projects from having is_default=FALSE,
|
||||
which breaks project creation when the service sets is_default=False.
|
||||
|
||||
SQLite: Recreate the table without the constraint (no ALTER TABLE support)
|
||||
Postgres: Use ALTER TABLE to drop the constraint directly
|
||||
"""
|
||||
connection = op.get_bind()
|
||||
is_sqlite = connection.dialect.name == "sqlite"
|
||||
|
||||
if is_sqlite:
|
||||
# For SQLite, we need to recreate the table without the UNIQUE constraint
|
||||
# Create a new table without the UNIQUE constraint on is_default
|
||||
op.create_table(
|
||||
"project_new",
|
||||
sa.Column("id", sa.Integer(), nullable=False),
|
||||
sa.Column("name", sa.String(), nullable=False),
|
||||
sa.Column("description", sa.Text(), nullable=True),
|
||||
sa.Column("permalink", sa.String(), nullable=False),
|
||||
sa.Column("path", sa.String(), nullable=False),
|
||||
sa.Column("is_active", sa.Boolean(), nullable=False),
|
||||
sa.Column("is_default", sa.Boolean(), nullable=True), # No UNIQUE constraint!
|
||||
sa.Column("created_at", sa.DateTime(), nullable=False),
|
||||
sa.Column("updated_at", sa.DateTime(), nullable=False),
|
||||
sa.PrimaryKeyConstraint("id"),
|
||||
sa.UniqueConstraint("name"),
|
||||
sa.UniqueConstraint("permalink"),
|
||||
)
|
||||
|
||||
# Copy data from old table to new table
|
||||
op.execute("INSERT INTO project_new SELECT * FROM project")
|
||||
|
||||
# Drop the old table
|
||||
op.drop_table("project")
|
||||
|
||||
# Rename the new table
|
||||
op.rename_table("project_new", "project")
|
||||
|
||||
# Recreate the indexes
|
||||
with op.batch_alter_table("project", schema=None) as batch_op:
|
||||
batch_op.create_index("ix_project_created_at", ["created_at"], unique=False)
|
||||
batch_op.create_index("ix_project_name", ["name"], unique=True)
|
||||
batch_op.create_index("ix_project_path", ["path"], unique=False)
|
||||
batch_op.create_index("ix_project_permalink", ["permalink"], unique=True)
|
||||
batch_op.create_index("ix_project_updated_at", ["updated_at"], unique=False)
|
||||
else:
|
||||
# For Postgres, we can simply drop the constraint
|
||||
with op.batch_alter_table("project", schema=None) as batch_op:
|
||||
batch_op.drop_constraint("project_is_default_key", type_="unique")
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
"""Add back the UNIQUE constraint on is_default column.
|
||||
|
||||
WARNING: This will break project creation again if multiple projects
|
||||
have is_default=FALSE.
|
||||
"""
|
||||
# Recreate the table with the UNIQUE constraint
|
||||
op.create_table(
|
||||
"project_old",
|
||||
sa.Column("id", sa.Integer(), nullable=False),
|
||||
sa.Column("name", sa.String(), nullable=False),
|
||||
sa.Column("description", sa.Text(), nullable=True),
|
||||
sa.Column("permalink", sa.String(), nullable=False),
|
||||
sa.Column("path", sa.String(), nullable=False),
|
||||
sa.Column("is_active", sa.Boolean(), nullable=False),
|
||||
sa.Column("is_default", sa.Boolean(), nullable=True),
|
||||
sa.Column("created_at", sa.DateTime(), nullable=False),
|
||||
sa.Column("updated_at", sa.DateTime(), nullable=False),
|
||||
sa.PrimaryKeyConstraint("id"),
|
||||
sa.UniqueConstraint("is_default"), # Add back the problematic constraint
|
||||
sa.UniqueConstraint("name"),
|
||||
sa.UniqueConstraint("permalink"),
|
||||
)
|
||||
|
||||
# Copy data (this may fail if multiple FALSE values exist)
|
||||
op.execute("INSERT INTO project_old SELECT * FROM project")
|
||||
|
||||
# Drop the current table and rename
|
||||
op.drop_table("project")
|
||||
op.rename_table("project_old", "project")
|
||||
|
||||
# Recreate indexes
|
||||
with op.batch_alter_table("project", schema=None) as batch_op:
|
||||
batch_op.create_index("ix_project_created_at", ["created_at"], unique=False)
|
||||
batch_op.create_index("ix_project_name", ["name"], unique=True)
|
||||
batch_op.create_index("ix_project_path", ["path"], unique=False)
|
||||
batch_op.create_index("ix_project_permalink", ["permalink"], unique=True)
|
||||
batch_op.create_index("ix_project_updated_at", ["updated_at"], unique=False)
|
||||
@@ -1,24 +0,0 @@
|
||||
"""Merge multiple heads
|
||||
|
||||
Revision ID: 6830751f5fb6
|
||||
Revises: a2b3c4d5e6f7, g9a0b3c4d5e6
|
||||
Create Date: 2025-12-29 12:46:46.476268
|
||||
|
||||
"""
|
||||
|
||||
from typing import Sequence, Union
|
||||
|
||||
|
||||
# revision identifiers, used by Alembic.
|
||||
revision: str = "6830751f5fb6"
|
||||
down_revision: Union[str, Sequence[str], None] = ("a2b3c4d5e6f7", "g9a0b3c4d5e6")
|
||||
branch_labels: Union[str, Sequence[str], None] = None
|
||||
depends_on: Union[str, Sequence[str], None] = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
pass
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
pass
|
||||
-49
@@ -1,49 +0,0 @@
|
||||
"""Add mtime and size columns to Entity for sync optimization
|
||||
|
||||
Revision ID: 9d9c1cb7d8f5
|
||||
Revises: a1b2c3d4e5f6
|
||||
Create Date: 2025-10-20 05:07:55.173849
|
||||
|
||||
"""
|
||||
|
||||
from typing import Sequence, Union
|
||||
|
||||
from alembic import op
|
||||
import sqlalchemy as sa
|
||||
|
||||
|
||||
# revision identifiers, used by Alembic.
|
||||
revision: str = "9d9c1cb7d8f5"
|
||||
down_revision: Union[str, None] = "a1b2c3d4e5f6"
|
||||
branch_labels: Union[str, Sequence[str], None] = None
|
||||
depends_on: Union[str, Sequence[str], None] = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
# ### commands auto generated by Alembic - please adjust! ###
|
||||
with op.batch_alter_table("entity", schema=None) as batch_op:
|
||||
batch_op.add_column(sa.Column("mtime", sa.Float(), nullable=True))
|
||||
batch_op.add_column(sa.Column("size", sa.Integer(), nullable=True))
|
||||
batch_op.drop_constraint(batch_op.f("fk_entity_project_id"), type_="foreignkey")
|
||||
batch_op.create_foreign_key(
|
||||
batch_op.f("fk_entity_project_id"), "project", ["project_id"], ["id"]
|
||||
)
|
||||
|
||||
# ### end Alembic commands ###
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
# ### commands auto generated by Alembic - please adjust! ###
|
||||
with op.batch_alter_table("entity", schema=None) as batch_op:
|
||||
batch_op.drop_constraint(batch_op.f("fk_entity_project_id"), type_="foreignkey")
|
||||
batch_op.create_foreign_key(
|
||||
batch_op.f("fk_entity_project_id"),
|
||||
"project",
|
||||
["project_id"],
|
||||
["id"],
|
||||
ondelete="CASCADE",
|
||||
)
|
||||
batch_op.drop_column("size")
|
||||
batch_op.drop_column("mtime")
|
||||
|
||||
# ### end Alembic commands ###
|
||||
@@ -1,49 +0,0 @@
|
||||
"""fix project foreign keys
|
||||
|
||||
Revision ID: a1b2c3d4e5f6
|
||||
Revises: 647e7a75e2cd
|
||||
Create Date: 2025-08-19 22:06:00.000000
|
||||
|
||||
"""
|
||||
|
||||
from typing import Sequence, Union
|
||||
|
||||
from alembic import op
|
||||
|
||||
|
||||
# revision identifiers, used by Alembic.
|
||||
revision: str = "a1b2c3d4e5f6"
|
||||
down_revision: Union[str, None] = "647e7a75e2cd"
|
||||
branch_labels: Union[str, Sequence[str], None] = None
|
||||
depends_on: Union[str, Sequence[str], None] = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
"""Re-establish foreign key constraints that were lost during project table recreation.
|
||||
|
||||
The migration 647e7a75e2cd recreated the project table but did not re-establish
|
||||
the foreign key constraint from entity.project_id to project.id, causing
|
||||
foreign key constraint failures when trying to delete projects with related entities.
|
||||
"""
|
||||
# SQLite doesn't allow adding foreign key constraints to existing tables easily
|
||||
# We need to be careful and handle the case where the constraint might already exist
|
||||
|
||||
with op.batch_alter_table("entity", schema=None) as batch_op:
|
||||
# Try to drop existing foreign key constraint (may not exist)
|
||||
try:
|
||||
batch_op.drop_constraint("fk_entity_project_id", type_="foreignkey")
|
||||
except Exception:
|
||||
# Constraint may not exist, which is fine - we'll create it next
|
||||
pass
|
||||
|
||||
# Add the foreign key constraint with CASCADE DELETE
|
||||
# This ensures that when a project is deleted, all related entities are also deleted
|
||||
batch_op.create_foreign_key(
|
||||
"fk_entity_project_id", "project", ["project_id"], ["id"], ondelete="CASCADE"
|
||||
)
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
"""Remove the foreign key constraint."""
|
||||
with op.batch_alter_table("entity", schema=None) as batch_op:
|
||||
batch_op.drop_constraint("fk_entity_project_id", type_="foreignkey")
|
||||
@@ -1,56 +0,0 @@
|
||||
"""Add cascade delete FK from search_index to entity
|
||||
|
||||
Revision ID: a2b3c4d5e6f7
|
||||
Revises: f8a9b2c3d4e5
|
||||
Create Date: 2025-12-02 07:00:00.000000
|
||||
|
||||
"""
|
||||
|
||||
from typing import Sequence, Union
|
||||
|
||||
from alembic import op
|
||||
|
||||
|
||||
# revision identifiers, used by Alembic.
|
||||
revision: str = "a2b3c4d5e6f7"
|
||||
down_revision: Union[str, None] = "f8a9b2c3d4e5"
|
||||
branch_labels: Union[str, Sequence[str], None] = None
|
||||
depends_on: Union[str, Sequence[str], None] = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
"""Add FK with CASCADE delete from search_index.entity_id to entity.id.
|
||||
|
||||
This migration is Postgres-only because:
|
||||
- SQLite uses FTS5 virtual tables which don't support foreign keys
|
||||
- The FK enables automatic cleanup of search_index entries when entities are deleted
|
||||
"""
|
||||
connection = op.get_bind()
|
||||
dialect = connection.dialect.name
|
||||
|
||||
if dialect == "postgresql":
|
||||
# First, clean up any orphaned search_index entries where entity no longer exists
|
||||
op.execute("""
|
||||
DELETE FROM search_index
|
||||
WHERE entity_id IS NOT NULL
|
||||
AND entity_id NOT IN (SELECT id FROM entity)
|
||||
""")
|
||||
|
||||
# Add FK with CASCADE - nullable FK allows search_index entries without entity_id
|
||||
op.create_foreign_key(
|
||||
"fk_search_index_entity_id",
|
||||
"search_index",
|
||||
"entity",
|
||||
["entity_id"],
|
||||
["id"],
|
||||
ondelete="CASCADE",
|
||||
)
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
"""Remove the FK constraint."""
|
||||
connection = op.get_bind()
|
||||
dialect = connection.dialect.name
|
||||
|
||||
if dialect == "postgresql":
|
||||
op.drop_constraint("fk_search_index_entity_id", "search_index", type_="foreignkey")
|
||||
@@ -1,44 +0,0 @@
|
||||
"""relation to_name unique index
|
||||
|
||||
Revision ID: b3c3938bacdb
|
||||
Revises: 3dae7c7b1564
|
||||
Create Date: 2025-02-22 14:59:30.668466
|
||||
|
||||
"""
|
||||
|
||||
from typing import Sequence, Union
|
||||
|
||||
from alembic import op
|
||||
|
||||
|
||||
# revision identifiers, used by Alembic.
|
||||
revision: str = "b3c3938bacdb"
|
||||
down_revision: Union[str, None] = "3dae7c7b1564"
|
||||
branch_labels: Union[str, Sequence[str], None] = None
|
||||
depends_on: Union[str, Sequence[str], None] = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
# SQLite doesn't support constraint changes through ALTER
|
||||
# Need to recreate table with desired constraints
|
||||
with op.batch_alter_table("relation") as batch_op:
|
||||
# Drop existing unique constraint
|
||||
batch_op.drop_constraint("uix_relation", type_="unique")
|
||||
|
||||
# Add new constraints
|
||||
batch_op.create_unique_constraint(
|
||||
"uix_relation_from_id_to_id", ["from_id", "to_id", "relation_type"]
|
||||
)
|
||||
batch_op.create_unique_constraint(
|
||||
"uix_relation_from_id_to_name", ["from_id", "to_name", "relation_type"]
|
||||
)
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
with op.batch_alter_table("relation") as batch_op:
|
||||
# Drop new constraints
|
||||
batch_op.drop_constraint("uix_relation_from_id_to_name", type_="unique")
|
||||
batch_op.drop_constraint("uix_relation_from_id_to_id", type_="unique")
|
||||
|
||||
# Restore original constraint
|
||||
batch_op.create_unique_constraint("uix_relation", ["from_id", "to_id", "relation_type"])
|
||||
@@ -1,113 +0,0 @@
|
||||
"""Update search index schema
|
||||
|
||||
Revision ID: cc7172b46608
|
||||
Revises: 502b60eaa905
|
||||
Create Date: 2025-02-28 18:48:23.244941
|
||||
|
||||
"""
|
||||
|
||||
from typing import Sequence, Union
|
||||
|
||||
from alembic import op
|
||||
|
||||
|
||||
# revision identifiers, used by Alembic.
|
||||
revision: str = "cc7172b46608"
|
||||
down_revision: Union[str, None] = "502b60eaa905"
|
||||
branch_labels: Union[str, Sequence[str], None] = None
|
||||
depends_on: Union[str, Sequence[str], None] = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
"""Upgrade database schema to use new search index with content_stems and content_snippet."""
|
||||
|
||||
# This migration is SQLite-specific (FTS5 virtual tables)
|
||||
# For Postgres, the search_index table is created via ORM models
|
||||
connection = op.get_bind()
|
||||
if connection.dialect.name != "sqlite":
|
||||
return
|
||||
|
||||
# First, drop the existing search_index table
|
||||
op.execute("DROP TABLE IF EXISTS search_index")
|
||||
|
||||
# Create new search_index with updated schema
|
||||
op.execute("""
|
||||
CREATE VIRTUAL TABLE IF NOT EXISTS search_index USING fts5(
|
||||
-- Core entity fields
|
||||
id UNINDEXED, -- Row ID
|
||||
title, -- Title for searching
|
||||
content_stems, -- Main searchable content split into stems
|
||||
content_snippet, -- File content snippet for display
|
||||
permalink, -- Stable identifier (now indexed for path search)
|
||||
file_path UNINDEXED, -- Physical location
|
||||
type UNINDEXED, -- entity/relation/observation
|
||||
|
||||
-- Relation fields
|
||||
from_id UNINDEXED, -- Source entity
|
||||
to_id UNINDEXED, -- Target entity
|
||||
relation_type UNINDEXED, -- Type of relation
|
||||
|
||||
-- Observation fields
|
||||
entity_id UNINDEXED, -- Parent entity
|
||||
category UNINDEXED, -- Observation category
|
||||
|
||||
-- Common fields
|
||||
metadata UNINDEXED, -- JSON metadata
|
||||
created_at UNINDEXED, -- Creation timestamp
|
||||
updated_at UNINDEXED, -- Last update
|
||||
|
||||
-- Configuration
|
||||
tokenize='unicode61 tokenchars 0x2F', -- Hex code for /
|
||||
prefix='1,2,3,4' -- Support longer prefixes for paths
|
||||
);
|
||||
""")
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
"""Downgrade database schema to use old search index."""
|
||||
|
||||
# This migration is SQLite-specific (FTS5 virtual tables)
|
||||
# For Postgres, the search_index table is managed via ORM models
|
||||
connection = op.get_bind()
|
||||
if connection.dialect.name != "sqlite":
|
||||
return
|
||||
|
||||
# Drop the updated search_index table
|
||||
op.execute("DROP TABLE IF EXISTS search_index")
|
||||
|
||||
# Recreate the original search_index schema
|
||||
op.execute("""
|
||||
CREATE VIRTUAL TABLE IF NOT EXISTS search_index USING fts5(
|
||||
-- Core entity fields
|
||||
id UNINDEXED, -- Row ID
|
||||
title, -- Title for searching
|
||||
content, -- Main searchable content
|
||||
permalink, -- Stable identifier (now indexed for path search)
|
||||
file_path UNINDEXED, -- Physical location
|
||||
type UNINDEXED, -- entity/relation/observation
|
||||
|
||||
-- Relation fields
|
||||
from_id UNINDEXED, -- Source entity
|
||||
to_id UNINDEXED, -- Target entity
|
||||
relation_type UNINDEXED, -- Type of relation
|
||||
|
||||
-- Observation fields
|
||||
entity_id UNINDEXED, -- Parent entity
|
||||
category UNINDEXED, -- Observation category
|
||||
|
||||
-- Common fields
|
||||
metadata UNINDEXED, -- JSON metadata
|
||||
created_at UNINDEXED, -- Creation timestamp
|
||||
updated_at UNINDEXED, -- Last update
|
||||
|
||||
-- Configuration
|
||||
tokenize='unicode61 tokenchars 0x2F', -- Hex code for /
|
||||
prefix='1,2,3,4' -- Support longer prefixes for paths
|
||||
);
|
||||
""")
|
||||
|
||||
# Print instruction to manually reindex after migration
|
||||
print("\n------------------------------------------------------------------")
|
||||
print("IMPORTANT: After downgrade completes, manually run the reindex command:")
|
||||
print("basic-memory sync")
|
||||
print("------------------------------------------------------------------\n")
|
||||
@@ -1,152 +0,0 @@
|
||||
"""Add structured metadata indexes for entity frontmatter
|
||||
|
||||
Revision ID: d7e8f9a0b1c2
|
||||
Revises: g9a0b3c4d5e6
|
||||
Create Date: 2026-01-31 12:00:00.000000
|
||||
|
||||
"""
|
||||
|
||||
from typing import Sequence, Union
|
||||
|
||||
import sqlalchemy as sa
|
||||
from alembic import op
|
||||
from sqlalchemy import text
|
||||
|
||||
|
||||
def column_exists(connection, table: str, column: str) -> bool:
|
||||
"""Check if a column exists in a table (idempotent migration support)."""
|
||||
if connection.dialect.name == "postgresql":
|
||||
result = connection.execute(
|
||||
text(
|
||||
"SELECT 1 FROM information_schema.columns "
|
||||
"WHERE table_name = :table AND column_name = :column"
|
||||
),
|
||||
{"table": table, "column": column},
|
||||
)
|
||||
return result.fetchone() is not None
|
||||
# SQLite
|
||||
result = connection.execute(text(f"PRAGMA table_info({table})"))
|
||||
columns = [row[1] for row in result]
|
||||
return column in columns
|
||||
|
||||
|
||||
def index_exists(connection, index_name: str) -> bool:
|
||||
"""Check if an index exists (idempotent migration support)."""
|
||||
if connection.dialect.name == "postgresql":
|
||||
result = connection.execute(
|
||||
text("SELECT 1 FROM pg_indexes WHERE indexname = :index_name"),
|
||||
{"index_name": index_name},
|
||||
)
|
||||
return result.fetchone() is not None
|
||||
# SQLite
|
||||
result = connection.execute(
|
||||
text("SELECT 1 FROM sqlite_master WHERE type='index' AND name = :index_name"),
|
||||
{"index_name": index_name},
|
||||
)
|
||||
return result.fetchone() is not None
|
||||
|
||||
|
||||
# revision identifiers, used by Alembic.
|
||||
revision: str = "d7e8f9a0b1c2"
|
||||
down_revision: Union[str, None] = "6830751f5fb6"
|
||||
branch_labels: Union[str, Sequence[str], None] = None
|
||||
depends_on: Union[str, Sequence[str], None] = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
"""Add JSONB/GiN indexes for Postgres and generated columns for SQLite."""
|
||||
connection = op.get_bind()
|
||||
dialect = connection.dialect.name
|
||||
|
||||
if dialect == "postgresql":
|
||||
# Ensure JSONB for efficient indexing
|
||||
result = connection.execute(
|
||||
text(
|
||||
"SELECT data_type FROM information_schema.columns "
|
||||
"WHERE table_name = 'entity' AND column_name = 'entity_metadata'"
|
||||
)
|
||||
).fetchone()
|
||||
if result and result[0] != "jsonb":
|
||||
op.execute(
|
||||
"ALTER TABLE entity ALTER COLUMN entity_metadata "
|
||||
"TYPE jsonb USING entity_metadata::jsonb"
|
||||
)
|
||||
|
||||
# General JSONB GIN index
|
||||
op.execute(
|
||||
"CREATE INDEX IF NOT EXISTS idx_entity_metadata_gin "
|
||||
"ON entity USING GIN (entity_metadata jsonb_path_ops)"
|
||||
)
|
||||
|
||||
# Common field indexes
|
||||
op.execute(
|
||||
"CREATE INDEX IF NOT EXISTS idx_entity_tags_json "
|
||||
"ON entity USING GIN ((entity_metadata -> 'tags'))"
|
||||
)
|
||||
op.execute(
|
||||
"CREATE INDEX IF NOT EXISTS idx_entity_frontmatter_type "
|
||||
"ON entity ((entity_metadata ->> 'type'))"
|
||||
)
|
||||
op.execute(
|
||||
"CREATE INDEX IF NOT EXISTS idx_entity_frontmatter_status "
|
||||
"ON entity ((entity_metadata ->> 'status'))"
|
||||
)
|
||||
return
|
||||
|
||||
# SQLite: add generated columns for common frontmatter fields
|
||||
if not column_exists(connection, "entity", "tags_json"):
|
||||
op.add_column(
|
||||
"entity",
|
||||
sa.Column(
|
||||
"tags_json",
|
||||
sa.Text(),
|
||||
sa.Computed("json_extract(entity_metadata, '$.tags')", persisted=True),
|
||||
),
|
||||
)
|
||||
if not column_exists(connection, "entity", "frontmatter_status"):
|
||||
op.add_column(
|
||||
"entity",
|
||||
sa.Column(
|
||||
"frontmatter_status",
|
||||
sa.Text(),
|
||||
sa.Computed("json_extract(entity_metadata, '$.status')", persisted=True),
|
||||
),
|
||||
)
|
||||
if not column_exists(connection, "entity", "frontmatter_type"):
|
||||
op.add_column(
|
||||
"entity",
|
||||
sa.Column(
|
||||
"frontmatter_type",
|
||||
sa.Text(),
|
||||
sa.Computed("json_extract(entity_metadata, '$.type')", persisted=True),
|
||||
),
|
||||
)
|
||||
|
||||
# Index generated columns
|
||||
if not index_exists(connection, "idx_entity_tags_json"):
|
||||
op.create_index("idx_entity_tags_json", "entity", ["tags_json"])
|
||||
if not index_exists(connection, "idx_entity_frontmatter_status"):
|
||||
op.create_index("idx_entity_frontmatter_status", "entity", ["frontmatter_status"])
|
||||
if not index_exists(connection, "idx_entity_frontmatter_type"):
|
||||
op.create_index("idx_entity_frontmatter_type", "entity", ["frontmatter_type"])
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
"""Best-effort downgrade (drop indexes, revert JSONB on Postgres)."""
|
||||
connection = op.get_bind()
|
||||
dialect = connection.dialect.name
|
||||
|
||||
if dialect == "postgresql":
|
||||
op.execute("DROP INDEX IF EXISTS idx_entity_frontmatter_status")
|
||||
op.execute("DROP INDEX IF EXISTS idx_entity_frontmatter_type")
|
||||
op.execute("DROP INDEX IF EXISTS idx_entity_tags_json")
|
||||
op.execute("DROP INDEX IF EXISTS idx_entity_metadata_gin")
|
||||
op.execute(
|
||||
"ALTER TABLE entity ALTER COLUMN entity_metadata TYPE json USING entity_metadata::json"
|
||||
)
|
||||
return
|
||||
|
||||
# SQLite: drop indexes (dropping generated columns requires table rebuild)
|
||||
op.execute("DROP INDEX IF EXISTS idx_entity_frontmatter_status")
|
||||
op.execute("DROP INDEX IF EXISTS idx_entity_frontmatter_type")
|
||||
op.execute("DROP INDEX IF EXISTS idx_entity_tags_json")
|
||||
-37
@@ -1,37 +0,0 @@
|
||||
"""Add scan watermark tracking to Project
|
||||
|
||||
Revision ID: e7e1f4367280
|
||||
Revises: 9d9c1cb7d8f5
|
||||
Create Date: 2025-10-20 16:42:46.625075
|
||||
|
||||
"""
|
||||
|
||||
from typing import Sequence, Union
|
||||
|
||||
from alembic import op
|
||||
import sqlalchemy as sa
|
||||
|
||||
|
||||
# revision identifiers, used by Alembic.
|
||||
revision: str = "e7e1f4367280"
|
||||
down_revision: Union[str, None] = "9d9c1cb7d8f5"
|
||||
branch_labels: Union[str, Sequence[str], None] = None
|
||||
depends_on: Union[str, Sequence[str], None] = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
# ### commands auto generated by Alembic - please adjust! ###
|
||||
with op.batch_alter_table("project", schema=None) as batch_op:
|
||||
batch_op.add_column(sa.Column("last_scan_timestamp", sa.Float(), nullable=True))
|
||||
batch_op.add_column(sa.Column("last_file_count", sa.Integer(), nullable=True))
|
||||
|
||||
# ### end Alembic commands ###
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
# ### commands auto generated by Alembic - please adjust! ###
|
||||
with op.batch_alter_table("project", schema=None) as batch_op:
|
||||
batch_op.drop_column("last_file_count")
|
||||
batch_op.drop_column("last_scan_timestamp")
|
||||
|
||||
# ### end Alembic commands ###
|
||||
-239
@@ -1,239 +0,0 @@
|
||||
"""Add project_id to relation/observation and pg_trgm for fuzzy link resolution
|
||||
|
||||
Revision ID: f8a9b2c3d4e5
|
||||
Revises: 314f1ea54dc4
|
||||
Create Date: 2025-12-01 12:00:00.000000
|
||||
|
||||
"""
|
||||
|
||||
from typing import Sequence, Union
|
||||
|
||||
import sqlalchemy as sa
|
||||
from alembic import op
|
||||
from sqlalchemy import text
|
||||
|
||||
|
||||
def column_exists(connection, table: str, column: str) -> bool:
|
||||
"""Check if a column exists in a table (idempotent migration support)."""
|
||||
if connection.dialect.name == "postgresql":
|
||||
result = connection.execute(
|
||||
text(
|
||||
"SELECT 1 FROM information_schema.columns "
|
||||
"WHERE table_name = :table AND column_name = :column"
|
||||
),
|
||||
{"table": table, "column": column},
|
||||
)
|
||||
return result.fetchone() is not None
|
||||
else:
|
||||
# SQLite
|
||||
result = connection.execute(text(f"PRAGMA table_info({table})"))
|
||||
columns = [row[1] for row in result]
|
||||
return column in columns
|
||||
|
||||
|
||||
def index_exists(connection, index_name: str) -> bool:
|
||||
"""Check if an index exists (idempotent migration support)."""
|
||||
if connection.dialect.name == "postgresql":
|
||||
result = connection.execute(
|
||||
text("SELECT 1 FROM pg_indexes WHERE indexname = :index_name"),
|
||||
{"index_name": index_name},
|
||||
)
|
||||
return result.fetchone() is not None
|
||||
else:
|
||||
# SQLite
|
||||
result = connection.execute(
|
||||
text("SELECT 1 FROM sqlite_master WHERE type='index' AND name = :index_name"),
|
||||
{"index_name": index_name},
|
||||
)
|
||||
return result.fetchone() is not None
|
||||
|
||||
|
||||
# revision identifiers, used by Alembic.
|
||||
revision: str = "f8a9b2c3d4e5"
|
||||
down_revision: Union[str, None] = "314f1ea54dc4"
|
||||
branch_labels: Union[str, Sequence[str], None] = None
|
||||
depends_on: Union[str, Sequence[str], None] = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
"""Add project_id to relation and observation tables, plus pg_trgm indexes.
|
||||
|
||||
This migration:
|
||||
1. Adds project_id column to relation and observation tables (denormalization)
|
||||
2. Backfills project_id from the associated entity
|
||||
3. Enables pg_trgm extension for trigram-based fuzzy matching (Postgres only)
|
||||
4. Creates GIN indexes on entity title and permalink for fast similarity searches
|
||||
5. Creates partial index on unresolved relations for efficient bulk resolution
|
||||
"""
|
||||
connection = op.get_bind()
|
||||
dialect = connection.dialect.name
|
||||
|
||||
# -------------------------------------------------------------------------
|
||||
# Add project_id to relation table
|
||||
# -------------------------------------------------------------------------
|
||||
|
||||
# Step 1: Add project_id column as nullable first (idempotent)
|
||||
if not column_exists(connection, "relation", "project_id"):
|
||||
op.add_column("relation", sa.Column("project_id", sa.Integer(), nullable=True))
|
||||
|
||||
# Step 2: Backfill project_id from entity.project_id via from_id
|
||||
if dialect == "postgresql":
|
||||
op.execute("""
|
||||
UPDATE relation
|
||||
SET project_id = entity.project_id
|
||||
FROM entity
|
||||
WHERE relation.from_id = entity.id
|
||||
""")
|
||||
else:
|
||||
# SQLite syntax
|
||||
op.execute("""
|
||||
UPDATE relation
|
||||
SET project_id = (
|
||||
SELECT entity.project_id
|
||||
FROM entity
|
||||
WHERE entity.id = relation.from_id
|
||||
)
|
||||
""")
|
||||
|
||||
# Step 3: Make project_id NOT NULL and add foreign key
|
||||
if dialect == "postgresql":
|
||||
op.alter_column("relation", "project_id", nullable=False)
|
||||
op.create_foreign_key(
|
||||
"fk_relation_project_id",
|
||||
"relation",
|
||||
"project",
|
||||
["project_id"],
|
||||
["id"],
|
||||
)
|
||||
else:
|
||||
# SQLite requires batch operations for ALTER COLUMN
|
||||
with op.batch_alter_table("relation") as batch_op:
|
||||
batch_op.alter_column("project_id", nullable=False)
|
||||
batch_op.create_foreign_key(
|
||||
"fk_relation_project_id",
|
||||
"project",
|
||||
["project_id"],
|
||||
["id"],
|
||||
)
|
||||
|
||||
# Step 4: Create index on relation.project_id (idempotent)
|
||||
if not index_exists(connection, "ix_relation_project_id"):
|
||||
op.create_index("ix_relation_project_id", "relation", ["project_id"])
|
||||
|
||||
# -------------------------------------------------------------------------
|
||||
# Add project_id to observation table
|
||||
# -------------------------------------------------------------------------
|
||||
|
||||
# Step 1: Add project_id column as nullable first (idempotent)
|
||||
if not column_exists(connection, "observation", "project_id"):
|
||||
op.add_column("observation", sa.Column("project_id", sa.Integer(), nullable=True))
|
||||
|
||||
# Step 2: Backfill project_id from entity.project_id via entity_id
|
||||
if dialect == "postgresql":
|
||||
op.execute("""
|
||||
UPDATE observation
|
||||
SET project_id = entity.project_id
|
||||
FROM entity
|
||||
WHERE observation.entity_id = entity.id
|
||||
""")
|
||||
else:
|
||||
# SQLite syntax
|
||||
op.execute("""
|
||||
UPDATE observation
|
||||
SET project_id = (
|
||||
SELECT entity.project_id
|
||||
FROM entity
|
||||
WHERE entity.id = observation.entity_id
|
||||
)
|
||||
""")
|
||||
|
||||
# Step 3: Make project_id NOT NULL and add foreign key
|
||||
if dialect == "postgresql":
|
||||
op.alter_column("observation", "project_id", nullable=False)
|
||||
op.create_foreign_key(
|
||||
"fk_observation_project_id",
|
||||
"observation",
|
||||
"project",
|
||||
["project_id"],
|
||||
["id"],
|
||||
)
|
||||
else:
|
||||
# SQLite requires batch operations for ALTER COLUMN
|
||||
with op.batch_alter_table("observation") as batch_op:
|
||||
batch_op.alter_column("project_id", nullable=False)
|
||||
batch_op.create_foreign_key(
|
||||
"fk_observation_project_id",
|
||||
"project",
|
||||
["project_id"],
|
||||
["id"],
|
||||
)
|
||||
|
||||
# Step 4: Create index on observation.project_id (idempotent)
|
||||
if not index_exists(connection, "ix_observation_project_id"):
|
||||
op.create_index("ix_observation_project_id", "observation", ["project_id"])
|
||||
|
||||
# Postgres-specific: pg_trgm and GIN indexes
|
||||
if dialect == "postgresql":
|
||||
# Enable pg_trgm extension for fuzzy string matching
|
||||
op.execute("CREATE EXTENSION IF NOT EXISTS pg_trgm")
|
||||
|
||||
# Create trigram indexes on entity table for fuzzy matching
|
||||
# GIN indexes with gin_trgm_ops support similarity searches
|
||||
op.execute("""
|
||||
CREATE INDEX IF NOT EXISTS idx_entity_title_trgm
|
||||
ON entity USING gin (title gin_trgm_ops)
|
||||
""")
|
||||
|
||||
op.execute("""
|
||||
CREATE INDEX IF NOT EXISTS idx_entity_permalink_trgm
|
||||
ON entity USING gin (permalink gin_trgm_ops)
|
||||
""")
|
||||
|
||||
# Create partial index on unresolved relations for efficient bulk resolution
|
||||
# This makes "WHERE to_id IS NULL AND project_id = X" queries very fast
|
||||
op.execute("""
|
||||
CREATE INDEX IF NOT EXISTS idx_relation_unresolved
|
||||
ON relation (project_id, to_name)
|
||||
WHERE to_id IS NULL
|
||||
""")
|
||||
|
||||
# Create index on relation.to_name for join performance in bulk resolution
|
||||
op.execute("""
|
||||
CREATE INDEX IF NOT EXISTS idx_relation_to_name
|
||||
ON relation (to_name)
|
||||
""")
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
"""Remove project_id from relation/observation and pg_trgm indexes."""
|
||||
connection = op.get_bind()
|
||||
dialect = connection.dialect.name
|
||||
|
||||
if dialect == "postgresql":
|
||||
# Drop Postgres-specific indexes
|
||||
op.execute("DROP INDEX IF EXISTS idx_relation_to_name")
|
||||
op.execute("DROP INDEX IF EXISTS idx_relation_unresolved")
|
||||
op.execute("DROP INDEX IF EXISTS idx_entity_permalink_trgm")
|
||||
op.execute("DROP INDEX IF EXISTS idx_entity_title_trgm")
|
||||
# Note: We don't drop the pg_trgm extension as other code may depend on it
|
||||
|
||||
# Drop project_id from observation
|
||||
op.drop_index("ix_observation_project_id", table_name="observation")
|
||||
op.drop_constraint("fk_observation_project_id", "observation", type_="foreignkey")
|
||||
op.drop_column("observation", "project_id")
|
||||
|
||||
# Drop project_id from relation
|
||||
op.drop_index("ix_relation_project_id", table_name="relation")
|
||||
op.drop_constraint("fk_relation_project_id", "relation", type_="foreignkey")
|
||||
op.drop_column("relation", "project_id")
|
||||
else:
|
||||
# SQLite requires batch operations
|
||||
op.drop_index("ix_observation_project_id", table_name="observation")
|
||||
with op.batch_alter_table("observation") as batch_op:
|
||||
batch_op.drop_constraint("fk_observation_project_id", type_="foreignkey")
|
||||
batch_op.drop_column("project_id")
|
||||
|
||||
op.drop_index("ix_relation_project_id", table_name="relation")
|
||||
with op.batch_alter_table("relation") as batch_op:
|
||||
batch_op.drop_constraint("fk_relation_project_id", type_="foreignkey")
|
||||
batch_op.drop_column("project_id")
|
||||
-173
@@ -1,173 +0,0 @@
|
||||
"""Add external_id UUID column to project and entity tables
|
||||
|
||||
Revision ID: g9a0b3c4d5e6
|
||||
Revises: f8a9b2c3d4e5
|
||||
Create Date: 2025-12-29 10:00:00.000000
|
||||
|
||||
"""
|
||||
|
||||
import uuid
|
||||
from typing import Sequence, Union
|
||||
|
||||
import sqlalchemy as sa
|
||||
from alembic import op
|
||||
from sqlalchemy import text
|
||||
|
||||
|
||||
def column_exists(connection, table: str, column: str) -> bool:
|
||||
"""Check if a column exists in a table (idempotent migration support)."""
|
||||
if connection.dialect.name == "postgresql":
|
||||
result = connection.execute(
|
||||
text(
|
||||
"SELECT 1 FROM information_schema.columns "
|
||||
"WHERE table_name = :table AND column_name = :column"
|
||||
),
|
||||
{"table": table, "column": column},
|
||||
)
|
||||
return result.fetchone() is not None
|
||||
else:
|
||||
# SQLite
|
||||
result = connection.execute(text(f"PRAGMA table_info({table})"))
|
||||
columns = [row[1] for row in result]
|
||||
return column in columns
|
||||
|
||||
|
||||
def index_exists(connection, index_name: str) -> bool:
|
||||
"""Check if an index exists (idempotent migration support)."""
|
||||
if connection.dialect.name == "postgresql":
|
||||
result = connection.execute(
|
||||
text("SELECT 1 FROM pg_indexes WHERE indexname = :index_name"),
|
||||
{"index_name": index_name},
|
||||
)
|
||||
return result.fetchone() is not None
|
||||
else:
|
||||
# SQLite
|
||||
result = connection.execute(
|
||||
text("SELECT 1 FROM sqlite_master WHERE type='index' AND name = :index_name"),
|
||||
{"index_name": index_name},
|
||||
)
|
||||
return result.fetchone() is not None
|
||||
|
||||
|
||||
# revision identifiers, used by Alembic.
|
||||
revision: str = "g9a0b3c4d5e6"
|
||||
down_revision: Union[str, None] = "f8a9b2c3d4e5"
|
||||
branch_labels: Union[str, Sequence[str], None] = None
|
||||
depends_on: Union[str, Sequence[str], None] = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
"""Add external_id UUID column to project and entity tables.
|
||||
|
||||
This migration:
|
||||
1. Adds external_id column to project table
|
||||
2. Adds external_id column to entity table
|
||||
3. Generates UUIDs for existing rows
|
||||
4. Creates unique indexes on both columns
|
||||
"""
|
||||
connection = op.get_bind()
|
||||
dialect = connection.dialect.name
|
||||
|
||||
# -------------------------------------------------------------------------
|
||||
# Add external_id to project table
|
||||
# -------------------------------------------------------------------------
|
||||
|
||||
if not column_exists(connection, "project", "external_id"):
|
||||
# Step 1: Add external_id column as nullable first
|
||||
op.add_column("project", sa.Column("external_id", sa.String(), nullable=True))
|
||||
|
||||
# Step 2: Generate UUIDs for existing rows
|
||||
if dialect == "postgresql":
|
||||
# Postgres has gen_random_uuid() function
|
||||
op.execute("""
|
||||
UPDATE project
|
||||
SET external_id = gen_random_uuid()::text
|
||||
WHERE external_id IS NULL
|
||||
""")
|
||||
else:
|
||||
# SQLite: need to generate UUIDs in Python
|
||||
result = connection.execute(text("SELECT id FROM project WHERE external_id IS NULL"))
|
||||
for row in result:
|
||||
new_uuid = str(uuid.uuid4())
|
||||
connection.execute(
|
||||
text("UPDATE project SET external_id = :uuid WHERE id = :id"),
|
||||
{"uuid": new_uuid, "id": row[0]},
|
||||
)
|
||||
|
||||
# Step 3: Make external_id NOT NULL
|
||||
if dialect == "postgresql":
|
||||
op.alter_column("project", "external_id", nullable=False)
|
||||
else:
|
||||
# SQLite requires batch operations for ALTER COLUMN
|
||||
with op.batch_alter_table("project") as batch_op:
|
||||
batch_op.alter_column("external_id", nullable=False)
|
||||
|
||||
# Step 4: Create unique index on project.external_id (idempotent)
|
||||
if not index_exists(connection, "ix_project_external_id"):
|
||||
op.create_index("ix_project_external_id", "project", ["external_id"], unique=True)
|
||||
|
||||
# -------------------------------------------------------------------------
|
||||
# Add external_id to entity table
|
||||
# -------------------------------------------------------------------------
|
||||
|
||||
if not column_exists(connection, "entity", "external_id"):
|
||||
# Step 1: Add external_id column as nullable first
|
||||
op.add_column("entity", sa.Column("external_id", sa.String(), nullable=True))
|
||||
|
||||
# Step 2: Generate UUIDs for existing rows
|
||||
if dialect == "postgresql":
|
||||
# Postgres has gen_random_uuid() function
|
||||
op.execute("""
|
||||
UPDATE entity
|
||||
SET external_id = gen_random_uuid()::text
|
||||
WHERE external_id IS NULL
|
||||
""")
|
||||
else:
|
||||
# SQLite: need to generate UUIDs in Python
|
||||
result = connection.execute(text("SELECT id FROM entity WHERE external_id IS NULL"))
|
||||
for row in result:
|
||||
new_uuid = str(uuid.uuid4())
|
||||
connection.execute(
|
||||
text("UPDATE entity SET external_id = :uuid WHERE id = :id"),
|
||||
{"uuid": new_uuid, "id": row[0]},
|
||||
)
|
||||
|
||||
# Step 3: Make external_id NOT NULL
|
||||
if dialect == "postgresql":
|
||||
op.alter_column("entity", "external_id", nullable=False)
|
||||
else:
|
||||
# SQLite requires batch operations for ALTER COLUMN
|
||||
with op.batch_alter_table("entity") as batch_op:
|
||||
batch_op.alter_column("external_id", nullable=False)
|
||||
|
||||
# Step 4: Create unique index on entity.external_id (idempotent)
|
||||
if not index_exists(connection, "ix_entity_external_id"):
|
||||
op.create_index("ix_entity_external_id", "entity", ["external_id"], unique=True)
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
"""Remove external_id columns from project and entity tables."""
|
||||
connection = op.get_bind()
|
||||
dialect = connection.dialect.name
|
||||
|
||||
# Drop from entity table
|
||||
if index_exists(connection, "ix_entity_external_id"):
|
||||
op.drop_index("ix_entity_external_id", table_name="entity")
|
||||
|
||||
if column_exists(connection, "entity", "external_id"):
|
||||
if dialect == "postgresql":
|
||||
op.drop_column("entity", "external_id")
|
||||
else:
|
||||
with op.batch_alter_table("entity") as batch_op:
|
||||
batch_op.drop_column("external_id")
|
||||
|
||||
# Drop from project table
|
||||
if index_exists(connection, "ix_project_external_id"):
|
||||
op.drop_index("ix_project_external_id", table_name="project")
|
||||
|
||||
if column_exists(connection, "project", "external_id"):
|
||||
if dialect == "postgresql":
|
||||
op.drop_column("project", "external_id")
|
||||
else:
|
||||
with op.batch_alter_table("project") as batch_op:
|
||||
batch_op.drop_column("external_id")
|
||||
@@ -1,5 +0,0 @@
|
||||
"""Basic Memory API module."""
|
||||
|
||||
from .app import app
|
||||
|
||||
__all__ = ["app"]
|
||||
@@ -1,137 +0,0 @@
|
||||
"""FastAPI application for basic-memory knowledge graph API."""
|
||||
|
||||
from contextlib import asynccontextmanager
|
||||
|
||||
from fastapi import FastAPI, HTTPException, Request
|
||||
from fastapi.exception_handlers import http_exception_handler
|
||||
from fastapi.routing import APIRouter
|
||||
from loguru import logger
|
||||
|
||||
from basic_memory import __version__ as version
|
||||
from basic_memory.api.container import ApiContainer, set_container
|
||||
from basic_memory.api.v2.routers import (
|
||||
knowledge_router as v2_knowledge,
|
||||
project_router as v2_project,
|
||||
memory_router as v2_memory,
|
||||
search_router as v2_search,
|
||||
resource_router as v2_resource,
|
||||
directory_router as v2_directory,
|
||||
prompt_router as v2_prompt,
|
||||
importer_router as v2_importer,
|
||||
)
|
||||
from basic_memory.api.v2.routers.project_router import (
|
||||
add_project,
|
||||
list_projects,
|
||||
synchronize_projects,
|
||||
)
|
||||
from basic_memory.config import init_api_logging
|
||||
from basic_memory.services.exceptions import EntityAlreadyExistsError
|
||||
from basic_memory.services.initialization import initialize_app
|
||||
|
||||
|
||||
@asynccontextmanager
|
||||
async def lifespan(app: FastAPI): # pragma: no cover
|
||||
"""Lifecycle manager for the FastAPI app. Not called in stdio mcp mode"""
|
||||
|
||||
# Initialize logging for API (stdout in cloud mode, file otherwise)
|
||||
init_api_logging()
|
||||
|
||||
# --- Composition Root ---
|
||||
# Create container and read config (single point of config access)
|
||||
container = ApiContainer.create()
|
||||
set_container(container)
|
||||
app.state.container = container
|
||||
|
||||
logger.info(f"Starting Basic Memory API (mode={container.mode.name})")
|
||||
|
||||
await initialize_app(container.config)
|
||||
|
||||
# Cache database connections in app state for performance
|
||||
logger.info("Initializing database and caching connections...")
|
||||
engine, session_maker = await container.init_database()
|
||||
app.state.engine = engine
|
||||
app.state.session_maker = session_maker
|
||||
logger.info("Database connections cached in app state")
|
||||
|
||||
# Create and start sync coordinator (lifecycle centralized in coordinator)
|
||||
sync_coordinator = container.create_sync_coordinator()
|
||||
await sync_coordinator.start()
|
||||
app.state.sync_coordinator = sync_coordinator
|
||||
|
||||
# Proceed with startup
|
||||
yield
|
||||
|
||||
# Shutdown - coordinator handles clean task cancellation
|
||||
logger.info("Shutting down Basic Memory API")
|
||||
await sync_coordinator.stop()
|
||||
|
||||
await container.shutdown_database()
|
||||
|
||||
|
||||
# Initialize FastAPI app
|
||||
app = FastAPI(
|
||||
title="Basic Memory API",
|
||||
description="Knowledge graph API for basic-memory",
|
||||
version=version,
|
||||
lifespan=lifespan,
|
||||
)
|
||||
|
||||
# Include v2 routers FIRST (more specific paths must match before /{project} catch-all)
|
||||
app.include_router(v2_knowledge, prefix="/v2/projects/{project_id}")
|
||||
app.include_router(v2_memory, prefix="/v2/projects/{project_id}")
|
||||
app.include_router(v2_search, prefix="/v2/projects/{project_id}")
|
||||
app.include_router(v2_resource, prefix="/v2/projects/{project_id}")
|
||||
app.include_router(v2_directory, prefix="/v2/projects/{project_id}")
|
||||
app.include_router(v2_prompt, prefix="/v2/projects/{project_id}")
|
||||
app.include_router(v2_importer, prefix="/v2/projects/{project_id}")
|
||||
app.include_router(v2_project, prefix="/v2")
|
||||
|
||||
# Legacy web app proxy paths (compat with /proxy/projects/projects)
|
||||
app.include_router(v2_project, prefix="/proxy/projects")
|
||||
|
||||
# Legacy v1 compat: older CLI versions (v0.18.0 and earlier) call /projects/...
|
||||
# Using router mount causes 307 redirect which proxy doesn't follow, so add explicit routes
|
||||
legacy_router = APIRouter(tags=["legacy"])
|
||||
legacy_router.add_api_route("/projects/projects", list_projects, methods=["GET"])
|
||||
legacy_router.add_api_route("/projects/projects", add_project, methods=["POST"])
|
||||
legacy_router.add_api_route("/projects/config/sync", synchronize_projects, methods=["POST"])
|
||||
app.include_router(legacy_router)
|
||||
|
||||
# V2 routers are the only public API surface
|
||||
|
||||
|
||||
@app.exception_handler(EntityAlreadyExistsError)
|
||||
async def entity_already_exists_error_handler(request: Request, exc: EntityAlreadyExistsError):
|
||||
"""Handle entity creation conflicts (e.g., file already exists).
|
||||
|
||||
This is expected behavior when users try to create notes that exist,
|
||||
so log at INFO level instead of ERROR.
|
||||
"""
|
||||
logger.info(
|
||||
"Entity already exists",
|
||||
url=str(request.url),
|
||||
method=request.method,
|
||||
path=request.url.path,
|
||||
error=str(exc),
|
||||
)
|
||||
return await http_exception_handler(
|
||||
request,
|
||||
HTTPException(
|
||||
status_code=409,
|
||||
detail="Note already exists. Use edit_note to modify it, or delete it first.",
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
@app.exception_handler(Exception)
|
||||
async def exception_handler(request, exc): # pragma: no cover
|
||||
logger.exception(
|
||||
"API unhandled exception",
|
||||
url=str(request.url),
|
||||
method=request.method,
|
||||
client=request.client.host if request.client else None,
|
||||
path=request.url.path,
|
||||
error_type=type(exc).__name__,
|
||||
error=str(exc),
|
||||
)
|
||||
return await http_exception_handler(request, HTTPException(status_code=500, detail=str(exc)))
|
||||
@@ -1,133 +0,0 @@
|
||||
"""API composition root for Basic Memory.
|
||||
|
||||
This container owns reading ConfigManager and environment variables for the
|
||||
API entrypoint. Downstream modules receive config/dependencies explicitly
|
||||
rather than reading globals.
|
||||
|
||||
Design principles:
|
||||
- Only this module reads ConfigManager directly
|
||||
- Runtime mode (cloud/local/test) is resolved here
|
||||
- Factories for services are provided, not singletons
|
||||
"""
|
||||
|
||||
from dataclasses import dataclass
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from sqlalchemy.ext.asyncio import AsyncEngine, async_sessionmaker, AsyncSession
|
||||
|
||||
from basic_memory import db
|
||||
from basic_memory.config import BasicMemoryConfig, ConfigManager
|
||||
from basic_memory.runtime import RuntimeMode, resolve_runtime_mode
|
||||
|
||||
if TYPE_CHECKING: # pragma: no cover
|
||||
from basic_memory.sync import SyncCoordinator
|
||||
|
||||
|
||||
@dataclass
|
||||
class ApiContainer:
|
||||
"""Composition root for the API entrypoint.
|
||||
|
||||
Holds resolved configuration and runtime context.
|
||||
Created once at app startup, then used to wire dependencies.
|
||||
"""
|
||||
|
||||
config: BasicMemoryConfig
|
||||
mode: RuntimeMode
|
||||
|
||||
# --- Database ---
|
||||
# Cached database connections (set during lifespan startup)
|
||||
engine: AsyncEngine | None = None
|
||||
session_maker: async_sessionmaker[AsyncSession] | None = None
|
||||
|
||||
@classmethod
|
||||
def create(cls) -> "ApiContainer": # pragma: no cover
|
||||
"""Create container by reading ConfigManager.
|
||||
|
||||
This is the single point where API reads global config.
|
||||
"""
|
||||
config = ConfigManager().config
|
||||
mode = resolve_runtime_mode(
|
||||
cloud_mode_enabled=config.cloud_mode_enabled,
|
||||
is_test_env=config.is_test_env,
|
||||
)
|
||||
return cls(config=config, mode=mode)
|
||||
|
||||
# --- Runtime Mode Properties ---
|
||||
|
||||
@property
|
||||
def should_sync_files(self) -> bool:
|
||||
"""Whether file sync should be started.
|
||||
|
||||
Sync is enabled when:
|
||||
- sync_changes is True in config
|
||||
- Not in test mode (tests manage their own sync)
|
||||
"""
|
||||
return self.config.sync_changes and not self.mode.is_test
|
||||
|
||||
@property
|
||||
def sync_skip_reason(self) -> str | None: # pragma: no cover
|
||||
"""Reason why sync is skipped, or None if sync should run.
|
||||
|
||||
Useful for logging why sync was disabled.
|
||||
"""
|
||||
if self.mode.is_test:
|
||||
return "Test environment detected"
|
||||
if not self.config.sync_changes:
|
||||
return "Sync changes disabled"
|
||||
return None
|
||||
|
||||
def create_sync_coordinator(self) -> "SyncCoordinator": # pragma: no cover
|
||||
"""Create a SyncCoordinator with this container's settings.
|
||||
|
||||
Returns:
|
||||
SyncCoordinator configured for this runtime environment
|
||||
"""
|
||||
# Deferred import to avoid circular dependency
|
||||
from basic_memory.sync import SyncCoordinator
|
||||
|
||||
return SyncCoordinator(
|
||||
config=self.config,
|
||||
should_sync=self.should_sync_files,
|
||||
skip_reason=self.sync_skip_reason,
|
||||
)
|
||||
|
||||
# --- Database Factory ---
|
||||
|
||||
async def init_database( # pragma: no cover
|
||||
self,
|
||||
) -> tuple[AsyncEngine, async_sessionmaker[AsyncSession]]:
|
||||
"""Initialize and cache database connections.
|
||||
|
||||
Returns:
|
||||
Tuple of (engine, session_maker)
|
||||
"""
|
||||
engine, session_maker = await db.get_or_create_db(self.config.database_path)
|
||||
self.engine = engine
|
||||
self.session_maker = session_maker
|
||||
return engine, session_maker
|
||||
|
||||
async def shutdown_database(self) -> None: # pragma: no cover
|
||||
"""Clean up database connections."""
|
||||
await db.shutdown_db()
|
||||
|
||||
|
||||
# Module-level container instance (set by lifespan)
|
||||
# This allows deps.py to access the container without reading ConfigManager
|
||||
_container: ApiContainer | None = None
|
||||
|
||||
|
||||
def get_container() -> ApiContainer:
|
||||
"""Get the current API container.
|
||||
|
||||
Raises:
|
||||
RuntimeError: If container hasn't been initialized
|
||||
"""
|
||||
if _container is None:
|
||||
raise RuntimeError("API container not initialized. Call set_container() first.")
|
||||
return _container
|
||||
|
||||
|
||||
def set_container(container: ApiContainer) -> None:
|
||||
"""Set the API container (called by lifespan)."""
|
||||
global _container
|
||||
_container = container
|
||||
@@ -1,292 +0,0 @@
|
||||
"""Template loading and rendering utilities for the Basic Memory API.
|
||||
|
||||
This module handles the loading and rendering of Handlebars templates from the
|
||||
templates directory, providing a consistent interface for all prompt-related
|
||||
formatting needs.
|
||||
"""
|
||||
|
||||
import textwrap
|
||||
from typing import Dict, Any, Optional, Callable
|
||||
from pathlib import Path
|
||||
import json
|
||||
import datetime
|
||||
|
||||
import pybars
|
||||
from loguru import logger
|
||||
|
||||
# Get the base path of the templates directory
|
||||
TEMPLATES_DIR = Path(__file__).parent.parent / "templates"
|
||||
|
||||
|
||||
# Custom helpers for Handlebars
|
||||
def _date_helper(this, *args):
|
||||
"""Format a date using the given format string."""
|
||||
if len(args) < 1: # pragma: no cover
|
||||
return ""
|
||||
|
||||
timestamp = args[0]
|
||||
format_str = args[1] if len(args) > 1 else "%Y-%m-%d %H:%M"
|
||||
|
||||
if hasattr(timestamp, "strftime"):
|
||||
result = timestamp.strftime(format_str)
|
||||
elif isinstance(timestamp, str):
|
||||
try:
|
||||
dt = datetime.datetime.fromisoformat(timestamp)
|
||||
result = dt.strftime(format_str)
|
||||
except ValueError:
|
||||
result = timestamp
|
||||
else:
|
||||
result = str(timestamp) # pragma: no cover
|
||||
|
||||
return pybars.strlist([result])
|
||||
|
||||
|
||||
def _default_helper(this, *args):
|
||||
"""Return a default value if the given value is None or empty."""
|
||||
if len(args) < 2: # pragma: no cover
|
||||
return ""
|
||||
|
||||
value = args[0]
|
||||
default_value = args[1]
|
||||
|
||||
result = default_value if value is None or value == "" else value
|
||||
# Use strlist for consistent handling of HTML escaping
|
||||
return pybars.strlist([str(result)])
|
||||
|
||||
|
||||
def _capitalize_helper(this, *args):
|
||||
"""Capitalize the first letter of a string."""
|
||||
if len(args) < 1: # pragma: no cover
|
||||
return ""
|
||||
|
||||
text = args[0]
|
||||
if not text or not isinstance(text, str): # pragma: no cover
|
||||
result = ""
|
||||
else:
|
||||
result = text.capitalize()
|
||||
|
||||
return pybars.strlist([result])
|
||||
|
||||
|
||||
def _round_helper(this, *args):
|
||||
"""Round a number to the specified number of decimal places."""
|
||||
if len(args) < 1:
|
||||
return ""
|
||||
|
||||
value = args[0]
|
||||
decimal_places = args[1] if len(args) > 1 else 2
|
||||
|
||||
try:
|
||||
result = str(round(float(value), int(decimal_places)))
|
||||
except (ValueError, TypeError):
|
||||
result = str(value)
|
||||
|
||||
return pybars.strlist([result])
|
||||
|
||||
|
||||
def _size_helper(this, *args):
|
||||
"""Return the size/length of a collection."""
|
||||
if len(args) < 1:
|
||||
return 0
|
||||
|
||||
value = args[0]
|
||||
if value is None:
|
||||
result = "0"
|
||||
elif isinstance(value, (list, tuple, dict, str)):
|
||||
result = str(len(value)) # pragma: no cover
|
||||
else: # pragma: no cover
|
||||
result = "0"
|
||||
|
||||
return pybars.strlist([result])
|
||||
|
||||
|
||||
def _json_helper(this, *args):
|
||||
"""Convert a value to a JSON string."""
|
||||
if len(args) < 1: # pragma: no cover
|
||||
return "{}"
|
||||
|
||||
value = args[0]
|
||||
# For pybars, we need to return a SafeString to prevent HTML escaping
|
||||
result = json.dumps(value) # pragma: no cover
|
||||
# Safe string implementation to prevent HTML escaping
|
||||
return pybars.strlist([result])
|
||||
|
||||
|
||||
def _math_helper(this, *args):
|
||||
"""Perform basic math operations."""
|
||||
if len(args) < 3:
|
||||
return pybars.strlist(["Math error: Insufficient arguments"])
|
||||
|
||||
lhs = args[0]
|
||||
operator = args[1]
|
||||
rhs = args[2]
|
||||
|
||||
try:
|
||||
lhs = float(lhs)
|
||||
rhs = float(rhs)
|
||||
if operator == "+":
|
||||
result = str(lhs + rhs)
|
||||
elif operator == "-":
|
||||
result = str(lhs - rhs)
|
||||
elif operator == "*":
|
||||
result = str(lhs * rhs)
|
||||
elif operator == "/":
|
||||
result = str(lhs / rhs)
|
||||
else:
|
||||
result = f"Unsupported operator: {operator}"
|
||||
except (ValueError, TypeError) as e:
|
||||
result = f"Math error: {e}"
|
||||
|
||||
return pybars.strlist([result])
|
||||
|
||||
|
||||
def _lt_helper(this, *args):
|
||||
"""Check if left hand side is less than right hand side."""
|
||||
if len(args) < 2:
|
||||
return False
|
||||
|
||||
lhs = args[0]
|
||||
rhs = args[1]
|
||||
|
||||
try:
|
||||
return float(lhs) < float(rhs)
|
||||
except (ValueError, TypeError):
|
||||
# Fall back to string comparison for non-numeric values
|
||||
return str(lhs) < str(rhs)
|
||||
|
||||
|
||||
def _if_cond_helper(this, options, condition):
|
||||
"""Block helper for custom if conditionals."""
|
||||
if condition:
|
||||
return options["fn"](this)
|
||||
elif "inverse" in options:
|
||||
return options["inverse"](this)
|
||||
return "" # pragma: no cover
|
||||
|
||||
|
||||
def _dedent_helper(this, options):
|
||||
"""Dedent a block of text to remove common leading whitespace.
|
||||
|
||||
Usage:
|
||||
{{#dedent}}
|
||||
This text will have its
|
||||
common leading whitespace removed
|
||||
while preserving relative indentation.
|
||||
{{/dedent}}
|
||||
"""
|
||||
if "fn" not in options: # pragma: no cover
|
||||
return ""
|
||||
|
||||
# Get the content from the block
|
||||
content = options["fn"](this)
|
||||
|
||||
# Convert to string if it's a strlist
|
||||
if (
|
||||
isinstance(content, list)
|
||||
or hasattr(content, "__iter__")
|
||||
and not isinstance(content, (str, bytes))
|
||||
):
|
||||
content_str = "".join(str(item) for item in content) # pragma: no cover
|
||||
else:
|
||||
content_str = str(content) # pragma: no cover
|
||||
|
||||
# Add trailing and leading newlines to ensure proper dedenting
|
||||
# This is critical for textwrap.dedent to work correctly with mixed content
|
||||
content_str = "\n" + content_str + "\n"
|
||||
|
||||
# Use textwrap to dedent the content and remove the extra newlines we added
|
||||
dedented = textwrap.dedent(content_str)[1:-1]
|
||||
|
||||
# Return as a SafeString to prevent HTML escaping
|
||||
return pybars.strlist([dedented]) # pragma: no cover
|
||||
|
||||
|
||||
class TemplateLoader:
|
||||
"""Loader for Handlebars templates.
|
||||
|
||||
This class is responsible for loading templates from disk and rendering
|
||||
them with the provided context data.
|
||||
"""
|
||||
|
||||
def __init__(self, template_dir: Optional[str] = None):
|
||||
"""Initialize the template loader.
|
||||
|
||||
Args:
|
||||
template_dir: Optional custom template directory path
|
||||
"""
|
||||
self.template_dir = Path(template_dir) if template_dir else TEMPLATES_DIR
|
||||
self.template_cache: Dict[str, Callable] = {}
|
||||
self.compiler = pybars.Compiler()
|
||||
|
||||
# Set up standard helpers
|
||||
self.helpers = {
|
||||
"date": _date_helper,
|
||||
"default": _default_helper,
|
||||
"capitalize": _capitalize_helper,
|
||||
"round": _round_helper,
|
||||
"size": _size_helper,
|
||||
"json": _json_helper,
|
||||
"math": _math_helper,
|
||||
"lt": _lt_helper,
|
||||
"if_cond": _if_cond_helper,
|
||||
"dedent": _dedent_helper,
|
||||
}
|
||||
|
||||
logger.debug(f"Initialized template loader with directory: {self.template_dir}")
|
||||
|
||||
def get_template(self, template_path: str) -> Callable:
|
||||
"""Get a template by path, using cache if available.
|
||||
|
||||
Args:
|
||||
template_path: The path to the template, relative to the templates directory
|
||||
|
||||
Returns:
|
||||
The compiled Handlebars template
|
||||
|
||||
Raises:
|
||||
FileNotFoundError: If the template doesn't exist
|
||||
"""
|
||||
if template_path in self.template_cache:
|
||||
return self.template_cache[template_path]
|
||||
|
||||
# Convert from Liquid-style path to Handlebars extension
|
||||
if template_path.endswith(".liquid"):
|
||||
template_path = template_path.replace(".liquid", ".hbs")
|
||||
elif not template_path.endswith(".hbs"):
|
||||
template_path = f"{template_path}.hbs"
|
||||
|
||||
full_path = self.template_dir / template_path
|
||||
|
||||
if not full_path.exists():
|
||||
raise FileNotFoundError(f"Template not found: {full_path}")
|
||||
|
||||
with open(full_path, "r", encoding="utf-8") as f:
|
||||
template_str = f.read()
|
||||
|
||||
template = self.compiler.compile(template_str)
|
||||
self.template_cache[template_path] = template
|
||||
|
||||
logger.debug(f"Loaded template: {template_path}")
|
||||
return template
|
||||
|
||||
async def render(self, template_path: str, context: Dict[str, Any]) -> str:
|
||||
"""Render a template with the given context.
|
||||
|
||||
Args:
|
||||
template_path: The path to the template, relative to the templates directory
|
||||
context: The context data to pass to the template
|
||||
|
||||
Returns:
|
||||
The rendered template as a string
|
||||
"""
|
||||
template = self.get_template(template_path)
|
||||
return template(context, helpers=self.helpers)
|
||||
|
||||
def clear_cache(self) -> None:
|
||||
"""Clear the template cache."""
|
||||
self.template_cache.clear()
|
||||
logger.debug("Template cache cleared")
|
||||
|
||||
|
||||
# Global template loader instance
|
||||
template_loader = TemplateLoader()
|
||||
@@ -1,35 +0,0 @@
|
||||
"""API v2 module - ID-based entity references.
|
||||
|
||||
Version 2 of the Basic Memory API uses integer entity IDs as the primary
|
||||
identifier for improved performance and stability.
|
||||
|
||||
Key changes from v1:
|
||||
- Entity lookups use integer IDs instead of paths/permalinks
|
||||
- Direct database queries instead of cascading resolution
|
||||
- Stable references that don't change with file moves
|
||||
- Better caching support
|
||||
|
||||
All v2 routers are registered with the /v2 prefix.
|
||||
"""
|
||||
|
||||
from basic_memory.api.v2.routers import (
|
||||
knowledge_router,
|
||||
memory_router,
|
||||
project_router,
|
||||
resource_router,
|
||||
search_router,
|
||||
directory_router,
|
||||
prompt_router,
|
||||
importer_router,
|
||||
)
|
||||
|
||||
__all__ = [
|
||||
"knowledge_router",
|
||||
"memory_router",
|
||||
"project_router",
|
||||
"resource_router",
|
||||
"search_router",
|
||||
"directory_router",
|
||||
"prompt_router",
|
||||
"importer_router",
|
||||
]
|
||||
@@ -1,21 +0,0 @@
|
||||
"""V2 API routers."""
|
||||
|
||||
from basic_memory.api.v2.routers.knowledge_router import router as knowledge_router
|
||||
from basic_memory.api.v2.routers.project_router import router as project_router
|
||||
from basic_memory.api.v2.routers.memory_router import router as memory_router
|
||||
from basic_memory.api.v2.routers.search_router import router as search_router
|
||||
from basic_memory.api.v2.routers.resource_router import router as resource_router
|
||||
from basic_memory.api.v2.routers.directory_router import router as directory_router
|
||||
from basic_memory.api.v2.routers.prompt_router import router as prompt_router
|
||||
from basic_memory.api.v2.routers.importer_router import router as importer_router
|
||||
|
||||
__all__ = [
|
||||
"knowledge_router",
|
||||
"project_router",
|
||||
"memory_router",
|
||||
"search_router",
|
||||
"resource_router",
|
||||
"directory_router",
|
||||
"prompt_router",
|
||||
"importer_router",
|
||||
]
|
||||
@@ -1,93 +0,0 @@
|
||||
"""V2 Directory Router - ID-based directory tree operations.
|
||||
|
||||
This router provides directory structure browsing for projects using
|
||||
external_id UUIDs instead of name-based identifiers.
|
||||
|
||||
Key improvements:
|
||||
- Direct project lookup via external_id UUIDs
|
||||
- Consistent with other v2 endpoints
|
||||
- Better performance through indexed queries
|
||||
"""
|
||||
|
||||
from typing import List, Optional
|
||||
|
||||
from fastapi import APIRouter, Query, Path
|
||||
|
||||
from basic_memory.deps import DirectoryServiceV2ExternalDep
|
||||
from basic_memory.schemas.directory import DirectoryNode
|
||||
|
||||
router = APIRouter(prefix="/directory", tags=["directory-v2"])
|
||||
|
||||
|
||||
@router.get("/tree", response_model=DirectoryNode, response_model_exclude_none=True)
|
||||
async def get_directory_tree(
|
||||
directory_service: DirectoryServiceV2ExternalDep,
|
||||
project_id: str = Path(..., description="Project external UUID"),
|
||||
):
|
||||
"""Get hierarchical directory structure from the knowledge base.
|
||||
|
||||
Args:
|
||||
directory_service: Service for directory operations
|
||||
project_id: Project external UUID
|
||||
|
||||
Returns:
|
||||
DirectoryNode representing the root of the hierarchical tree structure
|
||||
"""
|
||||
# Get a hierarchical directory tree for the specific project
|
||||
tree = await directory_service.get_directory_tree()
|
||||
|
||||
# Return the hierarchical tree
|
||||
return tree
|
||||
|
||||
|
||||
@router.get("/structure", response_model=DirectoryNode, response_model_exclude_none=True)
|
||||
async def get_directory_structure(
|
||||
directory_service: DirectoryServiceV2ExternalDep,
|
||||
project_id: str = Path(..., description="Project external UUID"),
|
||||
):
|
||||
"""Get folder structure for navigation (no files).
|
||||
|
||||
Optimized endpoint for folder tree navigation. Returns only directory nodes
|
||||
without file metadata. For full tree with files, use /directory/tree.
|
||||
|
||||
Args:
|
||||
directory_service: Service for directory operations
|
||||
project_id: Project external UUID
|
||||
|
||||
Returns:
|
||||
DirectoryNode tree containing only folders (type="directory")
|
||||
"""
|
||||
structure = await directory_service.get_directory_structure()
|
||||
return structure
|
||||
|
||||
|
||||
@router.get("/list", response_model=List[DirectoryNode], response_model_exclude_none=True)
|
||||
async def list_directory(
|
||||
directory_service: DirectoryServiceV2ExternalDep,
|
||||
project_id: str = Path(..., description="Project external UUID"),
|
||||
dir_name: str = Query("/", description="Directory path to list"),
|
||||
depth: int = Query(1, ge=1, le=10, description="Recursion depth (1-10)"),
|
||||
file_name_glob: Optional[str] = Query(
|
||||
None, description="Glob pattern for filtering file names"
|
||||
),
|
||||
):
|
||||
"""List directory contents with filtering and depth control.
|
||||
|
||||
Args:
|
||||
directory_service: Service for directory operations
|
||||
project_id: Project external UUID
|
||||
dir_name: Directory path to list (default: root "/")
|
||||
depth: Recursion depth (1-10, default: 1 for immediate children only)
|
||||
file_name_glob: Optional glob pattern for filtering file names (e.g., "*.md", "*meeting*")
|
||||
|
||||
Returns:
|
||||
List of DirectoryNode objects matching the criteria
|
||||
"""
|
||||
# Get directory listing with filtering
|
||||
nodes = await directory_service.list_directory(
|
||||
dir_name=dir_name,
|
||||
depth=depth,
|
||||
file_name_glob=file_name_glob,
|
||||
)
|
||||
|
||||
return nodes
|
||||
@@ -1,181 +0,0 @@
|
||||
"""V2 Import Router - ID-based data import operations.
|
||||
|
||||
This router uses v2 dependencies for consistent project handling with external_id UUIDs.
|
||||
Import endpoints use project_id in the path for consistency with other v2 endpoints.
|
||||
"""
|
||||
|
||||
import json
|
||||
import logging
|
||||
|
||||
from fastapi import APIRouter, Form, HTTPException, UploadFile, status, Path
|
||||
|
||||
from basic_memory.deps import (
|
||||
ChatGPTImporterV2ExternalDep,
|
||||
ClaudeConversationsImporterV2ExternalDep,
|
||||
ClaudeProjectsImporterV2ExternalDep,
|
||||
MemoryJsonImporterV2ExternalDep,
|
||||
)
|
||||
from basic_memory.importers import Importer
|
||||
from basic_memory.schemas.importer import (
|
||||
ChatImportResult,
|
||||
EntityImportResult,
|
||||
ProjectImportResult,
|
||||
)
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
router = APIRouter(prefix="/import", tags=["import-v2"])
|
||||
|
||||
|
||||
@router.post("/chatgpt", response_model=ChatImportResult)
|
||||
async def import_chatgpt(
|
||||
importer: ChatGPTImporterV2ExternalDep,
|
||||
file: UploadFile,
|
||||
project_id: str = Path(..., description="Project external UUID"),
|
||||
directory: str = Form("conversations"),
|
||||
) -> ChatImportResult:
|
||||
"""Import conversations from ChatGPT JSON export.
|
||||
|
||||
Args:
|
||||
project_id: Project external UUID from URL path
|
||||
file: The ChatGPT conversations.json file.
|
||||
directory: The directory to place the files in.
|
||||
importer: ChatGPT importer instance.
|
||||
|
||||
Returns:
|
||||
ChatImportResult with import statistics.
|
||||
|
||||
Raises:
|
||||
HTTPException: If import fails.
|
||||
"""
|
||||
logger.info(f"V2 Importing ChatGPT conversations for project {project_id}")
|
||||
return await import_file(importer, file, directory)
|
||||
|
||||
|
||||
@router.post("/claude/conversations", response_model=ChatImportResult)
|
||||
async def import_claude_conversations(
|
||||
importer: ClaudeConversationsImporterV2ExternalDep,
|
||||
file: UploadFile,
|
||||
project_id: str = Path(..., description="Project external UUID"),
|
||||
directory: str = Form("conversations"),
|
||||
) -> ChatImportResult:
|
||||
"""Import conversations from Claude conversations.json export.
|
||||
|
||||
Args:
|
||||
project_id: Project external UUID from URL path
|
||||
file: The Claude conversations.json file.
|
||||
directory: The directory to place the files in.
|
||||
importer: Claude conversations importer instance.
|
||||
|
||||
Returns:
|
||||
ChatImportResult with import statistics.
|
||||
|
||||
Raises:
|
||||
HTTPException: If import fails.
|
||||
"""
|
||||
logger.info(f"V2 Importing Claude conversations for project {project_id}")
|
||||
return await import_file(importer, file, directory)
|
||||
|
||||
|
||||
@router.post("/claude/projects", response_model=ProjectImportResult)
|
||||
async def import_claude_projects(
|
||||
importer: ClaudeProjectsImporterV2ExternalDep,
|
||||
file: UploadFile,
|
||||
project_id: str = Path(..., description="Project external UUID"),
|
||||
directory: str = Form("projects"),
|
||||
) -> ProjectImportResult:
|
||||
"""Import projects from Claude projects.json export.
|
||||
|
||||
Args:
|
||||
project_id: Project external UUID from URL path
|
||||
file: The Claude projects.json file.
|
||||
directory: The base directory to place the files in.
|
||||
importer: Claude projects importer instance.
|
||||
|
||||
Returns:
|
||||
ProjectImportResult with import statistics.
|
||||
|
||||
Raises:
|
||||
HTTPException: If import fails.
|
||||
"""
|
||||
logger.info(f"V2 Importing Claude projects for project {project_id}")
|
||||
return await import_file(importer, file, directory)
|
||||
|
||||
|
||||
@router.post("/memory-json", response_model=EntityImportResult)
|
||||
async def import_memory_json(
|
||||
importer: MemoryJsonImporterV2ExternalDep,
|
||||
file: UploadFile,
|
||||
project_id: str = Path(..., description="Project external UUID"),
|
||||
directory: str = Form("conversations"),
|
||||
) -> EntityImportResult:
|
||||
"""Import entities and relations from a memory.json file.
|
||||
|
||||
Args:
|
||||
project_id: Project external UUID from URL path
|
||||
file: The memory.json file.
|
||||
directory: Optional destination directory within the project.
|
||||
importer: Memory JSON importer instance.
|
||||
|
||||
Returns:
|
||||
EntityImportResult with import statistics.
|
||||
|
||||
Raises:
|
||||
HTTPException: If import fails.
|
||||
"""
|
||||
logger.info(f"V2 Importing memory.json for project {project_id}")
|
||||
try:
|
||||
file_data = []
|
||||
file_bytes = await file.read()
|
||||
file_str = file_bytes.decode("utf-8")
|
||||
for line in file_str.splitlines():
|
||||
json_data = json.loads(line)
|
||||
file_data.append(json_data)
|
||||
|
||||
result = await importer.import_data(file_data, directory)
|
||||
if not result.success: # pragma: no cover
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
|
||||
detail=result.error_message or "Import failed",
|
||||
)
|
||||
except Exception as e:
|
||||
logger.exception("V2 Import failed")
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
|
||||
detail=f"Import failed: {str(e)}",
|
||||
)
|
||||
return result
|
||||
|
||||
|
||||
async def import_file(importer: Importer, file: UploadFile, destination_directory: str):
|
||||
"""Helper function to import a file using an importer instance.
|
||||
|
||||
Args:
|
||||
importer: The importer instance to use
|
||||
file: The file to import
|
||||
destination_directory: Destination directory for imported content
|
||||
|
||||
Returns:
|
||||
Import result from the importer
|
||||
|
||||
Raises:
|
||||
HTTPException: If import fails
|
||||
"""
|
||||
try:
|
||||
# Process file
|
||||
json_data = json.load(file.file)
|
||||
result = await importer.import_data(json_data, destination_directory)
|
||||
if not result.success: # pragma: no cover
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
|
||||
detail=result.error_message or "Import failed",
|
||||
)
|
||||
|
||||
return result
|
||||
|
||||
except Exception as e:
|
||||
logger.exception("V2 Import failed")
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
|
||||
detail=f"Import failed: {str(e)}",
|
||||
)
|
||||
@@ -1,584 +0,0 @@
|
||||
"""V2 Knowledge Router - External ID-based entity operations.
|
||||
|
||||
This router provides external_id (UUID) based CRUD operations for entities,
|
||||
using stable string UUIDs that won't change with file moves or database migrations.
|
||||
|
||||
Key improvements:
|
||||
- Stable external UUIDs that won't change with file moves or renames
|
||||
- Better API ergonomics with consistent string identifiers
|
||||
- Direct database lookups via unique indexed column
|
||||
- Simplified caching strategies
|
||||
"""
|
||||
|
||||
from fastapi import APIRouter, HTTPException, BackgroundTasks, Depends, Response, Path, Query
|
||||
from loguru import logger
|
||||
|
||||
from basic_memory.deps import (
|
||||
EntityServiceV2ExternalDep,
|
||||
SearchServiceV2ExternalDep,
|
||||
LinkResolverV2ExternalDep,
|
||||
ProjectConfigV2ExternalDep,
|
||||
AppConfigDep,
|
||||
EntityRepositoryV2ExternalDep,
|
||||
ProjectExternalIdPathDep,
|
||||
TaskSchedulerDep,
|
||||
FileServiceV2ExternalDep,
|
||||
)
|
||||
from basic_memory.schemas import DeleteEntitiesResponse
|
||||
from basic_memory.schemas.base import Entity
|
||||
from basic_memory.schemas.request import EditEntityRequest
|
||||
from basic_memory.schemas.v2 import (
|
||||
EntityResolveRequest,
|
||||
EntityResolveResponse,
|
||||
EntityResponseV2,
|
||||
MoveEntityRequestV2,
|
||||
MoveDirectoryRequestV2,
|
||||
DeleteDirectoryRequestV2,
|
||||
)
|
||||
from basic_memory.schemas.response import DirectoryMoveResult, DirectoryDeleteResult
|
||||
|
||||
router = APIRouter(prefix="/knowledge", tags=["knowledge-v2"])
|
||||
|
||||
## Resolution endpoint
|
||||
|
||||
|
||||
@router.post("/resolve", response_model=EntityResolveResponse)
|
||||
async def resolve_identifier(
|
||||
project_id: ProjectExternalIdPathDep,
|
||||
data: EntityResolveRequest,
|
||||
link_resolver: LinkResolverV2ExternalDep,
|
||||
entity_repository: EntityRepositoryV2ExternalDep,
|
||||
) -> EntityResolveResponse:
|
||||
"""Resolve a string identifier (external_id, permalink, title, or path) to entity info.
|
||||
|
||||
This endpoint provides a bridge between v1-style identifiers and v2 external_ids.
|
||||
Use this to convert existing references to the new UUID-based format.
|
||||
|
||||
Args:
|
||||
data: Request containing the identifier to resolve
|
||||
|
||||
Returns:
|
||||
Entity external_id and metadata about how it was resolved
|
||||
|
||||
Raises:
|
||||
HTTPException: 404 if identifier cannot be resolved
|
||||
|
||||
Example:
|
||||
POST /v2/{project_id}/knowledge/resolve
|
||||
{"identifier": "specs/search"}
|
||||
|
||||
Returns:
|
||||
{
|
||||
"external_id": "550e8400-e29b-41d4-a716-446655440000",
|
||||
"entity_id": 123,
|
||||
"permalink": "specs/search",
|
||||
"file_path": "specs/search.md",
|
||||
"title": "Search Specification",
|
||||
"resolution_method": "permalink"
|
||||
}
|
||||
"""
|
||||
logger.info(f"API v2 request: resolve_identifier for '{data.identifier}'")
|
||||
|
||||
# Try to resolve by external_id first
|
||||
entity = await entity_repository.get_by_external_id(data.identifier)
|
||||
resolution_method = "external_id" if entity else "search"
|
||||
|
||||
# If not found by external_id, try other resolution methods
|
||||
# Pass source_path for context-aware resolution (prefers notes closer to source)
|
||||
# Pass strict to control fuzzy search fallback (default False allows fuzzy matching)
|
||||
if not entity:
|
||||
entity = await link_resolver.resolve_link(
|
||||
data.identifier, source_path=data.source_path, strict=data.strict
|
||||
)
|
||||
if entity:
|
||||
# Determine resolution method
|
||||
if entity.permalink == data.identifier:
|
||||
resolution_method = "permalink"
|
||||
elif entity.title == data.identifier:
|
||||
resolution_method = "title"
|
||||
elif entity.file_path == data.identifier:
|
||||
resolution_method = "path"
|
||||
else:
|
||||
resolution_method = "search"
|
||||
|
||||
if not entity:
|
||||
raise HTTPException(status_code=404, detail=f"Entity not found: '{data.identifier}'")
|
||||
|
||||
result = EntityResolveResponse(
|
||||
external_id=entity.external_id,
|
||||
entity_id=entity.id,
|
||||
permalink=entity.permalink,
|
||||
file_path=entity.file_path,
|
||||
title=entity.title,
|
||||
resolution_method=resolution_method,
|
||||
)
|
||||
|
||||
logger.info(
|
||||
f"API v2 response: resolved '{data.identifier}' to external_id={result.external_id} via {resolution_method}"
|
||||
)
|
||||
|
||||
return result
|
||||
|
||||
|
||||
## Read endpoints
|
||||
|
||||
|
||||
@router.get("/entities/{entity_id}", response_model=EntityResponseV2)
|
||||
async def get_entity_by_id(
|
||||
project_id: ProjectExternalIdPathDep,
|
||||
entity_repository: EntityRepositoryV2ExternalDep,
|
||||
entity_id: str = Path(..., description="Entity external ID (UUID)"),
|
||||
) -> EntityResponseV2:
|
||||
"""Get an entity by its external ID (UUID).
|
||||
|
||||
This is the primary entity retrieval method in v2, using stable UUID
|
||||
identifiers that won't change with file moves.
|
||||
|
||||
Args:
|
||||
entity_id: External ID (UUID string)
|
||||
|
||||
Returns:
|
||||
Complete entity with observations and relations
|
||||
|
||||
Raises:
|
||||
HTTPException: 404 if entity not found
|
||||
"""
|
||||
logger.info(f"API v2 request: get_entity_by_id entity_id={entity_id}")
|
||||
|
||||
entity = await entity_repository.get_by_external_id(entity_id)
|
||||
if not entity:
|
||||
raise HTTPException(
|
||||
status_code=404, detail=f"Entity with external_id '{entity_id}' not found"
|
||||
)
|
||||
|
||||
result = EntityResponseV2.model_validate(entity)
|
||||
logger.info(f"API v2 response: external_id={entity_id}, title='{result.title}'")
|
||||
|
||||
return result
|
||||
|
||||
|
||||
## Create endpoints
|
||||
|
||||
|
||||
@router.post("/entities", response_model=EntityResponseV2)
|
||||
async def create_entity(
|
||||
project_id: ProjectExternalIdPathDep,
|
||||
data: Entity,
|
||||
background_tasks: BackgroundTasks,
|
||||
entity_service: EntityServiceV2ExternalDep,
|
||||
search_service: SearchServiceV2ExternalDep,
|
||||
task_scheduler: TaskSchedulerDep,
|
||||
file_service: FileServiceV2ExternalDep,
|
||||
fast: bool = Query(
|
||||
True, description="If true, write quickly and defer indexing to background tasks."
|
||||
),
|
||||
) -> EntityResponseV2:
|
||||
"""Create a new entity.
|
||||
|
||||
Args:
|
||||
data: Entity data to create
|
||||
fast: If True, defer indexing to background tasks
|
||||
|
||||
Returns:
|
||||
Created entity with generated external_id (UUID) and file content
|
||||
"""
|
||||
logger.info(
|
||||
"API v2 request", endpoint="create_entity", entity_type=data.entity_type, title=data.title
|
||||
)
|
||||
|
||||
if fast:
|
||||
entity = await entity_service.fast_write_entity(data)
|
||||
task_scheduler.schedule(
|
||||
"reindex_entity",
|
||||
entity_id=entity.id,
|
||||
project_id=project_id,
|
||||
)
|
||||
else:
|
||||
entity = await entity_service.create_entity(data)
|
||||
await search_service.index_entity(entity, background_tasks=background_tasks)
|
||||
|
||||
result = EntityResponseV2.model_validate(entity)
|
||||
if fast:
|
||||
result = result.model_copy(update={"observations": [], "relations": []})
|
||||
|
||||
# Always read and return file content
|
||||
content = await file_service.read_file_content(entity.file_path)
|
||||
result = result.model_copy(update={"content": content})
|
||||
|
||||
logger.info(
|
||||
f"API v2 response: endpoint='create_entity' external_id={entity.external_id}, title={result.title}, permalink={result.permalink}, status_code=201"
|
||||
)
|
||||
return result
|
||||
|
||||
|
||||
## Update endpoints
|
||||
|
||||
|
||||
@router.put("/entities/{entity_id}", response_model=EntityResponseV2)
|
||||
async def update_entity_by_id(
|
||||
data: Entity,
|
||||
response: Response,
|
||||
background_tasks: BackgroundTasks,
|
||||
project_id: ProjectExternalIdPathDep,
|
||||
entity_service: EntityServiceV2ExternalDep,
|
||||
search_service: SearchServiceV2ExternalDep,
|
||||
entity_repository: EntityRepositoryV2ExternalDep,
|
||||
task_scheduler: TaskSchedulerDep,
|
||||
file_service: FileServiceV2ExternalDep,
|
||||
entity_id: str = Path(..., description="Entity external ID (UUID)"),
|
||||
fast: bool = Query(
|
||||
True, description="If true, write quickly and defer indexing to background tasks."
|
||||
),
|
||||
) -> EntityResponseV2:
|
||||
"""Update an entity by external ID.
|
||||
|
||||
If the entity doesn't exist, it will be created (upsert behavior).
|
||||
|
||||
Args:
|
||||
entity_id: External ID (UUID string)
|
||||
data: Updated entity data
|
||||
fast: If True, defer indexing to background tasks
|
||||
|
||||
Returns:
|
||||
Updated entity with file content
|
||||
"""
|
||||
logger.info(f"API v2 request: update_entity_by_id entity_id={entity_id}")
|
||||
|
||||
# Check if entity exists (external_id is the source of truth for v2)
|
||||
existing = await entity_repository.get_by_external_id(entity_id)
|
||||
created = existing is None
|
||||
|
||||
if fast:
|
||||
entity = await entity_service.fast_write_entity(data, external_id=entity_id)
|
||||
response.status_code = 200 if existing else 201
|
||||
task_scheduler.schedule(
|
||||
"reindex_entity",
|
||||
entity_id=entity.id,
|
||||
project_id=project_id,
|
||||
resolve_relations=created,
|
||||
)
|
||||
else:
|
||||
if existing:
|
||||
# Update the existing entity in-place to avoid path-based duplication
|
||||
entity = await entity_service.update_entity(existing, data)
|
||||
response.status_code = 200
|
||||
else:
|
||||
# Create new entity, then bind external_id to the requested UUID
|
||||
entity = await entity_service.create_entity(data)
|
||||
if entity.external_id != entity_id:
|
||||
entity = await entity_repository.update(
|
||||
entity.id,
|
||||
{"external_id": entity_id},
|
||||
)
|
||||
if not entity:
|
||||
raise HTTPException(
|
||||
status_code=404,
|
||||
detail=f"Entity with external_id '{entity_id}' not found",
|
||||
)
|
||||
response.status_code = 201
|
||||
|
||||
await search_service.index_entity(entity, background_tasks=background_tasks)
|
||||
|
||||
result = EntityResponseV2.model_validate(entity)
|
||||
if fast:
|
||||
result = result.model_copy(update={"observations": [], "relations": []})
|
||||
|
||||
# Always read and return file content
|
||||
content = await file_service.read_file_content(entity.file_path)
|
||||
result = result.model_copy(update={"content": content})
|
||||
|
||||
logger.info(
|
||||
f"API v2 response: external_id={entity_id}, created={created}, status_code={response.status_code}"
|
||||
)
|
||||
return result
|
||||
|
||||
|
||||
@router.patch("/entities/{entity_id}", response_model=EntityResponseV2)
|
||||
async def edit_entity_by_id(
|
||||
data: EditEntityRequest,
|
||||
background_tasks: BackgroundTasks,
|
||||
project_id: ProjectExternalIdPathDep,
|
||||
entity_service: EntityServiceV2ExternalDep,
|
||||
search_service: SearchServiceV2ExternalDep,
|
||||
entity_repository: EntityRepositoryV2ExternalDep,
|
||||
task_scheduler: TaskSchedulerDep,
|
||||
file_service: FileServiceV2ExternalDep,
|
||||
entity_id: str = Path(..., description="Entity external ID (UUID)"),
|
||||
fast: bool = Query(
|
||||
True, description="If true, write quickly and defer indexing to background tasks."
|
||||
),
|
||||
) -> EntityResponseV2:
|
||||
"""Edit an existing entity by external ID using operations like append, prepend, etc.
|
||||
|
||||
Args:
|
||||
entity_id: External ID (UUID string)
|
||||
data: Edit operation details
|
||||
fast: If True, defer indexing to background tasks
|
||||
|
||||
Returns:
|
||||
Updated entity with file content
|
||||
|
||||
Raises:
|
||||
HTTPException: 404 if entity not found, 400 if edit fails
|
||||
"""
|
||||
logger.info(
|
||||
f"API v2 request: edit_entity_by_id entity_id={entity_id}, operation='{data.operation}'"
|
||||
)
|
||||
|
||||
# Verify entity exists
|
||||
entity = await entity_repository.get_by_external_id(entity_id)
|
||||
if not entity: # pragma: no cover
|
||||
raise HTTPException(
|
||||
status_code=404, detail=f"Entity with external_id '{entity_id}' not found"
|
||||
)
|
||||
|
||||
try:
|
||||
if fast:
|
||||
updated_entity = await entity_service.fast_edit_entity(
|
||||
entity=entity,
|
||||
operation=data.operation,
|
||||
content=data.content,
|
||||
section=data.section,
|
||||
find_text=data.find_text,
|
||||
expected_replacements=data.expected_replacements,
|
||||
)
|
||||
task_scheduler.schedule(
|
||||
"reindex_entity",
|
||||
entity_id=updated_entity.id,
|
||||
project_id=project_id,
|
||||
)
|
||||
else:
|
||||
# Edit using the entity's permalink or path
|
||||
identifier = entity.permalink or entity.file_path
|
||||
updated_entity = await entity_service.edit_entity(
|
||||
identifier=identifier,
|
||||
operation=data.operation,
|
||||
content=data.content,
|
||||
section=data.section,
|
||||
find_text=data.find_text,
|
||||
expected_replacements=data.expected_replacements,
|
||||
)
|
||||
|
||||
await search_service.index_entity(updated_entity, background_tasks=background_tasks)
|
||||
|
||||
result = EntityResponseV2.model_validate(updated_entity)
|
||||
if fast:
|
||||
result = result.model_copy(update={"observations": [], "relations": []})
|
||||
|
||||
# Always read and return file content
|
||||
content = await file_service.read_file_content(updated_entity.file_path)
|
||||
result = result.model_copy(update={"content": content})
|
||||
|
||||
logger.info(
|
||||
f"API v2 response: external_id={entity_id}, operation='{data.operation}', status_code=200"
|
||||
)
|
||||
|
||||
return result
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Error editing entity {entity_id}: {e}")
|
||||
raise HTTPException(status_code=400, detail=str(e))
|
||||
|
||||
|
||||
## Delete endpoints
|
||||
|
||||
|
||||
@router.delete("/entities/{entity_id}", response_model=DeleteEntitiesResponse)
|
||||
async def delete_entity_by_id(
|
||||
background_tasks: BackgroundTasks,
|
||||
project_id: ProjectExternalIdPathDep,
|
||||
entity_service: EntityServiceV2ExternalDep,
|
||||
entity_repository: EntityRepositoryV2ExternalDep,
|
||||
entity_id: str = Path(..., description="Entity external ID (UUID)"),
|
||||
search_service=Depends(lambda: None), # Optional for now
|
||||
) -> DeleteEntitiesResponse:
|
||||
"""Delete an entity by external ID.
|
||||
|
||||
Args:
|
||||
entity_id: External ID (UUID string)
|
||||
|
||||
Returns:
|
||||
Deletion status
|
||||
|
||||
Note: Returns deleted=False if entity doesn't exist (idempotent)
|
||||
"""
|
||||
logger.info(f"API v2 request: delete_entity_by_id entity_id={entity_id}")
|
||||
|
||||
entity = await entity_repository.get_by_external_id(entity_id)
|
||||
if entity is None:
|
||||
logger.info(f"API v2 response: external_id={entity_id} not found, deleted=False")
|
||||
return DeleteEntitiesResponse(deleted=False)
|
||||
|
||||
# Delete the entity using internal ID
|
||||
deleted = await entity_service.delete_entity(entity.id)
|
||||
|
||||
# Remove from search index if search service available
|
||||
if search_service:
|
||||
background_tasks.add_task(search_service.handle_delete, entity) # pragma: no cover
|
||||
|
||||
logger.info(f"API v2 response: external_id={entity_id}, deleted={deleted}")
|
||||
|
||||
return DeleteEntitiesResponse(deleted=deleted)
|
||||
|
||||
|
||||
## Move endpoint
|
||||
|
||||
|
||||
@router.put("/entities/{entity_id}/move", response_model=EntityResponseV2)
|
||||
async def move_entity(
|
||||
data: MoveEntityRequestV2,
|
||||
background_tasks: BackgroundTasks,
|
||||
project_id: ProjectExternalIdPathDep,
|
||||
entity_service: EntityServiceV2ExternalDep,
|
||||
entity_repository: EntityRepositoryV2ExternalDep,
|
||||
project_config: ProjectConfigV2ExternalDep,
|
||||
app_config: AppConfigDep,
|
||||
search_service: SearchServiceV2ExternalDep,
|
||||
entity_id: str = Path(..., description="Entity external ID (UUID)"),
|
||||
) -> EntityResponseV2:
|
||||
"""Move an entity to a new file location.
|
||||
|
||||
V2 API uses external_id (UUID) in the URL path for stable references.
|
||||
The external_id will remain stable after the move.
|
||||
|
||||
Args:
|
||||
project_id: Project external ID from URL path
|
||||
entity_id: Entity external ID from URL path (primary identifier)
|
||||
data: Move request with destination path only
|
||||
|
||||
Returns:
|
||||
Updated entity with new file path
|
||||
"""
|
||||
logger.info(
|
||||
f"API v2 request: move_entity entity_id={entity_id}, destination='{data.destination_path}'"
|
||||
)
|
||||
|
||||
try:
|
||||
# First, get the entity by external_id to verify it exists
|
||||
entity = await entity_repository.get_by_external_id(entity_id)
|
||||
if not entity: # pragma: no cover
|
||||
raise HTTPException(
|
||||
status_code=404, detail=f"Entity with external_id '{entity_id}' not found"
|
||||
)
|
||||
|
||||
# Move the entity using its current file path as identifier
|
||||
moved_entity = await entity_service.move_entity(
|
||||
identifier=entity.file_path, # Use file path for resolution
|
||||
destination_path=data.destination_path,
|
||||
project_config=project_config,
|
||||
app_config=app_config,
|
||||
)
|
||||
|
||||
# Reindex at new location
|
||||
reindexed_entity = await entity_service.link_resolver.resolve_link(data.destination_path)
|
||||
if reindexed_entity:
|
||||
await search_service.index_entity(reindexed_entity, background_tasks=background_tasks)
|
||||
|
||||
result = EntityResponseV2.model_validate(moved_entity)
|
||||
|
||||
logger.info(f"API v2 response: moved external_id={entity_id} to '{data.destination_path}'")
|
||||
|
||||
return result
|
||||
|
||||
except HTTPException: # pragma: no cover
|
||||
raise # pragma: no cover
|
||||
except Exception as e:
|
||||
logger.error(f"Error moving entity: {e}")
|
||||
raise HTTPException(status_code=400, detail=str(e))
|
||||
|
||||
|
||||
## Move directory endpoint
|
||||
|
||||
|
||||
@router.post("/move-directory", response_model=DirectoryMoveResult)
|
||||
async def move_directory(
|
||||
data: MoveDirectoryRequestV2,
|
||||
background_tasks: BackgroundTasks,
|
||||
project_id: ProjectExternalIdPathDep,
|
||||
entity_service: EntityServiceV2ExternalDep,
|
||||
project_config: ProjectConfigV2ExternalDep,
|
||||
app_config: AppConfigDep,
|
||||
search_service: SearchServiceV2ExternalDep,
|
||||
) -> DirectoryMoveResult:
|
||||
"""Move all entities in a directory to a new location.
|
||||
|
||||
V2 API uses project external_id in the URL path for stable references.
|
||||
Moves all files within a source directory to a destination directory,
|
||||
updating database records and optionally updating permalinks.
|
||||
|
||||
Args:
|
||||
project_id: Project external ID from URL path
|
||||
data: Move request with source and destination directories
|
||||
|
||||
Returns:
|
||||
DirectoryMoveResult with counts and details of moved files
|
||||
"""
|
||||
logger.info(
|
||||
f"API v2 request: move_directory source='{data.source_directory}', destination='{data.destination_directory}'"
|
||||
)
|
||||
|
||||
try:
|
||||
# Move the directory using the service
|
||||
result = await entity_service.move_directory(
|
||||
source_directory=data.source_directory,
|
||||
destination_directory=data.destination_directory,
|
||||
project_config=project_config,
|
||||
app_config=app_config,
|
||||
)
|
||||
|
||||
# Reindex moved entities
|
||||
for file_path in result.moved_files:
|
||||
entity = await entity_service.link_resolver.resolve_link(file_path)
|
||||
if entity:
|
||||
await search_service.index_entity(entity, background_tasks=background_tasks)
|
||||
|
||||
logger.info(
|
||||
f"API v2 response: move_directory "
|
||||
f"total={result.total_files}, success={result.successful_moves}, failed={result.failed_moves}"
|
||||
)
|
||||
return result
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Error moving directory: {e}")
|
||||
raise HTTPException(status_code=400, detail=str(e))
|
||||
|
||||
|
||||
## Delete directory endpoint
|
||||
|
||||
|
||||
@router.post("/delete-directory", response_model=DirectoryDeleteResult)
|
||||
async def delete_directory(
|
||||
data: DeleteDirectoryRequestV2,
|
||||
project_id: ProjectExternalIdPathDep,
|
||||
entity_service: EntityServiceV2ExternalDep,
|
||||
) -> DirectoryDeleteResult:
|
||||
"""Delete all entities in a directory.
|
||||
|
||||
V2 API uses project external_id in the URL path for stable references.
|
||||
Deletes all files within a directory, updating database records and
|
||||
removing files from the filesystem.
|
||||
|
||||
Args:
|
||||
project_id: Project external ID from URL path
|
||||
data: Delete request with directory path
|
||||
|
||||
Returns:
|
||||
DirectoryDeleteResult with counts and details of deleted files
|
||||
"""
|
||||
logger.info(f"API v2 request: delete_directory directory='{data.directory}'")
|
||||
|
||||
try:
|
||||
# Delete the directory using the service
|
||||
result = await entity_service.delete_directory(
|
||||
directory=data.directory,
|
||||
)
|
||||
|
||||
logger.info(
|
||||
f"API v2 response: delete_directory "
|
||||
f"total={result.total_files}, success={result.successful_deletes}, failed={result.failed_deletes}"
|
||||
)
|
||||
return result
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Error deleting directory: {e}")
|
||||
raise HTTPException(status_code=400, detail=str(e))
|
||||
@@ -1,130 +0,0 @@
|
||||
"""V2 routes for memory:// URI operations.
|
||||
|
||||
This router uses external_id UUIDs for stable, API-friendly routing.
|
||||
V1 uses string-based project names which are less efficient and less stable.
|
||||
"""
|
||||
|
||||
from typing import Annotated, Optional
|
||||
|
||||
from fastapi import APIRouter, Query, Path
|
||||
from loguru import logger
|
||||
|
||||
from basic_memory.deps import ContextServiceV2ExternalDep, EntityRepositoryV2ExternalDep
|
||||
from basic_memory.schemas.base import TimeFrame, parse_timeframe
|
||||
from basic_memory.schemas.memory import (
|
||||
GraphContext,
|
||||
normalize_memory_url,
|
||||
)
|
||||
from basic_memory.schemas.search import SearchItemType
|
||||
from basic_memory.api.v2.utils import to_graph_context
|
||||
|
||||
# Note: No prefix here - it's added during registration as /v2/{project_id}/memory
|
||||
router = APIRouter(tags=["memory"])
|
||||
|
||||
|
||||
@router.get("/memory/recent", response_model=GraphContext)
|
||||
async def recent(
|
||||
context_service: ContextServiceV2ExternalDep,
|
||||
entity_repository: EntityRepositoryV2ExternalDep,
|
||||
project_id: str = Path(..., description="Project external UUID"),
|
||||
type: Annotated[list[SearchItemType] | None, Query()] = None,
|
||||
depth: int = 1,
|
||||
timeframe: TimeFrame = "7d",
|
||||
page: int = 1,
|
||||
page_size: int = 10,
|
||||
max_related: int = 10,
|
||||
) -> GraphContext:
|
||||
"""Get recent activity context for a project.
|
||||
|
||||
Args:
|
||||
project_id: Project external UUID from URL path
|
||||
context_service: Context service scoped to project
|
||||
entity_repository: Entity repository scoped to project
|
||||
type: Types of items to include (entities, relations, observations)
|
||||
depth: How many levels of related entities to include
|
||||
timeframe: Time window for recent activity (e.g., "7d", "1 week")
|
||||
page: Page number for pagination
|
||||
page_size: Number of items per page
|
||||
max_related: Maximum related entities to include per item
|
||||
|
||||
Returns:
|
||||
GraphContext with recent activity and related entities
|
||||
"""
|
||||
# return all types by default
|
||||
types = (
|
||||
[SearchItemType.ENTITY, SearchItemType.RELATION, SearchItemType.OBSERVATION]
|
||||
if not type
|
||||
else type
|
||||
)
|
||||
|
||||
logger.debug(
|
||||
f"V2 Getting recent context for project {project_id}: `{types}` depth: `{depth}` timeframe: `{timeframe}` page: `{page}` page_size: `{page_size}` max_related: `{max_related}`"
|
||||
)
|
||||
# Parse timeframe
|
||||
since = parse_timeframe(timeframe)
|
||||
limit = page_size
|
||||
offset = (page - 1) * page_size
|
||||
|
||||
# Build context
|
||||
context = await context_service.build_context(
|
||||
types=types, depth=depth, since=since, limit=limit, offset=offset, max_related=max_related
|
||||
)
|
||||
recent_context = await to_graph_context(
|
||||
context, entity_repository=entity_repository, page=page, page_size=page_size
|
||||
)
|
||||
logger.debug(f"V2 Recent context: {recent_context.model_dump_json()}")
|
||||
return recent_context
|
||||
|
||||
|
||||
# get_memory_context needs to be declared last so other paths can match
|
||||
|
||||
|
||||
@router.get("/memory/{uri:path}", response_model=GraphContext)
|
||||
async def get_memory_context(
|
||||
context_service: ContextServiceV2ExternalDep,
|
||||
entity_repository: EntityRepositoryV2ExternalDep,
|
||||
uri: str,
|
||||
project_id: str = Path(..., description="Project external UUID"),
|
||||
depth: int = 1,
|
||||
timeframe: Optional[TimeFrame] = None,
|
||||
page: int = 1,
|
||||
page_size: int = 10,
|
||||
max_related: int = 10,
|
||||
) -> GraphContext:
|
||||
"""Get rich context from memory:// URI.
|
||||
|
||||
V2 supports both legacy path-based URIs and new ID-based URIs:
|
||||
- Legacy: memory://path/to/note
|
||||
- ID-based: memory://id/123 or memory://123
|
||||
|
||||
Args:
|
||||
project_id: Project external UUID from URL path
|
||||
context_service: Context service scoped to project
|
||||
entity_repository: Entity repository scoped to project
|
||||
uri: Memory URI path (e.g., "id/123", "123", or "path/to/note")
|
||||
depth: How many levels of related entities to include
|
||||
timeframe: Optional time window for filtering related content
|
||||
page: Page number for pagination
|
||||
page_size: Number of items per page
|
||||
max_related: Maximum related entities to include
|
||||
|
||||
Returns:
|
||||
GraphContext with the entity and its related context
|
||||
"""
|
||||
logger.debug(
|
||||
f"V2 Getting context for project {project_id}, URI: `{uri}` depth: `{depth}` timeframe: `{timeframe}` page: `{page}` page_size: `{page_size}` max_related: `{max_related}`"
|
||||
)
|
||||
memory_url = normalize_memory_url(uri)
|
||||
|
||||
# Parse timeframe
|
||||
since = parse_timeframe(timeframe) if timeframe else None
|
||||
limit = page_size
|
||||
offset = (page - 1) * page_size
|
||||
|
||||
# Build context
|
||||
context = await context_service.build_context(
|
||||
memory_url, depth=depth, since=since, limit=limit, offset=offset, max_related=max_related
|
||||
)
|
||||
return await to_graph_context(
|
||||
context, entity_repository=entity_repository, page=page, page_size=page_size
|
||||
)
|
||||
@@ -1,550 +0,0 @@
|
||||
"""V2 Project Router - External ID-based project management operations.
|
||||
|
||||
This router provides external_id (UUID) based CRUD operations for projects,
|
||||
using stable string UUIDs that never change (unlike integer IDs or names).
|
||||
|
||||
Key improvements:
|
||||
- Stable external UUIDs that won't change with renames or database migrations
|
||||
- Better API ergonomics with consistent string identifiers
|
||||
- Direct database lookups via unique indexed column
|
||||
- Consistent with v2 entity operations
|
||||
"""
|
||||
|
||||
import os
|
||||
from typing import Optional
|
||||
|
||||
from fastapi import APIRouter, HTTPException, Body, Query, Path
|
||||
from loguru import logger
|
||||
|
||||
from basic_memory.deps import (
|
||||
ProjectServiceDep,
|
||||
ProjectRepositoryDep,
|
||||
ProjectConfigV2ExternalDep,
|
||||
SyncServiceV2ExternalDep,
|
||||
TaskSchedulerDep,
|
||||
ProjectExternalIdPathDep,
|
||||
)
|
||||
from basic_memory.schemas import SyncReportResponse
|
||||
from basic_memory.schemas.project_info import (
|
||||
ProjectItem,
|
||||
ProjectList,
|
||||
ProjectInfoRequest,
|
||||
ProjectInfoResponse,
|
||||
ProjectStatusResponse,
|
||||
)
|
||||
from basic_memory.schemas.v2 import ProjectResolveRequest, ProjectResolveResponse
|
||||
from basic_memory.utils import normalize_project_path, generate_permalink
|
||||
|
||||
router = APIRouter(prefix="/projects", tags=["project_management-v2"])
|
||||
|
||||
|
||||
@router.get("/", response_model=ProjectList)
|
||||
async def list_projects(
|
||||
project_service: ProjectServiceDep,
|
||||
) -> ProjectList:
|
||||
"""List all configured projects.
|
||||
|
||||
Returns:
|
||||
A list of all projects with metadata
|
||||
"""
|
||||
projects = await project_service.list_projects()
|
||||
default_project = project_service.default_project
|
||||
|
||||
project_items = [
|
||||
ProjectItem(
|
||||
id=project.id,
|
||||
external_id=project.external_id,
|
||||
name=project.name,
|
||||
path=normalize_project_path(project.path),
|
||||
is_default=project.is_default or False,
|
||||
)
|
||||
for project in projects
|
||||
]
|
||||
|
||||
return ProjectList(
|
||||
projects=project_items,
|
||||
default_project=default_project,
|
||||
)
|
||||
|
||||
|
||||
@router.post("/", response_model=ProjectStatusResponse, status_code=201)
|
||||
async def add_project(
|
||||
project_data: ProjectInfoRequest,
|
||||
project_service: ProjectServiceDep,
|
||||
) -> ProjectStatusResponse:
|
||||
"""Add a new project to configuration and database.
|
||||
|
||||
Args:
|
||||
project_data: The project name and path, with option to set as default
|
||||
|
||||
Returns:
|
||||
Response confirming the project was added
|
||||
"""
|
||||
# Check if project already exists before attempting to add
|
||||
existing_project = await project_service.get_project(project_data.name)
|
||||
if existing_project:
|
||||
# Project exists - check if paths match for true idempotency
|
||||
# Normalize paths for comparison (resolve symlinks, etc.)
|
||||
requested_path = os.path.abspath(os.path.expanduser(project_data.path))
|
||||
existing_path = os.path.abspath(os.path.expanduser(existing_project.path))
|
||||
|
||||
if requested_path == existing_path:
|
||||
# Same name, same path - return 200 OK (idempotent)
|
||||
return ProjectStatusResponse( # pyright: ignore [reportCallIssue]
|
||||
message=f"Project '{project_data.name}' already exists",
|
||||
status="success",
|
||||
default=existing_project.is_default or False,
|
||||
new_project=ProjectItem(
|
||||
id=existing_project.id,
|
||||
external_id=existing_project.external_id,
|
||||
name=existing_project.name,
|
||||
path=existing_project.path,
|
||||
is_default=existing_project.is_default or False,
|
||||
),
|
||||
)
|
||||
else:
|
||||
# Same name, different path - this is an error
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
detail=(
|
||||
f"Project '{project_data.name}' already exists with different path. "
|
||||
f"Existing: {existing_project.path}, Requested: {project_data.path}"
|
||||
),
|
||||
)
|
||||
|
||||
try: # pragma: no cover
|
||||
# The service layer handles cloud mode validation and path sanitization
|
||||
await project_service.add_project(
|
||||
project_data.name, project_data.path, set_default=project_data.set_default
|
||||
)
|
||||
|
||||
# Fetch the newly created project to get its ID
|
||||
new_project = await project_service.get_project(project_data.name)
|
||||
if not new_project:
|
||||
raise HTTPException(status_code=500, detail="Failed to retrieve newly created project")
|
||||
|
||||
return ProjectStatusResponse( # pyright: ignore [reportCallIssue]
|
||||
message=f"Project '{new_project.name}' added successfully",
|
||||
status="success",
|
||||
default=project_data.set_default,
|
||||
new_project=ProjectItem(
|
||||
id=new_project.id,
|
||||
external_id=new_project.external_id,
|
||||
name=new_project.name,
|
||||
path=new_project.path,
|
||||
is_default=new_project.is_default or False,
|
||||
),
|
||||
)
|
||||
except ValueError as e: # pragma: no cover
|
||||
raise HTTPException(status_code=400, detail=str(e))
|
||||
|
||||
|
||||
@router.post("/config/sync", response_model=ProjectStatusResponse)
|
||||
async def synchronize_projects(
|
||||
project_service: ProjectServiceDep,
|
||||
) -> ProjectStatusResponse:
|
||||
"""Synchronize projects between configuration file and database."""
|
||||
try: # pragma: no cover
|
||||
await project_service.synchronize_projects()
|
||||
|
||||
return ProjectStatusResponse( # pyright: ignore [reportCallIssue]
|
||||
message="Projects synchronized successfully between configuration and database",
|
||||
status="success",
|
||||
default=False,
|
||||
)
|
||||
except ValueError as e: # pragma: no cover
|
||||
raise HTTPException(status_code=400, detail=str(e))
|
||||
|
||||
|
||||
@router.post("/{project_id}/sync")
|
||||
async def sync_project(
|
||||
sync_service: SyncServiceV2ExternalDep,
|
||||
project_config: ProjectConfigV2ExternalDep,
|
||||
task_scheduler: TaskSchedulerDep,
|
||||
project_internal_id: ProjectExternalIdPathDep,
|
||||
force_full: bool = Query(
|
||||
False, description="Force full scan, bypassing watermark optimization"
|
||||
),
|
||||
run_in_background: bool = Query(True, description="Run in background"),
|
||||
):
|
||||
"""Force project filesystem sync to database."""
|
||||
if run_in_background:
|
||||
task_scheduler.schedule(
|
||||
"sync_project",
|
||||
project_id=project_internal_id,
|
||||
force_full=force_full,
|
||||
)
|
||||
logger.info(
|
||||
f"Filesystem sync initiated for project: {project_config.name} (force_full={force_full})"
|
||||
)
|
||||
|
||||
return {
|
||||
"status": "sync_started",
|
||||
"message": f"Filesystem sync initiated for project '{project_config.name}'",
|
||||
}
|
||||
|
||||
report = await sync_service.sync(
|
||||
project_config.home, project_config.name, force_full=force_full
|
||||
)
|
||||
logger.info(
|
||||
f"Filesystem sync completed for project: {project_config.name} (force_full={force_full})"
|
||||
)
|
||||
return SyncReportResponse.from_sync_report(report)
|
||||
|
||||
|
||||
@router.post("/{project_id}/status", response_model=SyncReportResponse)
|
||||
async def get_project_status(
|
||||
sync_service: SyncServiceV2ExternalDep,
|
||||
project_config: ProjectConfigV2ExternalDep,
|
||||
project_id: str = Path(..., description="Project external ID (UUID)"),
|
||||
force_full: bool = Query(
|
||||
False, description="Force full scan, bypassing watermark optimization"
|
||||
),
|
||||
) -> SyncReportResponse:
|
||||
"""Get sync status of files vs database for a project."""
|
||||
logger.info(f"API v2 request: get_project_status for project_id={project_id}")
|
||||
report = await sync_service.scan(project_config.home, force_full=force_full)
|
||||
return SyncReportResponse.from_sync_report(report)
|
||||
|
||||
|
||||
@router.post("/resolve", response_model=ProjectResolveResponse)
|
||||
async def resolve_project_identifier(
|
||||
data: ProjectResolveRequest,
|
||||
project_repository: ProjectRepositoryDep,
|
||||
) -> ProjectResolveResponse:
|
||||
"""Resolve a project identifier (name, permalink, or external_id) to project info.
|
||||
|
||||
This endpoint provides efficient lookup of projects by various identifiers
|
||||
without needing to fetch the entire project list. Supports:
|
||||
- External ID (UUID string) - preferred stable identifier
|
||||
- Permalink
|
||||
- Case-insensitive name matching
|
||||
|
||||
Args:
|
||||
data: Request containing the identifier to resolve
|
||||
|
||||
Returns:
|
||||
Project information including the external_id (UUID)
|
||||
|
||||
Raises:
|
||||
HTTPException: 404 if project not found
|
||||
|
||||
Example:
|
||||
POST /v2/projects/resolve
|
||||
{"identifier": "my-project"}
|
||||
|
||||
Returns:
|
||||
{
|
||||
"external_id": "550e8400-e29b-41d4-a716-446655440000",
|
||||
"project_id": 1,
|
||||
"name": "my-project",
|
||||
"permalink": "my-project",
|
||||
"path": "/path/to/project",
|
||||
"is_active": true,
|
||||
"is_default": false,
|
||||
"resolution_method": "name"
|
||||
}
|
||||
"""
|
||||
logger.info(f"API v2 request: resolve_project_identifier for '{data.identifier}'")
|
||||
|
||||
# Generate permalink for comparison
|
||||
identifier_permalink = generate_permalink(data.identifier)
|
||||
|
||||
resolution_method = "name"
|
||||
project = None
|
||||
|
||||
# Try external_id first (UUID format)
|
||||
project = await project_repository.get_by_external_id(data.identifier)
|
||||
if project:
|
||||
resolution_method = "external_id"
|
||||
|
||||
# If not found by external_id, try by permalink (exact match)
|
||||
if not project:
|
||||
project = await project_repository.get_by_permalink(identifier_permalink)
|
||||
if project:
|
||||
resolution_method = "permalink"
|
||||
|
||||
# If not found by permalink, try case-insensitive name search
|
||||
if not project:
|
||||
project = await project_repository.get_by_name_case_insensitive(data.identifier)
|
||||
if project:
|
||||
resolution_method = "name" # pragma: no cover
|
||||
|
||||
if not project:
|
||||
raise HTTPException(status_code=404, detail=f"Project not found: '{data.identifier}'")
|
||||
|
||||
return ProjectResolveResponse(
|
||||
external_id=project.external_id,
|
||||
project_id=project.id,
|
||||
name=project.name,
|
||||
permalink=generate_permalink(project.name),
|
||||
path=normalize_project_path(project.path),
|
||||
is_active=project.is_active if hasattr(project, "is_active") else True,
|
||||
is_default=project.is_default or False,
|
||||
resolution_method=resolution_method,
|
||||
)
|
||||
|
||||
|
||||
@router.get("/{project_id}", response_model=ProjectItem)
|
||||
async def get_project_by_id(
|
||||
project_repository: ProjectRepositoryDep,
|
||||
project_id: str = Path(..., description="Project external ID (UUID)"),
|
||||
) -> ProjectItem:
|
||||
"""Get project by its external ID (UUID).
|
||||
|
||||
This is the primary project retrieval method in v2, using stable UUID
|
||||
identifiers that won't change with project renames.
|
||||
|
||||
Args:
|
||||
project_id: External ID (UUID string)
|
||||
|
||||
Returns:
|
||||
Project information including external_id
|
||||
|
||||
Raises:
|
||||
HTTPException: 404 if project not found
|
||||
|
||||
Example:
|
||||
GET /v2/projects/550e8400-e29b-41d4-a716-446655440000
|
||||
"""
|
||||
logger.info(f"API v2 request: get_project_by_id for project_id={project_id}")
|
||||
|
||||
project = await project_repository.get_by_external_id(project_id)
|
||||
if not project:
|
||||
raise HTTPException(
|
||||
status_code=404, detail=f"Project with external_id '{project_id}' not found"
|
||||
)
|
||||
|
||||
return ProjectItem(
|
||||
id=project.id,
|
||||
external_id=project.external_id,
|
||||
name=project.name,
|
||||
path=normalize_project_path(project.path),
|
||||
is_default=project.is_default or False,
|
||||
)
|
||||
|
||||
|
||||
@router.get("/{project_id}/info", response_model=ProjectInfoResponse)
|
||||
async def get_project_info_by_id(
|
||||
project_service: ProjectServiceDep,
|
||||
project_repository: ProjectRepositoryDep,
|
||||
project_id: str = Path(..., description="Project external ID (UUID)"),
|
||||
) -> ProjectInfoResponse:
|
||||
"""Get detailed project information by external ID."""
|
||||
logger.info(f"API v2 request: get_project_info_by_id for project_id={project_id}")
|
||||
project = await project_repository.get_by_external_id(project_id)
|
||||
if not project:
|
||||
raise HTTPException(
|
||||
status_code=404, detail=f"Project with external_id '{project_id}' not found"
|
||||
)
|
||||
return await project_service.get_project_info(project.name)
|
||||
|
||||
|
||||
@router.patch("/{project_id}", response_model=ProjectStatusResponse)
|
||||
async def update_project_by_id(
|
||||
project_service: ProjectServiceDep,
|
||||
project_repository: ProjectRepositoryDep,
|
||||
project_id: str = Path(..., description="Project external ID (UUID)"),
|
||||
path: Optional[str] = Body(None, description="New absolute path for the project"),
|
||||
is_active: Optional[bool] = Body(None, description="Status of the project (active/inactive)"),
|
||||
) -> ProjectStatusResponse:
|
||||
"""Update a project's information by external ID.
|
||||
|
||||
Args:
|
||||
project_id: External ID (UUID string)
|
||||
path: Optional new absolute path for the project
|
||||
is_active: Optional status update for the project
|
||||
|
||||
Returns:
|
||||
Response confirming the project was updated
|
||||
|
||||
Raises:
|
||||
HTTPException: 400 if validation fails, 404 if project not found
|
||||
|
||||
Example:
|
||||
PATCH /v2/projects/550e8400-e29b-41d4-a716-446655440000
|
||||
{"path": "/new/path"}
|
||||
"""
|
||||
logger.info(f"API v2 request: update_project_by_id for project_id={project_id}")
|
||||
|
||||
try:
|
||||
# Validate that path is absolute if provided
|
||||
if path and not os.path.isabs(path):
|
||||
raise HTTPException(status_code=400, detail="Path must be absolute")
|
||||
|
||||
# Get original project info for the response
|
||||
old_project = await project_repository.get_by_external_id(project_id)
|
||||
if not old_project:
|
||||
raise HTTPException(
|
||||
status_code=404, detail=f"Project with external_id '{project_id}' not found"
|
||||
)
|
||||
|
||||
old_project_info = ProjectItem(
|
||||
id=old_project.id,
|
||||
external_id=old_project.external_id,
|
||||
name=old_project.name,
|
||||
path=old_project.path,
|
||||
is_default=old_project.is_default or False,
|
||||
)
|
||||
|
||||
# Update using project name (service layer still uses names internally)
|
||||
if path:
|
||||
await project_service.move_project(old_project.name, path)
|
||||
elif is_active is not None:
|
||||
await project_service.update_project(old_project.name, is_active=is_active)
|
||||
|
||||
# Get updated project info (use the same external_id)
|
||||
updated_project = await project_repository.get_by_external_id(project_id)
|
||||
if not updated_project: # pragma: no cover
|
||||
raise HTTPException(
|
||||
status_code=404,
|
||||
detail=f"Project with external_id '{project_id}' not found after update",
|
||||
)
|
||||
|
||||
return ProjectStatusResponse(
|
||||
message=f"Project '{updated_project.name}' updated successfully",
|
||||
status="success",
|
||||
default=old_project.is_default or False,
|
||||
old_project=old_project_info,
|
||||
new_project=ProjectItem(
|
||||
id=updated_project.id,
|
||||
external_id=updated_project.external_id,
|
||||
name=updated_project.name,
|
||||
path=updated_project.path,
|
||||
is_default=updated_project.is_default or False,
|
||||
),
|
||||
)
|
||||
except ValueError as e: # pragma: no cover
|
||||
raise HTTPException(status_code=400, detail=str(e)) # pragma: no cover
|
||||
|
||||
|
||||
@router.delete("/{project_id}", response_model=ProjectStatusResponse)
|
||||
async def delete_project_by_id(
|
||||
project_service: ProjectServiceDep,
|
||||
project_repository: ProjectRepositoryDep,
|
||||
project_id: str = Path(..., description="Project external ID (UUID)"),
|
||||
delete_notes: bool = Query(
|
||||
False, description="If True, delete project directory from filesystem"
|
||||
),
|
||||
) -> ProjectStatusResponse:
|
||||
"""Delete a project by external ID.
|
||||
|
||||
Args:
|
||||
project_id: External ID (UUID string)
|
||||
delete_notes: If True, delete the project directory from the filesystem
|
||||
|
||||
Returns:
|
||||
Response confirming the project was deleted
|
||||
|
||||
Raises:
|
||||
HTTPException: 400 if trying to delete default project, 404 if not found
|
||||
|
||||
Example:
|
||||
DELETE /v2/projects/550e8400-e29b-41d4-a716-446655440000?delete_notes=false
|
||||
"""
|
||||
logger.info(
|
||||
f"API v2 request: delete_project_by_id for project_id={project_id}, delete_notes={delete_notes}"
|
||||
)
|
||||
|
||||
try:
|
||||
old_project = await project_repository.get_by_external_id(project_id)
|
||||
if not old_project:
|
||||
raise HTTPException(
|
||||
status_code=404, detail=f"Project with external_id '{project_id}' not found"
|
||||
)
|
||||
|
||||
# Check if trying to delete the default project
|
||||
# Use is_default from database, not ConfigManager (which doesn't work in cloud mode)
|
||||
if old_project.is_default:
|
||||
available_projects = await project_service.list_projects()
|
||||
other_projects = [p.name for p in available_projects if p.external_id != project_id]
|
||||
detail = f"Cannot delete default project '{old_project.name}'. "
|
||||
if other_projects:
|
||||
detail += ( # pragma: no cover
|
||||
f"Set another project as default first. Available: {', '.join(other_projects)}"
|
||||
)
|
||||
else:
|
||||
detail += "This is the only project in your configuration." # pragma: no cover
|
||||
raise HTTPException(status_code=400, detail=detail)
|
||||
|
||||
# Delete using project name (service layer still uses names internally)
|
||||
await project_service.remove_project(old_project.name, delete_notes=delete_notes)
|
||||
|
||||
return ProjectStatusResponse(
|
||||
message=f"Project '{old_project.name}' removed successfully",
|
||||
status="success",
|
||||
default=False,
|
||||
old_project=ProjectItem(
|
||||
id=old_project.id,
|
||||
external_id=old_project.external_id,
|
||||
name=old_project.name,
|
||||
path=old_project.path,
|
||||
is_default=old_project.is_default or False,
|
||||
),
|
||||
new_project=None,
|
||||
)
|
||||
except ValueError as e: # pragma: no cover
|
||||
raise HTTPException(status_code=400, detail=str(e)) # pragma: no cover
|
||||
|
||||
|
||||
@router.put("/{project_id}/default", response_model=ProjectStatusResponse)
|
||||
async def set_default_project_by_id(
|
||||
project_service: ProjectServiceDep,
|
||||
project_repository: ProjectRepositoryDep,
|
||||
project_id: str = Path(..., description="Project external ID (UUID)"),
|
||||
) -> ProjectStatusResponse:
|
||||
"""Set a project as the default project by external ID.
|
||||
|
||||
Args:
|
||||
project_id: External ID (UUID string) to set as default
|
||||
|
||||
Returns:
|
||||
Response confirming the project was set as default
|
||||
|
||||
Raises:
|
||||
HTTPException: 404 if project not found
|
||||
|
||||
Example:
|
||||
PUT /v2/projects/550e8400-e29b-41d4-a716-446655440000/default
|
||||
"""
|
||||
logger.info(f"API v2 request: set_default_project_by_id for project_id={project_id}")
|
||||
|
||||
try:
|
||||
# Get the old default project from database
|
||||
default_project = await project_repository.get_default_project()
|
||||
if not default_project:
|
||||
raise HTTPException( # pragma: no cover
|
||||
status_code=404, detail="No default project is currently set"
|
||||
)
|
||||
|
||||
# Get the new default project by external_id
|
||||
new_default_project = await project_repository.get_by_external_id(project_id)
|
||||
if not new_default_project:
|
||||
raise HTTPException(
|
||||
status_code=404, detail=f"Project with external_id '{project_id}' not found"
|
||||
)
|
||||
|
||||
# Set as default using project name (service layer still uses names internally)
|
||||
await project_service.set_default_project(new_default_project.name)
|
||||
|
||||
return ProjectStatusResponse(
|
||||
message=f"Project '{new_default_project.name}' set as default successfully",
|
||||
status="success",
|
||||
default=True,
|
||||
old_project=ProjectItem(
|
||||
id=default_project.id,
|
||||
external_id=default_project.external_id,
|
||||
name=default_project.name,
|
||||
path=default_project.path,
|
||||
is_default=False,
|
||||
),
|
||||
new_project=ProjectItem(
|
||||
id=new_default_project.id,
|
||||
external_id=new_default_project.external_id,
|
||||
name=new_default_project.name,
|
||||
path=new_default_project.path,
|
||||
is_default=True,
|
||||
),
|
||||
)
|
||||
except ValueError as e: # pragma: no cover
|
||||
raise HTTPException(status_code=400, detail=str(e)) # pragma: no cover
|
||||
@@ -1,269 +0,0 @@
|
||||
"""V2 Prompt Router - ID-based prompt generation operations.
|
||||
|
||||
This router uses v2 dependencies for consistent project handling with external_id UUIDs.
|
||||
Prompt endpoints are action-based (not resource-based), so they don't
|
||||
have entity IDs in URLs - they generate formatted prompts from queries.
|
||||
"""
|
||||
|
||||
from datetime import datetime, timezone
|
||||
from fastapi import APIRouter, HTTPException, status, Path
|
||||
from loguru import logger
|
||||
|
||||
from basic_memory.api.v2.utils import to_graph_context, to_search_results
|
||||
from basic_memory.api.template_loader import template_loader
|
||||
from basic_memory.schemas.base import parse_timeframe
|
||||
from basic_memory.deps import (
|
||||
ContextServiceV2ExternalDep,
|
||||
EntityRepositoryV2ExternalDep,
|
||||
SearchServiceV2ExternalDep,
|
||||
EntityServiceV2ExternalDep,
|
||||
)
|
||||
from basic_memory.schemas.prompt import (
|
||||
ContinueConversationRequest,
|
||||
SearchPromptRequest,
|
||||
PromptResponse,
|
||||
PromptMetadata,
|
||||
)
|
||||
from basic_memory.schemas.search import SearchItemType, SearchQuery
|
||||
|
||||
router = APIRouter(prefix="/prompt", tags=["prompt-v2"])
|
||||
|
||||
|
||||
@router.post("/continue-conversation", response_model=PromptResponse)
|
||||
async def continue_conversation(
|
||||
search_service: SearchServiceV2ExternalDep,
|
||||
entity_service: EntityServiceV2ExternalDep,
|
||||
context_service: ContextServiceV2ExternalDep,
|
||||
entity_repository: EntityRepositoryV2ExternalDep,
|
||||
request: ContinueConversationRequest,
|
||||
project_id: str = Path(..., description="Project external UUID"),
|
||||
) -> PromptResponse:
|
||||
"""Generate a prompt for continuing a conversation.
|
||||
|
||||
This endpoint takes a topic and/or timeframe and generates a prompt with
|
||||
relevant context from the knowledge base.
|
||||
|
||||
Args:
|
||||
project_id: Project external UUID from URL path
|
||||
request: The request parameters
|
||||
|
||||
Returns:
|
||||
Formatted continuation prompt with context
|
||||
"""
|
||||
logger.info(
|
||||
f"V2 Generating continue conversation prompt for project {project_id}, "
|
||||
f"topic: {request.topic}, timeframe: {request.timeframe}"
|
||||
)
|
||||
|
||||
since = parse_timeframe(request.timeframe) if request.timeframe else None
|
||||
|
||||
# Initialize search results
|
||||
search_results = []
|
||||
|
||||
# Get data needed for template
|
||||
if request.topic:
|
||||
query = SearchQuery(text=request.topic, after_date=request.timeframe)
|
||||
results = await search_service.search(query, limit=request.search_items_limit)
|
||||
search_results = await to_search_results(entity_service, results)
|
||||
|
||||
# Build context from results
|
||||
all_hierarchical_results = []
|
||||
for result in search_results:
|
||||
if hasattr(result, "permalink") and result.permalink:
|
||||
# Get hierarchical context using the new dataclass-based approach
|
||||
context_result = await context_service.build_context(
|
||||
result.permalink,
|
||||
depth=request.depth,
|
||||
since=since,
|
||||
max_related=request.related_items_limit,
|
||||
include_observations=True, # Include observations for entities
|
||||
)
|
||||
|
||||
# Process results into the schema format
|
||||
graph_context = await to_graph_context(
|
||||
context_result, entity_repository=entity_repository
|
||||
)
|
||||
|
||||
# Add results to our collection (limit to top results for each permalink)
|
||||
if graph_context.results:
|
||||
all_hierarchical_results.extend(graph_context.results[:3])
|
||||
|
||||
# Limit to a reasonable number of total results
|
||||
all_hierarchical_results = all_hierarchical_results[:10]
|
||||
|
||||
template_context = {
|
||||
"topic": request.topic,
|
||||
"timeframe": request.timeframe,
|
||||
"hierarchical_results": all_hierarchical_results,
|
||||
"has_results": len(all_hierarchical_results) > 0,
|
||||
}
|
||||
else:
|
||||
# If no topic, get recent activity
|
||||
context_result = await context_service.build_context(
|
||||
types=[SearchItemType.ENTITY],
|
||||
depth=request.depth,
|
||||
since=since,
|
||||
max_related=request.related_items_limit,
|
||||
include_observations=True,
|
||||
)
|
||||
recent_context = await to_graph_context(context_result, entity_repository=entity_repository)
|
||||
|
||||
hierarchical_results = recent_context.results[:5] # Limit to top 5 recent items
|
||||
|
||||
template_context = {
|
||||
"topic": f"Recent Activity from ({request.timeframe})",
|
||||
"timeframe": request.timeframe,
|
||||
"hierarchical_results": hierarchical_results,
|
||||
"has_results": len(hierarchical_results) > 0,
|
||||
}
|
||||
|
||||
try:
|
||||
# Render template
|
||||
rendered_prompt = await template_loader.render(
|
||||
"prompts/continue_conversation.hbs", template_context
|
||||
)
|
||||
|
||||
# Calculate metadata
|
||||
# Count items of different types
|
||||
observation_count = 0
|
||||
relation_count = 0
|
||||
entity_count = 0
|
||||
|
||||
# Get the hierarchical results from the template context
|
||||
hierarchical_results_for_count = template_context.get("hierarchical_results", [])
|
||||
|
||||
# For topic-based search
|
||||
if request.topic:
|
||||
for item in hierarchical_results_for_count:
|
||||
if hasattr(item, "observations"):
|
||||
observation_count += len(item.observations) if item.observations else 0
|
||||
|
||||
if hasattr(item, "related_results"):
|
||||
for related in item.related_results or []:
|
||||
if hasattr(related, "type"):
|
||||
if related.type == "relation":
|
||||
relation_count += 1
|
||||
elif related.type == "entity": # pragma: no cover
|
||||
entity_count += 1 # pragma: no cover
|
||||
# For recent activity
|
||||
else:
|
||||
for item in hierarchical_results_for_count:
|
||||
if hasattr(item, "observations"):
|
||||
observation_count += len(item.observations) if item.observations else 0
|
||||
|
||||
if hasattr(item, "related_results"):
|
||||
for related in item.related_results or []:
|
||||
if hasattr(related, "type"):
|
||||
if related.type == "relation":
|
||||
relation_count += 1
|
||||
elif related.type == "entity": # pragma: no cover
|
||||
entity_count += 1 # pragma: no cover
|
||||
|
||||
# Build metadata
|
||||
metadata = {
|
||||
"query": request.topic,
|
||||
"timeframe": request.timeframe,
|
||||
"search_count": len(search_results)
|
||||
if request.topic
|
||||
else 0, # Original search results count
|
||||
"context_count": len(hierarchical_results_for_count),
|
||||
"observation_count": observation_count,
|
||||
"relation_count": relation_count,
|
||||
"total_items": (
|
||||
len(hierarchical_results_for_count)
|
||||
+ observation_count
|
||||
+ relation_count
|
||||
+ entity_count
|
||||
),
|
||||
"search_limit": request.search_items_limit,
|
||||
"context_depth": request.depth,
|
||||
"related_limit": request.related_items_limit,
|
||||
"generated_at": datetime.now(timezone.utc).isoformat(),
|
||||
}
|
||||
|
||||
prompt_metadata = PromptMetadata(**metadata)
|
||||
|
||||
return PromptResponse(
|
||||
prompt=rendered_prompt, context=template_context, metadata=prompt_metadata
|
||||
)
|
||||
except Exception as e:
|
||||
logger.error(f"Error rendering continue conversation template: {e}")
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
|
||||
detail=f"Error rendering prompt template: {str(e)}",
|
||||
)
|
||||
|
||||
|
||||
@router.post("/search", response_model=PromptResponse)
|
||||
async def search_prompt(
|
||||
search_service: SearchServiceV2ExternalDep,
|
||||
entity_service: EntityServiceV2ExternalDep,
|
||||
request: SearchPromptRequest,
|
||||
project_id: str = Path(..., description="Project external UUID"),
|
||||
page: int = 1,
|
||||
page_size: int = 10,
|
||||
) -> PromptResponse:
|
||||
"""Generate a prompt for search results.
|
||||
|
||||
This endpoint takes a search query and formats the results into a helpful
|
||||
prompt with context and suggestions.
|
||||
|
||||
Args:
|
||||
project_id: Project external UUID from URL path
|
||||
request: The search parameters
|
||||
page: The page number for pagination
|
||||
page_size: The number of results per page, defaults to 10
|
||||
|
||||
Returns:
|
||||
Formatted search results prompt with context
|
||||
"""
|
||||
logger.info(
|
||||
f"V2 Generating search prompt for project {project_id}, "
|
||||
f"query: {request.query}, timeframe: {request.timeframe}"
|
||||
)
|
||||
|
||||
limit = page_size
|
||||
offset = (page - 1) * page_size
|
||||
|
||||
query = SearchQuery(text=request.query, after_date=request.timeframe)
|
||||
results = await search_service.search(query, limit=limit, offset=offset)
|
||||
search_results = await to_search_results(entity_service, results)
|
||||
|
||||
template_context = {
|
||||
"query": request.query,
|
||||
"timeframe": request.timeframe,
|
||||
"results": search_results,
|
||||
"has_results": len(search_results) > 0,
|
||||
"result_count": len(search_results),
|
||||
}
|
||||
|
||||
try:
|
||||
# Render template
|
||||
rendered_prompt = await template_loader.render("prompts/search.hbs", template_context)
|
||||
|
||||
# Build metadata
|
||||
metadata = {
|
||||
"query": request.query,
|
||||
"timeframe": request.timeframe,
|
||||
"search_count": len(search_results),
|
||||
"context_count": len(search_results),
|
||||
"observation_count": 0, # Search results don't include observations
|
||||
"relation_count": 0, # Search results don't include relations
|
||||
"total_items": len(search_results),
|
||||
"search_limit": limit,
|
||||
"context_depth": 0, # No context depth for basic search
|
||||
"related_limit": 0, # No related items for basic search
|
||||
"generated_at": datetime.now(timezone.utc).isoformat(),
|
||||
}
|
||||
|
||||
prompt_metadata = PromptMetadata(**metadata)
|
||||
|
||||
return PromptResponse(
|
||||
prompt=rendered_prompt, context=template_context, metadata=prompt_metadata
|
||||
)
|
||||
except Exception as e:
|
||||
logger.error(f"Error rendering search template: {e}")
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
|
||||
detail=f"Error rendering prompt template: {str(e)}",
|
||||
)
|
||||
@@ -1,289 +0,0 @@
|
||||
"""V2 Resource Router - ID-based resource content operations.
|
||||
|
||||
This router uses entity external_ids (UUIDs) for all operations, with file paths
|
||||
in request bodies when needed. This is consistent with v2's external_id-first design.
|
||||
|
||||
Key differences from v1:
|
||||
- Uses UUID external_ids in URL paths instead of integer IDs or file paths
|
||||
- File paths are in request bodies for create/update operations
|
||||
- More RESTful: POST for create, PUT for update, GET for read
|
||||
"""
|
||||
|
||||
import uuid
|
||||
from pathlib import Path as PathLib
|
||||
|
||||
from fastapi import APIRouter, HTTPException, Response, Path
|
||||
from loguru import logger
|
||||
|
||||
from basic_memory.deps import (
|
||||
ProjectConfigV2ExternalDep,
|
||||
FileServiceV2ExternalDep,
|
||||
EntityRepositoryV2ExternalDep,
|
||||
SearchServiceV2ExternalDep,
|
||||
)
|
||||
from basic_memory.models.knowledge import Entity as EntityModel
|
||||
from basic_memory.schemas.v2.resource import (
|
||||
CreateResourceRequest,
|
||||
UpdateResourceRequest,
|
||||
ResourceResponse,
|
||||
)
|
||||
from basic_memory.utils import validate_project_path
|
||||
|
||||
router = APIRouter(prefix="/resource", tags=["resources-v2"])
|
||||
|
||||
|
||||
@router.get("/{entity_id}")
|
||||
async def get_resource_content(
|
||||
config: ProjectConfigV2ExternalDep,
|
||||
entity_repository: EntityRepositoryV2ExternalDep,
|
||||
file_service: FileServiceV2ExternalDep,
|
||||
project_id: str = Path(..., description="Project external UUID"),
|
||||
entity_id: str = Path(..., description="Entity external UUID"),
|
||||
) -> Response:
|
||||
"""Get raw resource content by entity external_id.
|
||||
|
||||
Args:
|
||||
project_id: Project external UUID from URL path
|
||||
entity_id: Entity external UUID
|
||||
config: Project configuration
|
||||
entity_repository: Entity repository for fetching entity data
|
||||
file_service: File service for reading file content
|
||||
|
||||
Returns:
|
||||
Response with entity content
|
||||
|
||||
Raises:
|
||||
HTTPException: 404 if entity or file not found
|
||||
"""
|
||||
logger.debug(f"V2 Getting content for project {project_id}, entity_id: {entity_id}")
|
||||
|
||||
# Get entity by external_id
|
||||
entity = await entity_repository.get_by_external_id(entity_id)
|
||||
if not entity:
|
||||
raise HTTPException(status_code=404, detail=f"Entity {entity_id} not found")
|
||||
|
||||
# Validate entity file path to prevent path traversal
|
||||
project_path = PathLib(config.home)
|
||||
if not validate_project_path(entity.file_path, project_path):
|
||||
logger.error( # pragma: no cover
|
||||
f"Invalid file path in entity {entity.id}: {entity.file_path}"
|
||||
)
|
||||
raise HTTPException( # pragma: no cover
|
||||
status_code=500,
|
||||
detail="Entity contains invalid file path",
|
||||
)
|
||||
|
||||
# Check file exists via file_service (for cloud compatibility)
|
||||
if not await file_service.exists(entity.file_path):
|
||||
raise HTTPException( # pragma: no cover
|
||||
status_code=404,
|
||||
detail=f"File not found: {entity.file_path}",
|
||||
)
|
||||
|
||||
# Read content via file_service as bytes (works with both local and S3)
|
||||
content = await file_service.read_file_bytes(entity.file_path)
|
||||
content_type = file_service.content_type(entity.file_path)
|
||||
|
||||
return Response(content=content, media_type=content_type)
|
||||
|
||||
|
||||
@router.post("", response_model=ResourceResponse)
|
||||
async def create_resource(
|
||||
data: CreateResourceRequest,
|
||||
config: ProjectConfigV2ExternalDep,
|
||||
file_service: FileServiceV2ExternalDep,
|
||||
entity_repository: EntityRepositoryV2ExternalDep,
|
||||
search_service: SearchServiceV2ExternalDep,
|
||||
project_id: str = Path(..., description="Project external UUID"),
|
||||
) -> ResourceResponse:
|
||||
"""Create a new resource file.
|
||||
|
||||
Args:
|
||||
project_id: Project external UUID from URL path
|
||||
data: Create resource request with file_path and content
|
||||
config: Project configuration
|
||||
file_service: File service for writing files
|
||||
entity_repository: Entity repository for creating entities
|
||||
search_service: Search service for indexing
|
||||
|
||||
Returns:
|
||||
ResourceResponse with file information including entity_id and external_id
|
||||
|
||||
Raises:
|
||||
HTTPException: 400 for invalid file paths, 409 if file already exists
|
||||
"""
|
||||
try:
|
||||
# Validate path to prevent path traversal attacks
|
||||
project_path = PathLib(config.home)
|
||||
if not validate_project_path(data.file_path, project_path):
|
||||
logger.warning(
|
||||
f"Invalid file path attempted: {data.file_path} in project {config.name}"
|
||||
)
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
detail=f"Invalid file path: {data.file_path}. "
|
||||
"Path must be relative and stay within project boundaries.",
|
||||
)
|
||||
|
||||
# Check if entity already exists
|
||||
existing_entity = await entity_repository.get_by_file_path(data.file_path)
|
||||
if existing_entity:
|
||||
raise HTTPException(
|
||||
status_code=409,
|
||||
detail=f"Resource already exists at {data.file_path} with entity_id {existing_entity.external_id}. "
|
||||
f"Use PUT /resource/{existing_entity.external_id} to update it.",
|
||||
)
|
||||
|
||||
# Cloud compatibility: avoid assuming a local filesystem path.
|
||||
# Delegate directory creation + writes to FileService (local or S3).
|
||||
await file_service.ensure_directory(PathLib(data.file_path).parent)
|
||||
checksum = await file_service.write_file(data.file_path, data.content)
|
||||
|
||||
# Get file info
|
||||
file_metadata = await file_service.get_file_metadata(data.file_path)
|
||||
|
||||
# Determine file details
|
||||
file_name = PathLib(data.file_path).name
|
||||
content_type = file_service.content_type(data.file_path)
|
||||
entity_type = "canvas" if data.file_path.endswith(".canvas") else "file"
|
||||
|
||||
# Create a new entity model
|
||||
# Explicitly set external_id to ensure NOT NULL constraint is satisfied (fixes #512)
|
||||
entity = EntityModel(
|
||||
external_id=str(uuid.uuid4()),
|
||||
title=file_name,
|
||||
entity_type=entity_type,
|
||||
content_type=content_type,
|
||||
file_path=data.file_path,
|
||||
checksum=checksum,
|
||||
created_at=file_metadata.created_at,
|
||||
updated_at=file_metadata.modified_at,
|
||||
)
|
||||
entity = await entity_repository.add(entity)
|
||||
|
||||
# Index the file for search
|
||||
await search_service.index_entity(entity) # pyright: ignore
|
||||
|
||||
# Return success response
|
||||
return ResourceResponse(
|
||||
entity_id=entity.id,
|
||||
external_id=entity.external_id,
|
||||
file_path=data.file_path,
|
||||
checksum=checksum,
|
||||
size=file_metadata.size,
|
||||
created_at=file_metadata.created_at.timestamp(),
|
||||
modified_at=file_metadata.modified_at.timestamp(),
|
||||
)
|
||||
except HTTPException:
|
||||
# Re-raise HTTP exceptions without wrapping
|
||||
raise
|
||||
except Exception as e: # pragma: no cover
|
||||
logger.error(f"Error creating resource {data.file_path}: {e}")
|
||||
raise HTTPException(status_code=500, detail=f"Failed to create resource: {str(e)}")
|
||||
|
||||
|
||||
@router.put("/{entity_id}", response_model=ResourceResponse)
|
||||
async def update_resource(
|
||||
data: UpdateResourceRequest,
|
||||
config: ProjectConfigV2ExternalDep,
|
||||
file_service: FileServiceV2ExternalDep,
|
||||
entity_repository: EntityRepositoryV2ExternalDep,
|
||||
search_service: SearchServiceV2ExternalDep,
|
||||
project_id: str = Path(..., description="Project external UUID"),
|
||||
entity_id: str = Path(..., description="Entity external UUID"),
|
||||
) -> ResourceResponse:
|
||||
"""Update an existing resource by entity external_id.
|
||||
|
||||
Can update content and optionally move the file to a new path.
|
||||
|
||||
Args:
|
||||
project_id: Project external UUID from URL path
|
||||
entity_id: Entity external UUID of the resource to update
|
||||
data: Update resource request with content and optional new file_path
|
||||
config: Project configuration
|
||||
file_service: File service for writing files
|
||||
entity_repository: Entity repository for updating entities
|
||||
search_service: Search service for indexing
|
||||
|
||||
Returns:
|
||||
ResourceResponse with updated file information
|
||||
|
||||
Raises:
|
||||
HTTPException: 404 if entity not found, 400 for invalid paths
|
||||
"""
|
||||
try:
|
||||
# Get existing entity by external_id
|
||||
entity = await entity_repository.get_by_external_id(entity_id)
|
||||
if not entity:
|
||||
raise HTTPException(status_code=404, detail=f"Entity {entity_id} not found")
|
||||
|
||||
# Determine target file path
|
||||
target_file_path = data.file_path if data.file_path else entity.file_path
|
||||
|
||||
# Validate path to prevent path traversal attacks
|
||||
project_path = PathLib(config.home)
|
||||
if not validate_project_path(target_file_path, project_path):
|
||||
logger.warning(
|
||||
f"Invalid file path attempted: {target_file_path} in project {config.name}"
|
||||
)
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
detail=f"Invalid file path: {target_file_path}. "
|
||||
"Path must be relative and stay within project boundaries.",
|
||||
)
|
||||
|
||||
# If moving file, handle the move
|
||||
if data.file_path and data.file_path != entity.file_path:
|
||||
# Ensure new parent directory exists (no-op for S3)
|
||||
await file_service.ensure_directory(PathLib(target_file_path).parent)
|
||||
|
||||
# If old file exists, remove it via file_service (for cloud compatibility)
|
||||
if await file_service.exists(entity.file_path):
|
||||
await file_service.delete_file(entity.file_path)
|
||||
else:
|
||||
# Ensure directory exists for in-place update
|
||||
await file_service.ensure_directory(PathLib(target_file_path).parent)
|
||||
|
||||
# Write content to target file
|
||||
checksum = await file_service.write_file(target_file_path, data.content)
|
||||
|
||||
# Get file info
|
||||
file_metadata = await file_service.get_file_metadata(target_file_path)
|
||||
|
||||
# Determine file details
|
||||
file_name = PathLib(target_file_path).name
|
||||
content_type = file_service.content_type(target_file_path)
|
||||
entity_type = "canvas" if target_file_path.endswith(".canvas") else "file"
|
||||
|
||||
# Update entity using internal ID
|
||||
updated_entity = await entity_repository.update(
|
||||
entity.id,
|
||||
{
|
||||
"title": file_name,
|
||||
"entity_type": entity_type,
|
||||
"content_type": content_type,
|
||||
"file_path": target_file_path,
|
||||
"checksum": checksum,
|
||||
"updated_at": file_metadata.modified_at,
|
||||
},
|
||||
)
|
||||
|
||||
# Index the updated file for search
|
||||
await search_service.index_entity(updated_entity) # pyright: ignore
|
||||
|
||||
# Return success response
|
||||
return ResourceResponse(
|
||||
entity_id=entity.id,
|
||||
external_id=entity.external_id,
|
||||
file_path=target_file_path,
|
||||
checksum=checksum,
|
||||
size=file_metadata.size,
|
||||
created_at=file_metadata.created_at.timestamp(),
|
||||
modified_at=file_metadata.modified_at.timestamp(),
|
||||
)
|
||||
except HTTPException:
|
||||
# Re-raise HTTP exceptions without wrapping
|
||||
raise
|
||||
except Exception as e: # pragma: no cover
|
||||
logger.error(f"Error updating resource {entity_id}: {e}")
|
||||
raise HTTPException(status_code=500, detail=f"Failed to update resource: {str(e)}")
|
||||
@@ -1,76 +0,0 @@
|
||||
"""V2 router for search operations.
|
||||
|
||||
This router uses external_id UUIDs for stable, API-friendly routing.
|
||||
V1 uses string-based project names which are less efficient and less stable.
|
||||
"""
|
||||
|
||||
from fastapi import APIRouter, Path
|
||||
|
||||
from basic_memory.api.v2.utils import to_search_results
|
||||
from basic_memory.schemas.search import SearchQuery, SearchResponse
|
||||
from basic_memory.deps import (
|
||||
SearchServiceV2ExternalDep,
|
||||
EntityServiceV2ExternalDep,
|
||||
TaskSchedulerDep,
|
||||
ProjectExternalIdPathDep,
|
||||
)
|
||||
|
||||
# Note: No prefix here - it's added during registration as /v2/{project_id}/search
|
||||
router = APIRouter(tags=["search"])
|
||||
|
||||
|
||||
@router.post("/search/", response_model=SearchResponse)
|
||||
async def search(
|
||||
query: SearchQuery,
|
||||
search_service: SearchServiceV2ExternalDep,
|
||||
entity_service: EntityServiceV2ExternalDep,
|
||||
project_id: str = Path(..., description="Project external UUID"),
|
||||
page: int = 1,
|
||||
page_size: int = 10,
|
||||
):
|
||||
"""Search across all knowledge and documents in a project.
|
||||
|
||||
V2 uses external_id UUIDs for stable API references.
|
||||
|
||||
Args:
|
||||
project_id: Project external UUID from URL path
|
||||
query: Search query parameters (text, filters, etc.)
|
||||
search_service: Search service scoped to project
|
||||
entity_service: Entity service scoped to project
|
||||
page: Page number for pagination
|
||||
page_size: Number of results per page
|
||||
|
||||
Returns:
|
||||
SearchResponse with paginated search results
|
||||
"""
|
||||
limit = page_size
|
||||
offset = (page - 1) * page_size
|
||||
results = await search_service.search(query, limit=limit, offset=offset)
|
||||
search_results = await to_search_results(entity_service, results)
|
||||
return SearchResponse(
|
||||
results=search_results,
|
||||
current_page=page,
|
||||
page_size=page_size,
|
||||
)
|
||||
|
||||
|
||||
@router.post("/search/reindex")
|
||||
async def reindex(
|
||||
task_scheduler: TaskSchedulerDep,
|
||||
project_id: ProjectExternalIdPathDep,
|
||||
):
|
||||
"""Recreate and populate the search index for a project.
|
||||
|
||||
This is a background operation that rebuilds the search index
|
||||
from scratch. Useful after bulk updates or if the index becomes
|
||||
corrupted.
|
||||
|
||||
Args:
|
||||
project_id: Project external UUID from URL path
|
||||
task_scheduler: Task scheduler for background work
|
||||
|
||||
Returns:
|
||||
Status message indicating reindex has been initiated
|
||||
"""
|
||||
task_scheduler.schedule("reindex_project", project_id=project_id)
|
||||
return {"status": "ok", "message": "Reindex initiated"}
|
||||
@@ -1,190 +0,0 @@
|
||||
from typing import Optional, List
|
||||
|
||||
from basic_memory.repository import EntityRepository
|
||||
from basic_memory.repository.search_repository import SearchIndexRow
|
||||
from basic_memory.schemas.memory import (
|
||||
EntitySummary,
|
||||
ObservationSummary,
|
||||
RelationSummary,
|
||||
MemoryMetadata,
|
||||
GraphContext,
|
||||
ContextResult,
|
||||
)
|
||||
from basic_memory.schemas.search import SearchItemType, SearchResult
|
||||
from basic_memory.services import EntityService
|
||||
from basic_memory.services.context_service import (
|
||||
ContextResultRow,
|
||||
ContextResult as ServiceContextResult,
|
||||
)
|
||||
|
||||
|
||||
async def to_graph_context(
|
||||
context_result: ServiceContextResult,
|
||||
entity_repository: EntityRepository,
|
||||
page: Optional[int] = None,
|
||||
page_size: Optional[int] = None,
|
||||
):
|
||||
# First pass: collect all entity IDs needed for external_id lookup
|
||||
# This includes: entity primary results, observation parent entities, relation from/to entities
|
||||
entity_ids_needed: set[int] = set()
|
||||
for context_item in context_result.results:
|
||||
for item in (
|
||||
[context_item.primary_result] + context_item.observations + context_item.related_results
|
||||
):
|
||||
if item.type == SearchItemType.ENTITY:
|
||||
# Entity's own ID for its external_id
|
||||
entity_ids_needed.add(item.id)
|
||||
elif item.type == SearchItemType.OBSERVATION:
|
||||
# Parent entity ID for entity_external_id
|
||||
if item.entity_id: # pyright: ignore
|
||||
entity_ids_needed.add(item.entity_id) # pyright: ignore
|
||||
elif item.type == SearchItemType.RELATION:
|
||||
# Source and target entity IDs for external_ids
|
||||
if item.from_id: # pyright: ignore
|
||||
entity_ids_needed.add(item.from_id) # pyright: ignore
|
||||
if item.to_id:
|
||||
entity_ids_needed.add(item.to_id)
|
||||
|
||||
# Batch fetch all entities at once - get both title and external_id
|
||||
entity_title_lookup: dict[int, str] = {}
|
||||
entity_external_id_lookup: dict[int, str] = {}
|
||||
if entity_ids_needed:
|
||||
entities = await entity_repository.find_by_ids(list(entity_ids_needed))
|
||||
for e in entities:
|
||||
entity_title_lookup[e.id] = e.title
|
||||
entity_external_id_lookup[e.id] = e.external_id
|
||||
|
||||
# Helper function to convert items to summaries
|
||||
def to_summary(item: SearchIndexRow | ContextResultRow):
|
||||
match item.type:
|
||||
case SearchItemType.ENTITY:
|
||||
return EntitySummary(
|
||||
external_id=entity_external_id_lookup.get(item.id, ""),
|
||||
entity_id=item.id,
|
||||
title=item.title, # pyright: ignore
|
||||
permalink=item.permalink,
|
||||
content=item.content,
|
||||
file_path=item.file_path,
|
||||
created_at=item.created_at,
|
||||
)
|
||||
case SearchItemType.OBSERVATION:
|
||||
entity_ext_id = None
|
||||
if item.entity_id: # pyright: ignore
|
||||
entity_ext_id = entity_external_id_lookup.get(item.entity_id) # pyright: ignore
|
||||
return ObservationSummary(
|
||||
observation_id=item.id,
|
||||
entity_id=item.entity_id, # pyright: ignore
|
||||
entity_external_id=entity_ext_id,
|
||||
title=entity_title_lookup.get(item.entity_id), # pyright: ignore
|
||||
file_path=item.file_path,
|
||||
category=item.category, # pyright: ignore
|
||||
content=item.content, # pyright: ignore
|
||||
permalink=item.permalink, # pyright: ignore
|
||||
created_at=item.created_at,
|
||||
)
|
||||
case SearchItemType.RELATION:
|
||||
from_title = entity_title_lookup.get(item.from_id) if item.from_id else None # pyright: ignore
|
||||
to_title = entity_title_lookup.get(item.to_id) if item.to_id else None
|
||||
from_ext_id = entity_external_id_lookup.get(item.from_id) if item.from_id else None # pyright: ignore
|
||||
to_ext_id = entity_external_id_lookup.get(item.to_id) if item.to_id else None
|
||||
return RelationSummary(
|
||||
relation_id=item.id,
|
||||
entity_id=item.entity_id, # pyright: ignore
|
||||
title=item.title, # pyright: ignore
|
||||
file_path=item.file_path,
|
||||
permalink=item.permalink, # pyright: ignore
|
||||
relation_type=item.relation_type, # pyright: ignore
|
||||
from_entity=from_title,
|
||||
from_entity_id=item.from_id, # pyright: ignore
|
||||
from_entity_external_id=from_ext_id,
|
||||
to_entity=to_title,
|
||||
to_entity_id=item.to_id,
|
||||
to_entity_external_id=to_ext_id,
|
||||
created_at=item.created_at,
|
||||
)
|
||||
case _: # pragma: no cover
|
||||
raise ValueError(f"Unexpected type: {item.type}")
|
||||
|
||||
# Process the hierarchical results
|
||||
hierarchical_results = []
|
||||
for context_item in context_result.results:
|
||||
# Process primary result
|
||||
primary_result = to_summary(context_item.primary_result)
|
||||
|
||||
# Process observations (always ObservationSummary, validated by context_service)
|
||||
observations = [to_summary(obs) for obs in context_item.observations]
|
||||
|
||||
# Process related results
|
||||
related = [to_summary(rel) for rel in context_item.related_results]
|
||||
|
||||
# Add to hierarchical results
|
||||
hierarchical_results.append(
|
||||
ContextResult(
|
||||
primary_result=primary_result,
|
||||
observations=observations, # pyright: ignore[reportArgumentType]
|
||||
related_results=related,
|
||||
)
|
||||
)
|
||||
|
||||
# Create schema metadata from service metadata
|
||||
metadata = MemoryMetadata(
|
||||
uri=context_result.metadata.uri,
|
||||
types=context_result.metadata.types,
|
||||
depth=context_result.metadata.depth,
|
||||
timeframe=context_result.metadata.timeframe,
|
||||
generated_at=context_result.metadata.generated_at,
|
||||
primary_count=context_result.metadata.primary_count,
|
||||
related_count=context_result.metadata.related_count,
|
||||
total_results=context_result.metadata.primary_count + context_result.metadata.related_count,
|
||||
total_relations=context_result.metadata.total_relations,
|
||||
total_observations=context_result.metadata.total_observations,
|
||||
)
|
||||
|
||||
# Return new GraphContext with just hierarchical results
|
||||
return GraphContext(
|
||||
results=hierarchical_results,
|
||||
metadata=metadata,
|
||||
page=page,
|
||||
page_size=page_size,
|
||||
)
|
||||
|
||||
|
||||
async def to_search_results(entity_service: EntityService, results: List[SearchIndexRow]):
|
||||
search_results = []
|
||||
for r in results:
|
||||
entities = await entity_service.get_entities_by_id([r.entity_id, r.from_id, r.to_id]) # pyright: ignore
|
||||
|
||||
# Determine which IDs to set based on type
|
||||
entity_id = None
|
||||
observation_id = None
|
||||
relation_id = None
|
||||
|
||||
if r.type == SearchItemType.ENTITY:
|
||||
entity_id = r.id
|
||||
elif r.type == SearchItemType.OBSERVATION:
|
||||
observation_id = r.id
|
||||
entity_id = r.entity_id # Parent entity
|
||||
elif r.type == SearchItemType.RELATION:
|
||||
relation_id = r.id
|
||||
entity_id = r.entity_id # Parent entity
|
||||
|
||||
search_results.append(
|
||||
SearchResult(
|
||||
title=r.title, # pyright: ignore
|
||||
type=r.type, # pyright: ignore
|
||||
permalink=r.permalink,
|
||||
score=r.score, # pyright: ignore
|
||||
entity=entities[0].permalink if entities else None,
|
||||
content=r.content,
|
||||
file_path=r.file_path,
|
||||
metadata=r.metadata,
|
||||
entity_id=entity_id,
|
||||
observation_id=observation_id,
|
||||
relation_id=relation_id,
|
||||
category=r.category,
|
||||
from_entity=entities[0].permalink if entities else None,
|
||||
to_entity=entities[1].permalink if len(entities) > 1 else None,
|
||||
relation_type=r.relation_type,
|
||||
)
|
||||
)
|
||||
return search_results
|
||||
@@ -1 +0,0 @@
|
||||
"""CLI tools for basic-memory"""
|
||||
@@ -1,76 +0,0 @@
|
||||
# This prevents DEBUG logs from appearing on stdout during module-level
|
||||
# initialization (e.g., template_loader.TemplateLoader() logs at DEBUG level).
|
||||
from loguru import logger
|
||||
|
||||
logger.remove()
|
||||
|
||||
from typing import Optional # noqa: E402
|
||||
|
||||
import typer # noqa: E402
|
||||
|
||||
from basic_memory.cli.container import CliContainer, set_container # noqa: E402
|
||||
from basic_memory.config import init_cli_logging # noqa: E402
|
||||
|
||||
|
||||
def version_callback(value: bool) -> None:
|
||||
"""Show version and exit."""
|
||||
if value: # pragma: no cover
|
||||
import basic_memory
|
||||
|
||||
typer.echo(f"Basic Memory version: {basic_memory.__version__}")
|
||||
raise typer.Exit()
|
||||
|
||||
|
||||
app = typer.Typer(name="basic-memory")
|
||||
|
||||
|
||||
@app.callback()
|
||||
def app_callback(
|
||||
ctx: typer.Context,
|
||||
version: Optional[bool] = typer.Option(
|
||||
None,
|
||||
"--version",
|
||||
"-v",
|
||||
help="Show version and exit.",
|
||||
callback=version_callback,
|
||||
is_eager=True,
|
||||
),
|
||||
) -> None:
|
||||
"""Basic Memory - Local-first personal knowledge management."""
|
||||
|
||||
# Initialize logging for CLI (file only, no stdout)
|
||||
init_cli_logging()
|
||||
|
||||
# --- Composition Root ---
|
||||
# Create container and read config (single point of config access)
|
||||
container = CliContainer.create()
|
||||
set_container(container)
|
||||
|
||||
# Run initialization for commands that don't use the API
|
||||
# Skip for 'mcp' command - it has its own lifespan that handles initialization
|
||||
# Skip for API-using commands (status, sync, etc.) - they handle initialization via deps.py
|
||||
# Skip for 'reset' command - it manages its own database lifecycle
|
||||
skip_init_commands = {"doctor", "mcp", "status", "sync", "project", "tool", "reset"}
|
||||
if (
|
||||
not version
|
||||
and ctx.invoked_subcommand is not None
|
||||
and ctx.invoked_subcommand not in skip_init_commands
|
||||
):
|
||||
from basic_memory.services.initialization import ensure_initialization
|
||||
|
||||
ensure_initialization(container.config)
|
||||
|
||||
|
||||
## import
|
||||
# Register sub-command groups
|
||||
import_app = typer.Typer(help="Import data from various sources")
|
||||
app.add_typer(import_app, name="import")
|
||||
|
||||
claude_app = typer.Typer(help="Import Conversations from Claude JSON export.")
|
||||
import_app.add_typer(claude_app, name="claude")
|
||||
|
||||
|
||||
## cloud
|
||||
|
||||
cloud_app = typer.Typer(help="Access Basic Memory Cloud")
|
||||
app.add_typer(cloud_app, name="cloud")
|
||||
@@ -1,300 +0,0 @@
|
||||
"""WorkOS OAuth Device Authorization for CLI."""
|
||||
|
||||
import base64
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
import secrets
|
||||
import time
|
||||
import webbrowser
|
||||
from contextlib import asynccontextmanager
|
||||
from collections.abc import AsyncIterator, Callable
|
||||
from typing import AsyncContextManager
|
||||
|
||||
import httpx
|
||||
from rich.console import Console
|
||||
|
||||
from basic_memory.config import ConfigManager
|
||||
|
||||
console = Console()
|
||||
|
||||
|
||||
class CLIAuth:
|
||||
"""Handles WorkOS OAuth Device Authorization for CLI tools."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
client_id: str,
|
||||
authkit_domain: str,
|
||||
http_client_factory: Callable[[], AsyncContextManager[httpx.AsyncClient]] | None = None,
|
||||
):
|
||||
self.client_id = client_id
|
||||
self.authkit_domain = authkit_domain
|
||||
app_config = ConfigManager().config
|
||||
# Store tokens in data dir
|
||||
self.token_file = app_config.data_dir_path / "basic-memory-cloud.json"
|
||||
# PKCE parameters
|
||||
self.code_verifier = None
|
||||
self.code_challenge = None
|
||||
self._http_client_factory = http_client_factory
|
||||
|
||||
@asynccontextmanager
|
||||
async def _get_http_client(self) -> AsyncIterator[httpx.AsyncClient]:
|
||||
"""Create an AsyncClient, optionally via injected factory.
|
||||
|
||||
Why: enables reliable tests without monkeypatching httpx internals while
|
||||
still using real httpx request/response objects.
|
||||
"""
|
||||
if self._http_client_factory:
|
||||
async with self._http_client_factory() as client:
|
||||
yield client
|
||||
else:
|
||||
async with httpx.AsyncClient() as client:
|
||||
yield client
|
||||
|
||||
def generate_pkce_pair(self) -> tuple[str, str]:
|
||||
"""Generate PKCE code verifier and challenge."""
|
||||
# Generate code verifier (43-128 characters)
|
||||
code_verifier = base64.urlsafe_b64encode(secrets.token_bytes(32)).decode("utf-8")
|
||||
code_verifier = code_verifier.rstrip("=")
|
||||
|
||||
# Generate code challenge (SHA256 hash of verifier)
|
||||
challenge_bytes = hashlib.sha256(code_verifier.encode("utf-8")).digest()
|
||||
code_challenge = base64.urlsafe_b64encode(challenge_bytes).decode("utf-8")
|
||||
code_challenge = code_challenge.rstrip("=")
|
||||
|
||||
return code_verifier, code_challenge
|
||||
|
||||
async def request_device_authorization(self) -> dict | None:
|
||||
"""Request device authorization from WorkOS with PKCE."""
|
||||
device_auth_url = f"{self.authkit_domain}/oauth2/device_authorization"
|
||||
|
||||
# Generate PKCE pair
|
||||
self.code_verifier, self.code_challenge = self.generate_pkce_pair()
|
||||
|
||||
data = {
|
||||
"client_id": self.client_id,
|
||||
"scope": "openid profile email offline_access",
|
||||
"code_challenge": self.code_challenge,
|
||||
"code_challenge_method": "S256",
|
||||
}
|
||||
|
||||
try:
|
||||
async with self._get_http_client() as client:
|
||||
response = await client.post(device_auth_url, data=data)
|
||||
|
||||
if response.status_code == 200:
|
||||
return response.json()
|
||||
else:
|
||||
console.print(
|
||||
f"[red]Device authorization failed: {response.status_code} - {response.text}[/red]"
|
||||
)
|
||||
return None
|
||||
except Exception as e:
|
||||
console.print(f"[red]Device authorization error: {e}[/red]")
|
||||
return None
|
||||
|
||||
def display_user_instructions(self, device_response: dict) -> None:
|
||||
"""Display user instructions for device authorization."""
|
||||
user_code = device_response["user_code"]
|
||||
verification_uri = device_response["verification_uri"]
|
||||
verification_uri_complete = device_response.get("verification_uri_complete")
|
||||
|
||||
console.print("\n[bold blue]Authentication Required[/bold blue]")
|
||||
console.print("\nTo authenticate, please visit:")
|
||||
console.print(f"[bold cyan]{verification_uri}[/bold cyan]")
|
||||
console.print(f"\nAnd enter this code: [bold yellow]{user_code}[/bold yellow]")
|
||||
|
||||
if verification_uri_complete:
|
||||
console.print("\nOr for one-click access, visit:")
|
||||
console.print(f"[bold green]{verification_uri_complete}[/bold green]")
|
||||
|
||||
# Try to open browser automatically
|
||||
try:
|
||||
console.print("\n[dim]Opening browser automatically...[/dim]")
|
||||
webbrowser.open(verification_uri_complete)
|
||||
except Exception:
|
||||
pass # Silently fail if browser can't be opened
|
||||
|
||||
console.print("\n[dim]Waiting for you to complete authentication in your browser...[/dim]")
|
||||
|
||||
async def poll_for_token(self, device_code: str, interval: int = 5) -> dict | None:
|
||||
"""Poll the token endpoint until user completes authentication."""
|
||||
token_url = f"{self.authkit_domain}/oauth2/token"
|
||||
|
||||
data = {
|
||||
"client_id": self.client_id,
|
||||
"device_code": device_code,
|
||||
"grant_type": "urn:ietf:params:oauth:grant-type:device_code",
|
||||
"code_verifier": self.code_verifier,
|
||||
}
|
||||
|
||||
max_attempts = 60 # 5 minutes with 5-second intervals
|
||||
current_interval = interval
|
||||
|
||||
for _attempt in range(max_attempts):
|
||||
try:
|
||||
async with self._get_http_client() as client:
|
||||
response = await client.post(token_url, data=data)
|
||||
|
||||
if response.status_code == 200:
|
||||
return response.json()
|
||||
|
||||
# Parse error response
|
||||
try:
|
||||
error_data = response.json()
|
||||
error = error_data.get("error")
|
||||
except Exception:
|
||||
error = "unknown_error"
|
||||
|
||||
if error == "authorization_pending":
|
||||
# User hasn't completed auth yet, keep polling
|
||||
pass
|
||||
elif error == "slow_down":
|
||||
# Increase polling interval
|
||||
current_interval += 5
|
||||
console.print("[yellow]Slowing down polling rate...[/yellow]")
|
||||
elif error == "access_denied":
|
||||
console.print("[red]Authentication was denied by user[/red]")
|
||||
return None
|
||||
elif error == "expired_token":
|
||||
console.print("[red]Device code has expired. Please try again.[/red]")
|
||||
return None
|
||||
else:
|
||||
console.print(f"[red]Token polling error: {error}[/red]")
|
||||
return None
|
||||
|
||||
except Exception as e:
|
||||
console.print(f"[red]Token polling request error: {e}[/red]")
|
||||
|
||||
# Wait before next poll
|
||||
await self._async_sleep(current_interval)
|
||||
|
||||
console.print("[red]Authentication timeout. Please try again.[/red]")
|
||||
return None
|
||||
|
||||
async def _async_sleep(self, seconds: int) -> None:
|
||||
"""Async sleep utility."""
|
||||
import asyncio
|
||||
|
||||
await asyncio.sleep(seconds)
|
||||
|
||||
def save_tokens(self, tokens: dict) -> None:
|
||||
"""Save tokens to project root as .bm-auth.json."""
|
||||
token_data = {
|
||||
"access_token": tokens["access_token"],
|
||||
"refresh_token": tokens.get("refresh_token"),
|
||||
"expires_at": int(time.time()) + tokens.get("expires_in", 3600),
|
||||
"token_type": tokens.get("token_type", "Bearer"),
|
||||
}
|
||||
|
||||
with open(self.token_file, "w") as f:
|
||||
json.dump(token_data, f, indent=2)
|
||||
|
||||
# Secure the token file
|
||||
os.chmod(self.token_file, 0o600)
|
||||
|
||||
console.print(f"[green]Tokens saved to {self.token_file}[/green]")
|
||||
|
||||
def load_tokens(self) -> dict | None:
|
||||
"""Load tokens from .bm-auth.json file."""
|
||||
if not self.token_file.exists():
|
||||
return None
|
||||
|
||||
try:
|
||||
with open(self.token_file) as f:
|
||||
return json.load(f)
|
||||
except (OSError, json.JSONDecodeError):
|
||||
return None
|
||||
|
||||
def is_token_valid(self, tokens: dict) -> bool:
|
||||
"""Check if stored token is still valid."""
|
||||
expires_at = tokens.get("expires_at", 0)
|
||||
# Add 60 second buffer for clock skew
|
||||
return time.time() < (expires_at - 60)
|
||||
|
||||
async def refresh_token(self, refresh_token: str) -> dict | None:
|
||||
"""Refresh access token using refresh token."""
|
||||
token_url = f"{self.authkit_domain}/oauth2/token"
|
||||
|
||||
data = {
|
||||
"client_id": self.client_id,
|
||||
"grant_type": "refresh_token",
|
||||
"refresh_token": refresh_token,
|
||||
}
|
||||
|
||||
try:
|
||||
async with self._get_http_client() as client:
|
||||
response = await client.post(token_url, data=data)
|
||||
|
||||
if response.status_code == 200:
|
||||
return response.json()
|
||||
else:
|
||||
console.print(
|
||||
f"[red]Token refresh failed: {response.status_code} - {response.text}[/red]"
|
||||
)
|
||||
return None
|
||||
except Exception as e:
|
||||
console.print(f"[red]Token refresh error: {e}[/red]")
|
||||
return None
|
||||
|
||||
async def get_valid_token(self) -> str | None:
|
||||
"""Get valid access token, refresh if needed."""
|
||||
tokens = self.load_tokens()
|
||||
if not tokens:
|
||||
return None
|
||||
|
||||
if self.is_token_valid(tokens):
|
||||
return tokens["access_token"]
|
||||
|
||||
# Token expired - try to refresh if we have a refresh token
|
||||
refresh_token = tokens.get("refresh_token")
|
||||
if refresh_token:
|
||||
console.print("[yellow]Access token expired, refreshing...[/yellow]")
|
||||
|
||||
new_tokens = await self.refresh_token(refresh_token)
|
||||
if new_tokens:
|
||||
# Save new tokens (may include rotated refresh token)
|
||||
self.save_tokens(new_tokens)
|
||||
console.print("[green]Token refreshed successfully[/green]")
|
||||
return new_tokens["access_token"]
|
||||
else:
|
||||
console.print("[yellow]Token refresh failed. Please run 'login' again.[/yellow]")
|
||||
return None
|
||||
else:
|
||||
console.print("[yellow]No refresh token available. Please run 'login' again.[/yellow]")
|
||||
return None
|
||||
|
||||
async def login(self) -> bool:
|
||||
"""Perform OAuth Device Authorization login flow."""
|
||||
console.print("[blue]Initiating authentication...[/blue]")
|
||||
|
||||
# Step 1: Request device authorization
|
||||
device_response = await self.request_device_authorization()
|
||||
if not device_response:
|
||||
return False
|
||||
|
||||
# Step 2: Display user instructions
|
||||
self.display_user_instructions(device_response)
|
||||
|
||||
# Step 3: Poll for token
|
||||
device_code = device_response["device_code"]
|
||||
interval = device_response.get("interval", 5)
|
||||
|
||||
tokens = await self.poll_for_token(device_code, interval)
|
||||
if not tokens:
|
||||
return False
|
||||
|
||||
# Step 4: Save tokens
|
||||
self.save_tokens(tokens)
|
||||
|
||||
console.print("\n[green]Successfully authenticated with Basic Memory Cloud![/green]")
|
||||
return True
|
||||
|
||||
def logout(self) -> None:
|
||||
"""Remove stored authentication tokens."""
|
||||
if self.token_file.exists():
|
||||
self.token_file.unlink()
|
||||
console.print("[green]Logged out successfully[/green]")
|
||||
else:
|
||||
console.print("[yellow]No stored authentication found[/yellow]")
|
||||
@@ -1,18 +0,0 @@
|
||||
"""CLI commands for basic-memory."""
|
||||
|
||||
from . import status, db, doctor, import_memory_json, mcp, import_claude_conversations
|
||||
from . import import_claude_projects, import_chatgpt, tool, project, format
|
||||
|
||||
__all__ = [
|
||||
"status",
|
||||
"db",
|
||||
"doctor",
|
||||
"import_memory_json",
|
||||
"mcp",
|
||||
"import_claude_conversations",
|
||||
"import_claude_projects",
|
||||
"import_chatgpt",
|
||||
"tool",
|
||||
"project",
|
||||
"format",
|
||||
]
|
||||
@@ -1,16 +0,0 @@
|
||||
"""Cloud commands package."""
|
||||
|
||||
from basic_memory.cli.app import cloud_app
|
||||
|
||||
# Import all commands to register them with typer
|
||||
from basic_memory.cli.commands.cloud.core_commands import * # noqa: F401,F403
|
||||
from basic_memory.cli.commands.cloud.api_client import get_authenticated_headers, get_cloud_config # noqa: F401
|
||||
from basic_memory.cli.commands.cloud.upload_command import * # noqa: F401,F403
|
||||
|
||||
# Register snapshot sub-command group
|
||||
from basic_memory.cli.commands.cloud.snapshot import snapshot_app
|
||||
|
||||
cloud_app.add_typer(snapshot_app, name="snapshot")
|
||||
|
||||
# Register restore command (directly on cloud_app via decorator)
|
||||
from basic_memory.cli.commands.cloud.restore import restore # noqa: F401, E402
|
||||
@@ -1,127 +0,0 @@
|
||||
"""Cloud API client utilities."""
|
||||
|
||||
from collections.abc import AsyncIterator
|
||||
from typing import Optional
|
||||
from contextlib import asynccontextmanager
|
||||
from typing import AsyncContextManager, Callable
|
||||
|
||||
import httpx
|
||||
import typer
|
||||
from rich.console import Console
|
||||
|
||||
from basic_memory.cli.auth import CLIAuth
|
||||
from basic_memory.config import ConfigManager
|
||||
|
||||
console = Console()
|
||||
|
||||
HttpClientFactory = Callable[[], AsyncContextManager[httpx.AsyncClient]]
|
||||
|
||||
|
||||
class CloudAPIError(Exception):
|
||||
"""Exception raised for cloud API errors."""
|
||||
|
||||
def __init__(
|
||||
self, message: str, status_code: Optional[int] = None, detail: Optional[dict] = None
|
||||
):
|
||||
super().__init__(message)
|
||||
self.status_code = status_code
|
||||
self.detail = detail or {}
|
||||
|
||||
|
||||
class SubscriptionRequiredError(CloudAPIError):
|
||||
"""Exception raised when user needs an active subscription."""
|
||||
|
||||
def __init__(self, message: str, subscribe_url: str):
|
||||
super().__init__(message, status_code=403, detail={"error": "subscription_required"})
|
||||
self.subscribe_url = subscribe_url
|
||||
|
||||
|
||||
def get_cloud_config() -> tuple[str, str, str]:
|
||||
"""Get cloud OAuth configuration from config."""
|
||||
config_manager = ConfigManager()
|
||||
config = config_manager.config
|
||||
return config.cloud_client_id, config.cloud_domain, config.cloud_host
|
||||
|
||||
|
||||
async def get_authenticated_headers(auth: CLIAuth | None = None) -> dict[str, str]:
|
||||
"""
|
||||
Get authentication headers with JWT token.
|
||||
handles jwt refresh if needed.
|
||||
"""
|
||||
client_id, domain, _ = get_cloud_config()
|
||||
auth_obj = auth or CLIAuth(client_id=client_id, authkit_domain=domain)
|
||||
token = await auth_obj.get_valid_token()
|
||||
if not token:
|
||||
console.print("[red]Not authenticated. Please run 'basic-memory cloud login' first.[/red]")
|
||||
raise typer.Exit(1)
|
||||
|
||||
return {"Authorization": f"Bearer {token}"}
|
||||
|
||||
|
||||
@asynccontextmanager
|
||||
async def _default_http_client(timeout: float) -> AsyncIterator[httpx.AsyncClient]:
|
||||
async with httpx.AsyncClient(timeout=timeout) as client:
|
||||
yield client
|
||||
|
||||
|
||||
async def make_api_request(
|
||||
method: str,
|
||||
url: str,
|
||||
headers: Optional[dict] = None,
|
||||
json_data: Optional[dict] = None,
|
||||
timeout: float = 30.0,
|
||||
*,
|
||||
auth: CLIAuth | None = None,
|
||||
http_client_factory: HttpClientFactory | None = None,
|
||||
) -> httpx.Response:
|
||||
"""Make an API request to the cloud service."""
|
||||
headers = headers or {}
|
||||
auth_headers = await get_authenticated_headers(auth=auth)
|
||||
headers.update(auth_headers)
|
||||
# Add debug headers to help with compression issues
|
||||
headers.setdefault("Accept-Encoding", "identity") # Disable compression for debugging
|
||||
|
||||
client_factory = http_client_factory or (lambda: _default_http_client(timeout))
|
||||
async with client_factory() as client:
|
||||
try:
|
||||
response = await client.request(method=method, url=url, headers=headers, json=json_data)
|
||||
response.raise_for_status()
|
||||
return response
|
||||
except httpx.HTTPError as e:
|
||||
# Check if this is a response error with response details
|
||||
if hasattr(e, "response") and e.response is not None: # pyright: ignore [reportAttributeAccessIssue]
|
||||
response = e.response # type: ignore
|
||||
|
||||
# Try to parse error detail from response
|
||||
error_detail = None
|
||||
try:
|
||||
error_detail = response.json()
|
||||
except Exception:
|
||||
# If JSON parsing fails, we'll handle it as a generic error
|
||||
pass
|
||||
|
||||
# Check for subscription_required error (403)
|
||||
if response.status_code == 403 and isinstance(error_detail, dict):
|
||||
# Handle both FastAPI HTTPException format (nested under "detail")
|
||||
# and direct format
|
||||
detail_obj = error_detail.get("detail", error_detail)
|
||||
if (
|
||||
isinstance(detail_obj, dict)
|
||||
and detail_obj.get("error") == "subscription_required"
|
||||
):
|
||||
message = detail_obj.get("message", "Active subscription required")
|
||||
subscribe_url = detail_obj.get(
|
||||
"subscribe_url", "https://basicmemory.com/subscribe"
|
||||
)
|
||||
raise SubscriptionRequiredError(
|
||||
message=message, subscribe_url=subscribe_url
|
||||
) from e
|
||||
|
||||
# Raise generic CloudAPIError with status code and detail
|
||||
raise CloudAPIError(
|
||||
f"API request failed: {e}",
|
||||
status_code=response.status_code,
|
||||
detail=error_detail if isinstance(error_detail, dict) else {},
|
||||
) from e
|
||||
|
||||
raise CloudAPIError(f"API request failed: {e}") from e
|
||||
@@ -1,110 +0,0 @@
|
||||
"""Cloud bisync utility functions for Basic Memory CLI."""
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
from basic_memory.cli.commands.cloud.api_client import make_api_request
|
||||
from basic_memory.config import ConfigManager
|
||||
from basic_memory.ignore_utils import create_default_bmignore, get_bmignore_path
|
||||
from basic_memory.schemas.cloud import MountCredentials, TenantMountInfo
|
||||
|
||||
|
||||
class BisyncError(Exception):
|
||||
"""Exception raised for bisync-related errors."""
|
||||
|
||||
pass
|
||||
|
||||
|
||||
async def get_mount_info() -> TenantMountInfo:
|
||||
"""Get current tenant information from cloud API."""
|
||||
try:
|
||||
config_manager = ConfigManager()
|
||||
config = config_manager.config
|
||||
host_url = config.cloud_host.rstrip("/")
|
||||
|
||||
response = await make_api_request(method="GET", url=f"{host_url}/tenant/mount/info")
|
||||
|
||||
return TenantMountInfo.model_validate(response.json())
|
||||
except Exception as e:
|
||||
raise BisyncError(f"Failed to get tenant info: {e}") from e
|
||||
|
||||
|
||||
async def generate_mount_credentials(tenant_id: str) -> MountCredentials:
|
||||
"""Generate scoped credentials for syncing."""
|
||||
try:
|
||||
config_manager = ConfigManager()
|
||||
config = config_manager.config
|
||||
host_url = config.cloud_host.rstrip("/")
|
||||
|
||||
response = await make_api_request(method="POST", url=f"{host_url}/tenant/mount/credentials")
|
||||
|
||||
return MountCredentials.model_validate(response.json())
|
||||
except Exception as e:
|
||||
raise BisyncError(f"Failed to generate credentials: {e}") from e
|
||||
|
||||
|
||||
def convert_bmignore_to_rclone_filters() -> Path:
|
||||
"""Convert .bmignore patterns to rclone filter format.
|
||||
|
||||
Reads ~/.basic-memory/.bmignore (gitignore-style) and converts to
|
||||
~/.basic-memory/.bmignore.rclone (rclone filter format).
|
||||
|
||||
Only regenerates if .bmignore has been modified since last conversion.
|
||||
|
||||
Returns:
|
||||
Path to converted rclone filter file
|
||||
"""
|
||||
# Ensure .bmignore exists
|
||||
create_default_bmignore()
|
||||
|
||||
bmignore_path = get_bmignore_path()
|
||||
# Create rclone filter path: ~/.basic-memory/.bmignore -> ~/.basic-memory/.bmignore.rclone
|
||||
rclone_filter_path = bmignore_path.parent / f"{bmignore_path.name}.rclone"
|
||||
|
||||
# Skip regeneration if rclone file is newer than bmignore
|
||||
if rclone_filter_path.exists():
|
||||
bmignore_mtime = bmignore_path.stat().st_mtime
|
||||
rclone_mtime = rclone_filter_path.stat().st_mtime
|
||||
if rclone_mtime >= bmignore_mtime:
|
||||
return rclone_filter_path
|
||||
|
||||
# Read .bmignore patterns
|
||||
patterns = []
|
||||
try:
|
||||
with bmignore_path.open("r", encoding="utf-8") as f:
|
||||
for line in f:
|
||||
line = line.strip()
|
||||
# Keep comments and empty lines
|
||||
if not line or line.startswith("#"):
|
||||
patterns.append(line)
|
||||
continue
|
||||
|
||||
# Convert gitignore pattern to rclone filter syntax
|
||||
# gitignore: node_modules → rclone: - node_modules/**
|
||||
# gitignore: *.pyc → rclone: - *.pyc
|
||||
if "*" in line:
|
||||
# Pattern already has wildcard, just add exclude prefix
|
||||
patterns.append(f"- {line}")
|
||||
else:
|
||||
# Directory pattern - add /** for recursive exclude
|
||||
patterns.append(f"- {line}/**")
|
||||
|
||||
except Exception:
|
||||
# If we can't read the file, create a minimal filter
|
||||
patterns = ["# Error reading .bmignore, using minimal filters", "- .git/**"]
|
||||
|
||||
# Write rclone filter file
|
||||
rclone_filter_path.write_text("\n".join(patterns) + "\n")
|
||||
|
||||
return rclone_filter_path
|
||||
|
||||
|
||||
def get_bisync_filter_path() -> Path:
|
||||
"""Get path to bisync filter file.
|
||||
|
||||
Uses ~/.basic-memory/.bmignore (converted to rclone format).
|
||||
The file is automatically created with default patterns on first use.
|
||||
|
||||
Returns:
|
||||
Path to rclone filter file
|
||||
"""
|
||||
return convert_bmignore_to_rclone_filters()
|
||||
@@ -1,108 +0,0 @@
|
||||
"""Shared utilities for cloud operations."""
|
||||
|
||||
from basic_memory.cli.commands.cloud.api_client import make_api_request
|
||||
from basic_memory.config import ConfigManager
|
||||
from basic_memory.schemas.cloud import (
|
||||
CloudProjectList,
|
||||
CloudProjectCreateRequest,
|
||||
CloudProjectCreateResponse,
|
||||
)
|
||||
from basic_memory.utils import generate_permalink
|
||||
|
||||
|
||||
class CloudUtilsError(Exception):
|
||||
"""Exception raised for cloud utility errors."""
|
||||
|
||||
pass
|
||||
|
||||
|
||||
async def fetch_cloud_projects(
|
||||
*,
|
||||
api_request=make_api_request,
|
||||
) -> CloudProjectList:
|
||||
"""Fetch list of projects from cloud API.
|
||||
|
||||
Returns:
|
||||
CloudProjectList with projects from cloud
|
||||
"""
|
||||
try:
|
||||
config_manager = ConfigManager()
|
||||
config = config_manager.config
|
||||
host_url = config.cloud_host.rstrip("/")
|
||||
|
||||
response = await api_request(method="GET", url=f"{host_url}/proxy/v2/projects/")
|
||||
|
||||
return CloudProjectList.model_validate(response.json())
|
||||
except Exception as e:
|
||||
raise CloudUtilsError(f"Failed to fetch cloud projects: {e}") from e
|
||||
|
||||
|
||||
async def create_cloud_project(
|
||||
project_name: str,
|
||||
*,
|
||||
api_request=make_api_request,
|
||||
) -> CloudProjectCreateResponse:
|
||||
"""Create a new project on cloud.
|
||||
|
||||
Args:
|
||||
project_name: Name of project to create
|
||||
|
||||
Returns:
|
||||
CloudProjectCreateResponse with project details from API
|
||||
"""
|
||||
try:
|
||||
config_manager = ConfigManager()
|
||||
config = config_manager.config
|
||||
host_url = config.cloud_host.rstrip("/")
|
||||
|
||||
# Use generate_permalink to ensure consistent naming
|
||||
project_path = generate_permalink(project_name)
|
||||
|
||||
project_data = CloudProjectCreateRequest(
|
||||
name=project_name,
|
||||
path=project_path,
|
||||
set_default=False,
|
||||
)
|
||||
|
||||
response = await api_request(
|
||||
method="POST",
|
||||
url=f"{host_url}/proxy/v2/projects/",
|
||||
headers={"Content-Type": "application/json"},
|
||||
json_data=project_data.model_dump(),
|
||||
)
|
||||
|
||||
return CloudProjectCreateResponse.model_validate(response.json())
|
||||
except Exception as e:
|
||||
raise CloudUtilsError(f"Failed to create cloud project '{project_name}': {e}") from e
|
||||
|
||||
|
||||
async def sync_project(project_name: str, force_full: bool = False) -> None:
|
||||
"""Trigger sync for a specific project on cloud.
|
||||
|
||||
Args:
|
||||
project_name: Name of project to sync
|
||||
force_full: If True, force a full scan bypassing watermark optimization
|
||||
"""
|
||||
try:
|
||||
from basic_memory.cli.commands.command_utils import run_sync
|
||||
|
||||
await run_sync(project=project_name, force_full=force_full)
|
||||
except Exception as e:
|
||||
raise CloudUtilsError(f"Failed to sync project '{project_name}': {e}") from e
|
||||
|
||||
|
||||
async def project_exists(project_name: str, *, api_request=make_api_request) -> bool:
|
||||
"""Check if a project exists on cloud.
|
||||
|
||||
Args:
|
||||
project_name: Name of project to check
|
||||
|
||||
Returns:
|
||||
True if project exists, False otherwise
|
||||
"""
|
||||
try:
|
||||
projects = await fetch_cloud_projects(api_request=api_request)
|
||||
project_names = {p.name for p in projects.projects}
|
||||
return project_name in project_names
|
||||
except Exception:
|
||||
return False
|
||||
@@ -1,193 +0,0 @@
|
||||
"""Core cloud commands for Basic Memory CLI."""
|
||||
|
||||
import typer
|
||||
from rich.console import Console
|
||||
|
||||
from basic_memory.cli.app import cloud_app
|
||||
from basic_memory.cli.commands.command_utils import run_with_cleanup
|
||||
from basic_memory.cli.auth import CLIAuth
|
||||
from basic_memory.config import ConfigManager
|
||||
from basic_memory.cli.commands.cloud.api_client import (
|
||||
CloudAPIError,
|
||||
SubscriptionRequiredError,
|
||||
get_cloud_config,
|
||||
make_api_request,
|
||||
)
|
||||
from basic_memory.cli.commands.cloud.bisync_commands import (
|
||||
BisyncError,
|
||||
generate_mount_credentials,
|
||||
get_mount_info,
|
||||
)
|
||||
from basic_memory.cli.commands.cloud.rclone_config import configure_rclone_remote
|
||||
from basic_memory.cli.commands.cloud.rclone_installer import (
|
||||
RcloneInstallError,
|
||||
install_rclone,
|
||||
)
|
||||
|
||||
console = Console()
|
||||
|
||||
|
||||
@cloud_app.command()
|
||||
def login():
|
||||
"""Authenticate with WorkOS using OAuth Device Authorization flow and enable cloud mode."""
|
||||
|
||||
async def _login():
|
||||
client_id, domain, host_url = get_cloud_config()
|
||||
auth = CLIAuth(client_id=client_id, authkit_domain=domain)
|
||||
|
||||
try:
|
||||
success = await auth.login()
|
||||
if not success:
|
||||
console.print("[red]Login failed[/red]")
|
||||
raise typer.Exit(1)
|
||||
|
||||
# Test subscription access by calling a protected endpoint
|
||||
console.print("[dim]Verifying subscription access...[/dim]")
|
||||
await make_api_request("GET", f"{host_url.rstrip('/')}/proxy/health")
|
||||
|
||||
# Enable cloud mode after successful login and subscription validation
|
||||
config_manager = ConfigManager()
|
||||
config = config_manager.load_config()
|
||||
config.cloud_mode = True
|
||||
config_manager.save_config(config)
|
||||
|
||||
console.print("[green]Cloud mode enabled[/green]")
|
||||
console.print(f"[dim]All CLI commands now work against {host_url}[/dim]")
|
||||
|
||||
except SubscriptionRequiredError as e:
|
||||
console.print("\n[red]Subscription Required[/red]\n")
|
||||
console.print(f"[yellow]{e.args[0]}[/yellow]\n")
|
||||
console.print(f"Subscribe at: [blue underline]{e.subscribe_url}[/blue underline]\n")
|
||||
console.print(
|
||||
"[dim]Once you have an active subscription, run [bold]bm cloud login[/bold] again.[/dim]"
|
||||
)
|
||||
raise typer.Exit(1)
|
||||
|
||||
run_with_cleanup(_login())
|
||||
|
||||
|
||||
@cloud_app.command()
|
||||
def logout():
|
||||
"""Disable cloud mode and return to local mode."""
|
||||
|
||||
# Disable cloud mode
|
||||
config_manager = ConfigManager()
|
||||
config = config_manager.load_config()
|
||||
config.cloud_mode = False
|
||||
config_manager.save_config(config)
|
||||
|
||||
console.print("[green]Cloud mode disabled[/green]")
|
||||
console.print("[dim]All CLI commands now work locally[/dim]")
|
||||
|
||||
|
||||
@cloud_app.command("status")
|
||||
def status() -> None:
|
||||
"""Check cloud mode status and cloud instance health."""
|
||||
# Check cloud mode
|
||||
config_manager = ConfigManager()
|
||||
config = config_manager.load_config()
|
||||
|
||||
console.print("[bold blue]Cloud Mode Status[/bold blue]")
|
||||
if config.cloud_mode:
|
||||
console.print(" Mode: [green]Cloud (enabled)[/green]")
|
||||
console.print(f" Host: {config.cloud_host}")
|
||||
console.print(" [dim]All CLI commands work against cloud[/dim]")
|
||||
else:
|
||||
console.print(" Mode: [yellow]Local (disabled)[/yellow]")
|
||||
console.print(" [dim]All CLI commands work locally[/dim]")
|
||||
console.print("\n[dim]To enable cloud mode, run: bm cloud login[/dim]")
|
||||
return
|
||||
|
||||
# Get cloud configuration
|
||||
_, _, host_url = get_cloud_config()
|
||||
host_url = host_url.rstrip("/")
|
||||
|
||||
# Prepare headers
|
||||
headers = {}
|
||||
|
||||
try:
|
||||
console.print("\n[blue]Checking cloud instance health...[/blue]")
|
||||
|
||||
# Make API request to check health
|
||||
response = run_with_cleanup(
|
||||
make_api_request(method="GET", url=f"{host_url}/proxy/health", headers=headers)
|
||||
)
|
||||
|
||||
health_data = response.json()
|
||||
|
||||
console.print("[green]Cloud instance is healthy[/green]")
|
||||
|
||||
# Display status details
|
||||
if "status" in health_data:
|
||||
console.print(f" Status: {health_data['status']}")
|
||||
if "version" in health_data:
|
||||
console.print(f" Version: {health_data['version']}")
|
||||
if "timestamp" in health_data:
|
||||
console.print(f" Timestamp: {health_data['timestamp']}")
|
||||
|
||||
console.print("\n[dim]To sync projects, use: bm project bisync --name <project>[/dim]")
|
||||
|
||||
except CloudAPIError as e:
|
||||
console.print(f"[red]Error checking cloud health: {e}[/red]")
|
||||
raise typer.Exit(1)
|
||||
except Exception as e:
|
||||
console.print(f"[red]Unexpected error: {e}[/red]")
|
||||
raise typer.Exit(1)
|
||||
|
||||
|
||||
@cloud_app.command("setup")
|
||||
def setup() -> None:
|
||||
"""Set up cloud sync by installing rclone and configuring credentials.
|
||||
|
||||
After setup, use project commands for syncing:
|
||||
bm project add <name> <path> --local-path ~/projects/<name>
|
||||
bm project bisync --name <name> --resync # First time
|
||||
bm project bisync --name <name> # Subsequent syncs
|
||||
"""
|
||||
console.print("[bold blue]Basic Memory Cloud Setup[/bold blue]")
|
||||
console.print("Setting up cloud sync with rclone...\n")
|
||||
|
||||
try:
|
||||
# Step 1: Install rclone
|
||||
console.print("[blue]Step 1: Installing rclone...[/blue]")
|
||||
install_rclone()
|
||||
|
||||
# Step 2: Get tenant info
|
||||
console.print("\n[blue]Step 2: Getting tenant information...[/blue]")
|
||||
tenant_info = run_with_cleanup(get_mount_info())
|
||||
console.print(f"[green]Found tenant: {tenant_info.tenant_id}[/green]")
|
||||
|
||||
# Step 3: Generate credentials
|
||||
console.print("\n[blue]Step 3: Generating sync credentials...[/blue]")
|
||||
creds = run_with_cleanup(generate_mount_credentials(tenant_info.tenant_id))
|
||||
console.print("[green]Generated secure credentials[/green]")
|
||||
|
||||
# Step 4: Configure rclone remote
|
||||
console.print("\n[blue]Step 4: Configuring rclone remote...[/blue]")
|
||||
configure_rclone_remote(
|
||||
access_key=creds.access_key,
|
||||
secret_key=creds.secret_key,
|
||||
)
|
||||
|
||||
console.print("\n[bold green]Cloud setup completed successfully![/bold green]")
|
||||
console.print("\n[bold]Next steps:[/bold]")
|
||||
console.print("1. Add a project with local sync path:")
|
||||
console.print(" bm project add research --local-path ~/Documents/research")
|
||||
console.print("\n Or configure sync for an existing project:")
|
||||
console.print(" bm project sync-setup research ~/Documents/research")
|
||||
console.print("\n2. Preview the initial sync (recommended):")
|
||||
console.print(" bm project bisync --name research --resync --dry-run")
|
||||
console.print("\n3. If all looks good, run the actual sync:")
|
||||
console.print(" bm project bisync --name research --resync")
|
||||
console.print("\n4. Subsequent syncs (no --resync needed):")
|
||||
console.print(" bm project bisync --name research")
|
||||
console.print(
|
||||
"\n[dim]Tip: Always use --dry-run first to preview changes before syncing[/dim]"
|
||||
)
|
||||
|
||||
except (RcloneInstallError, BisyncError, CloudAPIError) as e:
|
||||
console.print(f"\n[red]Setup failed: {e}[/red]")
|
||||
raise typer.Exit(1)
|
||||
except Exception as e:
|
||||
console.print(f"\n[red]Unexpected error during setup: {e}[/red]")
|
||||
raise typer.Exit(1)
|
||||
@@ -1,409 +0,0 @@
|
||||
"""Project-scoped rclone sync commands for Basic Memory Cloud.
|
||||
|
||||
This module provides simplified, project-scoped rclone operations:
|
||||
- Each project syncs independently
|
||||
- Uses single "basic-memory-cloud" remote (not tenant-specific)
|
||||
- Balanced defaults from SPEC-8 Phase 4 testing
|
||||
- Per-project bisync state tracking
|
||||
|
||||
Replaces tenant-wide sync with project-scoped workflows.
|
||||
"""
|
||||
|
||||
import re
|
||||
import subprocess
|
||||
from dataclasses import dataclass
|
||||
from functools import lru_cache
|
||||
from pathlib import Path
|
||||
from typing import Callable, Optional, Protocol
|
||||
|
||||
from loguru import logger
|
||||
from rich.console import Console
|
||||
|
||||
from basic_memory.cli.commands.cloud.rclone_installer import is_rclone_installed
|
||||
from basic_memory.utils import normalize_project_path
|
||||
|
||||
console = Console()
|
||||
|
||||
# Minimum rclone version for --create-empty-src-dirs support
|
||||
MIN_RCLONE_VERSION_EMPTY_DIRS = (1, 64, 0)
|
||||
|
||||
# Tigris edge caching returns stale data for users outside the origin region (iad).
|
||||
# These headers bypass edge cache and force reads/writes against the origin.
|
||||
# See: https://www.tigrisdata.com/docs/objects/consistency/
|
||||
TIGRIS_CONSISTENCY_HEADERS = [
|
||||
"--header-download", "X-Tigris-Consistent: true",
|
||||
"--header-upload", "X-Tigris-Consistent: true",
|
||||
]
|
||||
|
||||
|
||||
class RunResult(Protocol):
|
||||
returncode: int
|
||||
stdout: str
|
||||
|
||||
|
||||
RunFunc = Callable[..., RunResult]
|
||||
IsInstalledFunc = Callable[[], bool]
|
||||
|
||||
|
||||
class RcloneError(Exception):
|
||||
"""Exception raised for rclone command errors."""
|
||||
|
||||
pass
|
||||
|
||||
|
||||
def check_rclone_installed(is_installed: IsInstalledFunc = is_rclone_installed) -> None:
|
||||
"""Check if rclone is installed and raise helpful error if not.
|
||||
|
||||
Raises:
|
||||
RcloneError: If rclone is not installed with installation instructions
|
||||
"""
|
||||
if not is_installed():
|
||||
raise RcloneError(
|
||||
"rclone is not installed.\n\n"
|
||||
"Install rclone by running: bm cloud setup\n"
|
||||
"Or install manually from: https://rclone.org/downloads/\n\n"
|
||||
"Windows users: Ensure you have a package manager installed (winget, chocolatey, or scoop)"
|
||||
)
|
||||
|
||||
|
||||
@lru_cache(maxsize=1)
|
||||
def get_rclone_version(run: RunFunc = subprocess.run) -> tuple[int, int, int] | None:
|
||||
"""Get rclone version as (major, minor, patch) tuple.
|
||||
|
||||
Returns:
|
||||
Version tuple like (1, 64, 2), or None if version cannot be determined.
|
||||
|
||||
Note:
|
||||
Result is cached since rclone version won't change during runtime.
|
||||
"""
|
||||
try:
|
||||
result = run(["rclone", "version"], capture_output=True, text=True, timeout=10)
|
||||
# Parse "rclone v1.64.2" or "rclone v1.60.1-DEV"
|
||||
match = re.search(r"v(\d+)\.(\d+)\.(\d+)", result.stdout)
|
||||
if match:
|
||||
version = (int(match.group(1)), int(match.group(2)), int(match.group(3)))
|
||||
logger.debug(f"Detected rclone version: {version}")
|
||||
return version
|
||||
except Exception as e:
|
||||
logger.warning(f"Could not determine rclone version: {e}")
|
||||
return None
|
||||
|
||||
|
||||
def supports_create_empty_src_dirs(version: tuple[int, int, int] | None) -> bool:
|
||||
"""Check if installed rclone supports --create-empty-src-dirs flag.
|
||||
|
||||
Returns:
|
||||
True if rclone version >= 1.64.0, False otherwise.
|
||||
"""
|
||||
if version is None:
|
||||
# If we can't determine version, assume older and skip the flag
|
||||
return False
|
||||
return version >= MIN_RCLONE_VERSION_EMPTY_DIRS
|
||||
|
||||
|
||||
@dataclass
|
||||
class SyncProject:
|
||||
"""Project configured for cloud sync.
|
||||
|
||||
Attributes:
|
||||
name: Project name
|
||||
path: Cloud path (e.g., "app/data/research")
|
||||
local_sync_path: Local directory for syncing (optional)
|
||||
"""
|
||||
|
||||
name: str
|
||||
path: str
|
||||
local_sync_path: Optional[str] = None
|
||||
|
||||
|
||||
def get_bmignore_filter_path() -> Path:
|
||||
"""Get path to rclone filter file.
|
||||
|
||||
Uses ~/.basic-memory/.bmignore converted to rclone format.
|
||||
File is automatically created with default patterns on first use.
|
||||
|
||||
Returns:
|
||||
Path to rclone filter file
|
||||
"""
|
||||
# Import here to avoid circular dependency
|
||||
from basic_memory.cli.commands.cloud.bisync_commands import (
|
||||
convert_bmignore_to_rclone_filters,
|
||||
)
|
||||
|
||||
return convert_bmignore_to_rclone_filters()
|
||||
|
||||
|
||||
def get_project_bisync_state(project_name: str) -> Path:
|
||||
"""Get path to project's bisync state directory.
|
||||
|
||||
Args:
|
||||
project_name: Name of the project
|
||||
|
||||
Returns:
|
||||
Path to bisync state directory for this project
|
||||
"""
|
||||
return Path.home() / ".basic-memory" / "bisync-state" / project_name
|
||||
|
||||
|
||||
def bisync_initialized(project_name: str) -> bool:
|
||||
"""Check if bisync has been initialized for this project.
|
||||
|
||||
Args:
|
||||
project_name: Name of the project
|
||||
|
||||
Returns:
|
||||
True if bisync state exists, False otherwise
|
||||
"""
|
||||
state_path = get_project_bisync_state(project_name)
|
||||
return state_path.exists() and any(state_path.iterdir())
|
||||
|
||||
|
||||
def get_project_remote(project: SyncProject, bucket_name: str) -> str:
|
||||
"""Build rclone remote path for project.
|
||||
|
||||
Args:
|
||||
project: Project with cloud path
|
||||
bucket_name: S3 bucket name
|
||||
|
||||
Returns:
|
||||
Remote path like "basic-memory-cloud:bucket-name/basic-memory-llc"
|
||||
|
||||
Note:
|
||||
The API returns paths like "/app/data/basic-memory-llc" because the S3 bucket
|
||||
is mounted at /app/data on the fly machine. We need to strip the /app/data/
|
||||
prefix to get the actual S3 path within the bucket.
|
||||
"""
|
||||
# Normalize path to strip /app/data/ mount point prefix
|
||||
cloud_path = normalize_project_path(project.path).lstrip("/")
|
||||
return f"basic-memory-cloud:{bucket_name}/{cloud_path}"
|
||||
|
||||
|
||||
def project_sync(
|
||||
project: SyncProject,
|
||||
bucket_name: str,
|
||||
dry_run: bool = False,
|
||||
verbose: bool = False,
|
||||
*,
|
||||
run: RunFunc = subprocess.run,
|
||||
is_installed: IsInstalledFunc = is_rclone_installed,
|
||||
filter_path: Path | None = None,
|
||||
) -> bool:
|
||||
"""One-way sync: local → cloud.
|
||||
|
||||
Makes cloud identical to local using rclone sync.
|
||||
|
||||
Args:
|
||||
project: Project to sync
|
||||
bucket_name: S3 bucket name
|
||||
dry_run: Preview changes without applying
|
||||
verbose: Show detailed output
|
||||
|
||||
Returns:
|
||||
True if sync succeeded, False otherwise
|
||||
|
||||
Raises:
|
||||
RcloneError: If project has no local_sync_path configured or rclone not installed
|
||||
"""
|
||||
check_rclone_installed(is_installed=is_installed)
|
||||
|
||||
if not project.local_sync_path:
|
||||
raise RcloneError(f"Project {project.name} has no local_sync_path configured")
|
||||
|
||||
local_path = Path(project.local_sync_path).expanduser()
|
||||
remote_path = get_project_remote(project, bucket_name)
|
||||
filter_path = filter_path or get_bmignore_filter_path()
|
||||
|
||||
cmd = [
|
||||
"rclone",
|
||||
"sync",
|
||||
str(local_path),
|
||||
remote_path,
|
||||
*TIGRIS_CONSISTENCY_HEADERS,
|
||||
"--filter-from",
|
||||
str(filter_path),
|
||||
]
|
||||
|
||||
if verbose:
|
||||
cmd.append("--verbose")
|
||||
else:
|
||||
cmd.append("--progress")
|
||||
|
||||
if dry_run:
|
||||
cmd.append("--dry-run")
|
||||
|
||||
result = run(cmd, text=True)
|
||||
return result.returncode == 0
|
||||
|
||||
|
||||
def project_bisync(
|
||||
project: SyncProject,
|
||||
bucket_name: str,
|
||||
dry_run: bool = False,
|
||||
resync: bool = False,
|
||||
verbose: bool = False,
|
||||
*,
|
||||
run: RunFunc = subprocess.run,
|
||||
is_installed: IsInstalledFunc = is_rclone_installed,
|
||||
version: tuple[int, int, int] | None = None,
|
||||
filter_path: Path | None = None,
|
||||
state_path: Path | None = None,
|
||||
is_initialized: Callable[[str], bool] = bisync_initialized,
|
||||
) -> bool:
|
||||
"""Two-way sync: local ↔ cloud.
|
||||
|
||||
Uses rclone bisync with balanced defaults:
|
||||
- conflict_resolve: newer (auto-resolve to most recent)
|
||||
- max_delete: 25 (safety limit)
|
||||
- compare: modtime (ignore size differences from line ending conversions)
|
||||
- check_access: false (skip for performance)
|
||||
|
||||
Args:
|
||||
project: Project to sync
|
||||
bucket_name: S3 bucket name
|
||||
dry_run: Preview changes without applying
|
||||
resync: Force resync to establish new baseline
|
||||
verbose: Show detailed output
|
||||
|
||||
Returns:
|
||||
True if bisync succeeded, False otherwise
|
||||
|
||||
Raises:
|
||||
RcloneError: If project has no local_sync_path, needs --resync, or rclone not installed
|
||||
"""
|
||||
check_rclone_installed(is_installed=is_installed)
|
||||
|
||||
if not project.local_sync_path:
|
||||
raise RcloneError(f"Project {project.name} has no local_sync_path configured")
|
||||
|
||||
local_path = Path(project.local_sync_path).expanduser()
|
||||
remote_path = get_project_remote(project, bucket_name)
|
||||
filter_path = filter_path or get_bmignore_filter_path()
|
||||
state_path = state_path or get_project_bisync_state(project.name)
|
||||
|
||||
# Ensure state directory exists
|
||||
state_path.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
cmd = [
|
||||
"rclone",
|
||||
"bisync",
|
||||
str(local_path),
|
||||
remote_path,
|
||||
*TIGRIS_CONSISTENCY_HEADERS,
|
||||
"--resilient",
|
||||
"--conflict-resolve=newer",
|
||||
"--max-delete=25",
|
||||
"--compare=modtime", # Ignore size differences from line ending conversions
|
||||
"--filter-from",
|
||||
str(filter_path),
|
||||
"--workdir",
|
||||
str(state_path),
|
||||
]
|
||||
|
||||
# Add --create-empty-src-dirs if rclone version supports it (v1.64+)
|
||||
version = version if version is not None else get_rclone_version(run=run)
|
||||
if supports_create_empty_src_dirs(version):
|
||||
cmd.append("--create-empty-src-dirs")
|
||||
|
||||
if verbose:
|
||||
cmd.append("--verbose")
|
||||
else:
|
||||
cmd.append("--progress")
|
||||
|
||||
if dry_run:
|
||||
cmd.append("--dry-run")
|
||||
|
||||
if resync:
|
||||
cmd.append("--resync")
|
||||
|
||||
# Check if first run requires resync
|
||||
if not resync and not is_initialized(project.name) and not dry_run:
|
||||
raise RcloneError(
|
||||
f"First bisync for {project.name} requires --resync to establish baseline.\n"
|
||||
f"Run: bm project bisync --name {project.name} --resync"
|
||||
)
|
||||
|
||||
result = run(cmd, text=True)
|
||||
return result.returncode == 0
|
||||
|
||||
|
||||
def project_check(
|
||||
project: SyncProject,
|
||||
bucket_name: str,
|
||||
one_way: bool = False,
|
||||
*,
|
||||
run: RunFunc = subprocess.run,
|
||||
is_installed: IsInstalledFunc = is_rclone_installed,
|
||||
filter_path: Path | None = None,
|
||||
) -> bool:
|
||||
"""Check integrity between local and cloud.
|
||||
|
||||
Verifies files match without transferring data.
|
||||
|
||||
Args:
|
||||
project: Project to check
|
||||
bucket_name: S3 bucket name
|
||||
one_way: Only check for missing files on destination (faster)
|
||||
|
||||
Returns:
|
||||
True if files match, False if differences found
|
||||
|
||||
Raises:
|
||||
RcloneError: If project has no local_sync_path configured or rclone not installed
|
||||
"""
|
||||
check_rclone_installed(is_installed=is_installed)
|
||||
|
||||
if not project.local_sync_path:
|
||||
raise RcloneError(f"Project {project.name} has no local_sync_path configured")
|
||||
|
||||
local_path = Path(project.local_sync_path).expanduser()
|
||||
remote_path = get_project_remote(project, bucket_name)
|
||||
filter_path = filter_path or get_bmignore_filter_path()
|
||||
|
||||
cmd = [
|
||||
"rclone",
|
||||
"check",
|
||||
str(local_path),
|
||||
remote_path,
|
||||
*TIGRIS_CONSISTENCY_HEADERS,
|
||||
"--filter-from",
|
||||
str(filter_path),
|
||||
]
|
||||
|
||||
if one_way:
|
||||
cmd.append("--one-way")
|
||||
|
||||
result = run(cmd, capture_output=True, text=True)
|
||||
return result.returncode == 0
|
||||
|
||||
|
||||
def project_ls(
|
||||
project: SyncProject,
|
||||
bucket_name: str,
|
||||
path: Optional[str] = None,
|
||||
*,
|
||||
run: RunFunc = subprocess.run,
|
||||
is_installed: IsInstalledFunc = is_rclone_installed,
|
||||
) -> list[str]:
|
||||
"""List files in remote project.
|
||||
|
||||
Args:
|
||||
project: Project to list files from
|
||||
bucket_name: S3 bucket name
|
||||
path: Optional subdirectory within project
|
||||
|
||||
Returns:
|
||||
List of file paths
|
||||
|
||||
Raises:
|
||||
subprocess.CalledProcessError: If rclone command fails
|
||||
RcloneError: If rclone is not installed
|
||||
"""
|
||||
check_rclone_installed(is_installed=is_installed)
|
||||
|
||||
remote_path = get_project_remote(project, bucket_name)
|
||||
if path:
|
||||
remote_path = f"{remote_path}/{path}"
|
||||
|
||||
cmd = ["rclone", "ls", *TIGRIS_CONSISTENCY_HEADERS, remote_path]
|
||||
result = run(cmd, capture_output=True, text=True, check=True)
|
||||
return result.stdout.splitlines()
|
||||
@@ -1,110 +0,0 @@
|
||||
"""rclone configuration management for Basic Memory Cloud.
|
||||
|
||||
This module provides simplified rclone configuration for SPEC-20.
|
||||
Uses a single "basic-memory-cloud" remote for all operations.
|
||||
"""
|
||||
|
||||
import configparser
|
||||
import os
|
||||
import shutil
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
|
||||
from rich.console import Console
|
||||
|
||||
console = Console()
|
||||
|
||||
|
||||
class RcloneConfigError(Exception):
|
||||
"""Exception raised for rclone configuration errors."""
|
||||
|
||||
pass
|
||||
|
||||
|
||||
def get_rclone_config_path() -> Path:
|
||||
"""Get the path to rclone configuration file."""
|
||||
config_dir = Path.home() / ".config" / "rclone"
|
||||
config_dir.mkdir(parents=True, exist_ok=True)
|
||||
return config_dir / "rclone.conf"
|
||||
|
||||
|
||||
def backup_rclone_config() -> Optional[Path]:
|
||||
"""Create a backup of existing rclone config."""
|
||||
config_path = get_rclone_config_path()
|
||||
if not config_path.exists():
|
||||
return None
|
||||
|
||||
backup_path = config_path.with_suffix(f".conf.backup-{os.getpid()}")
|
||||
shutil.copy2(config_path, backup_path)
|
||||
console.print(f"[dim]Created backup: {backup_path}[/dim]")
|
||||
return backup_path
|
||||
|
||||
|
||||
def load_rclone_config() -> configparser.ConfigParser:
|
||||
"""Load existing rclone configuration."""
|
||||
config = configparser.ConfigParser()
|
||||
config_path = get_rclone_config_path()
|
||||
|
||||
if config_path.exists():
|
||||
config.read(config_path)
|
||||
|
||||
return config
|
||||
|
||||
|
||||
def save_rclone_config(config: configparser.ConfigParser) -> None:
|
||||
"""Save rclone configuration to file."""
|
||||
config_path = get_rclone_config_path()
|
||||
|
||||
with open(config_path, "w") as f:
|
||||
config.write(f)
|
||||
|
||||
console.print(f"[dim]Updated rclone config: {config_path}[/dim]")
|
||||
|
||||
|
||||
def configure_rclone_remote(
|
||||
access_key: str,
|
||||
secret_key: str,
|
||||
endpoint: str = "https://fly.storage.tigris.dev",
|
||||
region: str = "auto",
|
||||
) -> str:
|
||||
"""Configure single rclone remote named 'basic-memory-cloud'.
|
||||
|
||||
This is the simplified approach from SPEC-20 that uses one remote
|
||||
for all Basic Memory cloud operations (not tenant-specific).
|
||||
|
||||
Args:
|
||||
access_key: S3 access key ID
|
||||
secret_key: S3 secret access key
|
||||
endpoint: S3-compatible endpoint URL
|
||||
region: S3 region (default: auto)
|
||||
|
||||
Returns:
|
||||
The remote name: "basic-memory-cloud"
|
||||
"""
|
||||
# Backup existing config
|
||||
backup_rclone_config()
|
||||
|
||||
# Load existing config
|
||||
config = load_rclone_config()
|
||||
|
||||
# Single remote name (not tenant-specific)
|
||||
REMOTE_NAME = "basic-memory-cloud"
|
||||
|
||||
# Add/update the remote section
|
||||
if not config.has_section(REMOTE_NAME):
|
||||
config.add_section(REMOTE_NAME)
|
||||
|
||||
config.set(REMOTE_NAME, "type", "s3")
|
||||
config.set(REMOTE_NAME, "provider", "Other")
|
||||
config.set(REMOTE_NAME, "access_key_id", access_key)
|
||||
config.set(REMOTE_NAME, "secret_access_key", secret_key)
|
||||
config.set(REMOTE_NAME, "endpoint", endpoint)
|
||||
config.set(REMOTE_NAME, "region", region)
|
||||
# Prevent unnecessary encoding of filenames (only encode slashes and invalid UTF-8)
|
||||
# This prevents files with spaces like "Hello World.md" from being quoted
|
||||
config.set(REMOTE_NAME, "encoding", "Slash,InvalidUtf8")
|
||||
# Save updated config
|
||||
save_rclone_config(config)
|
||||
|
||||
console.print(f"[green]Configured rclone remote: {REMOTE_NAME}[/green]")
|
||||
return REMOTE_NAME
|
||||
@@ -1,263 +0,0 @@
|
||||
"""Cross-platform rclone installation utilities."""
|
||||
|
||||
import os
|
||||
import platform
|
||||
import shutil
|
||||
import subprocess
|
||||
from typing import Optional
|
||||
|
||||
from rich.console import Console
|
||||
|
||||
console = Console()
|
||||
|
||||
|
||||
class RcloneInstallError(Exception):
|
||||
"""Exception raised for rclone installation errors."""
|
||||
|
||||
pass
|
||||
|
||||
|
||||
def is_rclone_installed() -> bool:
|
||||
"""Check if rclone is already installed and available in PATH."""
|
||||
return shutil.which("rclone") is not None
|
||||
|
||||
|
||||
def get_platform() -> str:
|
||||
"""Get the current platform identifier."""
|
||||
system = platform.system().lower()
|
||||
if system == "darwin":
|
||||
return "macos"
|
||||
elif system == "linux":
|
||||
return "linux"
|
||||
elif system == "windows":
|
||||
return "windows"
|
||||
else:
|
||||
raise RcloneInstallError(f"Unsupported platform: {system}")
|
||||
|
||||
|
||||
def run_command(command: list[str], check: bool = True) -> subprocess.CompletedProcess:
|
||||
"""Run a command with proper error handling."""
|
||||
try:
|
||||
console.print(f"[dim]Running: {' '.join(command)}[/dim]")
|
||||
result = subprocess.run(command, capture_output=True, text=True, check=check)
|
||||
if result.stdout:
|
||||
console.print(f"[dim]Output: {result.stdout.strip()}[/dim]")
|
||||
return result
|
||||
except subprocess.CalledProcessError as e:
|
||||
console.print(f"[red]Command failed: {e}[/red]")
|
||||
if e.stderr:
|
||||
console.print(f"[red]Error output: {e.stderr}[/red]")
|
||||
raise RcloneInstallError(f"Command failed: {e}") from e
|
||||
except FileNotFoundError as e:
|
||||
raise RcloneInstallError(f"Command not found: {' '.join(command)}") from e
|
||||
|
||||
|
||||
def install_rclone_macos() -> None:
|
||||
"""Install rclone on macOS using Homebrew or official script."""
|
||||
# Try Homebrew first
|
||||
if shutil.which("brew"):
|
||||
try:
|
||||
console.print("[blue]Installing rclone via Homebrew...[/blue]")
|
||||
run_command(["brew", "install", "rclone"])
|
||||
console.print("[green]rclone installed via Homebrew[/green]")
|
||||
return
|
||||
except RcloneInstallError:
|
||||
console.print(
|
||||
"[yellow]Homebrew installation failed, trying official script...[/yellow]"
|
||||
)
|
||||
|
||||
# Fallback to official script
|
||||
console.print("[blue]Installing rclone via official script...[/blue]")
|
||||
try:
|
||||
run_command(["sh", "-c", "curl https://rclone.org/install.sh | sudo bash"])
|
||||
console.print("[green]rclone installed via official script[/green]")
|
||||
except RcloneInstallError:
|
||||
raise RcloneInstallError(
|
||||
"Failed to install rclone. Please install manually: brew install rclone"
|
||||
)
|
||||
|
||||
|
||||
def install_rclone_linux() -> None:
|
||||
"""Install rclone on Linux using package managers or official script."""
|
||||
# Try snap first (most universal)
|
||||
if shutil.which("snap"):
|
||||
try:
|
||||
console.print("[blue]Installing rclone via snap...[/blue]")
|
||||
run_command(["sudo", "snap", "install", "rclone"])
|
||||
console.print("[green]rclone installed via snap[/green]")
|
||||
return
|
||||
except RcloneInstallError:
|
||||
console.print("[yellow]Snap installation failed, trying apt...[/yellow]")
|
||||
|
||||
# Try apt (Debian/Ubuntu)
|
||||
if shutil.which("apt"):
|
||||
try:
|
||||
console.print("[blue]Installing rclone via apt...[/blue]")
|
||||
run_command(["sudo", "apt", "update"])
|
||||
run_command(["sudo", "apt", "install", "-y", "rclone"])
|
||||
console.print("[green]rclone installed via apt[/green]")
|
||||
return
|
||||
except RcloneInstallError:
|
||||
console.print("[yellow]apt installation failed, trying official script...[/yellow]")
|
||||
|
||||
# Fallback to official script
|
||||
console.print("[blue]Installing rclone via official script...[/blue]")
|
||||
try:
|
||||
run_command(["sh", "-c", "curl https://rclone.org/install.sh | sudo bash"])
|
||||
console.print("[green]rclone installed via official script[/green]")
|
||||
except RcloneInstallError:
|
||||
raise RcloneInstallError(
|
||||
"Failed to install rclone. Please install manually: sudo snap install rclone"
|
||||
)
|
||||
|
||||
|
||||
def install_rclone_windows() -> None:
|
||||
"""Install rclone on Windows using package managers."""
|
||||
# Try winget first (built into Windows 10+)
|
||||
if shutil.which("winget"):
|
||||
try:
|
||||
console.print("[blue]Installing rclone via winget...[/blue]")
|
||||
run_command(
|
||||
[
|
||||
"winget",
|
||||
"install",
|
||||
"Rclone.Rclone",
|
||||
"--accept-source-agreements",
|
||||
"--accept-package-agreements",
|
||||
]
|
||||
)
|
||||
console.print("[green]rclone installed via winget[/green]")
|
||||
return
|
||||
except RcloneInstallError:
|
||||
console.print("[yellow]winget installation failed, trying chocolatey...[/yellow]")
|
||||
|
||||
# Try chocolatey
|
||||
if shutil.which("choco"):
|
||||
try:
|
||||
console.print("[blue]Installing rclone via chocolatey...[/blue]")
|
||||
run_command(["choco", "install", "rclone", "-y"])
|
||||
console.print("[green]rclone installed via chocolatey[/green]")
|
||||
return
|
||||
except RcloneInstallError:
|
||||
console.print("[yellow]chocolatey installation failed, trying scoop...[/yellow]")
|
||||
|
||||
# Try scoop
|
||||
if shutil.which("scoop"):
|
||||
try:
|
||||
console.print("[blue]Installing rclone via scoop...[/blue]")
|
||||
run_command(["scoop", "install", "rclone"])
|
||||
console.print("[green]rclone installed via scoop[/green]")
|
||||
return
|
||||
except RcloneInstallError:
|
||||
console.print("[yellow]scoop installation failed[/yellow]")
|
||||
|
||||
# No package manager available - provide detailed instructions
|
||||
error_msg = (
|
||||
"Could not install rclone automatically.\n\n"
|
||||
"Windows requires a package manager to install rclone. Options:\n\n"
|
||||
"1. Install winget (recommended, built into Windows 11):\n"
|
||||
" - Windows 11: Already installed\n"
|
||||
" - Windows 10: Install 'App Installer' from Microsoft Store\n"
|
||||
" - Then run: bm cloud setup\n\n"
|
||||
"2. Install chocolatey:\n"
|
||||
" - Visit: https://chocolatey.org/install\n"
|
||||
" - Then run: bm cloud setup\n\n"
|
||||
"3. Install scoop:\n"
|
||||
" - Visit: https://scoop.sh\n"
|
||||
" - Then run: bm cloud setup\n\n"
|
||||
"4. Manual installation:\n"
|
||||
" - Download from: https://rclone.org/downloads/\n"
|
||||
" - Extract and add to PATH\n"
|
||||
)
|
||||
raise RcloneInstallError(error_msg)
|
||||
|
||||
|
||||
def install_rclone(platform_override: Optional[str] = None) -> None:
|
||||
"""Install rclone for the current platform."""
|
||||
if is_rclone_installed():
|
||||
console.print("[green]rclone is already installed[/green]")
|
||||
return
|
||||
|
||||
platform_name = platform_override or get_platform()
|
||||
console.print(f"[blue]Installing rclone for {platform_name}...[/blue]")
|
||||
|
||||
try:
|
||||
if platform_name == "macos":
|
||||
install_rclone_macos()
|
||||
elif platform_name == "linux":
|
||||
install_rclone_linux()
|
||||
elif platform_name == "windows":
|
||||
install_rclone_windows()
|
||||
refresh_windows_path()
|
||||
else:
|
||||
raise RcloneInstallError(f"Unsupported platform: {platform_name}")
|
||||
|
||||
# Verify installation
|
||||
if not is_rclone_installed():
|
||||
raise RcloneInstallError("rclone installation completed but command not found in PATH")
|
||||
|
||||
console.print("[green]rclone installation completed successfully[/green]")
|
||||
|
||||
except RcloneInstallError:
|
||||
raise
|
||||
except Exception as e:
|
||||
raise RcloneInstallError(f"Unexpected error during installation: {e}") from e
|
||||
|
||||
|
||||
def refresh_windows_path() -> None:
|
||||
"""Refresh the Windows PATH environment variable for the current session."""
|
||||
if platform.system().lower() != "windows":
|
||||
return
|
||||
|
||||
# Importing here after performing platform detection. Also note that we have to ignore pylance/pyright
|
||||
# warnings about winreg attributes so that "errors" don't appear on non-Windows platforms.
|
||||
import winreg
|
||||
|
||||
user_key_path = r"Environment"
|
||||
system_key_path = r"System\CurrentControlSet\Control\Session Manager\Environment"
|
||||
new_path = ""
|
||||
|
||||
# Read user PATH
|
||||
try:
|
||||
reg_key = winreg.OpenKey(winreg.HKEY_CURRENT_USER, user_key_path, 0, winreg.KEY_READ) # type: ignore[reportAttributeAccessIssue]
|
||||
user_path, _ = winreg.QueryValueEx(reg_key, "PATH") # type: ignore[reportAttributeAccessIssue]
|
||||
winreg.CloseKey(reg_key) # type: ignore[reportAttributeAccessIssue]
|
||||
except Exception:
|
||||
user_path = ""
|
||||
|
||||
# Read system PATH
|
||||
try:
|
||||
reg_key = winreg.OpenKey(winreg.HKEY_LOCAL_MACHINE, system_key_path, 0, winreg.KEY_READ) # type: ignore[reportAttributeAccessIssue]
|
||||
system_path, _ = winreg.QueryValueEx(reg_key, "PATH") # type: ignore[reportAttributeAccessIssue]
|
||||
winreg.CloseKey(reg_key) # type: ignore[reportAttributeAccessIssue]
|
||||
except Exception:
|
||||
system_path = ""
|
||||
|
||||
# Merge user and system PATHs (system first, then user)
|
||||
if system_path and user_path:
|
||||
new_path = system_path + ";" + user_path
|
||||
elif system_path:
|
||||
new_path = system_path
|
||||
elif user_path:
|
||||
new_path = user_path
|
||||
|
||||
if new_path:
|
||||
os.environ["PATH"] = new_path
|
||||
|
||||
|
||||
def get_rclone_version() -> Optional[str]:
|
||||
"""Get the installed rclone version."""
|
||||
if not is_rclone_installed():
|
||||
return None
|
||||
|
||||
try:
|
||||
result = run_command(["rclone", "version"], check=False)
|
||||
if result.returncode == 0:
|
||||
# Parse version from output (format: "rclone v1.64.0")
|
||||
lines = result.stdout.strip().split("\n")
|
||||
for line in lines:
|
||||
if line.startswith("rclone v"):
|
||||
return line.split()[1]
|
||||
return "unknown"
|
||||
except Exception:
|
||||
return "unknown"
|
||||
@@ -1,159 +0,0 @@
|
||||
"""Restore CLI commands for Basic Memory Cloud.
|
||||
|
||||
SPEC-29 Phase 3: CLI commands for restoring files from Tigris bucket snapshots.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
|
||||
import typer
|
||||
from rich.console import Console
|
||||
|
||||
from basic_memory.cli.app import cloud_app
|
||||
from basic_memory.cli.commands.cloud.api_client import (
|
||||
CloudAPIError,
|
||||
SubscriptionRequiredError,
|
||||
make_api_request,
|
||||
)
|
||||
from basic_memory.cli.commands.cloud.schemas import BucketSnapshotBrowseResponse
|
||||
from basic_memory.config import ConfigManager
|
||||
|
||||
console = Console()
|
||||
|
||||
|
||||
@cloud_app.command("restore")
|
||||
def restore(
|
||||
path: str = typer.Argument(
|
||||
...,
|
||||
help="Path to restore (file or folder, e.g., 'notes/project.md' or 'research/')",
|
||||
),
|
||||
snapshot_id: str = typer.Option(
|
||||
...,
|
||||
"--snapshot",
|
||||
"-s",
|
||||
help="ID of the snapshot to restore from",
|
||||
),
|
||||
force: bool = typer.Option(
|
||||
False,
|
||||
"--force",
|
||||
"-f",
|
||||
help="Skip confirmation prompt",
|
||||
),
|
||||
) -> None:
|
||||
"""Restore a file or folder from a snapshot.
|
||||
|
||||
This command restores files from a previous snapshot to the current bucket.
|
||||
The restored files will overwrite any existing files at the same path.
|
||||
|
||||
Examples:
|
||||
bm cloud restore notes/project.md --snapshot abc123
|
||||
bm cloud restore research/ --snapshot abc123
|
||||
bm cloud restore notes/project.md --snapshot abc123 --force
|
||||
"""
|
||||
|
||||
async def _restore():
|
||||
try:
|
||||
config_manager = ConfigManager()
|
||||
config = config_manager.config
|
||||
host_url = config.cloud_host.rstrip("/")
|
||||
|
||||
# Normalize path - remove leading slash if present
|
||||
normalized_path = path.lstrip("/")
|
||||
|
||||
if not force:
|
||||
# Show what will be restored
|
||||
console.print(f"[blue]Preparing to restore from snapshot {snapshot_id}[/blue]")
|
||||
console.print(f" Path: {normalized_path}")
|
||||
|
||||
# Try to browse the snapshot to show what files will be affected
|
||||
try:
|
||||
browse_url = f"{host_url}/api/bucket-snapshots/{snapshot_id}/browse"
|
||||
if normalized_path:
|
||||
browse_url += f"?prefix={normalized_path}"
|
||||
|
||||
response = await make_api_request(
|
||||
method="GET",
|
||||
url=browse_url,
|
||||
)
|
||||
browse_response = BucketSnapshotBrowseResponse.model_validate(response.json())
|
||||
|
||||
if browse_response.files:
|
||||
if len(browse_response.files) <= 10:
|
||||
console.print("\n Files to restore:")
|
||||
for file_info in browse_response.files:
|
||||
console.print(f" - {file_info.key}")
|
||||
else:
|
||||
console.print(
|
||||
f"\n {len(browse_response.files)} files will be restored"
|
||||
)
|
||||
console.print(" First 5 files:")
|
||||
for file_info in browse_response.files[:5]:
|
||||
console.print(f" - {file_info.key}")
|
||||
console.print(f" ... and {len(browse_response.files) - 5} more")
|
||||
else:
|
||||
console.print(
|
||||
f"\n[yellow]No files found matching '{normalized_path}' "
|
||||
f"in snapshot[/yellow]"
|
||||
)
|
||||
raise typer.Exit(0)
|
||||
|
||||
except CloudAPIError as browse_error:
|
||||
if browse_error.status_code == 404:
|
||||
console.print(f"[red]Snapshot not found: {snapshot_id}[/red]")
|
||||
raise typer.Exit(1)
|
||||
# If browse fails for other reasons, proceed with confirmation anyway
|
||||
pass
|
||||
|
||||
console.print(
|
||||
"\n[yellow]Warning: Restored files will overwrite existing files![/yellow]"
|
||||
)
|
||||
confirmed = typer.confirm("\nProceed with restore?")
|
||||
if not confirmed:
|
||||
console.print("[yellow]Restore cancelled[/yellow]")
|
||||
raise typer.Exit(0)
|
||||
|
||||
console.print(f"[blue]Restoring from snapshot {snapshot_id}...[/blue]")
|
||||
|
||||
response = await make_api_request(
|
||||
method="POST",
|
||||
url=f"{host_url}/api/bucket-snapshots/{snapshot_id}/restore",
|
||||
json_data={"path": normalized_path},
|
||||
)
|
||||
|
||||
data = response.json()
|
||||
restored_files = data.get("restored", [])
|
||||
returned_snapshot_id = data.get("snapshot_id", snapshot_id)
|
||||
|
||||
if restored_files:
|
||||
console.print(f"[green]Successfully restored {len(restored_files)} file(s)[/green]")
|
||||
if len(restored_files) <= 10:
|
||||
for file_path in restored_files:
|
||||
console.print(f" - {file_path}")
|
||||
else:
|
||||
console.print(" First 5 restored files:")
|
||||
for file_path in restored_files[:5]:
|
||||
console.print(f" - {file_path}")
|
||||
console.print(f" ... and {len(restored_files) - 5} more")
|
||||
console.print(f"\n[dim]Snapshot ID: {returned_snapshot_id}[/dim]")
|
||||
else:
|
||||
console.print("[yellow]No files were restored[/yellow]")
|
||||
console.print(f"[dim]No files matching '{normalized_path}' found in snapshot[/dim]")
|
||||
|
||||
except typer.Exit:
|
||||
# Re-raise typer.Exit without modification - it's used for clean exits
|
||||
raise
|
||||
except SubscriptionRequiredError as e:
|
||||
console.print("\n[red]Subscription Required[/red]\n")
|
||||
console.print(f"[yellow]{e.args[0]}[/yellow]\n")
|
||||
console.print(f"Subscribe at: [blue underline]{e.subscribe_url}[/blue underline]\n")
|
||||
raise typer.Exit(1)
|
||||
except CloudAPIError as e:
|
||||
if e.status_code == 404:
|
||||
console.print(f"[red]Snapshot not found: {snapshot_id}[/red]")
|
||||
else:
|
||||
console.print(f"[red]Failed to restore: {e}[/red]")
|
||||
raise typer.Exit(1)
|
||||
except Exception as e:
|
||||
console.print(f"[red]Unexpected error: {e}[/red]")
|
||||
raise typer.Exit(1)
|
||||
|
||||
asyncio.run(_restore())
|
||||
@@ -1,55 +0,0 @@
|
||||
"""Pydantic schemas for Basic Memory Cloud API responses.
|
||||
|
||||
These schemas mirror the API response models from basic-memory-cloud
|
||||
for type-safe parsing of API responses in CLI commands.
|
||||
"""
|
||||
|
||||
from datetime import datetime
|
||||
from uuid import UUID
|
||||
|
||||
from pydantic import BaseModel
|
||||
|
||||
|
||||
class BucketSnapshotFileInfo(BaseModel):
|
||||
"""File info from snapshot browse response."""
|
||||
|
||||
key: str
|
||||
size: int
|
||||
last_modified: datetime
|
||||
etag: str | None = None
|
||||
|
||||
|
||||
class BucketSnapshotBrowseResponse(BaseModel):
|
||||
"""Response from browsing snapshot contents."""
|
||||
|
||||
files: list[BucketSnapshotFileInfo]
|
||||
prefix: str
|
||||
snapshot_version: str
|
||||
|
||||
|
||||
class BucketSnapshotResponse(BaseModel):
|
||||
"""Response model for bucket snapshot data."""
|
||||
|
||||
id: UUID
|
||||
bucket_name: str
|
||||
snapshot_version: str
|
||||
name: str
|
||||
description: str | None
|
||||
auto: bool
|
||||
created_at: datetime
|
||||
created_by: UUID | None = None
|
||||
|
||||
|
||||
class BucketSnapshotListResponse(BaseModel):
|
||||
"""Response from listing bucket snapshots."""
|
||||
|
||||
snapshots: list[BucketSnapshotResponse]
|
||||
total: int
|
||||
|
||||
|
||||
class BucketSnapshotRestoreResponse(BaseModel):
|
||||
"""Response from restore operation."""
|
||||
|
||||
restored: list[str]
|
||||
snapshot_version: str
|
||||
snapshot_id: UUID
|
||||
@@ -1,370 +0,0 @@
|
||||
"""Snapshot CLI commands for Basic Memory Cloud.
|
||||
|
||||
SPEC-29 Phase 3: CLI commands for managing Tigris bucket snapshots.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
from datetime import datetime
|
||||
from typing import Optional
|
||||
|
||||
import typer
|
||||
from rich.console import Console
|
||||
from rich.table import Table
|
||||
|
||||
from basic_memory.cli.commands.cloud.api_client import (
|
||||
CloudAPIError,
|
||||
SubscriptionRequiredError,
|
||||
make_api_request,
|
||||
)
|
||||
from basic_memory.cli.commands.cloud.schemas import BucketSnapshotBrowseResponse
|
||||
from basic_memory.config import ConfigManager
|
||||
|
||||
console = Console()
|
||||
snapshot_app = typer.Typer(help="Manage bucket snapshots")
|
||||
|
||||
|
||||
def _format_timestamp(iso_timestamp: str) -> str:
|
||||
"""Format ISO timestamp to a human-readable format."""
|
||||
try:
|
||||
dt = datetime.fromisoformat(iso_timestamp.replace("Z", "+00:00"))
|
||||
return dt.strftime("%Y-%m-%d %H:%M:%S")
|
||||
except (ValueError, AttributeError):
|
||||
return iso_timestamp
|
||||
|
||||
|
||||
@snapshot_app.command("create")
|
||||
def create(
|
||||
description: str = typer.Argument(
|
||||
...,
|
||||
help="Description for the snapshot",
|
||||
),
|
||||
) -> None:
|
||||
"""Create a new bucket snapshot.
|
||||
|
||||
Examples:
|
||||
bm cloud snapshot create "before major refactor"
|
||||
bm cloud snapshot create "daily backup"
|
||||
"""
|
||||
|
||||
async def _create():
|
||||
try:
|
||||
config_manager = ConfigManager()
|
||||
config = config_manager.config
|
||||
host_url = config.cloud_host.rstrip("/")
|
||||
|
||||
console.print("[blue]Creating snapshot...[/blue]")
|
||||
|
||||
response = await make_api_request(
|
||||
method="POST",
|
||||
url=f"{host_url}/api/bucket-snapshots",
|
||||
json_data={"description": description},
|
||||
)
|
||||
|
||||
data = response.json()
|
||||
snapshot_id = data.get("id", "unknown")
|
||||
snapshot_version = data.get("snapshot_version", "unknown")
|
||||
created_at = _format_timestamp(data.get("created_at", ""))
|
||||
|
||||
console.print("[green]Snapshot created successfully[/green]")
|
||||
console.print(f" ID: {snapshot_id}")
|
||||
console.print(f" Version: {snapshot_version}")
|
||||
console.print(f" Created: {created_at}")
|
||||
console.print(f" Description: {description}")
|
||||
|
||||
except SubscriptionRequiredError as e:
|
||||
console.print("\n[red]Subscription Required[/red]\n")
|
||||
console.print(f"[yellow]{e.args[0]}[/yellow]\n")
|
||||
console.print(f"Subscribe at: [blue underline]{e.subscribe_url}[/blue underline]\n")
|
||||
raise typer.Exit(1)
|
||||
except CloudAPIError as e:
|
||||
console.print(f"[red]Failed to create snapshot: {e}[/red]")
|
||||
raise typer.Exit(1)
|
||||
except Exception as e:
|
||||
console.print(f"[red]Unexpected error: {e}[/red]")
|
||||
raise typer.Exit(1)
|
||||
|
||||
asyncio.run(_create())
|
||||
|
||||
|
||||
@snapshot_app.command("list")
|
||||
def list_snapshots(
|
||||
limit: int = typer.Option(
|
||||
10,
|
||||
"--limit",
|
||||
"-l",
|
||||
help="Maximum number of snapshots to display",
|
||||
),
|
||||
) -> None:
|
||||
"""List all bucket snapshots.
|
||||
|
||||
Examples:
|
||||
bm cloud snapshot list
|
||||
bm cloud snapshot list --limit 20
|
||||
"""
|
||||
|
||||
async def _list():
|
||||
try:
|
||||
config_manager = ConfigManager()
|
||||
config = config_manager.config
|
||||
host_url = config.cloud_host.rstrip("/")
|
||||
|
||||
console.print("[blue]Fetching snapshots...[/blue]")
|
||||
|
||||
response = await make_api_request(
|
||||
method="GET",
|
||||
url=f"{host_url}/api/bucket-snapshots",
|
||||
)
|
||||
|
||||
data = response.json()
|
||||
snapshots = data.get("snapshots", [])
|
||||
total = data.get("total", len(snapshots))
|
||||
|
||||
if not snapshots:
|
||||
console.print("[yellow]No snapshots found[/yellow]")
|
||||
console.print(
|
||||
'\n[dim]Create a snapshot with: bm cloud snapshot create "description"[/dim]'
|
||||
)
|
||||
return
|
||||
|
||||
# Create a table for displaying snapshots
|
||||
table = Table(title=f"Bucket Snapshots ({total} total)")
|
||||
table.add_column("ID", style="cyan", no_wrap=True)
|
||||
table.add_column("Description", style="white")
|
||||
table.add_column("Auto", style="dim")
|
||||
table.add_column("Created", style="green")
|
||||
|
||||
for snapshot in snapshots[:limit]:
|
||||
snapshot_id = snapshot.get("id", "unknown")
|
||||
desc = snapshot.get("description") or snapshot.get("name", "-")
|
||||
auto = "yes" if snapshot.get("auto", False) else "no"
|
||||
created_at = _format_timestamp(snapshot.get("created_at", ""))
|
||||
|
||||
table.add_row(snapshot_id, desc, auto, created_at)
|
||||
|
||||
console.print(table)
|
||||
|
||||
if total > limit:
|
||||
console.print(
|
||||
f"\n[dim]Showing {limit} of {total} snapshots. Use --limit to see more.[/dim]"
|
||||
)
|
||||
|
||||
except SubscriptionRequiredError as e:
|
||||
console.print("\n[red]Subscription Required[/red]\n")
|
||||
console.print(f"[yellow]{e.args[0]}[/yellow]\n")
|
||||
console.print(f"Subscribe at: [blue underline]{e.subscribe_url}[/blue underline]\n")
|
||||
raise typer.Exit(1)
|
||||
except CloudAPIError as e:
|
||||
console.print(f"[red]Failed to list snapshots: {e}[/red]")
|
||||
raise typer.Exit(1)
|
||||
except Exception as e:
|
||||
console.print(f"[red]Unexpected error: {e}[/red]")
|
||||
raise typer.Exit(1)
|
||||
|
||||
asyncio.run(_list())
|
||||
|
||||
|
||||
@snapshot_app.command("delete")
|
||||
def delete(
|
||||
snapshot_id: str = typer.Argument(
|
||||
...,
|
||||
help="The ID of the snapshot to delete",
|
||||
),
|
||||
force: bool = typer.Option(
|
||||
False,
|
||||
"--force",
|
||||
"-f",
|
||||
help="Skip confirmation prompt",
|
||||
),
|
||||
) -> None:
|
||||
"""Delete a bucket snapshot.
|
||||
|
||||
Examples:
|
||||
bm cloud snapshot delete abc123
|
||||
bm cloud snapshot delete abc123 --force
|
||||
"""
|
||||
|
||||
async def _delete():
|
||||
try:
|
||||
config_manager = ConfigManager()
|
||||
config = config_manager.config
|
||||
host_url = config.cloud_host.rstrip("/")
|
||||
|
||||
if not force:
|
||||
# Fetch snapshot details first to show what will be deleted
|
||||
console.print("[blue]Fetching snapshot details...[/blue]")
|
||||
try:
|
||||
response = await make_api_request(
|
||||
method="GET",
|
||||
url=f"{host_url}/api/bucket-snapshots/{snapshot_id}",
|
||||
)
|
||||
data = response.json()
|
||||
desc = data.get("description") or data.get("name", "unnamed")
|
||||
created_at = _format_timestamp(data.get("created_at", ""))
|
||||
console.print("\nSnapshot to delete:")
|
||||
console.print(f" ID: {snapshot_id}")
|
||||
console.print(f" Description: {desc}")
|
||||
console.print(f" Created: {created_at}")
|
||||
except CloudAPIError:
|
||||
# If we can't fetch details, proceed with confirmation anyway
|
||||
pass
|
||||
|
||||
confirmed = typer.confirm("\nAre you sure you want to delete this snapshot?")
|
||||
if not confirmed:
|
||||
console.print("[yellow]Deletion cancelled[/yellow]")
|
||||
raise typer.Exit(0)
|
||||
|
||||
console.print("[blue]Deleting snapshot...[/blue]")
|
||||
|
||||
await make_api_request(
|
||||
method="DELETE",
|
||||
url=f"{host_url}/api/bucket-snapshots/{snapshot_id}",
|
||||
)
|
||||
|
||||
console.print(f"[green]Snapshot {snapshot_id} deleted successfully[/green]")
|
||||
|
||||
except typer.Exit:
|
||||
# Re-raise typer.Exit without modification - it's used for clean exits
|
||||
raise
|
||||
except SubscriptionRequiredError as e:
|
||||
console.print("\n[red]Subscription Required[/red]\n")
|
||||
console.print(f"[yellow]{e.args[0]}[/yellow]\n")
|
||||
console.print(f"Subscribe at: [blue underline]{e.subscribe_url}[/blue underline]\n")
|
||||
raise typer.Exit(1)
|
||||
except CloudAPIError as e:
|
||||
if e.status_code == 404:
|
||||
console.print(f"[red]Snapshot not found: {snapshot_id}[/red]")
|
||||
else:
|
||||
console.print(f"[red]Failed to delete snapshot: {e}[/red]")
|
||||
raise typer.Exit(1)
|
||||
except Exception as e:
|
||||
console.print(f"[red]Unexpected error: {e}[/red]")
|
||||
raise typer.Exit(1)
|
||||
|
||||
asyncio.run(_delete())
|
||||
|
||||
|
||||
@snapshot_app.command("show")
|
||||
def show(
|
||||
snapshot_id: str = typer.Argument(
|
||||
...,
|
||||
help="The ID of the snapshot to show",
|
||||
),
|
||||
) -> None:
|
||||
"""Show details of a specific snapshot.
|
||||
|
||||
Examples:
|
||||
bm cloud snapshot show abc123
|
||||
"""
|
||||
|
||||
async def _show():
|
||||
try:
|
||||
config_manager = ConfigManager()
|
||||
config = config_manager.config
|
||||
host_url = config.cloud_host.rstrip("/")
|
||||
|
||||
response = await make_api_request(
|
||||
method="GET",
|
||||
url=f"{host_url}/api/bucket-snapshots/{snapshot_id}",
|
||||
)
|
||||
|
||||
data = response.json()
|
||||
|
||||
console.print("[bold blue]Snapshot Details[/bold blue]")
|
||||
console.print(f" ID: {data.get('id', 'unknown')}")
|
||||
console.print(f" Bucket: {data.get('bucket_name', 'unknown')}")
|
||||
console.print(f" Version: {data.get('snapshot_version', 'unknown')}")
|
||||
console.print(f" Name: {data.get('name', '-')}")
|
||||
console.print(f" Description: {data.get('description') or '-'}")
|
||||
console.print(f" Auto: {'yes' if data.get('auto', False) else 'no'}")
|
||||
console.print(f" Created: {_format_timestamp(data.get('created_at', ''))}")
|
||||
|
||||
except SubscriptionRequiredError as e:
|
||||
console.print("\n[red]Subscription Required[/red]\n")
|
||||
console.print(f"[yellow]{e.args[0]}[/yellow]\n")
|
||||
console.print(f"Subscribe at: [blue underline]{e.subscribe_url}[/blue underline]\n")
|
||||
raise typer.Exit(1)
|
||||
except CloudAPIError as e:
|
||||
if e.status_code == 404:
|
||||
console.print(f"[red]Snapshot not found: {snapshot_id}[/red]")
|
||||
else:
|
||||
console.print(f"[red]Failed to get snapshot details: {e}[/red]")
|
||||
raise typer.Exit(1)
|
||||
except Exception as e:
|
||||
console.print(f"[red]Unexpected error: {e}[/red]")
|
||||
raise typer.Exit(1)
|
||||
|
||||
asyncio.run(_show())
|
||||
|
||||
|
||||
@snapshot_app.command("browse")
|
||||
def browse(
|
||||
snapshot_id: str = typer.Argument(
|
||||
...,
|
||||
help="The ID of the snapshot to browse",
|
||||
),
|
||||
prefix: Optional[str] = typer.Option(
|
||||
None,
|
||||
"--prefix",
|
||||
"-p",
|
||||
help="Filter files by path prefix (e.g., 'notes/')",
|
||||
),
|
||||
) -> None:
|
||||
"""Browse contents of a snapshot.
|
||||
|
||||
Examples:
|
||||
bm cloud snapshot browse abc123
|
||||
bm cloud snapshot browse abc123 --prefix notes/
|
||||
"""
|
||||
|
||||
async def _browse():
|
||||
try:
|
||||
config_manager = ConfigManager()
|
||||
config = config_manager.config
|
||||
host_url = config.cloud_host.rstrip("/")
|
||||
|
||||
url = f"{host_url}/api/bucket-snapshots/{snapshot_id}/browse"
|
||||
if prefix:
|
||||
url += f"?prefix={prefix}"
|
||||
|
||||
response = await make_api_request(
|
||||
method="GET",
|
||||
url=url,
|
||||
)
|
||||
|
||||
browse_response = BucketSnapshotBrowseResponse.model_validate(response.json())
|
||||
|
||||
if not browse_response.files:
|
||||
if prefix:
|
||||
console.print(f"[yellow]No files found with prefix '{prefix}'[/yellow]")
|
||||
else:
|
||||
console.print("[yellow]No files found in snapshot[/yellow]")
|
||||
return
|
||||
|
||||
console.print(
|
||||
f"[bold blue]Snapshot Contents ({len(browse_response.files)} files)[/bold blue]"
|
||||
)
|
||||
for file_info in browse_response.files:
|
||||
size_kb = file_info.size // 1024
|
||||
console.print(f" {file_info.key} ({size_kb} KB)")
|
||||
|
||||
console.print(
|
||||
f"\n[dim]Use 'bm cloud restore <path> --snapshot {snapshot_id}' "
|
||||
f"to restore files[/dim]"
|
||||
)
|
||||
|
||||
except SubscriptionRequiredError as e:
|
||||
console.print("\n[red]Subscription Required[/red]\n")
|
||||
console.print(f"[yellow]{e.args[0]}[/yellow]\n")
|
||||
console.print(f"Subscribe at: [blue underline]{e.subscribe_url}[/blue underline]\n")
|
||||
raise typer.Exit(1)
|
||||
except CloudAPIError as e:
|
||||
if e.status_code == 404:
|
||||
console.print(f"[red]Snapshot not found: {snapshot_id}[/red]")
|
||||
else:
|
||||
console.print(f"[red]Failed to browse snapshot: {e}[/red]")
|
||||
raise typer.Exit(1)
|
||||
except Exception as e:
|
||||
console.print(f"[red]Unexpected error: {e}[/red]")
|
||||
raise typer.Exit(1)
|
||||
|
||||
asyncio.run(_browse())
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user