mirror of
https://github.com/basicmachines-co/basic-memory
synced 2026-06-21 13:47:35 +00:00
Compare commits
75 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| b8191d090f | |||
| 2ce8a8e4b0 | |||
| f8099cd004 | |||
| 688e0b0971 | |||
| ed09ea4ec7 | |||
| c85d9f74d7 | |||
| 84d2aaf641 | |||
| 7789864493 | |||
| c6215fd819 | |||
| b4c26a6133 | |||
| 3fdce683d7 | |||
| 782cb2df28 | |||
| 56c875f137 | |||
| 5049de7e2d | |||
| 49011768f7 | |||
| bc3557f000 | |||
| 611f5cd305 | |||
| 4ea392d284 | |||
| d491757980 | |||
| 7a69ca2c36 | |||
| 70a6ce3411 | |||
| 5b69fd65cd | |||
| 85a178a6b8 | |||
| e4b32d7bc9 | |||
| 9590b934cf | |||
| 735f239f9b | |||
| 3ee30e1f36 | |||
| ac401ea254 | |||
| fb2fd62ed9 | |||
| 126d1655e6 | |||
| 2abf626c46 | |||
| ba8e3d112d | |||
| 7108a7baf1 | |||
| 35884ef3a7 | |||
| 040be05a81 | |||
| c141d7d1e6 | |||
| b73aeb5ed8 | |||
| 9a0e0bd82d | |||
| 117fa44ecf | |||
| 69d7610d47 | |||
| f608cd13f1 | |||
| 2162ad57fe | |||
| dd6ca80716 | |||
| ae3eeb0cc1 | |||
| 602c55fe90 | |||
| 91bfe2dc92 | |||
| a3cae1064d | |||
| c5c70cb0f4 | |||
| 80ec860a1c | |||
| f64d5b2152 | |||
| 69a625acd1 | |||
| 53c29a37ca | |||
| d8c13bf1d3 | |||
| 3f70f5ed42 | |||
| 569a3de80b | |||
| ac08a8d024 | |||
| c13d4b1511 | |||
| 5b85d33a99 | |||
| e2bd3142dc | |||
| 40b7de9168 | |||
| 455a280707 | |||
| 634486107b | |||
| 23d34289fd | |||
| 5ec4087f7c | |||
| 71941c35dd | |||
| 020957cd76 | |||
| b3af6aa685 | |||
| 9437a5f83b | |||
| 3c1cc346df | |||
| 81616ab42e | |||
| 321471f29d | |||
| 73ea91fe0d | |||
| 03d4e97b90 | |||
| 98622a7a47 | |||
| 54dfa08aba |
@@ -0,0 +1,190 @@
|
||||
# /project:check-health - Project Health Assessment
|
||||
|
||||
Comprehensive health check of the Basic Memory project including code quality, test coverage, dependencies, and documentation.
|
||||
|
||||
## Usage
|
||||
```
|
||||
/project:check-health
|
||||
```
|
||||
|
||||
## Implementation
|
||||
|
||||
You are an expert DevOps engineer for the Basic Memory project. When the user runs `/project:check-health`, execute the following comprehensive assessment:
|
||||
|
||||
### Step 1: Git Repository Health
|
||||
1. **Repository Status**
|
||||
```bash
|
||||
git status
|
||||
git log --oneline -5
|
||||
git branch -vv
|
||||
```
|
||||
- Check working directory status
|
||||
- Verify branch alignment with remote
|
||||
- Check recent commit activity
|
||||
|
||||
2. **Branch Analysis**
|
||||
- Verify on main branch
|
||||
- Check if ahead/behind remote
|
||||
- Identify any untracked files
|
||||
|
||||
### Step 2: Code Quality Assessment
|
||||
1. **Linting and Formatting**
|
||||
```bash
|
||||
uv run ruff check .
|
||||
uv run pyright
|
||||
```
|
||||
- Count linting issues by severity
|
||||
- Check type annotation coverage
|
||||
- Verify code formatting compliance
|
||||
|
||||
2. **Test Suite Health**
|
||||
```bash
|
||||
uv run pytest --collect-only -q
|
||||
uv run pytest --co -q | wc -l
|
||||
```
|
||||
- Count total tests
|
||||
- Check for test discovery issues
|
||||
- Verify test structure integrity
|
||||
|
||||
### Step 3: Dependency Analysis
|
||||
1. **Dependency Health**
|
||||
```bash
|
||||
uv tree
|
||||
uv lock --dry-run
|
||||
```
|
||||
- Check for dependency conflicts
|
||||
- Identify outdated dependencies
|
||||
- Verify lock file consistency
|
||||
|
||||
2. **Security Scan**
|
||||
```bash
|
||||
uv run pip-audit --desc
|
||||
```
|
||||
- Scan for known vulnerabilities
|
||||
- Check dependency licenses
|
||||
- Identify security advisories
|
||||
|
||||
### Step 4: Performance Metrics
|
||||
1. **Test Performance**
|
||||
```bash
|
||||
uv run pytest --durations=10
|
||||
```
|
||||
- Identify slowest tests
|
||||
- Check overall test execution time
|
||||
- Monitor test suite growth
|
||||
|
||||
2. **Build Performance**
|
||||
```bash
|
||||
time uv run python -c "import basic_memory"
|
||||
```
|
||||
- Check import time
|
||||
- Validate package installation
|
||||
- Monitor startup performance
|
||||
|
||||
### Step 5: Documentation Health
|
||||
1. **Documentation Coverage**
|
||||
- Check README.md currency
|
||||
- Verify CLI documentation
|
||||
- Validate MCP tool documentation
|
||||
- Check changelog completeness
|
||||
|
||||
2. **API Documentation**
|
||||
- Verify docstring coverage
|
||||
- Check type annotation completeness
|
||||
- Validate example code
|
||||
|
||||
### Step 6: Project Metrics
|
||||
1. **Code Statistics**
|
||||
```bash
|
||||
find src -name "*.py" | xargs wc -l
|
||||
find tests -name "*.py" | xargs wc -l
|
||||
```
|
||||
- Lines of code trends
|
||||
- Test-to-code ratio
|
||||
- File organization metrics
|
||||
|
||||
## Health Report Format
|
||||
|
||||
Generate comprehensive health dashboard:
|
||||
|
||||
```
|
||||
🏥 Basic Memory Project Health Report
|
||||
|
||||
📊 OVERALL HEALTH: 🟢 EXCELLENT (92/100)
|
||||
|
||||
🗂️ GIT REPOSITORY
|
||||
✅ Clean working directory
|
||||
✅ Up to date with origin/main
|
||||
✅ Recent commit activity (5 commits this week)
|
||||
|
||||
🔍 CODE QUALITY
|
||||
✅ Linting: 0 errors, 2 warnings
|
||||
✅ Type checking: 100% coverage
|
||||
✅ Formatting: Compliant
|
||||
⚠️ Complex functions: 3 need refactoring
|
||||
|
||||
🧪 TEST SUITE
|
||||
✅ Total tests: 744
|
||||
✅ Test discovery: All tests found
|
||||
✅ Coverage: 98.2%
|
||||
⚡ Performance: 45.2s (good)
|
||||
|
||||
📦 DEPENDENCIES
|
||||
✅ Dependencies: Up to date
|
||||
✅ Security: No vulnerabilities
|
||||
✅ Conflicts: None detected
|
||||
⚠️ Outdated: 2 minor updates available
|
||||
|
||||
📖 DOCUMENTATION
|
||||
✅ README: Current
|
||||
✅ API docs: 95% coverage
|
||||
⚠️ CLI reference: Needs update
|
||||
✅ Changelog: Complete
|
||||
|
||||
📈 METRICS
|
||||
├── Source code: 15,432 lines
|
||||
├── Test code: 8,967 lines
|
||||
├── Test ratio: 58% (excellent)
|
||||
└── Complexity: Low (maintainable)
|
||||
|
||||
🎯 RECOMMENDATIONS:
|
||||
1. Update CLI documentation
|
||||
2. Refactor 3 complex functions
|
||||
3. Update minor dependencies
|
||||
4. Consider splitting large test files
|
||||
|
||||
🏆 PROJECT STATUS: Ready for v0.13.0 release!
|
||||
```
|
||||
|
||||
## Health Scoring
|
||||
|
||||
### Excellent (90-100)
|
||||
- All quality gates pass
|
||||
- High test coverage (>95%)
|
||||
- No security issues
|
||||
- Documentation current
|
||||
|
||||
### Good (75-89)
|
||||
- Minor issues present
|
||||
- Good test coverage (>90%)
|
||||
- No critical security issues
|
||||
- Most documentation current
|
||||
|
||||
### Needs Attention (60-74)
|
||||
- Several quality issues
|
||||
- Adequate test coverage (>80%)
|
||||
- Minor security concerns
|
||||
- Documentation gaps
|
||||
|
||||
### Critical (<60)
|
||||
- Major quality problems
|
||||
- Low test coverage (<80%)
|
||||
- Security vulnerabilities
|
||||
- Significant documentation issues
|
||||
|
||||
## Context
|
||||
- Provides comprehensive project overview
|
||||
- Identifies potential issues before they become problems
|
||||
- Tracks project health trends over time
|
||||
- Helps prioritize maintenance tasks
|
||||
- Supports release readiness decisions
|
||||
@@ -0,0 +1,62 @@
|
||||
# Basic Memory Custom Commands
|
||||
|
||||
This directory contains custom Claude Code slash commands for the Basic Memory project.
|
||||
|
||||
## Available Commands
|
||||
|
||||
### Release Management (`/project:release:*`)
|
||||
- `/project:release:beta` - Create beta releases with automated quality checks
|
||||
- `/project:release:release` - Create stable releases with comprehensive validation
|
||||
- `/project:release:release-check` - Pre-flight validation without making changes
|
||||
- `/project:release:changelog` - Generate changelog entries from commits
|
||||
|
||||
### Development (`/project:*`)
|
||||
- `/project:test-coverage` - Run tests with detailed coverage analysis
|
||||
- `/project:test-live` - Live testing suite using real Basic Memory installation
|
||||
- `/project:lint-fix` - Run comprehensive linting with auto-fix
|
||||
- `/project:check-health` - Comprehensive project health assessment
|
||||
|
||||
## Command Structure
|
||||
|
||||
Commands are organized by functionality:
|
||||
```
|
||||
.claude/commands/
|
||||
├── release/ # Release management commands
|
||||
│ ├── beta.md # /project:release:beta
|
||||
│ ├── release.md # /project:release:release
|
||||
│ ├── release-check.md # /project:release:release-check
|
||||
│ └── changelog.md # /project:release:changelog
|
||||
├── test-coverage.md # /project:test-coverage
|
||||
├── test-live.md # /project:test-live
|
||||
├── lint-fix.md # /project:lint-fix
|
||||
├── check-health.md # /project:check-health
|
||||
└── commands.md # This overview file
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
Commands are invoked using the `/project:` prefix:
|
||||
- `/project:release:beta v0.13.0b4`
|
||||
- `/project:test-coverage mcp`
|
||||
- `/project:test-live core`
|
||||
- `/project:release:release-check`
|
||||
- `/project:check-health`
|
||||
|
||||
## Implementation
|
||||
|
||||
Each command is implemented as a Markdown file containing structured prompts that:
|
||||
1. Validate preconditions
|
||||
2. Execute steps in the correct order
|
||||
3. Handle errors gracefully
|
||||
4. Provide clear status updates
|
||||
5. Return actionable results
|
||||
|
||||
## Tooling Integration
|
||||
|
||||
Commands leverage existing project tooling:
|
||||
- `just check` - Quality checks
|
||||
- `just test` - Test suite
|
||||
- `just update-deps` - Dependency updates
|
||||
- `uv` - Package management
|
||||
- `git` - Version control
|
||||
- GitHub Actions - CI/CD pipeline
|
||||
@@ -0,0 +1,145 @@
|
||||
# /project:lint-fix - Comprehensive Code Quality Fix
|
||||
|
||||
Run comprehensive linting and auto-fix code quality issues across the codebase.
|
||||
|
||||
## Usage
|
||||
```
|
||||
/project:lint-fix
|
||||
```
|
||||
|
||||
## Implementation
|
||||
|
||||
You are an expert code quality engineer for the Basic Memory project. When the user runs `/project:lint-fix`, execute the following steps:
|
||||
|
||||
### Step 1: Pre-flight Check
|
||||
1. **Verify Clean Working Directory**
|
||||
```bash
|
||||
git status --porcelain
|
||||
```
|
||||
- Check for uncommitted changes
|
||||
- Warn if working directory is not clean
|
||||
- Suggest stashing changes if needed
|
||||
|
||||
### Step 2: Import Organization
|
||||
1. **Fix Import Order and Cleanup**
|
||||
```bash
|
||||
uv run ruff check --select I --fix .
|
||||
```
|
||||
- Sort imports by category (standard, third-party, local)
|
||||
- Remove unused imports
|
||||
- Fix import spacing and organization
|
||||
|
||||
### Step 3: Code Formatting
|
||||
1. **Apply Consistent Formatting**
|
||||
```bash
|
||||
uv run ruff format .
|
||||
```
|
||||
- Format code according to project style
|
||||
- Fix line length issues (100 chars max)
|
||||
- Standardize quotes and spacing
|
||||
|
||||
### Step 4: Linting with Auto-fix
|
||||
1. **Fix Linting Issues**
|
||||
```bash
|
||||
uv run ruff check --fix .
|
||||
```
|
||||
- Auto-fix safe linting issues
|
||||
- Report any remaining manual fixes needed
|
||||
- Focus on code quality and best practices
|
||||
|
||||
### Step 5: Type Checking
|
||||
1. **Validate Type Annotations**
|
||||
```bash
|
||||
uv run pyright
|
||||
```
|
||||
- Check for type errors
|
||||
- Report any missing type annotations
|
||||
- Validate type compatibility
|
||||
|
||||
### Step 6: Report Generation
|
||||
Generate comprehensive quality report:
|
||||
|
||||
```
|
||||
🔧 Code Quality Fix Report
|
||||
|
||||
✅ FIXES APPLIED:
|
||||
├── Import organization: 12 files updated
|
||||
├── Code formatting: 8 files reformatted
|
||||
├── Auto-fixable lint issues: 23 issues resolved
|
||||
└── Total files processed: 156
|
||||
|
||||
⚠️ MANUAL ATTENTION NEEDED:
|
||||
├── Type annotations missing in entity_service.py:45
|
||||
├── Complex function needs refactoring in sync_service.py:123
|
||||
└── Unused variable in test_utils.py:67
|
||||
|
||||
🎯 QUALITY SCORE: 96.2% (excellent)
|
||||
|
||||
📁 Run `git diff` to review all changes
|
||||
```
|
||||
|
||||
## Error Handling
|
||||
|
||||
### Common Issues
|
||||
- **Merge Conflicts**: Provide resolution guidance
|
||||
- **Syntax Errors**: Point to specific files and lines
|
||||
- **Type Errors**: Suggest specific fixes
|
||||
- **Import Errors**: Check for missing dependencies
|
||||
|
||||
### Recovery Steps
|
||||
- If auto-fixes introduce issues, provide rollback instructions
|
||||
- If type checking fails, suggest incremental fixes
|
||||
- If tests break, provide debugging guidance
|
||||
|
||||
## Quality Gates
|
||||
|
||||
### Must Pass
|
||||
- [ ] All auto-fixable lint issues resolved
|
||||
- [ ] Code formatting consistent
|
||||
- [ ] No syntax errors
|
||||
- [ ] Import organization clean
|
||||
|
||||
### Should Pass (Warnings)
|
||||
- [ ] No type checking errors
|
||||
- [ ] No complex function warnings
|
||||
- [ ] No unused variables/imports
|
||||
- [ ] Consistent naming conventions
|
||||
|
||||
## Output Examples
|
||||
|
||||
### Successful Fix
|
||||
```
|
||||
🎉 CODE QUALITY IMPROVED!
|
||||
|
||||
✅ All auto-fixes applied successfully
|
||||
📏 Code formatting: 100% compliant
|
||||
🔍 Linting: No issues found
|
||||
🏷️ Type checking: All passed
|
||||
|
||||
Ready for commit! Use:
|
||||
git add -A && git commit -m "style: fix code quality issues"
|
||||
```
|
||||
|
||||
### Issues Requiring Attention
|
||||
```
|
||||
⚠️ PARTIAL SUCCESS - MANUAL FIXES NEEDED
|
||||
|
||||
✅ Auto-fixes applied: 45 issues
|
||||
❌ Manual fixes needed: 3 issues
|
||||
|
||||
Priority fixes:
|
||||
1. Fix type annotation in services/entity_service.py:142
|
||||
2. Simplify complex function in sync/sync_service.py:67
|
||||
3. Remove unused import in tests/conftest.py:12
|
||||
|
||||
Run these commands:
|
||||
# Fix specific file
|
||||
uv run pyright src/basic_memory/services/entity_service.py
|
||||
```
|
||||
|
||||
## Context
|
||||
- Uses ruff for fast Python linting and formatting
|
||||
- Uses pyright for type checking
|
||||
- Follows project code style guidelines (100 char line length)
|
||||
- Maintains backward compatibility
|
||||
- Integrates with existing pre-commit hooks
|
||||
@@ -0,0 +1,95 @@
|
||||
# /beta - Create Beta Release
|
||||
|
||||
Create a new beta release using the automated justfile target with quality checks and tagging.
|
||||
|
||||
## Usage
|
||||
```
|
||||
/beta <version>
|
||||
```
|
||||
|
||||
**Parameters:**
|
||||
- `version` (required): Beta version like `v0.13.2b1` or `v0.13.2rc1`
|
||||
|
||||
## Implementation
|
||||
|
||||
You are an expert release manager for the Basic Memory project. When the user runs `/beta`, execute the following steps:
|
||||
|
||||
### Step 1: Pre-flight Validation
|
||||
1. Verify version format matches `v\d+\.\d+\.\d+(b\d+|rc\d+)` pattern
|
||||
2. Check current git status for uncommitted changes
|
||||
3. Verify we're on the `main` branch
|
||||
4. Confirm no existing tag with this version
|
||||
|
||||
### Step 2: Use Justfile Automation
|
||||
Execute the automated beta release process:
|
||||
```bash
|
||||
just beta <version>
|
||||
```
|
||||
|
||||
The justfile target handles:
|
||||
- ✅ Beta version format validation (supports b1, b2, rc1, etc.)
|
||||
- ✅ Git status and branch checks
|
||||
- ✅ Quality checks (`just check` - lint, format, type-check, tests)
|
||||
- ✅ Version update in `src/basic_memory/__init__.py`
|
||||
- ✅ Automatic commit with proper message
|
||||
- ✅ Tag creation and pushing to GitHub
|
||||
- ✅ Beta release workflow trigger
|
||||
|
||||
### Step 3: Monitor Beta Release
|
||||
1. Check GitHub Actions workflow starts successfully
|
||||
2. Monitor workflow at: https://github.com/basicmachines-co/basic-memory/actions
|
||||
3. Verify PyPI pre-release publication
|
||||
4. Test beta installation: `uv tool install basic-memory --pre`
|
||||
|
||||
### Step 4: Beta Testing Instructions
|
||||
Provide users with beta testing instructions:
|
||||
|
||||
```bash
|
||||
# Install/upgrade to beta
|
||||
uv tool install basic-memory --pre
|
||||
|
||||
# Or upgrade existing installation
|
||||
uv tool upgrade basic-memory --prerelease=allow
|
||||
```
|
||||
|
||||
## Version Guidelines
|
||||
- **First beta**: `v0.13.2b1`
|
||||
- **Subsequent betas**: `v0.13.2b2`, `v0.13.2b3`, etc.
|
||||
- **Release candidates**: `v0.13.2rc1`, `v0.13.2rc2`, etc.
|
||||
- **Final release**: `v0.13.2` (use `/release` command)
|
||||
|
||||
## Error Handling
|
||||
- If `just beta` fails, examine the error output for specific issues
|
||||
- If quality checks fail, fix issues and retry
|
||||
- If version format is invalid, correct and retry
|
||||
- If tag already exists, increment version number
|
||||
|
||||
## Success Output
|
||||
```
|
||||
✅ Beta Release v0.13.2b1 Created Successfully!
|
||||
|
||||
🏷️ Tag: v0.13.2b1
|
||||
🚀 GitHub Actions: Running
|
||||
📦 PyPI: Will be available in ~5 minutes as pre-release
|
||||
|
||||
Install/test with:
|
||||
uv tool install basic-memory --pre
|
||||
|
||||
Monitor release: https://github.com/basicmachines-co/basic-memory/actions
|
||||
```
|
||||
|
||||
## Beta Testing Workflow
|
||||
1. **Create beta**: Use `/beta v0.13.2b1`
|
||||
2. **Test features**: Install and validate new functionality
|
||||
3. **Fix issues**: Address bugs found during testing
|
||||
4. **Iterate**: Create `v0.13.2b2` if needed
|
||||
5. **Release candidate**: Create `v0.13.2rc1` when stable
|
||||
6. **Final release**: Use `/release v0.13.2` when ready
|
||||
|
||||
## Context
|
||||
- Beta releases are pre-releases for testing new features
|
||||
- Automatically published to PyPI with pre-release flag
|
||||
- Uses the automated justfile target for consistency
|
||||
- Version is automatically updated in `__init__.py`
|
||||
- Ideal for validating changes before stable release
|
||||
- Supports both beta (b1, b2) and release candidate (rc1, rc2) versions
|
||||
@@ -0,0 +1,157 @@
|
||||
# /changelog - Generate or Update Changelog Entry
|
||||
|
||||
Analyze commits and generate formatted changelog entry for a version.
|
||||
|
||||
## Usage
|
||||
```
|
||||
/changelog <version> [type]
|
||||
```
|
||||
|
||||
**Parameters:**
|
||||
- `version` (required): Version like `v0.13.0` or `v0.13.0b4`
|
||||
- `type` (optional): `beta`, `rc`, or `stable` (default: `stable`)
|
||||
|
||||
## Implementation
|
||||
|
||||
You are an expert technical writer for the Basic Memory project. When the user runs `/changelog`, execute the following steps:
|
||||
|
||||
### Step 1: Version Analysis
|
||||
1. **Determine Commit Range**
|
||||
```bash
|
||||
# Find last release tag
|
||||
git tag -l "v*" --sort=-version:refname | grep -v "b\|rc" | head -1
|
||||
|
||||
# Get commits since last release
|
||||
git log --oneline ${last_tag}..HEAD
|
||||
```
|
||||
|
||||
2. **Parse Conventional Commits**
|
||||
- Extract feat: (features)
|
||||
- Extract fix: (bug fixes)
|
||||
- Extract BREAKING CHANGE: (breaking changes)
|
||||
- Extract chore:, docs:, test: (other improvements)
|
||||
|
||||
### Step 2: Categorize Changes
|
||||
1. **Features (feat:)**
|
||||
- New MCP tools
|
||||
- New CLI commands
|
||||
- New API endpoints
|
||||
- Major functionality additions
|
||||
|
||||
2. **Bug Fixes (fix:)**
|
||||
- User-facing bug fixes
|
||||
- Critical issues resolved
|
||||
- Performance improvements
|
||||
- Security fixes
|
||||
|
||||
3. **Technical Improvements**
|
||||
- Test coverage improvements
|
||||
- Code quality enhancements
|
||||
- Dependency updates
|
||||
- Documentation updates
|
||||
|
||||
4. **Breaking Changes**
|
||||
- API changes
|
||||
- Configuration changes
|
||||
- Behavior changes
|
||||
- Migration requirements
|
||||
|
||||
### Step 3: Generate Changelog Entry
|
||||
Create formatted entry following existing CHANGELOG.md style:
|
||||
|
||||
```markdown
|
||||
## v0.13.0 (2025-06-03)
|
||||
|
||||
### Features
|
||||
|
||||
- **Multi-Project Management System** - Switch between projects instantly during conversations
|
||||
([`993e88a`](https://github.com/basicmachines-co/basic-memory/commit/993e88a))
|
||||
- Instant project switching with session context
|
||||
- Project-specific operations and isolation
|
||||
- Project discovery and management tools
|
||||
|
||||
- **Advanced Note Editing** - Incremental editing with append, prepend, find/replace, and section operations
|
||||
([`6fc3904`](https://github.com/basicmachines-co/basic-memory/commit/6fc3904))
|
||||
- `edit_note` tool with multiple operation types
|
||||
- Smart frontmatter-aware editing
|
||||
- Validation and error handling
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
- **#118**: Fix YAML tag formatting to follow standard specification
|
||||
([`2dc7e27`](https://github.com/basicmachines-co/basic-memory/commit/2dc7e27))
|
||||
|
||||
- **#110**: Make --project flag work consistently across CLI commands
|
||||
([`02dd91a`](https://github.com/basicmachines-co/basic-memory/commit/02dd91a))
|
||||
|
||||
### Technical Improvements
|
||||
|
||||
- **Comprehensive Testing** - 100% test coverage with integration testing
|
||||
([`468a22f`](https://github.com/basicmachines-co/basic-memory/commit/468a22f))
|
||||
- MCP integration test suite
|
||||
- End-to-end testing framework
|
||||
- Performance and edge case validation
|
||||
|
||||
### Breaking Changes
|
||||
|
||||
- **Database Migration**: Automatic migration from per-project to unified database.
|
||||
Data will be re-index from the filesystem, resulting in no data loss.
|
||||
- **Configuration Changes**: Projects now synced between config.json and database
|
||||
- **Full Backward Compatibility**: All existing setups continue to work seamlessly
|
||||
```
|
||||
|
||||
### Step 4: Integration
|
||||
1. **Update CHANGELOG.md**
|
||||
- Insert new entry at top
|
||||
- Maintain consistent formatting
|
||||
- Include commit links and issue references
|
||||
|
||||
2. **Validation**
|
||||
- Check all major changes are captured
|
||||
- Verify commit links work
|
||||
- Ensure issue numbers are correct
|
||||
|
||||
## Smart Analysis Features
|
||||
|
||||
### Automatic Classification
|
||||
- Detect feature additions from file changes
|
||||
- Identify bug fixes from commit messages
|
||||
- Find breaking changes from code analysis
|
||||
- Extract issue numbers from commit messages
|
||||
|
||||
### Content Enhancement
|
||||
- Add context for technical changes
|
||||
- Include migration guidance for breaking changes
|
||||
- Suggest installation/upgrade instructions
|
||||
- Link to relevant documentation
|
||||
|
||||
## Output Format
|
||||
|
||||
### For Beta Releases
|
||||
```markdown
|
||||
## v0.13.0b4 (2025-06-03)
|
||||
|
||||
### Beta Changes Since v0.13.0b3
|
||||
|
||||
- Fix FastMCP API compatibility issues
|
||||
- Update dependencies to latest versions
|
||||
- Resolve setuptools import error
|
||||
|
||||
### Installation
|
||||
```bash
|
||||
uv tool install basic-memory --prerelease=allow
|
||||
```
|
||||
|
||||
### Known Issues
|
||||
- [List any known issues for beta testing]
|
||||
```
|
||||
|
||||
### For Stable Releases
|
||||
Full changelog with complete feature list, organized by impact and category.
|
||||
|
||||
## Context
|
||||
- Follows existing CHANGELOG.md format and style
|
||||
- Uses conventional commit standards
|
||||
- Includes GitHub commit links for traceability
|
||||
- Focuses on user-facing changes and value
|
||||
- Maintains consistency with previous entries
|
||||
@@ -0,0 +1,131 @@
|
||||
# /release-check - Pre-flight Release Validation
|
||||
|
||||
Comprehensive pre-flight check for release readiness without making any changes.
|
||||
|
||||
## Usage
|
||||
```
|
||||
/release-check [version]
|
||||
```
|
||||
|
||||
**Parameters:**
|
||||
- `version` (optional): Version to validate like `v0.13.0`. If not provided, determines from context.
|
||||
|
||||
## Implementation
|
||||
|
||||
You are an expert QA engineer for the Basic Memory project. When the user runs `/release-check`, execute the following validation steps:
|
||||
|
||||
### Step 1: Environment Validation
|
||||
1. **Git Status Check**
|
||||
- Verify working directory is clean
|
||||
- Confirm on `main` branch
|
||||
- Check if ahead/behind origin
|
||||
|
||||
2. **Version Validation**
|
||||
- Validate version format if provided
|
||||
- Check for existing tags with same version
|
||||
- Verify version increments properly from last release
|
||||
|
||||
### Step 2: Code Quality Gates
|
||||
1. **Test Suite Validation**
|
||||
```bash
|
||||
just test
|
||||
```
|
||||
- All tests must pass
|
||||
- Check test coverage (target: 95%+)
|
||||
- Validate no skipped critical tests
|
||||
|
||||
2. **Code Quality Checks**
|
||||
```bash
|
||||
just lint
|
||||
just type-check
|
||||
```
|
||||
- No linting errors
|
||||
- No type checking errors
|
||||
- Code formatting is consistent
|
||||
|
||||
### Step 3: Documentation Validation
|
||||
1. **Changelog Check**
|
||||
- CHANGELOG.md contains entry for target version
|
||||
- Entry includes all major features and fixes
|
||||
- Breaking changes are documented
|
||||
|
||||
2. **Documentation Currency**
|
||||
- README.md reflects current functionality
|
||||
- CLI reference is up to date
|
||||
- MCP tools are documented
|
||||
|
||||
### Step 4: Dependency Validation
|
||||
1. **Security Scan**
|
||||
- No known vulnerabilities in dependencies
|
||||
- All dependencies are at appropriate versions
|
||||
- No conflicting dependency versions
|
||||
|
||||
2. **Build Validation**
|
||||
- Package builds successfully
|
||||
- All required files are included
|
||||
- No missing dependencies
|
||||
|
||||
### Step 5: Issue Tracking Validation
|
||||
1. **GitHub Issues Check**
|
||||
- No critical open issues blocking release
|
||||
- All milestone issues are resolved
|
||||
- High-priority bugs are fixed
|
||||
|
||||
2. **Testing Coverage**
|
||||
- Integration tests pass
|
||||
- MCP tool tests pass
|
||||
- Cross-platform compatibility verified
|
||||
|
||||
## Report Format
|
||||
|
||||
Generate a comprehensive report:
|
||||
|
||||
```
|
||||
🔍 Release Readiness Check for v0.13.0
|
||||
|
||||
✅ PASSED CHECKS:
|
||||
├── Git status clean
|
||||
├── On main branch
|
||||
├── All tests passing (744/744)
|
||||
├── Test coverage: 98.2%
|
||||
├── Type checking passed
|
||||
├── Linting passed
|
||||
├── CHANGELOG.md updated
|
||||
└── No critical issues open
|
||||
|
||||
⚠️ WARNINGS:
|
||||
├── 2 medium-priority issues still open
|
||||
└── Documentation could be updated
|
||||
|
||||
❌ BLOCKING ISSUES:
|
||||
└── None found
|
||||
|
||||
🎯 RELEASE READINESS: ✅ READY
|
||||
|
||||
Recommended next steps:
|
||||
1. Address warnings if desired
|
||||
2. Run `/release v0.13.0` when ready
|
||||
```
|
||||
|
||||
## Validation Criteria
|
||||
|
||||
### Must Pass (Blocking)
|
||||
- [ ] All tests pass
|
||||
- [ ] No type errors
|
||||
- [ ] No linting errors
|
||||
- [ ] Working directory clean
|
||||
- [ ] On main branch
|
||||
- [ ] CHANGELOG.md has version entry
|
||||
- [ ] No critical open issues
|
||||
|
||||
### Should Pass (Warnings)
|
||||
- [ ] Test coverage >95%
|
||||
- [ ] No medium-priority open issues
|
||||
- [ ] Documentation up to date
|
||||
- [ ] No dependency vulnerabilities
|
||||
|
||||
## Context
|
||||
- This is a read-only validation - makes no changes
|
||||
- Provides confidence before running actual release
|
||||
- Helps identify issues early in release process
|
||||
- Can be run multiple times safely
|
||||
@@ -0,0 +1,92 @@
|
||||
# /release - Create Stable Release
|
||||
|
||||
Create a stable release using the automated justfile target with comprehensive validation.
|
||||
|
||||
## Usage
|
||||
```
|
||||
/release <version>
|
||||
```
|
||||
|
||||
**Parameters:**
|
||||
- `version` (required): Release version like `v0.13.2`
|
||||
|
||||
## Implementation
|
||||
|
||||
You are an expert release manager for the Basic Memory project. When the user runs `/release`, execute the following steps:
|
||||
|
||||
### Step 1: Pre-flight Validation
|
||||
1. Verify version format matches `v\d+\.\d+\.\d+` pattern
|
||||
2. Check current git status for uncommitted changes
|
||||
3. Verify we're on the `main` branch
|
||||
4. Confirm no existing tag with this version
|
||||
|
||||
#### Documentation Validation
|
||||
1. **Changelog Check**
|
||||
- CHANGELOG.md contains entry for target version
|
||||
- Entry includes all major features and fixes
|
||||
- Breaking changes are documented
|
||||
|
||||
### Step 2: Use Justfile Automation
|
||||
Execute the automated release process:
|
||||
```bash
|
||||
just release <version>
|
||||
```
|
||||
|
||||
The justfile target handles:
|
||||
- ✅ Version format validation
|
||||
- ✅ Git status and branch checks
|
||||
- ✅ Quality checks (`just check` - lint, format, type-check, tests)
|
||||
- ✅ Version update in `src/basic_memory/__init__.py`
|
||||
- ✅ Automatic commit with proper message
|
||||
- ✅ Tag creation and pushing to GitHub
|
||||
- ✅ Release workflow trigger
|
||||
|
||||
### Step 3: Monitor Release Process
|
||||
1. Check that GitHub Actions workflow starts successfully
|
||||
2. Monitor workflow completion at: https://github.com/basicmachines-co/basic-memory/actions
|
||||
3. Verify PyPI publication
|
||||
4. Test installation: `uv tool install basic-memory`
|
||||
|
||||
### Step 4: Post-Release Validation
|
||||
1. Verify GitHub release is created automatically
|
||||
2. Check PyPI publication
|
||||
3. Validate release assets
|
||||
4. Update any post-release documentation
|
||||
|
||||
## Pre-conditions Check
|
||||
Before starting, verify:
|
||||
- [ ] All beta testing is complete
|
||||
- [ ] Critical bugs are fixed
|
||||
- [ ] Breaking changes are documented
|
||||
- [ ] CHANGELOG.md is updated (if needed)
|
||||
- [ ] Version number follows semantic versioning
|
||||
|
||||
## Error Handling
|
||||
- If `just release` fails, examine the error output for specific issues
|
||||
- If quality checks fail, fix issues and retry
|
||||
- If changelog entry missing, update CHANGELOG.md and commit before retrying
|
||||
- If GitHub Actions fail, check workflow logs for debugging
|
||||
|
||||
## Success Output
|
||||
```
|
||||
🎉 Stable Release v0.13.2 Created Successfully!
|
||||
|
||||
🏷️ Tag: v0.13.2
|
||||
📋 GitHub Release: https://github.com/basicmachines-co/basic-memory/releases/tag/v0.13.2
|
||||
📦 PyPI: https://pypi.org/project/basic-memory/0.13.2/
|
||||
🚀 GitHub Actions: Completed
|
||||
|
||||
Install with:
|
||||
uv tool install basic-memory
|
||||
|
||||
Users can now upgrade:
|
||||
uv tool upgrade basic-memory
|
||||
```
|
||||
|
||||
## Context
|
||||
- This creates production releases used by end users
|
||||
- Must pass all quality gates before proceeding
|
||||
- Uses the automated justfile target for consistency
|
||||
- Version is automatically updated in `__init__.py`
|
||||
- Triggers automated GitHub release with changelog
|
||||
- Leverages uv-dynamic-versioning for package version management
|
||||
@@ -0,0 +1,131 @@
|
||||
# /test-coverage - Run Tests with Coverage Analysis
|
||||
|
||||
Execute test suite with comprehensive coverage reporting and analysis.
|
||||
|
||||
## Usage
|
||||
```
|
||||
/test-coverage [pattern]
|
||||
```
|
||||
|
||||
**Parameters:**
|
||||
- `pattern` (optional): Test pattern to run specific tests (e.g., `test_mcp`, `*integration*`)
|
||||
|
||||
## Implementation
|
||||
|
||||
You are an expert QA engineer for the Basic Memory project. When the user runs `/test-coverage`, execute the following steps:
|
||||
|
||||
### Step 1: Test Execution
|
||||
1. **Run Tests with Coverage**
|
||||
```bash
|
||||
# Full test suite
|
||||
uv run pytest --cov=basic_memory --cov-report=html --cov-report=term -v
|
||||
|
||||
# Or with pattern if provided
|
||||
uv run pytest tests/*{pattern}* --cov=basic_memory --cov-report=html --cov-report=term -v
|
||||
```
|
||||
|
||||
2. **Generate Coverage Reports**
|
||||
- Terminal summary with percentages
|
||||
- HTML report for detailed analysis
|
||||
- Identify files below coverage threshold
|
||||
|
||||
### Step 2: Coverage Analysis
|
||||
1. **Summary Statistics**
|
||||
- Overall coverage percentage
|
||||
- Number of files with 100% coverage
|
||||
- Files below 95% threshold
|
||||
- Total lines covered/missed
|
||||
|
||||
2. **Detailed Breakdown**
|
||||
- Coverage by module/package
|
||||
- Identify untested code paths
|
||||
- Find missing edge case tests
|
||||
|
||||
### Step 3: Report Generation
|
||||
Generate comprehensive coverage report:
|
||||
|
||||
```
|
||||
🧪 Test Coverage Report
|
||||
|
||||
📊 OVERALL COVERAGE: 98.2% (target: 95%+)
|
||||
|
||||
✅ EXCELLENT COVERAGE (>95%):
|
||||
├── basic_memory/mcp/: 99.1%
|
||||
├── basic_memory/services/: 98.8%
|
||||
├── basic_memory/repository/: 97.9%
|
||||
└── basic_memory/api/: 96.2%
|
||||
|
||||
⚠️ NEEDS ATTENTION (<95%):
|
||||
├── basic_memory/sync/: 94.1% (missing 12 lines)
|
||||
└── basic_memory/importers/: 91.8% (missing 23 lines)
|
||||
|
||||
🎯 SPECIFIC GAPS:
|
||||
├── sync_service.py:142-145 (error handling)
|
||||
├── importer_base.py:67-70 (edge case)
|
||||
└── file_utils.py:89 (exception path)
|
||||
|
||||
📁 HTML Report: htmlcov/index.html
|
||||
🚀 Run `open htmlcov/index.html` to view detailed report
|
||||
```
|
||||
|
||||
### Step 4: Actionable Recommendations
|
||||
1. **Coverage Improvements**
|
||||
- Suggest specific tests to add
|
||||
- Identify edge cases to cover
|
||||
- Recommend integration tests
|
||||
|
||||
2. **Quality Insights**
|
||||
- Highlight well-tested modules
|
||||
- Point out testing patterns to follow
|
||||
- Suggest refactoring for testability
|
||||
|
||||
## Advanced Analysis
|
||||
|
||||
### Performance Metrics
|
||||
- Test execution time by module
|
||||
- Slowest tests identification
|
||||
- Coverage collection overhead
|
||||
|
||||
### Integration Coverage
|
||||
- MCP tool integration tests
|
||||
- API endpoint coverage
|
||||
- Database operation coverage
|
||||
- File system operation coverage
|
||||
|
||||
## Output Examples
|
||||
|
||||
### Full Coverage Success
|
||||
```
|
||||
🎉 EXCELLENT COVERAGE!
|
||||
|
||||
📊 Coverage: 98.7% (744 tests passed)
|
||||
✅ All modules above 95% threshold
|
||||
🏆 23 files with 100% coverage
|
||||
⚡ Tests completed in 45.2s
|
||||
|
||||
Ready for release! 🚀
|
||||
```
|
||||
|
||||
### Coverage Issues Found
|
||||
```
|
||||
⚠️ COVERAGE GAPS DETECTED
|
||||
|
||||
📊 Coverage: 92.1% (below 95% target)
|
||||
❌ 3 modules need attention
|
||||
🔍 43 uncovered lines found
|
||||
|
||||
Priority fixes:
|
||||
1. Add tests for error handling in sync_service.py
|
||||
2. Cover edge cases in importer_base.py
|
||||
3. Test exception paths in file_utils.py
|
||||
|
||||
Run specific tests:
|
||||
uv run pytest tests/sync/ -v
|
||||
```
|
||||
|
||||
## Context
|
||||
- Uses pytest with coverage plugin
|
||||
- Generates both terminal and HTML reports
|
||||
- Focuses on actionable improvement suggestions
|
||||
- Integrates with existing test infrastructure
|
||||
- Helps maintain high code quality standards
|
||||
@@ -0,0 +1,410 @@
|
||||
# /project:test-live - Live Basic Memory Testing Suite
|
||||
|
||||
Execute comprehensive real-world testing of Basic Memory using the installed version, following the methodology in TESTING.md. All test results are recorded as notes in a dedicated test project.
|
||||
|
||||
## Usage
|
||||
```
|
||||
/project:test-live [phase]
|
||||
```
|
||||
|
||||
**Parameters:**
|
||||
- `phase` (optional): Specific test phase to run (`core`, `features`, `edge`, `workflows`, `stress`, or `all`)
|
||||
|
||||
## Implementation
|
||||
|
||||
You are an expert QA engineer conducting live testing of Basic Memory.
|
||||
When the user runs `/project:test-live`, execute comprehensive testing following the TESTING.md methodology:
|
||||
|
||||
### Pre-Test Setup
|
||||
|
||||
1. **Environment Verification**
|
||||
- Verify basic-memory is installed and accessible via MCP
|
||||
- Check version and confirm it's the expected release
|
||||
- Test MCP connection and tool availability
|
||||
|
||||
2. **Test Project Creation**
|
||||
|
||||
Run the bash `date` command to get the current date/time.
|
||||
|
||||
```
|
||||
Create project: "basic-memory-testing-[timestamp]"
|
||||
Location: ~/basic-memory-testing-[timestamp]
|
||||
Purpose: Record all test observations and results
|
||||
```
|
||||
|
||||
Make sure to switch to the newly created project with the `switch_project()` tool.
|
||||
|
||||
3. **Baseline Documentation**
|
||||
Create initial test session note with:
|
||||
- Test environment details
|
||||
- Version being tested
|
||||
- Test objectives and scope
|
||||
- Start timestamp
|
||||
|
||||
### Phase 1: Core Functionality Validation
|
||||
|
||||
Test all fundamental MCP tools systematically:
|
||||
|
||||
**write_note Tests:**
|
||||
- Basic note creation with various content types
|
||||
- Frontmatter handling (tags, custom fields)
|
||||
- Special characters in titles and content
|
||||
- Unicode and emoji support
|
||||
- Empty notes and minimal content
|
||||
|
||||
**read_note Tests:**
|
||||
- Read by title, permalink, memory:// URLs
|
||||
- Non-existent notes (error handling)
|
||||
- Notes with complex formatting
|
||||
- Performance with large notes
|
||||
|
||||
**view_note Tests:**
|
||||
- View notes as formatted artifacts (Claude Desktop)
|
||||
- Title extraction from frontmatter and headings
|
||||
- Unicode and emoji content in artifacts
|
||||
- Error handling for non-existent notes
|
||||
- Artifact display quality and readability
|
||||
|
||||
**search_notes Tests:**
|
||||
- Simple text queries
|
||||
- Tag-based searches
|
||||
- Boolean operators and complex queries
|
||||
- Empty/no results scenarios
|
||||
- Performance with growing knowledge base
|
||||
|
||||
**Recent Activity Tests:**
|
||||
- Various timeframes ("today", "1 week", "1d")
|
||||
- Type filtering (if available)
|
||||
- Empty project scenarios
|
||||
- Performance with many recent changes
|
||||
|
||||
**Context Building Tests:**
|
||||
- Different depth levels (1, 2, 3+)
|
||||
- Various timeframes
|
||||
- Relation traversal accuracy
|
||||
- Performance with complex graphs
|
||||
|
||||
### Phase 2: v0.13.0 Feature Deep Dive
|
||||
|
||||
**Project Management:**
|
||||
- Create multiple projects dynamically
|
||||
- Switch between projects mid-conversation
|
||||
- Cross-project operations
|
||||
- Project discovery and status
|
||||
- Default project behavior
|
||||
- Invalid project handling
|
||||
|
||||
**Advanced Note Editing:**
|
||||
- `edit_note` with append operations
|
||||
- Prepend operations
|
||||
- Find/replace with validation
|
||||
- Section replacement under headers
|
||||
- Error scenarios (invalid operations)
|
||||
- Frontmatter preservation
|
||||
|
||||
**File Management:**
|
||||
- `move_note` within same project
|
||||
- Move between projects
|
||||
- Automatic folder creation
|
||||
- Special characters in paths
|
||||
- Database consistency validation
|
||||
- Search index updates after moves
|
||||
|
||||
### Phase 3: Edge Case Exploration
|
||||
|
||||
**Boundary Testing:**
|
||||
- Very long titles and content (stress limits)
|
||||
- Empty projects and notes
|
||||
- Unicode, emojis, special symbols
|
||||
- Deeply nested folder structures
|
||||
- Circular relations and self-references
|
||||
- Maximum relation depths
|
||||
|
||||
**Error Scenarios:**
|
||||
- Invalid memory:// URLs
|
||||
- Missing files referenced in database
|
||||
- Invalid project names and paths
|
||||
- Malformed note structures
|
||||
- Concurrent operation conflicts
|
||||
|
||||
**Performance Testing:**
|
||||
- Create 100+ notes rapidly
|
||||
- Complex search queries
|
||||
- Deep relation chains (5+ levels)
|
||||
- Rapid successive operations
|
||||
- Memory usage monitoring
|
||||
|
||||
### Phase 4: Real-World Workflow Scenarios
|
||||
|
||||
**Meeting Notes Pipeline:**
|
||||
1. Create meeting notes with action items
|
||||
2. Extract action items using edit_note
|
||||
3. Build relations to project documents
|
||||
4. Update progress incrementally
|
||||
5. Search and track completion
|
||||
|
||||
**Research Knowledge Building:**
|
||||
1. Create research topic hierarchy
|
||||
2. Build complex relation networks
|
||||
3. Add incremental findings over time
|
||||
4. Search for connections and patterns
|
||||
5. Reorganize as knowledge evolves
|
||||
|
||||
**Multi-Project Workflow:**
|
||||
1. Technical documentation project
|
||||
2. Personal recipe collection project
|
||||
3. Learning/course notes project
|
||||
4. Switch contexts during conversation
|
||||
5. Cross-reference related concepts
|
||||
|
||||
**Content Evolution:**
|
||||
1. Start with basic notes
|
||||
2. Enhance with relations and observations
|
||||
3. Reorganize file structure using moves
|
||||
4. Update content with edit operations
|
||||
5. Validate knowledge graph integrity
|
||||
|
||||
### Phase 5: Creative Stress Testing
|
||||
|
||||
**Creative Exploration:**
|
||||
- Rapid project creation/switching patterns
|
||||
- Unusual but valid markdown structures
|
||||
- Creative observation categories
|
||||
- Novel relation types and patterns
|
||||
- Unexpected tool combinations
|
||||
|
||||
**Stress Scenarios:**
|
||||
- Bulk operations (many notes quickly)
|
||||
- Complex nested moves and edits
|
||||
- Deep context building
|
||||
- Complex boolean search expressions
|
||||
- Resource constraint testing
|
||||
|
||||
## Test Observation Format
|
||||
|
||||
Record ALL observations immediately as Basic Memory notes:
|
||||
|
||||
```markdown
|
||||
---
|
||||
title: Test Session [Phase] YYYY-MM-DD HH:MM
|
||||
tags: [testing, v0.13.0, live-testing, [phase]]
|
||||
permalink: test-session-[phase]-[timestamp]
|
||||
---
|
||||
|
||||
# Test Session [Phase] - [Date/Time]
|
||||
|
||||
## Environment
|
||||
- Basic Memory version: [version]
|
||||
- MCP connection: [status]
|
||||
- Test project: [name]
|
||||
- Phase focus: [description]
|
||||
|
||||
## Test Results
|
||||
|
||||
### ✅ Successful Operations
|
||||
- [timestamp] write_note: Created note with emoji title 📝 #functionality
|
||||
- [timestamp] search_notes: Boolean query returned 23 results in 0.4s #performance
|
||||
- [timestamp] edit_note: Append operation preserved frontmatter #reliability
|
||||
|
||||
### ⚠️ Issues Discovered
|
||||
- [timestamp] move_note: Slow with deep folder paths (2.1s) #performance
|
||||
- [timestamp] search_notes: Unicode query returned unexpected results #bug
|
||||
- [timestamp] project switch: Context lost for memory:// URLs #issue
|
||||
|
||||
### 🚀 Enhancements Identified
|
||||
- edit_note could benefit from preview mode #ux-improvement
|
||||
- search_notes needs fuzzy matching for typos #feature-idea
|
||||
- move_note could auto-suggest folder creation #usability
|
||||
|
||||
### 📊 Performance Metrics
|
||||
- Average write_note time: 0.3s
|
||||
- Search with 100+ notes: 0.6s
|
||||
- Project switch overhead: 0.1s
|
||||
- Memory usage: [observed levels]
|
||||
|
||||
## Relations
|
||||
- tests [[Basic Memory v0.13.0]]
|
||||
- part_of [[Live Testing Suite]]
|
||||
- found_issues [[Bug Report: Unicode Search]]
|
||||
- discovered [[Performance Optimization Opportunities]]
|
||||
```
|
||||
|
||||
## Quality Assessment Areas
|
||||
|
||||
**User Experience & Usability:**
|
||||
- Tool instruction clarity and examples
|
||||
- Error message actionability
|
||||
- Response time acceptability
|
||||
- Tool consistency and discoverability
|
||||
- Learning curve and intuitiveness
|
||||
|
||||
**System Behavior:**
|
||||
- Context preservation across operations
|
||||
- memory:// URL navigation reliability
|
||||
- Multi-step workflow cohesion
|
||||
- Edge case graceful handling
|
||||
- Recovery from user errors
|
||||
|
||||
**Documentation Alignment:**
|
||||
- Tool output clarity and helpfulness
|
||||
- Behavior vs. documentation accuracy
|
||||
- Example validity and usefulness
|
||||
- Real-world vs. documented workflows
|
||||
|
||||
**Mental Model Validation:**
|
||||
- Natural user expectation alignment
|
||||
- Surprising behavior identification
|
||||
- Mistake recovery ease
|
||||
- Knowledge graph concept naturalness
|
||||
|
||||
**Performance & Reliability:**
|
||||
- Operation completion times
|
||||
- Consistency across sessions
|
||||
- Scaling behavior with growth
|
||||
- Unexpected slowness identification
|
||||
|
||||
## Error Documentation Protocol
|
||||
|
||||
For each error discovered:
|
||||
|
||||
1. **Immediate Recording**
|
||||
- Create dedicated error note
|
||||
- Include exact reproduction steps
|
||||
- Capture error messages verbatim
|
||||
- Note system state when error occurred
|
||||
|
||||
2. **Error Note Format**
|
||||
```markdown
|
||||
---
|
||||
title: Bug Report - [Short Description]
|
||||
tags: [bug, testing, v0.13.0, [severity]]
|
||||
---
|
||||
|
||||
# Bug Report: [Description]
|
||||
|
||||
## Reproduction Steps
|
||||
1. [Exact steps to reproduce]
|
||||
2. [Include all parameters used]
|
||||
3. [Note any special conditions]
|
||||
|
||||
## Expected Behavior
|
||||
[What should have happened]
|
||||
|
||||
## Actual Behavior
|
||||
[What actually happened]
|
||||
|
||||
## Error Messages
|
||||
```
|
||||
[Exact error text]
|
||||
```
|
||||
|
||||
## Environment
|
||||
- Version: [version]
|
||||
- Project: [name]
|
||||
- Timestamp: [when]
|
||||
|
||||
## Severity
|
||||
- [ ] Critical (blocks major functionality)
|
||||
- [ ] High (impacts user experience)
|
||||
- [ ] Medium (workaround available)
|
||||
- [ ] Low (minor inconvenience)
|
||||
|
||||
## Relations
|
||||
- discovered_during [[Test Session [Phase]]]
|
||||
- affects [[Feature Name]]
|
||||
```
|
||||
|
||||
## Success Metrics Tracking
|
||||
|
||||
**Quantitative Measures:**
|
||||
- Test scenario completion rate
|
||||
- Bug discovery count with severity
|
||||
- Performance benchmark establishment
|
||||
- Tool coverage completeness
|
||||
|
||||
**Qualitative Measures:**
|
||||
- Conversation flow naturalness
|
||||
- Knowledge graph quality
|
||||
- User experience insights
|
||||
- System reliability assessment
|
||||
|
||||
## Test Execution Flow
|
||||
|
||||
1. **Setup Phase** (5 minutes)
|
||||
- Verify environment and create test project
|
||||
- Record baseline system state
|
||||
- Establish performance benchmarks
|
||||
|
||||
2. **Core Testing** (15-20 minutes per phase)
|
||||
- Execute test scenarios systematically
|
||||
- Record observations immediately
|
||||
- Note timestamps for performance tracking
|
||||
- Explore variations when interesting behaviors occur
|
||||
|
||||
3. **Documentation** (5 minutes per phase)
|
||||
- Create phase summary note
|
||||
- Link related test observations
|
||||
- Update running issues list
|
||||
- Record enhancement ideas
|
||||
|
||||
4. **Analysis Phase** (10 minutes)
|
||||
- Review all observations across phases
|
||||
- Identify patterns and trends
|
||||
- Create comprehensive summary report
|
||||
- Generate development recommendations
|
||||
|
||||
## Expected Outcomes
|
||||
|
||||
**System Validation:**
|
||||
- v0.13.0 feature verification in real usage
|
||||
- Edge case discovery beyond unit tests
|
||||
- Performance baseline establishment
|
||||
- Bug identification with reproduction cases
|
||||
|
||||
**Knowledge Base Creation:**
|
||||
- Comprehensive testing documentation
|
||||
- Real usage examples for user guides
|
||||
- Edge case scenarios for future testing
|
||||
- Performance insights for optimization
|
||||
|
||||
**Development Insights:**
|
||||
- Prioritized bug fix list
|
||||
- Enhancement ideas from real usage
|
||||
- Architecture validation results
|
||||
- User experience improvement areas
|
||||
|
||||
## Post-Test Deliverables
|
||||
|
||||
1. **Test Summary Note**
|
||||
- Overall results and findings
|
||||
- Critical issues requiring immediate attention
|
||||
- Enhancement opportunities discovered
|
||||
- System readiness assessment
|
||||
|
||||
2. **Bug Report Collection**
|
||||
- All discovered issues with reproduction steps
|
||||
- Severity and impact assessments
|
||||
- Suggested fixes where applicable
|
||||
|
||||
3. **Performance Baseline**
|
||||
- Timing data for all operations
|
||||
- Scaling behavior observations
|
||||
- Resource usage patterns
|
||||
|
||||
4. **UX Improvement Recommendations**
|
||||
- Usability enhancement suggestions
|
||||
- Documentation improvement areas
|
||||
- Tool design optimization ideas
|
||||
|
||||
5. **Updated TESTING.md**
|
||||
- Incorporate new test scenarios discovered
|
||||
- Update based on real execution experience
|
||||
- Add performance benchmarks and targets
|
||||
|
||||
## Context
|
||||
- Uses installed basic-memory version (not development)
|
||||
- Tests complete MCP→API→DB→File stack
|
||||
- Creates living documentation in Basic Memory itself
|
||||
- Follows integration over isolation philosophy
|
||||
- Focuses on real usage patterns over checklist validation
|
||||
- Generates actionable insights for development team
|
||||
@@ -0,0 +1,55 @@
|
||||
# OAuth Configuration for Basic Memory MCP Server
|
||||
# Copy this file to .env and update the values
|
||||
|
||||
# Enable OAuth authentication
|
||||
FASTMCP_AUTH_ENABLED=true
|
||||
|
||||
# OAuth provider type: basic, github, google, or supabase
|
||||
# - basic: Built-in OAuth provider with in-memory storage
|
||||
# - github: Integrate with GitHub OAuth
|
||||
# - google: Integrate with Google OAuth
|
||||
# - supabase: Integrate with Supabase Auth (recommended for production)
|
||||
FASTMCP_AUTH_PROVIDER=basic
|
||||
|
||||
# OAuth issuer URL (your MCP server URL)
|
||||
FASTMCP_AUTH_ISSUER_URL=http://localhost:8000
|
||||
|
||||
# Documentation URL for OAuth endpoints
|
||||
FASTMCP_AUTH_DOCS_URL=http://localhost:8000/docs/oauth
|
||||
|
||||
# Required scopes (comma-separated)
|
||||
# Examples: read,write,admin
|
||||
FASTMCP_AUTH_REQUIRED_SCOPES=read,write
|
||||
|
||||
# Secret key for JWT tokens (auto-generated if not set)
|
||||
# FASTMCP_AUTH_SECRET_KEY=your-secret-key-here
|
||||
|
||||
# Enable client registration endpoint
|
||||
FASTMCP_AUTH_CLIENT_REGISTRATION_ENABLED=true
|
||||
|
||||
# Enable token revocation endpoint
|
||||
FASTMCP_AUTH_REVOCATION_ENABLED=true
|
||||
|
||||
# Default scopes for new clients
|
||||
FASTMCP_AUTH_DEFAULT_SCOPES=read
|
||||
|
||||
# Valid scopes that can be requested
|
||||
FASTMCP_AUTH_VALID_SCOPES=read,write,admin
|
||||
|
||||
# Client secret expiry in seconds (optional)
|
||||
# FASTMCP_AUTH_CLIENT_SECRET_EXPIRY=86400
|
||||
|
||||
# GitHub OAuth settings (if using github provider)
|
||||
# GITHUB_CLIENT_ID=your-github-client-id
|
||||
# GITHUB_CLIENT_SECRET=your-github-client-secret
|
||||
|
||||
# Google OAuth settings (if using google provider)
|
||||
# GOOGLE_CLIENT_ID=your-google-client-id
|
||||
# GOOGLE_CLIENT_SECRET=your-google-client-secret
|
||||
|
||||
# Supabase settings (if using supabase provider)
|
||||
# SUPABASE_URL=https://your-project.supabase.co
|
||||
# SUPABASE_ANON_KEY=your-anon-key
|
||||
# SUPABASE_SERVICE_KEY=your-service-key # Optional, for admin operations
|
||||
# SUPABASE_JWT_SECRET=your-jwt-secret # Optional, for token validation
|
||||
# SUPABASE_ALLOWED_CLIENTS=client1,client2 # Comma-separated list of allowed client IDs
|
||||
@@ -1,16 +0,0 @@
|
||||
name: Claude Code Integration
|
||||
|
||||
on:
|
||||
issue_comment:
|
||||
types: [ created ]
|
||||
pull_request_review_comment:
|
||||
types: [ created ]
|
||||
|
||||
jobs:
|
||||
claude-integration:
|
||||
uses: basicmachines-co/claude-code-github-action/.github/workflows/claude-full.yml@v0.11.0
|
||||
with:
|
||||
issue-label: 'claude-fix' # Optional: customize the trigger label
|
||||
secrets:
|
||||
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
|
||||
PERSONAL_ACCESS_TOKEN: ${{ secrets.CLAUDE_TOKEN }}
|
||||
@@ -0,0 +1,114 @@
|
||||
name: Claude Code
|
||||
|
||||
on:
|
||||
issue_comment:
|
||||
types: [created]
|
||||
pull_request_review_comment:
|
||||
types: [created]
|
||||
issues:
|
||||
types: [opened, assigned]
|
||||
pull_request_review:
|
||||
types: [submitted]
|
||||
|
||||
jobs:
|
||||
claude:
|
||||
if: |
|
||||
(github.event_name == 'issue_comment' && contains(github.event.comment.body, '@claude')) ||
|
||||
(github.event_name == 'pull_request_review_comment' && contains(github.event.comment.body, '@claude')) ||
|
||||
(github.event_name == 'pull_request_review' && contains(github.event.review.body, '@claude')) ||
|
||||
(github.event_name == 'issues' && (contains(github.event.issue.body, '@claude') || contains(github.event.issue.title, '@claude')))
|
||||
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
contents: read
|
||||
pull-requests: read
|
||||
issues: read
|
||||
id-token: write
|
||||
steps:
|
||||
- name: Check user permissions
|
||||
id: check_membership
|
||||
uses: actions/github-script@v7
|
||||
with:
|
||||
script: |
|
||||
let actor;
|
||||
if (context.eventName === 'issue_comment') {
|
||||
actor = context.payload.comment.user.login;
|
||||
} else if (context.eventName === 'pull_request_review_comment') {
|
||||
actor = context.payload.comment.user.login;
|
||||
} else if (context.eventName === 'pull_request_review') {
|
||||
actor = context.payload.review.user.login;
|
||||
} else if (context.eventName === 'issues') {
|
||||
actor = context.payload.issue.user.login;
|
||||
}
|
||||
|
||||
console.log(`Checking permissions for user: ${actor}`);
|
||||
|
||||
// List of explicitly allowed users (organization members)
|
||||
const allowedUsers = [
|
||||
'phernandez',
|
||||
'groksrc',
|
||||
'nellins',
|
||||
'bm-claudeai'
|
||||
];
|
||||
|
||||
if (allowedUsers.includes(actor)) {
|
||||
console.log(`User ${actor} is in the allowed list`);
|
||||
core.setOutput('is_member', true);
|
||||
return;
|
||||
}
|
||||
|
||||
// Fallback: Check if user has repository permissions
|
||||
try {
|
||||
const collaboration = await github.rest.repos.getCollaboratorPermissionLevel({
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
username: actor
|
||||
});
|
||||
|
||||
const permission = collaboration.data.permission;
|
||||
console.log(`User ${actor} has permission level: ${permission}`);
|
||||
|
||||
// Allow if user has push access or higher (write, maintain, admin)
|
||||
const allowed = ['write', 'maintain', 'admin'].includes(permission);
|
||||
|
||||
core.setOutput('is_member', allowed);
|
||||
|
||||
if (!allowed) {
|
||||
core.notice(`User ${actor} does not have sufficient repository permissions (has: ${permission})`);
|
||||
}
|
||||
} catch (error) {
|
||||
console.log(`Error checking permissions: ${error.message}`);
|
||||
|
||||
// Final fallback: Check if user is a public member of the organization
|
||||
try {
|
||||
const membership = await github.rest.orgs.getMembershipForUser({
|
||||
org: 'basicmachines-co',
|
||||
username: actor
|
||||
});
|
||||
|
||||
const allowed = membership.data.state === 'active';
|
||||
core.setOutput('is_member', allowed);
|
||||
|
||||
if (!allowed) {
|
||||
core.notice(`User ${actor} is not a public member of basicmachines-co organization`);
|
||||
}
|
||||
} catch (membershipError) {
|
||||
console.log(`Error checking organization membership: ${membershipError.message}`);
|
||||
core.setOutput('is_member', false);
|
||||
core.notice(`User ${actor} does not have access to this repository`);
|
||||
}
|
||||
}
|
||||
|
||||
- name: Checkout repository
|
||||
if: steps.check_membership.outputs.is_member == 'true'
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 1
|
||||
|
||||
- name: Run Claude Code
|
||||
if: steps.check_membership.outputs.is_member == 'true'
|
||||
id: claude
|
||||
uses: anthropics/claude-code-action@beta
|
||||
with:
|
||||
anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
|
||||
allowed_tools: Bash(uv run pytest),Bash(uv run ruff check . --fix),Bash(uv run ruff format .),Bash(uv run pyright),Bash(just test),Bash(just lint),Bash(just format),Bash(just type-check),Bash(just check),Read,Write,Edit,MultiEdit,Glob,Grep,LS
|
||||
@@ -0,0 +1,53 @@
|
||||
name: Dev Release
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
workflow_dispatch: # Allow manual triggering
|
||||
|
||||
jobs:
|
||||
dev-release:
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
id-token: write
|
||||
contents: write
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: "3.12"
|
||||
|
||||
- name: Install uv
|
||||
run: |
|
||||
pip install uv
|
||||
|
||||
- name: Install dependencies and build
|
||||
run: |
|
||||
uv venv
|
||||
uv sync
|
||||
uv build
|
||||
|
||||
- name: Check if this is a dev version
|
||||
id: check_version
|
||||
run: |
|
||||
VERSION=$(uv run python -c "import basic_memory; print(basic_memory.__version__)")
|
||||
echo "version=$VERSION" >> $GITHUB_OUTPUT
|
||||
if [[ "$VERSION" == *"dev"* ]]; then
|
||||
echo "is_dev=true" >> $GITHUB_OUTPUT
|
||||
echo "Dev version detected: $VERSION"
|
||||
else
|
||||
echo "is_dev=false" >> $GITHUB_OUTPUT
|
||||
echo "Release version detected: $VERSION, skipping dev release"
|
||||
fi
|
||||
|
||||
- name: Publish dev version to PyPI
|
||||
if: steps.check_version.outputs.is_dev == 'true'
|
||||
uses: pypa/gh-action-pypi-publish@release/v1
|
||||
with:
|
||||
password: ${{ secrets.PYPI_TOKEN }}
|
||||
skip-existing: true # Don't fail if version already exists
|
||||
@@ -1,96 +1,82 @@
|
||||
name: Release
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
version_type:
|
||||
description: 'Type of version bump (major, minor, patch)'
|
||||
required: true
|
||||
default: 'patch'
|
||||
type: choice
|
||||
options:
|
||||
- patch
|
||||
- minor
|
||||
- major
|
||||
push:
|
||||
tags:
|
||||
- 'v*' # Trigger on version tags like v1.0.0, v0.13.0, etc.
|
||||
|
||||
jobs:
|
||||
release:
|
||||
runs-on: ubuntu-latest
|
||||
concurrency: release
|
||||
permissions:
|
||||
id-token: write
|
||||
contents: write
|
||||
outputs:
|
||||
released: ${{ steps.release.outputs.released }}
|
||||
tag: ${{ steps.release.outputs.tag }}
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Python Semantic Release
|
||||
id: release
|
||||
uses: python-semantic-release/python-semantic-release@master
|
||||
with:
|
||||
github_token: ${{ secrets.GITHUB_TOKEN }}
|
||||
|
||||
- name: Publish to PyPI
|
||||
uses: pypa/gh-action-pypi-publish@release/v1
|
||||
if: steps.release.outputs.released == 'true'
|
||||
with:
|
||||
password: ${{ secrets.PYPI_TOKEN }}
|
||||
|
||||
- name: Publish to GitHub Release Assets
|
||||
uses: python-semantic-release/publish-action@v9.8.9
|
||||
if: steps.release.outputs.released == 'true'
|
||||
with:
|
||||
github_token: ${{ secrets.GITHUB_TOKEN }}
|
||||
tag: ${{ steps.release.outputs.tag }}
|
||||
|
||||
build-macos:
|
||||
needs: release
|
||||
if: needs.release.outputs.released == 'true'
|
||||
runs-on: macos-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
ref: ${{ needs.release.outputs.tag }}
|
||||
|
||||
- name: Set up Python "3.12"
|
||||
uses: actions/setup-python@v4
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: "3.12"
|
||||
cache: 'pip'
|
||||
|
||||
- name: Install librsvg
|
||||
run: brew install librsvg
|
||||
|
||||
- name: Install uv
|
||||
run: |
|
||||
pip install uv
|
||||
|
||||
- name: Create virtual env
|
||||
- name: Install dependencies and build
|
||||
run: |
|
||||
uv venv
|
||||
|
||||
- name: Install dependencies
|
||||
run: |
|
||||
uv sync
|
||||
uv build
|
||||
|
||||
- name: Build macOS installer
|
||||
- name: Verify build succeeded
|
||||
run: |
|
||||
make installer-mac
|
||||
xattr -dr com.apple.quarantine "installer/build/Basic Memory Installer.app"
|
||||
# Verify that build artifacts exist
|
||||
ls -la dist/
|
||||
echo "Build completed successfully"
|
||||
|
||||
- name: Zip macOS installer
|
||||
run: |
|
||||
cd installer/build
|
||||
zip -ry "Basic-Memory-Installer-${{ needs.release.outputs.tag }}.zip" "Basic Memory Installer.app"
|
||||
|
||||
- name: Upload macOS installer
|
||||
uses: softprops/action-gh-release@v1
|
||||
- name: Create GitHub Release
|
||||
uses: softprops/action-gh-release@v2
|
||||
with:
|
||||
files: installer/build/Basic-Memory-Installer-${{ needs.release.outputs.tag }}.zip
|
||||
tag_name: ${{ needs.release.outputs.tag }}
|
||||
files: |
|
||||
dist/*.whl
|
||||
dist/*.tar.gz
|
||||
generate_release_notes: true
|
||||
tag_name: ${{ github.ref_name }}
|
||||
token: ${{ secrets.GITHUB_TOKEN }}
|
||||
|
||||
- name: Publish to PyPI
|
||||
uses: pypa/gh-action-pypi-publish@release/v1
|
||||
with:
|
||||
password: ${{ secrets.PYPI_TOKEN }}
|
||||
|
||||
homebrew:
|
||||
name: Update Homebrew Formula
|
||||
needs: release
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
contents: write
|
||||
actions: read
|
||||
steps:
|
||||
- name: Update Homebrew formula
|
||||
uses: mislav/bump-homebrew-formula-action@v3
|
||||
with:
|
||||
# Formula name in homebrew-basic-memory repo
|
||||
formula-name: basic-memory
|
||||
# The tap repository
|
||||
homebrew-tap: basicmachines-co/homebrew-basic-memory
|
||||
# Base branch of the tap repository
|
||||
base-branch: main
|
||||
# Download URL will be automatically constructed from the tag
|
||||
download-url: https://github.com/basicmachines-co/basic-memory/archive/refs/tags/${{ github.ref_name }}.tar.gz
|
||||
# Commit message for the formula update
|
||||
commit-message: |
|
||||
{{formulaName}} {{version}}
|
||||
|
||||
Created by https://github.com/basicmachines-co/basic-memory/actions/runs/${{ github.run_id }}
|
||||
env:
|
||||
# Personal Access Token with repo scope for homebrew-basic-memory repo
|
||||
COMMITTER_TOKEN: ${{ secrets.HOMEBREW_TOKEN }}
|
||||
|
||||
@@ -35,6 +35,10 @@ jobs:
|
||||
run: |
|
||||
pip install uv
|
||||
|
||||
- name: Install just
|
||||
run: |
|
||||
curl --proto '=https' --tlsv1.2 -sSf https://just.systems/install.sh | bash -s -- --to /usr/local/bin
|
||||
|
||||
- name: Create virtual env
|
||||
run: |
|
||||
uv venv
|
||||
@@ -45,9 +49,9 @@ jobs:
|
||||
|
||||
- name: Run type checks
|
||||
run: |
|
||||
uv run make type-check
|
||||
just type-check
|
||||
|
||||
- name: Run tests
|
||||
run: |
|
||||
uv pip install pytest pytest-cov
|
||||
uv run make test
|
||||
just test
|
||||
|
||||
+3
-2
@@ -42,7 +42,7 @@ ENV/
|
||||
|
||||
# macOS
|
||||
.DS_Store
|
||||
/.coverage.*
|
||||
.coverage.*
|
||||
|
||||
# obsidian docs:
|
||||
/docs/.obsidian/
|
||||
@@ -51,4 +51,5 @@ ENV/
|
||||
|
||||
|
||||
# claude action
|
||||
claude-output
|
||||
claude-output
|
||||
**/.claude/settings.local.json
|
||||
@@ -0,0 +1,14 @@
|
||||
{
|
||||
"mcpServers": {
|
||||
"basic-memory": {
|
||||
"command": "uv",
|
||||
"args": [
|
||||
"--directory",
|
||||
"/Users/phernandez/dev/basicmachines/basic-memory",
|
||||
"run",
|
||||
"src/basic_memory/cli/main.py",
|
||||
"mcp"
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,42 @@
|
||||
# OAuth Quick Start
|
||||
|
||||
Basic Memory supports OAuth authentication for secure access control. For detailed documentation, see [OAuth Authentication Guide](docs/OAuth%20Authentication%20Guide.md).
|
||||
|
||||
## Quick Test with MCP Inspector
|
||||
|
||||
```bash
|
||||
# 1. Set a consistent secret key
|
||||
export FASTMCP_AUTH_SECRET_KEY="test-secret-key"
|
||||
|
||||
# 2. Start server with OAuth
|
||||
FASTMCP_AUTH_ENABLED=true basic-memory mcp --transport streamable-http
|
||||
|
||||
# 3. In another terminal, get a test token
|
||||
export FASTMCP_AUTH_SECRET_KEY="test-secret-key" # Same key!
|
||||
basic-memory auth test-auth
|
||||
|
||||
# 4. Copy the access token and use in MCP Inspector:
|
||||
# - Server URL: http://localhost:8000/mcp
|
||||
# - Transport: streamable-http
|
||||
# - Custom Headers:
|
||||
# Authorization: Bearer YOUR_ACCESS_TOKEN
|
||||
# Accept: application/json, text/event-stream
|
||||
```
|
||||
|
||||
## OAuth Endpoints
|
||||
|
||||
- `GET /authorize` - Authorization endpoint
|
||||
- `POST /token` - Token exchange endpoint
|
||||
- `GET /.well-known/oauth-authorization-server` - OAuth metadata
|
||||
|
||||
## Common Issues
|
||||
|
||||
1. **401 Unauthorized**: Make sure you're using the same secret key for both server and client
|
||||
2. **404 Not Found**: Use `/authorize` not `/auth/authorize`
|
||||
3. **Token Invalid**: Tokens don't persist across server restarts with basic provider
|
||||
|
||||
## Documentation
|
||||
|
||||
- [OAuth Authentication Guide](docs/OAuth%20Authentication%20Guide.md) - Complete setup guide
|
||||
- [Supabase OAuth Setup](docs/Supabase%20OAuth%20Setup.md) - Production deployment
|
||||
- [External OAuth Providers](docs/External%20OAuth%20Providers.md) - GitHub, Google integration
|
||||
+400
-1
@@ -1,5 +1,404 @@
|
||||
# CHANGELOG
|
||||
|
||||
## v0.13.6 (2025-06-18)
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
- **Custom Entity Types** - Support for custom entity types in write_note
|
||||
([`7789864`](https://github.com/basicmachines-co/basic-memory/commit/77898644933589c2da9bdd60571d54137a5309ed))
|
||||
- Fixed `entity_type` parameter for `write_note` MCP tool to respect value passed in
|
||||
- Frontmatter `type` field automatically respected when no explicit parameter provided
|
||||
- Maintains backward compatibility with default "note" type
|
||||
|
||||
- **#139**: Fix "UNIQUE constraint failed: entity.permalink" database error
|
||||
([`c6215fd`](https://github.com/basicmachines-co/basic-memory/commit/c6215fd819f9564ead91cf3a950f855241446096))
|
||||
- Implement SQLAlchemy UPSERT strategy to handle permalink conflicts gracefully
|
||||
- Eliminates crashes when creating notes with existing titles in same folders
|
||||
- Seamlessly updates existing entities instead of failing with constraint errors
|
||||
|
||||
- **Database Migration Performance** - Eliminate redundant migration initialization
|
||||
([`84d2aaf`](https://github.com/basicmachines-co/basic-memory/commit/84d2aaf6414dd083af4b0df73f6c8139b63468f6))
|
||||
- Fix duplicate migration calls that slowed system startup
|
||||
- Improve performance with multiple projects (tested with 28+ projects)
|
||||
- Add migration deduplication safeguards with comprehensive test coverage
|
||||
|
||||
- **User Experience** - Correct spelling error in continue_conversation prompt
|
||||
([`b4c26a6`](https://github.com/basicmachines-co/basic-memory/commit/b4c26a613379e6f2ba655efe3d7d8d40c27999e5))
|
||||
- Fix "Chose a folder" → "Choose a folder" in MCP prompt instructions
|
||||
- Improve grammar and clarity in user-facing prompt text
|
||||
|
||||
### Documentation
|
||||
|
||||
- **Website Updates** - Add new website and community links to README
|
||||
([`3fdce68`](https://github.com/basicmachines-co/basic-memory/commit/3fdce683d7ad8b6f4855d7138d5ff2136d4c07bc))
|
||||
|
||||
- **Project Documentation** - Update README.md and CLAUDE.md with latest project information
|
||||
([`782cb2d`](https://github.com/basicmachines-co/basic-memory/commit/782cb2df28803482d209135a054e67cc32d7363e))
|
||||
|
||||
### Technical Improvements
|
||||
|
||||
- **Comprehensive Test Coverage** - Add extensive test suites for new features
|
||||
- Custom entity type validation with 8 new test scenarios
|
||||
- UPSERT behavior testing with edge case coverage
|
||||
- Migration deduplication testing with 6 test scenarios
|
||||
- Database constraint handling validation
|
||||
|
||||
- **Code Quality** - Enhanced error handling and validation
|
||||
- Improved SQLAlchemy patterns with modern UPSERT operations
|
||||
- Better conflict resolution strategies for entity management
|
||||
- Strengthened database consistency guarantees
|
||||
|
||||
### Performance
|
||||
|
||||
- **Database Operations** - Faster startup and improved scalability
|
||||
- Reduced migration overhead for multi-project setups
|
||||
- Optimized conflict resolution for entity creation
|
||||
- Enhanced performance with growing knowledge bases
|
||||
|
||||
### Migration Guide
|
||||
|
||||
This release includes automatic database improvements. No manual migration required:
|
||||
|
||||
- Existing notes and entity types continue working unchanged
|
||||
- New `entity_type` parameter is optional and backward compatible
|
||||
- Database performance improvements apply automatically
|
||||
- All existing MCP tool behavior preserved
|
||||
|
||||
### Installation
|
||||
|
||||
```bash
|
||||
# Latest stable release
|
||||
uv tool install basic-memory
|
||||
|
||||
# Update existing installation
|
||||
uv tool upgrade basic-memory
|
||||
```
|
||||
|
||||
## v0.13.5 (2025-06-11)
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
- **MCP Tools**: Renamed `create_project` tool to `create_memory_project` for namespace isolation
|
||||
- **Namespace**: Continued namespace isolation effort to prevent conflicts with other MCP servers
|
||||
|
||||
### Changes
|
||||
|
||||
- Tool functionality remains identical - only the name changed from `create_project` to `create_memory_project`
|
||||
- All integration tests updated to use the new tool name
|
||||
- Completes namespace isolation for project management tools alongside `list_memory_projects`
|
||||
|
||||
## v0.13.4 (2025-06-11)
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
- **MCP Tools**: Renamed `list_projects` tool to `list_memory_projects` to avoid naming conflicts with other MCP servers
|
||||
- **Namespace**: Improved tool naming specificity for better MCP server integration and isolation
|
||||
|
||||
### Changes
|
||||
|
||||
- Tool functionality remains identical - only the name changed from `list_projects` to `list_memory_projects`
|
||||
- All integration tests updated to use the new tool name
|
||||
- Better namespace isolation for Basic Memory MCP tools
|
||||
|
||||
## v0.13.3 (2025-06-11)
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
- **Projects**: Fixed case-insensitive project switching where switching succeeded but subsequent operations failed due to session state inconsistency
|
||||
- **Config**: Enhanced config manager with case-insensitive project lookup using permalink-based matching
|
||||
- **MCP Tools**: Updated project management tools to store canonical project names from database instead of user input
|
||||
- **API**: Improved project service to handle both name and permalink lookups consistently
|
||||
|
||||
### Technical Improvements
|
||||
|
||||
- Added comprehensive case-insensitive project switching test coverage with 5 new integration test scenarios
|
||||
- Fixed permalink generation inconsistencies where different case inputs could generate different permalinks
|
||||
- Enhanced project URL construction to use permalinks consistently across all API calls
|
||||
- Improved error handling and session state management for project operations
|
||||
|
||||
### Changes
|
||||
|
||||
- Project switching now preserves canonical project names from database in session state
|
||||
- All project operations use permalink-based lookups for case-insensitive matching
|
||||
- Enhanced test coverage ensures reliable case-insensitive project operations
|
||||
|
||||
## v0.13.2 (2025-06-11)
|
||||
|
||||
### Features
|
||||
|
||||
- **Release Management**: Added automated release management system with version control in `__init__.py`
|
||||
- **Automation**: Implemented justfile targets for `release` and `beta` commands with comprehensive quality gates
|
||||
- **CI/CD**: Enhanced release process with automatic version updates, git tagging, and GitHub release creation
|
||||
|
||||
### Development Experience
|
||||
|
||||
- Added `.claude/commands/release/` directory with automation documentation
|
||||
- Implemented release validation including lint, type-check, and test execution
|
||||
- Streamlined release workflow from manual process to single-command automation
|
||||
|
||||
### Technical Improvements
|
||||
|
||||
- Updated package version management to use actual version numbers instead of dynamic versioning
|
||||
- Added release process documentation and command references
|
||||
- Enhanced justfile with comprehensive release automation targets
|
||||
|
||||
## v0.13.1 (2025-06-11)
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
- **CLI**: Fixed `basic-memory project` project management commands that were not working in v0.13.0 (#129)
|
||||
- **Projects**: Resolved case sensitivity issues when switching between projects that caused "Project not found" errors (#127)
|
||||
- **API**: Standardized CLI project command endpoints and improved error handling
|
||||
- **Core**: Implemented consistent project name handling using permalinks to avoid case-related conflicts
|
||||
|
||||
### Changes
|
||||
|
||||
- Renamed `basic-memory project sync` command to `basic-memory project sync-config` for clarity
|
||||
- Improved project switching reliability across different case variations
|
||||
- Removed redundant server status messages from CLI error outputs
|
||||
|
||||
## v0.13.0 (2025-06-11)
|
||||
|
||||
### Overview
|
||||
|
||||
Basic Memory v0.13.0 is a **major release** that transforms Basic Memory into a true multi-project knowledge management system. This release introduces fluid project switching, advanced note editing capabilities, robust file management, and production-ready OAuth authentication - all while maintaining full backward compatibility.
|
||||
|
||||
**What's New for Users:**
|
||||
- 🎯 **Switch between projects instantly** during conversations with Claude
|
||||
- ✏️ **Edit notes incrementally** without rewriting entire documents
|
||||
- 📁 **Move and organize notes** with full database consistency
|
||||
- 📖 **View notes as formatted artifacts** for better readability in Claude Desktop
|
||||
- 🔍 **Search frontmatter tags** to discover content more easily
|
||||
- 🔐 **OAuth authentication** for secure remote access
|
||||
- ⚡ **Development builds** automatically published for beta testing
|
||||
|
||||
**Key v0.13.0 Accomplishments:**
|
||||
- ✅ **Complete Project Management System** - Project switching and project-specific operations
|
||||
- ✅ **Advanced Note Editing** - Incremental editing with append, prepend, find/replace, and section operations
|
||||
- ✅ **View Notes as Artifacts in Claude Desktop/Web** - Use the view_note tool to view a note as an artifact
|
||||
- ✅ **File Management System** - Full move operations with database consistency and rollback protection
|
||||
- ✅ **Enhanced Search Capabilities** - Frontmatter tags now searchable, improved content discoverability
|
||||
- ✅ **Unified Database Architecture** - Single app-level database for better performance and project management
|
||||
|
||||
### Major Features
|
||||
|
||||
#### 1. Multiple Project Management
|
||||
|
||||
**Switch between projects instantly during conversations:**
|
||||
|
||||
```
|
||||
💬 "What projects do I have?"
|
||||
🤖 Available projects:
|
||||
• main (current, default)
|
||||
• work-notes
|
||||
• personal-journal
|
||||
• code-snippets
|
||||
|
||||
💬 "Switch to work-notes"
|
||||
🤖 ✓ Switched to work-notes project
|
||||
|
||||
Project Summary:
|
||||
• 47 entities
|
||||
• 125 observations
|
||||
• 23 relations
|
||||
|
||||
💬 "What did I work on yesterday?"
|
||||
🤖 [Shows recent activity from work-notes project]
|
||||
```
|
||||
|
||||
**Key Capabilities:**
|
||||
- **Instant Project Switching**: Change project context mid-conversation without restart
|
||||
- **Project-Specific Operations**: Operations work within the currently active project context
|
||||
- **Project Discovery**: List all available projects with status indicators
|
||||
- **Session Context**: Maintains active project throughout conversation
|
||||
- **Backward Compatibility**: Existing single-project setups continue to work seamlessly
|
||||
|
||||
#### 2. Advanced Note Editing
|
||||
|
||||
**Edit notes incrementally without rewriting entire documents:**
|
||||
|
||||
```python
|
||||
# Append new sections to existing notes
|
||||
edit_note("project-planning", "append", "\n## New Requirements\n- Feature X\n- Feature Y")
|
||||
|
||||
# Prepend timestamps to meeting notes
|
||||
edit_note("meeting-notes", "prepend", "## 2025-05-27 Update\n- Progress update...")
|
||||
|
||||
# Replace specific sections under headers
|
||||
edit_note("api-spec", "replace_section", "New implementation details", section="## Implementation")
|
||||
|
||||
# Find and replace with validation
|
||||
edit_note("config", "find_replace", "v0.13.0", find_text="v0.12.0", expected_replacements=2)
|
||||
```
|
||||
|
||||
**Key Capabilities:**
|
||||
- **Append Operations**: Add content to end of notes (most common use case)
|
||||
- **Prepend Operations**: Add content to beginning of notes
|
||||
- **Section Replacement**: Replace content under specific markdown headers
|
||||
- **Find & Replace**: Simple text replacements with occurrence counting
|
||||
- **Smart Error Handling**: Helpful guidance when operations fail
|
||||
- **Project Context**: Works within the active project with session awareness
|
||||
|
||||
#### 3. Smart File Management
|
||||
|
||||
**Move and organize notes:**
|
||||
|
||||
```python
|
||||
# Simple moves with automatic folder creation
|
||||
move_note("my-note", "work/projects/my-note.md")
|
||||
|
||||
# Organize within the active project
|
||||
move_note("shared-doc", "archive/old-docs/shared-doc.md")
|
||||
|
||||
# Rename operations
|
||||
move_note("old-name", "same-folder/new-name.md")
|
||||
```
|
||||
|
||||
**Key Capabilities:**
|
||||
- **Database Consistency**: Updates file paths, permalinks, and checksums automatically
|
||||
- **Search Reindexing**: Maintains search functionality after moves
|
||||
- **Folder Creation**: Automatically creates destination directories
|
||||
- **Project Isolation**: Operates within the currently active project
|
||||
- **Link Preservation**: Maintains internal links and references
|
||||
|
||||
#### 4. Enhanced Search & Discovery
|
||||
|
||||
**Find content more easily with improved search capabilities:**
|
||||
|
||||
- **Frontmatter Tag Search**: Tags from YAML frontmatter are now indexed and searchable
|
||||
- **Improved Content Discovery**: Search across titles, content, tags, and metadata
|
||||
- **Project-Scoped Search**: Search within the currently active project
|
||||
- **Better Search Quality**: Enhanced FTS5 indexing with tag content inclusion
|
||||
|
||||
**Example:**
|
||||
```yaml
|
||||
---
|
||||
title: Coffee Brewing Methods
|
||||
tags: [coffee, brewing, equipment]
|
||||
---
|
||||
```
|
||||
Now searchable by: "coffee", "brewing", "equipment", or "Coffee Brewing Methods"
|
||||
|
||||
#### 5. Unified Database Architecture
|
||||
|
||||
**Single app-level database for better performance and project management:**
|
||||
|
||||
- **Migration from Per-Project DBs**: Moved from multiple SQLite files to single app database
|
||||
- **Project Isolation**: Proper data separation with project_id foreign keys
|
||||
- **Better Performance**: Optimized queries and reduced file I/O
|
||||
|
||||
### Complete MCP Tool Suite
|
||||
|
||||
#### New Project Management Tools
|
||||
- **`list_projects()`** - Discover and list all available projects with status
|
||||
- **`switch_project(project_name)`** - Change active project context during conversations
|
||||
- **`get_current_project()`** - Show currently active project with statistics
|
||||
- **`set_default_project(project_name)`** - Update default project configuration
|
||||
- **`sync_status()`** - Check file synchronization status and background operations
|
||||
|
||||
#### New Note Operations Tools
|
||||
- **`edit_note()`** - Incremental note editing (append, prepend, find/replace, section replace)
|
||||
- **`move_note()`** - Move notes with database consistency and search reindexing
|
||||
- **`view_note()`** - Display notes as formatted artifacts for better readability in Claude Desktop
|
||||
|
||||
#### Enhanced Existing Tools
|
||||
All existing tools now support:
|
||||
- **Session context awareness** (operates within the currently active project)
|
||||
- **Enhanced error messages** with project context metadata
|
||||
- **Improved response formatting** with project information footers
|
||||
- **Project isolation** ensures operations stay within the correct project boundaries
|
||||
|
||||
|
||||
### User Experience Improvements
|
||||
|
||||
#### Installation Options
|
||||
|
||||
**Multiple ways to install and test Basic Memory:**
|
||||
|
||||
```bash
|
||||
# Stable release
|
||||
uv tool install basic-memory
|
||||
|
||||
# Beta/pre-releases
|
||||
uv tool install basic-memory --pre
|
||||
```
|
||||
|
||||
|
||||
#### Bug Fixes & Quality Improvements
|
||||
|
||||
**Major issues resolved in v0.13.0:**
|
||||
|
||||
- **#118**: Fixed YAML tag formatting to follow standard specification
|
||||
- **#110**: Fixed `--project` flag consistency across all CLI commands
|
||||
- **#107**: Fixed write_note update failures with existing notes
|
||||
- **#93**: Fixed custom permalink handling in frontmatter
|
||||
- **#52**: Enhanced search capabilities with frontmatter tag indexing
|
||||
- **FTS5 Search**: Fixed special character handling in search queries
|
||||
- **Error Handling**: Improved error messages and validation across all tools
|
||||
|
||||
### Breaking Changes & Migration
|
||||
|
||||
#### For Existing Users
|
||||
|
||||
**Automatic Migration**: First run will automatically migrate existing data to the new unified database structure. No manual action required.
|
||||
|
||||
**What Changes:**
|
||||
- Database location: Moved to `~/.basic-memory/memory.db` (unified across projects)
|
||||
- Configuration: Projects defined in `~/.basic-memory/config.json` are synced with database
|
||||
|
||||
**What Stays the Same:**
|
||||
- All existing notes and data remain unchanged
|
||||
- Default project behavior maintained for single-project users
|
||||
- All existing MCP tools continue to work without modification
|
||||
|
||||
### Documentation & Resources
|
||||
|
||||
#### New Documentation
|
||||
- [Project Management Guide](docs/Project%20Management.md) - Multi-project workflows
|
||||
- [Note Editing Guide](docs/Note%20Editing.md) - Advanced editing techniques
|
||||
|
||||
#### Updated Documentation
|
||||
- [README.md](README.md) - Installation options and beta build instructions
|
||||
- [CONTRIBUTING.md](CONTRIBUTING.md) - Release process and version management
|
||||
- [CLAUDE.md](CLAUDE.md) - Development workflow and CI/CD documentation
|
||||
- [Claude.ai Integration](docs/Claude.ai%20Integration.md) - Updated MCP tool examples
|
||||
|
||||
#### Quick Start Examples
|
||||
|
||||
**Project Switching:**
|
||||
```
|
||||
💬 "Switch to my work project and show recent activity"
|
||||
🤖 [Calls switch_project("work") then recent_activity()]
|
||||
```
|
||||
|
||||
**Note Editing:**
|
||||
```
|
||||
💬 "Add a section about deployment to my API docs"
|
||||
🤖 [Calls edit_note("api-docs", "append", "## Deployment\n...")]
|
||||
```
|
||||
|
||||
**File Organization:**
|
||||
```
|
||||
💬 "Move my old meeting notes to the archive folder"
|
||||
🤖 [Calls move_note("meeting-notes", "archive/old-meetings.md")]
|
||||
```
|
||||
|
||||
|
||||
|
||||
## v0.12.3 (2025-04-17)
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
- Add extra logic for permalink generation with mixed Latin unicode and Chinese characters
|
||||
([`73ea91f`](https://github.com/basicmachines-co/basic-memory/commit/73ea91fe0d1f7ab89b99a1b691d59fe608b7fcbb))
|
||||
|
||||
Signed-off-by: phernandez <paul@basicmachines.co>
|
||||
|
||||
- Modify recent_activity args to be strings instead of enums
|
||||
([`3c1cc34`](https://github.com/basicmachines-co/basic-memory/commit/3c1cc346df519e703fae6412d43a92c7232c6226))
|
||||
|
||||
Signed-off-by: phernandez <paul@basicmachines.co>
|
||||
|
||||
|
||||
## v0.12.2 (2025-04-08)
|
||||
|
||||
@@ -770,4 +1169,4 @@ Co-authored-by: phernandez <phernandez@basicmachines.co>
|
||||
### Chores
|
||||
|
||||
- Remove basic-foundation src ref in pyproject.toml
|
||||
([`29fce8b`](https://github.com/basicmachines-co/basic-memory/commit/29fce8b0b922d54d7799bf2534107ee6cfb961b8))
|
||||
([`29fce8b`](https://github.com/basicmachines-co/basic-memory/commit/29fce8b0b922d54d7799bf2534107ee6cfb961b8))
|
||||
@@ -14,15 +14,15 @@ See the [README.md](README.md) file for a project overview.
|
||||
|
||||
### Build and Test Commands
|
||||
|
||||
- Install: `make install` or `pip install -e ".[dev]"`
|
||||
- Run tests: `uv run pytest -p pytest_mock -v` or `make test`
|
||||
- Install: `just install` or `pip install -e ".[dev]"`
|
||||
- Run tests: `uv run pytest -p pytest_mock -v` or `just test`
|
||||
- Single test: `pytest tests/path/to/test_file.py::test_function_name`
|
||||
- Lint: `make lint` or `ruff check . --fix`
|
||||
- Type check: `make type-check` or `uv run pyright`
|
||||
- Format: `make format` or `uv run ruff format .`
|
||||
- Run all code checks: `make check` (runs lint, format, type-check, test)
|
||||
- Create db migration: `make migration m="Your migration message"`
|
||||
- Run development MCP Inspector: `make run-inspector`
|
||||
- Lint: `just lint` or `ruff check . --fix`
|
||||
- Type check: `just type-check` or `uv run pyright`
|
||||
- Format: `just format` or `uv run ruff format .`
|
||||
- Run all code checks: `just check` (runs lint, format, type-check, test)
|
||||
- Create db migration: `just migration "Your migration message"`
|
||||
- Run development MCP Inspector: `just run-inspector`
|
||||
|
||||
### Code Style Guidelines
|
||||
|
||||
@@ -65,6 +65,7 @@ See the [README.md](README.md) file for a project overview.
|
||||
- Test database uses in-memory SQLite
|
||||
- Avoid creating mocks in tests in most circumstances.
|
||||
- Each test runs in a standalone environment with in memory SQLite and tmp_file directory
|
||||
- Do not use mocks in tests if possible. Tests run with an in memory sqlite db, so they are not needed. See fixtures in conftest.py
|
||||
|
||||
## BASIC MEMORY PRODUCT USAGE
|
||||
|
||||
@@ -96,15 +97,26 @@ See the [README.md](README.md) file for a project overview.
|
||||
|
||||
**Content Management:**
|
||||
- `write_note(title, content, folder, tags)` - Create/update markdown notes with semantic observations and relations
|
||||
- `read_note(identifier, page, page_size)` - Read notes by title, permalink, or memory:// URL with knowledge graph
|
||||
awareness
|
||||
- `read_file(path)` - Read raw file content (text, images, binaries) without knowledge graph processing
|
||||
- `read_note(identifier, page, page_size)` - Read notes by title, permalink, or memory:// URL with knowledge graph awareness
|
||||
- `edit_note(identifier, operation, content)` - Edit notes incrementally (append, prepend, find/replace, section replace)
|
||||
- `move_note(identifier, destination_path)` - Move notes with database consistency and search reindexing
|
||||
- `view_note(identifier)` - Display notes as formatted artifacts for better readability in Claude Desktop
|
||||
- `read_content(path)` - Read raw file content (text, images, binaries) without knowledge graph processing
|
||||
- `delete_note(identifier)` - Delete notes from knowledge base
|
||||
|
||||
**Project Management:**
|
||||
- `list_memory_projects()` - List all available projects with status indicators
|
||||
- `switch_project(project_name)` - Switch to different project context during conversations
|
||||
- `get_current_project()` - Show currently active project with statistics
|
||||
- `create_memory_project(name, path, set_default)` - Create new Basic Memory projects
|
||||
- `delete_project(name)` - Delete projects from configuration and database
|
||||
- `set_default_project(name)` - Set default project in config
|
||||
- `sync_status()` - Check file synchronization status and background operations
|
||||
|
||||
**Knowledge Graph Navigation:**
|
||||
- `build_context(url, depth, timeframe)` - Navigate the knowledge graph via memory:// URLs for conversation
|
||||
continuity
|
||||
- `recent_activity(type, depth, timeframe)` - Get recently updated information with specified timeframe (e.g., "
|
||||
1d", "1 week")
|
||||
- `build_context(url, depth, timeframe)` - Navigate the knowledge graph via memory:// URLs for conversation continuity
|
||||
- `recent_activity(type, depth, timeframe)` - Get recently updated information with specified timeframe (e.g., "1d", "1 week")
|
||||
- `list_directory(dir_name, depth, file_name_glob)` - List directory contents with filtering and depth control
|
||||
|
||||
**Search & Discovery:**
|
||||
- `search_notes(query, page, page_size)` - Full-text search across all content with filtering options
|
||||
@@ -172,4 +184,72 @@ With GitHub integration, the development workflow includes:
|
||||
4. **Documentation maintenance** - Claude can keep documentation updated as the code evolves
|
||||
|
||||
With this integration, the AI assistant is a full-fledged team member rather than just a tool for generating code
|
||||
snippets.
|
||||
snippets.
|
||||
|
||||
|
||||
### Basic Memory Pro
|
||||
|
||||
Basic Memory Pro is a desktop GUI application that wraps the basic-memory CLI/MCP tools:
|
||||
|
||||
- Built with Tauri (Rust), React (TypeScript), and a Python FastAPI sidecar
|
||||
- Provides visual knowledge graph exploration and project management
|
||||
- Uses the same core codebase but adds a desktop-friendly interface
|
||||
- Project configuration is shared between CLI and Pro versions
|
||||
- Multiple project support with visual switching interface
|
||||
|
||||
local repo: /Users/phernandez/dev/basicmachines/basic-memory-pro
|
||||
github: https://github.com/basicmachines-co/basic-memory-pro
|
||||
|
||||
## Release and Version Management
|
||||
|
||||
Basic Memory uses `uv-dynamic-versioning` for automatic version management based on git tags:
|
||||
|
||||
### Version Types
|
||||
- **Development versions**: Automatically generated from commits (e.g., `0.12.4.dev26+468a22f`)
|
||||
- **Beta releases**: Created by tagging with beta suffixes (e.g., `v0.13.0b1`, `v0.13.0rc1`)
|
||||
- **Stable releases**: Created by tagging with version numbers (e.g., `v0.13.0`)
|
||||
|
||||
### Release Workflows
|
||||
|
||||
#### Development Builds (Automatic)
|
||||
- Triggered on every push to `main` branch
|
||||
- Publishes dev versions like `0.12.4.dev26+468a22f` to PyPI
|
||||
- Allows continuous testing of latest changes
|
||||
- Users install with: `pip install basic-memory --pre --force-reinstall`
|
||||
|
||||
#### Beta/RC Releases (Manual)
|
||||
- Create beta tag: `git tag v0.13.0b1 && git push origin v0.13.0b1`
|
||||
- Automatically builds and publishes to PyPI as pre-release
|
||||
- Users install with: `pip install basic-memory --pre`
|
||||
- Use for milestone testing before stable release
|
||||
|
||||
#### Stable Releases (Automated)
|
||||
- Use the automated release system: `just release v0.13.0`
|
||||
- Includes comprehensive quality checks (lint, format, type-check, tests)
|
||||
- Automatically updates version in `__init__.py`
|
||||
- Creates git tag and pushes to GitHub
|
||||
- Triggers GitHub Actions workflow for:
|
||||
- PyPI publication
|
||||
- Homebrew formula update (requires HOMEBREW_TOKEN secret)
|
||||
|
||||
**Manual method (legacy):**
|
||||
- Create version tag: `git tag v0.13.0 && git push origin v0.13.0`
|
||||
|
||||
#### Homebrew Formula Updates
|
||||
- Automatically triggered after successful PyPI release
|
||||
- Updates formula in `basicmachines-co/homebrew-basic-memory` repo
|
||||
- Requires `HOMEBREW_TOKEN` secret in GitHub repository settings:
|
||||
- Create a fine-grained Personal Access Token with `Contents: Read and Write` and `Actions: Read` scopes on `basicmachines-co/homebrew-basic-memory`
|
||||
- Add as repository secret named `HOMEBREW_TOKEN` in `basicmachines-co/basic-memory`
|
||||
- Formula updates include new version URL and SHA256 checksum
|
||||
|
||||
### For Development
|
||||
- **Automated releases**: Use `just release v0.13.x` for stable releases and `just beta v0.13.0b1` for beta releases
|
||||
- **Quality gates**: All releases require passing lint, format, type-check, and test suites
|
||||
- **Version management**: Versions automatically derived from git tags via `uv-dynamic-versioning`
|
||||
- **Configuration**: `pyproject.toml` uses `dynamic = ["version"]`
|
||||
- **Release automation**: `__init__.py` updated automatically during release process
|
||||
- **CI/CD**: GitHub Actions handles building and PyPI publication
|
||||
|
||||
## Development Notes
|
||||
- make sure you sign off on commits
|
||||
|
||||
+44
-8
@@ -15,8 +15,8 @@ project and how to get started as a developer.
|
||||
|
||||
2. **Install Dependencies**:
|
||||
```bash
|
||||
# Using make (recommended)
|
||||
make install
|
||||
# Using just (recommended)
|
||||
just install
|
||||
|
||||
# Or using uv
|
||||
uv install -e ".[dev]"
|
||||
@@ -25,10 +25,12 @@ project and how to get started as a developer.
|
||||
pip install -e ".[dev]"
|
||||
```
|
||||
|
||||
> **Note**: Basic Memory uses [just](https://just.systems) as a modern command runner. Install with `brew install just` or `cargo install just`.
|
||||
|
||||
3. **Run the Tests**:
|
||||
```bash
|
||||
# Run all tests
|
||||
make test
|
||||
just test
|
||||
# or
|
||||
uv run pytest -p pytest_mock -v
|
||||
|
||||
@@ -49,16 +51,16 @@ project and how to get started as a developer.
|
||||
4. **Check Code Quality**:
|
||||
```bash
|
||||
# Run all checks at once
|
||||
make check
|
||||
just check
|
||||
|
||||
# Or run individual checks
|
||||
make lint # Run linting
|
||||
make format # Format code
|
||||
make type-check # Type checking
|
||||
just lint # Run linting
|
||||
just format # Format code
|
||||
just type-check # Type checking
|
||||
```
|
||||
5. **Test Your Changes**: Ensure all tests pass locally and maintain 100% test coverage.
|
||||
```bash
|
||||
make test
|
||||
just test
|
||||
```
|
||||
6. **Submit a PR**: Submit a pull request with a detailed description of your changes.
|
||||
|
||||
@@ -144,6 +146,40 @@ agreement to the DCO.
|
||||
- **Database Testing**: Use in-memory SQLite for testing database operations
|
||||
- **Fixtures**: Use async pytest fixtures for setup and teardown
|
||||
|
||||
## Release Process
|
||||
|
||||
Basic Memory uses automatic versioning based on git tags with `uv-dynamic-versioning`. Here's how releases work:
|
||||
|
||||
### Version Management
|
||||
- **Development versions**: Automatically generated from git commits (e.g., `0.12.4.dev26+468a22f`)
|
||||
- **Beta releases**: Created by tagging with beta suffixes (e.g., `git tag v0.13.0b1`)
|
||||
- **Stable releases**: Created by tagging with version numbers (e.g., `git tag v0.13.0`)
|
||||
|
||||
### Release Workflows
|
||||
|
||||
#### Development Builds
|
||||
- Automatically published to PyPI on every commit to `main`
|
||||
- Version format: `0.12.4.dev26+468a22f` (base version + dev + commit count + hash)
|
||||
- Users install with: `pip install basic-memory --pre --force-reinstall`
|
||||
|
||||
#### Beta Releases
|
||||
1. Create and push a beta tag: `git tag v0.13.0b1 && git push origin v0.13.0b1`
|
||||
2. GitHub Actions automatically builds and publishes to PyPI
|
||||
3. Users install with: `pip install basic-memory --pre`
|
||||
|
||||
#### Stable Releases
|
||||
1. Create and push a version tag: `git tag v0.13.0 && git push origin v0.13.0`
|
||||
2. GitHub Actions automatically:
|
||||
- Builds the package with version `0.13.0`
|
||||
- Creates GitHub release with auto-generated notes
|
||||
- Publishes to PyPI
|
||||
3. Users install with: `pip install basic-memory`
|
||||
|
||||
### For Contributors
|
||||
- No manual version bumping required
|
||||
- Versions are automatically derived from git tags
|
||||
- Focus on code changes, not version management
|
||||
|
||||
## Creating Issues
|
||||
|
||||
If you're planning to work on something, please create an issue first to discuss the approach. Include:
|
||||
|
||||
@@ -1,67 +0,0 @@
|
||||
.PHONY: install test test-module lint clean format type-check installer-mac installer-win check
|
||||
|
||||
install:
|
||||
pip install -e ".[dev]"
|
||||
|
||||
test:
|
||||
uv run pytest -p pytest_mock -v
|
||||
|
||||
# Run tests for a specific module
|
||||
# Usage: make test-module m=path/to/module.py [cov=module_path]
|
||||
test-module:
|
||||
@if [ -z "$(m)" ]; then \
|
||||
echo "Usage: make test-module m=path/to/module.py [cov=module_path]"; \
|
||||
exit 1; \
|
||||
fi; \
|
||||
if [ -z "$(cov)" ]; then \
|
||||
uv run pytest $(m) -v; \
|
||||
else \
|
||||
uv run pytest $(m) -v --cov=$(cov); \
|
||||
fi
|
||||
|
||||
lint:
|
||||
ruff check . --fix
|
||||
|
||||
type-check:
|
||||
uv run pyright
|
||||
|
||||
clean:
|
||||
find . -type f -name '*.pyc' -delete
|
||||
find . -type d -name '__pycache__' -exec rm -r {} +
|
||||
rm -rf installer/build/
|
||||
rm -rf installer/dist/
|
||||
rm -f rw.*.dmg
|
||||
rm -rf dist
|
||||
rm -rf installer/build
|
||||
rm -rf installer/dist
|
||||
rm -f .coverage.*
|
||||
|
||||
format:
|
||||
uv run ruff format .
|
||||
|
||||
# run inspector tool
|
||||
run-inspector:
|
||||
uv run mcp dev src/basic_memory/mcp/main.py
|
||||
|
||||
# Build app installer
|
||||
installer-mac:
|
||||
cd installer && chmod +x make_icons.sh && ./make_icons.sh
|
||||
cd installer && uv run python setup.py bdist_mac
|
||||
|
||||
installer-win:
|
||||
cd installer && uv run python setup.py bdist_win32
|
||||
|
||||
|
||||
update-deps:
|
||||
uv lock --upgrade
|
||||
|
||||
check: lint format type-check test
|
||||
|
||||
|
||||
# Target for generating Alembic migrations with a message from command line
|
||||
migration:
|
||||
@if [ -z "$(m)" ]; then \
|
||||
echo "Usage: make migration m=\"Your migration message\""; \
|
||||
exit 1; \
|
||||
fi; \
|
||||
cd src/basic_memory/alembic && alembic revision --autogenerate -m "$(m)"
|
||||
@@ -13,8 +13,11 @@ Basic Memory lets you build persistent knowledge through natural conversations w
|
||||
Claude, while keeping everything in simple Markdown files on your computer. It uses the Model Context Protocol (MCP) to
|
||||
enable any compatible LLM to read and write to your local knowledge base.
|
||||
|
||||
- Website: https://basicmachines.co
|
||||
- Website: https://basicmemory.com
|
||||
- Company: https://basicmachines.co
|
||||
- Documentation: https://memory.basicmachines.co
|
||||
- Discord: https://discord.gg/tyvKNccgqN
|
||||
- YouTube: https://www.youtube.com/@basicmachines-co
|
||||
|
||||
## Pick up your conversation right where you left off
|
||||
|
||||
@@ -61,8 +64,7 @@ Memory for Claude Desktop:
|
||||
npx -y @smithery/cli install @basicmachines-co/basic-memory --client claude
|
||||
```
|
||||
|
||||
This installs and configures Basic Memory without requiring manual edits to the Claude Desktop configuration file. The
|
||||
Smithery server hosts the MCP server component, while your data remains stored locally as Markdown files.
|
||||
This installs and configures Basic Memory without requiring manual edits to the Claude Desktop configuration file. Note: The Smithery installation uses their hosted MCP server, while your data remains stored locally as Markdown files.
|
||||
|
||||
### Glama.ai
|
||||
|
||||
@@ -153,7 +155,8 @@ The note embeds semantic content and links to other topics via simple Markdown f
|
||||
|
||||
3. You see this file on your computer in real time in the current project directory (default `~/$HOME/basic-memory`).
|
||||
|
||||
- Realtime sync is enabled by default with the v0.12.0 version
|
||||
- Realtime sync is enabled by default starting with v0.12.0
|
||||
- Project switching during conversations is supported starting with v0.13.0
|
||||
|
||||
4. In a chat with the LLM, you can reference a topic:
|
||||
|
||||
@@ -333,9 +336,9 @@ config:
|
||||
"command": "uvx",
|
||||
"args": [
|
||||
"basic-memory",
|
||||
"mcp",
|
||||
"--project",
|
||||
"your-project-name"
|
||||
"your-project-name",
|
||||
"mcp"
|
||||
]
|
||||
}
|
||||
}
|
||||
@@ -351,10 +354,20 @@ Basic Memory will sync the files in your project in real time if you make manual
|
||||
```
|
||||
write_note(title, content, folder, tags) - Create or update notes
|
||||
read_note(identifier, page, page_size) - Read notes by title or permalink
|
||||
edit_note(identifier, operation, content) - Edit notes incrementally (append, prepend, find/replace)
|
||||
move_note(identifier, destination_path) - Move notes with database consistency
|
||||
view_note(identifier) - Display notes as formatted artifacts for better readability
|
||||
build_context(url, depth, timeframe) - Navigate knowledge graph via memory:// URLs
|
||||
search_notes(query, page, page_size) - Search across your knowledge base
|
||||
recent_activity(type, depth, timeframe) - Find recently updated information
|
||||
canvas(nodes, edges, title, folder) - Generate knowledge visualizations
|
||||
list_memory_projects() - List all available projects with status
|
||||
switch_project(project_name) - Switch to different project context
|
||||
get_current_project() - Show current project and statistics
|
||||
create_memory_project(name, path, set_default) - Create new projects
|
||||
delete_project(name) - Delete projects from configuration
|
||||
set_default_project(name) - Set default project
|
||||
sync_status() - Check file synchronization status
|
||||
```
|
||||
|
||||
5. Example prompts to try:
|
||||
@@ -365,6 +378,10 @@ canvas(nodes, edges, title, folder) - Generate knowledge visualizations
|
||||
"Create a canvas visualization of my project components"
|
||||
"Read my notes on the authentication system"
|
||||
"What have I been working on in the past week?"
|
||||
"Switch to my work-notes project"
|
||||
"List all my available projects"
|
||||
"Edit my coffee brewing note to add a new technique"
|
||||
"Move my old meeting notes to the archive folder"
|
||||
```
|
||||
|
||||
## Futher info
|
||||
@@ -376,6 +393,24 @@ See the [Documentation](https://memory.basicmachines.co/) for more info, includi
|
||||
- [Managing multiple Projects](https://memory.basicmachines.co/docs/cli-reference#project)
|
||||
- [Importing data from OpenAI/Claude Projects](https://memory.basicmachines.co/docs/cli-reference#import)
|
||||
|
||||
## Installation Options
|
||||
|
||||
### Stable Release
|
||||
```bash
|
||||
pip install basic-memory
|
||||
```
|
||||
|
||||
### Beta/Pre-releases
|
||||
```bash
|
||||
pip install basic-memory --pre
|
||||
```
|
||||
|
||||
### Development Builds
|
||||
Development versions are automatically published on every commit to main with versions like `0.12.4.dev26+468a22f`:
|
||||
```bash
|
||||
pip install basic-memory --pre --force-reinstall
|
||||
```
|
||||
|
||||
## License
|
||||
|
||||
AGPL-3.0
|
||||
|
||||
@@ -1,13 +0,0 @@
|
||||
Starting issue-fix mode at Sat Apr 5 18:02:54 UTC 2025
|
||||
Fetching issue #75 details
|
||||
Using repository: basicmachines-co/basic-memory
|
||||
Checking if phernandez is a member of organization basicmachines-co
|
||||
User phernandez is a member of organization basicmachines-co. Proceeding with Claude fix.
|
||||
Creating a new branch: fix-issue-75-20250405180254
|
||||
From https://github.com/basicmachines-co/basic-memory
|
||||
* branch main -> FETCH_HEAD
|
||||
Switched to a new branch 'fix-issue-75-20250405180254'
|
||||
branch 'fix-issue-75-20250405180254' set up to track 'origin/main'.
|
||||
Prompt saved to ./claude-output/claude_prompt_75.txt for debugging
|
||||
Running Claude to fix the issue...
|
||||
Committing changes...
|
||||
@@ -1,35 +0,0 @@
|
||||
Let's summarize the changes we've made to fix issue #75:
|
||||
|
||||
1. We updated the `search_notes` tool in `/src/basic_memory/mcp/tools/search.py` to accept primitive types as parameters instead of a complex Pydantic `SearchQuery` object. This makes it easier for LLMs like Cursor to make proper tool calls.
|
||||
|
||||
2. We converted the internal implementation to create a SearchQuery object from the primitive parameters, maintaining backward compatibility with the existing API.
|
||||
|
||||
3. We updated tests in `/tests/mcp/test_tool_search.py` to use the new function signature with primitive parameters.
|
||||
|
||||
4. We updated code in `/src/basic_memory/mcp/tools/read_note.py` to use the new function signature when making calls to `search_notes`.
|
||||
|
||||
5. We updated code in `/src/basic_memory/mcp/prompts/search.py` to use the new function signature when making calls to `search_notes`.
|
||||
|
||||
These changes should make it easier for Cursor and other LLMs to use the search_notes tool by eliminating the complex Pydantic object parameter in favor of simple primitive parameters.
|
||||
|
||||
---SUMMARY---
|
||||
Fixed issue #75 where Cursor was having errors calling the search_notes tool. The problem was that the search_notes tool was expecting a complex Pydantic object (SearchQuery) as input, which was confusing Cursor.
|
||||
|
||||
Changes:
|
||||
1. Modified the search_notes tool to accept primitive types (strings, lists, etc.) as parameters instead of a complex Pydantic object
|
||||
2. Updated the implementation to create a SearchQuery object internally from these primitive parameters
|
||||
3. Updated all call sites in the codebase that were using the old function signature
|
||||
4. Updated tests to use the new function signature
|
||||
|
||||
The fix makes it easier for LLMs like Cursor to make proper calls to the search_notes tool, which will resolve the reported error messages:
|
||||
- "Parameter 'query' must be of type undefined, got object"
|
||||
- "Parameter 'query' must be of type undefined, got string"
|
||||
- "Invalid type for parameter 'query' in tool search_notes"
|
||||
|
||||
Files modified:
|
||||
- src/basic_memory/mcp/tools/search.py
|
||||
- src/basic_memory/mcp/tools/read_note.py
|
||||
- src/basic_memory/mcp/prompts/search.py
|
||||
- tests/mcp/test_tool_search.py
|
||||
- tests/mcp/test_tool_read_note.py
|
||||
---END SUMMARY---
|
||||
@@ -1,49 +0,0 @@
|
||||
You are Claude, an AI assistant tasked with fixing issues in a GitHub repository.
|
||||
|
||||
Issue #75: [BUG] Cursor has errors calling search tool
|
||||
|
||||
Issue Description:
|
||||
## Bug Description
|
||||
|
||||
|
||||
|
||||
> Cursor cannot figure out how to structure the parameters for that tool call. No matter what Cursor seems to try it gets the errors.
|
||||
>
|
||||
> ```Looking at the error messages more carefully:
|
||||
> - When I pass an object: "Parameter 'query' must be of type undefined, got object"
|
||||
> - When I pass a string: "Parameter 'query' must be of type undefined, got string"
|
||||
>
|
||||
>
|
||||
>
|
||||
> and then it reports: "Invalid type for parameter 'query' in tool search_notes"
|
||||
> Any chance you can give me some guidance with this?
|
||||
>
|
||||
|
||||
## Steps To Reproduce
|
||||
Steps to reproduce the behavior:
|
||||
|
||||
try using search tool in Cursor.
|
||||
|
||||
## Possible Solution
|
||||
|
||||
The tool args should probably be plain text and not json to make it easier to call.
|
||||
Additional Instructions from User Comment:
|
||||
let make a PR to implement option #1.
|
||||
Your task is to:
|
||||
1. Analyze the issue carefully to understand the problem
|
||||
2. Look through the repository to identify the relevant files that need to be modified
|
||||
3. Make precise changes to fix the issue
|
||||
4. Use the Edit tool to modify files directly when needed
|
||||
5. Be minimal in your changes - only modify what's necessary to fix the issue
|
||||
|
||||
After making changes, provide a summary of what you did in this format:
|
||||
|
||||
---SUMMARY---
|
||||
[Your detailed summary of changes, including which files were modified and how]
|
||||
---END SUMMARY---
|
||||
|
||||
Remember:
|
||||
- Be specific in your changes
|
||||
- Only modify files that are necessary to fix the issue
|
||||
- Follow existing code style and conventions
|
||||
- Make the minimal changes needed to resolve the issue
|
||||
+178
-145
@@ -10,6 +10,26 @@ You can [download](https://github.com/basicmachines-co/basic-memory/blob/main/do
|
||||
|
||||
This guide helps you, the AI assistant, use Basic Memory tools effectively when working with users. It covers reading, writing, and navigating knowledge through the Model Context Protocol (MCP).
|
||||
|
||||
## Quick Reference
|
||||
|
||||
**Essential Tools:**
|
||||
- `write_note()` - Create/update notes (primary tool)
|
||||
- `read_note()` - Read existing content
|
||||
- `search_notes()` - Find information
|
||||
- `edit_note()` - Modify existing notes incrementally (v0.13.0)
|
||||
- `move_note()` - Organize files with database consistency (v0.13.0)
|
||||
|
||||
**Project Management (v0.13.0):**
|
||||
- `list_projects()` - Show available projects
|
||||
- `switch_project()` - Change active project
|
||||
- `get_current_project()` - Current project info
|
||||
|
||||
**Key Principles:**
|
||||
1. **Build connections** - Rich knowledge graphs > isolated notes
|
||||
2. **Ask permission** - "Would you like me to record this?"
|
||||
3. **Use exact titles** - For accurate `[[WikiLinks]]`
|
||||
4. **Leverage v0.13.0** - Edit incrementally, organize proactively, switch projects contextually
|
||||
|
||||
## Overview
|
||||
|
||||
Basic Memory allows you and users to record context in local Markdown files, building a rich knowledge base through natural conversations. The system automatically creates a semantic knowledge graph from simple text patterns.
|
||||
@@ -37,50 +57,67 @@ Remember that a knowledge graph with 10 heavily connected notes is more valuable
|
||||
|
||||
## Core Tools Reference
|
||||
|
||||
```python
|
||||
# Writing knowledge - THE MOST IMPORTANT TOOL!
|
||||
response = await write_note(
|
||||
title="Search Design", # Required: Note title
|
||||
content="# Search Design\n...", # Required: Note content
|
||||
folder="specs", # Optional: Folder to save in
|
||||
tags=["search", "design"], # Optional: Tags for categorization
|
||||
verbose=True # Optional: Get parsing details
|
||||
)
|
||||
### Essential Content Management
|
||||
|
||||
# Reading knowledge
|
||||
content = await read_note("Search Design") # By title
|
||||
content = await read_note("specs/search-design") # By path
|
||||
content = await read_note("memory://specs/search") # By memory URL
|
||||
|
||||
# Searching for knowledge
|
||||
results = await search_notes(
|
||||
query="authentication system", # Text to search for
|
||||
page=1, # Optional: Pagination
|
||||
page_size=10 # Optional: Results per page
|
||||
**Writing knowledge** (most important tool):
|
||||
```
|
||||
write_note(
|
||||
title="Search Design",
|
||||
content="# Search Design\n...",
|
||||
folder="specs", # Optional
|
||||
tags=["search", "design"], # v0.13.0: now searchable!
|
||||
project="work-notes" # v0.13.0: target specific project
|
||||
)
|
||||
```
|
||||
|
||||
# Building context from the knowledge graph
|
||||
context = await build_context(
|
||||
url="memory://specs/search", # Starting point
|
||||
depth=2, # Optional: How many hops to follow
|
||||
timeframe="1 month" # Optional: Recent timeframe
|
||||
)
|
||||
**Reading knowledge:**
|
||||
```
|
||||
read_note("Search Design") # By title
|
||||
read_note("specs/search-design") # By path
|
||||
read_note("memory://specs/search") # By memory URL
|
||||
```
|
||||
|
||||
# Checking recent changes
|
||||
activity = await recent_activity(
|
||||
type="all", # Optional: Entity types to include
|
||||
depth=1, # Optional: Related items to include
|
||||
timeframe="1 week" # Optional: Time window
|
||||
)
|
||||
**Viewing notes as formatted artifacts (Claude Desktop):**
|
||||
```
|
||||
view_note("Search Design") # Creates readable artifact
|
||||
view_note("specs/search-design") # By permalink
|
||||
view_note("memory://specs/search") # By memory URL
|
||||
```
|
||||
|
||||
# Creating a knowledge visualization
|
||||
canvas_result = await canvas(
|
||||
nodes=[{"id": "note1", "label": "Search Design"}], # Nodes to display
|
||||
edges=[{"from": "note1", "to": "note2"}], # Connections
|
||||
title="Project Overview", # Canvas title
|
||||
folder="diagrams" # Storage location
|
||||
**Incremental editing** (v0.13.0):
|
||||
```
|
||||
edit_note(
|
||||
identifier="Search Design", # Must be EXACT title/permalink (strict matching)
|
||||
operation="append", # append, prepend, find_replace, replace_section
|
||||
content="\n## New Section\nContent here..."
|
||||
)
|
||||
```
|
||||
**⚠️ Important:** `edit_note` requires exact identifiers (no fuzzy matching). Use `search_notes()` first if uncertain.
|
||||
|
||||
**File organization** (v0.13.0):
|
||||
```
|
||||
move_note(
|
||||
identifier="Old Note", # Must be EXACT title/permalink (strict matching)
|
||||
destination="archive/old-note.md" # Folders created automatically
|
||||
)
|
||||
```
|
||||
**⚠️ Important:** `move_note` requires exact identifiers (no fuzzy matching). Use `search_notes()` first if uncertain.
|
||||
|
||||
### Project Management (v0.13.0)
|
||||
|
||||
```
|
||||
list_projects() # Show available projects
|
||||
switch_project("work-notes") # Change active project
|
||||
get_current_project() # Current project info
|
||||
```
|
||||
|
||||
### Search & Discovery
|
||||
|
||||
```
|
||||
search_notes("authentication system") # v0.13.0: includes frontmatter tags
|
||||
build_context("memory://specs/search") # Follow knowledge graph connections
|
||||
recent_activity(timeframe="1 week") # Check what's been updated
|
||||
```
|
||||
|
||||
## memory:// URLs Explained
|
||||
|
||||
@@ -158,6 +195,30 @@ Users will interact with Basic Memory in patterns like:
|
||||
[Then build_context() to understand connections]
|
||||
```
|
||||
|
||||
4. **Editing existing notes (v0.13.0)**:
|
||||
```
|
||||
Human: "Add a section about deployment to my API documentation"
|
||||
|
||||
You: I'll add that section to your existing documentation.
|
||||
[Use edit_note() with operation="append" to add new content]
|
||||
```
|
||||
|
||||
5. **Project management (v0.13.0)**:
|
||||
```
|
||||
Human: "Switch to my work project and show recent activity"
|
||||
|
||||
You: I'll switch to your work project and check what's been updated recently.
|
||||
[Use switch_project() then recent_activity()]
|
||||
```
|
||||
|
||||
6. **File organization (v0.13.0)**:
|
||||
```
|
||||
Human: "Move my old meeting notes to the archive folder"
|
||||
|
||||
You: I'll organize those notes for you.
|
||||
[Use move_note() to relocate files with database consistency]
|
||||
```
|
||||
|
||||
## Key Things to Remember
|
||||
|
||||
1. **Files are Truth**
|
||||
@@ -174,16 +235,27 @@ Users will interact with Basic Memory in patterns like:
|
||||
- Combine related information
|
||||
|
||||
3. **Writing Knowledge Wisely**
|
||||
- Using the same title+folder will overwrite existing notes
|
||||
- Structure content with clear headings and sections
|
||||
- Use semantic markup for observations and relations
|
||||
- Same title+folder overwrites existing notes
|
||||
- Structure with clear headings and semantic markup
|
||||
- Use tags for searchability (v0.13.0: frontmatter tags indexed)
|
||||
- Keep files organized in logical folders
|
||||
|
||||
4. **Leverage v0.13.0 Features**
|
||||
- **Edit incrementally**: Use `edit_note()` for small changes vs rewriting
|
||||
- **Switch projects**: Change context when user mentions different work areas
|
||||
- **Organize proactively**: Move old content to archive folders
|
||||
- **Cross-project operations**: Create notes in specific projects while maintaining context
|
||||
|
||||
## Common Knowledge Patterns
|
||||
|
||||
### Capturing Decisions
|
||||
|
||||
```markdown
|
||||
---
|
||||
title: Coffee Brewing Methods
|
||||
tags: [coffee, brewing, pour-over, techniques] # v0.13.0: Now searchable!
|
||||
---
|
||||
|
||||
# Coffee Brewing Methods
|
||||
|
||||
## Context
|
||||
@@ -196,11 +268,13 @@ Pour over is my preferred method for light to medium roasts because it highlight
|
||||
- [technique] Blooming the coffee grounds for 30 seconds improves extraction #brewing
|
||||
- [preference] Water temperature between 195-205°F works best #temperature
|
||||
- [equipment] Gooseneck kettle provides better control of water flow #tools
|
||||
- [timing] Total brew time of 3-4 minutes produces optimal extraction #process
|
||||
|
||||
## Relations
|
||||
- pairs_with [[Light Roast Beans]]
|
||||
- contrasts_with [[French Press Method]]
|
||||
- requires [[Proper Grinding Technique]]
|
||||
- part_of [[Morning Coffee Routine]]
|
||||
```
|
||||
|
||||
### Recording Project Structure
|
||||
@@ -241,119 +315,77 @@ Discussed strategies for improving the chocolate chip cookie recipe.
|
||||
- pairs_with [[Homemade Ice Cream]]
|
||||
```
|
||||
|
||||
## v0.13.0 Workflow Examples
|
||||
|
||||
### Multi-Project Conversations
|
||||
|
||||
**User:** "I need to update my work documentation and also add a personal recipe note."
|
||||
|
||||
**Workflow:**
|
||||
1. `list_projects()` - Check available projects
|
||||
2. `write_note(title="Sprint Planning", project="work-notes")` - Work content
|
||||
3. `write_note(title="Weekend Recipes", project="personal")` - Personal content
|
||||
|
||||
### Incremental Note Building
|
||||
|
||||
**User:** "Add a troubleshooting section to my setup guide."
|
||||
|
||||
**Workflow:**
|
||||
1. `edit_note(identifier="Setup Guide", operation="append", content="\n## Troubleshooting\n...")`
|
||||
|
||||
**User:** "Update the authentication section in my API docs."
|
||||
|
||||
**Workflow:**
|
||||
1. `edit_note(identifier="API Documentation", operation="replace_section", section="## Authentication")`
|
||||
|
||||
### Smart File Organization
|
||||
|
||||
**User:** "My notes are getting messy in the main folder."
|
||||
|
||||
**Workflow:**
|
||||
1. `move_note("Old Meeting Notes", "archive/2024/old-meetings.md")`
|
||||
2. `move_note("Project Notes", "projects/client-work/notes.md")`
|
||||
|
||||
### Creating Effective Relations
|
||||
|
||||
When creating relations, you can:
|
||||
1. Reference existing entities by their exact title
|
||||
2. Create forward references to entities that don't exist yet
|
||||
When creating relations:
|
||||
1. **Reference existing entities** by their exact title: `[[Exact Title]]`
|
||||
2. **Create forward references** to entities that don't exist yet - they'll be linked automatically when created
|
||||
3. **Search first** to find existing entities to reference
|
||||
4. **Use meaningful relation types**: `implements`, `requires`, `part_of` vs generic `relates_to`
|
||||
|
||||
```python
|
||||
# Example workflow for creating notes with effective relations
|
||||
async def create_note_with_effective_relations():
|
||||
# Search for existing entities to reference
|
||||
search_results = await search_notes("travel")
|
||||
existing_entities = [result.title for result in search_results.primary_results]
|
||||
|
||||
# Check if specific entities exist
|
||||
packing_tips_exists = "Packing Tips" in existing_entities
|
||||
japan_travel_exists = "Japan Travel Guide" in existing_entities
|
||||
|
||||
# Prepare relations section - include both existing and forward references
|
||||
relations_section = "## Relations\n"
|
||||
|
||||
# Existing reference - exact match to known entity
|
||||
if packing_tips_exists:
|
||||
relations_section += "- references [[Packing Tips]]\n"
|
||||
else:
|
||||
# Forward reference - will be linked when that entity is created later
|
||||
relations_section += "- references [[Packing Tips]]\n"
|
||||
|
||||
# Another possible reference
|
||||
if japan_travel_exists:
|
||||
relations_section += "- part_of [[Japan Travel Guide]]\n"
|
||||
|
||||
# You can also check recently modified notes to reference them
|
||||
recent = await recent_activity(timeframe="1 week")
|
||||
recent_titles = [item.title for item in recent.primary_results]
|
||||
|
||||
if "Transportation Options" in recent_titles:
|
||||
relations_section += "- relates_to [[Transportation Options]]\n"
|
||||
|
||||
# Always include meaningful forward references, even if they don't exist yet
|
||||
relations_section += "- located_in [[Tokyo]]\n"
|
||||
relations_section += "- visited_during [[Spring 2023 Trip]]\n"
|
||||
|
||||
# Now create the note with both verified and forward relations
|
||||
content = f"""# Tokyo Neighborhood Guide
|
||||
|
||||
## Overview
|
||||
Details about different Tokyo neighborhoods and their unique characteristics.
|
||||
**Example workflow:**
|
||||
1. `search_notes("travel")` to find existing travel-related notes
|
||||
2. Reference found entities: `- part_of [[Japan Travel Guide]]`
|
||||
3. Add forward references: `- located_in [[Tokyo]]` (even if Tokyo note doesn't exist yet)
|
||||
|
||||
## Observations
|
||||
- [area] Shibuya is a busy shopping district #shopping
|
||||
- [transportation] Yamanote Line connects major neighborhoods #transit
|
||||
- [recommendation] Visit Shimokitazawa for vintage shopping #unique
|
||||
- [tip] Get a Suica card for easy train travel #convenience
|
||||
## Common Issues & Solutions
|
||||
|
||||
{relations_section}
|
||||
"""
|
||||
|
||||
result = await write_note(
|
||||
title="Tokyo Neighborhood Guide",
|
||||
content=content,
|
||||
verbose=True
|
||||
)
|
||||
|
||||
# You can check which relations were resolved and which are forward references
|
||||
if result and 'relations' in result:
|
||||
resolved = [r['to_name'] for r in result['relations'] if r.get('target_id')]
|
||||
forward_refs = [r['to_name'] for r in result['relations'] if not r.get('target_id')]
|
||||
|
||||
print(f"Resolved relations: {resolved}")
|
||||
print(f"Forward references that will be resolved later: {forward_refs}")
|
||||
```
|
||||
**Missing Content:**
|
||||
- Try `search_notes()` with broader terms if `read_note()` fails
|
||||
- Use fuzzy matching: search for partial titles
|
||||
|
||||
## Error Handling
|
||||
**Forward References:**
|
||||
- These are normal! Basic Memory links them automatically when target notes are created
|
||||
- Inform users: "I've created forward references that will be linked when you create those notes"
|
||||
|
||||
Common issues to watch for:
|
||||
**Sync Issues:**
|
||||
- If information seems outdated, suggest `basic-memory sync`
|
||||
- Use `recent_activity()` to check if content is current
|
||||
|
||||
1. **Missing Content**
|
||||
```python
|
||||
try:
|
||||
content = await read_note("Document")
|
||||
except:
|
||||
# Try search instead
|
||||
results = await search_notes("Document")
|
||||
if results and results.primary_results:
|
||||
# Found something similar
|
||||
content = await read_note(results.primary_results[0].permalink)
|
||||
```
|
||||
|
||||
2. **Forward References (Unresolved Relations)**
|
||||
```python
|
||||
response = await write_note(..., verbose=True)
|
||||
# Check for forward references (unresolved relations)
|
||||
forward_refs = []
|
||||
for relation in response.get('relations', []):
|
||||
if not relation.get('target_id'):
|
||||
forward_refs.append(relation.get('to_name'))
|
||||
|
||||
if forward_refs:
|
||||
# This is a feature, not an error! Inform the user about forward references
|
||||
print(f"Note created with forward references to: {forward_refs}")
|
||||
print("These will be automatically linked when those notes are created.")
|
||||
|
||||
# Optionally suggest creating those entities now
|
||||
print("Would you like me to create any of these notes now to complete the connections?")
|
||||
```
|
||||
|
||||
3. **Sync Issues**
|
||||
```python
|
||||
# If information seems outdated
|
||||
activity = await recent_activity(timeframe="1 hour")
|
||||
if not activity or not activity.primary_results:
|
||||
print("It seems there haven't been recent updates. You might need to run 'basic-memory sync'.")
|
||||
```
|
||||
**Strict Mode for Edit/Move Operations:**
|
||||
- `edit_note()` and `move_note()` require **exact identifiers** (no fuzzy matching for safety)
|
||||
- If identifier not found: use `search_notes()` first to find the exact title/permalink
|
||||
- Error messages will guide you to find correct identifiers
|
||||
- Example workflow:
|
||||
```
|
||||
# ❌ This might fail if identifier isn't exact
|
||||
edit_note("Meeting Note", "append", "content")
|
||||
|
||||
# ✅ Safe approach: search first, then use exact result
|
||||
results = search_notes("meeting")
|
||||
edit_note("Meeting Notes 2024", "append", "content") # Use exact title from search
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
@@ -395,4 +427,5 @@ Common issues to watch for:
|
||||
- Offer to create summaries of scattered information
|
||||
- Suggest potential missing relations: "I notice this might relate to [topic], would you like me to add that connection?"
|
||||
|
||||
|
||||
Built with ♥️ by Basic Machines
|
||||
|
||||
+101
-19
@@ -10,6 +10,25 @@ Basic Memory provides command line tools for managing your knowledge base. This
|
||||
|
||||
## Core Commands
|
||||
|
||||
### auth (New in v0.13.0)
|
||||
|
||||
Manage OAuth authentication for secure remote access:
|
||||
|
||||
```bash
|
||||
# Test authentication setup
|
||||
basic-memory auth test-auth
|
||||
|
||||
# Register OAuth client
|
||||
basic-memory auth register-client
|
||||
```
|
||||
|
||||
Supports multiple authentication providers:
|
||||
- **Basic Provider**: For development and testing
|
||||
- **Supabase Provider**: For production deployments
|
||||
- **External Providers**: GitHub, Google integration framework
|
||||
|
||||
See [[OAuth Authentication Guide]] for complete setup instructions.
|
||||
|
||||
### sync
|
||||
|
||||
Keeps files and the knowledge graph in sync:
|
||||
@@ -45,9 +64,9 @@ To change the properties, set the following values:
|
||||
```
|
||||
|
||||
Thanks for using Basic Memory!
|
||||
### import
|
||||
### import (Enhanced in v0.13.0)
|
||||
|
||||
Imports external knowledge sources:
|
||||
Imports external knowledge sources with support for project targeting:
|
||||
|
||||
```bash
|
||||
# Claude conversations
|
||||
@@ -59,12 +78,19 @@ basic-memory import claude projects
|
||||
# ChatGPT history
|
||||
basic-memory import chatgpt
|
||||
|
||||
# ChatGPT history
|
||||
# Memory JSON format
|
||||
basic-memory import memory-json /path/to/memory.json
|
||||
|
||||
# Import to specific project (v0.13.0)
|
||||
basic-memory --project=work import claude conversations
|
||||
```
|
||||
|
||||
> **Note**: After importing, run `basic-memory sync` to index the new files.
|
||||
**New in v0.13.0:**
|
||||
- **Project Targeting**: Import directly to specific projects
|
||||
- **Real-time Sync**: Imported content available immediately
|
||||
- **Unified Database**: All imports stored in centralized database
|
||||
|
||||
> **Note**: Changes sync automatically - no manual sync required in v0.13.0.
|
||||
### status
|
||||
|
||||
Shows system status information:
|
||||
@@ -81,28 +107,32 @@ basic-memory status --json
|
||||
```
|
||||
|
||||
|
||||
### project
|
||||
### project (Enhanced in v0.13.0)
|
||||
|
||||
Create multiple projects to manage your knowledge.
|
||||
Manage multiple projects with the new unified database architecture. Projects can now be switched instantly during conversations without restart.
|
||||
|
||||
```bash
|
||||
# List all configured projects
|
||||
# List all configured projects with status
|
||||
basic-memory project list
|
||||
|
||||
# Add a new project
|
||||
basic-memory project add work ~/work-basic-memory
|
||||
# Create a new project
|
||||
basic-memory project create work ~/work-basic-memory
|
||||
|
||||
# Set the default project
|
||||
basic-memory project default work
|
||||
basic-memory project set-default work
|
||||
|
||||
# Remove a project (doesn't delete files)
|
||||
basic-memory project remove personal
|
||||
# Delete a project (doesn't delete files)
|
||||
basic-memory project delete personal
|
||||
|
||||
# Show current project
|
||||
basic-memory project current
|
||||
# Show detailed project statistics
|
||||
basic-memory project info
|
||||
```
|
||||
|
||||
> Be sure to restart Claude Desktop after changing projects.
|
||||
**New in v0.13.0:**
|
||||
- **Unified Database**: All projects share a single database for better performance
|
||||
- **Instant Switching**: Switch projects during conversations without restart
|
||||
- **Enhanced Commands**: Updated project commands with better status information
|
||||
- **Project Statistics**: Detailed info about entities, observations, and relations
|
||||
|
||||
#### Using Projects in Commands
|
||||
|
||||
@@ -122,6 +152,34 @@ You can also set the `BASIC_MEMORY_PROJECT` environment variable:
|
||||
BASIC_MEMORY_PROJECT=work basic-memory sync
|
||||
```
|
||||
|
||||
### tool (Enhanced in v0.13.0)
|
||||
|
||||
Direct access to MCP tools via CLI with new editing and file management capabilities:
|
||||
|
||||
```bash
|
||||
# Create notes
|
||||
basic-memory tool write-note --title "My Note" --content "Content here"
|
||||
|
||||
# Edit notes incrementally (v0.13.0)
|
||||
echo "New content" | basic-memory tool edit-note --title "My Note" --operation append
|
||||
|
||||
# Move notes (v0.13.0)
|
||||
basic-memory tool move-note --identifier "My Note" --destination "archive/my-note.md"
|
||||
|
||||
# Search notes
|
||||
basic-memory tool search-notes --query "authentication"
|
||||
|
||||
# Project management (v0.13.0)
|
||||
basic-memory tool list-projects
|
||||
basic-memory tool switch-project --project-name "work"
|
||||
```
|
||||
|
||||
**New in v0.13.0:**
|
||||
- **edit-note**: Incremental editing (append, prepend, find/replace, section replace)
|
||||
- **move-note**: File management with database consistency
|
||||
- **Project tools**: list-projects, switch-project, get-current-project
|
||||
- **Cross-project operations**: Use `--project` flag with any tool
|
||||
|
||||
### help
|
||||
|
||||
The full list of commands and help for each can be viewed with the `--help` argument.
|
||||
@@ -144,10 +202,11 @@ The full list of commands and help for each can be viewed with the `--help` argu
|
||||
│ --help Show this message and exit. │
|
||||
╰───────────────────────────────────────────────────────────────────────────────────────────╯
|
||||
╭─ Commands ────────────────────────────────────────────────────────────────────────────────╮
|
||||
│ sync Sync knowledge files with the database. │
|
||||
│ status Show sync status between files and database. │
|
||||
│ reset Reset database (drop all tables and recreate). │
|
||||
│ mcp Run the MCP server for Claude Desktop integration. │
|
||||
│ auth OAuth authentication management (v0.13.0) │
|
||||
│ sync Sync knowledge files with the database │
|
||||
│ status Show sync status between files and database │
|
||||
│ reset Reset database (drop all tables and recreate) │
|
||||
│ mcp Run the MCP server for Claude Desktop integration │
|
||||
│ import Import data from various sources │
|
||||
│ tool Direct access to MCP tools via CLI │
|
||||
│ project Manage multiple Basic Memory projects │
|
||||
@@ -302,6 +361,29 @@ You can then use the `/mcp` command in the REPL:
|
||||
• basic-memory: connected
|
||||
```
|
||||
|
||||
## Version Management (New in v0.13.0)
|
||||
|
||||
Basic Memory v0.13.0 introduces automatic version management and multiple installation options:
|
||||
|
||||
```bash
|
||||
# Stable releases
|
||||
pip install basic-memory
|
||||
|
||||
# Beta/pre-releases
|
||||
pip install basic-memory --pre
|
||||
|
||||
# Latest development builds (auto-published)
|
||||
pip install basic-memory --pre --force-reinstall
|
||||
|
||||
# Check current version
|
||||
basic-memory --version
|
||||
```
|
||||
|
||||
**Version Types:**
|
||||
- **Stable**: `0.13.0` (manual git tags)
|
||||
- **Beta**: `0.13.0b1` (manual git tags)
|
||||
- **Development**: `0.12.4.dev26+468a22f` (automatic from commits)
|
||||
|
||||
## Troubleshooting Common Issues
|
||||
|
||||
### Sync Conflicts
|
||||
|
||||
@@ -0,0 +1,335 @@
|
||||
# Claude.ai Integration Guide
|
||||
|
||||
This guide explains how to connect Basic Memory to Claude.ai, enabling Claude to read and write to your personal knowledge base.
|
||||
|
||||
## Overview
|
||||
|
||||
When connected to Claude.ai, Basic Memory provides:
|
||||
- Persistent memory across conversations
|
||||
- Knowledge graph navigation
|
||||
- Note-taking and search capabilities
|
||||
- File organization and management
|
||||
|
||||
## Prerequisites
|
||||
|
||||
1. Basic Memory MCP server with OAuth enabled
|
||||
2. Public HTTPS URL (or tunneling service for testing)
|
||||
3. Claude.ai account (Free, Pro, or Enterprise)
|
||||
|
||||
## Quick Start (Testing)
|
||||
|
||||
### 1. Start MCP Server with OAuth
|
||||
|
||||
```bash
|
||||
# Enable OAuth with basic provider
|
||||
export FASTMCP_AUTH_ENABLED=true
|
||||
export FASTMCP_AUTH_PROVIDER=basic
|
||||
|
||||
# Start server on all interfaces
|
||||
basic-memory mcp --transport streamable-http --host 0.0.0.0 --port 8000
|
||||
```
|
||||
|
||||
### 2. Make Server Accessible
|
||||
|
||||
For testing, use ngrok:
|
||||
|
||||
```bash
|
||||
# Install ngrok
|
||||
brew install ngrok # macOS
|
||||
# or download from https://ngrok.com
|
||||
|
||||
# Create tunnel
|
||||
ngrok http 8000
|
||||
```
|
||||
|
||||
Note the HTTPS URL (e.g., `https://abc123.ngrok.io`)
|
||||
|
||||
### 3. Register OAuth Client
|
||||
|
||||
```bash
|
||||
# Register a client for Claude
|
||||
basic-memory auth register-client --client-id claude-ai
|
||||
|
||||
# Save the credentials!
|
||||
# Client ID: claude-ai
|
||||
# Client Secret: xxx...
|
||||
```
|
||||
|
||||
### 4. Connect in Claude.ai
|
||||
|
||||
1. Go to Claude.ai → Settings → Integrations
|
||||
2. Click "Add More"
|
||||
3. Enter your server URL: `https://abc123.ngrok.io/mcp`
|
||||
4. Click "Connect"
|
||||
5. Authorize the connection
|
||||
|
||||
### 5. Use in Conversations
|
||||
|
||||
- Click the tools icon (🔧) in the chat
|
||||
- Select "Basic Memory"
|
||||
- Try commands like:
|
||||
- "Create a note about our meeting"
|
||||
- "Search for project ideas"
|
||||
- "Show recent notes"
|
||||
|
||||
## Production Setup
|
||||
|
||||
### 1. Deploy with Supabase Auth
|
||||
|
||||
```bash
|
||||
# .env file
|
||||
FASTMCP_AUTH_ENABLED=true
|
||||
FASTMCP_AUTH_PROVIDER=supabase
|
||||
SUPABASE_URL=https://your-project.supabase.co
|
||||
SUPABASE_ANON_KEY=your-anon-key
|
||||
SUPABASE_SERVICE_KEY=your-service-key
|
||||
```
|
||||
|
||||
### 2. Deploy to Cloud
|
||||
|
||||
Options for deployment:
|
||||
|
||||
#### Vercel
|
||||
```json
|
||||
// vercel.json
|
||||
{
|
||||
"functions": {
|
||||
"api/mcp.py": {
|
||||
"runtime": "python3.9"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
#### Railway
|
||||
```bash
|
||||
# Install Railway CLI
|
||||
brew install railway
|
||||
|
||||
# Deploy
|
||||
railway init
|
||||
railway up
|
||||
```
|
||||
|
||||
#### Docker
|
||||
```dockerfile
|
||||
FROM python:3.12
|
||||
WORKDIR /app
|
||||
COPY . .
|
||||
RUN pip install -e .
|
||||
CMD ["basic-memory", "mcp", "--transport", "streamable-http"]
|
||||
```
|
||||
|
||||
### 3. Configure for Organization
|
||||
|
||||
For Claude.ai Enterprise:
|
||||
|
||||
1. **Admin Setup**:
|
||||
- Go to Organizational Settings
|
||||
- Navigate to Integrations
|
||||
- Add MCP server URL for all users
|
||||
- Configure allowed scopes
|
||||
|
||||
2. **User Permissions**:
|
||||
- Users connect individually
|
||||
- Each user has their own auth token
|
||||
- Scopes determine access level
|
||||
|
||||
## Security Best Practices
|
||||
|
||||
### 1. Use HTTPS
|
||||
- Required for OAuth
|
||||
- Encrypt all data in transit
|
||||
- Use proper SSL certificates
|
||||
|
||||
### 2. Implement Scopes
|
||||
```bash
|
||||
# Configure required scopes
|
||||
FASTMCP_AUTH_REQUIRED_SCOPES=read,write
|
||||
|
||||
# User-specific scopes
|
||||
read: Can search and read notes
|
||||
write: Can create and update notes
|
||||
admin: Can manage all data
|
||||
```
|
||||
|
||||
### 3. Token Security
|
||||
- Short-lived access tokens (1 hour)
|
||||
- Refresh token rotation
|
||||
- Secure token storage
|
||||
|
||||
### 4. Rate Limiting
|
||||
```python
|
||||
# In your MCP server
|
||||
from fastapi import HTTPException
|
||||
from slowapi import Limiter
|
||||
|
||||
limiter = Limiter(key_func=get_remote_address)
|
||||
|
||||
@app.get("/mcp")
|
||||
@limiter.limit("100/minute")
|
||||
async def mcp_endpoint():
|
||||
# Handle MCP requests
|
||||
```
|
||||
|
||||
## Advanced Features
|
||||
|
||||
### 1. Custom Tools
|
||||
|
||||
Create specialized tools for Claude:
|
||||
|
||||
```python
|
||||
@mcp.tool()
|
||||
async def analyze_notes(topic: str) -> str:
|
||||
"""Analyze all notes on a specific topic."""
|
||||
# Search and analyze implementation
|
||||
return analysis
|
||||
```
|
||||
|
||||
### 2. Context Preservation
|
||||
|
||||
Use memory:// URLs to maintain context:
|
||||
|
||||
```python
|
||||
@mcp.tool()
|
||||
async def continue_conversation(memory_url: str) -> str:
|
||||
"""Continue from a previous conversation."""
|
||||
context = await build_context(memory_url)
|
||||
return context
|
||||
```
|
||||
|
||||
### 3. Multi-User Support
|
||||
|
||||
With Supabase, each user has isolated data:
|
||||
|
||||
```sql
|
||||
-- Row-level security
|
||||
CREATE POLICY "Users see own notes"
|
||||
ON notes FOR SELECT
|
||||
USING (auth.uid() = user_id);
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Connection Issues
|
||||
|
||||
1. **"Failed to connect"**
|
||||
- Verify server is running
|
||||
- Check HTTPS is working
|
||||
- Confirm OAuth is enabled
|
||||
|
||||
2. **"Authorization failed"**
|
||||
- Check client credentials
|
||||
- Verify redirect URLs
|
||||
- Review OAuth logs
|
||||
|
||||
3. **"No tools available"**
|
||||
- Ensure MCP tools are registered
|
||||
- Check required scopes
|
||||
- Verify transport type
|
||||
|
||||
### Debug Mode
|
||||
|
||||
Enable detailed logging:
|
||||
|
||||
```bash
|
||||
# Server side
|
||||
export FASTMCP_LOG_LEVEL=DEBUG
|
||||
export LOGURU_LEVEL=DEBUG
|
||||
|
||||
# Check logs
|
||||
tail -f logs/mcp.log
|
||||
```
|
||||
|
||||
### Test Connection
|
||||
|
||||
```bash
|
||||
# Test OAuth flow
|
||||
curl https://your-server.com/mcp/.well-known/oauth-authorization-server
|
||||
|
||||
# Should return OAuth metadata
|
||||
{
|
||||
"issuer": "https://your-server.com",
|
||||
"authorization_endpoint": "https://your-server.com/auth/authorize",
|
||||
...
|
||||
}
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Regular Backups**
|
||||
- Export your knowledge base
|
||||
- Use version control
|
||||
- Multiple storage locations
|
||||
|
||||
2. **Access Control**
|
||||
- Principle of least privilege
|
||||
- Regular token rotation
|
||||
- Audit access logs
|
||||
|
||||
3. **Performance**
|
||||
- Index frequently searched fields
|
||||
- Optimize large knowledge bases
|
||||
- Use caching where appropriate
|
||||
|
||||
4. **User Experience**
|
||||
- Clear tool descriptions
|
||||
- Helpful error messages
|
||||
- Quick response times
|
||||
|
||||
## Examples
|
||||
|
||||
### Creating Notes
|
||||
|
||||
```
|
||||
User: Create a note about the meeting with the product team
|
||||
|
||||
Claude: I'll create a note about your meeting with the product team.
|
||||
|
||||
[Uses write_note tool]
|
||||
|
||||
Note created: "Meeting with Product Team - 2024-01-15"
|
||||
Location: Work/Meetings/
|
||||
|
||||
I've documented the meeting notes. The note includes the date, attendees, and key discussion points.
|
||||
```
|
||||
|
||||
### Searching Knowledge
|
||||
|
||||
```
|
||||
User: What did we discuss about the API redesign?
|
||||
|
||||
Claude: Let me search for information about the API redesign.
|
||||
|
||||
[Uses search_notes tool]
|
||||
|
||||
I found 3 relevant notes about the API redesign:
|
||||
|
||||
1. "API Redesign Proposal" (2024-01-10)
|
||||
- RESTful architecture
|
||||
- Version 2.0 specifications
|
||||
- Migration timeline
|
||||
|
||||
2. "Technical Review: API Changes" (2024-01-12)
|
||||
- Breaking changes documented
|
||||
- Backwards compatibility plan
|
||||
|
||||
3. "Meeting: API Implementation" (2024-01-14)
|
||||
- Team assignments
|
||||
- Q1 deliverables
|
||||
```
|
||||
|
||||
## Next Steps
|
||||
|
||||
1. Set up production deployment
|
||||
2. Configure organizational access
|
||||
3. Create custom tools for your workflow
|
||||
4. Implement advanced security features
|
||||
5. Monitor usage and performance
|
||||
|
||||
## Resources
|
||||
|
||||
- [Basic Memory Documentation](../README.md)
|
||||
- [OAuth Setup Guide](OAuth%20Authentication.md)
|
||||
- [MCP Specification](https://modelcontextprotocol.io)
|
||||
- [Claude.ai Help Center](https://support.anthropic.com)
|
||||
@@ -20,14 +20,25 @@ The easiest way to install basic memory is via `uv`. See the [uv installation gu
|
||||
|
||||
### 1. Install Basic Memory
|
||||
|
||||
```bash
|
||||
# Install with uv (recommended).
|
||||
uv tool install basic-memory
|
||||
**v0.13.0 offers multiple installation options:**
|
||||
|
||||
# Or with pip
|
||||
pip install basic-memory
|
||||
```bash
|
||||
# Stable release (recommended)
|
||||
uv tool install basic-memory
|
||||
# or: pip install basic-memory
|
||||
|
||||
# Beta releases (new features, testing)
|
||||
pip install basic-memory --pre
|
||||
|
||||
# Development builds (latest changes)
|
||||
pip install basic-memory --pre --force-reinstall
|
||||
```
|
||||
|
||||
**Version Information:**
|
||||
- **Stable**: Latest tested release (e.g., `0.13.0`)
|
||||
- **Beta**: Pre-release versions (e.g., `0.13.0b1`)
|
||||
- **Development**: Auto-published from git commits (e.g., `0.12.4.dev26+468a22f`)
|
||||
|
||||
> **Important**: You need to install Basic Memory using one of the commands above to use the command line tools.
|
||||
|
||||
Using `uv tool install` will install the basic-memory package in a standalone virtual environment. See the [UV docs](https://docs.astral.sh/uv/concepts/tools/) for more info.
|
||||
@@ -99,30 +110,49 @@ To disable realtime sync, you can update the config. See [[CLI Reference#sync]].
|
||||
To update Basic Memory when new versions are released:
|
||||
|
||||
```bash
|
||||
# Update with uv (recommended)
|
||||
# Update stable release
|
||||
uv tool upgrade basic-memory
|
||||
# or: pip install --upgrade basic-memory
|
||||
|
||||
# Or with pip
|
||||
pip install --upgrade basic-memory
|
||||
# Update to latest beta (v0.13.0)
|
||||
pip install --upgrade basic-memory --pre
|
||||
|
||||
# Get latest development build
|
||||
pip install --upgrade basic-memory --pre --force-reinstall
|
||||
```
|
||||
|
||||
> **Note**: After updating, you'll need to restart Claude Desktop and your sync process for changes to take effect.
|
||||
**v0.13.0 Update Benefits:**
|
||||
- **Fluid project switching** during conversations
|
||||
- **Advanced note editing** capabilities
|
||||
- **Smart file management** with move operations
|
||||
- **Enhanced search** with frontmatter tag support
|
||||
|
||||
### 5. Change the default project directory
|
||||
> **Note**: After updating, restart Claude Desktop for changes to take effect. No sync restart needed in v0.13.0.
|
||||
|
||||
By default, Basic Memory will create a project in the `basic-memory` folder in your home directory. You can change this via the `project` [[CLI Reference#project|cli command]].
|
||||
### 5. Multi-Project Setup (Enhanced in v0.13.0)
|
||||
|
||||
By default, Basic Memory creates a project in `~/basic-memory`. v0.13.0 introduces **fluid project management** - switch between projects instantly during conversations.
|
||||
|
||||
```
|
||||
# Add a new project
|
||||
basic-memory project add work ~/work-basic-memory
|
||||
# Create a new project
|
||||
basic-memory project create work ~/work-basic-memory
|
||||
|
||||
# Set the default project
|
||||
basic-memory project default work
|
||||
basic-memory project set-default work
|
||||
|
||||
# List all configured projects
|
||||
basic-memory project list
|
||||
# List all projects with status
|
||||
basic-memory project list
|
||||
|
||||
# Get detailed project information
|
||||
basic-memory project info
|
||||
```
|
||||
|
||||
**New in v0.13.0:**
|
||||
- **Instant switching**: Change projects during conversations without restart
|
||||
- **Unified database**: All projects in single `~/.basic-memory/memory.db`
|
||||
- **Better performance**: Optimized queries and reduced file I/O
|
||||
- **Session context**: Maintains active project throughout conversations
|
||||
|
||||
## Troubleshooting Installation
|
||||
|
||||
### Common Issues
|
||||
@@ -168,6 +198,7 @@ If you encounter permission errors:
|
||||
---
|
||||
title: Coffee Brewing Methods
|
||||
permalink: coffee-brewing-methods
|
||||
tags: [coffee, brewing, equipment] # v0.13.0: Now searchable!
|
||||
---
|
||||
|
||||
# Coffee Brewing Methods
|
||||
@@ -180,11 +211,10 @@ If you encounter permission errors:
|
||||
- relates_to [[Other Coffee Topics]]
|
||||
```
|
||||
|
||||
5. **Start the sync process** in a Terminal window (optional):
|
||||
```bash
|
||||
basic-memory sync --watch
|
||||
```
|
||||
Keep this running in the background.
|
||||
**v0.13.0 Improvements:**
|
||||
- **Real-time sync**: Changes appear immediately, no background sync needed
|
||||
- **Searchable tags**: Frontmatter tags are now indexed for search
|
||||
- **Better file organization**: Enhanced file management capabilities
|
||||
|
||||
## Using Special Prompts
|
||||
|
||||
@@ -250,14 +280,37 @@ Or directly reference notes using memory:// URLs:
|
||||
You: "Take a look at memory://coffee-brewing-methods and let's discuss how to improve my technique."
|
||||
```
|
||||
|
||||
### Building On Previous Knowledge
|
||||
### Building On Previous Knowledge (Enhanced in v0.13.0)
|
||||
|
||||
Basic Memory enables continuous knowledge building:
|
||||
|
||||
1. **Reference previous discussions** in new conversations
|
||||
2. **Add to existing notes** through conversations
|
||||
3. **Create connections** between related topics
|
||||
4. **Follow relationships** to build comprehensive context
|
||||
2. **Edit notes incrementally** without rewriting entire documents
|
||||
3. **Move and organize notes** as your knowledge base grows
|
||||
4. **Switch between projects** instantly during conversations
|
||||
5. **Search by tags** to find related content quickly
|
||||
6. **Create connections** between related topics
|
||||
7. **Follow relationships** to build comprehensive context
|
||||
|
||||
### v0.13.0 Workflow Examples
|
||||
|
||||
**Incremental Editing:**
|
||||
```
|
||||
You: "Add a section about espresso to my coffee brewing notes"
|
||||
Claude: [Uses edit_note to append new section]
|
||||
```
|
||||
|
||||
**File Organization:**
|
||||
```
|
||||
You: "Move my old meeting notes to an archive folder"
|
||||
Claude: [Uses move_note with database consistency]
|
||||
```
|
||||
|
||||
**Project Switching:**
|
||||
```
|
||||
You: "Switch to my work project and show recent activity"
|
||||
Claude: [Switches projects and shows work-specific content]
|
||||
```
|
||||
|
||||
## Importing Existing Conversations
|
||||
|
||||
@@ -271,17 +324,24 @@ basic-memory import claude conversations
|
||||
basic-memory import chatgpt
|
||||
```
|
||||
|
||||
After importing, the changes will be synced. Initial syncs may take a few moments. You can see info about your project by running `basic-memrory project info`.
|
||||
After importing, changes sync automatically in real-time. You can see project statistics by running `basic-memory project info`.
|
||||
|
||||
## Quick Tips
|
||||
|
||||
- Basic Memory will sync changes from your project in real time.
|
||||
### General Usage
|
||||
- Basic Memory syncs changes in real-time (no manual sync needed)
|
||||
- Use special prompts (Continue Conversation, Recent Activity, Search) to start contextual discussions
|
||||
- Build connections between notes for a richer knowledge graph
|
||||
- Use direct `memory://` URLs with a permalink when you need precise context. See [[User Guide#Using memory // URLs]]
|
||||
- Use git to version control your knowledge base (git integration is on the roadmap)
|
||||
- Use direct `memory://` URLs with permalinks for precise context
|
||||
- Review and edit AI-generated notes for accuracy
|
||||
|
||||
### v0.13.0 Features
|
||||
- **Switch projects instantly**: "Switch to my work project" - no restart needed
|
||||
- **Edit notes incrementally**: "Add a section about..." instead of rewriting
|
||||
- **Organize with moves**: "Move this to my archive folder" with database consistency
|
||||
- **Search by tags**: Frontmatter tags are now searchable
|
||||
- **Try beta builds**: `pip install basic-memory --pre` for latest features
|
||||
|
||||
## Next Steps
|
||||
|
||||
After getting started, explore these areas:
|
||||
@@ -290,4 +350,6 @@ After getting started, explore these areas:
|
||||
2. **Understand the [[Knowledge Format]]** to learn how knowledge is structured
|
||||
3. **Set up [[Obsidian Integration]]** for visual knowledge navigation
|
||||
4. **Learn about [[Canvas]]** visualizations for mapping concepts
|
||||
5. **Review the [[CLI Reference]]** for command line tools
|
||||
5. **Review the [[CLI Reference]]** for command line tools
|
||||
6. **Explore [[OAuth Authentication Guide]]** for secure remote access (v0.13.0)
|
||||
7. **Set up multiple projects** for different knowledge areas (v0.13.0)
|
||||
@@ -0,0 +1,259 @@
|
||||
# OAuth Authentication Guide
|
||||
|
||||
Basic Memory MCP server supports OAuth 2.1 authentication for secure access control. This guide covers setup, testing, and production deployment.
|
||||
|
||||
## Quick Start
|
||||
|
||||
### 1. Enable OAuth
|
||||
|
||||
```bash
|
||||
# Set environment variable
|
||||
export FASTMCP_AUTH_ENABLED=true
|
||||
|
||||
# Or use .env file
|
||||
echo "FASTMCP_AUTH_ENABLED=true" >> .env
|
||||
```
|
||||
|
||||
### 2. Start the Server
|
||||
|
||||
```bash
|
||||
basic-memory mcp --transport streamable-http
|
||||
```
|
||||
|
||||
### 3. Test with MCP Inspector
|
||||
|
||||
Since the basic auth provider uses in-memory storage with per-instance secret keys, you'll need to use a consistent approach:
|
||||
|
||||
#### Option A: Use Environment Variable for Secret Key
|
||||
|
||||
```bash
|
||||
# Set a fixed secret key for testing
|
||||
export FASTMCP_AUTH_SECRET_KEY="your-test-secret-key"
|
||||
|
||||
# Start the server
|
||||
FASTMCP_AUTH_ENABLED=true basic-memory mcp --transport streamable-http
|
||||
|
||||
# In another terminal, register a client
|
||||
basic-memory auth register-client --client-id=test-client
|
||||
|
||||
# Get a token using the same secret key
|
||||
basic-memory auth test-auth
|
||||
```
|
||||
|
||||
#### Option B: Use the Built-in Test Endpoint
|
||||
|
||||
```bash
|
||||
# Start server with OAuth
|
||||
FASTMCP_AUTH_ENABLED=true basic-memory mcp --transport streamable-http
|
||||
|
||||
# Register a client and get token in one step
|
||||
curl -X POST http://localhost:8000/register \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"client_metadata": {"client_name": "Test Client"}}'
|
||||
|
||||
# Use the returned client_id and client_secret
|
||||
curl -X POST http://localhost:8000/token \
|
||||
-H "Content-Type: application/x-www-form-urlencoded" \
|
||||
-d "grant_type=client_credentials&client_id=YOUR_CLIENT_ID&client_secret=YOUR_CLIENT_SECRET"
|
||||
```
|
||||
|
||||
### 4. Configure MCP Inspector
|
||||
|
||||
1. Open MCP Inspector
|
||||
2. Configure:
|
||||
- Server URL: `http://localhost:8000/mcp/` (note the trailing slash!)
|
||||
- Transport: `streamable-http`
|
||||
- Custom Headers:
|
||||
```
|
||||
Authorization: Bearer YOUR_ACCESS_TOKEN
|
||||
Accept: application/json, text/event-stream
|
||||
```
|
||||
|
||||
## OAuth Endpoints
|
||||
|
||||
The server provides these OAuth endpoints automatically:
|
||||
|
||||
- `GET /authorize` - Authorization endpoint
|
||||
- `POST /token` - Token exchange endpoint
|
||||
- `GET /.well-known/oauth-authorization-server` - OAuth metadata
|
||||
- `POST /register` - Client registration (if enabled)
|
||||
- `POST /revoke` - Token revocation (if enabled)
|
||||
|
||||
## OAuth Flow
|
||||
|
||||
### Standard Authorization Code Flow
|
||||
|
||||
1. **Get Authorization Code**:
|
||||
```bash
|
||||
curl "http://localhost:8000/authorize?client_id=YOUR_CLIENT_ID&redirect_uri=http://localhost:8000/callback&response_type=code&code_challenge=YOUR_CHALLENGE&code_challenge_method=S256"
|
||||
```
|
||||
|
||||
2. **Exchange Code for Token**:
|
||||
```bash
|
||||
curl -X POST http://localhost:8000/token \
|
||||
-H "Content-Type: application/x-www-form-urlencoded" \
|
||||
-d "grant_type=authorization_code&code=AUTH_CODE&client_id=CLIENT_ID&client_secret=CLIENT_SECRET&code_verifier=YOUR_VERIFIER"
|
||||
```
|
||||
|
||||
3. **Use Access Token**:
|
||||
```bash
|
||||
curl http://localhost:8000/mcp \
|
||||
-H "Authorization: Bearer ACCESS_TOKEN"
|
||||
```
|
||||
|
||||
## Production Deployment
|
||||
|
||||
### Using Supabase Auth
|
||||
|
||||
For production, use Supabase for persistent auth storage:
|
||||
|
||||
```bash
|
||||
# Configure environment
|
||||
FASTMCP_AUTH_ENABLED=true
|
||||
FASTMCP_AUTH_PROVIDER=supabase
|
||||
SUPABASE_URL=https://your-project.supabase.co
|
||||
SUPABASE_ANON_KEY=your-anon-key
|
||||
SUPABASE_SERVICE_KEY=your-service-key
|
||||
|
||||
# Start server
|
||||
basic-memory mcp --transport streamable-http --host 0.0.0.0
|
||||
```
|
||||
|
||||
### Security Requirements
|
||||
|
||||
1. **HTTPS Required**: OAuth requires HTTPS in production (localhost exception for testing)
|
||||
2. **PKCE Support**: Claude.ai requires PKCE for authorization
|
||||
3. **Token Expiration**: Access tokens expire after 1 hour
|
||||
4. **Scopes**: Supported scopes are `read`, `write`, and `admin`
|
||||
|
||||
## Connecting from Claude.ai
|
||||
|
||||
1. **Deploy with HTTPS**:
|
||||
```bash
|
||||
# Use ngrok for testing
|
||||
ngrok http 8000
|
||||
|
||||
# Or deploy to cloud provider
|
||||
```
|
||||
|
||||
2. **Configure in Claude.ai**:
|
||||
- Go to Settings → Integrations
|
||||
- Click "Add More"
|
||||
- Enter: `https://your-server.com/mcp`
|
||||
- Click "Connect"
|
||||
- Authorize in the popup window
|
||||
|
||||
## Debugging
|
||||
|
||||
### Common Issues
|
||||
|
||||
1. **401 Unauthorized**:
|
||||
- Check token is valid and not expired
|
||||
- Verify secret key consistency
|
||||
- Ensure bearer token format: `Authorization: Bearer TOKEN`
|
||||
|
||||
2. **404 on Auth Endpoints**:
|
||||
- Endpoints are at root, not under `/auth`
|
||||
- Use `/authorize` not `/auth/authorize`
|
||||
|
||||
3. **Token Validation Fails**:
|
||||
- Basic provider uses in-memory storage
|
||||
- Tokens don't persist across server restarts
|
||||
- Use same secret key for testing
|
||||
|
||||
### Debug Commands
|
||||
|
||||
```bash
|
||||
# Check OAuth metadata
|
||||
curl http://localhost:8000/.well-known/oauth-authorization-server
|
||||
|
||||
# Enable debug logging
|
||||
export FASTMCP_LOG_LEVEL=DEBUG
|
||||
|
||||
# Test token directly
|
||||
curl http://localhost:8000/mcp \
|
||||
-H "Authorization: Bearer YOUR_TOKEN" \
|
||||
-v
|
||||
```
|
||||
|
||||
## Provider Options
|
||||
|
||||
- **basic**: In-memory storage (development only)
|
||||
- **supabase**: Recommended for production
|
||||
- **github**: GitHub OAuth integration
|
||||
- **google**: Google OAuth integration
|
||||
|
||||
## Example Test Script
|
||||
|
||||
```python
|
||||
import httpx
|
||||
import asyncio
|
||||
from urllib.parse import urlparse, parse_qs
|
||||
|
||||
async def test_oauth_flow():
|
||||
"""Test the full OAuth flow"""
|
||||
client_id = "test-client"
|
||||
client_secret = "test-secret"
|
||||
|
||||
async with httpx.AsyncClient() as client:
|
||||
# 1. Get authorization code
|
||||
auth_response = await client.get(
|
||||
"http://localhost:8000/authorize",
|
||||
params={
|
||||
"client_id": client_id,
|
||||
"redirect_uri": "http://localhost:8000/callback",
|
||||
"response_type": "code",
|
||||
"code_challenge": "test-challenge",
|
||||
"code_challenge_method": "S256",
|
||||
"state": "test-state"
|
||||
}
|
||||
)
|
||||
|
||||
# Extract code from redirect URL
|
||||
redirect_url = auth_response.headers.get("Location")
|
||||
parsed = urlparse(redirect_url)
|
||||
code = parse_qs(parsed.query)["code"][0]
|
||||
|
||||
# 2. Exchange for token
|
||||
token_response = await client.post(
|
||||
"http://localhost:8000/token",
|
||||
data={
|
||||
"grant_type": "authorization_code",
|
||||
"code": code,
|
||||
"client_id": client_id,
|
||||
"client_secret": client_secret,
|
||||
"code_verifier": "test-verifier",
|
||||
"redirect_uri": "http://localhost:8000/callback"
|
||||
}
|
||||
)
|
||||
|
||||
tokens = token_response.json()
|
||||
print(f"Access token: {tokens['access_token']}")
|
||||
|
||||
# 3. Test MCP endpoint
|
||||
mcp_response = await client.post(
|
||||
"http://localhost:8000/mcp",
|
||||
headers={"Authorization": f"Bearer {tokens['access_token']}"},
|
||||
json={"method": "initialize", "params": {}}
|
||||
)
|
||||
|
||||
print(f"MCP Response: {mcp_response.status_code}")
|
||||
|
||||
asyncio.run(test_oauth_flow())
|
||||
```
|
||||
|
||||
## Environment Variables
|
||||
|
||||
| Variable | Description | Default |
|
||||
|----------|-------------|---------|
|
||||
| `FASTMCP_AUTH_ENABLED` | Enable OAuth authentication | `false` |
|
||||
| `FASTMCP_AUTH_PROVIDER` | OAuth provider type | `basic` |
|
||||
| `FASTMCP_AUTH_SECRET_KEY` | JWT signing key (basic provider) | Random |
|
||||
| `FASTMCP_AUTH_ISSUER_URL` | OAuth issuer URL | `http://localhost:8000` |
|
||||
| `FASTMCP_AUTH_REQUIRED_SCOPES` | Required scopes (comma-separated) | `read,write` |
|
||||
|
||||
## Next Steps
|
||||
|
||||
- [Supabase OAuth Setup](./Supabase%20OAuth%20Setup.md) - Production auth setup
|
||||
- [External OAuth Providers](./External%20OAuth%20Providers.md) - GitHub, Google integration
|
||||
- [MCP OAuth Specification](https://modelcontextprotocol.io/specification/2025-03-26/basic/authorization) - Official spec
|
||||
@@ -0,0 +1,311 @@
|
||||
# Supabase OAuth Setup for Basic Memory
|
||||
|
||||
This guide explains how to set up Supabase as the OAuth provider for Basic Memory MCP server in production.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
1. A Supabase project (create one at [supabase.com](https://supabase.com))
|
||||
2. Basic Memory MCP server deployed
|
||||
3. Environment variables configuration
|
||||
|
||||
## Overview
|
||||
|
||||
The Supabase OAuth provider offers:
|
||||
- Production-ready authentication with persistent storage
|
||||
- User management through Supabase Auth
|
||||
- JWT token validation
|
||||
- Integration with Supabase's security features
|
||||
- Support for social logins (GitHub, Google, etc.)
|
||||
|
||||
## Setup Steps
|
||||
|
||||
### 1. Get Supabase Credentials
|
||||
|
||||
From your Supabase project dashboard:
|
||||
|
||||
1. Go to Settings > API
|
||||
2. Copy these values:
|
||||
- `Project URL` → `SUPABASE_URL`
|
||||
- `anon public` key → `SUPABASE_ANON_KEY`
|
||||
- `service_role` key → `SUPABASE_SERVICE_KEY` (keep this secret!)
|
||||
- JWT secret → `SUPABASE_JWT_SECRET` (under Settings > API > JWT Settings)
|
||||
|
||||
### 2. Configure Environment Variables
|
||||
|
||||
Create a `.env` file:
|
||||
|
||||
```bash
|
||||
# Enable OAuth
|
||||
FASTMCP_AUTH_ENABLED=true
|
||||
FASTMCP_AUTH_PROVIDER=supabase
|
||||
|
||||
# Your MCP server URL
|
||||
FASTMCP_AUTH_ISSUER_URL=https://your-mcp-server.com
|
||||
|
||||
# Supabase configuration
|
||||
SUPABASE_URL=https://your-project.supabase.co
|
||||
SUPABASE_ANON_KEY=your-anon-key
|
||||
SUPABASE_SERVICE_KEY=your-service-key
|
||||
SUPABASE_JWT_SECRET=your-jwt-secret
|
||||
|
||||
# Allowed OAuth clients (comma-separated)
|
||||
SUPABASE_ALLOWED_CLIENTS=web-app,mobile-app,cli-tool
|
||||
|
||||
# Required scopes
|
||||
FASTMCP_AUTH_REQUIRED_SCOPES=read,write
|
||||
```
|
||||
|
||||
### 3. Create OAuth Clients Table (Optional)
|
||||
|
||||
For production, create a table to store OAuth clients in Supabase:
|
||||
|
||||
```sql
|
||||
CREATE TABLE oauth_clients (
|
||||
id UUID DEFAULT gen_random_uuid() PRIMARY KEY,
|
||||
client_id TEXT UNIQUE NOT NULL,
|
||||
client_secret TEXT NOT NULL,
|
||||
name TEXT,
|
||||
redirect_uris TEXT[],
|
||||
allowed_scopes TEXT[],
|
||||
created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(),
|
||||
updated_at TIMESTAMP WITH TIME ZONE DEFAULT NOW()
|
||||
);
|
||||
|
||||
-- Create an index for faster lookups
|
||||
CREATE INDEX idx_oauth_clients_client_id ON oauth_clients(client_id);
|
||||
|
||||
-- RLS policies
|
||||
ALTER TABLE oauth_clients ENABLE ROW LEVEL SECURITY;
|
||||
|
||||
-- Only service role can manage clients
|
||||
CREATE POLICY "Service role can manage clients" ON oauth_clients
|
||||
FOR ALL USING (auth.jwt()->>'role' = 'service_role');
|
||||
```
|
||||
|
||||
### 4. Set Up Auth Flow
|
||||
|
||||
The Supabase OAuth provider handles the following flow:
|
||||
|
||||
1. **Client Authorization Request**
|
||||
```
|
||||
GET /authorize?client_id=web-app&redirect_uri=https://app.com/callback
|
||||
```
|
||||
|
||||
2. **Redirect to Supabase Auth**
|
||||
- User authenticates with Supabase (email/password, magic link, or social login)
|
||||
- Supabase redirects back to your MCP server
|
||||
|
||||
3. **Token Exchange**
|
||||
```
|
||||
POST /token
|
||||
Content-Type: application/x-www-form-urlencoded
|
||||
|
||||
grant_type=authorization_code&code=xxx&client_id=web-app
|
||||
```
|
||||
|
||||
4. **Access Protected Resources**
|
||||
```
|
||||
GET /mcp
|
||||
Authorization: Bearer <access_token>
|
||||
```
|
||||
|
||||
### 5. Enable Social Logins (Optional)
|
||||
|
||||
In Supabase dashboard:
|
||||
|
||||
1. Go to Authentication > Providers
|
||||
2. Enable desired providers (GitHub, Google, etc.)
|
||||
3. Configure OAuth apps for each provider
|
||||
4. Users can now log in via social providers
|
||||
|
||||
### 6. User Management
|
||||
|
||||
Supabase provides:
|
||||
- User registration and login
|
||||
- Password reset flows
|
||||
- Email verification
|
||||
- User metadata storage
|
||||
- Admin APIs for user management
|
||||
|
||||
Access user data in your MCP tools:
|
||||
|
||||
```python
|
||||
# In your MCP tool
|
||||
async def get_user_info(ctx: Context):
|
||||
# The token is already validated by the OAuth middleware
|
||||
user_id = ctx.auth.user_id
|
||||
email = ctx.auth.email
|
||||
|
||||
# Use Supabase client to get more user data if needed
|
||||
user = await supabase.auth.admin.get_user_by_id(user_id)
|
||||
return user
|
||||
```
|
||||
|
||||
### 7. Production Deployment
|
||||
|
||||
1. **Environment Security**
|
||||
- Never expose `SUPABASE_SERVICE_KEY`
|
||||
- Use environment variables, not hardcoded values
|
||||
- Rotate keys periodically
|
||||
|
||||
2. **HTTPS Required**
|
||||
- Always use HTTPS in production
|
||||
- Configure proper SSL certificates
|
||||
|
||||
3. **Rate Limiting**
|
||||
- Implement rate limiting for auth endpoints
|
||||
- Use Supabase's built-in rate limiting
|
||||
|
||||
4. **Monitoring**
|
||||
- Monitor auth logs in Supabase dashboard
|
||||
- Set up alerts for suspicious activity
|
||||
|
||||
## Testing
|
||||
|
||||
### Local Development
|
||||
|
||||
For local testing with Supabase:
|
||||
|
||||
```bash
|
||||
# Start MCP server with Supabase auth
|
||||
FASTMCP_AUTH_ENABLED=true \
|
||||
FASTMCP_AUTH_PROVIDER=supabase \
|
||||
SUPABASE_URL=http://localhost:54321 \
|
||||
SUPABASE_ANON_KEY=your-local-anon-key \
|
||||
bm mcp --transport streamable-http
|
||||
```
|
||||
|
||||
### Test Authentication Flow
|
||||
|
||||
```python
|
||||
import httpx
|
||||
import asyncio
|
||||
|
||||
async def test_supabase_auth():
|
||||
# 1. Register/login with Supabase directly
|
||||
supabase_url = "https://your-project.supabase.co"
|
||||
|
||||
# 2. Get MCP authorization URL
|
||||
response = await httpx.get(
|
||||
"http://localhost:8000/authorize",
|
||||
params={
|
||||
"client_id": "web-app",
|
||||
"redirect_uri": "http://localhost:3000/callback",
|
||||
"response_type": "code",
|
||||
}
|
||||
)
|
||||
|
||||
# 3. User logs in via Supabase
|
||||
# 4. Exchange code for MCP tokens
|
||||
# 5. Access protected resources
|
||||
|
||||
asyncio.run(test_supabase_auth())
|
||||
```
|
||||
|
||||
## Advanced Configuration
|
||||
|
||||
### Custom User Metadata
|
||||
|
||||
Store additional user data in Supabase:
|
||||
|
||||
```sql
|
||||
-- Add custom fields to auth.users
|
||||
ALTER TABLE auth.users
|
||||
ADD COLUMN IF NOT EXISTS metadata JSONB DEFAULT '{}';
|
||||
|
||||
-- Or create a separate profiles table
|
||||
CREATE TABLE profiles (
|
||||
id UUID REFERENCES auth.users PRIMARY KEY,
|
||||
username TEXT UNIQUE,
|
||||
avatar_url TEXT,
|
||||
bio TEXT,
|
||||
created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW()
|
||||
);
|
||||
```
|
||||
|
||||
### Row Level Security (RLS)
|
||||
|
||||
Protect user data with RLS:
|
||||
|
||||
```sql
|
||||
-- Users can only access their own data
|
||||
CREATE POLICY "Users can view own profile" ON profiles
|
||||
FOR SELECT USING (auth.uid() = id);
|
||||
|
||||
CREATE POLICY "Users can update own profile" ON profiles
|
||||
FOR UPDATE USING (auth.uid() = id);
|
||||
```
|
||||
|
||||
### Custom Claims
|
||||
|
||||
Add custom claims to JWT tokens:
|
||||
|
||||
```sql
|
||||
-- Function to add custom claims
|
||||
CREATE OR REPLACE FUNCTION custom_jwt_claims()
|
||||
RETURNS JSON AS $$
|
||||
BEGIN
|
||||
RETURN json_build_object(
|
||||
'user_role', current_setting('request.jwt.claims')::json->>'user_role',
|
||||
'permissions', current_setting('request.jwt.claims')::json->>'permissions'
|
||||
);
|
||||
END;
|
||||
$$ LANGUAGE plpgsql;
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Common Issues
|
||||
|
||||
1. **Invalid JWT Secret**
|
||||
- Ensure `SUPABASE_JWT_SECRET` matches your Supabase project
|
||||
- Check Settings > API > JWT Settings in Supabase dashboard
|
||||
|
||||
2. **CORS Errors**
|
||||
- Configure CORS in your MCP server
|
||||
- Add allowed origins in Supabase dashboard
|
||||
|
||||
3. **Token Validation Fails**
|
||||
- Verify tokens are being passed correctly
|
||||
- Check token expiration times
|
||||
- Ensure scopes match requirements
|
||||
|
||||
4. **User Not Found**
|
||||
- Confirm user exists in Supabase Auth
|
||||
- Check if email is verified (if required)
|
||||
- Verify client permissions
|
||||
|
||||
### Debug Mode
|
||||
|
||||
Enable debug logging:
|
||||
|
||||
```bash
|
||||
export FASTMCP_LOG_LEVEL=DEBUG
|
||||
export SUPABASE_LOG_LEVEL=debug
|
||||
```
|
||||
|
||||
## Security Best Practices
|
||||
|
||||
1. **Secure Keys**: Never commit secrets to version control
|
||||
2. **Least Privilege**: Use minimal required scopes
|
||||
3. **Token Rotation**: Implement refresh token rotation
|
||||
4. **Audit Logs**: Monitor authentication events
|
||||
5. **Rate Limiting**: Protect against brute force attacks
|
||||
6. **HTTPS Only**: Always use encrypted connections
|
||||
|
||||
## Migration from Basic Auth
|
||||
|
||||
To migrate from the basic auth provider:
|
||||
|
||||
1. Export existing user data
|
||||
2. Import users into Supabase Auth
|
||||
3. Update client applications to use new auth flow
|
||||
4. Gradually transition users to Supabase login
|
||||
|
||||
## Next Steps
|
||||
|
||||
- Set up email templates in Supabase
|
||||
- Configure password policies
|
||||
- Implement MFA (multi-factor authentication)
|
||||
- Add social login providers
|
||||
- Create admin dashboard for user management
|
||||
+127
-26
@@ -388,6 +388,59 @@ Maintain context for complex projects over time:
|
||||
|
||||
## Advanced Features
|
||||
|
||||
### Note Editing (New in v0.13.0)
|
||||
|
||||
**Edit notes incrementally without rewriting entire documents:**
|
||||
|
||||
```
|
||||
💬 "Add a new section about deployment to my API documentation"
|
||||
🤖 [Uses edit_note to append new section]
|
||||
|
||||
💬 "Update the date at the top of my meeting notes"
|
||||
🤖 [Uses edit_note to prepend new timestamp]
|
||||
|
||||
💬 "Replace the implementation section in my design doc"
|
||||
🤖 [Uses edit_note to replace specific section]
|
||||
```
|
||||
|
||||
Available editing operations:
|
||||
- **Append**: Add content to end of notes
|
||||
- **Prepend**: Add content to beginning of notes
|
||||
- **Replace Section**: Replace content under specific headers
|
||||
- **Find & Replace**: Simple text replacements with validation
|
||||
|
||||
### File Management (New in v0.13.0)
|
||||
|
||||
**Move and organize notes with full database consistency:**
|
||||
|
||||
```
|
||||
💬 "Move my old meeting notes to the archive folder"
|
||||
🤖 [Uses move_note with automatic folder creation and database updates]
|
||||
|
||||
💬 "Reorganize my project files into a better structure"
|
||||
🤖 [Moves files while maintaining search indexes and links]
|
||||
```
|
||||
|
||||
Move operations include:
|
||||
- **Database Consistency**: Updates file paths, permalinks, and checksums
|
||||
- **Search Reindexing**: Maintains search functionality after moves
|
||||
- **Folder Creation**: Automatically creates destination directories
|
||||
- **Project Isolation**: Moves are contained within the current project
|
||||
- **Rollback Protection**: Ensures data integrity during failed operations
|
||||
|
||||
### Enhanced Search (New in v0.13.0)
|
||||
|
||||
**Frontmatter tags are now searchable:**
|
||||
|
||||
```yaml
|
||||
---
|
||||
title: Coffee Brewing Methods
|
||||
tags: [coffee, brewing, equipment]
|
||||
---
|
||||
```
|
||||
|
||||
Now searchable by: "coffee", "brewing", "equipment", or "Coffee Brewing Methods"
|
||||
|
||||
### Importing External Knowledge
|
||||
|
||||
Import existing conversations:
|
||||
@@ -398,9 +451,12 @@ basic-memory import claude conversations
|
||||
|
||||
# From ChatGPT
|
||||
basic-memory import chatgpt
|
||||
|
||||
# Target specific projects (v0.13.0)
|
||||
basic-memory --project=work import claude conversations
|
||||
```
|
||||
|
||||
After importing, run `basic-memory sync` to index everything.
|
||||
After importing, changes sync automatically in real-time.
|
||||
|
||||
### Obsidian Integration
|
||||
|
||||
@@ -460,12 +516,47 @@ basic-memory import claude conversations
|
||||
basic-memory import chatgpt
|
||||
```
|
||||
|
||||
## Multiple Projects
|
||||
## Multiple Projects (v0.13.0)
|
||||
|
||||
Basic Memory supports managing multiple separate knowledge bases through projects. This feature allows you to maintain
|
||||
separate knowledge graphs for different purposes (e.g., personal notes, work projects, research topics).
|
||||
Basic Memory v0.13.0 introduces **fluid project management** - the ability to switch between projects instantly during conversations without restart. This allows you to maintain separate knowledge graphs for different purposes while seamlessly switching between them.
|
||||
|
||||
Basic Memory keeps a list of projects in a config file: ` ~/.basic-memory/config.json`
|
||||
### Instant Project Switching (New in v0.13.0)
|
||||
|
||||
**Switch projects during conversations:**
|
||||
|
||||
```
|
||||
💬 "What projects do I have?"
|
||||
🤖 Available projects:
|
||||
• main (current, default)
|
||||
• work-notes
|
||||
• personal-journal
|
||||
• code-snippets
|
||||
|
||||
💬 "Switch to work-notes"
|
||||
🤖 ✓ Switched to work-notes project
|
||||
|
||||
Project Summary:
|
||||
• 47 entities
|
||||
• 125 observations
|
||||
• 23 relations
|
||||
|
||||
💬 "What did I work on yesterday?"
|
||||
🤖 [Shows recent activity from work-notes project]
|
||||
```
|
||||
|
||||
### Project-Specific Operations (New in v0.13.0)
|
||||
|
||||
Some MCP tools support optional project parameters for targeting specific projects:
|
||||
|
||||
```
|
||||
💬 "Create a note about this meeting in my personal-notes project"
|
||||
🤖 [Creates note in personal-notes project]
|
||||
|
||||
💬 "Switch to my work project"
|
||||
🤖 [Switches project context, then all operations work within that project]
|
||||
```
|
||||
|
||||
**Note**: Operations like search, move, and edit work within the currently active project. To work with content in different projects, switch to that project first or use the project parameter where supported.
|
||||
|
||||
### Managing Projects
|
||||
|
||||
@@ -474,16 +565,16 @@ Basic Memory keeps a list of projects in a config file: ` ~/.basic-memory/config
|
||||
basic-memory project list
|
||||
|
||||
# Add a new project
|
||||
basic-memory project add work ~/work-basic-memory
|
||||
basic-memory project create work ~/work-basic-memory
|
||||
|
||||
# Set the default project
|
||||
basic-memory project default work
|
||||
basic-memory project set-default work
|
||||
|
||||
# Remove a project (doesn't delete files)
|
||||
basic-memory project remove personal
|
||||
basic-memory project delete personal
|
||||
|
||||
# Show current project
|
||||
basic-memory project current
|
||||
# Show current project statistics
|
||||
basic-memory project info
|
||||
```
|
||||
|
||||
### Using Projects in Commands
|
||||
@@ -504,23 +595,32 @@ You can also set the `BASIC_MEMORY_PROJECT` environment variable:
|
||||
BASIC_MEMORY_PROJECT=work basic-memory sync
|
||||
```
|
||||
|
||||
### Project Isolation
|
||||
### Unified Database Architecture (New in v0.13.0)
|
||||
|
||||
Each project maintains:
|
||||
Basic Memory v0.13.0 uses a unified database architecture:
|
||||
|
||||
- Its own collection of markdown files in the specified directory
|
||||
- A separate SQLite database for that project
|
||||
- Complete knowledge graph isolation from other projects
|
||||
- **Single Database**: All projects share `~/.basic-memory/memory.db`
|
||||
- **Project Isolation**: Proper data separation with project context
|
||||
- **Better Performance**: Optimized queries and reduced file I/O
|
||||
- **Easier Backup**: Single database file contains all project data
|
||||
- **Session Context**: Maintains active project throughout conversations
|
||||
|
||||
## Workflow Tips
|
||||
|
||||
1. Run sync in watch mode for automatic updates
|
||||
2. Use git for version control of your knowledge base
|
||||
3. Review and edit AI-created content for accuracy
|
||||
4. Periodically organize and refine your knowledge structure
|
||||
5. Build rich connections between related ideas
|
||||
6. Use forward references to plan future documentation
|
||||
7. Start conversations with special prompts to leverage existing knowledge
|
||||
### General Workflow
|
||||
1. **Project Organization**: Use multiple projects to separate different areas (work, personal, research)
|
||||
2. **Session Context**: Switch projects during conversations without restart (v0.13.0)
|
||||
3. **Real-time Sync**: Changes sync automatically - no need to run watch mode
|
||||
4. **Review Content**: Edit AI-created content for accuracy
|
||||
5. **Build Connections**: Create rich relationships between related ideas
|
||||
6. **Use Special Prompts**: Start conversations with context from your knowledge base
|
||||
|
||||
### v0.13.0 Workflow Enhancements
|
||||
7. **Incremental Editing**: Use edit_note for small changes instead of rewriting entire documents
|
||||
8. **File Organization**: Move and reorganize notes as your knowledge base grows
|
||||
9. **Project-Specific Creation**: Create notes in specific projects using project parameters
|
||||
10. **Search Tags**: Use frontmatter tags to improve content discoverability
|
||||
11. **Project Statistics**: Monitor project growth and activity with project info commands
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
@@ -528,9 +628,8 @@ Each project maintains:
|
||||
|
||||
If changes aren't showing up:
|
||||
|
||||
1. Verify `basic-memory sync --watch` is running
|
||||
2. Run `basic-memory status` to check system state
|
||||
3. Try a manual sync with `basic-memory sync`
|
||||
1. Run `basic-memory status` to check system state
|
||||
2. Try a manual sync with `basic-memory sync`
|
||||
|
||||
### Missing Content
|
||||
|
||||
@@ -553,4 +652,6 @@ If relations aren't working:
|
||||
- implements [[Knowledge Format]] (How knowledge is structured)
|
||||
- relates_to [[Getting Started with Basic Memory]] (Setup and first steps)
|
||||
- relates_to [[Canvas]] (Creating visual knowledge maps)
|
||||
- relates_to [[CLI Reference]] (Command line tools)
|
||||
- relates_to [[CLI Reference]] (Command line tools)
|
||||
- enhanced_in_v0.13.0 [[OAuth Authentication Guide]] (Production authentication)
|
||||
- enhanced_in_v0.13.0 [[Project Management]] (Multi-project workflows)
|
||||
@@ -0,0 +1,69 @@
|
||||
---
|
||||
title: Test Note Creation - Basic Functionality
|
||||
type: note
|
||||
permalink: testing/test-note-creation-basic-functionality
|
||||
tags:
|
||||
- '["testing"'
|
||||
- '"core-functionality"'
|
||||
- '"note-creation"]'
|
||||
---
|
||||
|
||||
---
|
||||
title: Test Note Creation - Basic Functionality
|
||||
tags: [testing, core-functionality, note-creation, edited]
|
||||
test_status: active
|
||||
last_edited: 2025-06-01
|
||||
---
|
||||
|
||||
# Test Note Creation - Basic Functionality
|
||||
|
||||
## Test Status: COMPREHENSIVE TESTING IN PROGRESS
|
||||
Testing basic note creation with various content types and structures.
|
||||
|
||||
## Content Types Tested
|
||||
- Plain text content ✓
|
||||
- Markdown formatting **bold**, *italic*
|
||||
- Lists:
|
||||
- Bullet points
|
||||
- Numbered items
|
||||
- Code blocks: `inline code`
|
||||
|
||||
```python
|
||||
# Block code
|
||||
def test_function():
|
||||
return "Hello, Basic Memory!"
|
||||
```
|
||||
|
||||
## Special Characters
|
||||
- Unicode: café, naïve, résumé
|
||||
- Emojis: 🚀 🔬 📝
|
||||
- Symbols: @#$%^&*()
|
||||
|
||||
## Frontmatter Testing
|
||||
This note should have proper frontmatter parsing.
|
||||
|
||||
## Relations to Test
|
||||
- connects_to [[Another Test Note]]
|
||||
- validates [[Core Functionality Tests]]
|
||||
|
||||
## Observations
|
||||
- [success] Note creation initiated
|
||||
- [test] Content variety included
|
||||
- [validation] Special characters included
|
||||
|
||||
|
||||
## Edit Test Results
|
||||
- [success] Note reading via title lookup ✓
|
||||
- [success] Search functionality returns relevant results ✓
|
||||
- [success] Special characters (unicode, emojis) preserved ✓
|
||||
- [test] Now testing append edit operation ✓
|
||||
|
||||
## Performance Notes
|
||||
- Note creation: Instantaneous
|
||||
- Note reading: Fast response
|
||||
- Search: Good relevance scoring
|
||||
|
||||
## Next Tests
|
||||
- Edit operations (append, prepend, find_replace)
|
||||
- Move operations
|
||||
- Cross-project functionality
|
||||
Binary file not shown.
@@ -1,26 +0,0 @@
|
||||
# Basic Memory Installer
|
||||
|
||||
This installer configures Basic Memory to work with Claude Desktop.
|
||||
|
||||
## Installation
|
||||
|
||||
1. Download the latest installer from the [releases page](https://github.com/basicmachines-co/basic-memory/releases)
|
||||
2. Unzip the downloaded file
|
||||
3. Since the app is currently unsigned, you'll need to:
|
||||
|
||||
On your Mac, choose Apple menu > System Settings, then click Privacy & Security in the sidebar. (You may need to
|
||||
scroll down.)
|
||||
|
||||
Go to Security, then click Open.
|
||||
|
||||
Click Open Anyway.
|
||||
|
||||
This button is available for about an hour after you try to open the app.
|
||||
|
||||
Enter your login password, then click OK.
|
||||
|
||||
https://support.apple.com/guide/mac-help/apple-cant-check-app-for-malicious-software-mchleab3a043/mac
|
||||
|
||||
5. Restart Claude Desktop
|
||||
|
||||
The warning only appears the first time you open the app. Future updates will include proper code signing.
|
||||
@@ -1,64 +0,0 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<svg viewBox="0 0 512 512" xmlns="http://www.w3.org/2000/svg">
|
||||
<!-- Background -->
|
||||
<rect x="0" y="0" width="512" height="512" rx="64" fill="#111111"/>
|
||||
|
||||
<!-- Define arrowhead marker -->
|
||||
<defs>
|
||||
<marker id="arrowhead"
|
||||
markerWidth="10"
|
||||
markerHeight="10"
|
||||
refX="8"
|
||||
refY="5"
|
||||
orient="auto">
|
||||
<path d="M 0 0 L 10 5 L 0 10 Z"
|
||||
fill="#00cc00"/>
|
||||
</marker>
|
||||
</defs>
|
||||
|
||||
<!-- State 1 (initial) -->
|
||||
<circle cx="156" cy="256" r="30" fill="none" stroke="#00cc00" stroke-width="3"/>
|
||||
|
||||
<!-- State 2 (accept) -->
|
||||
<circle cx="356" cy="176" r="34" fill="none" stroke="#00cc00" stroke-width="3"/>
|
||||
<circle cx="356" cy="176" r="28" fill="none" stroke="#00cc00" stroke-width="3"/>
|
||||
|
||||
<!-- State 3 (accept) -->
|
||||
<circle cx="356" cy="336" r="34" fill="none" stroke="#00cc00" stroke-width="3"/>
|
||||
<circle cx="356" cy="336" r="28" fill="none" stroke="#00cc00" stroke-width="3"/>
|
||||
|
||||
<!-- Initial arrow -->
|
||||
<path d="M 96 256 L 126 256"
|
||||
stroke="#00cc00" stroke-width="3" fill="none"
|
||||
marker-end="url(#arrowhead)"/>
|
||||
|
||||
<!-- State transitions -->
|
||||
<!-- 1 -> 2 -->
|
||||
<path d="M 180 240
|
||||
Q 260 200, 320 176"
|
||||
stroke="#00cc00" stroke-width="3" fill="none"
|
||||
marker-end="url(#arrowhead)"/>
|
||||
|
||||
<!-- 1 -> 3 -->
|
||||
<path d="M 180 272
|
||||
Q 260 312, 320 336"
|
||||
stroke="#00cc00" stroke-width="3" fill="none"
|
||||
marker-end="url(#arrowhead)"/>
|
||||
|
||||
<!-- Self loops -->
|
||||
<path d="M 356 142
|
||||
Q 396 142, 396 176
|
||||
Q 396 210, 356 210
|
||||
Q 316 210, 316 176
|
||||
Q 316 142, 356 142"
|
||||
stroke="#00cc00" stroke-width="2" fill="none"
|
||||
marker-end="url(#arrowhead)"/>
|
||||
|
||||
<path d="M 356 302
|
||||
Q 396 302, 396 336
|
||||
Q 396 370, 356 370
|
||||
Q 316 370, 316 336
|
||||
Q 316 302, 356 302"
|
||||
stroke="#00cc00" stroke-width="2" fill="none"
|
||||
marker-end="url(#arrowhead)"/>
|
||||
</svg>
|
||||
|
Before Width: | Height: | Size: 2.0 KiB |
@@ -1,93 +0,0 @@
|
||||
import json
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
# Use tkinter for GUI alerts on macOS
|
||||
if sys.platform == "darwin":
|
||||
import tkinter as tk
|
||||
from tkinter import messagebox
|
||||
|
||||
|
||||
def ensure_uv_installed():
|
||||
"""Check if uv is installed, install if not."""
|
||||
try:
|
||||
subprocess.run(["uv", "--version"], capture_output=True, check=True)
|
||||
except (subprocess.CalledProcessError, FileNotFoundError):
|
||||
print("Installing uv package manager...")
|
||||
subprocess.run(
|
||||
[
|
||||
"curl",
|
||||
"-LsSf",
|
||||
"https://astral.sh/uv/install.sh",
|
||||
"|",
|
||||
"sh",
|
||||
],
|
||||
shell=True,
|
||||
)
|
||||
|
||||
|
||||
def get_config_path():
|
||||
"""Get Claude Desktop config path for current platform."""
|
||||
if sys.platform == "darwin":
|
||||
return Path.home() / "Library/Application Support/Claude/claude_desktop_config.json"
|
||||
elif sys.platform == "win32":
|
||||
return Path.home() / "AppData/Roaming/Claude/claude_desktop_config.json"
|
||||
else:
|
||||
raise RuntimeError(f"Unsupported platform: {sys.platform}")
|
||||
|
||||
|
||||
def update_claude_config():
|
||||
"""Update Claude Desktop config to include basic-memory."""
|
||||
config_path = get_config_path()
|
||||
config_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
# Load existing config or create new
|
||||
if config_path.exists():
|
||||
config = json.loads(config_path.read_text(encoding="utf-8"))
|
||||
else:
|
||||
config = {"mcpServers": {}}
|
||||
|
||||
# Add/update basic-memory config
|
||||
config["mcpServers"]["basic-memory"] = {
|
||||
"command": "uvx",
|
||||
"args": ["basic-memory@latest", "mcp"],
|
||||
}
|
||||
|
||||
# Write back config
|
||||
config_path.write_text(json.dumps(config, indent=2))
|
||||
|
||||
|
||||
def print_completion_message():
|
||||
"""Show completion message with helpful tips."""
|
||||
message = """Installation complete! Basic Memory is now available in Claude Desktop.
|
||||
|
||||
Please restart Claude Desktop for changes to take effect.
|
||||
|
||||
Quick Start:
|
||||
1. You can run sync directly using: uvx basic-memory sync
|
||||
2. Optionally, install globally with: uv pip install basic-memory
|
||||
|
||||
Built with ♥️ by Basic Machines."""
|
||||
|
||||
if sys.platform == "darwin":
|
||||
# Show GUI message on macOS
|
||||
root = tk.Tk()
|
||||
root.withdraw() # Hide the main window
|
||||
messagebox.showinfo("Basic Memory", message)
|
||||
root.destroy()
|
||||
else:
|
||||
# Fallback to console output
|
||||
print(message)
|
||||
|
||||
|
||||
def main():
|
||||
print("Welcome to Basic Memory installer")
|
||||
ensure_uv_installed()
|
||||
print("Configuring Claude Desktop...")
|
||||
update_claude_config()
|
||||
print_completion_message()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -1,27 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# Convert SVG to PNG at various required sizes
|
||||
rsvg-convert -h 16 -w 16 icon.svg > icon_16x16.png
|
||||
rsvg-convert -h 32 -w 32 icon.svg > icon_32x32.png
|
||||
rsvg-convert -h 128 -w 128 icon.svg > icon_128x128.png
|
||||
rsvg-convert -h 256 -w 256 icon.svg > icon_256x256.png
|
||||
rsvg-convert -h 512 -w 512 icon.svg > icon_512x512.png
|
||||
|
||||
# Create iconset directory
|
||||
mkdir -p Basic.iconset
|
||||
|
||||
# Move files into iconset with Mac-specific names
|
||||
cp icon_16x16.png Basic.iconset/icon_16x16.png
|
||||
cp icon_32x32.png Basic.iconset/icon_16x16@2x.png
|
||||
cp icon_32x32.png Basic.iconset/icon_32x32.png
|
||||
cp icon_128x128.png Basic.iconset/icon_32x32@2x.png
|
||||
cp icon_256x256.png Basic.iconset/icon_128x128.png
|
||||
cp icon_512x512.png Basic.iconset/icon_256x256.png
|
||||
cp icon_512x512.png Basic.iconset/icon_512x512.png
|
||||
|
||||
# Convert iconset to icns
|
||||
iconutil -c icns Basic.iconset
|
||||
|
||||
# Clean up
|
||||
rm -rf Basic.iconset
|
||||
rm icon_*.png
|
||||
@@ -1,40 +0,0 @@
|
||||
from cx_Freeze import setup, Executable
|
||||
import sys
|
||||
|
||||
# Build options for all platforms
|
||||
build_exe_options = {
|
||||
"packages": ["json", "pathlib"],
|
||||
"excludes": ["unittest", "pydoc", "test"],
|
||||
}
|
||||
|
||||
# Platform-specific options
|
||||
if sys.platform == "win32":
|
||||
base = "Win32GUI" # Use GUI base for Windows
|
||||
build_exe_options.update(
|
||||
{
|
||||
"include_msvcr": True,
|
||||
}
|
||||
)
|
||||
target_name = "Basic Memory Installer.exe"
|
||||
else: # darwin
|
||||
base = None # Don't use GUI base for macOS
|
||||
target_name = "Basic Memory Installer"
|
||||
|
||||
executables = [
|
||||
Executable(script="installer.py", target_name=target_name, base=base, icon="Basic.icns")
|
||||
]
|
||||
|
||||
setup(
|
||||
name="basic-memory",
|
||||
version=open("../pyproject.toml").read().split('version = "', 1)[1].split('"', 1)[0],
|
||||
description="Basic Memory - Local-first knowledge management",
|
||||
options={
|
||||
"build_exe": build_exe_options,
|
||||
"bdist_mac": {
|
||||
"bundle_name": "Basic Memory Installer",
|
||||
"iconfile": "Basic.icns",
|
||||
"codesign_identity": "-", # Force ad-hoc signing
|
||||
},
|
||||
},
|
||||
executables=executables,
|
||||
)
|
||||
@@ -0,0 +1,182 @@
|
||||
# Basic Memory - Modern Command Runner
|
||||
|
||||
# Install dependencies
|
||||
install:
|
||||
pip install -e ".[dev]"
|
||||
|
||||
# Run unit tests in parallel
|
||||
test-unit:
|
||||
uv run pytest -p pytest_mock -v -n auto
|
||||
|
||||
# Run integration tests in parallel
|
||||
test-int:
|
||||
uv run pytest -p pytest_mock -v --no-cov -n auto test-int
|
||||
|
||||
# Run all tests
|
||||
test: test-unit test-int
|
||||
|
||||
# Lint and fix code
|
||||
lint:
|
||||
ruff check . --fix
|
||||
|
||||
# Type check code
|
||||
type-check:
|
||||
uv run pyright
|
||||
|
||||
# Clean build artifacts and cache files
|
||||
clean:
|
||||
find . -type f -name '*.pyc' -delete
|
||||
find . -type d -name '__pycache__' -exec rm -r {} +
|
||||
rm -rf installer/build/ installer/dist/ dist/
|
||||
rm -f rw.*.dmg .coverage.*
|
||||
|
||||
# Format code with ruff
|
||||
format:
|
||||
uv run ruff format .
|
||||
|
||||
# Run MCP inspector tool
|
||||
run-inspector:
|
||||
npx @modelcontextprotocol/inspector
|
||||
|
||||
# Build macOS installer
|
||||
installer-mac:
|
||||
cd installer && chmod +x make_icons.sh && ./make_icons.sh
|
||||
cd installer && uv run python setup.py bdist_mac
|
||||
|
||||
# Build Windows installer
|
||||
installer-win:
|
||||
cd installer && uv run python setup.py bdist_win32
|
||||
|
||||
# Update all dependencies to latest versions
|
||||
update-deps:
|
||||
uv sync --upgrade
|
||||
|
||||
# Run all code quality checks and tests
|
||||
check: lint format type-check test
|
||||
|
||||
# Generate Alembic migration with descriptive message
|
||||
migration message:
|
||||
cd src/basic_memory/alembic && alembic revision --autogenerate -m "{{message}}"
|
||||
|
||||
# Create a stable release (e.g., just release v0.13.2)
|
||||
release version:
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
# Validate version format
|
||||
if [[ ! "{{version}}" =~ ^v[0-9]+\.[0-9]+\.[0-9]+$ ]]; then
|
||||
echo "❌ Invalid version format. Use: v0.13.2"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Extract version number without 'v' prefix
|
||||
VERSION_NUM=$(echo "{{version}}" | sed 's/^v//')
|
||||
|
||||
echo "🚀 Creating stable release {{version}}"
|
||||
|
||||
# Pre-flight checks
|
||||
echo "📋 Running pre-flight checks..."
|
||||
if [[ -n $(git status --porcelain) ]]; then
|
||||
echo "❌ Uncommitted changes found. Please commit or stash them first."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [[ $(git branch --show-current) != "main" ]]; then
|
||||
echo "❌ Not on main branch. Switch to main first."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Check if tag already exists
|
||||
if git tag -l "{{version}}" | grep -q "{{version}}"; then
|
||||
echo "❌ Tag {{version}} already exists"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Run quality checks
|
||||
echo "🔍 Running quality checks..."
|
||||
just check
|
||||
|
||||
# Update version in __init__.py
|
||||
echo "📝 Updating version in __init__.py..."
|
||||
sed -i.bak "s/__version__ = \".*\"/__version__ = \"$VERSION_NUM\"/" src/basic_memory/__init__.py
|
||||
rm -f src/basic_memory/__init__.py.bak
|
||||
|
||||
# Commit version update
|
||||
git add src/basic_memory/__init__.py
|
||||
git commit -m "chore: update version to $VERSION_NUM for {{version}} release"
|
||||
|
||||
# Create and push tag
|
||||
echo "🏷️ Creating tag {{version}}..."
|
||||
git tag "{{version}}"
|
||||
|
||||
echo "📤 Pushing to GitHub..."
|
||||
git push origin main
|
||||
git push origin "{{version}}"
|
||||
|
||||
echo "✅ Release {{version}} created successfully!"
|
||||
echo "📦 GitHub Actions will build and publish to PyPI"
|
||||
echo "🔗 Monitor at: https://github.com/basicmachines-co/basic-memory/actions"
|
||||
|
||||
# Create a beta release (e.g., just beta v0.13.2b1)
|
||||
beta version:
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
# Validate version format (allow beta/rc suffixes)
|
||||
if [[ ! "{{version}}" =~ ^v[0-9]+\.[0-9]+\.[0-9]+(b[0-9]+|rc[0-9]+)$ ]]; then
|
||||
echo "❌ Invalid beta version format. Use: v0.13.2b1 or v0.13.2rc1"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Extract version number without 'v' prefix
|
||||
VERSION_NUM=$(echo "{{version}}" | sed 's/^v//')
|
||||
|
||||
echo "🧪 Creating beta release {{version}}"
|
||||
|
||||
# Pre-flight checks
|
||||
echo "📋 Running pre-flight checks..."
|
||||
if [[ -n $(git status --porcelain) ]]; then
|
||||
echo "❌ Uncommitted changes found. Please commit or stash them first."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [[ $(git branch --show-current) != "main" ]]; then
|
||||
echo "❌ Not on main branch. Switch to main first."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Check if tag already exists
|
||||
if git tag -l "{{version}}" | grep -q "{{version}}"; then
|
||||
echo "❌ Tag {{version}} already exists"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Run quality checks
|
||||
echo "🔍 Running quality checks..."
|
||||
just check
|
||||
|
||||
# Update version in __init__.py
|
||||
echo "📝 Updating version in __init__.py..."
|
||||
sed -i.bak "s/__version__ = \".*\"/__version__ = \"$VERSION_NUM\"/" src/basic_memory/__init__.py
|
||||
rm -f src/basic_memory/__init__.py.bak
|
||||
|
||||
# Commit version update
|
||||
git add src/basic_memory/__init__.py
|
||||
git commit -m "chore: update version to $VERSION_NUM for {{version}} beta release"
|
||||
|
||||
# Create and push tag
|
||||
echo "🏷️ Creating tag {{version}}..."
|
||||
git tag "{{version}}"
|
||||
|
||||
echo "📤 Pushing to GitHub..."
|
||||
git push origin main
|
||||
git push origin "{{version}}"
|
||||
|
||||
echo "✅ Beta release {{version}} created successfully!"
|
||||
echo "📦 GitHub Actions will build and publish to PyPI as pre-release"
|
||||
echo "🔗 Monitor at: https://github.com/basicmachines-co/basic-memory/actions"
|
||||
echo "📥 Install with: uv tool install basic-memory --pre"
|
||||
|
||||
# List all available recipes
|
||||
default:
|
||||
@just --list
|
||||
+44
-25
@@ -1,6 +1,6 @@
|
||||
[project]
|
||||
name = "basic-memory"
|
||||
version = "0.12.2"
|
||||
dynamic = ["version"]
|
||||
description = "Local-first knowledge management combining Zettelkasten with knowledge graphs"
|
||||
readme = "README.md"
|
||||
requires-python = ">=3.12.1"
|
||||
@@ -28,8 +28,12 @@ dependencies = [
|
||||
"watchfiles>=1.0.4",
|
||||
"fastapi[standard]>=0.115.8",
|
||||
"alembic>=1.14.1",
|
||||
"qasync>=0.27.1",
|
||||
"pillow>=11.1.0",
|
||||
"pybars3>=0.9.7",
|
||||
"fastmcp>=2.3.4",
|
||||
"pyjwt>=2.10.1",
|
||||
"python-dotenv>=1.1.0",
|
||||
"pytest-aio>=1.9.0",
|
||||
]
|
||||
|
||||
|
||||
@@ -43,7 +47,7 @@ basic-memory = "basic_memory.cli.main:app"
|
||||
bm = "basic_memory.cli.main:app"
|
||||
|
||||
[build-system]
|
||||
requires = ["hatchling"]
|
||||
requires = ["hatchling", "uv-dynamic-versioning>=0.7.0"]
|
||||
build-backend = "hatchling.build"
|
||||
|
||||
[tool.pytest.ini_options]
|
||||
@@ -65,16 +69,19 @@ dev-dependencies = [
|
||||
"pytest-cov>=4.1.0",
|
||||
"pytest-mock>=3.12.0",
|
||||
"pytest-asyncio>=0.24.0",
|
||||
"pytest-xdist>=3.0.0",
|
||||
"ruff>=0.1.6",
|
||||
"pytest>=8.3.4",
|
||||
"pytest-cov>=4.1.0",
|
||||
"pytest-mock>=3.12.0",
|
||||
"pytest-asyncio>=0.24.0",
|
||||
"ruff>=0.1.6",
|
||||
"cx-freeze>=7.2.10",
|
||||
"pyqt6>=6.8.1",
|
||||
]
|
||||
|
||||
[tool.hatch.version]
|
||||
source = "uv-dynamic-versioning"
|
||||
|
||||
[tool.uv-dynamic-versioning]
|
||||
vcs = "git"
|
||||
style = "pep440"
|
||||
bump = true
|
||||
fallback-version = "0.0.0"
|
||||
|
||||
[tool.pyright]
|
||||
include = ["src/"]
|
||||
exclude = ["**/__pycache__"]
|
||||
@@ -85,23 +92,35 @@ reportMissingTypeStubs = false
|
||||
pythonVersion = "3.12"
|
||||
|
||||
|
||||
[tool.semantic_release]
|
||||
version_variables = [
|
||||
"src/basic_memory/__init__.py:__version__",
|
||||
]
|
||||
version_toml = [
|
||||
"pyproject.toml:project.version",
|
||||
]
|
||||
major_on_zero = false
|
||||
branch = "main"
|
||||
changelog_file = "CHANGELOG.md"
|
||||
build_command = "pip install uv && uv build"
|
||||
dist_path = "dist/"
|
||||
upload_to_pypi = true
|
||||
commit_message = "chore(release): {version} [skip ci]"
|
||||
|
||||
[tool.coverage.run]
|
||||
concurrency = ["thread", "gevent"]
|
||||
|
||||
[tool.coverage.report]
|
||||
exclude_lines = [
|
||||
"pragma: no cover",
|
||||
"def __repr__",
|
||||
"if self.debug:",
|
||||
"if settings.DEBUG",
|
||||
"raise AssertionError",
|
||||
"raise NotImplementedError",
|
||||
"if 0:",
|
||||
"if __name__ == .__main__.:",
|
||||
"class .*\\bProtocol\\):",
|
||||
"@(abc\\.)?abstractmethod",
|
||||
]
|
||||
|
||||
# Exclude specific modules that are difficult to test comprehensively
|
||||
omit = [
|
||||
"*/external_auth_provider.py", # External HTTP calls to OAuth providers
|
||||
"*/supabase_auth_provider.py", # External HTTP calls to Supabase APIs
|
||||
"*/watch_service.py", # File system watching - complex integration testing
|
||||
"*/background_sync.py", # Background processes
|
||||
"*/cli/main.py", # CLI entry point
|
||||
"*/mcp/tools/project_management.py", # Covered by integration tests
|
||||
"*/mcp/tools/sync_status.py", # Covered by integration tests
|
||||
"*/services/migration_service.py", # Complex migration scenarios
|
||||
]
|
||||
|
||||
[tool.logfire]
|
||||
ignore_no_config = true
|
||||
ignore_no_config = true
|
||||
@@ -1,3 +1,7 @@
|
||||
"""basic-memory - Local-first knowledge management combining Zettelkasten with knowledge graphs"""
|
||||
|
||||
__version__ = "0.12.2"
|
||||
# Package version - updated by release automation
|
||||
__version__ = "0.13.7"
|
||||
|
||||
# API version for FastAPI - independent of package version
|
||||
__api_version__ = "v0"
|
||||
|
||||
@@ -8,12 +8,12 @@ from sqlalchemy import pool
|
||||
|
||||
from alembic import context
|
||||
|
||||
from basic_memory.models import Base
|
||||
|
||||
# set config.env to "test" for pytest to prevent logging to file in utils.setup_logging()
|
||||
os.environ["BASIC_MEMORY_ENV"] = "test"
|
||||
|
||||
from basic_memory.config import config as app_config
|
||||
# Import after setting environment variable # noqa: E402
|
||||
from basic_memory.config import app_config # noqa: E402
|
||||
from basic_memory.models import Base # noqa: E402
|
||||
|
||||
# this is the Alembic Config object, which provides
|
||||
# access to the values within the .ini file in use.
|
||||
|
||||
@@ -0,0 +1,108 @@
|
||||
"""add projects table
|
||||
|
||||
Revision ID: 5fe1ab1ccebe
|
||||
Revises: cc7172b46608
|
||||
Create Date: 2025-05-14 09:05:18.214357
|
||||
|
||||
"""
|
||||
|
||||
from typing import Sequence, Union
|
||||
|
||||
from alembic import op
|
||||
import sqlalchemy as sa
|
||||
|
||||
|
||||
# revision identifiers, used by Alembic.
|
||||
revision: str = "5fe1ab1ccebe"
|
||||
down_revision: Union[str, None] = "cc7172b46608"
|
||||
branch_labels: Union[str, Sequence[str], None] = None
|
||||
depends_on: Union[str, Sequence[str], None] = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
# ### commands auto generated by Alembic - please adjust! ###
|
||||
op.create_table(
|
||||
"project",
|
||||
sa.Column("id", sa.Integer(), nullable=False),
|
||||
sa.Column("name", sa.String(), nullable=False),
|
||||
sa.Column("description", sa.Text(), nullable=True),
|
||||
sa.Column("permalink", sa.String(), nullable=False),
|
||||
sa.Column("path", sa.String(), nullable=False),
|
||||
sa.Column("is_active", sa.Boolean(), nullable=False),
|
||||
sa.Column("is_default", sa.Boolean(), nullable=True),
|
||||
sa.Column("created_at", sa.DateTime(), nullable=False),
|
||||
sa.Column("updated_at", sa.DateTime(), nullable=False),
|
||||
sa.PrimaryKeyConstraint("id"),
|
||||
sa.UniqueConstraint("is_default"),
|
||||
sa.UniqueConstraint("name"),
|
||||
sa.UniqueConstraint("permalink"),
|
||||
if_not_exists=True,
|
||||
)
|
||||
with op.batch_alter_table("project", schema=None) as batch_op:
|
||||
batch_op.create_index(
|
||||
"ix_project_created_at", ["created_at"], unique=False, if_not_exists=True
|
||||
)
|
||||
batch_op.create_index("ix_project_name", ["name"], unique=True, if_not_exists=True)
|
||||
batch_op.create_index("ix_project_path", ["path"], unique=False, if_not_exists=True)
|
||||
batch_op.create_index(
|
||||
"ix_project_permalink", ["permalink"], unique=True, if_not_exists=True
|
||||
)
|
||||
batch_op.create_index(
|
||||
"ix_project_updated_at", ["updated_at"], unique=False, if_not_exists=True
|
||||
)
|
||||
|
||||
with op.batch_alter_table("entity", schema=None) as batch_op:
|
||||
batch_op.add_column(sa.Column("project_id", sa.Integer(), nullable=False))
|
||||
batch_op.drop_index(
|
||||
"uix_entity_permalink",
|
||||
sqlite_where=sa.text("content_type = 'text/markdown' AND permalink IS NOT NULL"),
|
||||
)
|
||||
batch_op.drop_index("ix_entity_file_path")
|
||||
batch_op.create_index(batch_op.f("ix_entity_file_path"), ["file_path"], unique=False)
|
||||
batch_op.create_index("ix_entity_project_id", ["project_id"], unique=False)
|
||||
batch_op.create_index(
|
||||
"uix_entity_file_path_project", ["file_path", "project_id"], unique=True
|
||||
)
|
||||
batch_op.create_index(
|
||||
"uix_entity_permalink_project",
|
||||
["permalink", "project_id"],
|
||||
unique=True,
|
||||
sqlite_where=sa.text("content_type = 'text/markdown' AND permalink IS NOT NULL"),
|
||||
)
|
||||
batch_op.create_foreign_key("fk_entity_project_id", "project", ["project_id"], ["id"])
|
||||
|
||||
# drop the search index table. it will be recreated
|
||||
op.drop_table("search_index")
|
||||
|
||||
# ### end Alembic commands ###
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
# ### commands auto generated by Alembic - please adjust! ###
|
||||
with op.batch_alter_table("entity", schema=None) as batch_op:
|
||||
batch_op.drop_constraint("fk_entity_project_id", type_="foreignkey")
|
||||
batch_op.drop_index(
|
||||
"uix_entity_permalink_project",
|
||||
sqlite_where=sa.text("content_type = 'text/markdown' AND permalink IS NOT NULL"),
|
||||
)
|
||||
batch_op.drop_index("uix_entity_file_path_project")
|
||||
batch_op.drop_index("ix_entity_project_id")
|
||||
batch_op.drop_index(batch_op.f("ix_entity_file_path"))
|
||||
batch_op.create_index("ix_entity_file_path", ["file_path"], unique=1)
|
||||
batch_op.create_index(
|
||||
"uix_entity_permalink",
|
||||
["permalink"],
|
||||
unique=1,
|
||||
sqlite_where=sa.text("content_type = 'text/markdown' AND permalink IS NOT NULL"),
|
||||
)
|
||||
batch_op.drop_column("project_id")
|
||||
|
||||
with op.batch_alter_table("project", schema=None) as batch_op:
|
||||
batch_op.drop_index("ix_project_updated_at")
|
||||
batch_op.drop_index("ix_project_permalink")
|
||||
batch_op.drop_index("ix_project_path")
|
||||
batch_op.drop_index("ix_project_name")
|
||||
batch_op.drop_index("ix_project_created_at")
|
||||
|
||||
op.drop_table("project")
|
||||
# ### end Alembic commands ###
|
||||
@@ -0,0 +1,104 @@
|
||||
"""project constraint fix
|
||||
|
||||
Revision ID: 647e7a75e2cd
|
||||
Revises: 5fe1ab1ccebe
|
||||
Create Date: 2025-06-03 12:48:30.162566
|
||||
|
||||
"""
|
||||
|
||||
from typing import Sequence, Union
|
||||
|
||||
from alembic import op
|
||||
import sqlalchemy as sa
|
||||
|
||||
|
||||
# revision identifiers, used by Alembic.
|
||||
revision: str = "647e7a75e2cd"
|
||||
down_revision: Union[str, None] = "5fe1ab1ccebe"
|
||||
branch_labels: Union[str, Sequence[str], None] = None
|
||||
depends_on: Union[str, Sequence[str], None] = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
"""Remove the problematic UNIQUE constraint on is_default column.
|
||||
|
||||
The UNIQUE constraint prevents multiple projects from having is_default=FALSE,
|
||||
which breaks project creation when the service sets is_default=False.
|
||||
|
||||
Since SQLite doesn't support dropping specific constraints easily, we'll
|
||||
recreate the table without the problematic constraint.
|
||||
"""
|
||||
# For SQLite, we need to recreate the table without the UNIQUE constraint
|
||||
# Create a new table without the UNIQUE constraint on is_default
|
||||
op.create_table(
|
||||
"project_new",
|
||||
sa.Column("id", sa.Integer(), nullable=False),
|
||||
sa.Column("name", sa.String(), nullable=False),
|
||||
sa.Column("description", sa.Text(), nullable=True),
|
||||
sa.Column("permalink", sa.String(), nullable=False),
|
||||
sa.Column("path", sa.String(), nullable=False),
|
||||
sa.Column("is_active", sa.Boolean(), nullable=False),
|
||||
sa.Column("is_default", sa.Boolean(), nullable=True), # No UNIQUE constraint!
|
||||
sa.Column("created_at", sa.DateTime(), nullable=False),
|
||||
sa.Column("updated_at", sa.DateTime(), nullable=False),
|
||||
sa.PrimaryKeyConstraint("id"),
|
||||
sa.UniqueConstraint("name"),
|
||||
sa.UniqueConstraint("permalink"),
|
||||
)
|
||||
|
||||
# Copy data from old table to new table
|
||||
op.execute("INSERT INTO project_new SELECT * FROM project")
|
||||
|
||||
# Drop the old table
|
||||
op.drop_table("project")
|
||||
|
||||
# Rename the new table
|
||||
op.rename_table("project_new", "project")
|
||||
|
||||
# Recreate the indexes
|
||||
with op.batch_alter_table("project", schema=None) as batch_op:
|
||||
batch_op.create_index("ix_project_created_at", ["created_at"], unique=False)
|
||||
batch_op.create_index("ix_project_name", ["name"], unique=True)
|
||||
batch_op.create_index("ix_project_path", ["path"], unique=False)
|
||||
batch_op.create_index("ix_project_permalink", ["permalink"], unique=True)
|
||||
batch_op.create_index("ix_project_updated_at", ["updated_at"], unique=False)
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
"""Add back the UNIQUE constraint on is_default column.
|
||||
|
||||
WARNING: This will break project creation again if multiple projects
|
||||
have is_default=FALSE.
|
||||
"""
|
||||
# Recreate the table with the UNIQUE constraint
|
||||
op.create_table(
|
||||
"project_old",
|
||||
sa.Column("id", sa.Integer(), nullable=False),
|
||||
sa.Column("name", sa.String(), nullable=False),
|
||||
sa.Column("description", sa.Text(), nullable=True),
|
||||
sa.Column("permalink", sa.String(), nullable=False),
|
||||
sa.Column("path", sa.String(), nullable=False),
|
||||
sa.Column("is_active", sa.Boolean(), nullable=False),
|
||||
sa.Column("is_default", sa.Boolean(), nullable=True),
|
||||
sa.Column("created_at", sa.DateTime(), nullable=False),
|
||||
sa.Column("updated_at", sa.DateTime(), nullable=False),
|
||||
sa.PrimaryKeyConstraint("id"),
|
||||
sa.UniqueConstraint("is_default"), # Add back the problematic constraint
|
||||
sa.UniqueConstraint("name"),
|
||||
sa.UniqueConstraint("permalink"),
|
||||
)
|
||||
|
||||
# Copy data (this may fail if multiple FALSE values exist)
|
||||
op.execute("INSERT INTO project_old SELECT * FROM project")
|
||||
|
||||
# Drop the current table and rename
|
||||
op.drop_table("project")
|
||||
op.rename_table("project_old", "project")
|
||||
|
||||
# Recreate indexes
|
||||
with op.batch_alter_table("project", schema=None) as batch_op:
|
||||
batch_op.create_index("ix_project_created_at", ["created_at"], unique=False)
|
||||
batch_op.create_index("ix_project_name", ["name"], unique=True)
|
||||
batch_op.create_index("ix_project_path", ["path"], unique=False)
|
||||
batch_op.create_index("ix_project_permalink", ["permalink"], unique=True)
|
||||
batch_op.create_index("ix_project_updated_at", ["updated_at"], unique=False)
|
||||
@@ -56,12 +56,6 @@ def upgrade() -> None:
|
||||
);
|
||||
""")
|
||||
|
||||
# Print instruction to manually reindex after migration
|
||||
print("\n------------------------------------------------------------------")
|
||||
print("IMPORTANT: After migration completes, manually run the reindex command:")
|
||||
print("basic-memory sync")
|
||||
print("------------------------------------------------------------------\n")
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
"""Downgrade database schema to use old search index."""
|
||||
|
||||
+43
-13
@@ -1,29 +1,50 @@
|
||||
"""FastAPI application for basic-memory knowledge graph API."""
|
||||
|
||||
import asyncio
|
||||
from contextlib import asynccontextmanager
|
||||
|
||||
from fastapi import FastAPI, HTTPException
|
||||
from fastapi.exception_handlers import http_exception_handler
|
||||
from loguru import logger
|
||||
|
||||
from basic_memory import __version__ as version
|
||||
from basic_memory import db
|
||||
from basic_memory.api.routers import knowledge, memory, project_info, resource, search
|
||||
from basic_memory.config import config as project_config
|
||||
from basic_memory.services.initialization import initialize_app
|
||||
from basic_memory.api.routers import (
|
||||
directory_router,
|
||||
importer_router,
|
||||
knowledge,
|
||||
management,
|
||||
memory,
|
||||
project,
|
||||
resource,
|
||||
search,
|
||||
prompt_router,
|
||||
)
|
||||
from basic_memory.config import app_config
|
||||
from basic_memory.services.initialization import initialize_app, initialize_file_sync
|
||||
|
||||
|
||||
@asynccontextmanager
|
||||
async def lifespan(app: FastAPI): # pragma: no cover
|
||||
"""Lifecycle manager for the FastAPI app."""
|
||||
# Initialize database and file sync services
|
||||
watch_task = await initialize_app(project_config)
|
||||
# Initialize app and database
|
||||
logger.info("Starting Basic Memory API")
|
||||
await initialize_app(app_config)
|
||||
|
||||
logger.info(f"Sync changes enabled: {app_config.sync_changes}")
|
||||
if app_config.sync_changes:
|
||||
# start file sync task in background
|
||||
app.state.sync_task = asyncio.create_task(initialize_file_sync(app_config))
|
||||
else:
|
||||
logger.info("Sync changes disabled. Skipping file sync service.")
|
||||
|
||||
# proceed with startup
|
||||
yield
|
||||
|
||||
logger.info("Shutting down Basic Memory API")
|
||||
if watch_task:
|
||||
watch_task.cancel()
|
||||
if app.state.sync_task:
|
||||
logger.info("Stopping sync...")
|
||||
app.state.sync_task.cancel() # pyright: ignore
|
||||
|
||||
await db.shutdown_db()
|
||||
|
||||
@@ -32,17 +53,26 @@ async def lifespan(app: FastAPI): # pragma: no cover
|
||||
app = FastAPI(
|
||||
title="Basic Memory API",
|
||||
description="Knowledge graph API for basic-memory",
|
||||
version="0.1.0",
|
||||
version=version,
|
||||
lifespan=lifespan,
|
||||
)
|
||||
|
||||
|
||||
# Include routers
|
||||
app.include_router(knowledge.router)
|
||||
app.include_router(search.router)
|
||||
app.include_router(memory.router)
|
||||
app.include_router(resource.router)
|
||||
app.include_router(project_info.router)
|
||||
app.include_router(knowledge.router, prefix="/{project}")
|
||||
app.include_router(memory.router, prefix="/{project}")
|
||||
app.include_router(resource.router, prefix="/{project}")
|
||||
app.include_router(search.router, prefix="/{project}")
|
||||
app.include_router(project.project_router, prefix="/{project}")
|
||||
app.include_router(directory_router.router, prefix="/{project}")
|
||||
app.include_router(prompt_router.router, prefix="/{project}")
|
||||
app.include_router(importer_router.router, prefix="/{project}")
|
||||
|
||||
# Project resource router works accross projects
|
||||
app.include_router(project.project_resource_router)
|
||||
app.include_router(management.router)
|
||||
|
||||
# Auth routes are handled by FastMCP automatically when auth is enabled
|
||||
|
||||
|
||||
@app.exception_handler(Exception)
|
||||
|
||||
@@ -1,9 +1,11 @@
|
||||
"""API routers."""
|
||||
|
||||
from . import knowledge_router as knowledge
|
||||
from . import management_router as management
|
||||
from . import memory_router as memory
|
||||
from . import project_router as project
|
||||
from . import resource_router as resource
|
||||
from . import search_router as search
|
||||
from . import project_info_router as project_info
|
||||
from . import prompt_router as prompt
|
||||
|
||||
__all__ = ["knowledge", "memory", "resource", "search", "project_info"]
|
||||
__all__ = ["knowledge", "management", "memory", "project", "resource", "search", "prompt"]
|
||||
|
||||
@@ -0,0 +1,63 @@
|
||||
"""Router for directory tree operations."""
|
||||
|
||||
from typing import List, Optional
|
||||
|
||||
from fastapi import APIRouter, Query
|
||||
|
||||
from basic_memory.deps import DirectoryServiceDep, ProjectIdDep
|
||||
from basic_memory.schemas.directory import DirectoryNode
|
||||
|
||||
router = APIRouter(prefix="/directory", tags=["directory"])
|
||||
|
||||
|
||||
@router.get("/tree", response_model=DirectoryNode)
|
||||
async def get_directory_tree(
|
||||
directory_service: DirectoryServiceDep,
|
||||
project_id: ProjectIdDep,
|
||||
):
|
||||
"""Get hierarchical directory structure from the knowledge base.
|
||||
|
||||
Args:
|
||||
directory_service: Service for directory operations
|
||||
project_id: ID of the current project
|
||||
|
||||
Returns:
|
||||
DirectoryNode representing the root of the hierarchical tree structure
|
||||
"""
|
||||
# Get a hierarchical directory tree for the specific project
|
||||
tree = await directory_service.get_directory_tree()
|
||||
|
||||
# Return the hierarchical tree
|
||||
return tree
|
||||
|
||||
|
||||
@router.get("/list", response_model=List[DirectoryNode])
|
||||
async def list_directory(
|
||||
directory_service: DirectoryServiceDep,
|
||||
project_id: ProjectIdDep,
|
||||
dir_name: str = Query("/", description="Directory path to list"),
|
||||
depth: int = Query(1, ge=1, le=10, description="Recursion depth (1-10)"),
|
||||
file_name_glob: Optional[str] = Query(
|
||||
None, description="Glob pattern for filtering file names"
|
||||
),
|
||||
):
|
||||
"""List directory contents with filtering and depth control.
|
||||
|
||||
Args:
|
||||
directory_service: Service for directory operations
|
||||
project_id: ID of the current project
|
||||
dir_name: Directory path to list (default: root "/")
|
||||
depth: Recursion depth (1-10, default: 1 for immediate children only)
|
||||
file_name_glob: Optional glob pattern for filtering file names (e.g., "*.md", "*meeting*")
|
||||
|
||||
Returns:
|
||||
List of DirectoryNode objects matching the criteria
|
||||
"""
|
||||
# Get directory listing with filtering
|
||||
nodes = await directory_service.list_directory(
|
||||
dir_name=dir_name,
|
||||
depth=depth,
|
||||
file_name_glob=file_name_glob,
|
||||
)
|
||||
|
||||
return nodes
|
||||
@@ -0,0 +1,152 @@
|
||||
"""Import router for Basic Memory API."""
|
||||
|
||||
import json
|
||||
import logging
|
||||
|
||||
from fastapi import APIRouter, Form, HTTPException, UploadFile, status
|
||||
|
||||
from basic_memory.deps import (
|
||||
ChatGPTImporterDep,
|
||||
ClaudeConversationsImporterDep,
|
||||
ClaudeProjectsImporterDep,
|
||||
MemoryJsonImporterDep,
|
||||
)
|
||||
from basic_memory.importers import Importer
|
||||
from basic_memory.schemas.importer import (
|
||||
ChatImportResult,
|
||||
EntityImportResult,
|
||||
ProjectImportResult,
|
||||
)
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
router = APIRouter(prefix="/import", tags=["import"])
|
||||
|
||||
|
||||
@router.post("/chatgpt", response_model=ChatImportResult)
|
||||
async def import_chatgpt(
|
||||
importer: ChatGPTImporterDep,
|
||||
file: UploadFile,
|
||||
folder: str = Form("conversations"),
|
||||
) -> ChatImportResult:
|
||||
"""Import conversations from ChatGPT JSON export.
|
||||
|
||||
Args:
|
||||
file: The ChatGPT conversations.json file.
|
||||
folder: The folder to place the files in.
|
||||
markdown_processor: MarkdownProcessor instance.
|
||||
|
||||
Returns:
|
||||
ChatImportResult with import statistics.
|
||||
|
||||
Raises:
|
||||
HTTPException: If import fails.
|
||||
"""
|
||||
return await import_file(importer, file, folder)
|
||||
|
||||
|
||||
@router.post("/claude/conversations", response_model=ChatImportResult)
|
||||
async def import_claude_conversations(
|
||||
importer: ClaudeConversationsImporterDep,
|
||||
file: UploadFile,
|
||||
folder: str = Form("conversations"),
|
||||
) -> ChatImportResult:
|
||||
"""Import conversations from Claude conversations.json export.
|
||||
|
||||
Args:
|
||||
file: The Claude conversations.json file.
|
||||
folder: The folder to place the files in.
|
||||
markdown_processor: MarkdownProcessor instance.
|
||||
|
||||
Returns:
|
||||
ChatImportResult with import statistics.
|
||||
|
||||
Raises:
|
||||
HTTPException: If import fails.
|
||||
"""
|
||||
return await import_file(importer, file, folder)
|
||||
|
||||
|
||||
@router.post("/claude/projects", response_model=ProjectImportResult)
|
||||
async def import_claude_projects(
|
||||
importer: ClaudeProjectsImporterDep,
|
||||
file: UploadFile,
|
||||
folder: str = Form("projects"),
|
||||
) -> ProjectImportResult:
|
||||
"""Import projects from Claude projects.json export.
|
||||
|
||||
Args:
|
||||
file: The Claude projects.json file.
|
||||
base_folder: The base folder to place the files in.
|
||||
markdown_processor: MarkdownProcessor instance.
|
||||
|
||||
Returns:
|
||||
ProjectImportResult with import statistics.
|
||||
|
||||
Raises:
|
||||
HTTPException: If import fails.
|
||||
"""
|
||||
return await import_file(importer, file, folder)
|
||||
|
||||
|
||||
@router.post("/memory-json", response_model=EntityImportResult)
|
||||
async def import_memory_json(
|
||||
importer: MemoryJsonImporterDep,
|
||||
file: UploadFile,
|
||||
folder: str = Form("conversations"),
|
||||
) -> EntityImportResult:
|
||||
"""Import entities and relations from a memory.json file.
|
||||
|
||||
Args:
|
||||
file: The memory.json file.
|
||||
destination_folder: Optional destination folder within the project.
|
||||
markdown_processor: MarkdownProcessor instance.
|
||||
|
||||
Returns:
|
||||
EntityImportResult with import statistics.
|
||||
|
||||
Raises:
|
||||
HTTPException: If import fails.
|
||||
"""
|
||||
try:
|
||||
file_data = []
|
||||
file_bytes = await file.read()
|
||||
file_str = file_bytes.decode("utf-8")
|
||||
for line in file_str.splitlines():
|
||||
json_data = json.loads(line)
|
||||
file_data.append(json_data)
|
||||
|
||||
result = await importer.import_data(file_data, folder)
|
||||
if not result.success: # pragma: no cover
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
|
||||
detail=result.error_message or "Import failed",
|
||||
)
|
||||
except Exception as e:
|
||||
logger.exception("Import failed")
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
|
||||
detail=f"Import failed: {str(e)}",
|
||||
)
|
||||
return result
|
||||
|
||||
|
||||
async def import_file(importer: Importer, file: UploadFile, destination_folder: str):
|
||||
try:
|
||||
# Process file
|
||||
json_data = json.load(file.file)
|
||||
result = await importer.import_data(json_data, destination_folder)
|
||||
if not result.success: # pragma: no cover
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
|
||||
detail=result.error_message or "Import failed",
|
||||
)
|
||||
|
||||
return result
|
||||
|
||||
except Exception as e:
|
||||
logger.exception("Import failed")
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
|
||||
detail=f"Import failed: {str(e)}",
|
||||
)
|
||||
@@ -10,6 +10,11 @@ from basic_memory.deps import (
|
||||
get_search_service,
|
||||
SearchServiceDep,
|
||||
LinkResolverDep,
|
||||
ProjectPathDep,
|
||||
FileServiceDep,
|
||||
ProjectConfigDep,
|
||||
AppConfigDep,
|
||||
SyncServiceDep,
|
||||
)
|
||||
from basic_memory.schemas import (
|
||||
EntityListResponse,
|
||||
@@ -17,8 +22,8 @@ from basic_memory.schemas import (
|
||||
DeleteEntitiesResponse,
|
||||
DeleteEntitiesRequest,
|
||||
)
|
||||
from basic_memory.schemas.request import EditEntityRequest, MoveEntityRequest
|
||||
from basic_memory.schemas.base import Permalink, Entity
|
||||
from basic_memory.services.exceptions import EntityNotFoundError
|
||||
|
||||
router = APIRouter(prefix="/knowledge", tags=["knowledge"])
|
||||
|
||||
@@ -44,43 +49,37 @@ async def create_entity(
|
||||
result = EntityResponse.model_validate(entity)
|
||||
|
||||
logger.info(
|
||||
"API response",
|
||||
endpoint="create_entity",
|
||||
title=result.title,
|
||||
permalink=result.permalink,
|
||||
status_code=201,
|
||||
f"API response: endpoint='create_entity' title={result.title}, permalink={result.permalink}, status_code=201"
|
||||
)
|
||||
return result
|
||||
|
||||
|
||||
@router.put("/entities/{permalink:path}", response_model=EntityResponse)
|
||||
async def create_or_update_entity(
|
||||
project: ProjectPathDep,
|
||||
permalink: Permalink,
|
||||
data: Entity,
|
||||
response: Response,
|
||||
background_tasks: BackgroundTasks,
|
||||
entity_service: EntityServiceDep,
|
||||
search_service: SearchServiceDep,
|
||||
file_service: FileServiceDep,
|
||||
sync_service: SyncServiceDep,
|
||||
) -> EntityResponse:
|
||||
"""Create or update an entity. If entity exists, it will be updated, otherwise created."""
|
||||
logger.info(
|
||||
"API request",
|
||||
endpoint="create_or_update_entity",
|
||||
permalink=permalink,
|
||||
entity_type=data.entity_type,
|
||||
title=data.title,
|
||||
f"API request: create_or_update_entity for {project=}, {permalink=}, {data.entity_type=}, {data.title=}"
|
||||
)
|
||||
|
||||
# Validate permalink matches
|
||||
if data.permalink != permalink:
|
||||
logger.warning(
|
||||
"API validation error",
|
||||
endpoint="create_or_update_entity",
|
||||
permalink=permalink,
|
||||
data_permalink=data.permalink,
|
||||
error="Permalink mismatch",
|
||||
f"API validation error: creating/updating entity with permalink mismatch - url={permalink}, data={data.permalink}",
|
||||
)
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
detail=f"Entity permalink {data.permalink} must match URL path: '{permalink}'",
|
||||
)
|
||||
raise HTTPException(status_code=400, detail="Entity permalink must match URL path")
|
||||
|
||||
# Try create_or_update operation
|
||||
entity, created = await entity_service.create_or_update_entity(data)
|
||||
@@ -88,41 +87,144 @@ async def create_or_update_entity(
|
||||
|
||||
# reindex
|
||||
await search_service.index_entity(entity, background_tasks=background_tasks)
|
||||
|
||||
# Attempt immediate relation resolution when creating new entities
|
||||
# This helps resolve forward references when related entities are created in the same session
|
||||
if created:
|
||||
try:
|
||||
await sync_service.resolve_relations()
|
||||
logger.debug(f"Resolved relations after creating entity: {entity.permalink}")
|
||||
except Exception as e: # pragma: no cover
|
||||
# Don't fail the entire request if relation resolution fails
|
||||
logger.warning(f"Failed to resolve relations after entity creation: {e}")
|
||||
|
||||
result = EntityResponse.model_validate(entity)
|
||||
|
||||
logger.info(
|
||||
"API response",
|
||||
endpoint="create_or_update_entity",
|
||||
title=result.title,
|
||||
permalink=result.permalink,
|
||||
created=created,
|
||||
status_code=response.status_code,
|
||||
f"API response: {result.title=}, {result.permalink=}, {created=}, status_code={response.status_code}"
|
||||
)
|
||||
return result
|
||||
|
||||
|
||||
@router.patch("/entities/{identifier:path}", response_model=EntityResponse)
|
||||
async def edit_entity(
|
||||
identifier: str,
|
||||
data: EditEntityRequest,
|
||||
background_tasks: BackgroundTasks,
|
||||
entity_service: EntityServiceDep,
|
||||
search_service: SearchServiceDep,
|
||||
) -> EntityResponse:
|
||||
"""Edit an existing entity using various operations like append, prepend, find_replace, or replace_section.
|
||||
|
||||
This endpoint allows for targeted edits without requiring the full entity content.
|
||||
"""
|
||||
logger.info(
|
||||
f"API request: endpoint='edit_entity', identifier='{identifier}', operation='{data.operation}'"
|
||||
)
|
||||
|
||||
try:
|
||||
# Edit the entity using the service
|
||||
entity = await entity_service.edit_entity(
|
||||
identifier=identifier,
|
||||
operation=data.operation,
|
||||
content=data.content,
|
||||
section=data.section,
|
||||
find_text=data.find_text,
|
||||
expected_replacements=data.expected_replacements,
|
||||
)
|
||||
|
||||
# Reindex the updated entity
|
||||
await search_service.index_entity(entity, background_tasks=background_tasks)
|
||||
|
||||
# Return the updated entity response
|
||||
result = EntityResponse.model_validate(entity)
|
||||
|
||||
logger.info(
|
||||
"API response",
|
||||
endpoint="edit_entity",
|
||||
identifier=identifier,
|
||||
operation=data.operation,
|
||||
permalink=result.permalink,
|
||||
status_code=200,
|
||||
)
|
||||
|
||||
return result
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Error editing entity: {e}")
|
||||
raise HTTPException(status_code=400, detail=str(e))
|
||||
|
||||
|
||||
@router.post("/move")
|
||||
async def move_entity(
|
||||
data: MoveEntityRequest,
|
||||
background_tasks: BackgroundTasks,
|
||||
entity_service: EntityServiceDep,
|
||||
project_config: ProjectConfigDep,
|
||||
app_config: AppConfigDep,
|
||||
search_service: SearchServiceDep,
|
||||
) -> EntityResponse:
|
||||
"""Move an entity to a new file location with project consistency.
|
||||
|
||||
This endpoint moves a note to a different path while maintaining project
|
||||
consistency and optionally updating permalinks based on configuration.
|
||||
"""
|
||||
logger.info(
|
||||
f"API request: endpoint='move_entity', identifier='{data.identifier}', destination='{data.destination_path}'"
|
||||
)
|
||||
|
||||
try:
|
||||
# Move the entity using the service
|
||||
moved_entity = await entity_service.move_entity(
|
||||
identifier=data.identifier,
|
||||
destination_path=data.destination_path,
|
||||
project_config=project_config,
|
||||
app_config=app_config,
|
||||
)
|
||||
|
||||
# Get the moved entity to reindex it
|
||||
entity = await entity_service.link_resolver.resolve_link(data.destination_path)
|
||||
if entity:
|
||||
await search_service.index_entity(entity, background_tasks=background_tasks)
|
||||
|
||||
logger.info(
|
||||
"API response",
|
||||
endpoint="move_entity",
|
||||
identifier=data.identifier,
|
||||
destination=data.destination_path,
|
||||
status_code=200,
|
||||
)
|
||||
result = EntityResponse.model_validate(moved_entity)
|
||||
return result
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Error moving entity: {e}")
|
||||
raise HTTPException(status_code=400, detail=str(e))
|
||||
|
||||
|
||||
## Read endpoints
|
||||
|
||||
|
||||
@router.get("/entities/{permalink:path}", response_model=EntityResponse)
|
||||
@router.get("/entities/{identifier:path}", response_model=EntityResponse)
|
||||
async def get_entity(
|
||||
entity_service: EntityServiceDep,
|
||||
permalink: str,
|
||||
link_resolver: LinkResolverDep,
|
||||
identifier: str,
|
||||
) -> EntityResponse:
|
||||
"""Get a specific entity by ID.
|
||||
"""Get a specific entity by file path or permalink..
|
||||
|
||||
Args:
|
||||
permalink: Entity path ID
|
||||
content: If True, include full file content
|
||||
identifier: Entity file path or permalink
|
||||
:param entity_service: EntityService
|
||||
:param link_resolver: LinkResolver
|
||||
"""
|
||||
logger.info(f"request: get_entity with permalink={permalink}")
|
||||
try:
|
||||
entity = await entity_service.get_by_permalink(permalink)
|
||||
result = EntityResponse.model_validate(entity)
|
||||
return result
|
||||
except EntityNotFoundError:
|
||||
raise HTTPException(status_code=404, detail=f"Entity with {permalink} not found")
|
||||
logger.info(f"request: get_entity with identifier={identifier}")
|
||||
entity = await link_resolver.resolve_link(identifier)
|
||||
if not entity:
|
||||
raise HTTPException(status_code=404, detail=f"Entity {identifier} not found")
|
||||
|
||||
result = EntityResponse.model_validate(entity)
|
||||
return result
|
||||
|
||||
|
||||
@router.get("/entities", response_model=EntityListResponse)
|
||||
@@ -161,8 +263,8 @@ async def delete_entity(
|
||||
# Delete the entity
|
||||
deleted = await entity_service.delete_entity(entity.permalink or entity.id)
|
||||
|
||||
# Remove from search index
|
||||
background_tasks.add_task(search_service.delete_by_permalink, entity.permalink)
|
||||
# Remove from search index (entity, observations, and relations)
|
||||
background_tasks.add_task(search_service.handle_delete, entity)
|
||||
|
||||
result = DeleteEntitiesResponse(deleted=deleted)
|
||||
return result
|
||||
|
||||
@@ -0,0 +1,78 @@
|
||||
"""Management router for basic-memory API."""
|
||||
|
||||
import asyncio
|
||||
|
||||
from fastapi import APIRouter, Request
|
||||
from loguru import logger
|
||||
from pydantic import BaseModel
|
||||
|
||||
from basic_memory.config import app_config
|
||||
from basic_memory.deps import SyncServiceDep, ProjectRepositoryDep
|
||||
|
||||
router = APIRouter(prefix="/management", tags=["management"])
|
||||
|
||||
|
||||
class WatchStatusResponse(BaseModel):
|
||||
"""Response model for watch status."""
|
||||
|
||||
running: bool
|
||||
"""Whether the watch service is currently running."""
|
||||
|
||||
|
||||
@router.get("/watch/status", response_model=WatchStatusResponse)
|
||||
async def get_watch_status(request: Request) -> WatchStatusResponse:
|
||||
"""Get the current status of the watch service."""
|
||||
return WatchStatusResponse(
|
||||
running=request.app.state.watch_task is not None and not request.app.state.watch_task.done()
|
||||
)
|
||||
|
||||
|
||||
@router.post("/watch/start", response_model=WatchStatusResponse)
|
||||
async def start_watch_service(
|
||||
request: Request, project_repository: ProjectRepositoryDep, sync_service: SyncServiceDep
|
||||
) -> WatchStatusResponse:
|
||||
"""Start the watch service if it's not already running."""
|
||||
|
||||
# needed because of circular imports from sync -> app
|
||||
from basic_memory.sync import WatchService
|
||||
from basic_memory.sync.background_sync import create_background_sync_task
|
||||
|
||||
if request.app.state.watch_task is not None and not request.app.state.watch_task.done():
|
||||
# Watch service is already running
|
||||
return WatchStatusResponse(running=True)
|
||||
|
||||
# Create and start a new watch service
|
||||
logger.info("Starting watch service via management API")
|
||||
|
||||
# Get services needed for the watch task
|
||||
watch_service = WatchService(
|
||||
app_config=app_config,
|
||||
project_repository=project_repository,
|
||||
)
|
||||
|
||||
# Create and store the task
|
||||
watch_task = create_background_sync_task(sync_service, watch_service)
|
||||
request.app.state.watch_task = watch_task
|
||||
|
||||
return WatchStatusResponse(running=True)
|
||||
|
||||
|
||||
@router.post("/watch/stop", response_model=WatchStatusResponse)
|
||||
async def stop_watch_service(request: Request) -> WatchStatusResponse: # pragma: no cover
|
||||
"""Stop the watch service if it's running."""
|
||||
if request.app.state.watch_task is None or request.app.state.watch_task.done():
|
||||
# Watch service is not running
|
||||
return WatchStatusResponse(running=False)
|
||||
|
||||
# Cancel the running task
|
||||
logger.info("Stopping watch service via management API")
|
||||
request.app.state.watch_task.cancel()
|
||||
|
||||
# Wait for it to be properly cancelled
|
||||
try:
|
||||
await request.app.state.watch_task
|
||||
except asyncio.CancelledError:
|
||||
pass
|
||||
|
||||
request.app.state.watch_task = None
|
||||
return WatchStatusResponse(running=False)
|
||||
@@ -1,78 +1,22 @@
|
||||
"""Routes for memory:// URI operations."""
|
||||
|
||||
from typing import Annotated
|
||||
from typing import Annotated, Optional
|
||||
|
||||
from dateparser import parse
|
||||
from fastapi import APIRouter, Query
|
||||
from loguru import logger
|
||||
|
||||
from basic_memory.deps import ContextServiceDep, EntityRepositoryDep
|
||||
from basic_memory.repository import EntityRepository
|
||||
from basic_memory.repository.search_repository import SearchIndexRow
|
||||
from basic_memory.schemas.base import TimeFrame
|
||||
from basic_memory.schemas.base import TimeFrame, parse_timeframe
|
||||
from basic_memory.schemas.memory import (
|
||||
GraphContext,
|
||||
RelationSummary,
|
||||
EntitySummary,
|
||||
ObservationSummary,
|
||||
MemoryMetadata,
|
||||
normalize_memory_url,
|
||||
)
|
||||
from basic_memory.schemas.search import SearchItemType
|
||||
from basic_memory.services.context_service import ContextResultRow
|
||||
from basic_memory.api.routers.utils import to_graph_context
|
||||
|
||||
router = APIRouter(prefix="/memory", tags=["memory"])
|
||||
|
||||
|
||||
async def to_graph_context(context, entity_repository: EntityRepository, page: int, page_size: int):
|
||||
# return results
|
||||
async def to_summary(item: SearchIndexRow | ContextResultRow):
|
||||
match item.type:
|
||||
case SearchItemType.ENTITY:
|
||||
return EntitySummary(
|
||||
title=item.title, # pyright: ignore
|
||||
permalink=item.permalink,
|
||||
content=item.content,
|
||||
file_path=item.file_path,
|
||||
created_at=item.created_at,
|
||||
)
|
||||
case SearchItemType.OBSERVATION:
|
||||
return ObservationSummary(
|
||||
title=item.title, # pyright: ignore
|
||||
file_path=item.file_path,
|
||||
category=item.category, # pyright: ignore
|
||||
content=item.content, # pyright: ignore
|
||||
permalink=item.permalink, # pyright: ignore
|
||||
created_at=item.created_at,
|
||||
)
|
||||
case SearchItemType.RELATION:
|
||||
from_entity = await entity_repository.find_by_id(item.from_id) # pyright: ignore
|
||||
to_entity = await entity_repository.find_by_id(item.to_id) if item.to_id else None
|
||||
return RelationSummary(
|
||||
title=item.title, # pyright: ignore
|
||||
file_path=item.file_path,
|
||||
permalink=item.permalink, # pyright: ignore
|
||||
relation_type=item.type,
|
||||
from_entity=from_entity.permalink, # pyright: ignore
|
||||
to_entity=to_entity.permalink if to_entity else None,
|
||||
created_at=item.created_at,
|
||||
)
|
||||
case _: # pragma: no cover
|
||||
raise ValueError(f"Unexpected type: {item.type}")
|
||||
|
||||
primary_results = [await to_summary(r) for r in context["primary_results"]]
|
||||
related_results = [await to_summary(r) for r in context["related_results"]]
|
||||
metadata = MemoryMetadata.model_validate(context["metadata"])
|
||||
# Transform to GraphContext
|
||||
return GraphContext(
|
||||
primary_results=primary_results,
|
||||
related_results=related_results,
|
||||
metadata=metadata,
|
||||
page=page,
|
||||
page_size=page_size,
|
||||
)
|
||||
|
||||
|
||||
@router.get("/recent", response_model=GraphContext)
|
||||
async def recent(
|
||||
context_service: ContextServiceDep,
|
||||
@@ -95,7 +39,7 @@ async def recent(
|
||||
f"Getting recent context: `{types}` depth: `{depth}` timeframe: `{timeframe}` page: `{page}` page_size: `{page_size}` max_related: `{max_related}`"
|
||||
)
|
||||
# Parse timeframe
|
||||
since = parse(timeframe)
|
||||
since = parse_timeframe(timeframe)
|
||||
limit = page_size
|
||||
offset = (page - 1) * page_size
|
||||
|
||||
@@ -119,7 +63,7 @@ async def get_memory_context(
|
||||
entity_repository: EntityRepositoryDep,
|
||||
uri: str,
|
||||
depth: int = 1,
|
||||
timeframe: TimeFrame = "7d",
|
||||
timeframe: Optional[TimeFrame] = None,
|
||||
page: int = 1,
|
||||
page_size: int = 10,
|
||||
max_related: int = 10,
|
||||
@@ -133,7 +77,7 @@ async def get_memory_context(
|
||||
memory_url = normalize_memory_url(uri)
|
||||
|
||||
# Parse timeframe
|
||||
since = parse(timeframe)
|
||||
since = parse_timeframe(timeframe) if timeframe else None
|
||||
limit = page_size
|
||||
offset = (page - 1) * page_size
|
||||
|
||||
|
||||
@@ -1,274 +0,0 @@
|
||||
"""Router for statistics and system information."""
|
||||
|
||||
import json
|
||||
from datetime import datetime
|
||||
|
||||
from basic_memory.config import config, config_manager
|
||||
from basic_memory.deps import (
|
||||
ProjectInfoRepositoryDep,
|
||||
)
|
||||
from basic_memory.repository.project_info_repository import ProjectInfoRepository
|
||||
from basic_memory.schemas import (
|
||||
ProjectInfoResponse,
|
||||
ProjectStatistics,
|
||||
ActivityMetrics,
|
||||
SystemStatus,
|
||||
)
|
||||
from basic_memory.sync.watch_service import WATCH_STATUS_JSON
|
||||
from fastapi import APIRouter
|
||||
from sqlalchemy import text
|
||||
|
||||
router = APIRouter(prefix="/stats", tags=["statistics"])
|
||||
|
||||
|
||||
@router.get("/project-info", response_model=ProjectInfoResponse)
|
||||
async def get_project_info(
|
||||
repository: ProjectInfoRepositoryDep,
|
||||
) -> ProjectInfoResponse:
|
||||
"""Get comprehensive information about the current Basic Memory project."""
|
||||
# Get statistics
|
||||
statistics = await get_statistics(repository)
|
||||
|
||||
# Get activity metrics
|
||||
activity = await get_activity_metrics(repository)
|
||||
|
||||
# Get system status
|
||||
system = await get_system_status()
|
||||
|
||||
# Get project configuration information
|
||||
project_name = config.project
|
||||
project_path = str(config.home)
|
||||
available_projects = config_manager.projects
|
||||
default_project = config_manager.default_project
|
||||
|
||||
# Construct the response
|
||||
return ProjectInfoResponse(
|
||||
project_name=project_name,
|
||||
project_path=project_path,
|
||||
available_projects=available_projects,
|
||||
default_project=default_project,
|
||||
statistics=statistics,
|
||||
activity=activity,
|
||||
system=system,
|
||||
)
|
||||
|
||||
|
||||
async def get_statistics(repository: ProjectInfoRepository) -> ProjectStatistics:
|
||||
"""Get statistics about the current project."""
|
||||
# Get basic counts
|
||||
entity_count_result = await repository.execute_query(text("SELECT COUNT(*) FROM entity"))
|
||||
total_entities = entity_count_result.scalar() or 0
|
||||
|
||||
observation_count_result = await repository.execute_query(
|
||||
text("SELECT COUNT(*) FROM observation")
|
||||
)
|
||||
total_observations = observation_count_result.scalar() or 0
|
||||
|
||||
relation_count_result = await repository.execute_query(text("SELECT COUNT(*) FROM relation"))
|
||||
total_relations = relation_count_result.scalar() or 0
|
||||
|
||||
unresolved_count_result = await repository.execute_query(
|
||||
text("SELECT COUNT(*) FROM relation WHERE to_id IS NULL")
|
||||
)
|
||||
total_unresolved = unresolved_count_result.scalar() or 0
|
||||
|
||||
# Get entity counts by type
|
||||
entity_types_result = await repository.execute_query(
|
||||
text("SELECT entity_type, COUNT(*) FROM entity GROUP BY entity_type")
|
||||
)
|
||||
entity_types = {row[0]: row[1] for row in entity_types_result.fetchall()}
|
||||
|
||||
# Get observation counts by category
|
||||
category_result = await repository.execute_query(
|
||||
text("SELECT category, COUNT(*) FROM observation GROUP BY category")
|
||||
)
|
||||
observation_categories = {row[0]: row[1] for row in category_result.fetchall()}
|
||||
|
||||
# Get relation counts by type
|
||||
relation_types_result = await repository.execute_query(
|
||||
text("SELECT relation_type, COUNT(*) FROM relation GROUP BY relation_type")
|
||||
)
|
||||
relation_types = {row[0]: row[1] for row in relation_types_result.fetchall()}
|
||||
|
||||
# Find most connected entities (most outgoing relations)
|
||||
connected_result = await repository.execute_query(
|
||||
text("""
|
||||
SELECT e.id, e.title, e.permalink, COUNT(r.id) AS relation_count
|
||||
FROM entity e
|
||||
JOIN relation r ON e.id = r.from_id
|
||||
GROUP BY e.id
|
||||
ORDER BY relation_count DESC
|
||||
LIMIT 10
|
||||
""")
|
||||
)
|
||||
most_connected = [
|
||||
{"id": row[0], "title": row[1], "permalink": row[2], "relation_count": row[3]}
|
||||
for row in connected_result.fetchall()
|
||||
]
|
||||
|
||||
# Count isolated entities (no relations)
|
||||
isolated_result = await repository.execute_query(
|
||||
text("""
|
||||
SELECT COUNT(e.id)
|
||||
FROM entity e
|
||||
LEFT JOIN relation r1 ON e.id = r1.from_id
|
||||
LEFT JOIN relation r2 ON e.id = r2.to_id
|
||||
WHERE r1.id IS NULL AND r2.id IS NULL
|
||||
""")
|
||||
)
|
||||
isolated_count = isolated_result.scalar() or 0
|
||||
|
||||
return ProjectStatistics(
|
||||
total_entities=total_entities,
|
||||
total_observations=total_observations,
|
||||
total_relations=total_relations,
|
||||
total_unresolved_relations=total_unresolved,
|
||||
entity_types=entity_types,
|
||||
observation_categories=observation_categories,
|
||||
relation_types=relation_types,
|
||||
most_connected_entities=most_connected,
|
||||
isolated_entities=isolated_count,
|
||||
)
|
||||
|
||||
|
||||
async def get_activity_metrics(repository: ProjectInfoRepository) -> ActivityMetrics:
|
||||
"""Get activity metrics for the current project."""
|
||||
# Get recently created entities
|
||||
created_result = await repository.execute_query(
|
||||
text("""
|
||||
SELECT id, title, permalink, entity_type, created_at
|
||||
FROM entity
|
||||
ORDER BY created_at DESC
|
||||
LIMIT 10
|
||||
""")
|
||||
)
|
||||
recently_created = [
|
||||
{
|
||||
"id": row[0],
|
||||
"title": row[1],
|
||||
"permalink": row[2],
|
||||
"entity_type": row[3],
|
||||
"created_at": row[4],
|
||||
}
|
||||
for row in created_result.fetchall()
|
||||
]
|
||||
|
||||
# Get recently updated entities
|
||||
updated_result = await repository.execute_query(
|
||||
text("""
|
||||
SELECT id, title, permalink, entity_type, updated_at
|
||||
FROM entity
|
||||
ORDER BY updated_at DESC
|
||||
LIMIT 10
|
||||
""")
|
||||
)
|
||||
recently_updated = [
|
||||
{
|
||||
"id": row[0],
|
||||
"title": row[1],
|
||||
"permalink": row[2],
|
||||
"entity_type": row[3],
|
||||
"updated_at": row[4],
|
||||
}
|
||||
for row in updated_result.fetchall()
|
||||
]
|
||||
|
||||
# Get monthly growth over the last 6 months
|
||||
# Calculate the start of 6 months ago
|
||||
now = datetime.now()
|
||||
six_months_ago = datetime(
|
||||
now.year - (1 if now.month <= 6 else 0), ((now.month - 6) % 12) or 12, 1
|
||||
)
|
||||
|
||||
# Query for monthly entity creation
|
||||
entity_growth_result = await repository.execute_query(
|
||||
text(f"""
|
||||
SELECT
|
||||
strftime('%Y-%m', created_at) AS month,
|
||||
COUNT(*) AS count
|
||||
FROM entity
|
||||
WHERE created_at >= '{six_months_ago.isoformat()}'
|
||||
GROUP BY month
|
||||
ORDER BY month
|
||||
""")
|
||||
)
|
||||
entity_growth = {row[0]: row[1] for row in entity_growth_result.fetchall()}
|
||||
|
||||
# Query for monthly observation creation
|
||||
observation_growth_result = await repository.execute_query(
|
||||
text(f"""
|
||||
SELECT
|
||||
strftime('%Y-%m', created_at) AS month,
|
||||
COUNT(*) AS count
|
||||
FROM observation
|
||||
INNER JOIN entity ON observation.entity_id = entity.id
|
||||
WHERE entity.created_at >= '{six_months_ago.isoformat()}'
|
||||
GROUP BY month
|
||||
ORDER BY month
|
||||
""")
|
||||
)
|
||||
observation_growth = {row[0]: row[1] for row in observation_growth_result.fetchall()}
|
||||
|
||||
# Query for monthly relation creation
|
||||
relation_growth_result = await repository.execute_query(
|
||||
text(f"""
|
||||
SELECT
|
||||
strftime('%Y-%m', created_at) AS month,
|
||||
COUNT(*) AS count
|
||||
FROM relation
|
||||
INNER JOIN entity ON relation.from_id = entity.id
|
||||
WHERE entity.created_at >= '{six_months_ago.isoformat()}'
|
||||
GROUP BY month
|
||||
ORDER BY month
|
||||
""")
|
||||
)
|
||||
relation_growth = {row[0]: row[1] for row in relation_growth_result.fetchall()}
|
||||
|
||||
# Combine all monthly growth data
|
||||
monthly_growth = {}
|
||||
for month in set(
|
||||
list(entity_growth.keys()) + list(observation_growth.keys()) + list(relation_growth.keys())
|
||||
):
|
||||
monthly_growth[month] = {
|
||||
"entities": entity_growth.get(month, 0),
|
||||
"observations": observation_growth.get(month, 0),
|
||||
"relations": relation_growth.get(month, 0),
|
||||
"total": (
|
||||
entity_growth.get(month, 0)
|
||||
+ observation_growth.get(month, 0)
|
||||
+ relation_growth.get(month, 0)
|
||||
),
|
||||
}
|
||||
|
||||
return ActivityMetrics(
|
||||
recently_created=recently_created,
|
||||
recently_updated=recently_updated,
|
||||
monthly_growth=monthly_growth,
|
||||
)
|
||||
|
||||
|
||||
async def get_system_status() -> SystemStatus:
|
||||
"""Get system status information."""
|
||||
import basic_memory
|
||||
|
||||
# Get database information
|
||||
db_path = config.database_path
|
||||
db_size = db_path.stat().st_size if db_path.exists() else 0
|
||||
db_size_readable = f"{db_size / (1024 * 1024):.2f} MB"
|
||||
|
||||
# Get watch service status if available
|
||||
watch_status = None
|
||||
watch_status_path = config.home / ".basic-memory" / WATCH_STATUS_JSON
|
||||
if watch_status_path.exists():
|
||||
try:
|
||||
watch_status = json.loads(watch_status_path.read_text(encoding="utf-8"))
|
||||
except Exception: # pragma: no cover
|
||||
pass
|
||||
|
||||
return SystemStatus(
|
||||
version=basic_memory.__version__,
|
||||
database_path=str(db_path),
|
||||
database_size=db_size_readable,
|
||||
watch_status=watch_status,
|
||||
timestamp=datetime.now(),
|
||||
)
|
||||
@@ -0,0 +1,234 @@
|
||||
"""Router for project management."""
|
||||
|
||||
from fastapi import APIRouter, HTTPException, Path, Body
|
||||
from typing import Optional
|
||||
|
||||
from basic_memory.deps import ProjectServiceDep, ProjectPathDep
|
||||
from basic_memory.schemas import ProjectInfoResponse
|
||||
from basic_memory.schemas.project_info import (
|
||||
ProjectList,
|
||||
ProjectItem,
|
||||
ProjectInfoRequest,
|
||||
ProjectStatusResponse,
|
||||
)
|
||||
|
||||
# Router for resources in a specific project
|
||||
project_router = APIRouter(prefix="/project", tags=["project"])
|
||||
|
||||
# Router for managing project resources
|
||||
project_resource_router = APIRouter(prefix="/projects", tags=["project_management"])
|
||||
|
||||
|
||||
@project_router.get("/info", response_model=ProjectInfoResponse)
|
||||
async def get_project_info(
|
||||
project_service: ProjectServiceDep,
|
||||
project: ProjectPathDep,
|
||||
) -> ProjectInfoResponse:
|
||||
"""Get comprehensive information about the specified Basic Memory project."""
|
||||
return await project_service.get_project_info(project)
|
||||
|
||||
|
||||
# Update a project
|
||||
@project_router.patch("/{name}", response_model=ProjectStatusResponse)
|
||||
async def update_project(
|
||||
project_service: ProjectServiceDep,
|
||||
project_name: str = Path(..., description="Name of the project to update"),
|
||||
path: Optional[str] = Body(None, description="New path for the project"),
|
||||
is_active: Optional[bool] = Body(None, description="Status of the project (active/inactive)"),
|
||||
) -> ProjectStatusResponse:
|
||||
"""Update a project's information in configuration and database.
|
||||
|
||||
Args:
|
||||
project_name: The name of the project to update
|
||||
path: Optional new path for the project
|
||||
is_active: Optional status update for the project
|
||||
|
||||
Returns:
|
||||
Response confirming the project was updated
|
||||
"""
|
||||
try: # pragma: no cover
|
||||
# Get original project info for the response
|
||||
old_project_info = ProjectItem(
|
||||
name=project_name,
|
||||
path=project_service.projects.get(project_name, ""),
|
||||
)
|
||||
|
||||
await project_service.update_project(project_name, updated_path=path, is_active=is_active)
|
||||
|
||||
# Get updated project info
|
||||
updated_path = path if path else project_service.projects.get(project_name, "")
|
||||
|
||||
return ProjectStatusResponse(
|
||||
message=f"Project '{project_name}' updated successfully",
|
||||
status="success",
|
||||
default=(project_name == project_service.default_project),
|
||||
old_project=old_project_info,
|
||||
new_project=ProjectItem(name=project_name, path=updated_path),
|
||||
)
|
||||
except ValueError as e: # pragma: no cover
|
||||
raise HTTPException(status_code=400, detail=str(e))
|
||||
|
||||
|
||||
# List all available projects
|
||||
@project_resource_router.get("/projects", response_model=ProjectList)
|
||||
async def list_projects(
|
||||
project_service: ProjectServiceDep,
|
||||
) -> ProjectList:
|
||||
"""List all configured projects.
|
||||
|
||||
Returns:
|
||||
A list of all projects with metadata
|
||||
"""
|
||||
projects = await project_service.list_projects()
|
||||
default_project = project_service.default_project
|
||||
|
||||
project_items = [
|
||||
ProjectItem(
|
||||
name=project.name,
|
||||
path=project.path,
|
||||
is_default=project.is_default or False,
|
||||
)
|
||||
for project in projects
|
||||
]
|
||||
|
||||
return ProjectList(
|
||||
projects=project_items,
|
||||
default_project=default_project,
|
||||
)
|
||||
|
||||
|
||||
# Add a new project
|
||||
@project_resource_router.post("/projects", response_model=ProjectStatusResponse)
|
||||
async def add_project(
|
||||
project_data: ProjectInfoRequest,
|
||||
project_service: ProjectServiceDep,
|
||||
) -> ProjectStatusResponse:
|
||||
"""Add a new project to configuration and database.
|
||||
|
||||
Args:
|
||||
project_data: The project name and path, with option to set as default
|
||||
|
||||
Returns:
|
||||
Response confirming the project was added
|
||||
"""
|
||||
try: # pragma: no cover
|
||||
await project_service.add_project(
|
||||
project_data.name, project_data.path, set_default=project_data.set_default
|
||||
)
|
||||
|
||||
return ProjectStatusResponse( # pyright: ignore [reportCallIssue]
|
||||
message=f"Project '{project_data.name}' added successfully",
|
||||
status="success",
|
||||
default=project_data.set_default,
|
||||
new_project=ProjectItem(
|
||||
name=project_data.name, path=project_data.path, is_default=project_data.set_default
|
||||
),
|
||||
)
|
||||
except ValueError as e: # pragma: no cover
|
||||
raise HTTPException(status_code=400, detail=str(e))
|
||||
|
||||
|
||||
# Remove a project
|
||||
@project_resource_router.delete("/{name}", response_model=ProjectStatusResponse)
|
||||
async def remove_project(
|
||||
project_service: ProjectServiceDep,
|
||||
name: str = Path(..., description="Name of the project to remove"),
|
||||
) -> ProjectStatusResponse:
|
||||
"""Remove a project from configuration and database.
|
||||
|
||||
Args:
|
||||
name: The name of the project to remove
|
||||
|
||||
Returns:
|
||||
Response confirming the project was removed
|
||||
"""
|
||||
try:
|
||||
old_project = await project_service.get_project(name)
|
||||
if not old_project: # pragma: no cover
|
||||
raise HTTPException(
|
||||
status_code=404, detail=f"Project: '{name}' does not exist"
|
||||
) # pragma: no cover
|
||||
|
||||
await project_service.remove_project(name)
|
||||
|
||||
return ProjectStatusResponse(
|
||||
message=f"Project '{name}' removed successfully",
|
||||
status="success",
|
||||
default=False,
|
||||
old_project=ProjectItem(name=old_project.name, path=old_project.path),
|
||||
new_project=None,
|
||||
)
|
||||
except ValueError as e: # pragma: no cover
|
||||
raise HTTPException(status_code=400, detail=str(e))
|
||||
|
||||
|
||||
# Set a project as default
|
||||
@project_resource_router.put("/{name}/default", response_model=ProjectStatusResponse)
|
||||
async def set_default_project(
|
||||
project_service: ProjectServiceDep,
|
||||
name: str = Path(..., description="Name of the project to set as default"),
|
||||
) -> ProjectStatusResponse:
|
||||
"""Set a project as the default project.
|
||||
|
||||
Args:
|
||||
name: The name of the project to set as default
|
||||
|
||||
Returns:
|
||||
Response confirming the project was set as default
|
||||
"""
|
||||
try:
|
||||
# Get the old default project
|
||||
default_name = project_service.default_project
|
||||
default_project = await project_service.get_project(default_name)
|
||||
if not default_project: # pragma: no cover
|
||||
raise HTTPException( # pragma: no cover
|
||||
status_code=404, detail=f"Default Project: '{default_name}' does not exist"
|
||||
)
|
||||
|
||||
# get the new project
|
||||
new_default_project = await project_service.get_project(name)
|
||||
if not new_default_project: # pragma: no cover
|
||||
raise HTTPException(
|
||||
status_code=404, detail=f"Project: '{name}' does not exist"
|
||||
) # pragma: no cover
|
||||
|
||||
await project_service.set_default_project(name)
|
||||
|
||||
return ProjectStatusResponse(
|
||||
message=f"Project '{name}' set as default successfully",
|
||||
status="success",
|
||||
default=True,
|
||||
old_project=ProjectItem(name=default_name, path=default_project.path),
|
||||
new_project=ProjectItem(
|
||||
name=name,
|
||||
path=new_default_project.path,
|
||||
is_default=True,
|
||||
),
|
||||
)
|
||||
except ValueError as e: # pragma: no cover
|
||||
raise HTTPException(status_code=400, detail=str(e))
|
||||
|
||||
|
||||
# Synchronize projects between config and database
|
||||
@project_resource_router.post("/sync", response_model=ProjectStatusResponse)
|
||||
async def synchronize_projects(
|
||||
project_service: ProjectServiceDep,
|
||||
) -> ProjectStatusResponse:
|
||||
"""Synchronize projects between configuration file and database.
|
||||
|
||||
Ensures that all projects in the configuration file exist in the database
|
||||
and vice versa.
|
||||
|
||||
Returns:
|
||||
Response confirming synchronization was completed
|
||||
"""
|
||||
try: # pragma: no cover
|
||||
await project_service.synchronize_projects()
|
||||
|
||||
return ProjectStatusResponse( # pyright: ignore [reportCallIssue]
|
||||
message="Projects synchronized successfully between configuration and database",
|
||||
status="success",
|
||||
default=False,
|
||||
)
|
||||
except ValueError as e: # pragma: no cover
|
||||
raise HTTPException(status_code=400, detail=str(e))
|
||||
@@ -0,0 +1,260 @@
|
||||
"""Router for prompt-related operations.
|
||||
|
||||
This router is responsible for rendering various prompts using Handlebars templates.
|
||||
It centralizes all prompt formatting logic that was previously in the MCP prompts.
|
||||
"""
|
||||
|
||||
from datetime import datetime, timezone
|
||||
from fastapi import APIRouter, HTTPException, status
|
||||
from loguru import logger
|
||||
|
||||
from basic_memory.api.routers.utils import to_graph_context, to_search_results
|
||||
from basic_memory.api.template_loader import template_loader
|
||||
from basic_memory.schemas.base import parse_timeframe
|
||||
from basic_memory.deps import (
|
||||
ContextServiceDep,
|
||||
EntityRepositoryDep,
|
||||
SearchServiceDep,
|
||||
EntityServiceDep,
|
||||
)
|
||||
from basic_memory.schemas.prompt import (
|
||||
ContinueConversationRequest,
|
||||
SearchPromptRequest,
|
||||
PromptResponse,
|
||||
PromptMetadata,
|
||||
)
|
||||
from basic_memory.schemas.search import SearchItemType, SearchQuery
|
||||
|
||||
router = APIRouter(prefix="/prompt", tags=["prompt"])
|
||||
|
||||
|
||||
@router.post("/continue-conversation", response_model=PromptResponse)
|
||||
async def continue_conversation(
|
||||
search_service: SearchServiceDep,
|
||||
entity_service: EntityServiceDep,
|
||||
context_service: ContextServiceDep,
|
||||
entity_repository: EntityRepositoryDep,
|
||||
request: ContinueConversationRequest,
|
||||
) -> PromptResponse:
|
||||
"""Generate a prompt for continuing a conversation.
|
||||
|
||||
This endpoint takes a topic and/or timeframe and generates a prompt with
|
||||
relevant context from the knowledge base.
|
||||
|
||||
Args:
|
||||
request: The request parameters
|
||||
|
||||
Returns:
|
||||
Formatted continuation prompt with context
|
||||
"""
|
||||
logger.info(
|
||||
f"Generating continue conversation prompt, topic: {request.topic}, timeframe: {request.timeframe}"
|
||||
)
|
||||
|
||||
since = parse_timeframe(request.timeframe) if request.timeframe else None
|
||||
|
||||
# Initialize search results
|
||||
search_results = []
|
||||
|
||||
# Get data needed for template
|
||||
if request.topic:
|
||||
query = SearchQuery(text=request.topic, after_date=request.timeframe)
|
||||
results = await search_service.search(query, limit=request.search_items_limit)
|
||||
search_results = await to_search_results(entity_service, results)
|
||||
|
||||
# Build context from results
|
||||
all_hierarchical_results = []
|
||||
for result in search_results:
|
||||
if hasattr(result, "permalink") and result.permalink:
|
||||
# Get hierarchical context using the new dataclass-based approach
|
||||
context_result = await context_service.build_context(
|
||||
result.permalink,
|
||||
depth=request.depth,
|
||||
since=since,
|
||||
max_related=request.related_items_limit,
|
||||
include_observations=True, # Include observations for entities
|
||||
)
|
||||
|
||||
# Process results into the schema format
|
||||
graph_context = await to_graph_context(
|
||||
context_result, entity_repository=entity_repository
|
||||
)
|
||||
|
||||
# Add results to our collection (limit to top results for each permalink)
|
||||
if graph_context.results:
|
||||
all_hierarchical_results.extend(graph_context.results[:3])
|
||||
|
||||
# Limit to a reasonable number of total results
|
||||
all_hierarchical_results = all_hierarchical_results[:10]
|
||||
|
||||
template_context = {
|
||||
"topic": request.topic,
|
||||
"timeframe": request.timeframe,
|
||||
"hierarchical_results": all_hierarchical_results,
|
||||
"has_results": len(all_hierarchical_results) > 0,
|
||||
}
|
||||
else:
|
||||
# If no topic, get recent activity
|
||||
context_result = await context_service.build_context(
|
||||
types=[SearchItemType.ENTITY],
|
||||
depth=request.depth,
|
||||
since=since,
|
||||
max_related=request.related_items_limit,
|
||||
include_observations=True,
|
||||
)
|
||||
recent_context = await to_graph_context(context_result, entity_repository=entity_repository)
|
||||
|
||||
hierarchical_results = recent_context.results[:5] # Limit to top 5 recent items
|
||||
|
||||
template_context = {
|
||||
"topic": f"Recent Activity from ({request.timeframe})",
|
||||
"timeframe": request.timeframe,
|
||||
"hierarchical_results": hierarchical_results,
|
||||
"has_results": len(hierarchical_results) > 0,
|
||||
}
|
||||
|
||||
try:
|
||||
# Render template
|
||||
rendered_prompt = await template_loader.render(
|
||||
"prompts/continue_conversation.hbs", template_context
|
||||
)
|
||||
|
||||
# Calculate metadata
|
||||
# Count items of different types
|
||||
observation_count = 0
|
||||
relation_count = 0
|
||||
entity_count = 0
|
||||
|
||||
# Get the hierarchical results from the template context
|
||||
hierarchical_results_for_count = template_context.get("hierarchical_results", [])
|
||||
|
||||
# For topic-based search
|
||||
if request.topic:
|
||||
for item in hierarchical_results_for_count:
|
||||
if hasattr(item, "observations"):
|
||||
observation_count += len(item.observations) if item.observations else 0
|
||||
|
||||
if hasattr(item, "related_results"):
|
||||
for related in item.related_results or []:
|
||||
if hasattr(related, "type"):
|
||||
if related.type == "relation":
|
||||
relation_count += 1
|
||||
elif related.type == "entity": # pragma: no cover
|
||||
entity_count += 1 # pragma: no cover
|
||||
# For recent activity
|
||||
else:
|
||||
for item in hierarchical_results_for_count:
|
||||
if hasattr(item, "observations"):
|
||||
observation_count += len(item.observations) if item.observations else 0
|
||||
|
||||
if hasattr(item, "related_results"):
|
||||
for related in item.related_results or []:
|
||||
if hasattr(related, "type"):
|
||||
if related.type == "relation":
|
||||
relation_count += 1
|
||||
elif related.type == "entity": # pragma: no cover
|
||||
entity_count += 1 # pragma: no cover
|
||||
|
||||
# Build metadata
|
||||
metadata = {
|
||||
"query": request.topic,
|
||||
"timeframe": request.timeframe,
|
||||
"search_count": len(search_results)
|
||||
if request.topic
|
||||
else 0, # Original search results count
|
||||
"context_count": len(hierarchical_results_for_count),
|
||||
"observation_count": observation_count,
|
||||
"relation_count": relation_count,
|
||||
"total_items": (
|
||||
len(hierarchical_results_for_count)
|
||||
+ observation_count
|
||||
+ relation_count
|
||||
+ entity_count
|
||||
),
|
||||
"search_limit": request.search_items_limit,
|
||||
"context_depth": request.depth,
|
||||
"related_limit": request.related_items_limit,
|
||||
"generated_at": datetime.now(timezone.utc).isoformat(),
|
||||
}
|
||||
|
||||
prompt_metadata = PromptMetadata(**metadata)
|
||||
|
||||
return PromptResponse(
|
||||
prompt=rendered_prompt, context=template_context, metadata=prompt_metadata
|
||||
)
|
||||
except Exception as e:
|
||||
logger.error(f"Error rendering continue conversation template: {e}")
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
|
||||
detail=f"Error rendering prompt template: {str(e)}",
|
||||
)
|
||||
|
||||
|
||||
@router.post("/search", response_model=PromptResponse)
|
||||
async def search_prompt(
|
||||
search_service: SearchServiceDep,
|
||||
entity_service: EntityServiceDep,
|
||||
request: SearchPromptRequest,
|
||||
page: int = 1,
|
||||
page_size: int = 10,
|
||||
) -> PromptResponse:
|
||||
"""Generate a prompt for search results.
|
||||
|
||||
This endpoint takes a search query and formats the results into a helpful
|
||||
prompt with context and suggestions.
|
||||
|
||||
Args:
|
||||
request: The search parameters
|
||||
page: The page number for pagination
|
||||
page_size: The number of results per page, defaults to 10
|
||||
|
||||
Returns:
|
||||
Formatted search results prompt with context
|
||||
"""
|
||||
logger.info(f"Generating search prompt, query: {request.query}, timeframe: {request.timeframe}")
|
||||
|
||||
limit = page_size
|
||||
offset = (page - 1) * page_size
|
||||
|
||||
query = SearchQuery(text=request.query, after_date=request.timeframe)
|
||||
results = await search_service.search(query, limit=limit, offset=offset)
|
||||
search_results = await to_search_results(entity_service, results)
|
||||
|
||||
template_context = {
|
||||
"query": request.query,
|
||||
"timeframe": request.timeframe,
|
||||
"results": search_results,
|
||||
"has_results": len(search_results) > 0,
|
||||
"result_count": len(search_results),
|
||||
}
|
||||
|
||||
try:
|
||||
# Render template
|
||||
rendered_prompt = await template_loader.render("prompts/search.hbs", template_context)
|
||||
|
||||
# Build metadata
|
||||
metadata = {
|
||||
"query": request.query,
|
||||
"timeframe": request.timeframe,
|
||||
"search_count": len(search_results),
|
||||
"context_count": len(search_results),
|
||||
"observation_count": 0, # Search results don't include observations
|
||||
"relation_count": 0, # Search results don't include relations
|
||||
"total_items": len(search_results),
|
||||
"search_limit": limit,
|
||||
"context_depth": 0, # No context depth for basic search
|
||||
"related_limit": 0, # No related items for basic search
|
||||
"generated_at": datetime.now(timezone.utc).isoformat(),
|
||||
}
|
||||
|
||||
prompt_metadata = PromptMetadata(**metadata)
|
||||
|
||||
return PromptResponse(
|
||||
prompt=rendered_prompt, context=template_context, metadata=prompt_metadata
|
||||
)
|
||||
except Exception as e:
|
||||
logger.error(f"Error rendering search template: {e}")
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
|
||||
detail=f"Error rendering prompt template: {str(e)}",
|
||||
)
|
||||
@@ -2,7 +2,8 @@
|
||||
|
||||
from fastapi import APIRouter, BackgroundTasks
|
||||
|
||||
from basic_memory.schemas.search import SearchQuery, SearchResult, SearchResponse
|
||||
from basic_memory.api.routers.utils import to_search_results
|
||||
from basic_memory.schemas.search import SearchQuery, SearchResponse
|
||||
from basic_memory.deps import SearchServiceDep, EntityServiceDep
|
||||
|
||||
router = APIRouter(prefix="/search", tags=["search"])
|
||||
@@ -20,26 +21,7 @@ async def search(
|
||||
limit = page_size
|
||||
offset = (page - 1) * page_size
|
||||
results = await search_service.search(query, limit=limit, offset=offset)
|
||||
|
||||
search_results = []
|
||||
for r in results:
|
||||
entities = await entity_service.get_entities_by_id([r.entity_id, r.from_id, r.to_id]) # pyright: ignore
|
||||
search_results.append(
|
||||
SearchResult(
|
||||
title=r.title, # pyright: ignore
|
||||
type=r.type, # pyright: ignore
|
||||
permalink=r.permalink,
|
||||
score=r.score, # pyright: ignore
|
||||
entity=entities[0].permalink if entities else None,
|
||||
content=r.content,
|
||||
file_path=r.file_path,
|
||||
metadata=r.metadata,
|
||||
category=r.category,
|
||||
from_entity=entities[0].permalink if entities else None,
|
||||
to_entity=entities[1].permalink if len(entities) > 1 else None,
|
||||
relation_type=r.relation_type,
|
||||
)
|
||||
)
|
||||
search_results = await to_search_results(entity_service, results)
|
||||
return SearchResponse(
|
||||
results=search_results,
|
||||
current_page=page,
|
||||
|
||||
@@ -0,0 +1,130 @@
|
||||
from typing import Optional, List
|
||||
|
||||
from basic_memory.repository import EntityRepository
|
||||
from basic_memory.repository.search_repository import SearchIndexRow
|
||||
from basic_memory.schemas.memory import (
|
||||
EntitySummary,
|
||||
ObservationSummary,
|
||||
RelationSummary,
|
||||
MemoryMetadata,
|
||||
GraphContext,
|
||||
ContextResult,
|
||||
)
|
||||
from basic_memory.schemas.search import SearchItemType, SearchResult
|
||||
from basic_memory.services import EntityService
|
||||
from basic_memory.services.context_service import (
|
||||
ContextResultRow,
|
||||
ContextResult as ServiceContextResult,
|
||||
)
|
||||
|
||||
|
||||
async def to_graph_context(
|
||||
context_result: ServiceContextResult,
|
||||
entity_repository: EntityRepository,
|
||||
page: Optional[int] = None,
|
||||
page_size: Optional[int] = None,
|
||||
):
|
||||
# Helper function to convert items to summaries
|
||||
async def to_summary(item: SearchIndexRow | ContextResultRow):
|
||||
match item.type:
|
||||
case SearchItemType.ENTITY:
|
||||
return EntitySummary(
|
||||
title=item.title, # pyright: ignore
|
||||
permalink=item.permalink,
|
||||
content=item.content,
|
||||
file_path=item.file_path,
|
||||
created_at=item.created_at,
|
||||
)
|
||||
case SearchItemType.OBSERVATION:
|
||||
return ObservationSummary(
|
||||
title=item.title, # pyright: ignore
|
||||
file_path=item.file_path,
|
||||
category=item.category, # pyright: ignore
|
||||
content=item.content, # pyright: ignore
|
||||
permalink=item.permalink, # pyright: ignore
|
||||
created_at=item.created_at,
|
||||
)
|
||||
case SearchItemType.RELATION:
|
||||
from_entity = await entity_repository.find_by_id(item.from_id) # pyright: ignore
|
||||
to_entity = await entity_repository.find_by_id(item.to_id) if item.to_id else None
|
||||
return RelationSummary(
|
||||
title=item.title, # pyright: ignore
|
||||
file_path=item.file_path,
|
||||
permalink=item.permalink, # pyright: ignore
|
||||
relation_type=item.relation_type, # pyright: ignore
|
||||
from_entity=from_entity.title, # pyright: ignore
|
||||
to_entity=to_entity.title if to_entity else None,
|
||||
created_at=item.created_at,
|
||||
)
|
||||
case _: # pragma: no cover
|
||||
raise ValueError(f"Unexpected type: {item.type}")
|
||||
|
||||
# Process the hierarchical results
|
||||
hierarchical_results = []
|
||||
for context_item in context_result.results:
|
||||
# Process primary result
|
||||
primary_result = await to_summary(context_item.primary_result)
|
||||
|
||||
# Process observations
|
||||
observations = []
|
||||
for obs in context_item.observations:
|
||||
observations.append(await to_summary(obs))
|
||||
|
||||
# Process related results
|
||||
related = []
|
||||
for rel in context_item.related_results:
|
||||
related.append(await to_summary(rel))
|
||||
|
||||
# Add to hierarchical results
|
||||
hierarchical_results.append(
|
||||
ContextResult(
|
||||
primary_result=primary_result,
|
||||
observations=observations,
|
||||
related_results=related,
|
||||
)
|
||||
)
|
||||
|
||||
# Create schema metadata from service metadata
|
||||
metadata = MemoryMetadata(
|
||||
uri=context_result.metadata.uri,
|
||||
types=context_result.metadata.types,
|
||||
depth=context_result.metadata.depth,
|
||||
timeframe=context_result.metadata.timeframe,
|
||||
generated_at=context_result.metadata.generated_at,
|
||||
primary_count=context_result.metadata.primary_count,
|
||||
related_count=context_result.metadata.related_count,
|
||||
total_results=context_result.metadata.primary_count + context_result.metadata.related_count,
|
||||
total_relations=context_result.metadata.total_relations,
|
||||
total_observations=context_result.metadata.total_observations,
|
||||
)
|
||||
|
||||
# Return new GraphContext with just hierarchical results
|
||||
return GraphContext(
|
||||
results=hierarchical_results,
|
||||
metadata=metadata,
|
||||
page=page,
|
||||
page_size=page_size,
|
||||
)
|
||||
|
||||
|
||||
async def to_search_results(entity_service: EntityService, results: List[SearchIndexRow]):
|
||||
search_results = []
|
||||
for r in results:
|
||||
entities = await entity_service.get_entities_by_id([r.entity_id, r.from_id, r.to_id]) # pyright: ignore
|
||||
search_results.append(
|
||||
SearchResult(
|
||||
title=r.title, # pyright: ignore
|
||||
type=r.type, # pyright: ignore
|
||||
permalink=r.permalink,
|
||||
score=r.score, # pyright: ignore
|
||||
entity=entities[0].permalink if entities else None,
|
||||
content=r.content,
|
||||
file_path=r.file_path,
|
||||
metadata=r.metadata,
|
||||
category=r.category,
|
||||
from_entity=entities[0].permalink if entities else None,
|
||||
to_entity=entities[1].permalink if len(entities) > 1 else None,
|
||||
relation_type=r.relation_type,
|
||||
)
|
||||
)
|
||||
return search_results
|
||||
@@ -0,0 +1,292 @@
|
||||
"""Template loading and rendering utilities for the Basic Memory API.
|
||||
|
||||
This module handles the loading and rendering of Handlebars templates from the
|
||||
templates directory, providing a consistent interface for all prompt-related
|
||||
formatting needs.
|
||||
"""
|
||||
|
||||
import textwrap
|
||||
from typing import Dict, Any, Optional, Callable
|
||||
from pathlib import Path
|
||||
import json
|
||||
import datetime
|
||||
|
||||
import pybars
|
||||
from loguru import logger
|
||||
|
||||
# Get the base path of the templates directory
|
||||
TEMPLATES_DIR = Path(__file__).parent.parent / "templates"
|
||||
|
||||
|
||||
# Custom helpers for Handlebars
|
||||
def _date_helper(this, *args):
|
||||
"""Format a date using the given format string."""
|
||||
if len(args) < 1: # pragma: no cover
|
||||
return ""
|
||||
|
||||
timestamp = args[0]
|
||||
format_str = args[1] if len(args) > 1 else "%Y-%m-%d %H:%M"
|
||||
|
||||
if hasattr(timestamp, "strftime"):
|
||||
result = timestamp.strftime(format_str)
|
||||
elif isinstance(timestamp, str):
|
||||
try:
|
||||
dt = datetime.datetime.fromisoformat(timestamp)
|
||||
result = dt.strftime(format_str)
|
||||
except ValueError:
|
||||
result = timestamp
|
||||
else:
|
||||
result = str(timestamp) # pragma: no cover
|
||||
|
||||
return pybars.strlist([result])
|
||||
|
||||
|
||||
def _default_helper(this, *args):
|
||||
"""Return a default value if the given value is None or empty."""
|
||||
if len(args) < 2: # pragma: no cover
|
||||
return ""
|
||||
|
||||
value = args[0]
|
||||
default_value = args[1]
|
||||
|
||||
result = default_value if value is None or value == "" else value
|
||||
# Use strlist for consistent handling of HTML escaping
|
||||
return pybars.strlist([str(result)])
|
||||
|
||||
|
||||
def _capitalize_helper(this, *args):
|
||||
"""Capitalize the first letter of a string."""
|
||||
if len(args) < 1: # pragma: no cover
|
||||
return ""
|
||||
|
||||
text = args[0]
|
||||
if not text or not isinstance(text, str): # pragma: no cover
|
||||
result = ""
|
||||
else:
|
||||
result = text.capitalize()
|
||||
|
||||
return pybars.strlist([result])
|
||||
|
||||
|
||||
def _round_helper(this, *args):
|
||||
"""Round a number to the specified number of decimal places."""
|
||||
if len(args) < 1:
|
||||
return ""
|
||||
|
||||
value = args[0]
|
||||
decimal_places = args[1] if len(args) > 1 else 2
|
||||
|
||||
try:
|
||||
result = str(round(float(value), int(decimal_places)))
|
||||
except (ValueError, TypeError):
|
||||
result = str(value)
|
||||
|
||||
return pybars.strlist([result])
|
||||
|
||||
|
||||
def _size_helper(this, *args):
|
||||
"""Return the size/length of a collection."""
|
||||
if len(args) < 1:
|
||||
return 0
|
||||
|
||||
value = args[0]
|
||||
if value is None:
|
||||
result = "0"
|
||||
elif isinstance(value, (list, tuple, dict, str)):
|
||||
result = str(len(value)) # pragma: no cover
|
||||
else: # pragma: no cover
|
||||
result = "0"
|
||||
|
||||
return pybars.strlist([result])
|
||||
|
||||
|
||||
def _json_helper(this, *args):
|
||||
"""Convert a value to a JSON string."""
|
||||
if len(args) < 1: # pragma: no cover
|
||||
return "{}"
|
||||
|
||||
value = args[0]
|
||||
# For pybars, we need to return a SafeString to prevent HTML escaping
|
||||
result = json.dumps(value) # pragma: no cover
|
||||
# Safe string implementation to prevent HTML escaping
|
||||
return pybars.strlist([result])
|
||||
|
||||
|
||||
def _math_helper(this, *args):
|
||||
"""Perform basic math operations."""
|
||||
if len(args) < 3:
|
||||
return pybars.strlist(["Math error: Insufficient arguments"])
|
||||
|
||||
lhs = args[0]
|
||||
operator = args[1]
|
||||
rhs = args[2]
|
||||
|
||||
try:
|
||||
lhs = float(lhs)
|
||||
rhs = float(rhs)
|
||||
if operator == "+":
|
||||
result = str(lhs + rhs)
|
||||
elif operator == "-":
|
||||
result = str(lhs - rhs)
|
||||
elif operator == "*":
|
||||
result = str(lhs * rhs)
|
||||
elif operator == "/":
|
||||
result = str(lhs / rhs)
|
||||
else:
|
||||
result = f"Unsupported operator: {operator}"
|
||||
except (ValueError, TypeError) as e:
|
||||
result = f"Math error: {e}"
|
||||
|
||||
return pybars.strlist([result])
|
||||
|
||||
|
||||
def _lt_helper(this, *args):
|
||||
"""Check if left hand side is less than right hand side."""
|
||||
if len(args) < 2:
|
||||
return False
|
||||
|
||||
lhs = args[0]
|
||||
rhs = args[1]
|
||||
|
||||
try:
|
||||
return float(lhs) < float(rhs)
|
||||
except (ValueError, TypeError):
|
||||
# Fall back to string comparison for non-numeric values
|
||||
return str(lhs) < str(rhs)
|
||||
|
||||
|
||||
def _if_cond_helper(this, options, condition):
|
||||
"""Block helper for custom if conditionals."""
|
||||
if condition:
|
||||
return options["fn"](this)
|
||||
elif "inverse" in options:
|
||||
return options["inverse"](this)
|
||||
return "" # pragma: no cover
|
||||
|
||||
|
||||
def _dedent_helper(this, options):
|
||||
"""Dedent a block of text to remove common leading whitespace.
|
||||
|
||||
Usage:
|
||||
{{#dedent}}
|
||||
This text will have its
|
||||
common leading whitespace removed
|
||||
while preserving relative indentation.
|
||||
{{/dedent}}
|
||||
"""
|
||||
if "fn" not in options: # pragma: no cover
|
||||
return ""
|
||||
|
||||
# Get the content from the block
|
||||
content = options["fn"](this)
|
||||
|
||||
# Convert to string if it's a strlist
|
||||
if (
|
||||
isinstance(content, list)
|
||||
or hasattr(content, "__iter__")
|
||||
and not isinstance(content, (str, bytes))
|
||||
):
|
||||
content_str = "".join(str(item) for item in content) # pragma: no cover
|
||||
else:
|
||||
content_str = str(content) # pragma: no cover
|
||||
|
||||
# Add trailing and leading newlines to ensure proper dedenting
|
||||
# This is critical for textwrap.dedent to work correctly with mixed content
|
||||
content_str = "\n" + content_str + "\n"
|
||||
|
||||
# Use textwrap to dedent the content and remove the extra newlines we added
|
||||
dedented = textwrap.dedent(content_str)[1:-1]
|
||||
|
||||
# Return as a SafeString to prevent HTML escaping
|
||||
return pybars.strlist([dedented]) # pragma: no cover
|
||||
|
||||
|
||||
class TemplateLoader:
|
||||
"""Loader for Handlebars templates.
|
||||
|
||||
This class is responsible for loading templates from disk and rendering
|
||||
them with the provided context data.
|
||||
"""
|
||||
|
||||
def __init__(self, template_dir: Optional[str] = None):
|
||||
"""Initialize the template loader.
|
||||
|
||||
Args:
|
||||
template_dir: Optional custom template directory path
|
||||
"""
|
||||
self.template_dir = Path(template_dir) if template_dir else TEMPLATES_DIR
|
||||
self.template_cache: Dict[str, Callable] = {}
|
||||
self.compiler = pybars.Compiler()
|
||||
|
||||
# Set up standard helpers
|
||||
self.helpers = {
|
||||
"date": _date_helper,
|
||||
"default": _default_helper,
|
||||
"capitalize": _capitalize_helper,
|
||||
"round": _round_helper,
|
||||
"size": _size_helper,
|
||||
"json": _json_helper,
|
||||
"math": _math_helper,
|
||||
"lt": _lt_helper,
|
||||
"if_cond": _if_cond_helper,
|
||||
"dedent": _dedent_helper,
|
||||
}
|
||||
|
||||
logger.debug(f"Initialized template loader with directory: {self.template_dir}")
|
||||
|
||||
def get_template(self, template_path: str) -> Callable:
|
||||
"""Get a template by path, using cache if available.
|
||||
|
||||
Args:
|
||||
template_path: The path to the template, relative to the templates directory
|
||||
|
||||
Returns:
|
||||
The compiled Handlebars template
|
||||
|
||||
Raises:
|
||||
FileNotFoundError: If the template doesn't exist
|
||||
"""
|
||||
if template_path in self.template_cache:
|
||||
return self.template_cache[template_path]
|
||||
|
||||
# Convert from Liquid-style path to Handlebars extension
|
||||
if template_path.endswith(".liquid"):
|
||||
template_path = template_path.replace(".liquid", ".hbs")
|
||||
elif not template_path.endswith(".hbs"):
|
||||
template_path = f"{template_path}.hbs"
|
||||
|
||||
full_path = self.template_dir / template_path
|
||||
|
||||
if not full_path.exists():
|
||||
raise FileNotFoundError(f"Template not found: {full_path}")
|
||||
|
||||
with open(full_path, "r", encoding="utf-8") as f:
|
||||
template_str = f.read()
|
||||
|
||||
template = self.compiler.compile(template_str)
|
||||
self.template_cache[template_path] = template
|
||||
|
||||
logger.debug(f"Loaded template: {template_path}")
|
||||
return template
|
||||
|
||||
async def render(self, template_path: str, context: Dict[str, Any]) -> str:
|
||||
"""Render a template with the given context.
|
||||
|
||||
Args:
|
||||
template_path: The path to the template, relative to the templates directory
|
||||
context: The context data to pass to the template
|
||||
|
||||
Returns:
|
||||
The rendered template as a string
|
||||
"""
|
||||
template = self.get_template(template_path)
|
||||
return template(context, helpers=self.helpers)
|
||||
|
||||
def clear_cache(self) -> None:
|
||||
"""Clear the template cache."""
|
||||
self.template_cache.clear()
|
||||
logger.debug("Template cache cleared")
|
||||
|
||||
|
||||
# Global template loader instance
|
||||
template_loader = TemplateLoader()
|
||||
+22
-23
@@ -2,6 +2,9 @@ from typing import Optional
|
||||
|
||||
import typer
|
||||
|
||||
from basic_memory.config import get_project_config
|
||||
from basic_memory.mcp.project_session import session
|
||||
|
||||
|
||||
def version_callback(value: bool) -> None:
|
||||
"""Show version and exit."""
|
||||
@@ -39,31 +42,27 @@ def app_callback(
|
||||
) -> None:
|
||||
"""Basic Memory - Local-first personal knowledge management."""
|
||||
|
||||
# We use the project option to set the BASIC_MEMORY_PROJECT environment variable
|
||||
# The config module will pick this up when loading
|
||||
if project: # pragma: no cover
|
||||
import os
|
||||
import importlib
|
||||
from basic_memory import config as config_module
|
||||
|
||||
# Set the environment variable
|
||||
os.environ["BASIC_MEMORY_PROJECT"] = project
|
||||
|
||||
# Reload the config module to pick up the new project
|
||||
importlib.reload(config_module)
|
||||
|
||||
# Update the local reference
|
||||
global config
|
||||
from basic_memory.config import config as new_config
|
||||
|
||||
config = new_config
|
||||
|
||||
# Run migrations for every command unless --version was specified
|
||||
# Run initialization for every command unless --version was specified
|
||||
if not version and ctx.invoked_subcommand is not None:
|
||||
from basic_memory.config import config
|
||||
from basic_memory.services.initialization import ensure_initialize_database
|
||||
from basic_memory.config import app_config
|
||||
from basic_memory.services.initialization import ensure_initialization
|
||||
|
||||
ensure_initialize_database(config)
|
||||
ensure_initialization(app_config)
|
||||
|
||||
# Initialize MCP session with the specified project or default
|
||||
if project: # pragma: no cover
|
||||
# Use the project specified via --project flag
|
||||
current_project_config = get_project_config(project)
|
||||
session.set_current_project(current_project_config.name)
|
||||
|
||||
# Update the global config to use this project
|
||||
from basic_memory.config import update_current_project
|
||||
|
||||
update_current_project(project)
|
||||
else:
|
||||
# Use the default project
|
||||
current_project = app_config.default_project
|
||||
session.set_current_project(current_project)
|
||||
|
||||
|
||||
# Register sub-command groups
|
||||
|
||||
@@ -1,9 +1,10 @@
|
||||
"""CLI commands for basic-memory."""
|
||||
|
||||
from . import status, sync, db, import_memory_json, mcp, import_claude_conversations
|
||||
from . import auth, status, sync, db, import_memory_json, mcp, import_claude_conversations
|
||||
from . import import_claude_projects, import_chatgpt, tool, project
|
||||
|
||||
__all__ = [
|
||||
"auth",
|
||||
"status",
|
||||
"sync",
|
||||
"db",
|
||||
|
||||
@@ -0,0 +1,136 @@
|
||||
"""OAuth management commands."""
|
||||
|
||||
import typer
|
||||
from typing import Optional
|
||||
from pydantic import AnyHttpUrl
|
||||
|
||||
from basic_memory.cli.app import app
|
||||
from basic_memory.mcp.auth_provider import BasicMemoryOAuthProvider
|
||||
from mcp.shared.auth import OAuthClientInformationFull
|
||||
|
||||
|
||||
auth_app = typer.Typer(help="OAuth client management commands")
|
||||
app.add_typer(auth_app, name="auth")
|
||||
|
||||
|
||||
@auth_app.command()
|
||||
def register_client(
|
||||
client_id: Optional[str] = typer.Option(
|
||||
None, help="Client ID (auto-generated if not provided)"
|
||||
),
|
||||
client_secret: Optional[str] = typer.Option(
|
||||
None, help="Client secret (auto-generated if not provided)"
|
||||
),
|
||||
issuer_url: str = typer.Option("http://localhost:8000", help="OAuth issuer URL"),
|
||||
):
|
||||
"""Register a new OAuth client for Basic Memory MCP server."""
|
||||
|
||||
# Create provider instance
|
||||
provider = BasicMemoryOAuthProvider(issuer_url=issuer_url)
|
||||
|
||||
# Create client info with required redirect_uris
|
||||
client_info = OAuthClientInformationFull(
|
||||
client_id=client_id or "", # Provider will generate if empty
|
||||
client_secret=client_secret or "", # Provider will generate if empty
|
||||
redirect_uris=[AnyHttpUrl("http://localhost:8000/callback")], # Default redirect URI
|
||||
client_name="Basic Memory OAuth Client",
|
||||
grant_types=["authorization_code", "refresh_token"],
|
||||
)
|
||||
|
||||
# Register the client
|
||||
import asyncio
|
||||
|
||||
asyncio.run(provider.register_client(client_info))
|
||||
|
||||
typer.echo("Client registered successfully!")
|
||||
typer.echo(f"Client ID: {client_info.client_id}")
|
||||
typer.echo(f"Client Secret: {client_info.client_secret}")
|
||||
typer.echo("\nSave these credentials securely - the client secret cannot be retrieved later.")
|
||||
|
||||
|
||||
@auth_app.command()
|
||||
def test_auth(
|
||||
issuer_url: str = typer.Option("http://localhost:8000", help="OAuth issuer URL"),
|
||||
):
|
||||
"""Test OAuth authentication flow.
|
||||
|
||||
IMPORTANT: Use the same FASTMCP_AUTH_SECRET_KEY environment variable
|
||||
as your MCP server for tokens to validate correctly.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import secrets
|
||||
from mcp.server.auth.provider import AuthorizationParams
|
||||
from pydantic import AnyHttpUrl
|
||||
|
||||
async def test_flow():
|
||||
# Create provider with same secret key as server
|
||||
provider = BasicMemoryOAuthProvider(issuer_url=issuer_url)
|
||||
|
||||
# Register a test client
|
||||
client_info = OAuthClientInformationFull(
|
||||
client_id=secrets.token_urlsafe(16),
|
||||
client_secret=secrets.token_urlsafe(32),
|
||||
redirect_uris=[AnyHttpUrl("http://localhost:8000/callback")],
|
||||
client_name="Test OAuth Client",
|
||||
grant_types=["authorization_code", "refresh_token"],
|
||||
)
|
||||
await provider.register_client(client_info)
|
||||
typer.echo(f"Registered test client: {client_info.client_id}")
|
||||
|
||||
# Get the client
|
||||
client = await provider.get_client(client_info.client_id)
|
||||
if not client:
|
||||
typer.echo("Error: Client not found after registration", err=True)
|
||||
return
|
||||
|
||||
# Create authorization request
|
||||
auth_params = AuthorizationParams(
|
||||
state="test-state",
|
||||
scopes=["read", "write"],
|
||||
code_challenge="test-challenge",
|
||||
redirect_uri=AnyHttpUrl("http://localhost:8000/callback"),
|
||||
redirect_uri_provided_explicitly=True,
|
||||
)
|
||||
|
||||
# Get authorization URL
|
||||
auth_url = await provider.authorize(client, auth_params)
|
||||
typer.echo(f"Authorization URL: {auth_url}")
|
||||
|
||||
# Extract auth code from URL
|
||||
from urllib.parse import urlparse, parse_qs
|
||||
|
||||
parsed = urlparse(auth_url)
|
||||
params = parse_qs(parsed.query)
|
||||
auth_code = params.get("code", [None])[0]
|
||||
|
||||
if not auth_code:
|
||||
typer.echo("Error: No authorization code in URL", err=True)
|
||||
return
|
||||
|
||||
# Load the authorization code
|
||||
code_obj = await provider.load_authorization_code(client, auth_code)
|
||||
if not code_obj:
|
||||
typer.echo("Error: Invalid authorization code", err=True)
|
||||
return
|
||||
|
||||
# Exchange for tokens
|
||||
token = await provider.exchange_authorization_code(client, code_obj)
|
||||
typer.echo(f"Access token: {token.access_token}")
|
||||
typer.echo(f"Refresh token: {token.refresh_token}")
|
||||
typer.echo(f"Expires in: {token.expires_in} seconds")
|
||||
|
||||
# Validate access token
|
||||
access_token_obj = await provider.load_access_token(token.access_token)
|
||||
if access_token_obj:
|
||||
typer.echo("Access token validated successfully!")
|
||||
typer.echo(f"Client ID: {access_token_obj.client_id}")
|
||||
typer.echo(f"Scopes: {access_token_obj.scopes}")
|
||||
else:
|
||||
typer.echo("Error: Invalid access token", err=True)
|
||||
|
||||
asyncio.run(test_flow())
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
auth_app()
|
||||
@@ -7,7 +7,7 @@ from loguru import logger
|
||||
|
||||
from basic_memory import db
|
||||
from basic_memory.cli.app import app
|
||||
from basic_memory.config import config
|
||||
from basic_memory.config import app_config
|
||||
|
||||
|
||||
@app.command()
|
||||
@@ -18,7 +18,7 @@ def reset(
|
||||
if typer.confirm("This will delete all data in your db. Are you sure?"):
|
||||
logger.info("Resetting database...")
|
||||
# Get database path
|
||||
db_path = config.database_path
|
||||
db_path = app_config.app_database_path
|
||||
|
||||
# Delete the database file if it exists
|
||||
if db_path.exists():
|
||||
@@ -26,7 +26,7 @@ def reset(
|
||||
logger.info(f"Database file deleted: {db_path}")
|
||||
|
||||
# Create a new empty database
|
||||
asyncio.run(db.run_migrations(config))
|
||||
asyncio.run(db.run_migrations(app_config))
|
||||
logger.info("Database reset complete")
|
||||
|
||||
if reindex:
|
||||
|
||||
@@ -2,203 +2,21 @@
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
from typing import Dict, Any, List, Annotated, Set, Optional
|
||||
from typing import Annotated
|
||||
|
||||
import typer
|
||||
from basic_memory.cli.app import import_app
|
||||
from basic_memory.config import config
|
||||
from basic_memory.importers import ChatGPTImporter
|
||||
from basic_memory.markdown import EntityParser, MarkdownProcessor
|
||||
from basic_memory.markdown.schemas import EntityMarkdown, EntityFrontmatter
|
||||
from loguru import logger
|
||||
from rich.console import Console
|
||||
from rich.panel import Panel
|
||||
from rich.progress import Progress, SpinnerColumn, TextColumn, BarColumn
|
||||
|
||||
console = Console()
|
||||
|
||||
|
||||
def clean_filename(text: str) -> str:
|
||||
"""Convert text to safe filename."""
|
||||
clean = "".join(c if c.isalnum() else "-" for c in text.lower()).strip("-")
|
||||
return clean
|
||||
|
||||
|
||||
def format_timestamp(ts: float) -> str:
|
||||
"""Format Unix timestamp for display."""
|
||||
dt = datetime.fromtimestamp(ts)
|
||||
return dt.strftime("%Y-%m-%d %H:%M:%S")
|
||||
|
||||
|
||||
def get_message_content(message: Dict[str, Any]) -> str:
|
||||
"""Extract clean message content."""
|
||||
if not message or "content" not in message:
|
||||
return "" # pragma: no cover
|
||||
|
||||
content = message["content"]
|
||||
if content.get("content_type") == "text":
|
||||
return "\n".join(content.get("parts", []))
|
||||
elif content.get("content_type") == "code":
|
||||
return f"```{content.get('language', '')}\n{content.get('text', '')}\n```"
|
||||
return "" # pragma: no cover
|
||||
|
||||
|
||||
def traverse_messages(
|
||||
mapping: Dict[str, Any], root_id: Optional[str], seen: Set[str]
|
||||
) -> List[Dict[str, Any]]:
|
||||
"""Traverse message tree and return messages in order."""
|
||||
messages = []
|
||||
node = mapping.get(root_id) if root_id else None
|
||||
|
||||
while node:
|
||||
if node["id"] not in seen and node.get("message"):
|
||||
seen.add(node["id"])
|
||||
messages.append(node["message"])
|
||||
|
||||
# Follow children
|
||||
children = node.get("children", [])
|
||||
for child_id in children:
|
||||
child_msgs = traverse_messages(mapping, child_id, seen)
|
||||
messages.extend(child_msgs)
|
||||
|
||||
break # Don't follow siblings
|
||||
|
||||
return messages
|
||||
|
||||
|
||||
def format_chat_markdown(
|
||||
title: str,
|
||||
mapping: Dict[str, Any],
|
||||
root_id: Optional[str],
|
||||
created_at: float,
|
||||
modified_at: float,
|
||||
) -> str:
|
||||
"""Format chat as clean markdown."""
|
||||
|
||||
# Start with title
|
||||
lines = [f"# {title}\n"]
|
||||
|
||||
# Traverse message tree
|
||||
seen_msgs = set()
|
||||
messages = traverse_messages(mapping, root_id, seen_msgs)
|
||||
|
||||
# Format each message
|
||||
for msg in messages:
|
||||
# Skip hidden messages
|
||||
if msg.get("metadata", {}).get("is_visually_hidden_from_conversation"):
|
||||
continue
|
||||
|
||||
# Get author and timestamp
|
||||
author = msg["author"]["role"].title()
|
||||
ts = format_timestamp(msg["create_time"]) if msg.get("create_time") else ""
|
||||
|
||||
# Add message header
|
||||
lines.append(f"### {author} ({ts})")
|
||||
|
||||
# Add message content
|
||||
content = get_message_content(msg)
|
||||
if content:
|
||||
lines.append(content)
|
||||
|
||||
# Add spacing
|
||||
lines.append("")
|
||||
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def format_chat_content(folder: str, conversation: Dict[str, Any]) -> EntityMarkdown:
|
||||
"""Convert chat conversation to Basic Memory entity."""
|
||||
|
||||
# Extract timestamps
|
||||
created_at = conversation["create_time"]
|
||||
modified_at = conversation["update_time"]
|
||||
|
||||
root_id = None
|
||||
# Find root message
|
||||
for node_id, node in conversation["mapping"].items():
|
||||
if node.get("parent") is None:
|
||||
root_id = node_id
|
||||
break
|
||||
|
||||
# Generate permalink
|
||||
date_prefix = datetime.fromtimestamp(created_at).strftime("%Y%m%d")
|
||||
clean_title = clean_filename(conversation["title"])
|
||||
|
||||
# Format content
|
||||
content = format_chat_markdown(
|
||||
title=conversation["title"],
|
||||
mapping=conversation["mapping"],
|
||||
root_id=root_id,
|
||||
created_at=created_at,
|
||||
modified_at=modified_at,
|
||||
)
|
||||
|
||||
# Create entity
|
||||
entity = EntityMarkdown(
|
||||
frontmatter=EntityFrontmatter(
|
||||
metadata={
|
||||
"type": "conversation",
|
||||
"title": conversation["title"],
|
||||
"created": format_timestamp(created_at),
|
||||
"modified": format_timestamp(modified_at),
|
||||
"permalink": f"{folder}/{date_prefix}-{clean_title}",
|
||||
}
|
||||
),
|
||||
content=content,
|
||||
)
|
||||
|
||||
return entity
|
||||
|
||||
|
||||
async def process_chatgpt_json(
|
||||
json_path: Path, folder: str, markdown_processor: MarkdownProcessor
|
||||
) -> Dict[str, int]:
|
||||
"""Import conversations from ChatGPT JSON format."""
|
||||
|
||||
with Progress(
|
||||
SpinnerColumn(),
|
||||
TextColumn("[progress.description]{task.description}"),
|
||||
BarColumn(),
|
||||
TextColumn("[progress.percentage]{task.percentage:>3.0f}%"),
|
||||
console=console,
|
||||
) as progress:
|
||||
read_task = progress.add_task("Reading chat data...", total=None)
|
||||
|
||||
# Read conversations
|
||||
conversations = json.loads(json_path.read_text(encoding="utf-8"))
|
||||
progress.update(read_task, total=len(conversations))
|
||||
|
||||
# Process each conversation
|
||||
messages_imported = 0
|
||||
chats_imported = 0
|
||||
|
||||
for chat in conversations:
|
||||
# Convert to entity
|
||||
entity = format_chat_content(folder, chat)
|
||||
|
||||
# Write file
|
||||
file_path = config.home / f"{entity.frontmatter.metadata['permalink']}.md"
|
||||
# logger.info(f"Writing file: {file_path.absolute()}")
|
||||
await markdown_processor.write_file(file_path, entity)
|
||||
|
||||
# Count messages
|
||||
msg_count = sum(
|
||||
1
|
||||
for node in chat["mapping"].values()
|
||||
if node.get("message")
|
||||
and not node.get("message", {})
|
||||
.get("metadata", {})
|
||||
.get("is_visually_hidden_from_conversation")
|
||||
)
|
||||
|
||||
chats_imported += 1
|
||||
messages_imported += msg_count
|
||||
progress.update(read_task, advance=1)
|
||||
|
||||
return {"conversations": chats_imported, "messages": messages_imported}
|
||||
|
||||
|
||||
async def get_markdown_processor() -> MarkdownProcessor:
|
||||
"""Get MarkdownProcessor instance."""
|
||||
entity_parser = EntityParser(config.home)
|
||||
@@ -225,30 +43,36 @@ def import_chatgpt(
|
||||
"""
|
||||
|
||||
try:
|
||||
if conversations_json:
|
||||
if not conversations_json.exists():
|
||||
typer.echo(f"Error: File not found: {conversations_json}", err=True)
|
||||
raise typer.Exit(1)
|
||||
if not conversations_json.exists(): # pragma: no cover
|
||||
typer.echo(f"Error: File not found: {conversations_json}", err=True)
|
||||
raise typer.Exit(1)
|
||||
|
||||
# Get markdown processor
|
||||
markdown_processor = asyncio.run(get_markdown_processor())
|
||||
# Get markdown processor
|
||||
markdown_processor = asyncio.run(get_markdown_processor())
|
||||
|
||||
# Process the file
|
||||
base_path = config.home / folder
|
||||
console.print(f"\nImporting chats from {conversations_json}...writing to {base_path}")
|
||||
results = asyncio.run(
|
||||
process_chatgpt_json(conversations_json, folder, markdown_processor)
|
||||
)
|
||||
|
||||
# Show results
|
||||
console.print(
|
||||
Panel(
|
||||
f"[green]Import complete![/green]\n\n"
|
||||
f"Imported {results['conversations']} conversations\n"
|
||||
f"Containing {results['messages']} messages",
|
||||
expand=False,
|
||||
)
|
||||
# Process the file
|
||||
base_path = config.home / folder
|
||||
console.print(f"\nImporting chats from {conversations_json}...writing to {base_path}")
|
||||
|
||||
# Create importer and run import
|
||||
importer = ChatGPTImporter(config.home, markdown_processor)
|
||||
with conversations_json.open("r", encoding="utf-8") as file:
|
||||
json_data = json.load(file)
|
||||
result = asyncio.run(importer.import_data(json_data, folder))
|
||||
|
||||
if not result.success: # pragma: no cover
|
||||
typer.echo(f"Error during import: {result.error_message}", err=True)
|
||||
raise typer.Exit(1)
|
||||
|
||||
# Show results
|
||||
console.print(
|
||||
Panel(
|
||||
f"[green]Import complete![/green]\n\n"
|
||||
f"Imported {result.conversations} conversations\n"
|
||||
f"Containing {result.messages} messages",
|
||||
expand=False,
|
||||
)
|
||||
)
|
||||
|
||||
console.print("\nRun 'basic-memory sync' to index the new files.")
|
||||
|
||||
|
||||
@@ -2,156 +2,21 @@
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
from typing import Dict, Any, List, Annotated
|
||||
from typing import Annotated
|
||||
|
||||
import typer
|
||||
from basic_memory.cli.app import claude_app
|
||||
from basic_memory.config import config
|
||||
from basic_memory.importers.claude_conversations_importer import ClaudeConversationsImporter
|
||||
from basic_memory.markdown import EntityParser, MarkdownProcessor
|
||||
from basic_memory.markdown.schemas import EntityMarkdown, EntityFrontmatter
|
||||
from loguru import logger
|
||||
from rich.console import Console
|
||||
from rich.panel import Panel
|
||||
from rich.progress import Progress, SpinnerColumn, TextColumn, BarColumn
|
||||
|
||||
console = Console()
|
||||
|
||||
|
||||
def clean_filename(text: str) -> str:
|
||||
"""Convert text to safe filename."""
|
||||
# Remove invalid characters and convert spaces
|
||||
clean = "".join(c if c.isalnum() else "-" for c in text.lower()).strip("-")
|
||||
return clean
|
||||
|
||||
|
||||
def format_timestamp(ts: str) -> str:
|
||||
"""Format ISO timestamp for display."""
|
||||
dt = datetime.fromisoformat(ts.replace("Z", "+00:00"))
|
||||
return dt.strftime("%Y-%m-%d %H:%M:%S")
|
||||
|
||||
|
||||
def format_chat_markdown(
|
||||
name: str, messages: List[Dict[str, Any]], created_at: str, modified_at: str, permalink: str
|
||||
) -> str:
|
||||
"""Format chat as clean markdown."""
|
||||
|
||||
# Start with frontmatter and title
|
||||
lines = [
|
||||
f"# {name}\n",
|
||||
]
|
||||
|
||||
# Add messages
|
||||
for msg in messages:
|
||||
# Format timestamp
|
||||
ts = format_timestamp(msg["created_at"])
|
||||
|
||||
# Add message header
|
||||
lines.append(f"### {msg['sender'].title()} ({ts})")
|
||||
|
||||
# Handle message content
|
||||
content = msg.get("text", "")
|
||||
if msg.get("content"):
|
||||
content = " ".join(c.get("text", "") for c in msg["content"])
|
||||
lines.append(content)
|
||||
|
||||
# Handle attachments
|
||||
attachments = msg.get("attachments", [])
|
||||
for attachment in attachments:
|
||||
if "file_name" in attachment:
|
||||
lines.append(f"\n**Attachment: {attachment['file_name']}**")
|
||||
if "extracted_content" in attachment:
|
||||
lines.append("```")
|
||||
lines.append(attachment["extracted_content"])
|
||||
lines.append("```")
|
||||
|
||||
# Add spacing between messages
|
||||
lines.append("")
|
||||
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def format_chat_content(
|
||||
base_path: Path, name: str, messages: List[Dict[str, Any]], created_at: str, modified_at: str
|
||||
) -> EntityMarkdown:
|
||||
"""Convert chat messages to Basic Memory entity format."""
|
||||
|
||||
# Generate permalink
|
||||
date_prefix = datetime.fromisoformat(created_at.replace("Z", "+00:00")).strftime("%Y%m%d")
|
||||
clean_title = clean_filename(name)
|
||||
permalink = f"{base_path}/{date_prefix}-{clean_title}"
|
||||
|
||||
# Format content
|
||||
content = format_chat_markdown(
|
||||
name=name,
|
||||
messages=messages,
|
||||
created_at=created_at,
|
||||
modified_at=modified_at,
|
||||
permalink=permalink,
|
||||
)
|
||||
|
||||
# Create entity
|
||||
entity = EntityMarkdown(
|
||||
frontmatter=EntityFrontmatter(
|
||||
metadata={
|
||||
"type": "conversation",
|
||||
"title": name,
|
||||
"created": created_at,
|
||||
"modified": modified_at,
|
||||
"permalink": permalink,
|
||||
}
|
||||
),
|
||||
content=content,
|
||||
)
|
||||
|
||||
return entity
|
||||
|
||||
|
||||
async def process_conversations_json(
|
||||
json_path: Path, base_path: Path, markdown_processor: MarkdownProcessor
|
||||
) -> Dict[str, int]:
|
||||
"""Import chat data from conversations2.json format."""
|
||||
|
||||
with Progress(
|
||||
SpinnerColumn(),
|
||||
TextColumn("[progress.description]{task.description}"),
|
||||
BarColumn(),
|
||||
TextColumn("[progress.percentage]{task.percentage:>3.0f}%"),
|
||||
console=console,
|
||||
) as progress:
|
||||
read_task = progress.add_task("Reading chat data...", total=None)
|
||||
|
||||
# Read chat data - handle array of arrays format
|
||||
data = json.loads(json_path.read_text(encoding="utf-8"))
|
||||
conversations = [chat for chat in data]
|
||||
progress.update(read_task, total=len(conversations))
|
||||
|
||||
# Process each conversation
|
||||
messages_imported = 0
|
||||
chats_imported = 0
|
||||
|
||||
for chat in conversations:
|
||||
# Convert to entity
|
||||
entity = format_chat_content(
|
||||
base_path=base_path,
|
||||
name=chat["name"],
|
||||
messages=chat["chat_messages"],
|
||||
created_at=chat["created_at"],
|
||||
modified_at=chat["updated_at"],
|
||||
)
|
||||
|
||||
# Write file
|
||||
file_path = Path(f"{entity.frontmatter.metadata['permalink']}.md")
|
||||
await markdown_processor.write_file(file_path, entity)
|
||||
|
||||
chats_imported += 1
|
||||
messages_imported += len(chat["chat_messages"])
|
||||
progress.update(read_task, advance=1)
|
||||
|
||||
return {"conversations": chats_imported, "messages": messages_imported}
|
||||
|
||||
|
||||
async def get_markdown_processor() -> MarkdownProcessor:
|
||||
"""Get MarkdownProcessor instance."""
|
||||
entity_parser = EntityParser(config.home)
|
||||
@@ -185,19 +50,28 @@ def import_claude(
|
||||
# Get markdown processor
|
||||
markdown_processor = asyncio.run(get_markdown_processor())
|
||||
|
||||
# Create the importer
|
||||
importer = ClaudeConversationsImporter(config.home, markdown_processor)
|
||||
|
||||
# Process the file
|
||||
base_path = config.home / folder
|
||||
console.print(f"\nImporting chats from {conversations_json}...writing to {base_path}")
|
||||
results = asyncio.run(
|
||||
process_conversations_json(conversations_json, base_path, markdown_processor)
|
||||
)
|
||||
|
||||
# Run the import
|
||||
with conversations_json.open("r", encoding="utf-8") as file:
|
||||
json_data = json.load(file)
|
||||
result = asyncio.run(importer.import_data(json_data, folder))
|
||||
|
||||
if not result.success: # pragma: no cover
|
||||
typer.echo(f"Error during import: {result.error_message}", err=True)
|
||||
raise typer.Exit(1)
|
||||
|
||||
# Show results
|
||||
console.print(
|
||||
Panel(
|
||||
f"[green]Import complete![/green]\n\n"
|
||||
f"Imported {results['conversations']} conversations\n"
|
||||
f"Containing {results['messages']} messages",
|
||||
f"Imported {result.conversations} conversations\n"
|
||||
f"Containing {result.messages} messages",
|
||||
expand=False,
|
||||
)
|
||||
)
|
||||
|
||||
@@ -3,138 +3,20 @@
|
||||
import asyncio
|
||||
import json
|
||||
from pathlib import Path
|
||||
from typing import Dict, Any, Annotated, Optional
|
||||
from typing import Annotated
|
||||
|
||||
import typer
|
||||
from basic_memory.cli.app import claude_app
|
||||
from basic_memory.config import config
|
||||
from basic_memory.importers.claude_projects_importer import ClaudeProjectsImporter
|
||||
from basic_memory.markdown import EntityParser, MarkdownProcessor
|
||||
from basic_memory.markdown.schemas import EntityMarkdown, EntityFrontmatter
|
||||
from loguru import logger
|
||||
from rich.console import Console
|
||||
from rich.panel import Panel
|
||||
from rich.progress import Progress, SpinnerColumn, TextColumn, BarColumn
|
||||
|
||||
console = Console()
|
||||
|
||||
|
||||
def clean_filename(text: str) -> str:
|
||||
"""Convert text to safe filename."""
|
||||
clean = "".join(c if c.isalnum() else "-" for c in text.lower()).strip("-")
|
||||
return clean
|
||||
|
||||
|
||||
def format_project_markdown(project: Dict[str, Any], doc: Dict[str, Any]) -> EntityMarkdown:
|
||||
"""Format a project document as a Basic Memory entity."""
|
||||
|
||||
# Extract timestamps
|
||||
created_at = doc.get("created_at") or project["created_at"]
|
||||
modified_at = project["updated_at"]
|
||||
|
||||
# Generate clean names for organization
|
||||
project_dir = clean_filename(project["name"])
|
||||
doc_file = clean_filename(doc["filename"])
|
||||
|
||||
# Create entity
|
||||
entity = EntityMarkdown(
|
||||
frontmatter=EntityFrontmatter(
|
||||
metadata={
|
||||
"type": "project_doc",
|
||||
"title": doc["filename"],
|
||||
"created": created_at,
|
||||
"modified": modified_at,
|
||||
"permalink": f"{project_dir}/docs/{doc_file}",
|
||||
"project_name": project["name"],
|
||||
"project_uuid": project["uuid"],
|
||||
"doc_uuid": doc["uuid"],
|
||||
}
|
||||
),
|
||||
content=doc["content"],
|
||||
)
|
||||
|
||||
return entity
|
||||
|
||||
|
||||
def format_prompt_markdown(project: Dict[str, Any]) -> Optional[EntityMarkdown]:
|
||||
"""Format project prompt template as a Basic Memory entity."""
|
||||
|
||||
if not project.get("prompt_template"):
|
||||
return None
|
||||
|
||||
# Extract timestamps
|
||||
created_at = project["created_at"]
|
||||
modified_at = project["updated_at"]
|
||||
|
||||
# Generate clean project directory name
|
||||
project_dir = clean_filename(project["name"])
|
||||
|
||||
# Create entity
|
||||
entity = EntityMarkdown(
|
||||
frontmatter=EntityFrontmatter(
|
||||
metadata={
|
||||
"type": "prompt_template",
|
||||
"title": f"Prompt Template: {project['name']}",
|
||||
"created": created_at,
|
||||
"modified": modified_at,
|
||||
"permalink": f"{project_dir}/prompt-template",
|
||||
"project_name": project["name"],
|
||||
"project_uuid": project["uuid"],
|
||||
}
|
||||
),
|
||||
content=f"# Prompt Template: {project['name']}\n\n{project['prompt_template']}",
|
||||
)
|
||||
|
||||
return entity
|
||||
|
||||
|
||||
async def process_projects_json(
|
||||
json_path: Path, base_path: Path, markdown_processor: MarkdownProcessor
|
||||
) -> Dict[str, int]:
|
||||
"""Import project data from Claude.ai projects.json format."""
|
||||
|
||||
with Progress(
|
||||
SpinnerColumn(),
|
||||
TextColumn("[progress.description]{task.description}"),
|
||||
BarColumn(),
|
||||
TextColumn("[progress.percentage]{task.percentage:>3.0f}%"),
|
||||
console=console,
|
||||
) as progress:
|
||||
read_task = progress.add_task("Reading project data...", total=None)
|
||||
|
||||
# Read project data
|
||||
data = json.loads(json_path.read_text(encoding="utf-8"))
|
||||
progress.update(read_task, total=len(data))
|
||||
|
||||
# Track import counts
|
||||
docs_imported = 0
|
||||
prompts_imported = 0
|
||||
|
||||
# Process each project
|
||||
for project in data:
|
||||
project_dir = clean_filename(project["name"])
|
||||
|
||||
# Create project directories
|
||||
docs_dir = base_path / project_dir / "docs"
|
||||
docs_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
# Import prompt template if it exists
|
||||
if prompt_entity := format_prompt_markdown(project):
|
||||
file_path = base_path / f"{prompt_entity.frontmatter.metadata['permalink']}.md"
|
||||
await markdown_processor.write_file(file_path, prompt_entity)
|
||||
prompts_imported += 1
|
||||
|
||||
# Import project documents
|
||||
for doc in project.get("docs", []):
|
||||
entity = format_project_markdown(project, doc)
|
||||
file_path = base_path / f"{entity.frontmatter.metadata['permalink']}.md"
|
||||
await markdown_processor.write_file(file_path, entity)
|
||||
docs_imported += 1
|
||||
|
||||
progress.update(read_task, advance=1)
|
||||
|
||||
return {"documents": docs_imported, "prompts": prompts_imported}
|
||||
|
||||
|
||||
async def get_markdown_processor() -> MarkdownProcessor:
|
||||
"""Get MarkdownProcessor instance."""
|
||||
entity_parser = EntityParser(config.home)
|
||||
@@ -160,30 +42,38 @@ def import_projects(
|
||||
After importing, run 'basic-memory sync' to index the new files.
|
||||
"""
|
||||
try:
|
||||
if projects_json:
|
||||
if not projects_json.exists():
|
||||
typer.echo(f"Error: File not found: {projects_json}", err=True)
|
||||
raise typer.Exit(1)
|
||||
if not projects_json.exists():
|
||||
typer.echo(f"Error: File not found: {projects_json}", err=True)
|
||||
raise typer.Exit(1)
|
||||
|
||||
# Get markdown processor
|
||||
markdown_processor = asyncio.run(get_markdown_processor())
|
||||
# Get markdown processor
|
||||
markdown_processor = asyncio.run(get_markdown_processor())
|
||||
|
||||
# Process the file
|
||||
base_path = config.home / base_folder if base_folder else config.home
|
||||
console.print(f"\nImporting projects from {projects_json}...writing to {base_path}")
|
||||
results = asyncio.run(
|
||||
process_projects_json(projects_json, base_path, markdown_processor)
|
||||
)
|
||||
|
||||
# Show results
|
||||
console.print(
|
||||
Panel(
|
||||
f"[green]Import complete![/green]\n\n"
|
||||
f"Imported {results['documents']} project documents\n"
|
||||
f"Imported {results['prompts']} prompt templates",
|
||||
expand=False,
|
||||
)
|
||||
# Create the importer
|
||||
importer = ClaudeProjectsImporter(config.home, markdown_processor)
|
||||
|
||||
# Process the file
|
||||
base_path = config.home / base_folder if base_folder else config.home
|
||||
console.print(f"\nImporting projects from {projects_json}...writing to {base_path}")
|
||||
|
||||
# Run the import
|
||||
with projects_json.open("r", encoding="utf-8") as file:
|
||||
json_data = json.load(file)
|
||||
result = asyncio.run(importer.import_data(json_data, base_folder))
|
||||
|
||||
if not result.success: # pragma: no cover
|
||||
typer.echo(f"Error during import: {result.error_message}", err=True)
|
||||
raise typer.Exit(1)
|
||||
|
||||
# Show results
|
||||
console.print(
|
||||
Panel(
|
||||
f"[green]Import complete![/green]\n\n"
|
||||
f"Imported {result.documents} project documents\n"
|
||||
f"Imported {result.prompts} prompt templates",
|
||||
expand=False,
|
||||
)
|
||||
)
|
||||
|
||||
console.print("\nRun 'basic-memory sync' to index the new files.")
|
||||
|
||||
|
||||
@@ -3,94 +3,20 @@
|
||||
import asyncio
|
||||
import json
|
||||
from pathlib import Path
|
||||
from typing import Dict, Any, List, Annotated
|
||||
from typing import Annotated
|
||||
|
||||
import typer
|
||||
from basic_memory.cli.app import import_app
|
||||
from basic_memory.config import config
|
||||
from basic_memory.importers.memory_json_importer import MemoryJsonImporter
|
||||
from basic_memory.markdown import EntityParser, MarkdownProcessor
|
||||
from loguru import logger
|
||||
from rich.console import Console
|
||||
from rich.panel import Panel
|
||||
from rich.progress import Progress, SpinnerColumn, TextColumn, BarColumn
|
||||
|
||||
from basic_memory.cli.app import import_app
|
||||
from basic_memory.config import config
|
||||
from basic_memory.markdown import EntityParser, MarkdownProcessor
|
||||
from basic_memory.markdown.schemas import EntityMarkdown, EntityFrontmatter, Observation, Relation
|
||||
|
||||
console = Console()
|
||||
|
||||
|
||||
async def process_memory_json(
|
||||
json_path: Path, base_path: Path, markdown_processor: MarkdownProcessor
|
||||
):
|
||||
"""Import entities from memory.json using markdown processor."""
|
||||
|
||||
# First pass - collect all relations by source entity
|
||||
entity_relations: Dict[str, List[Relation]] = {}
|
||||
entities: Dict[str, Dict[str, Any]] = {}
|
||||
|
||||
with Progress(
|
||||
SpinnerColumn(),
|
||||
TextColumn("[progress.description]{task.description}"),
|
||||
BarColumn(),
|
||||
TextColumn("[progress.percentage]{task.percentage:>3.0f}%"),
|
||||
console=console,
|
||||
) as progress:
|
||||
read_task = progress.add_task("Reading memory.json...", total=None)
|
||||
|
||||
# First pass - collect entities and relations
|
||||
with open(json_path, encoding="utf-8") as f:
|
||||
lines = f.readlines()
|
||||
progress.update(read_task, total=len(lines))
|
||||
|
||||
for line in lines:
|
||||
data = json.loads(line)
|
||||
if data["type"] == "entity":
|
||||
entities[data["name"]] = data
|
||||
elif data["type"] == "relation":
|
||||
# Store relation with its source entity
|
||||
source = data.get("from") or data.get("from_id")
|
||||
if source not in entity_relations:
|
||||
entity_relations[source] = []
|
||||
entity_relations[source].append(
|
||||
Relation(
|
||||
type=data.get("relationType") or data.get("relation_type"),
|
||||
target=data.get("to") or data.get("to_id"),
|
||||
)
|
||||
)
|
||||
progress.update(read_task, advance=1)
|
||||
|
||||
# Second pass - create and write entities
|
||||
write_task = progress.add_task("Creating entities...", total=len(entities))
|
||||
|
||||
entities_created = 0
|
||||
for name, entity_data in entities.items():
|
||||
entity = EntityMarkdown(
|
||||
frontmatter=EntityFrontmatter(
|
||||
metadata={
|
||||
"type": entity_data["entityType"],
|
||||
"title": name,
|
||||
"permalink": f"{entity_data['entityType']}/{name}",
|
||||
}
|
||||
),
|
||||
content=f"# {name}\n",
|
||||
observations=[Observation(content=obs) for obs in entity_data["observations"]],
|
||||
relations=entity_relations.get(
|
||||
name, []
|
||||
), # Add any relations where this entity is the source
|
||||
)
|
||||
|
||||
# Let markdown processor handle writing
|
||||
file_path = base_path / f"{entity_data['entityType']}/{name}.md"
|
||||
await markdown_processor.write_file(file_path, entity)
|
||||
entities_created += 1
|
||||
progress.update(write_task, advance=1)
|
||||
|
||||
return {
|
||||
"entities": entities_created,
|
||||
"relations": sum(len(rels) for rels in entity_relations.values()),
|
||||
}
|
||||
|
||||
|
||||
async def get_markdown_processor() -> MarkdownProcessor:
|
||||
"""Get MarkdownProcessor instance."""
|
||||
entity_parser = EntityParser(config.home)
|
||||
@@ -102,6 +28,9 @@ def memory_json(
|
||||
json_path: Annotated[Path, typer.Argument(..., help="Path to memory.json file")] = Path(
|
||||
"memory.json"
|
||||
),
|
||||
destination_folder: Annotated[
|
||||
str, typer.Option(help="Optional destination folder within the project")
|
||||
] = "",
|
||||
):
|
||||
"""Import entities and relations from a memory.json file.
|
||||
|
||||
@@ -121,17 +50,31 @@ def memory_json(
|
||||
# Get markdown processor
|
||||
markdown_processor = asyncio.run(get_markdown_processor())
|
||||
|
||||
# Create the importer
|
||||
importer = MemoryJsonImporter(config.home, markdown_processor)
|
||||
|
||||
# Process the file
|
||||
base_path = config.home
|
||||
base_path = config.home if not destination_folder else config.home / destination_folder
|
||||
console.print(f"\nImporting from {json_path}...writing to {base_path}")
|
||||
results = asyncio.run(process_memory_json(json_path, base_path, markdown_processor))
|
||||
|
||||
# Run the import for json log format
|
||||
file_data = []
|
||||
with json_path.open("r", encoding="utf-8") as file:
|
||||
for line in file:
|
||||
json_data = json.loads(line)
|
||||
file_data.append(json_data)
|
||||
result = asyncio.run(importer.import_data(file_data, destination_folder))
|
||||
|
||||
if not result.success: # pragma: no cover
|
||||
typer.echo(f"Error during import: {result.error_message}", err=True)
|
||||
raise typer.Exit(1)
|
||||
|
||||
# Show results
|
||||
console.print(
|
||||
Panel(
|
||||
f"[green]Import complete![/green]\n\n"
|
||||
f"Created {results['entities']} entities\n"
|
||||
f"Added {results['relations']} relations",
|
||||
f"Created {result.entities} entities\n"
|
||||
f"Added {result.relations} relations",
|
||||
expand=False,
|
||||
)
|
||||
)
|
||||
|
||||
@@ -1,6 +1,8 @@
|
||||
"""MCP server command."""
|
||||
"""MCP server command with streamable HTTP transport."""
|
||||
|
||||
import asyncio
|
||||
import typer
|
||||
|
||||
import basic_memory
|
||||
from basic_memory.cli.app import app
|
||||
|
||||
# Import mcp instance
|
||||
@@ -9,27 +11,78 @@ from basic_memory.mcp.server import mcp as mcp_server # pragma: no cover
|
||||
# Import mcp tools to register them
|
||||
import basic_memory.mcp.tools # noqa: F401 # pragma: no cover
|
||||
|
||||
# Import prompts to register them
|
||||
import basic_memory.mcp.prompts # noqa: F401 # pragma: no cover
|
||||
from loguru import logger
|
||||
|
||||
|
||||
@app.command()
|
||||
def mcp(): # pragma: no cover
|
||||
"""Run the MCP server"""
|
||||
from basic_memory.config import config
|
||||
import asyncio
|
||||
from basic_memory.services.initialization import initialize_database
|
||||
def mcp(
|
||||
transport: str = typer.Option("stdio", help="Transport type: stdio, streamable-http, or sse"),
|
||||
host: str = typer.Option(
|
||||
"0.0.0.0", help="Host for HTTP transports (use 0.0.0.0 to allow external connections)"
|
||||
),
|
||||
port: int = typer.Option(8000, help="Port for HTTP transports"),
|
||||
path: str = typer.Option("/mcp", help="Path prefix for streamable-http transport"),
|
||||
): # pragma: no cover
|
||||
"""Run the MCP server with configurable transport options.
|
||||
|
||||
# First, run just the database migrations synchronously
|
||||
asyncio.run(initialize_database(config))
|
||||
This command starts an MCP server using one of three transport options:
|
||||
|
||||
# Load config to check if sync is enabled
|
||||
from basic_memory.config import config_manager
|
||||
- stdio: Standard I/O (good for local usage)
|
||||
- streamable-http: Recommended for web deployments (default)
|
||||
- sse: Server-Sent Events (for compatibility with existing clients)
|
||||
"""
|
||||
|
||||
basic_memory_config = config_manager.load_config()
|
||||
# Check if OAuth is enabled
|
||||
import os
|
||||
|
||||
if basic_memory_config.sync_changes:
|
||||
# For now, we'll just log that sync will be handled by the MCP server
|
||||
from loguru import logger
|
||||
auth_enabled = os.getenv("FASTMCP_AUTH_ENABLED", "false").lower() == "true"
|
||||
if auth_enabled:
|
||||
logger.info("OAuth authentication is ENABLED")
|
||||
logger.info(f"Issuer URL: {os.getenv('FASTMCP_AUTH_ISSUER_URL', 'http://localhost:8000')}")
|
||||
if os.getenv("FASTMCP_AUTH_REQUIRED_SCOPES"):
|
||||
logger.info(f"Required scopes: {os.getenv('FASTMCP_AUTH_REQUIRED_SCOPES')}")
|
||||
else:
|
||||
logger.info("OAuth authentication is DISABLED")
|
||||
|
||||
logger.info("File sync will be handled by the MCP server")
|
||||
from basic_memory.config import app_config
|
||||
from basic_memory.services.initialization import initialize_file_sync
|
||||
|
||||
# Start the MCP server
|
||||
mcp_server.run()
|
||||
# Start the MCP server with the specified transport
|
||||
|
||||
# Use unified thread-based sync approach for both transports
|
||||
import threading
|
||||
|
||||
def run_file_sync():
|
||||
"""Run file sync in a separate thread with its own event loop."""
|
||||
loop = asyncio.new_event_loop()
|
||||
asyncio.set_event_loop(loop)
|
||||
try:
|
||||
loop.run_until_complete(initialize_file_sync(app_config))
|
||||
except Exception as e:
|
||||
logger.error(f"File sync error: {e}", err=True)
|
||||
finally:
|
||||
loop.close()
|
||||
|
||||
logger.info(f"Sync changes enabled: {app_config.sync_changes}")
|
||||
if app_config.sync_changes:
|
||||
# Start the sync thread
|
||||
sync_thread = threading.Thread(target=run_file_sync, daemon=True)
|
||||
sync_thread.start()
|
||||
logger.info("Started file sync in background")
|
||||
|
||||
# Now run the MCP server (blocks)
|
||||
logger.info(f"Starting MCP server with {transport.upper()} transport")
|
||||
|
||||
if transport == "stdio":
|
||||
mcp_server.run(
|
||||
transport=transport,
|
||||
)
|
||||
elif transport == "streamable-http" or transport == "sse":
|
||||
mcp_server.run(
|
||||
transport=transport,
|
||||
host=host,
|
||||
port=port,
|
||||
path=path,
|
||||
)
|
||||
|
||||
@@ -9,13 +9,21 @@ from rich.console import Console
|
||||
from rich.table import Table
|
||||
|
||||
from basic_memory.cli.app import app
|
||||
from basic_memory.config import ConfigManager, config
|
||||
from basic_memory.mcp.tools.project_info import project_info
|
||||
from basic_memory.mcp.project_session import session
|
||||
from basic_memory.mcp.resources.project_info import project_info
|
||||
import json
|
||||
from datetime import datetime
|
||||
|
||||
from rich.panel import Panel
|
||||
from rich.tree import Tree
|
||||
from basic_memory.mcp.async_client import client
|
||||
from basic_memory.mcp.tools.utils import call_get
|
||||
from basic_memory.schemas.project_info import ProjectList
|
||||
from basic_memory.mcp.tools.utils import call_post
|
||||
from basic_memory.schemas.project_info import ProjectStatusResponse
|
||||
from basic_memory.mcp.tools.utils import call_delete
|
||||
from basic_memory.mcp.tools.utils import call_put
|
||||
from basic_memory.utils import generate_permalink
|
||||
|
||||
console = Console()
|
||||
|
||||
@@ -28,112 +36,120 @@ def format_path(path: str) -> str:
|
||||
"""Format a path for display, using ~ for home directory."""
|
||||
home = str(Path.home())
|
||||
if path.startswith(home):
|
||||
return path.replace(home, "~", 1)
|
||||
return path.replace(home, "~", 1) # pragma: no cover
|
||||
return path
|
||||
|
||||
|
||||
@project_app.command("list")
|
||||
def list_projects() -> None:
|
||||
"""List all configured projects."""
|
||||
config_manager = ConfigManager()
|
||||
projects = config_manager.projects
|
||||
# Use API to list projects
|
||||
try:
|
||||
response = asyncio.run(call_get(client, "/projects/projects"))
|
||||
result = ProjectList.model_validate(response.json())
|
||||
|
||||
table = Table(title="Basic Memory Projects")
|
||||
table.add_column("Name", style="cyan")
|
||||
table.add_column("Path", style="green")
|
||||
table.add_column("Default", style="yellow")
|
||||
table.add_column("Active", style="magenta")
|
||||
table = Table(title="Basic Memory Projects")
|
||||
table.add_column("Name", style="cyan")
|
||||
table.add_column("Path", style="green")
|
||||
table.add_column("Default", style="yellow")
|
||||
table.add_column("Active", style="magenta")
|
||||
|
||||
default_project = config_manager.default_project
|
||||
active_project = config.project
|
||||
for project in result.projects:
|
||||
is_default = "✓" if project.is_default else ""
|
||||
is_active = "✓" if session.get_current_project() == project.name else ""
|
||||
table.add_row(project.name, format_path(project.path), is_default, is_active)
|
||||
|
||||
for name, path in projects.items():
|
||||
is_default = "✓" if name == default_project else ""
|
||||
is_active = "✓" if name == active_project else ""
|
||||
table.add_row(name, format_path(path), is_default, is_active)
|
||||
|
||||
console.print(table)
|
||||
console.print(table)
|
||||
except Exception as e:
|
||||
console.print(f"[red]Error listing projects: {str(e)}[/red]")
|
||||
raise typer.Exit(1)
|
||||
|
||||
|
||||
@project_app.command("add")
|
||||
def add_project(
|
||||
name: str = typer.Argument(..., help="Name of the project"),
|
||||
path: str = typer.Argument(..., help="Path to the project directory"),
|
||||
set_default: bool = typer.Option(False, "--default", help="Set as default project"),
|
||||
) -> None:
|
||||
"""Add a new project."""
|
||||
config_manager = ConfigManager()
|
||||
# Resolve to absolute path
|
||||
resolved_path = os.path.abspath(os.path.expanduser(path))
|
||||
|
||||
try:
|
||||
# Resolve to absolute path
|
||||
resolved_path = os.path.abspath(os.path.expanduser(path))
|
||||
config_manager.add_project(name, resolved_path)
|
||||
console.print(f"[green]Project '{name}' added at {format_path(resolved_path)}[/green]")
|
||||
data = {"name": name, "path": resolved_path, "set_default": set_default}
|
||||
|
||||
# Display usage hint
|
||||
console.print("\nTo use this project:")
|
||||
console.print(f" basic-memory --project={name} <command>")
|
||||
console.print(" # or")
|
||||
console.print(f" basic-memory project default {name}")
|
||||
except ValueError as e:
|
||||
console.print(f"[red]Error: {e}[/red]")
|
||||
response = asyncio.run(call_post(client, "/projects/projects", json=data))
|
||||
result = ProjectStatusResponse.model_validate(response.json())
|
||||
|
||||
console.print(f"[green]{result.message}[/green]")
|
||||
except Exception as e:
|
||||
console.print(f"[red]Error adding project: {str(e)}[/red]")
|
||||
raise typer.Exit(1)
|
||||
|
||||
# Display usage hint
|
||||
console.print("\nTo use this project:")
|
||||
console.print(f" basic-memory --project={name} <command>")
|
||||
console.print(" # or")
|
||||
console.print(f" basic-memory project default {name}")
|
||||
|
||||
|
||||
@project_app.command("remove")
|
||||
def remove_project(
|
||||
name: str = typer.Argument(..., help="Name of the project to remove"),
|
||||
) -> None:
|
||||
"""Remove a project from configuration."""
|
||||
config_manager = ConfigManager()
|
||||
|
||||
try:
|
||||
config_manager.remove_project(name)
|
||||
console.print(f"[green]Project '{name}' removed from configuration[/green]")
|
||||
console.print("[yellow]Note: The project files have not been deleted from disk.[/yellow]")
|
||||
except ValueError as e: # pragma: no cover
|
||||
console.print(f"[red]Error: {e}[/red]")
|
||||
project_name = generate_permalink(name)
|
||||
response = asyncio.run(call_delete(client, f"/projects/{project_name}"))
|
||||
result = ProjectStatusResponse.model_validate(response.json())
|
||||
|
||||
console.print(f"[green]{result.message}[/green]")
|
||||
except Exception as e:
|
||||
console.print(f"[red]Error removing project: {str(e)}[/red]")
|
||||
raise typer.Exit(1)
|
||||
|
||||
# Show this message regardless of method used
|
||||
console.print("[yellow]Note: The project files have not been deleted from disk.[/yellow]")
|
||||
|
||||
|
||||
@project_app.command("default")
|
||||
def set_default_project(
|
||||
name: str = typer.Argument(..., help="Name of the project to set as default"),
|
||||
) -> None:
|
||||
"""Set the default project and activate it for the current session."""
|
||||
config_manager = ConfigManager()
|
||||
|
||||
try:
|
||||
# Set the default project
|
||||
config_manager.set_default_project(name)
|
||||
project_name = generate_permalink(name)
|
||||
|
||||
# Also activate it for the current session by setting the environment variable
|
||||
os.environ["BASIC_MEMORY_PROJECT"] = name
|
||||
response = asyncio.run(call_put(client, f"projects/{project_name}/default"))
|
||||
result = ProjectStatusResponse.model_validate(response.json())
|
||||
|
||||
# Reload configuration to apply the change
|
||||
from importlib import reload
|
||||
from basic_memory import config as config_module
|
||||
|
||||
reload(config_module)
|
||||
console.print(f"[green]Project '{name}' set as default and activated[/green]")
|
||||
except ValueError as e: # pragma: no cover
|
||||
console.print(f"[red]Error: {e}[/red]")
|
||||
console.print(f"[green]{result.message}[/green]")
|
||||
except Exception as e:
|
||||
console.print(f"[red]Error setting default project: {str(e)}[/red]")
|
||||
raise typer.Exit(1)
|
||||
|
||||
# Reload configuration to apply the change
|
||||
from importlib import reload
|
||||
from basic_memory import config as config_module
|
||||
|
||||
@project_app.command("current")
|
||||
def show_current_project() -> None:
|
||||
"""Show the current project."""
|
||||
config_manager = ConfigManager()
|
||||
current = os.environ.get("BASIC_MEMORY_PROJECT", config_manager.default_project)
|
||||
reload(config_module)
|
||||
|
||||
console.print("[green]Project activated for current session[/green]")
|
||||
|
||||
|
||||
@project_app.command("sync-config")
|
||||
def synchronize_projects() -> None:
|
||||
"""Synchronize project config between configuration file and database."""
|
||||
# Call the API to synchronize projects
|
||||
|
||||
try:
|
||||
path = config_manager.get_project_path(current)
|
||||
console.print(f"Current project: [cyan]{current}[/cyan]")
|
||||
console.print(f"Path: [green]{format_path(str(path))}[/green]")
|
||||
console.print(f"Database: [blue]{format_path(str(config.database_path))}[/blue]")
|
||||
except ValueError: # pragma: no cover
|
||||
console.print(f"[yellow]Warning: Project '{current}' not found in configuration[/yellow]")
|
||||
console.print(f"Using default project: [cyan]{config_manager.default_project}[/cyan]")
|
||||
response = asyncio.run(call_post(client, "/projects/sync"))
|
||||
result = ProjectStatusResponse.model_validate(response.json())
|
||||
|
||||
console.print(f"[green]{result.message}[/green]")
|
||||
except Exception as e: # pragma: no cover
|
||||
console.print(f"[red]Error synchronizing projects: {str(e)}[/red]")
|
||||
raise typer.Exit(1)
|
||||
|
||||
|
||||
@project_app.command("info")
|
||||
@@ -143,7 +159,7 @@ def display_project_info(
|
||||
"""Display detailed information and statistics about the current project."""
|
||||
try:
|
||||
# Get project info
|
||||
info = asyncio.run(project_info())
|
||||
info = asyncio.run(project_info.fn()) # type: ignore # pyright: ignore [reportAttributeAccessIssue]
|
||||
|
||||
if json_output:
|
||||
# Convert to JSON and print
|
||||
@@ -190,7 +206,7 @@ def display_project_info(
|
||||
console.print(entity_types_table)
|
||||
|
||||
# Most connected entities
|
||||
if info.statistics.most_connected_entities:
|
||||
if info.statistics.most_connected_entities: # pragma: no cover
|
||||
connected_table = Table(title="🔗 Most Connected Entities")
|
||||
connected_table.add_column("Title", style="blue")
|
||||
connected_table.add_column("Permalink", style="cyan")
|
||||
@@ -204,7 +220,7 @@ def display_project_info(
|
||||
console.print(connected_table)
|
||||
|
||||
# Recent activity
|
||||
if info.activity.recently_updated:
|
||||
if info.activity.recently_updated: # pragma: no cover
|
||||
recent_table = Table(title="🕒 Recent Activity")
|
||||
recent_table.add_column("Title", style="blue")
|
||||
recent_table.add_column("Type", style="cyan")
|
||||
@@ -266,9 +282,10 @@ def display_project_info(
|
||||
projects_table.add_column("Path", style="cyan")
|
||||
projects_table.add_column("Default", style="green")
|
||||
|
||||
for name, path in info.available_projects.items():
|
||||
for name, proj_info in info.available_projects.items():
|
||||
is_default = name == info.default_project
|
||||
projects_table.add_row(name, path, "✓" if is_default else "")
|
||||
project_path = proj_info["path"]
|
||||
projects_table.add_row(name, project_path, "✓" if is_default else "")
|
||||
|
||||
console.print(projects_table)
|
||||
|
||||
|
||||
@@ -9,10 +9,11 @@ from rich.console import Console
|
||||
from rich.panel import Panel
|
||||
from rich.tree import Tree
|
||||
|
||||
from basic_memory import db
|
||||
from basic_memory.cli.app import app
|
||||
from basic_memory.cli.commands.sync import get_sync_service
|
||||
from basic_memory.config import config
|
||||
from basic_memory.sync import SyncService
|
||||
from basic_memory.config import config, app_config
|
||||
from basic_memory.repository import ProjectRepository
|
||||
from basic_memory.sync.sync_service import SyncReport
|
||||
|
||||
# Create rich console
|
||||
@@ -86,9 +87,9 @@ def build_directory_summary(counts: Dict[str, int]) -> str:
|
||||
return " ".join(parts)
|
||||
|
||||
|
||||
def display_changes(title: str, changes: SyncReport, verbose: bool = False):
|
||||
def display_changes(project_name: str, title: str, changes: SyncReport, verbose: bool = False):
|
||||
"""Display changes using Rich for better visualization."""
|
||||
tree = Tree(title)
|
||||
tree = Tree(f"{project_name}: {title}")
|
||||
|
||||
if changes.total == 0:
|
||||
tree.add("No changes")
|
||||
@@ -121,11 +122,21 @@ def display_changes(title: str, changes: SyncReport, verbose: bool = False):
|
||||
console.print(Panel(tree, expand=False))
|
||||
|
||||
|
||||
async def run_status(sync_service: SyncService, verbose: bool = False):
|
||||
async def run_status(verbose: bool = False): # pragma: no cover
|
||||
"""Check sync status of files vs database."""
|
||||
# Check knowledge/ directory
|
||||
|
||||
_, session_maker = await db.get_or_create_db(
|
||||
db_path=app_config.database_path, db_type=db.DatabaseType.FILESYSTEM
|
||||
)
|
||||
project_repository = ProjectRepository(session_maker)
|
||||
project = await project_repository.get_by_name(config.project)
|
||||
if not project: # pragma: no cover
|
||||
raise Exception(f"Project '{config.project}' not found")
|
||||
|
||||
sync_service = await get_sync_service(project)
|
||||
knowledge_changes = await sync_service.scan(config.home)
|
||||
display_changes("Status", knowledge_changes, verbose)
|
||||
display_changes(project.name, "Status", knowledge_changes, verbose)
|
||||
|
||||
|
||||
@app.command()
|
||||
@@ -134,9 +145,8 @@ def status(
|
||||
):
|
||||
"""Show sync status between files and database."""
|
||||
try:
|
||||
sync_service = asyncio.run(get_sync_service())
|
||||
asyncio.run(run_status(sync_service, verbose)) # pragma: no cover
|
||||
asyncio.run(run_status(verbose)) # pragma: no cover
|
||||
except Exception as e:
|
||||
logger.exception(f"Error checking status: {e}")
|
||||
logger.error(f"Error checking status: {e}")
|
||||
typer.echo(f"Error checking status: {e}", err=True)
|
||||
raise typer.Exit(code=1) # pragma: no cover
|
||||
|
||||
@@ -16,10 +16,12 @@ from basic_memory.cli.app import app
|
||||
from basic_memory.config import config
|
||||
from basic_memory.markdown import EntityParser
|
||||
from basic_memory.markdown.markdown_processor import MarkdownProcessor
|
||||
from basic_memory.models import Project
|
||||
from basic_memory.repository import (
|
||||
EntityRepository,
|
||||
ObservationRepository,
|
||||
RelationRepository,
|
||||
ProjectRepository,
|
||||
)
|
||||
from basic_memory.repository.search_repository import SearchRepository
|
||||
from basic_memory.services import EntityService, FileService
|
||||
@@ -27,7 +29,7 @@ from basic_memory.services.link_resolver import LinkResolver
|
||||
from basic_memory.services.search_service import SearchService
|
||||
from basic_memory.sync import SyncService
|
||||
from basic_memory.sync.sync_service import SyncReport
|
||||
from basic_memory.sync.watch_service import WatchService
|
||||
from basic_memory.config import app_config
|
||||
|
||||
console = Console()
|
||||
|
||||
@@ -38,21 +40,22 @@ class ValidationIssue:
|
||||
error: str
|
||||
|
||||
|
||||
async def get_sync_service(): # pragma: no cover
|
||||
async def get_sync_service(project: Project) -> SyncService: # pragma: no cover
|
||||
"""Get sync service instance with all dependencies."""
|
||||
_, session_maker = await db.get_or_create_db(
|
||||
db_path=config.database_path, db_type=db.DatabaseType.FILESYSTEM
|
||||
db_path=app_config.database_path, db_type=db.DatabaseType.FILESYSTEM
|
||||
)
|
||||
|
||||
entity_parser = EntityParser(config.home)
|
||||
project_path = Path(project.path)
|
||||
entity_parser = EntityParser(project_path)
|
||||
markdown_processor = MarkdownProcessor(entity_parser)
|
||||
file_service = FileService(config.home, markdown_processor)
|
||||
file_service = FileService(project_path, markdown_processor)
|
||||
|
||||
# Initialize repositories
|
||||
entity_repository = EntityRepository(session_maker)
|
||||
observation_repository = ObservationRepository(session_maker)
|
||||
relation_repository = RelationRepository(session_maker)
|
||||
search_repository = SearchRepository(session_maker)
|
||||
entity_repository = EntityRepository(session_maker, project_id=project.id)
|
||||
observation_repository = ObservationRepository(session_maker, project_id=project.id)
|
||||
relation_repository = RelationRepository(session_maker, project_id=project.id)
|
||||
search_repository = SearchRepository(session_maker, project_id=project.id)
|
||||
|
||||
# Initialize services
|
||||
search_service = SearchService(search_repository, entity_repository, file_service)
|
||||
@@ -70,7 +73,7 @@ async def get_sync_service(): # pragma: no cover
|
||||
|
||||
# Create sync service
|
||||
sync_service = SyncService(
|
||||
config=config,
|
||||
app_config=app_config,
|
||||
entity_service=entity_service,
|
||||
entity_parser=entity_parser,
|
||||
entity_repository=entity_repository,
|
||||
@@ -153,8 +156,16 @@ def display_detailed_sync_results(knowledge: SyncReport):
|
||||
console.print(knowledge_tree)
|
||||
|
||||
|
||||
async def run_sync(verbose: bool = False, watch: bool = False, console_status: bool = False):
|
||||
async def run_sync(verbose: bool = False):
|
||||
"""Run sync operation."""
|
||||
_, session_maker = await db.get_or_create_db(
|
||||
db_path=app_config.database_path, db_type=db.DatabaseType.FILESYSTEM
|
||||
)
|
||||
project_repository = ProjectRepository(session_maker)
|
||||
project = await project_repository.get_by_name(config.project)
|
||||
if not project: # pragma: no cover
|
||||
raise Exception(f"Project '{config.project}' not found")
|
||||
|
||||
import time
|
||||
|
||||
start_time = time.time()
|
||||
@@ -162,50 +173,33 @@ async def run_sync(verbose: bool = False, watch: bool = False, console_status: b
|
||||
logger.info(
|
||||
"Sync command started",
|
||||
project=config.project,
|
||||
watch_mode=watch,
|
||||
verbose=verbose,
|
||||
directory=str(config.home),
|
||||
)
|
||||
|
||||
sync_service = await get_sync_service()
|
||||
sync_service = await get_sync_service(project)
|
||||
|
||||
# Start watching if requested
|
||||
if watch:
|
||||
logger.info("Starting watch service after initial sync")
|
||||
watch_service = WatchService(
|
||||
sync_service=sync_service,
|
||||
file_service=sync_service.entity_service.file_service,
|
||||
config=config,
|
||||
)
|
||||
logger.info("Running one-time sync")
|
||||
knowledge_changes = await sync_service.sync(config.home, project_name=project.name)
|
||||
|
||||
# full sync - no progress bars in watch mode
|
||||
await sync_service.sync(config.home)
|
||||
# Log results
|
||||
duration_ms = int((time.time() - start_time) * 1000)
|
||||
logger.info(
|
||||
"Sync command completed",
|
||||
project=config.project,
|
||||
total_changes=knowledge_changes.total,
|
||||
new_files=len(knowledge_changes.new),
|
||||
modified_files=len(knowledge_changes.modified),
|
||||
deleted_files=len(knowledge_changes.deleted),
|
||||
moved_files=len(knowledge_changes.moves),
|
||||
duration_ms=duration_ms,
|
||||
)
|
||||
|
||||
# watch changes
|
||||
await watch_service.run() # pragma: no cover
|
||||
# Display results
|
||||
if verbose:
|
||||
display_detailed_sync_results(knowledge_changes)
|
||||
else:
|
||||
# one time sync
|
||||
logger.info("Running one-time sync")
|
||||
knowledge_changes = await sync_service.sync(config.home)
|
||||
|
||||
# Log results
|
||||
duration_ms = int((time.time() - start_time) * 1000)
|
||||
logger.info(
|
||||
"Sync command completed",
|
||||
project=config.project,
|
||||
total_changes=knowledge_changes.total,
|
||||
new_files=len(knowledge_changes.new),
|
||||
modified_files=len(knowledge_changes.modified),
|
||||
deleted_files=len(knowledge_changes.deleted),
|
||||
moved_files=len(knowledge_changes.moves),
|
||||
duration_ms=duration_ms,
|
||||
)
|
||||
|
||||
# Display results
|
||||
if verbose:
|
||||
display_detailed_sync_results(knowledge_changes)
|
||||
else:
|
||||
display_sync_summary(knowledge_changes) # pragma: no cover
|
||||
display_sync_summary(knowledge_changes) # pragma: no cover
|
||||
|
||||
|
||||
@app.command()
|
||||
@@ -216,22 +210,15 @@ def sync(
|
||||
"-v",
|
||||
help="Show detailed sync information.",
|
||||
),
|
||||
watch: bool = typer.Option(
|
||||
False,
|
||||
"--watch",
|
||||
"-w",
|
||||
help="Start watching for changes after sync.",
|
||||
),
|
||||
) -> None:
|
||||
"""Sync knowledge files with the database."""
|
||||
try:
|
||||
# Show which project we're syncing
|
||||
if not watch: # Don't show in watch mode as it would break the UI
|
||||
typer.echo(f"Syncing project: {config.project}")
|
||||
typer.echo(f"Project path: {config.home}")
|
||||
typer.echo(f"Syncing project: {config.project}")
|
||||
typer.echo(f"Project path: {config.home}")
|
||||
|
||||
# Run sync
|
||||
asyncio.run(run_sync(verbose=verbose, watch=watch))
|
||||
asyncio.run(run_sync(verbose=verbose))
|
||||
|
||||
except Exception as e: # pragma: no cover
|
||||
if not isinstance(e, typer.Exit):
|
||||
@@ -240,7 +227,6 @@ def sync(
|
||||
f"project={config.project},"
|
||||
f"error={str(e)},"
|
||||
f"error_type={type(e).__name__},"
|
||||
f"watch_mode={watch},"
|
||||
f"directory={str(config.home)}",
|
||||
)
|
||||
typer.echo(f"Error during sync: {e}", err=True)
|
||||
|
||||
@@ -90,7 +90,7 @@ def write_note(
|
||||
typer.echo("Empty content provided. Please provide non-empty content.", err=True)
|
||||
raise typer.Exit(1)
|
||||
|
||||
note = asyncio.run(mcp_write_note(title, content, folder, tags))
|
||||
note = asyncio.run(mcp_write_note.fn(title, content, folder, tags))
|
||||
rprint(note)
|
||||
except Exception as e: # pragma: no cover
|
||||
if not isinstance(e, typer.Exit):
|
||||
@@ -103,7 +103,7 @@ def write_note(
|
||||
def read_note(identifier: str, page: int = 1, page_size: int = 10):
|
||||
"""Read a markdown note from the knowledge base."""
|
||||
try:
|
||||
note = asyncio.run(mcp_read_note(identifier, page, page_size))
|
||||
note = asyncio.run(mcp_read_note.fn(identifier, page, page_size))
|
||||
rprint(note)
|
||||
except Exception as e: # pragma: no cover
|
||||
if not isinstance(e, typer.Exit):
|
||||
@@ -124,7 +124,7 @@ def build_context(
|
||||
"""Get context needed to continue a discussion."""
|
||||
try:
|
||||
context = asyncio.run(
|
||||
mcp_build_context(
|
||||
mcp_build_context.fn(
|
||||
url=url,
|
||||
depth=depth,
|
||||
timeframe=timeframe,
|
||||
@@ -157,7 +157,7 @@ def recent_activity(
|
||||
"""Get recent activity across the knowledge base."""
|
||||
try:
|
||||
context = asyncio.run(
|
||||
mcp_recent_activity(
|
||||
mcp_recent_activity.fn(
|
||||
type=type, # pyright: ignore [reportArgumentType]
|
||||
depth=depth,
|
||||
timeframe=timeframe,
|
||||
@@ -210,7 +210,7 @@ def search_notes(
|
||||
search_type = "text" if search_type is None else search_type
|
||||
|
||||
results = asyncio.run(
|
||||
mcp_search(
|
||||
mcp_search.fn(
|
||||
query,
|
||||
search_type=search_type,
|
||||
page=page,
|
||||
@@ -241,7 +241,7 @@ def continue_conversation(
|
||||
"""Prompt to continue a previous conversation or work session."""
|
||||
try:
|
||||
# Prompt functions return formatted strings directly
|
||||
session = asyncio.run(mcp_continue_conversation(topic=topic, timeframe=timeframe))
|
||||
session = asyncio.run(mcp_continue_conversation.fn(topic=topic, timeframe=timeframe)) # type: ignore
|
||||
rprint(session)
|
||||
except Exception as e: # pragma: no cover
|
||||
if not isinstance(e, typer.Exit):
|
||||
|
||||
@@ -4,6 +4,7 @@ from basic_memory.cli.app import app # pragma: no cover
|
||||
|
||||
# Register commands
|
||||
from basic_memory.cli.commands import ( # noqa: F401 # pragma: no cover
|
||||
auth,
|
||||
db,
|
||||
import_chatgpt,
|
||||
import_claude_conversations,
|
||||
@@ -15,12 +16,7 @@ from basic_memory.cli.commands import ( # noqa: F401 # pragma: no cover
|
||||
sync,
|
||||
tool,
|
||||
)
|
||||
from basic_memory.config import config
|
||||
from basic_memory.services.initialization import ensure_initialization
|
||||
|
||||
if __name__ == "__main__": # pragma: no cover
|
||||
# Run initialization if we are starting as a module
|
||||
ensure_initialization(config)
|
||||
|
||||
# start the app
|
||||
app()
|
||||
|
||||
+159
-88
@@ -2,76 +2,48 @@
|
||||
|
||||
import json
|
||||
import os
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import Any, Dict, Literal, Optional
|
||||
from typing import Any, Dict, Literal, Optional, List, Tuple
|
||||
|
||||
from loguru import logger
|
||||
from pydantic import Field, field_validator
|
||||
from pydantic_settings import BaseSettings, SettingsConfigDict
|
||||
|
||||
import basic_memory
|
||||
from basic_memory.utils import setup_logging
|
||||
from basic_memory.utils import setup_logging, generate_permalink
|
||||
|
||||
|
||||
DATABASE_NAME = "memory.db"
|
||||
APP_DATABASE_NAME = "memory.db" # Using the same name but in the app directory
|
||||
DATA_DIR_NAME = ".basic-memory"
|
||||
CONFIG_FILE_NAME = "config.json"
|
||||
WATCH_STATUS_JSON = "watch-status.json"
|
||||
|
||||
Environment = Literal["test", "dev", "user"]
|
||||
|
||||
|
||||
class ProjectConfig(BaseSettings):
|
||||
@dataclass
|
||||
class ProjectConfig:
|
||||
"""Configuration for a specific basic-memory project."""
|
||||
|
||||
env: Environment = Field(default="dev", description="Environment name")
|
||||
|
||||
# Default to ~/basic-memory but allow override with env var: BASIC_MEMORY_HOME
|
||||
home: Path = Field(
|
||||
default_factory=lambda: Path.home() / "basic-memory",
|
||||
description="Base path for basic-memory files",
|
||||
)
|
||||
|
||||
# Name of the project
|
||||
project: str = Field(default="default", description="Project name")
|
||||
|
||||
# Watch service configuration
|
||||
sync_delay: int = Field(
|
||||
default=1000, description="Milliseconds to wait after changes before syncing", gt=0
|
||||
)
|
||||
|
||||
# update permalinks on move
|
||||
update_permalinks_on_move: bool = Field(
|
||||
default=False,
|
||||
description="Whether to update permalinks when files are moved or renamed. default (False)",
|
||||
)
|
||||
|
||||
model_config = SettingsConfigDict(
|
||||
env_prefix="BASIC_MEMORY_",
|
||||
extra="ignore",
|
||||
env_file=".env",
|
||||
env_file_encoding="utf-8",
|
||||
)
|
||||
name: str
|
||||
home: Path
|
||||
|
||||
@property
|
||||
def database_path(self) -> Path:
|
||||
"""Get SQLite database path."""
|
||||
database_path = self.home / DATA_DIR_NAME / DATABASE_NAME
|
||||
if not database_path.exists():
|
||||
database_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
database_path.touch()
|
||||
return database_path
|
||||
def project(self):
|
||||
return self.name
|
||||
|
||||
@field_validator("home")
|
||||
@classmethod
|
||||
def ensure_path_exists(cls, v: Path) -> Path: # pragma: no cover
|
||||
"""Ensure project path exists."""
|
||||
if not v.exists():
|
||||
v.mkdir(parents=True)
|
||||
return v
|
||||
@property
|
||||
def project_url(self) -> str: # pragma: no cover
|
||||
return f"/{generate_permalink(self.name)}"
|
||||
|
||||
|
||||
class BasicMemoryConfig(BaseSettings):
|
||||
"""Pydantic model for Basic Memory global configuration."""
|
||||
|
||||
env: Environment = Field(default="dev", description="Environment name")
|
||||
|
||||
projects: Dict[str, str] = Field(
|
||||
default_factory=lambda: {"main": str(Path.home() / "basic-memory")},
|
||||
description="Mapping of project names to their filesystem paths",
|
||||
@@ -81,8 +53,15 @@ class BasicMemoryConfig(BaseSettings):
|
||||
description="Name of the default project to use",
|
||||
)
|
||||
|
||||
# overridden by ~/.basic-memory/config.json
|
||||
log_level: str = "INFO"
|
||||
|
||||
# Watch service configuration
|
||||
sync_delay: int = Field(
|
||||
default=1000, description="Milliseconds to wait after changes before syncing", gt=0
|
||||
)
|
||||
|
||||
# update permalinks on move
|
||||
update_permalinks_on_move: bool = Field(
|
||||
default=False,
|
||||
description="Whether to update permalinks when files are moved or renamed. default (False)",
|
||||
@@ -96,25 +75,84 @@ class BasicMemoryConfig(BaseSettings):
|
||||
model_config = SettingsConfigDict(
|
||||
env_prefix="BASIC_MEMORY_",
|
||||
extra="ignore",
|
||||
env_file=".env",
|
||||
env_file_encoding="utf-8",
|
||||
)
|
||||
|
||||
def get_project_path(self, project_name: Optional[str] = None) -> Path: # pragma: no cover
|
||||
"""Get the path for a specific project or the default project."""
|
||||
name = project_name or self.default_project
|
||||
|
||||
if name not in self.projects:
|
||||
raise ValueError(f"Project '{name}' not found in configuration")
|
||||
|
||||
return Path(self.projects[name])
|
||||
|
||||
def model_post_init(self, __context: Any) -> None:
|
||||
"""Ensure configuration is valid after initialization."""
|
||||
# Ensure main project exists
|
||||
if "main" not in self.projects:
|
||||
if "main" not in self.projects: # pragma: no cover
|
||||
self.projects["main"] = str(Path.home() / "basic-memory")
|
||||
|
||||
# Ensure default project is valid
|
||||
if self.default_project not in self.projects:
|
||||
if self.default_project not in self.projects: # pragma: no cover
|
||||
self.default_project = "main"
|
||||
|
||||
@property
|
||||
def app_database_path(self) -> Path:
|
||||
"""Get the path to the app-level database.
|
||||
|
||||
This is the single database that will store all knowledge data
|
||||
across all projects.
|
||||
"""
|
||||
database_path = Path.home() / DATA_DIR_NAME / APP_DATABASE_NAME
|
||||
if not database_path.exists(): # pragma: no cover
|
||||
database_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
database_path.touch()
|
||||
return database_path
|
||||
|
||||
@property
|
||||
def database_path(self) -> Path:
|
||||
"""Get SQLite database path.
|
||||
|
||||
Rreturns the app-level database path
|
||||
for backward compatibility in the codebase.
|
||||
"""
|
||||
|
||||
# Load the app-level database path from the global config
|
||||
config = config_manager.load_config() # pragma: no cover
|
||||
return config.app_database_path # pragma: no cover
|
||||
|
||||
@property
|
||||
def project_list(self) -> List[ProjectConfig]: # pragma: no cover
|
||||
"""Get all configured projects as ProjectConfig objects."""
|
||||
return [ProjectConfig(name=name, home=Path(path)) for name, path in self.projects.items()]
|
||||
|
||||
@field_validator("projects")
|
||||
@classmethod
|
||||
def ensure_project_paths_exists(cls, v: Dict[str, str]) -> Dict[str, str]: # pragma: no cover
|
||||
"""Ensure project path exists."""
|
||||
for name, path_value in v.items():
|
||||
path = Path(path_value)
|
||||
if not Path(path).exists():
|
||||
try:
|
||||
path.mkdir(parents=True)
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to create project path: {e}")
|
||||
raise e
|
||||
return v
|
||||
|
||||
|
||||
class ConfigManager:
|
||||
"""Manages Basic Memory configuration."""
|
||||
|
||||
def __init__(self) -> None:
|
||||
"""Initialize the configuration manager."""
|
||||
self.config_dir = Path.home() / DATA_DIR_NAME
|
||||
home = os.getenv("HOME", Path.home())
|
||||
if isinstance(home, str):
|
||||
home = Path(home)
|
||||
|
||||
self.config_dir = home / DATA_DIR_NAME
|
||||
self.config_file = self.config_dir / CONFIG_FILE_NAME
|
||||
|
||||
# Ensure config directory exists
|
||||
@@ -129,7 +167,7 @@ class ConfigManager:
|
||||
try:
|
||||
data = json.loads(self.config_file.read_text(encoding="utf-8"))
|
||||
return BasicMemoryConfig(**data)
|
||||
except Exception as e:
|
||||
except Exception as e: # pragma: no cover
|
||||
logger.error(f"Failed to load config: {e}")
|
||||
config = BasicMemoryConfig()
|
||||
self.save_config(config)
|
||||
@@ -156,37 +194,28 @@ class ConfigManager:
|
||||
"""Get the default project name."""
|
||||
return self.config.default_project
|
||||
|
||||
def get_project_path(self, project_name: Optional[str] = None) -> Path:
|
||||
"""Get the path for a specific project or the default project."""
|
||||
name = project_name or self.config.default_project
|
||||
|
||||
# Check if specified in environment variable
|
||||
if not project_name and "BASIC_MEMORY_PROJECT" in os.environ:
|
||||
name = os.environ["BASIC_MEMORY_PROJECT"]
|
||||
|
||||
if name not in self.config.projects:
|
||||
raise ValueError(f"Project '{name}' not found in configuration")
|
||||
|
||||
return Path(self.config.projects[name])
|
||||
|
||||
def add_project(self, name: str, path: str) -> None:
|
||||
def add_project(self, name: str, path: str) -> ProjectConfig:
|
||||
"""Add a new project to the configuration."""
|
||||
if name in self.config.projects:
|
||||
project_name, _ = self.get_project(name)
|
||||
if project_name: # pragma: no cover
|
||||
raise ValueError(f"Project '{name}' already exists")
|
||||
|
||||
# Ensure the path exists
|
||||
project_path = Path(path)
|
||||
project_path.mkdir(parents=True, exist_ok=True)
|
||||
project_path.mkdir(parents=True, exist_ok=True) # pragma: no cover
|
||||
|
||||
self.config.projects[name] = str(project_path)
|
||||
self.save_config(self.config)
|
||||
return ProjectConfig(name=name, home=project_path)
|
||||
|
||||
def remove_project(self, name: str) -> None:
|
||||
"""Remove a project from the configuration."""
|
||||
if name not in self.config.projects:
|
||||
|
||||
project_name, path = self.get_project(name)
|
||||
if not project_name: # pragma: no cover
|
||||
raise ValueError(f"Project '{name}' not found")
|
||||
|
||||
if name == self.config.default_project:
|
||||
if project_name == self.config.default_project: # pragma: no cover
|
||||
raise ValueError(f"Cannot remove the default project '{name}'")
|
||||
|
||||
del self.config.projects[name]
|
||||
@@ -194,41 +223,79 @@ class ConfigManager:
|
||||
|
||||
def set_default_project(self, name: str) -> None:
|
||||
"""Set the default project."""
|
||||
if name not in self.config.projects: # pragma: no cover
|
||||
project_name, path = self.get_project(name)
|
||||
if not project_name: # pragma: no cover
|
||||
raise ValueError(f"Project '{name}' not found")
|
||||
|
||||
self.config.default_project = name
|
||||
self.save_config(self.config)
|
||||
|
||||
def get_project(self, name: str) -> Tuple[str, str] | Tuple[None, None]:
|
||||
"""Look up a project from the configuration by name or permalink"""
|
||||
project_permalink = generate_permalink(name)
|
||||
for name, path in app_config.projects.items():
|
||||
if project_permalink == generate_permalink(name):
|
||||
return name, path
|
||||
return None, None
|
||||
|
||||
|
||||
def get_project_config(project_name: Optional[str] = None) -> ProjectConfig:
|
||||
"""Get a project configuration for the specified project."""
|
||||
config_manager = ConfigManager()
|
||||
"""
|
||||
Get the project configuration for the current session.
|
||||
If project_name is provided, it will be used instead of the default project.
|
||||
"""
|
||||
|
||||
# Get project name from environment variable or use provided name or default
|
||||
actual_project_name = os.environ.get(
|
||||
"BASIC_MEMORY_PROJECT", project_name or config_manager.default_project
|
||||
)
|
||||
actual_project_name = None
|
||||
|
||||
update_permalinks_on_move = config_manager.load_config().update_permalinks_on_move
|
||||
try:
|
||||
project_path = config_manager.get_project_path(actual_project_name)
|
||||
return ProjectConfig(
|
||||
home=project_path,
|
||||
project=actual_project_name,
|
||||
update_permalinks_on_move=update_permalinks_on_move,
|
||||
# load the config from file
|
||||
global app_config
|
||||
app_config = config_manager.load_config()
|
||||
|
||||
# Get project name from environment variable
|
||||
os_project_name = os.environ.get("BASIC_MEMORY_PROJECT", None)
|
||||
if os_project_name: # pragma: no cover
|
||||
logger.warning(
|
||||
f"BASIC_MEMORY_PROJECT is not supported anymore. Use the --project flag or set the default project in the config instead. Setting default project to {os_project_name}"
|
||||
)
|
||||
except ValueError: # pragma: no cover
|
||||
logger.warning(f"Project '{actual_project_name}' not found, using default")
|
||||
project_path = config_manager.get_project_path(config_manager.default_project)
|
||||
return ProjectConfig(home=project_path, project=config_manager.default_project)
|
||||
actual_project_name = project_name
|
||||
# if the project_name is passed in, use it
|
||||
elif not project_name:
|
||||
# use default
|
||||
actual_project_name = app_config.default_project
|
||||
else: # pragma: no cover
|
||||
actual_project_name = project_name
|
||||
|
||||
# the config contains a dict[str,str] of project names and absolute paths
|
||||
assert actual_project_name is not None, "actual_project_name cannot be None"
|
||||
|
||||
project_permalink = generate_permalink(actual_project_name)
|
||||
|
||||
for name, path in app_config.projects.items():
|
||||
if project_permalink == generate_permalink(name):
|
||||
return ProjectConfig(name=name, home=Path(path))
|
||||
|
||||
# otherwise raise error
|
||||
raise ValueError(f"Project '{actual_project_name}' not found") # pragma: no cover
|
||||
|
||||
|
||||
# Create config manager
|
||||
config_manager = ConfigManager()
|
||||
|
||||
# Load project config for current context
|
||||
config = get_project_config()
|
||||
# Export the app-level config
|
||||
app_config: BasicMemoryConfig = config_manager.config
|
||||
|
||||
# Load project config for the default project (backward compatibility)
|
||||
config: ProjectConfig = get_project_config()
|
||||
|
||||
|
||||
def update_current_project(project_name: str) -> None:
|
||||
"""Update the global config to use a different project.
|
||||
|
||||
This is used by the CLI when --project flag is specified.
|
||||
"""
|
||||
global config
|
||||
config = get_project_config(project_name) # pragma: no cover
|
||||
|
||||
|
||||
# setup logging to a single log file in user home directory
|
||||
user_home = Path.home()
|
||||
@@ -236,6 +303,7 @@ log_dir = user_home / DATA_DIR_NAME
|
||||
log_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
|
||||
# Process info for logging
|
||||
def get_process_name(): # pragma: no cover
|
||||
"""
|
||||
get the type of process for logging
|
||||
@@ -258,6 +326,9 @@ process_name = get_process_name()
|
||||
_LOGGING_SETUP = False
|
||||
|
||||
|
||||
# Logging
|
||||
|
||||
|
||||
def setup_basic_memory_logging(): # pragma: no cover
|
||||
"""Set up logging for basic-memory, ensuring it only happens once."""
|
||||
global _LOGGING_SETUP
|
||||
@@ -267,7 +338,7 @@ def setup_basic_memory_logging(): # pragma: no cover
|
||||
return
|
||||
|
||||
setup_logging(
|
||||
env=config.env,
|
||||
env=config_manager.config.env,
|
||||
home_dir=user_home, # Use user home for logs
|
||||
log_level=config_manager.load_config().log_level,
|
||||
log_file=f"{DATA_DIR_NAME}/basic-memory-{process_name}.log",
|
||||
|
||||
+47
-11
@@ -4,8 +4,7 @@ from enum import Enum, auto
|
||||
from pathlib import Path
|
||||
from typing import AsyncGenerator, Optional
|
||||
|
||||
|
||||
from basic_memory.config import ProjectConfig
|
||||
from basic_memory.config import BasicMemoryConfig
|
||||
from alembic import command
|
||||
from alembic.config import Config
|
||||
|
||||
@@ -24,6 +23,7 @@ from basic_memory.repository.search_repository import SearchRepository
|
||||
# Module level state
|
||||
_engine: Optional[AsyncEngine] = None
|
||||
_session_maker: Optional[async_sessionmaker[AsyncSession]] = None
|
||||
_migrations_completed: bool = False
|
||||
|
||||
|
||||
class DatabaseType(Enum):
|
||||
@@ -73,18 +73,35 @@ async def scoped_session(
|
||||
await factory.remove()
|
||||
|
||||
|
||||
def _create_engine_and_session(
|
||||
db_path: Path, db_type: DatabaseType = DatabaseType.FILESYSTEM
|
||||
) -> tuple[AsyncEngine, async_sessionmaker[AsyncSession]]:
|
||||
"""Internal helper to create engine and session maker."""
|
||||
db_url = DatabaseType.get_db_url(db_path, db_type)
|
||||
logger.debug(f"Creating engine for db_url: {db_url}")
|
||||
engine = create_async_engine(db_url, connect_args={"check_same_thread": False})
|
||||
session_maker = async_sessionmaker(engine, expire_on_commit=False)
|
||||
return engine, session_maker
|
||||
|
||||
|
||||
async def get_or_create_db(
|
||||
db_path: Path,
|
||||
db_type: DatabaseType = DatabaseType.FILESYSTEM,
|
||||
ensure_migrations: bool = True,
|
||||
app_config: Optional["BasicMemoryConfig"] = None,
|
||||
) -> tuple[AsyncEngine, async_sessionmaker[AsyncSession]]: # pragma: no cover
|
||||
"""Get or create database engine and session maker."""
|
||||
global _engine, _session_maker
|
||||
|
||||
if _engine is None:
|
||||
db_url = DatabaseType.get_db_url(db_path, db_type)
|
||||
logger.debug(f"Creating engine for db_url: {db_url}")
|
||||
_engine = create_async_engine(db_url, connect_args={"check_same_thread": False})
|
||||
_session_maker = async_sessionmaker(_engine, expire_on_commit=False)
|
||||
_engine, _session_maker = _create_engine_and_session(db_path, db_type)
|
||||
|
||||
# Run migrations automatically unless explicitly disabled
|
||||
if ensure_migrations:
|
||||
if app_config is None:
|
||||
from basic_memory.config import app_config as global_app_config
|
||||
app_config = global_app_config
|
||||
await run_migrations(app_config, db_type)
|
||||
|
||||
# These checks should never fail since we just created the engine and session maker
|
||||
# if they were None, but we'll check anyway for the type checker
|
||||
@@ -101,12 +118,13 @@ async def get_or_create_db(
|
||||
|
||||
async def shutdown_db() -> None: # pragma: no cover
|
||||
"""Clean up database connections."""
|
||||
global _engine, _session_maker
|
||||
global _engine, _session_maker, _migrations_completed
|
||||
|
||||
if _engine:
|
||||
await _engine.dispose()
|
||||
_engine = None
|
||||
_session_maker = None
|
||||
_migrations_completed = False
|
||||
|
||||
|
||||
@asynccontextmanager
|
||||
@@ -120,7 +138,7 @@ async def engine_session_factory(
|
||||
for each test. For production use, use get_or_create_db() instead.
|
||||
"""
|
||||
|
||||
global _engine, _session_maker
|
||||
global _engine, _session_maker, _migrations_completed
|
||||
|
||||
db_url = DatabaseType.get_db_url(db_path, db_type)
|
||||
logger.debug(f"Creating engine for db_url: {db_url}")
|
||||
@@ -144,12 +162,20 @@ async def engine_session_factory(
|
||||
await _engine.dispose()
|
||||
_engine = None
|
||||
_session_maker = None
|
||||
_migrations_completed = False
|
||||
|
||||
|
||||
async def run_migrations(
|
||||
app_config: ProjectConfig, database_type=DatabaseType.FILESYSTEM
|
||||
app_config: BasicMemoryConfig, database_type=DatabaseType.FILESYSTEM, force: bool = False
|
||||
): # pragma: no cover
|
||||
"""Run any pending alembic migrations."""
|
||||
global _migrations_completed
|
||||
|
||||
# Skip if migrations already completed unless forced
|
||||
if _migrations_completed and not force:
|
||||
logger.debug("Migrations already completed in this session, skipping")
|
||||
return
|
||||
|
||||
logger.info("Running database migrations...")
|
||||
try:
|
||||
# Get the absolute path to the alembic directory relative to this file
|
||||
@@ -171,8 +197,18 @@ async def run_migrations(
|
||||
command.upgrade(config, "head")
|
||||
logger.info("Migrations completed successfully")
|
||||
|
||||
_, session_maker = await get_or_create_db(app_config.database_path, database_type)
|
||||
await SearchRepository(session_maker).init_search_index()
|
||||
# Get session maker - ensure we don't trigger recursive migration calls
|
||||
if _session_maker is None:
|
||||
_, session_maker = _create_engine_and_session(app_config.database_path, database_type)
|
||||
else:
|
||||
session_maker = _session_maker
|
||||
|
||||
# initialize the search Index schema
|
||||
# the project_id is not used for init_search_index, so we pass a dummy value
|
||||
await SearchRepository(session_maker, 1).init_search_index()
|
||||
|
||||
# Mark migrations as completed
|
||||
_migrations_completed = True
|
||||
except Exception as e: # pragma: no cover
|
||||
logger.error(f"Error running migrations: {e}")
|
||||
raise
|
||||
|
||||
+227
-30
@@ -1,50 +1,87 @@
|
||||
"""Dependency injection functions for basic-memory services."""
|
||||
|
||||
from typing import Annotated
|
||||
from loguru import logger
|
||||
|
||||
from fastapi import Depends
|
||||
from fastapi import Depends, HTTPException, Path, status
|
||||
from sqlalchemy.ext.asyncio import (
|
||||
AsyncSession,
|
||||
AsyncEngine,
|
||||
async_sessionmaker,
|
||||
)
|
||||
import pathlib
|
||||
|
||||
from basic_memory import db
|
||||
from basic_memory.config import ProjectConfig, config
|
||||
from basic_memory.config import ProjectConfig, BasicMemoryConfig
|
||||
from basic_memory.importers import (
|
||||
ChatGPTImporter,
|
||||
ClaudeConversationsImporter,
|
||||
ClaudeProjectsImporter,
|
||||
MemoryJsonImporter,
|
||||
)
|
||||
from basic_memory.markdown import EntityParser
|
||||
from basic_memory.markdown.markdown_processor import MarkdownProcessor
|
||||
from basic_memory.repository.entity_repository import EntityRepository
|
||||
from basic_memory.repository.observation_repository import ObservationRepository
|
||||
from basic_memory.repository.project_info_repository import ProjectInfoRepository
|
||||
from basic_memory.repository.project_repository import ProjectRepository
|
||||
from basic_memory.repository.relation_repository import RelationRepository
|
||||
from basic_memory.repository.search_repository import SearchRepository
|
||||
from basic_memory.services import (
|
||||
EntityService,
|
||||
)
|
||||
from basic_memory.services import EntityService, ProjectService
|
||||
from basic_memory.services.context_service import ContextService
|
||||
from basic_memory.services.directory_service import DirectoryService
|
||||
from basic_memory.services.file_service import FileService
|
||||
from basic_memory.services.link_resolver import LinkResolver
|
||||
from basic_memory.services.search_service import SearchService
|
||||
from basic_memory.sync import SyncService
|
||||
from basic_memory.config import app_config
|
||||
|
||||
|
||||
def get_app_config() -> BasicMemoryConfig: # pragma: no cover
|
||||
return app_config
|
||||
|
||||
|
||||
AppConfigDep = Annotated[BasicMemoryConfig, Depends(get_app_config)] # pragma: no cover
|
||||
|
||||
|
||||
## project
|
||||
|
||||
|
||||
def get_project_config() -> ProjectConfig: # pragma: no cover
|
||||
return config
|
||||
async def get_project_config(
|
||||
project: "ProjectPathDep", project_repository: "ProjectRepositoryDep"
|
||||
) -> ProjectConfig: # pragma: no cover
|
||||
"""Get the current project referenced from request state.
|
||||
|
||||
Args:
|
||||
request: The current request object
|
||||
project_repository: Repository for project operations
|
||||
|
||||
Returns:
|
||||
The resolved project config
|
||||
|
||||
Raises:
|
||||
HTTPException: If project is not found
|
||||
"""
|
||||
|
||||
project_obj = await project_repository.get_by_permalink(str(project))
|
||||
if project_obj:
|
||||
return ProjectConfig(name=project_obj.name, home=pathlib.Path(project_obj.path))
|
||||
|
||||
# Not found
|
||||
raise HTTPException( # pragma: no cover
|
||||
status_code=status.HTTP_404_NOT_FOUND, detail=f"Project '{project}' not found."
|
||||
)
|
||||
|
||||
|
||||
ProjectConfigDep = Annotated[ProjectConfig, Depends(get_project_config)] # pragma: no cover
|
||||
|
||||
|
||||
## sqlalchemy
|
||||
|
||||
|
||||
async def get_engine_factory(
|
||||
project_config: ProjectConfigDep,
|
||||
app_config: AppConfigDep,
|
||||
) -> tuple[AsyncEngine, async_sessionmaker[AsyncSession]]: # pragma: no cover
|
||||
"""Get engine and session maker."""
|
||||
engine, session_maker = await db.get_or_create_db(project_config.database_path)
|
||||
engine, session_maker = await db.get_or_create_db(app_config.database_path)
|
||||
return engine, session_maker
|
||||
|
||||
|
||||
@@ -65,11 +102,70 @@ SessionMakerDep = Annotated[async_sessionmaker, Depends(get_session_maker)]
|
||||
## repositories
|
||||
|
||||
|
||||
async def get_project_repository(
|
||||
session_maker: SessionMakerDep,
|
||||
) -> ProjectRepository:
|
||||
"""Get the project repository."""
|
||||
return ProjectRepository(session_maker)
|
||||
|
||||
|
||||
ProjectRepositoryDep = Annotated[ProjectRepository, Depends(get_project_repository)]
|
||||
ProjectPathDep = Annotated[str, Path()] # Use Path dependency to extract from URL
|
||||
|
||||
|
||||
async def get_project_id(
|
||||
project_repository: ProjectRepositoryDep,
|
||||
project: ProjectPathDep,
|
||||
) -> int:
|
||||
"""Get the current project ID from request state.
|
||||
|
||||
When using sub-applications with /{project} mounting, the project value
|
||||
is stored in request.state by middleware.
|
||||
|
||||
Args:
|
||||
request: The current request object
|
||||
project_repository: Repository for project operations
|
||||
|
||||
Returns:
|
||||
The resolved project ID
|
||||
|
||||
Raises:
|
||||
HTTPException: If project is not found
|
||||
"""
|
||||
|
||||
# Try by permalink first (most common case with URL paths)
|
||||
project_obj = await project_repository.get_by_permalink(str(project))
|
||||
if project_obj:
|
||||
return project_obj.id
|
||||
|
||||
# Try by name if permalink lookup fails
|
||||
project_obj = await project_repository.get_by_name(str(project)) # pragma: no cover
|
||||
if project_obj: # pragma: no cover
|
||||
return project_obj.id
|
||||
|
||||
# Not found
|
||||
raise HTTPException( # pragma: no cover
|
||||
status_code=status.HTTP_404_NOT_FOUND, detail=f"Project '{project}' not found."
|
||||
)
|
||||
|
||||
|
||||
"""
|
||||
The project_id dependency is used in the following:
|
||||
- EntityRepository
|
||||
- ObservationRepository
|
||||
- RelationRepository
|
||||
- SearchRepository
|
||||
- ProjectInfoRepository
|
||||
"""
|
||||
ProjectIdDep = Annotated[int, Depends(get_project_id)]
|
||||
|
||||
|
||||
async def get_entity_repository(
|
||||
session_maker: SessionMakerDep,
|
||||
project_id: ProjectIdDep,
|
||||
) -> EntityRepository:
|
||||
"""Create an EntityRepository instance."""
|
||||
return EntityRepository(session_maker)
|
||||
"""Create an EntityRepository instance for the current project."""
|
||||
return EntityRepository(session_maker, project_id=project_id)
|
||||
|
||||
|
||||
EntityRepositoryDep = Annotated[EntityRepository, Depends(get_entity_repository)]
|
||||
@@ -77,9 +173,10 @@ EntityRepositoryDep = Annotated[EntityRepository, Depends(get_entity_repository)
|
||||
|
||||
async def get_observation_repository(
|
||||
session_maker: SessionMakerDep,
|
||||
project_id: ProjectIdDep,
|
||||
) -> ObservationRepository:
|
||||
"""Create an ObservationRepository instance."""
|
||||
return ObservationRepository(session_maker)
|
||||
"""Create an ObservationRepository instance for the current project."""
|
||||
return ObservationRepository(session_maker, project_id=project_id)
|
||||
|
||||
|
||||
ObservationRepositoryDep = Annotated[ObservationRepository, Depends(get_observation_repository)]
|
||||
@@ -87,9 +184,10 @@ ObservationRepositoryDep = Annotated[ObservationRepository, Depends(get_observat
|
||||
|
||||
async def get_relation_repository(
|
||||
session_maker: SessionMakerDep,
|
||||
project_id: ProjectIdDep,
|
||||
) -> RelationRepository:
|
||||
"""Create a RelationRepository instance."""
|
||||
return RelationRepository(session_maker)
|
||||
"""Create a RelationRepository instance for the current project."""
|
||||
return RelationRepository(session_maker, project_id=project_id)
|
||||
|
||||
|
||||
RelationRepositoryDep = Annotated[RelationRepository, Depends(get_relation_repository)]
|
||||
@@ -97,22 +195,17 @@ RelationRepositoryDep = Annotated[RelationRepository, Depends(get_relation_repos
|
||||
|
||||
async def get_search_repository(
|
||||
session_maker: SessionMakerDep,
|
||||
project_id: ProjectIdDep,
|
||||
) -> SearchRepository:
|
||||
"""Create a SearchRepository instance."""
|
||||
return SearchRepository(session_maker)
|
||||
"""Create a SearchRepository instance for the current project."""
|
||||
return SearchRepository(session_maker, project_id=project_id)
|
||||
|
||||
|
||||
SearchRepositoryDep = Annotated[SearchRepository, Depends(get_search_repository)]
|
||||
|
||||
|
||||
def get_project_info_repository(
|
||||
session_maker: SessionMakerDep,
|
||||
):
|
||||
"""Dependency for StatsRepository."""
|
||||
return ProjectInfoRepository(session_maker)
|
||||
|
||||
|
||||
ProjectInfoRepositoryDep = Annotated[ProjectInfoRepository, Depends(get_project_info_repository)]
|
||||
# ProjectInfoRepository is deprecated and will be removed in a future version.
|
||||
# Use ProjectRepository instead, which has the same functionality plus more project-specific operations.
|
||||
|
||||
## services
|
||||
|
||||
@@ -134,7 +227,12 @@ MarkdownProcessorDep = Annotated[MarkdownProcessor, Depends(get_markdown_process
|
||||
async def get_file_service(
|
||||
project_config: ProjectConfigDep, markdown_processor: MarkdownProcessorDep
|
||||
) -> FileService:
|
||||
return FileService(project_config.home, markdown_processor)
|
||||
logger.debug(
|
||||
f"Creating FileService for project: {project_config.name}, base_path: {project_config.home}"
|
||||
)
|
||||
file_service = FileService(project_config.home, markdown_processor)
|
||||
logger.debug(f"Created FileService for project: {file_service} ")
|
||||
return file_service
|
||||
|
||||
|
||||
FileServiceDep = Annotated[FileService, Depends(get_file_service)]
|
||||
@@ -184,9 +282,108 @@ LinkResolverDep = Annotated[LinkResolver, Depends(get_link_resolver)]
|
||||
|
||||
|
||||
async def get_context_service(
|
||||
search_repository: SearchRepositoryDep, entity_repository: EntityRepositoryDep
|
||||
search_repository: SearchRepositoryDep,
|
||||
entity_repository: EntityRepositoryDep,
|
||||
observation_repository: ObservationRepositoryDep,
|
||||
) -> ContextService:
|
||||
return ContextService(search_repository, entity_repository)
|
||||
return ContextService(
|
||||
search_repository=search_repository,
|
||||
entity_repository=entity_repository,
|
||||
observation_repository=observation_repository,
|
||||
)
|
||||
|
||||
|
||||
ContextServiceDep = Annotated[ContextService, Depends(get_context_service)]
|
||||
|
||||
|
||||
async def get_sync_service(
|
||||
entity_service: EntityServiceDep,
|
||||
entity_parser: EntityParserDep,
|
||||
entity_repository: EntityRepositoryDep,
|
||||
relation_repository: RelationRepositoryDep,
|
||||
search_service: SearchServiceDep,
|
||||
file_service: FileServiceDep,
|
||||
) -> SyncService: # pragma: no cover
|
||||
"""
|
||||
|
||||
:rtype: object
|
||||
"""
|
||||
return SyncService(
|
||||
app_config=app_config,
|
||||
entity_service=entity_service,
|
||||
entity_parser=entity_parser,
|
||||
entity_repository=entity_repository,
|
||||
relation_repository=relation_repository,
|
||||
search_service=search_service,
|
||||
file_service=file_service,
|
||||
)
|
||||
|
||||
|
||||
SyncServiceDep = Annotated[SyncService, Depends(get_sync_service)]
|
||||
|
||||
|
||||
async def get_project_service(
|
||||
project_repository: ProjectRepositoryDep,
|
||||
) -> ProjectService:
|
||||
"""Create ProjectService with repository."""
|
||||
return ProjectService(repository=project_repository)
|
||||
|
||||
|
||||
ProjectServiceDep = Annotated[ProjectService, Depends(get_project_service)]
|
||||
|
||||
|
||||
async def get_directory_service(
|
||||
entity_repository: EntityRepositoryDep,
|
||||
) -> DirectoryService:
|
||||
"""Create DirectoryService with dependencies."""
|
||||
return DirectoryService(
|
||||
entity_repository=entity_repository,
|
||||
)
|
||||
|
||||
|
||||
DirectoryServiceDep = Annotated[DirectoryService, Depends(get_directory_service)]
|
||||
|
||||
|
||||
# Import
|
||||
|
||||
|
||||
async def get_chatgpt_importer(
|
||||
project_config: ProjectConfigDep, markdown_processor: MarkdownProcessorDep
|
||||
) -> ChatGPTImporter:
|
||||
"""Create ChatGPTImporter with dependencies."""
|
||||
return ChatGPTImporter(project_config.home, markdown_processor)
|
||||
|
||||
|
||||
ChatGPTImporterDep = Annotated[ChatGPTImporter, Depends(get_chatgpt_importer)]
|
||||
|
||||
|
||||
async def get_claude_conversations_importer(
|
||||
project_config: ProjectConfigDep, markdown_processor: MarkdownProcessorDep
|
||||
) -> ClaudeConversationsImporter:
|
||||
"""Create ChatGPTImporter with dependencies."""
|
||||
return ClaudeConversationsImporter(project_config.home, markdown_processor)
|
||||
|
||||
|
||||
ClaudeConversationsImporterDep = Annotated[
|
||||
ClaudeConversationsImporter, Depends(get_claude_conversations_importer)
|
||||
]
|
||||
|
||||
|
||||
async def get_claude_projects_importer(
|
||||
project_config: ProjectConfigDep, markdown_processor: MarkdownProcessorDep
|
||||
) -> ClaudeProjectsImporter:
|
||||
"""Create ChatGPTImporter with dependencies."""
|
||||
return ClaudeProjectsImporter(project_config.home, markdown_processor)
|
||||
|
||||
|
||||
ClaudeProjectsImporterDep = Annotated[ClaudeProjectsImporter, Depends(get_claude_projects_importer)]
|
||||
|
||||
|
||||
async def get_memory_json_importer(
|
||||
project_config: ProjectConfigDep, markdown_processor: MarkdownProcessorDep
|
||||
) -> MemoryJsonImporter:
|
||||
"""Create ChatGPTImporter with dependencies."""
|
||||
return MemoryJsonImporter(project_config.home, markdown_processor)
|
||||
|
||||
|
||||
MemoryJsonImporterDep = Annotated[MemoryJsonImporter, Depends(get_memory_json_importer)]
|
||||
|
||||
@@ -0,0 +1,27 @@
|
||||
"""Import services for Basic Memory."""
|
||||
|
||||
from basic_memory.importers.base import Importer
|
||||
from basic_memory.importers.chatgpt_importer import ChatGPTImporter
|
||||
from basic_memory.importers.claude_conversations_importer import (
|
||||
ClaudeConversationsImporter,
|
||||
)
|
||||
from basic_memory.importers.claude_projects_importer import ClaudeProjectsImporter
|
||||
from basic_memory.importers.memory_json_importer import MemoryJsonImporter
|
||||
from basic_memory.schemas.importer import (
|
||||
ChatImportResult,
|
||||
EntityImportResult,
|
||||
ImportResult,
|
||||
ProjectImportResult,
|
||||
)
|
||||
|
||||
__all__ = [
|
||||
"Importer",
|
||||
"ChatGPTImporter",
|
||||
"ClaudeConversationsImporter",
|
||||
"ClaudeProjectsImporter",
|
||||
"MemoryJsonImporter",
|
||||
"ImportResult",
|
||||
"ChatImportResult",
|
||||
"EntityImportResult",
|
||||
"ProjectImportResult",
|
||||
]
|
||||
@@ -0,0 +1,79 @@
|
||||
"""Base import service for Basic Memory."""
|
||||
|
||||
import logging
|
||||
from abc import abstractmethod
|
||||
from pathlib import Path
|
||||
from typing import Any, Optional, TypeVar
|
||||
|
||||
from basic_memory.markdown.markdown_processor import MarkdownProcessor
|
||||
from basic_memory.markdown.schemas import EntityMarkdown
|
||||
from basic_memory.schemas.importer import ImportResult
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
T = TypeVar("T", bound=ImportResult)
|
||||
|
||||
|
||||
class Importer[T: ImportResult]:
|
||||
"""Base class for all import services."""
|
||||
|
||||
def __init__(self, base_path: Path, markdown_processor: MarkdownProcessor):
|
||||
"""Initialize the import service.
|
||||
|
||||
Args:
|
||||
markdown_processor: MarkdownProcessor instance for writing markdown files.
|
||||
"""
|
||||
self.base_path = base_path.resolve() # Get absolute path
|
||||
self.markdown_processor = markdown_processor
|
||||
|
||||
@abstractmethod
|
||||
async def import_data(self, source_data, destination_folder: str, **kwargs: Any) -> T:
|
||||
"""Import data from source file to destination folder.
|
||||
|
||||
Args:
|
||||
source_path: Path to the source file.
|
||||
destination_folder: Destination folder within the project.
|
||||
**kwargs: Additional keyword arguments for specific import types.
|
||||
|
||||
Returns:
|
||||
ImportResult containing statistics and status of the import.
|
||||
"""
|
||||
pass # pragma: no cover
|
||||
|
||||
async def write_entity(self, entity: EntityMarkdown, file_path: Path) -> None:
|
||||
"""Write entity to file using markdown processor.
|
||||
|
||||
Args:
|
||||
entity: EntityMarkdown instance to write.
|
||||
file_path: Path to write the entity to.
|
||||
"""
|
||||
await self.markdown_processor.write_file(file_path, entity)
|
||||
|
||||
def ensure_folder_exists(self, folder: str) -> Path:
|
||||
"""Ensure folder exists, create if it doesn't.
|
||||
|
||||
Args:
|
||||
base_path: Base path of the project.
|
||||
folder: Folder name or path within the project.
|
||||
|
||||
Returns:
|
||||
Path to the folder.
|
||||
"""
|
||||
folder_path = self.base_path / folder
|
||||
folder_path.mkdir(parents=True, exist_ok=True)
|
||||
return folder_path
|
||||
|
||||
@abstractmethod
|
||||
def handle_error(
|
||||
self, message: str, error: Optional[Exception] = None
|
||||
) -> T: # pragma: no cover
|
||||
"""Handle errors during import.
|
||||
|
||||
Args:
|
||||
message: Error message.
|
||||
error: Optional exception that caused the error.
|
||||
|
||||
Returns:
|
||||
ImportResult with error information.
|
||||
"""
|
||||
pass
|
||||
@@ -0,0 +1,222 @@
|
||||
"""ChatGPT import service for Basic Memory."""
|
||||
|
||||
import logging
|
||||
from datetime import datetime
|
||||
from typing import Any, Dict, List, Optional, Set
|
||||
|
||||
from basic_memory.markdown.schemas import EntityFrontmatter, EntityMarkdown
|
||||
from basic_memory.importers.base import Importer
|
||||
from basic_memory.schemas.importer import ChatImportResult
|
||||
from basic_memory.importers.utils import clean_filename, format_timestamp
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class ChatGPTImporter(Importer[ChatImportResult]):
|
||||
"""Service for importing ChatGPT conversations."""
|
||||
|
||||
async def import_data(
|
||||
self, source_data, destination_folder: str, **kwargs: Any
|
||||
) -> ChatImportResult:
|
||||
"""Import conversations from ChatGPT JSON export.
|
||||
|
||||
Args:
|
||||
source_path: Path to the ChatGPT conversations.json file.
|
||||
destination_folder: Destination folder within the project.
|
||||
**kwargs: Additional keyword arguments.
|
||||
|
||||
Returns:
|
||||
ChatImportResult containing statistics and status of the import.
|
||||
"""
|
||||
try: # pragma: no cover
|
||||
# Ensure the destination folder exists
|
||||
self.ensure_folder_exists(destination_folder)
|
||||
conversations = source_data
|
||||
|
||||
# Process each conversation
|
||||
messages_imported = 0
|
||||
chats_imported = 0
|
||||
|
||||
for chat in conversations:
|
||||
# Convert to entity
|
||||
entity = self._format_chat_content(destination_folder, chat)
|
||||
|
||||
# Write file
|
||||
file_path = self.base_path / f"{entity.frontmatter.metadata['permalink']}.md"
|
||||
await self.write_entity(entity, file_path)
|
||||
|
||||
# Count messages
|
||||
msg_count = sum(
|
||||
1
|
||||
for node in chat["mapping"].values()
|
||||
if node.get("message")
|
||||
and not node.get("message", {})
|
||||
.get("metadata", {})
|
||||
.get("is_visually_hidden_from_conversation")
|
||||
)
|
||||
|
||||
chats_imported += 1
|
||||
messages_imported += msg_count
|
||||
|
||||
return ChatImportResult(
|
||||
import_count={"conversations": chats_imported, "messages": messages_imported},
|
||||
success=True,
|
||||
conversations=chats_imported,
|
||||
messages=messages_imported,
|
||||
)
|
||||
|
||||
except Exception as e: # pragma: no cover
|
||||
logger.exception("Failed to import ChatGPT conversations")
|
||||
return self.handle_error("Failed to import ChatGPT conversations", e) # pyright: ignore [reportReturnType]
|
||||
|
||||
def _format_chat_content(
|
||||
self, folder: str, conversation: Dict[str, Any]
|
||||
) -> EntityMarkdown: # pragma: no cover
|
||||
"""Convert chat conversation to Basic Memory entity.
|
||||
|
||||
Args:
|
||||
folder: Destination folder name.
|
||||
conversation: ChatGPT conversation data.
|
||||
|
||||
Returns:
|
||||
EntityMarkdown instance representing the conversation.
|
||||
"""
|
||||
# Extract timestamps
|
||||
created_at = conversation["create_time"]
|
||||
modified_at = conversation["update_time"]
|
||||
|
||||
root_id = None
|
||||
# Find root message
|
||||
for node_id, node in conversation["mapping"].items():
|
||||
if node.get("parent") is None:
|
||||
root_id = node_id
|
||||
break
|
||||
|
||||
# Generate permalink
|
||||
date_prefix = datetime.fromtimestamp(created_at).strftime("%Y%m%d")
|
||||
clean_title = clean_filename(conversation["title"])
|
||||
|
||||
# Format content
|
||||
content = self._format_chat_markdown(
|
||||
title=conversation["title"],
|
||||
mapping=conversation["mapping"],
|
||||
root_id=root_id,
|
||||
created_at=created_at,
|
||||
modified_at=modified_at,
|
||||
)
|
||||
|
||||
# Create entity
|
||||
entity = EntityMarkdown(
|
||||
frontmatter=EntityFrontmatter(
|
||||
metadata={
|
||||
"type": "conversation",
|
||||
"title": conversation["title"],
|
||||
"created": format_timestamp(created_at),
|
||||
"modified": format_timestamp(modified_at),
|
||||
"permalink": f"{folder}/{date_prefix}-{clean_title}",
|
||||
}
|
||||
),
|
||||
content=content,
|
||||
)
|
||||
|
||||
return entity
|
||||
|
||||
def _format_chat_markdown(
|
||||
self,
|
||||
title: str,
|
||||
mapping: Dict[str, Any],
|
||||
root_id: Optional[str],
|
||||
created_at: float,
|
||||
modified_at: float,
|
||||
) -> str: # pragma: no cover
|
||||
"""Format chat as clean markdown.
|
||||
|
||||
Args:
|
||||
title: Chat title.
|
||||
mapping: Message mapping.
|
||||
root_id: Root message ID.
|
||||
created_at: Creation timestamp.
|
||||
modified_at: Modification timestamp.
|
||||
|
||||
Returns:
|
||||
Formatted markdown content.
|
||||
"""
|
||||
# Start with title
|
||||
lines = [f"# {title}\n"]
|
||||
|
||||
# Traverse message tree
|
||||
seen_msgs: Set[str] = set()
|
||||
messages = self._traverse_messages(mapping, root_id, seen_msgs)
|
||||
|
||||
# Format each message
|
||||
for msg in messages:
|
||||
# Skip hidden messages
|
||||
if msg.get("metadata", {}).get("is_visually_hidden_from_conversation"):
|
||||
continue
|
||||
|
||||
# Get author and timestamp
|
||||
author = msg["author"]["role"].title()
|
||||
ts = format_timestamp(msg["create_time"]) if msg.get("create_time") else ""
|
||||
|
||||
# Add message header
|
||||
lines.append(f"### {author} ({ts})")
|
||||
|
||||
# Add message content
|
||||
content = self._get_message_content(msg)
|
||||
if content:
|
||||
lines.append(content)
|
||||
|
||||
# Add spacing
|
||||
lines.append("")
|
||||
|
||||
return "\n".join(lines)
|
||||
|
||||
def _get_message_content(self, message: Dict[str, Any]) -> str: # pragma: no cover
|
||||
"""Extract clean message content.
|
||||
|
||||
Args:
|
||||
message: Message data.
|
||||
|
||||
Returns:
|
||||
Cleaned message content.
|
||||
"""
|
||||
if not message or "content" not in message:
|
||||
return ""
|
||||
|
||||
content = message["content"]
|
||||
if content.get("content_type") == "text":
|
||||
return "\n".join(content.get("parts", []))
|
||||
elif content.get("content_type") == "code":
|
||||
return f"```{content.get('language', '')}\n{content.get('text', '')}\n```"
|
||||
return ""
|
||||
|
||||
def _traverse_messages(
|
||||
self, mapping: Dict[str, Any], root_id: Optional[str], seen: Set[str]
|
||||
) -> List[Dict[str, Any]]: # pragma: no cover
|
||||
"""Traverse message tree and return messages in order.
|
||||
|
||||
Args:
|
||||
mapping: Message mapping.
|
||||
root_id: Root message ID.
|
||||
seen: Set of seen message IDs.
|
||||
|
||||
Returns:
|
||||
List of message data.
|
||||
"""
|
||||
messages = []
|
||||
node = mapping.get(root_id) if root_id else None
|
||||
|
||||
while node:
|
||||
if node["id"] not in seen and node.get("message"):
|
||||
seen.add(node["id"])
|
||||
messages.append(node["message"])
|
||||
|
||||
# Follow children
|
||||
children = node.get("children", [])
|
||||
for child_id in children:
|
||||
child_msgs = self._traverse_messages(mapping, child_id, seen)
|
||||
messages.extend(child_msgs)
|
||||
|
||||
break # Don't follow siblings
|
||||
|
||||
return messages
|
||||
@@ -0,0 +1,172 @@
|
||||
"""Claude conversations import service for Basic Memory."""
|
||||
|
||||
import logging
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
from typing import Any, Dict, List
|
||||
|
||||
from basic_memory.markdown.schemas import EntityFrontmatter, EntityMarkdown
|
||||
from basic_memory.importers.base import Importer
|
||||
from basic_memory.schemas.importer import ChatImportResult
|
||||
from basic_memory.importers.utils import clean_filename, format_timestamp
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class ClaudeConversationsImporter(Importer[ChatImportResult]):
|
||||
"""Service for importing Claude conversations."""
|
||||
|
||||
async def import_data(
|
||||
self, source_data, destination_folder: str, **kwargs: Any
|
||||
) -> ChatImportResult:
|
||||
"""Import conversations from Claude JSON export.
|
||||
|
||||
Args:
|
||||
source_data: Path to the Claude conversations.json file.
|
||||
destination_folder: Destination folder within the project.
|
||||
**kwargs: Additional keyword arguments.
|
||||
|
||||
Returns:
|
||||
ChatImportResult containing statistics and status of the import.
|
||||
"""
|
||||
try:
|
||||
# Ensure the destination folder exists
|
||||
folder_path = self.ensure_folder_exists(destination_folder)
|
||||
|
||||
conversations = source_data
|
||||
|
||||
# Process each conversation
|
||||
messages_imported = 0
|
||||
chats_imported = 0
|
||||
|
||||
for chat in conversations:
|
||||
# Convert to entity
|
||||
entity = self._format_chat_content(
|
||||
base_path=folder_path,
|
||||
name=chat["name"],
|
||||
messages=chat["chat_messages"],
|
||||
created_at=chat["created_at"],
|
||||
modified_at=chat["updated_at"],
|
||||
)
|
||||
|
||||
# Write file
|
||||
file_path = self.base_path / Path(f"{entity.frontmatter.metadata['permalink']}.md")
|
||||
await self.write_entity(entity, file_path)
|
||||
|
||||
chats_imported += 1
|
||||
messages_imported += len(chat["chat_messages"])
|
||||
|
||||
return ChatImportResult(
|
||||
import_count={"conversations": chats_imported, "messages": messages_imported},
|
||||
success=True,
|
||||
conversations=chats_imported,
|
||||
messages=messages_imported,
|
||||
)
|
||||
|
||||
except Exception as e: # pragma: no cover
|
||||
logger.exception("Failed to import Claude conversations")
|
||||
return self.handle_error("Failed to import Claude conversations", e) # pyright: ignore [reportReturnType]
|
||||
|
||||
def _format_chat_content(
|
||||
self,
|
||||
base_path: Path,
|
||||
name: str,
|
||||
messages: List[Dict[str, Any]],
|
||||
created_at: str,
|
||||
modified_at: str,
|
||||
) -> EntityMarkdown:
|
||||
"""Convert chat messages to Basic Memory entity format.
|
||||
|
||||
Args:
|
||||
base_path: Base path for the entity.
|
||||
name: Chat name.
|
||||
messages: List of chat messages.
|
||||
created_at: Creation timestamp.
|
||||
modified_at: Modification timestamp.
|
||||
|
||||
Returns:
|
||||
EntityMarkdown instance representing the conversation.
|
||||
"""
|
||||
# Generate permalink
|
||||
date_prefix = datetime.fromisoformat(created_at.replace("Z", "+00:00")).strftime("%Y%m%d")
|
||||
clean_title = clean_filename(name)
|
||||
permalink = f"{base_path.name}/{date_prefix}-{clean_title}"
|
||||
|
||||
# Format content
|
||||
content = self._format_chat_markdown(
|
||||
name=name,
|
||||
messages=messages,
|
||||
created_at=created_at,
|
||||
modified_at=modified_at,
|
||||
permalink=permalink,
|
||||
)
|
||||
|
||||
# Create entity
|
||||
entity = EntityMarkdown(
|
||||
frontmatter=EntityFrontmatter(
|
||||
metadata={
|
||||
"type": "conversation",
|
||||
"title": name,
|
||||
"created": created_at,
|
||||
"modified": modified_at,
|
||||
"permalink": permalink,
|
||||
}
|
||||
),
|
||||
content=content,
|
||||
)
|
||||
|
||||
return entity
|
||||
|
||||
def _format_chat_markdown(
|
||||
self,
|
||||
name: str,
|
||||
messages: List[Dict[str, Any]],
|
||||
created_at: str,
|
||||
modified_at: str,
|
||||
permalink: str,
|
||||
) -> str:
|
||||
"""Format chat as clean markdown.
|
||||
|
||||
Args:
|
||||
name: Chat name.
|
||||
messages: List of chat messages.
|
||||
created_at: Creation timestamp.
|
||||
modified_at: Modification timestamp.
|
||||
permalink: Permalink for the entity.
|
||||
|
||||
Returns:
|
||||
Formatted markdown content.
|
||||
"""
|
||||
# Start with frontmatter and title
|
||||
lines = [
|
||||
f"# {name}\n",
|
||||
]
|
||||
|
||||
# Add messages
|
||||
for msg in messages:
|
||||
# Format timestamp
|
||||
ts = format_timestamp(msg["created_at"])
|
||||
|
||||
# Add message header
|
||||
lines.append(f"### {msg['sender'].title()} ({ts})")
|
||||
|
||||
# Handle message content
|
||||
content = msg.get("text", "")
|
||||
if msg.get("content"):
|
||||
content = " ".join(c.get("text", "") for c in msg["content"])
|
||||
lines.append(content)
|
||||
|
||||
# Handle attachments
|
||||
attachments = msg.get("attachments", [])
|
||||
for attachment in attachments:
|
||||
if "file_name" in attachment:
|
||||
lines.append(f"\n**Attachment: {attachment['file_name']}**")
|
||||
if "extracted_content" in attachment:
|
||||
lines.append("```")
|
||||
lines.append(attachment["extracted_content"])
|
||||
lines.append("```")
|
||||
|
||||
# Add spacing between messages
|
||||
lines.append("")
|
||||
|
||||
return "\n".join(lines)
|
||||
@@ -0,0 +1,148 @@
|
||||
"""Claude projects import service for Basic Memory."""
|
||||
|
||||
import logging
|
||||
from typing import Any, Dict, Optional
|
||||
|
||||
from basic_memory.markdown.schemas import EntityFrontmatter, EntityMarkdown
|
||||
from basic_memory.importers.base import Importer
|
||||
from basic_memory.schemas.importer import ProjectImportResult
|
||||
from basic_memory.importers.utils import clean_filename
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class ClaudeProjectsImporter(Importer[ProjectImportResult]):
|
||||
"""Service for importing Claude projects."""
|
||||
|
||||
async def import_data(
|
||||
self, source_data, destination_folder: str, **kwargs: Any
|
||||
) -> ProjectImportResult:
|
||||
"""Import projects from Claude JSON export.
|
||||
|
||||
Args:
|
||||
source_path: Path to the Claude projects.json file.
|
||||
destination_folder: Base folder for projects within the project.
|
||||
**kwargs: Additional keyword arguments.
|
||||
|
||||
Returns:
|
||||
ProjectImportResult containing statistics and status of the import.
|
||||
"""
|
||||
try:
|
||||
# Ensure the base folder exists
|
||||
base_path = self.base_path
|
||||
if destination_folder:
|
||||
base_path = self.ensure_folder_exists(destination_folder)
|
||||
|
||||
projects = source_data
|
||||
|
||||
# Process each project
|
||||
docs_imported = 0
|
||||
prompts_imported = 0
|
||||
|
||||
for project in projects:
|
||||
project_dir = clean_filename(project["name"])
|
||||
|
||||
# Create project directories
|
||||
docs_dir = base_path / project_dir / "docs"
|
||||
docs_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
# Import prompt template if it exists
|
||||
if prompt_entity := self._format_prompt_markdown(project):
|
||||
file_path = base_path / f"{prompt_entity.frontmatter.metadata['permalink']}.md"
|
||||
await self.write_entity(prompt_entity, file_path)
|
||||
prompts_imported += 1
|
||||
|
||||
# Import project documents
|
||||
for doc in project.get("docs", []):
|
||||
entity = self._format_project_markdown(project, doc)
|
||||
file_path = base_path / f"{entity.frontmatter.metadata['permalink']}.md"
|
||||
await self.write_entity(entity, file_path)
|
||||
docs_imported += 1
|
||||
|
||||
return ProjectImportResult(
|
||||
import_count={"documents": docs_imported, "prompts": prompts_imported},
|
||||
success=True,
|
||||
documents=docs_imported,
|
||||
prompts=prompts_imported,
|
||||
)
|
||||
|
||||
except Exception as e: # pragma: no cover
|
||||
logger.exception("Failed to import Claude projects")
|
||||
return self.handle_error("Failed to import Claude projects", e) # pyright: ignore [reportReturnType]
|
||||
|
||||
def _format_project_markdown(
|
||||
self, project: Dict[str, Any], doc: Dict[str, Any]
|
||||
) -> EntityMarkdown:
|
||||
"""Format a project document as a Basic Memory entity.
|
||||
|
||||
Args:
|
||||
project: Project data.
|
||||
doc: Document data.
|
||||
|
||||
Returns:
|
||||
EntityMarkdown instance representing the document.
|
||||
"""
|
||||
# Extract timestamps
|
||||
created_at = doc.get("created_at") or project["created_at"]
|
||||
modified_at = project["updated_at"]
|
||||
|
||||
# Generate clean names for organization
|
||||
project_dir = clean_filename(project["name"])
|
||||
doc_file = clean_filename(doc["filename"])
|
||||
|
||||
# Create entity
|
||||
entity = EntityMarkdown(
|
||||
frontmatter=EntityFrontmatter(
|
||||
metadata={
|
||||
"type": "project_doc",
|
||||
"title": doc["filename"],
|
||||
"created": created_at,
|
||||
"modified": modified_at,
|
||||
"permalink": f"{project_dir}/docs/{doc_file}",
|
||||
"project_name": project["name"],
|
||||
"project_uuid": project["uuid"],
|
||||
"doc_uuid": doc["uuid"],
|
||||
}
|
||||
),
|
||||
content=doc["content"],
|
||||
)
|
||||
|
||||
return entity
|
||||
|
||||
def _format_prompt_markdown(self, project: Dict[str, Any]) -> Optional[EntityMarkdown]:
|
||||
"""Format project prompt template as a Basic Memory entity.
|
||||
|
||||
Args:
|
||||
project: Project data.
|
||||
|
||||
Returns:
|
||||
EntityMarkdown instance representing the prompt template, or None if
|
||||
no prompt template exists.
|
||||
"""
|
||||
if not project.get("prompt_template"):
|
||||
return None
|
||||
|
||||
# Extract timestamps
|
||||
created_at = project["created_at"]
|
||||
modified_at = project["updated_at"]
|
||||
|
||||
# Generate clean project directory name
|
||||
project_dir = clean_filename(project["name"])
|
||||
|
||||
# Create entity
|
||||
entity = EntityMarkdown(
|
||||
frontmatter=EntityFrontmatter(
|
||||
metadata={
|
||||
"type": "prompt_template",
|
||||
"title": f"Prompt Template: {project['name']}",
|
||||
"created": created_at,
|
||||
"modified": modified_at,
|
||||
"permalink": f"{project_dir}/prompt-template",
|
||||
"project_name": project["name"],
|
||||
"project_uuid": project["uuid"],
|
||||
}
|
||||
),
|
||||
content=f"# Prompt Template: {project['name']}\n\n{project['prompt_template']}",
|
||||
)
|
||||
|
||||
return entity
|
||||
@@ -0,0 +1,93 @@
|
||||
"""Memory JSON import service for Basic Memory."""
|
||||
|
||||
import logging
|
||||
from typing import Any, Dict, List
|
||||
|
||||
from basic_memory.config import config
|
||||
from basic_memory.markdown.schemas import EntityFrontmatter, EntityMarkdown, Observation, Relation
|
||||
from basic_memory.importers.base import Importer
|
||||
from basic_memory.schemas.importer import EntityImportResult
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class MemoryJsonImporter(Importer[EntityImportResult]):
|
||||
"""Service for importing memory.json format data."""
|
||||
|
||||
async def import_data(
|
||||
self, source_data, destination_folder: str = "", **kwargs: Any
|
||||
) -> EntityImportResult:
|
||||
"""Import entities and relations from a memory.json file.
|
||||
|
||||
Args:
|
||||
source_data: Path to the memory.json file.
|
||||
destination_folder: Optional destination folder within the project.
|
||||
**kwargs: Additional keyword arguments.
|
||||
|
||||
Returns:
|
||||
EntityImportResult containing statistics and status of the import.
|
||||
"""
|
||||
try:
|
||||
# First pass - collect all relations by source entity
|
||||
entity_relations: Dict[str, List[Relation]] = {}
|
||||
entities: Dict[str, Dict[str, Any]] = {}
|
||||
|
||||
# Ensure the base path exists
|
||||
base_path = config.home # pragma: no cover
|
||||
if destination_folder: # pragma: no cover
|
||||
base_path = self.ensure_folder_exists(destination_folder)
|
||||
|
||||
# First pass - collect entities and relations
|
||||
for line in source_data:
|
||||
data = line
|
||||
if data["type"] == "entity":
|
||||
entities[data["name"]] = data
|
||||
elif data["type"] == "relation":
|
||||
# Store relation with its source entity
|
||||
source = data.get("from") or data.get("from_id")
|
||||
if source not in entity_relations:
|
||||
entity_relations[source] = []
|
||||
entity_relations[source].append(
|
||||
Relation(
|
||||
type=data.get("relationType") or data.get("relation_type"),
|
||||
target=data.get("to") or data.get("to_id"),
|
||||
)
|
||||
)
|
||||
|
||||
# Second pass - create and write entities
|
||||
entities_created = 0
|
||||
for name, entity_data in entities.items():
|
||||
# Ensure entity type directory exists
|
||||
entity_type_dir = base_path / entity_data["entityType"]
|
||||
entity_type_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
entity = EntityMarkdown(
|
||||
frontmatter=EntityFrontmatter(
|
||||
metadata={
|
||||
"type": entity_data["entityType"],
|
||||
"title": name,
|
||||
"permalink": f"{entity_data['entityType']}/{name}",
|
||||
}
|
||||
),
|
||||
content=f"# {name}\n",
|
||||
observations=[Observation(content=obs) for obs in entity_data["observations"]],
|
||||
relations=entity_relations.get(name, []),
|
||||
)
|
||||
|
||||
# Write entity file
|
||||
file_path = base_path / f"{entity_data['entityType']}/{name}.md"
|
||||
await self.write_entity(entity, file_path)
|
||||
entities_created += 1
|
||||
|
||||
relations_count = sum(len(rels) for rels in entity_relations.values())
|
||||
|
||||
return EntityImportResult(
|
||||
import_count={"entities": entities_created, "relations": relations_count},
|
||||
success=True,
|
||||
entities=entities_created,
|
||||
relations=relations_count,
|
||||
)
|
||||
|
||||
except Exception as e: # pragma: no cover
|
||||
logger.exception("Failed to import memory.json")
|
||||
return self.handle_error("Failed to import memory.json", e) # pyright: ignore [reportReturnType]
|
||||
@@ -0,0 +1,58 @@
|
||||
"""Utility functions for import services."""
|
||||
|
||||
import re
|
||||
from datetime import datetime
|
||||
from typing import Any
|
||||
|
||||
|
||||
def clean_filename(name: str) -> str: # pragma: no cover
|
||||
"""Clean a string to be used as a filename.
|
||||
|
||||
Args:
|
||||
name: The string to clean.
|
||||
|
||||
Returns:
|
||||
A cleaned string suitable for use as a filename.
|
||||
"""
|
||||
# Replace common punctuation and whitespace with underscores
|
||||
name = re.sub(r"[\s\-,.:/\\\[\]\(\)]+", "_", name)
|
||||
# Remove any non-alphanumeric or underscore characters
|
||||
name = re.sub(r"[^\w]+", "", name)
|
||||
# Ensure the name isn't too long
|
||||
if len(name) > 100: # pragma: no cover
|
||||
name = name[:100]
|
||||
# Ensure the name isn't empty
|
||||
if not name: # pragma: no cover
|
||||
name = "untitled"
|
||||
return name
|
||||
|
||||
|
||||
def format_timestamp(timestamp: Any) -> str: # pragma: no cover
|
||||
"""Format a timestamp for use in a filename or title.
|
||||
|
||||
Args:
|
||||
timestamp: A timestamp in various formats.
|
||||
|
||||
Returns:
|
||||
A formatted string representation of the timestamp.
|
||||
"""
|
||||
if isinstance(timestamp, str):
|
||||
try:
|
||||
# Try ISO format
|
||||
timestamp = datetime.fromisoformat(timestamp.replace("Z", "+00:00"))
|
||||
except ValueError:
|
||||
try:
|
||||
# Try unix timestamp as string
|
||||
timestamp = datetime.fromtimestamp(float(timestamp))
|
||||
except ValueError:
|
||||
# Return as is if we can't parse it
|
||||
return timestamp
|
||||
elif isinstance(timestamp, (int, float)):
|
||||
# Unix timestamp
|
||||
timestamp = datetime.fromtimestamp(timestamp)
|
||||
|
||||
if isinstance(timestamp, datetime):
|
||||
return timestamp.strftime("%Y-%m-%d %H:%M:%S")
|
||||
|
||||
# Return as is if we can't format it
|
||||
return str(timestamp) # pragma: no cover
|
||||
@@ -92,7 +92,6 @@ class EntityParser:
|
||||
async def parse_file(self, path: Path | str) -> EntityMarkdown:
|
||||
"""Parse markdown file into EntityMarkdown."""
|
||||
|
||||
# TODO move to api endpoint to check if absolute path was requested
|
||||
# Check if the path is already absolute
|
||||
if (
|
||||
isinstance(path, Path)
|
||||
@@ -101,12 +100,16 @@ class EntityParser:
|
||||
):
|
||||
absolute_path = Path(path)
|
||||
else:
|
||||
absolute_path = self.base_path / path
|
||||
absolute_path = self.get_file_path(path)
|
||||
|
||||
# Parse frontmatter and content using python-frontmatter
|
||||
file_content = absolute_path.read_text(encoding="utf-8")
|
||||
return await self.parse_file_content(absolute_path, file_content)
|
||||
|
||||
def get_file_path(self, path):
|
||||
"""Get absolute path for a file using the base path for the project."""
|
||||
return self.base_path / path
|
||||
|
||||
async def parse_file_content(self, absolute_path, file_content):
|
||||
post = frontmatter.loads(file_content)
|
||||
# Extract file stat info
|
||||
|
||||
@@ -0,0 +1,270 @@
|
||||
"""OAuth authentication provider for Basic Memory MCP server."""
|
||||
|
||||
import secrets
|
||||
from datetime import datetime, timedelta
|
||||
from typing import Dict, Optional
|
||||
|
||||
import jwt
|
||||
from mcp.server.auth.provider import (
|
||||
OAuthAuthorizationServerProvider,
|
||||
AuthorizationParams,
|
||||
AuthorizationCode,
|
||||
RefreshToken,
|
||||
AccessToken,
|
||||
)
|
||||
from mcp.shared.auth import OAuthClientInformationFull, OAuthToken
|
||||
from loguru import logger
|
||||
|
||||
|
||||
class BasicMemoryAuthorizationCode(AuthorizationCode):
|
||||
"""Extended authorization code with additional metadata."""
|
||||
|
||||
issuer_state: Optional[str] = None
|
||||
|
||||
|
||||
class BasicMemoryRefreshToken(RefreshToken):
|
||||
"""Extended refresh token with additional metadata."""
|
||||
|
||||
pass
|
||||
|
||||
|
||||
class BasicMemoryAccessToken(AccessToken):
|
||||
"""Extended access token with additional metadata."""
|
||||
|
||||
pass
|
||||
|
||||
|
||||
class BasicMemoryOAuthProvider(
|
||||
OAuthAuthorizationServerProvider[
|
||||
BasicMemoryAuthorizationCode, BasicMemoryRefreshToken, BasicMemoryAccessToken
|
||||
]
|
||||
):
|
||||
"""OAuth provider for Basic Memory MCP server.
|
||||
|
||||
This is a simple in-memory implementation that can be extended
|
||||
to integrate with external OAuth providers or use persistent storage.
|
||||
"""
|
||||
|
||||
def __init__(self, issuer_url: str = "http://localhost:8000", secret_key: Optional[str] = None):
|
||||
self.issuer_url = issuer_url
|
||||
# Use environment variable for secret key if available, otherwise generate
|
||||
import os
|
||||
|
||||
self.secret_key = (
|
||||
secret_key or os.getenv("FASTMCP_AUTH_SECRET_KEY") or secrets.token_urlsafe(32)
|
||||
)
|
||||
|
||||
# In-memory storage - in production, use a proper database
|
||||
self.clients: Dict[str, OAuthClientInformationFull] = {}
|
||||
self.authorization_codes: Dict[str, BasicMemoryAuthorizationCode] = {}
|
||||
self.refresh_tokens: Dict[str, BasicMemoryRefreshToken] = {}
|
||||
self.access_tokens: Dict[str, BasicMemoryAccessToken] = {}
|
||||
|
||||
async def get_client(self, client_id: str) -> Optional[OAuthClientInformationFull]:
|
||||
"""Get a client by ID."""
|
||||
return self.clients.get(client_id)
|
||||
|
||||
async def register_client(self, client_info: OAuthClientInformationFull) -> None:
|
||||
"""Register a new OAuth client."""
|
||||
# Generate client ID if not provided
|
||||
if not client_info.client_id:
|
||||
client_info.client_id = secrets.token_urlsafe(16)
|
||||
|
||||
# Generate client secret if not provided
|
||||
if not client_info.client_secret:
|
||||
client_info.client_secret = secrets.token_urlsafe(32)
|
||||
|
||||
self.clients[client_info.client_id] = client_info
|
||||
logger.info(f"Registered OAuth client: {client_info.client_id}")
|
||||
|
||||
async def authorize(
|
||||
self, client: OAuthClientInformationFull, params: AuthorizationParams
|
||||
) -> str:
|
||||
"""Create an authorization URL for the OAuth flow.
|
||||
|
||||
For basic-memory, we'll implement a simple authorization flow.
|
||||
In production, this might redirect to an external provider.
|
||||
"""
|
||||
# Generate authorization code
|
||||
auth_code = secrets.token_urlsafe(32)
|
||||
|
||||
# Store authorization code with metadata
|
||||
self.authorization_codes[auth_code] = BasicMemoryAuthorizationCode(
|
||||
code=auth_code,
|
||||
scopes=params.scopes or [],
|
||||
expires_at=(datetime.utcnow() + timedelta(minutes=10)).timestamp(),
|
||||
client_id=client.client_id,
|
||||
code_challenge=params.code_challenge,
|
||||
redirect_uri=params.redirect_uri,
|
||||
redirect_uri_provided_explicitly=params.redirect_uri_provided_explicitly,
|
||||
issuer_state=params.state,
|
||||
)
|
||||
|
||||
# In a real implementation, we'd redirect to an authorization page
|
||||
# For now, we'll just return the redirect URL with the code
|
||||
redirect_uri = str(params.redirect_uri)
|
||||
separator = "&" if "?" in redirect_uri else "?"
|
||||
|
||||
auth_url = f"{redirect_uri}{separator}code={auth_code}"
|
||||
if params.state:
|
||||
auth_url += f"&state={params.state}"
|
||||
|
||||
return auth_url
|
||||
|
||||
async def load_authorization_code(
|
||||
self, client: OAuthClientInformationFull, authorization_code: str
|
||||
) -> Optional[BasicMemoryAuthorizationCode]:
|
||||
"""Load an authorization code."""
|
||||
code = self.authorization_codes.get(authorization_code)
|
||||
|
||||
if code and code.client_id == client.client_id:
|
||||
# Check if expired
|
||||
if datetime.utcnow().timestamp() > code.expires_at:
|
||||
del self.authorization_codes[authorization_code]
|
||||
return None
|
||||
return code
|
||||
|
||||
return None
|
||||
|
||||
async def exchange_authorization_code(
|
||||
self, client: OAuthClientInformationFull, authorization_code: BasicMemoryAuthorizationCode
|
||||
) -> OAuthToken:
|
||||
"""Exchange an authorization code for tokens."""
|
||||
# Generate tokens
|
||||
access_token = self._generate_access_token(client.client_id, authorization_code.scopes)
|
||||
refresh_token = secrets.token_urlsafe(32)
|
||||
|
||||
# Store tokens
|
||||
expires_at = (datetime.utcnow() + timedelta(hours=1)).timestamp()
|
||||
|
||||
self.access_tokens[access_token] = BasicMemoryAccessToken(
|
||||
token=access_token,
|
||||
client_id=client.client_id,
|
||||
scopes=authorization_code.scopes,
|
||||
expires_at=int(expires_at),
|
||||
)
|
||||
|
||||
self.refresh_tokens[refresh_token] = BasicMemoryRefreshToken(
|
||||
token=refresh_token,
|
||||
client_id=client.client_id,
|
||||
scopes=authorization_code.scopes,
|
||||
)
|
||||
|
||||
# Remove used authorization code
|
||||
del self.authorization_codes[authorization_code.code]
|
||||
|
||||
return OAuthToken(
|
||||
access_token=access_token,
|
||||
token_type="bearer",
|
||||
expires_in=3600, # 1 hour
|
||||
refresh_token=refresh_token,
|
||||
scope=" ".join(authorization_code.scopes) if authorization_code.scopes else None,
|
||||
)
|
||||
|
||||
async def load_refresh_token(
|
||||
self, client: OAuthClientInformationFull, refresh_token: str
|
||||
) -> Optional[BasicMemoryRefreshToken]:
|
||||
"""Load a refresh token."""
|
||||
token = self.refresh_tokens.get(refresh_token)
|
||||
|
||||
if token and token.client_id == client.client_id:
|
||||
return token
|
||||
|
||||
return None
|
||||
|
||||
async def exchange_refresh_token(
|
||||
self,
|
||||
client: OAuthClientInformationFull,
|
||||
refresh_token: BasicMemoryRefreshToken,
|
||||
scopes: list[str],
|
||||
) -> OAuthToken:
|
||||
"""Exchange a refresh token for new tokens."""
|
||||
# Use requested scopes or original scopes
|
||||
token_scopes = scopes if scopes else refresh_token.scopes
|
||||
|
||||
# Generate new tokens
|
||||
new_access_token = self._generate_access_token(client.client_id, token_scopes)
|
||||
new_refresh_token = secrets.token_urlsafe(32)
|
||||
|
||||
# Store new tokens
|
||||
expires_at = (datetime.utcnow() + timedelta(hours=1)).timestamp()
|
||||
|
||||
self.access_tokens[new_access_token] = BasicMemoryAccessToken(
|
||||
token=new_access_token,
|
||||
client_id=client.client_id,
|
||||
scopes=token_scopes,
|
||||
expires_at=int(expires_at),
|
||||
)
|
||||
|
||||
self.refresh_tokens[new_refresh_token] = BasicMemoryRefreshToken(
|
||||
token=new_refresh_token,
|
||||
client_id=client.client_id,
|
||||
scopes=token_scopes,
|
||||
)
|
||||
|
||||
# Remove old tokens
|
||||
del self.refresh_tokens[refresh_token.token]
|
||||
|
||||
return OAuthToken(
|
||||
access_token=new_access_token,
|
||||
token_type="bearer",
|
||||
expires_in=3600, # 1 hour
|
||||
refresh_token=new_refresh_token,
|
||||
scope=" ".join(token_scopes) if token_scopes else None,
|
||||
)
|
||||
|
||||
async def load_access_token(self, token: str) -> Optional[BasicMemoryAccessToken]:
|
||||
"""Load and validate an access token."""
|
||||
logger.debug("Loading access token, checking in-memory store first")
|
||||
access_token = self.access_tokens.get(token)
|
||||
|
||||
if access_token:
|
||||
# Check if expired
|
||||
if access_token.expires_at and datetime.utcnow().timestamp() > access_token.expires_at:
|
||||
logger.debug("Token found in memory but expired, removing")
|
||||
del self.access_tokens[token]
|
||||
return None
|
||||
logger.debug("Token found in memory and valid")
|
||||
return access_token
|
||||
|
||||
# Try to decode as JWT
|
||||
logger.debug("Token not in memory, attempting JWT decode with secret key")
|
||||
try:
|
||||
# Decode with audience verification - PyJWT expects the audience to match
|
||||
payload = jwt.decode(
|
||||
token,
|
||||
self.secret_key,
|
||||
algorithms=["HS256"],
|
||||
audience="basic-memory", # Expecting this audience
|
||||
issuer=self.issuer_url, # And this issuer
|
||||
)
|
||||
logger.debug(f"JWT decoded successfully: {payload}")
|
||||
return BasicMemoryAccessToken(
|
||||
token=token,
|
||||
client_id=payload.get("sub", ""),
|
||||
scopes=payload.get("scopes", []),
|
||||
expires_at=payload.get("exp"),
|
||||
)
|
||||
except jwt.InvalidTokenError as e:
|
||||
logger.error(f"JWT decode failed: {e}")
|
||||
return None
|
||||
|
||||
async def revoke_token(self, token: BasicMemoryAccessToken | BasicMemoryRefreshToken) -> None:
|
||||
"""Revoke an access or refresh token."""
|
||||
if isinstance(token, BasicMemoryAccessToken):
|
||||
self.access_tokens.pop(token.token, None)
|
||||
else:
|
||||
self.refresh_tokens.pop(token.token, None)
|
||||
|
||||
def _generate_access_token(self, client_id: str, scopes: list[str]) -> str:
|
||||
"""Generate a JWT access token."""
|
||||
payload = {
|
||||
"iss": self.issuer_url,
|
||||
"sub": client_id,
|
||||
"aud": "basic-memory",
|
||||
"exp": datetime.utcnow() + timedelta(hours=1),
|
||||
"iat": datetime.utcnow(),
|
||||
"scopes": scopes,
|
||||
}
|
||||
|
||||
return jwt.encode(payload, self.secret_key, algorithm="HS256")
|
||||
@@ -0,0 +1,321 @@
|
||||
"""External OAuth provider integration for Basic Memory MCP server."""
|
||||
|
||||
import os
|
||||
from typing import Optional, Dict, Any
|
||||
from dataclasses import dataclass
|
||||
|
||||
import httpx
|
||||
from loguru import logger
|
||||
from mcp.server.auth.provider import (
|
||||
OAuthAuthorizationServerProvider,
|
||||
AuthorizationParams,
|
||||
AuthorizationCode,
|
||||
RefreshToken,
|
||||
AccessToken,
|
||||
construct_redirect_uri,
|
||||
)
|
||||
from mcp.shared.auth import OAuthClientInformationFull, OAuthToken
|
||||
|
||||
|
||||
@dataclass
|
||||
class ExternalAuthorizationCode(AuthorizationCode):
|
||||
"""Authorization code with external provider metadata."""
|
||||
|
||||
external_code: Optional[str] = None
|
||||
state: Optional[str] = None
|
||||
|
||||
|
||||
@dataclass
|
||||
class ExternalRefreshToken(RefreshToken):
|
||||
"""Refresh token with external provider metadata."""
|
||||
|
||||
external_token: Optional[str] = None
|
||||
|
||||
|
||||
@dataclass
|
||||
class ExternalAccessToken(AccessToken):
|
||||
"""Access token with external provider metadata."""
|
||||
|
||||
external_token: Optional[str] = None
|
||||
|
||||
|
||||
class ExternalOAuthProvider(
|
||||
OAuthAuthorizationServerProvider[
|
||||
ExternalAuthorizationCode, ExternalRefreshToken, ExternalAccessToken
|
||||
]
|
||||
):
|
||||
"""OAuth provider that delegates to external OAuth providers.
|
||||
|
||||
This provider can integrate with services like:
|
||||
- GitHub OAuth
|
||||
- Google OAuth
|
||||
- Auth0
|
||||
- Okta
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
issuer_url: str,
|
||||
external_provider: str,
|
||||
external_client_id: str,
|
||||
external_client_secret: str,
|
||||
external_authorize_url: str,
|
||||
external_token_url: str,
|
||||
external_userinfo_url: Optional[str] = None,
|
||||
):
|
||||
self.issuer_url = issuer_url
|
||||
self.external_provider = external_provider
|
||||
self.external_client_id = external_client_id
|
||||
self.external_client_secret = external_client_secret
|
||||
self.external_authorize_url = external_authorize_url
|
||||
self.external_token_url = external_token_url
|
||||
self.external_userinfo_url = external_userinfo_url
|
||||
|
||||
# In-memory storage - in production, use a database
|
||||
self.clients: Dict[str, OAuthClientInformationFull] = {}
|
||||
self.codes: Dict[str, ExternalAuthorizationCode] = {}
|
||||
self.tokens: Dict[str, Any] = {}
|
||||
|
||||
self.http_client = httpx.AsyncClient()
|
||||
|
||||
async def get_client(self, client_id: str) -> Optional[OAuthClientInformationFull]:
|
||||
"""Get a client by ID."""
|
||||
return self.clients.get(client_id)
|
||||
|
||||
async def register_client(self, client_info: OAuthClientInformationFull) -> None:
|
||||
"""Register a new OAuth client."""
|
||||
self.clients[client_info.client_id] = client_info
|
||||
logger.info(f"Registered external OAuth client: {client_info.client_id}")
|
||||
|
||||
async def authorize(
|
||||
self, client: OAuthClientInformationFull, params: AuthorizationParams
|
||||
) -> str:
|
||||
"""Create authorization URL redirecting to external provider."""
|
||||
# Store authorization request
|
||||
import secrets
|
||||
|
||||
state = secrets.token_urlsafe(32)
|
||||
|
||||
self.codes[state] = ExternalAuthorizationCode(
|
||||
code=state,
|
||||
scopes=params.scopes or [],
|
||||
expires_at=0, # Will be set by external provider
|
||||
client_id=client.client_id,
|
||||
code_challenge=params.code_challenge,
|
||||
redirect_uri=params.redirect_uri,
|
||||
redirect_uri_provided_explicitly=params.redirect_uri_provided_explicitly,
|
||||
state=params.state,
|
||||
)
|
||||
|
||||
# Build external provider URL
|
||||
external_params = {
|
||||
"client_id": self.external_client_id,
|
||||
"redirect_uri": f"{self.issuer_url}/callback",
|
||||
"response_type": "code",
|
||||
"state": state,
|
||||
"scope": " ".join(params.scopes or []),
|
||||
}
|
||||
|
||||
return construct_redirect_uri(self.external_authorize_url, **external_params)
|
||||
|
||||
async def handle_callback(self, code: str, state: str) -> str:
|
||||
"""Handle callback from external provider."""
|
||||
# Get original authorization request
|
||||
auth_code = self.codes.get(state)
|
||||
if not auth_code:
|
||||
raise ValueError("Invalid state parameter")
|
||||
|
||||
# Exchange code with external provider
|
||||
token_data = {
|
||||
"grant_type": "authorization_code",
|
||||
"code": code,
|
||||
"redirect_uri": f"{self.issuer_url}/callback",
|
||||
"client_id": self.external_client_id,
|
||||
"client_secret": self.external_client_secret,
|
||||
}
|
||||
|
||||
response = await self.http_client.post(
|
||||
self.external_token_url,
|
||||
data=token_data,
|
||||
)
|
||||
response.raise_for_status()
|
||||
external_tokens = response.json()
|
||||
|
||||
# Store external tokens
|
||||
import secrets
|
||||
|
||||
internal_code = secrets.token_urlsafe(32)
|
||||
|
||||
self.codes[internal_code] = ExternalAuthorizationCode(
|
||||
code=internal_code,
|
||||
scopes=auth_code.scopes,
|
||||
expires_at=0,
|
||||
client_id=auth_code.client_id,
|
||||
code_challenge=auth_code.code_challenge,
|
||||
redirect_uri=auth_code.redirect_uri,
|
||||
redirect_uri_provided_explicitly=auth_code.redirect_uri_provided_explicitly,
|
||||
external_code=code,
|
||||
state=auth_code.state,
|
||||
)
|
||||
|
||||
self.tokens[internal_code] = external_tokens
|
||||
|
||||
# Redirect to original client
|
||||
return construct_redirect_uri(
|
||||
str(auth_code.redirect_uri),
|
||||
code=internal_code,
|
||||
state=auth_code.state,
|
||||
)
|
||||
|
||||
async def load_authorization_code(
|
||||
self, client: OAuthClientInformationFull, authorization_code: str
|
||||
) -> Optional[ExternalAuthorizationCode]:
|
||||
"""Load an authorization code."""
|
||||
code = self.codes.get(authorization_code)
|
||||
if code and code.client_id == client.client_id:
|
||||
return code
|
||||
return None
|
||||
|
||||
async def exchange_authorization_code(
|
||||
self, client: OAuthClientInformationFull, authorization_code: ExternalAuthorizationCode
|
||||
) -> OAuthToken:
|
||||
"""Exchange authorization code for tokens."""
|
||||
# Get stored external tokens
|
||||
external_tokens = self.tokens.get(authorization_code.code)
|
||||
if not external_tokens:
|
||||
raise ValueError("No tokens found for authorization code")
|
||||
|
||||
# Map external tokens to MCP tokens
|
||||
access_token = external_tokens.get("access_token")
|
||||
refresh_token = external_tokens.get("refresh_token")
|
||||
expires_in = external_tokens.get("expires_in", 3600)
|
||||
|
||||
# Store the mapping
|
||||
self.tokens[access_token] = {
|
||||
"client_id": client.client_id,
|
||||
"external_token": access_token,
|
||||
"scopes": authorization_code.scopes,
|
||||
}
|
||||
|
||||
if refresh_token:
|
||||
self.tokens[refresh_token] = {
|
||||
"client_id": client.client_id,
|
||||
"external_token": refresh_token,
|
||||
"scopes": authorization_code.scopes,
|
||||
}
|
||||
|
||||
# Clean up authorization code
|
||||
del self.codes[authorization_code.code]
|
||||
|
||||
return OAuthToken(
|
||||
access_token=access_token,
|
||||
token_type="bearer",
|
||||
expires_in=expires_in,
|
||||
refresh_token=refresh_token,
|
||||
scope=" ".join(authorization_code.scopes) if authorization_code.scopes else None,
|
||||
)
|
||||
|
||||
async def load_refresh_token(
|
||||
self, client: OAuthClientInformationFull, refresh_token: str
|
||||
) -> Optional[ExternalRefreshToken]:
|
||||
"""Load a refresh token."""
|
||||
token_info = self.tokens.get(refresh_token)
|
||||
if token_info and token_info["client_id"] == client.client_id:
|
||||
return ExternalRefreshToken(
|
||||
token=refresh_token,
|
||||
client_id=client.client_id,
|
||||
scopes=token_info["scopes"],
|
||||
external_token=token_info.get("external_token"),
|
||||
)
|
||||
return None
|
||||
|
||||
async def exchange_refresh_token(
|
||||
self,
|
||||
client: OAuthClientInformationFull,
|
||||
refresh_token: ExternalRefreshToken,
|
||||
scopes: list[str],
|
||||
) -> OAuthToken:
|
||||
"""Exchange refresh token for new tokens."""
|
||||
# Exchange with external provider
|
||||
token_data = {
|
||||
"grant_type": "refresh_token",
|
||||
"refresh_token": refresh_token.external_token or refresh_token.token,
|
||||
"client_id": self.external_client_id,
|
||||
"client_secret": self.external_client_secret,
|
||||
}
|
||||
|
||||
response = await self.http_client.post(
|
||||
self.external_token_url,
|
||||
data=token_data,
|
||||
)
|
||||
response.raise_for_status()
|
||||
external_tokens = response.json()
|
||||
|
||||
# Update stored tokens
|
||||
new_access_token = external_tokens.get("access_token")
|
||||
new_refresh_token = external_tokens.get("refresh_token", refresh_token.token)
|
||||
expires_in = external_tokens.get("expires_in", 3600)
|
||||
|
||||
self.tokens[new_access_token] = {
|
||||
"client_id": client.client_id,
|
||||
"external_token": new_access_token,
|
||||
"scopes": scopes or refresh_token.scopes,
|
||||
}
|
||||
|
||||
if new_refresh_token != refresh_token.token:
|
||||
self.tokens[new_refresh_token] = {
|
||||
"client_id": client.client_id,
|
||||
"external_token": new_refresh_token,
|
||||
"scopes": scopes or refresh_token.scopes,
|
||||
}
|
||||
del self.tokens[refresh_token.token]
|
||||
|
||||
return OAuthToken(
|
||||
access_token=new_access_token,
|
||||
token_type="bearer",
|
||||
expires_in=expires_in,
|
||||
refresh_token=new_refresh_token,
|
||||
scope=" ".join(scopes or refresh_token.scopes),
|
||||
)
|
||||
|
||||
async def load_access_token(self, token: str) -> Optional[ExternalAccessToken]:
|
||||
"""Load and validate an access token."""
|
||||
token_info = self.tokens.get(token)
|
||||
if token_info:
|
||||
return ExternalAccessToken(
|
||||
token=token,
|
||||
client_id=token_info["client_id"],
|
||||
scopes=token_info["scopes"],
|
||||
external_token=token_info.get("external_token"),
|
||||
)
|
||||
return None
|
||||
|
||||
async def revoke_token(self, token: ExternalAccessToken | ExternalRefreshToken) -> None:
|
||||
"""Revoke a token."""
|
||||
self.tokens.pop(token.token, None)
|
||||
|
||||
|
||||
def create_github_provider() -> ExternalOAuthProvider:
|
||||
"""Create an OAuth provider for GitHub integration."""
|
||||
return ExternalOAuthProvider(
|
||||
issuer_url=os.getenv("FASTMCP_AUTH_ISSUER_URL", "http://localhost:8000"),
|
||||
external_provider="github",
|
||||
external_client_id=os.getenv("GITHUB_CLIENT_ID", ""),
|
||||
external_client_secret=os.getenv("GITHUB_CLIENT_SECRET", ""),
|
||||
external_authorize_url="https://github.com/login/oauth/authorize",
|
||||
external_token_url="https://github.com/login/oauth/access_token",
|
||||
external_userinfo_url="https://api.github.com/user",
|
||||
)
|
||||
|
||||
|
||||
def create_google_provider() -> ExternalOAuthProvider:
|
||||
"""Create an OAuth provider for Google integration."""
|
||||
return ExternalOAuthProvider(
|
||||
issuer_url=os.getenv("FASTMCP_AUTH_ISSUER_URL", "http://localhost:8000"),
|
||||
external_provider="google",
|
||||
external_client_id=os.getenv("GOOGLE_CLIENT_ID", ""),
|
||||
external_client_secret=os.getenv("GOOGLE_CLIENT_SECRET", ""),
|
||||
external_authorize_url="https://accounts.google.com/o/oauth2/v2/auth",
|
||||
external_token_url="https://oauth2.googleapis.com/token",
|
||||
external_userinfo_url="https://www.googleapis.com/oauth2/v1/userinfo",
|
||||
)
|
||||
@@ -1,24 +0,0 @@
|
||||
"""Main MCP entrypoint for Basic Memory.
|
||||
|
||||
Creates and configures the shared MCP instance and handles server startup.
|
||||
"""
|
||||
|
||||
from loguru import logger # pragma: no cover
|
||||
|
||||
from basic_memory.config import config # pragma: no cover
|
||||
|
||||
# Import shared mcp instance
|
||||
from basic_memory.mcp.server import mcp # pragma: no cover
|
||||
|
||||
# Import tools to register them
|
||||
import basic_memory.mcp.tools # noqa: F401 # pragma: no cover
|
||||
|
||||
# Import prompts to register them
|
||||
import basic_memory.mcp.prompts # noqa: F401 # pragma: no cover
|
||||
|
||||
|
||||
if __name__ == "__main__": # pragma: no cover
|
||||
home_dir = config.home
|
||||
logger.info("Starting Basic Memory MCP server")
|
||||
logger.info(f"Home directory: {home_dir}")
|
||||
mcp.run()
|
||||
@@ -0,0 +1,103 @@
|
||||
"""Project session management for Basic Memory MCP server.
|
||||
|
||||
Provides simple in-memory project context for MCP tools, allowing users to switch
|
||||
between projects during a conversation without restarting the server.
|
||||
"""
|
||||
|
||||
from dataclasses import dataclass
|
||||
from typing import Optional
|
||||
from loguru import logger
|
||||
|
||||
from basic_memory.config import ProjectConfig, get_project_config
|
||||
|
||||
|
||||
@dataclass
|
||||
class ProjectSession:
|
||||
"""Simple in-memory project context for MCP session.
|
||||
|
||||
This class manages the current project context that tools use when no explicit
|
||||
project is specified. It's initialized with the default project from config
|
||||
and can be changed during the conversation.
|
||||
"""
|
||||
|
||||
current_project: Optional[str] = None
|
||||
default_project: Optional[str] = None
|
||||
|
||||
def initialize(self, default_project: str) -> None:
|
||||
"""Set the default project from config on startup.
|
||||
|
||||
Args:
|
||||
default_project: The project name from configuration
|
||||
"""
|
||||
self.default_project = default_project
|
||||
self.current_project = default_project
|
||||
logger.info(f"Initialized project session with default project: {default_project}")
|
||||
|
||||
def get_current_project(self) -> str:
|
||||
"""Get the currently active project name.
|
||||
|
||||
Returns:
|
||||
The current project name, falling back to default, then 'main'
|
||||
"""
|
||||
return self.current_project or self.default_project or "main"
|
||||
|
||||
def set_current_project(self, project_name: str) -> None:
|
||||
"""Set the current project context.
|
||||
|
||||
Args:
|
||||
project_name: The project to switch to
|
||||
"""
|
||||
previous = self.current_project
|
||||
self.current_project = project_name
|
||||
logger.info(f"Switched project context: {previous} -> {project_name}")
|
||||
|
||||
def get_default_project(self) -> str:
|
||||
"""Get the default project name from startup.
|
||||
|
||||
Returns:
|
||||
The default project name, or 'main' if not set
|
||||
"""
|
||||
return self.default_project or "main" # pragma: no cover
|
||||
|
||||
def reset_to_default(self) -> None: # pragma: no cover
|
||||
"""Reset current project back to the default project."""
|
||||
self.current_project = self.default_project # pragma: no cover
|
||||
logger.info(f"Reset project context to default: {self.default_project}") # pragma: no cover
|
||||
|
||||
|
||||
# Global session instance
|
||||
session = ProjectSession()
|
||||
|
||||
|
||||
def get_active_project(project_override: Optional[str] = None) -> ProjectConfig:
|
||||
"""Get the active project name for a tool call.
|
||||
|
||||
This is the main function tools should use to determine which project
|
||||
to operate on.
|
||||
|
||||
Args:
|
||||
project_override: Optional explicit project name from tool parameter
|
||||
|
||||
Returns:
|
||||
The project name to use (override takes precedence over session context)
|
||||
"""
|
||||
if project_override: # pragma: no cover
|
||||
project = get_project_config(project_override)
|
||||
session.set_current_project(project_override)
|
||||
return project
|
||||
|
||||
current_project = session.get_current_project()
|
||||
return get_project_config(current_project)
|
||||
|
||||
|
||||
def add_project_metadata(result: str, project_name: str) -> str:
|
||||
"""Add project context as metadata footer for LLM awareness.
|
||||
|
||||
Args:
|
||||
result: The tool result string
|
||||
project_name: The project name that was used
|
||||
|
||||
Returns:
|
||||
Result with project metadata footer
|
||||
"""
|
||||
return f"{result}\n\n<!-- Project: {project_name} -->" # pragma: no cover
|
||||
@@ -10,10 +10,12 @@ from basic_memory.mcp.prompts import continue_conversation
|
||||
from basic_memory.mcp.prompts import recent_activity
|
||||
from basic_memory.mcp.prompts import search
|
||||
from basic_memory.mcp.prompts import ai_assistant_guide
|
||||
from basic_memory.mcp.prompts import sync_status
|
||||
|
||||
__all__ = [
|
||||
"ai_assistant_guide",
|
||||
"continue_conversation",
|
||||
"recent_activity",
|
||||
"search",
|
||||
"sync_status",
|
||||
]
|
||||
|
||||
@@ -4,20 +4,17 @@ These prompts help users continue conversations and work across sessions,
|
||||
providing context from previous interactions to maintain continuity.
|
||||
"""
|
||||
|
||||
from textwrap import dedent
|
||||
from typing import Annotated, Optional
|
||||
|
||||
from loguru import logger
|
||||
from pydantic import Field
|
||||
|
||||
from basic_memory.mcp.prompts.utils import PromptContext, PromptContextItem, format_prompt_context
|
||||
from basic_memory.config import get_project_config
|
||||
from basic_memory.mcp.async_client import client
|
||||
from basic_memory.mcp.server import mcp
|
||||
from basic_memory.mcp.tools.build_context import build_context
|
||||
from basic_memory.mcp.tools.recent_activity import recent_activity
|
||||
from basic_memory.mcp.tools.search import search_notes
|
||||
from basic_memory.mcp.tools.utils import call_post
|
||||
from basic_memory.schemas.base import TimeFrame
|
||||
from basic_memory.schemas.memory import GraphContext
|
||||
from basic_memory.schemas.search import SearchItemType
|
||||
from basic_memory.schemas.prompt import ContinueConversationRequest
|
||||
|
||||
|
||||
@mcp.prompt(
|
||||
@@ -45,67 +42,20 @@ async def continue_conversation(
|
||||
"""
|
||||
logger.info(f"Continuing session, topic: {topic}, timeframe: {timeframe}")
|
||||
|
||||
# If topic provided, search for it
|
||||
if topic:
|
||||
search_results = await search_notes(
|
||||
query=topic, after_date=timeframe, entity_types=[SearchItemType.ENTITY]
|
||||
)
|
||||
# Create request model
|
||||
request = ContinueConversationRequest( # pyright: ignore [reportCallIssue]
|
||||
topic=topic, timeframe=timeframe
|
||||
)
|
||||
|
||||
# Build context from results
|
||||
contexts = []
|
||||
for result in search_results.results:
|
||||
if hasattr(result, "permalink") and result.permalink:
|
||||
context: GraphContext = await build_context(f"memory://{result.permalink}")
|
||||
if context.primary_results:
|
||||
contexts.append(
|
||||
PromptContextItem(
|
||||
primary_results=context.primary_results[:1], # pyright: ignore
|
||||
related_results=context.related_results[:3], # pyright: ignore
|
||||
)
|
||||
)
|
||||
project_url = get_project_config().project_url
|
||||
|
||||
# get context for the top 3 results
|
||||
prompt_context = format_prompt_context(
|
||||
PromptContext(topic=topic, timeframe=timeframe, results=contexts) # pyright: ignore
|
||||
)
|
||||
# Call the prompt API endpoint
|
||||
response = await call_post(
|
||||
client,
|
||||
f"{project_url}/prompt/continue-conversation",
|
||||
json=request.model_dump(exclude_none=True),
|
||||
)
|
||||
|
||||
else:
|
||||
# If no topic, get recent activity
|
||||
timeframe = timeframe or "7d"
|
||||
recent: GraphContext = await recent_activity(
|
||||
timeframe=timeframe, type=[SearchItemType.ENTITY]
|
||||
)
|
||||
prompt_context = format_prompt_context(
|
||||
PromptContext(
|
||||
topic=f"Recent Activity from ({timeframe})",
|
||||
timeframe=timeframe,
|
||||
results=[
|
||||
PromptContextItem(
|
||||
primary_results=recent.primary_results[:5], # pyright: ignore
|
||||
related_results=recent.related_results[:2], # pyright: ignore
|
||||
)
|
||||
],
|
||||
)
|
||||
)
|
||||
|
||||
# Add next steps with strong encouragement to write
|
||||
next_steps = dedent(f"""
|
||||
## Next Steps
|
||||
|
||||
You can:
|
||||
- Explore more with: `search_notes({{"text": "{topic}"}})`
|
||||
- See what's changed: `recent_activity(timeframe="{timeframe or "7d"}")`
|
||||
- **Record new learnings or decisions from this conversation:** `write_note(title="[Create a meaningful title]", content="[Content with observations and relations]")`
|
||||
|
||||
## Knowledge Capture Recommendation
|
||||
|
||||
As you continue this conversation, **actively look for opportunities to:**
|
||||
1. Record key information, decisions, or insights that emerge
|
||||
2. Link new knowledge to existing topics
|
||||
3. Suggest capturing important context when appropriate
|
||||
4. Create forward references to topics that might be created later
|
||||
|
||||
Remember that capturing knowledge during conversations is one of the most valuable aspects of Basic Memory.
|
||||
""")
|
||||
|
||||
return prompt_context + next_steps
|
||||
# Extract the rendered prompt from the response
|
||||
result = response.json()
|
||||
return result["prompt"]
|
||||
|
||||
@@ -38,7 +38,19 @@ async def recent_activity_prompt(
|
||||
"""
|
||||
logger.info(f"Getting recent activity, timeframe: {timeframe}")
|
||||
|
||||
recent = await recent_activity(timeframe=timeframe, type=[SearchItemType.ENTITY])
|
||||
recent = await recent_activity.fn(timeframe=timeframe, type=[SearchItemType.ENTITY])
|
||||
|
||||
# Extract primary results from the hierarchical structure
|
||||
primary_results = []
|
||||
related_results = []
|
||||
|
||||
if recent.results:
|
||||
# Take up to 5 primary results
|
||||
for item in recent.results[:5]:
|
||||
primary_results.append(item.primary_result)
|
||||
# Add up to 2 related results per primary item
|
||||
if item.related_results:
|
||||
related_results.extend(item.related_results[:2])
|
||||
|
||||
prompt_context = format_prompt_context(
|
||||
PromptContext(
|
||||
@@ -46,14 +58,18 @@ async def recent_activity_prompt(
|
||||
timeframe=timeframe,
|
||||
results=[
|
||||
PromptContextItem(
|
||||
primary_results=recent.primary_results[:5],
|
||||
related_results=recent.related_results[:2],
|
||||
primary_results=primary_results,
|
||||
related_results=related_results[:10], # Limit total related results
|
||||
)
|
||||
],
|
||||
)
|
||||
)
|
||||
|
||||
# Add suggestions for summarizing recent activity
|
||||
first_title = "Recent Topic"
|
||||
if primary_results and len(primary_results) > 0:
|
||||
first_title = primary_results[0].title
|
||||
|
||||
capture_suggestions = f"""
|
||||
## Opportunity to Capture Activity Summary
|
||||
|
||||
@@ -76,7 +92,7 @@ async def recent_activity_prompt(
|
||||
- [insight] [Connection between different activities]
|
||||
|
||||
## Relations
|
||||
- summarizes [[{recent.primary_results[0].title if recent.primary_results else "Recent Topic"}]]
|
||||
- summarizes [[{first_title}]]
|
||||
- relates_to [[Project Overview]]
|
||||
'''
|
||||
)
|
||||
|
||||
@@ -3,16 +3,17 @@
|
||||
These prompts help users search and explore their knowledge base.
|
||||
"""
|
||||
|
||||
from textwrap import dedent
|
||||
from typing import Annotated, Optional
|
||||
|
||||
from loguru import logger
|
||||
from pydantic import Field
|
||||
|
||||
from basic_memory.config import get_project_config
|
||||
from basic_memory.mcp.async_client import client
|
||||
from basic_memory.mcp.server import mcp
|
||||
from basic_memory.mcp.tools.search import search_notes as search_tool
|
||||
from basic_memory.mcp.tools.utils import call_post
|
||||
from basic_memory.schemas.base import TimeFrame
|
||||
from basic_memory.schemas.search import SearchResponse
|
||||
from basic_memory.schemas.prompt import SearchPromptRequest
|
||||
|
||||
|
||||
@mcp.prompt(
|
||||
@@ -40,143 +41,16 @@ async def search_prompt(
|
||||
"""
|
||||
logger.info(f"Searching knowledge base, query: {query}, timeframe: {timeframe}")
|
||||
|
||||
search_results = await search_tool(query=query, after_date=timeframe)
|
||||
return format_search_results(query, search_results, timeframe)
|
||||
# Create request model
|
||||
request = SearchPromptRequest(query=query, timeframe=timeframe)
|
||||
|
||||
project_url = get_project_config().project_url
|
||||
|
||||
def format_search_results(
|
||||
query: str, results: SearchResponse, timeframe: Optional[TimeFrame] = None
|
||||
) -> str:
|
||||
"""Format search results into a helpful summary.
|
||||
# Call the prompt API endpoint
|
||||
response = await call_post(
|
||||
client, f"{project_url}/prompt/search", json=request.model_dump(exclude_none=True)
|
||||
)
|
||||
|
||||
Args:
|
||||
query: The search query
|
||||
results: Search results object
|
||||
timeframe: How far back results were searched
|
||||
|
||||
Returns:
|
||||
Formatted search results summary
|
||||
"""
|
||||
if not results.results:
|
||||
return dedent(f"""
|
||||
# Search Results for: "{query}"
|
||||
|
||||
I couldn't find any results for this query.
|
||||
|
||||
## Opportunity to Capture Knowledge!
|
||||
|
||||
This is an excellent opportunity to create new knowledge on this topic. Consider:
|
||||
|
||||
```python
|
||||
await write_note(
|
||||
title="{query.capitalize()}",
|
||||
content=f'''
|
||||
# {query.capitalize()}
|
||||
|
||||
## Overview
|
||||
[Summary of what we've discussed about {query}]
|
||||
|
||||
## Observations
|
||||
- [category] [First observation about {query}]
|
||||
- [category] [Second observation about {query}]
|
||||
|
||||
## Relations
|
||||
- relates_to [[Other Relevant Topic]]
|
||||
'''
|
||||
)
|
||||
```
|
||||
|
||||
## Other Suggestions
|
||||
- Try a different search term
|
||||
- Broaden your search criteria
|
||||
- Check recent activity with `recent_activity(timeframe="1w")`
|
||||
""")
|
||||
|
||||
# Start building our summary with header
|
||||
time_info = f" (after {timeframe})" if timeframe else ""
|
||||
summary = dedent(f"""
|
||||
# Search Results for: "{query}"{time_info}
|
||||
|
||||
This is a memory search session.
|
||||
Please use the available basic-memory tools to gather relevant context before responding.
|
||||
I found {len(results.results)} results that match your query.
|
||||
|
||||
Here are the most relevant results:
|
||||
""")
|
||||
|
||||
# Add each search result
|
||||
for i, result in enumerate(results.results[:5]): # Limit to top 5 results
|
||||
summary += dedent(f"""
|
||||
## {i + 1}. {result.title}
|
||||
- **Type**: {result.type.value}
|
||||
""")
|
||||
|
||||
# Add creation date if available in metadata
|
||||
if result.metadata and "created_at" in result.metadata:
|
||||
created_at = result.metadata["created_at"]
|
||||
if hasattr(created_at, "strftime"):
|
||||
summary += (
|
||||
f"- **Created**: {created_at.strftime('%Y-%m-%d %H:%M')}\n" # pragma: no cover
|
||||
)
|
||||
elif isinstance(created_at, str):
|
||||
summary += f"- **Created**: {created_at}\n"
|
||||
|
||||
# Add score and excerpt
|
||||
summary += f"- **Relevance Score**: {result.score:.2f}\n"
|
||||
|
||||
# Add excerpt if available in metadata
|
||||
if result.content:
|
||||
summary += f"- **Excerpt**:\n{result.content}\n"
|
||||
|
||||
# Add permalink for retrieving content
|
||||
if result.permalink:
|
||||
summary += dedent(f"""
|
||||
You can view this content with: `read_note("{result.permalink}")`
|
||||
Or explore its context with: `build_context("memory://{result.permalink}")`
|
||||
""")
|
||||
else:
|
||||
summary += dedent(f"""
|
||||
You can view this file with: `read_file("{result.file_path}")`
|
||||
""") # pragma: no cover
|
||||
|
||||
# Add next steps with strong write encouragement
|
||||
summary += dedent(f"""
|
||||
## Next Steps
|
||||
|
||||
You can:
|
||||
- Refine your search: `search_notes("{query} AND additional_term")`
|
||||
- Exclude terms: `search_notes("{query} NOT exclude_term")`
|
||||
- View more results: `search_notes("{query}", after_date=None)`
|
||||
- Check recent activity: `recent_activity()`
|
||||
|
||||
## Synthesize and Capture Knowledge
|
||||
|
||||
Consider creating a new note that synthesizes what you've learned:
|
||||
|
||||
```python
|
||||
await write_note(
|
||||
title="Synthesis of {query.capitalize()} Information",
|
||||
content='''
|
||||
# Synthesis of {query.capitalize()} Information
|
||||
|
||||
## Overview
|
||||
[Synthesis of the search results and your conversation]
|
||||
|
||||
## Key Insights
|
||||
[Summary of main points learned from these results]
|
||||
|
||||
## Observations
|
||||
- [insight] [Important observation from search results]
|
||||
- [connection] [How this connects to other topics]
|
||||
|
||||
## Relations
|
||||
- relates_to [[{results.results[0].title if results.results else "Related Topic"}]]
|
||||
- extends [[Another Relevant Topic]]
|
||||
'''
|
||||
)
|
||||
```
|
||||
|
||||
Remember that capturing synthesized knowledge is one of the most valuable features of Basic Memory.
|
||||
""")
|
||||
|
||||
return summary
|
||||
# Extract the rendered prompt from the response
|
||||
result = response.json()
|
||||
return result["prompt"]
|
||||
|
||||
@@ -0,0 +1,116 @@
|
||||
"""Sync status prompt for Basic Memory MCP server."""
|
||||
|
||||
from basic_memory.mcp.server import mcp
|
||||
|
||||
|
||||
@mcp.prompt(
|
||||
description="""Get sync status with recommendations for AI assistants.
|
||||
|
||||
This prompt provides both current sync status and guidance on how
|
||||
AI assistants should respond when sync operations are in progress or completed.
|
||||
""",
|
||||
)
|
||||
async def sync_status_prompt() -> str:
|
||||
"""Get sync status with AI assistant guidance.
|
||||
|
||||
This prompt provides detailed sync status information along with
|
||||
recommendations for how AI assistants should handle different sync states.
|
||||
|
||||
Returns:
|
||||
Formatted sync status with AI assistant guidance
|
||||
"""
|
||||
try: # pragma: no cover
|
||||
from basic_memory.services.migration_service import migration_manager
|
||||
|
||||
state = migration_manager.state
|
||||
|
||||
# Build status report
|
||||
lines = [
|
||||
"# Basic Memory Sync Status",
|
||||
"",
|
||||
f"**Current Status**: {state.status.value.replace('_', ' ').title()}",
|
||||
f"**System Ready**: {'Yes' if migration_manager.is_ready else 'No'}",
|
||||
"",
|
||||
]
|
||||
|
||||
if migration_manager.is_ready:
|
||||
lines.extend(
|
||||
[
|
||||
"✅ **All sync operations completed** - System is fully operational",
|
||||
"",
|
||||
"All Basic Memory tools are available and functioning normally.",
|
||||
"File indexing is complete and knowledge graphs are up to date.",
|
||||
"You can proceed with any knowledge management tasks.",
|
||||
]
|
||||
)
|
||||
else:
|
||||
lines.append(f"**Status Message**: {state.message}")
|
||||
|
||||
if state.status.value == "in_progress":
|
||||
if state.projects_total > 0:
|
||||
progress = f" ({state.projects_migrated}/{state.projects_total})"
|
||||
lines.append(f"**Progress**: {progress}")
|
||||
|
||||
lines.extend(
|
||||
[
|
||||
"",
|
||||
"🔄 **File synchronization in progress** - Processing files and building knowledge graphs",
|
||||
"",
|
||||
"**Impact**: Some tools may show status messages instead of normal responses",
|
||||
"until sync completes (usually 1-3 minutes).",
|
||||
]
|
||||
)
|
||||
|
||||
elif state.status.value == "failed":
|
||||
lines.extend(
|
||||
[
|
||||
"",
|
||||
f"❌ **Sync failed**: {state.error or 'Unknown error'}",
|
||||
"",
|
||||
"**Impact**: System may have limited functionality until issue is resolved.",
|
||||
]
|
||||
)
|
||||
|
||||
# Add AI assistant recommendations
|
||||
if not migration_manager.is_ready:
|
||||
lines.extend(
|
||||
[
|
||||
"",
|
||||
"---",
|
||||
"",
|
||||
"## AI Assistant Recommendations",
|
||||
"",
|
||||
"**When sync is in progress:**",
|
||||
"- Inform the user about the background file processing",
|
||||
"- Suggest using `sync_status()` tool to check progress",
|
||||
"- Explain that tools will work normally once sync completes",
|
||||
"- Avoid creating complex workflows until sync is done",
|
||||
"",
|
||||
"**What to tell users:**",
|
||||
"- 'Basic Memory is processing your files and building knowledge graphs'",
|
||||
"- 'This usually takes 1-3 minutes depending on your content size'",
|
||||
"- 'You can check progress anytime with the sync_status tool'",
|
||||
"- 'Full functionality will be available once processing completes'",
|
||||
"",
|
||||
"**User-friendly language:**",
|
||||
"- Say 'processing files' instead of 'migration' or 'sync'",
|
||||
"- Say 'building knowledge graphs' instead of 'indexing'",
|
||||
"- Say 'setting up your knowledge base' instead of 'running migrations'",
|
||||
]
|
||||
)
|
||||
|
||||
return "\n".join(lines)
|
||||
|
||||
except Exception as e: # pragma: no cover
|
||||
return f"""# Sync Status - Error
|
||||
|
||||
❌ **Unable to check sync status**: {str(e)}
|
||||
|
||||
## AI Assistant Recommendations
|
||||
|
||||
**When status is unavailable:**
|
||||
- Assume the system is likely working normally
|
||||
- Try proceeding with normal operations
|
||||
- If users report issues, suggest checking logs or restarting
|
||||
- Use user-friendly language about 'setting up the knowledge base'
|
||||
"""
|
||||
@@ -35,7 +35,7 @@ def format_prompt_context(context: PromptContext) -> str:
|
||||
Returns:
|
||||
Formatted continuation summary
|
||||
"""
|
||||
if not context.results:
|
||||
if not context.results: # pragma: no cover
|
||||
return dedent(f"""
|
||||
# Continuing conversation on: {context.topic}
|
||||
|
||||
@@ -138,11 +138,11 @@ def format_prompt_context(context: PromptContext) -> str:
|
||||
- type: **{related.type}**
|
||||
- title: {related.title}
|
||||
""")
|
||||
if related.permalink:
|
||||
if related.permalink: # pragma: no cover
|
||||
section_content += (
|
||||
f'You can view this document with: `read_note("{related.permalink}")`'
|
||||
)
|
||||
else:
|
||||
else: # pragma: no cover
|
||||
section_content += (
|
||||
f'You can view this file with: `read_file("{related.file_path}")`'
|
||||
)
|
||||
|
||||
+6
-2
@@ -2,13 +2,15 @@
|
||||
|
||||
from loguru import logger
|
||||
|
||||
from basic_memory.mcp.project_session import get_active_project
|
||||
from basic_memory.mcp.async_client import client
|
||||
from basic_memory.mcp.server import mcp
|
||||
from basic_memory.mcp.tools.utils import call_get
|
||||
from basic_memory.schemas import ProjectInfoResponse
|
||||
|
||||
|
||||
@mcp.tool(
|
||||
@mcp.resource(
|
||||
uri="memory://project_info",
|
||||
description="Get information and statistics about the current Basic Memory project.",
|
||||
)
|
||||
async def project_info() -> ProjectInfoResponse:
|
||||
@@ -43,9 +45,11 @@ async def project_info() -> ProjectInfoResponse:
|
||||
print(f"Basic Memory version: {info.system.version}")
|
||||
"""
|
||||
logger.info("Getting project info")
|
||||
project_config = get_active_project()
|
||||
project_url = project_config.project_url
|
||||
|
||||
# Call the API endpoint
|
||||
response = await call_get(client, "/stats/project-info")
|
||||
response = await call_get(client, f"{project_url}/project/info")
|
||||
|
||||
# Convert response to ProjectInfoResponse
|
||||
return ProjectInfoResponse.model_validate(response.json())
|
||||
@@ -1,37 +1,110 @@
|
||||
"""Enhanced FastMCP server instance for Basic Memory."""
|
||||
"""
|
||||
Basic Memory FastMCP server.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
from contextlib import asynccontextmanager
|
||||
from typing import AsyncIterator, Optional
|
||||
|
||||
from mcp.server.fastmcp import FastMCP
|
||||
from mcp.server.fastmcp.utilities.logging import configure_logging as mcp_configure_logging
|
||||
from dataclasses import dataclass
|
||||
from typing import AsyncIterator, Optional, Any
|
||||
|
||||
from basic_memory.config import config as project_config
|
||||
from dotenv import load_dotenv
|
||||
from fastmcp import FastMCP
|
||||
from fastmcp.utilities.logging import configure_logging as mcp_configure_logging
|
||||
from mcp.server.auth.settings import AuthSettings
|
||||
|
||||
from basic_memory.config import app_config
|
||||
from basic_memory.services.initialization import initialize_app
|
||||
from basic_memory.mcp.auth_provider import BasicMemoryOAuthProvider
|
||||
from basic_memory.mcp.project_session import session
|
||||
from basic_memory.mcp.external_auth_provider import (
|
||||
create_github_provider,
|
||||
create_google_provider,
|
||||
)
|
||||
from basic_memory.mcp.supabase_auth_provider import SupabaseOAuthProvider
|
||||
|
||||
# mcp console logging
|
||||
mcp_configure_logging(level="ERROR")
|
||||
|
||||
load_dotenv()
|
||||
|
||||
|
||||
@dataclass
|
||||
class AppContext:
|
||||
watch_task: Optional[asyncio.Task]
|
||||
migration_manager: Optional[Any] = None
|
||||
|
||||
|
||||
@asynccontextmanager
|
||||
async def app_lifespan(server: FastMCP) -> AsyncIterator[AppContext]: # pragma: no cover
|
||||
"""Manage application lifecycle with type-safe context"""
|
||||
# Initialize on startup
|
||||
watch_task = await initialize_app(project_config)
|
||||
try:
|
||||
yield AppContext(watch_task=watch_task)
|
||||
finally:
|
||||
# Cleanup on shutdown
|
||||
if watch_task:
|
||||
watch_task.cancel()
|
||||
# Initialize on startup (now returns migration_manager)
|
||||
migration_manager = await initialize_app(app_config)
|
||||
|
||||
# Initialize project session with default project
|
||||
session.initialize(app_config.default_project)
|
||||
|
||||
try:
|
||||
yield AppContext(watch_task=None, migration_manager=migration_manager)
|
||||
finally:
|
||||
# Cleanup on shutdown - migration tasks will be cancelled automatically
|
||||
pass
|
||||
|
||||
|
||||
# OAuth configuration function
|
||||
def create_auth_config() -> tuple[AuthSettings | None, Any | None]:
|
||||
"""Create OAuth configuration if enabled."""
|
||||
# Check if OAuth is enabled via environment variable
|
||||
import os
|
||||
|
||||
if os.getenv("FASTMCP_AUTH_ENABLED", "false").lower() == "true":
|
||||
from pydantic import AnyHttpUrl
|
||||
|
||||
# Configure OAuth settings
|
||||
issuer_url = os.getenv("FASTMCP_AUTH_ISSUER_URL", "http://localhost:8000")
|
||||
required_scopes = os.getenv("FASTMCP_AUTH_REQUIRED_SCOPES", "read,write")
|
||||
docs_url = os.getenv("FASTMCP_AUTH_DOCS_URL") or "http://localhost:8000/docs/oauth"
|
||||
|
||||
auth_settings = AuthSettings(
|
||||
issuer_url=AnyHttpUrl(issuer_url),
|
||||
service_documentation_url=AnyHttpUrl(docs_url),
|
||||
required_scopes=required_scopes.split(",") if required_scopes else ["read", "write"],
|
||||
)
|
||||
|
||||
# Create OAuth provider based on type
|
||||
provider_type = os.getenv("FASTMCP_AUTH_PROVIDER", "basic").lower()
|
||||
|
||||
if provider_type == "github":
|
||||
auth_provider = create_github_provider()
|
||||
elif provider_type == "google":
|
||||
auth_provider = create_google_provider()
|
||||
elif provider_type == "supabase":
|
||||
supabase_url = os.getenv("SUPABASE_URL")
|
||||
supabase_anon_key = os.getenv("SUPABASE_ANON_KEY")
|
||||
supabase_service_key = os.getenv("SUPABASE_SERVICE_KEY")
|
||||
|
||||
if not supabase_url or not supabase_anon_key:
|
||||
raise ValueError("SUPABASE_URL and SUPABASE_ANON_KEY must be set for Supabase auth")
|
||||
|
||||
auth_provider = SupabaseOAuthProvider(
|
||||
supabase_url=supabase_url,
|
||||
supabase_anon_key=supabase_anon_key,
|
||||
supabase_service_key=supabase_service_key,
|
||||
issuer_url=issuer_url,
|
||||
)
|
||||
else: # default to "basic"
|
||||
auth_provider = BasicMemoryOAuthProvider(issuer_url=issuer_url)
|
||||
|
||||
return auth_settings, auth_provider
|
||||
|
||||
return None, None
|
||||
|
||||
|
||||
# Create auth configuration
|
||||
auth_settings, auth_provider = create_auth_config()
|
||||
|
||||
# Create the shared server instance
|
||||
mcp = FastMCP("Basic Memory", log_level="ERROR", lifespan=app_lifespan)
|
||||
mcp = FastMCP(
|
||||
name="Basic Memory",
|
||||
log_level="DEBUG",
|
||||
auth=auth_provider,
|
||||
)
|
||||
|
||||
@@ -0,0 +1,463 @@
|
||||
"""Supabase OAuth provider for Basic Memory MCP server."""
|
||||
|
||||
import os
|
||||
import secrets
|
||||
from dataclasses import dataclass
|
||||
from datetime import datetime, timedelta
|
||||
from typing import Optional, Dict, Any
|
||||
|
||||
import httpx
|
||||
import jwt
|
||||
from loguru import logger
|
||||
from mcp.server.auth.provider import (
|
||||
OAuthAuthorizationServerProvider,
|
||||
AuthorizationParams,
|
||||
AuthorizationCode,
|
||||
RefreshToken,
|
||||
AccessToken,
|
||||
TokenError,
|
||||
AuthorizeError,
|
||||
)
|
||||
from mcp.shared.auth import OAuthClientInformationFull, OAuthToken
|
||||
|
||||
|
||||
@dataclass
|
||||
class SupabaseAuthorizationCode(AuthorizationCode):
|
||||
"""Authorization code with Supabase metadata."""
|
||||
|
||||
user_id: Optional[str] = None
|
||||
email: Optional[str] = None
|
||||
|
||||
|
||||
@dataclass
|
||||
class SupabaseRefreshToken(RefreshToken):
|
||||
"""Refresh token with Supabase metadata."""
|
||||
|
||||
supabase_refresh_token: Optional[str] = None
|
||||
user_id: Optional[str] = None
|
||||
|
||||
|
||||
@dataclass
|
||||
class SupabaseAccessToken(AccessToken):
|
||||
"""Access token with Supabase metadata."""
|
||||
|
||||
supabase_access_token: Optional[str] = None
|
||||
user_id: Optional[str] = None
|
||||
email: Optional[str] = None
|
||||
|
||||
|
||||
class SupabaseOAuthProvider(
|
||||
OAuthAuthorizationServerProvider[
|
||||
SupabaseAuthorizationCode, SupabaseRefreshToken, SupabaseAccessToken
|
||||
]
|
||||
):
|
||||
"""OAuth provider that integrates with Supabase Auth.
|
||||
|
||||
This provider uses Supabase as the authentication backend while
|
||||
maintaining compatibility with MCP's OAuth requirements.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
supabase_url: str,
|
||||
supabase_anon_key: str,
|
||||
supabase_service_key: Optional[str] = None,
|
||||
issuer_url: str = "http://localhost:8000",
|
||||
):
|
||||
self.supabase_url = supabase_url.rstrip("/")
|
||||
self.supabase_anon_key = supabase_anon_key
|
||||
self.supabase_service_key = supabase_service_key or supabase_anon_key
|
||||
self.issuer_url = issuer_url
|
||||
|
||||
# HTTP client for Supabase API calls
|
||||
self.http_client = httpx.AsyncClient()
|
||||
|
||||
# Temporary storage for auth flows (in production, use Supabase DB)
|
||||
self.pending_auth_codes: Dict[str, SupabaseAuthorizationCode] = {}
|
||||
self.mcp_to_supabase_tokens: Dict[str, Dict[str, Any]] = {}
|
||||
|
||||
async def get_client(self, client_id: str) -> Optional[OAuthClientInformationFull]:
|
||||
"""Get a client from Supabase.
|
||||
|
||||
In production, this would query a clients table in Supabase.
|
||||
"""
|
||||
# For now, we'll validate against a configured list of allowed clients
|
||||
# In production, query Supabase DB for client info
|
||||
allowed_clients = os.getenv("SUPABASE_ALLOWED_CLIENTS", "").split(",")
|
||||
|
||||
if client_id in allowed_clients:
|
||||
return OAuthClientInformationFull(
|
||||
client_id=client_id,
|
||||
client_secret="", # Supabase handles secrets
|
||||
redirect_uris=[], # Supabase handles redirect URIs
|
||||
)
|
||||
|
||||
return None
|
||||
|
||||
async def register_client(self, client_info: OAuthClientInformationFull) -> None:
|
||||
"""Register a new OAuth client in Supabase.
|
||||
|
||||
In production, this would insert into a clients table.
|
||||
"""
|
||||
# For development, we just log the registration
|
||||
logger.info(f"Would register client {client_info.client_id} in Supabase")
|
||||
|
||||
# In production:
|
||||
# await self.supabase.table('oauth_clients').insert({
|
||||
# 'client_id': client_info.client_id,
|
||||
# 'client_secret': client_info.client_secret,
|
||||
# 'metadata': client_info.client_metadata,
|
||||
# }).execute()
|
||||
|
||||
async def authorize(
|
||||
self, client: OAuthClientInformationFull, params: AuthorizationParams
|
||||
) -> str:
|
||||
"""Create authorization URL redirecting to Supabase Auth.
|
||||
|
||||
This initiates the OAuth flow with Supabase as the identity provider.
|
||||
"""
|
||||
# Generate state for this auth request
|
||||
state = secrets.token_urlsafe(32)
|
||||
|
||||
# Store the authorization request
|
||||
self.pending_auth_codes[state] = SupabaseAuthorizationCode(
|
||||
code=state,
|
||||
scopes=params.scopes or [],
|
||||
expires_at=(datetime.utcnow() + timedelta(minutes=10)).timestamp(),
|
||||
client_id=client.client_id,
|
||||
code_challenge=params.code_challenge,
|
||||
redirect_uri=params.redirect_uri,
|
||||
redirect_uri_provided_explicitly=params.redirect_uri_provided_explicitly,
|
||||
)
|
||||
|
||||
# Build Supabase auth URL
|
||||
auth_params = {
|
||||
"redirect_to": f"{self.issuer_url}/auth/callback",
|
||||
"scopes": " ".join(params.scopes or ["openid", "email"]),
|
||||
"state": state,
|
||||
}
|
||||
|
||||
# Use Supabase's OAuth endpoint
|
||||
auth_url = f"{self.supabase_url}/auth/v1/authorize"
|
||||
query_string = "&".join(f"{k}={v}" for k, v in auth_params.items())
|
||||
|
||||
return f"{auth_url}?{query_string}"
|
||||
|
||||
async def handle_supabase_callback(self, code: str, state: str) -> str:
|
||||
"""Handle callback from Supabase after user authentication."""
|
||||
# Get the original auth request
|
||||
auth_request = self.pending_auth_codes.get(state)
|
||||
if not auth_request:
|
||||
raise AuthorizeError(
|
||||
error="invalid_request",
|
||||
error_description="Invalid state parameter",
|
||||
)
|
||||
|
||||
# Exchange code with Supabase for tokens
|
||||
token_response = await self.http_client.post(
|
||||
f"{self.supabase_url}/auth/v1/token",
|
||||
json={
|
||||
"grant_type": "authorization_code",
|
||||
"code": code,
|
||||
"redirect_uri": f"{self.issuer_url}/auth/callback",
|
||||
},
|
||||
headers={
|
||||
"apikey": self.supabase_anon_key,
|
||||
"Authorization": f"Bearer {self.supabase_anon_key}",
|
||||
},
|
||||
)
|
||||
|
||||
if not token_response.is_success:
|
||||
raise AuthorizeError(
|
||||
error="server_error",
|
||||
error_description="Failed to exchange code with Supabase",
|
||||
)
|
||||
|
||||
supabase_tokens = token_response.json()
|
||||
|
||||
# Get user info from Supabase
|
||||
user_response = await self.http_client.get(
|
||||
f"{self.supabase_url}/auth/v1/user",
|
||||
headers={
|
||||
"apikey": self.supabase_anon_key,
|
||||
"Authorization": f"Bearer {supabase_tokens['access_token']}",
|
||||
},
|
||||
)
|
||||
|
||||
user_data = user_response.json() if user_response.is_success else {}
|
||||
|
||||
# Generate MCP authorization code
|
||||
mcp_code = secrets.token_urlsafe(32)
|
||||
|
||||
# Update auth request with user info
|
||||
auth_request.code = mcp_code
|
||||
auth_request.user_id = user_data.get("id")
|
||||
auth_request.email = user_data.get("email")
|
||||
|
||||
# Store mapping
|
||||
self.pending_auth_codes[mcp_code] = auth_request
|
||||
self.mcp_to_supabase_tokens[mcp_code] = {
|
||||
"supabase_tokens": supabase_tokens,
|
||||
"user": user_data,
|
||||
}
|
||||
|
||||
# Clean up old state
|
||||
del self.pending_auth_codes[state]
|
||||
|
||||
# Redirect back to client
|
||||
redirect_uri = str(auth_request.redirect_uri)
|
||||
separator = "&" if "?" in redirect_uri else "?"
|
||||
|
||||
return f"{redirect_uri}{separator}code={mcp_code}&state={state}"
|
||||
|
||||
async def load_authorization_code(
|
||||
self, client: OAuthClientInformationFull, authorization_code: str
|
||||
) -> Optional[SupabaseAuthorizationCode]:
|
||||
"""Load an authorization code."""
|
||||
code = self.pending_auth_codes.get(authorization_code)
|
||||
|
||||
if code and code.client_id == client.client_id:
|
||||
# Check expiration
|
||||
if datetime.utcnow().timestamp() > code.expires_at:
|
||||
del self.pending_auth_codes[authorization_code]
|
||||
return None
|
||||
return code
|
||||
|
||||
return None
|
||||
|
||||
async def exchange_authorization_code(
|
||||
self, client: OAuthClientInformationFull, authorization_code: SupabaseAuthorizationCode
|
||||
) -> OAuthToken:
|
||||
"""Exchange authorization code for tokens."""
|
||||
# Get stored Supabase tokens
|
||||
token_data = self.mcp_to_supabase_tokens.get(authorization_code.code)
|
||||
if not token_data:
|
||||
raise TokenError(error="invalid_grant", error_description="Invalid authorization code")
|
||||
|
||||
supabase_tokens = token_data["supabase_tokens"]
|
||||
user = token_data["user"]
|
||||
|
||||
# Generate MCP tokens that wrap Supabase tokens
|
||||
access_token = self._generate_mcp_token(
|
||||
client_id=client.client_id,
|
||||
user_id=user.get("id", ""),
|
||||
email=user.get("email", ""),
|
||||
scopes=authorization_code.scopes,
|
||||
supabase_access_token=supabase_tokens["access_token"],
|
||||
)
|
||||
|
||||
refresh_token = secrets.token_urlsafe(32)
|
||||
|
||||
# Store the token mapping
|
||||
self.mcp_to_supabase_tokens[access_token] = {
|
||||
"client_id": client.client_id,
|
||||
"user_id": user.get("id"),
|
||||
"email": user.get("email"),
|
||||
"supabase_access_token": supabase_tokens["access_token"],
|
||||
"supabase_refresh_token": supabase_tokens["refresh_token"],
|
||||
"scopes": authorization_code.scopes,
|
||||
}
|
||||
|
||||
# Store refresh token mapping
|
||||
self.mcp_to_supabase_tokens[refresh_token] = {
|
||||
"client_id": client.client_id,
|
||||
"user_id": user.get("id"),
|
||||
"supabase_refresh_token": supabase_tokens["refresh_token"],
|
||||
"scopes": authorization_code.scopes,
|
||||
}
|
||||
|
||||
# Clean up authorization code
|
||||
del self.pending_auth_codes[authorization_code.code]
|
||||
del self.mcp_to_supabase_tokens[authorization_code.code]
|
||||
|
||||
return OAuthToken(
|
||||
access_token=access_token,
|
||||
token_type="bearer",
|
||||
expires_in=supabase_tokens.get("expires_in", 3600),
|
||||
refresh_token=refresh_token,
|
||||
scope=" ".join(authorization_code.scopes) if authorization_code.scopes else None,
|
||||
)
|
||||
|
||||
async def load_refresh_token(
|
||||
self, client: OAuthClientInformationFull, refresh_token: str
|
||||
) -> Optional[SupabaseRefreshToken]:
|
||||
"""Load a refresh token."""
|
||||
token_data = self.mcp_to_supabase_tokens.get(refresh_token)
|
||||
|
||||
if token_data and token_data["client_id"] == client.client_id:
|
||||
return SupabaseRefreshToken(
|
||||
token=refresh_token,
|
||||
client_id=client.client_id,
|
||||
scopes=token_data["scopes"],
|
||||
supabase_refresh_token=token_data["supabase_refresh_token"],
|
||||
user_id=token_data.get("user_id"),
|
||||
)
|
||||
|
||||
return None
|
||||
|
||||
async def exchange_refresh_token(
|
||||
self,
|
||||
client: OAuthClientInformationFull,
|
||||
refresh_token: SupabaseRefreshToken,
|
||||
scopes: list[str],
|
||||
) -> OAuthToken:
|
||||
"""Exchange refresh token for new tokens using Supabase."""
|
||||
# Refresh with Supabase
|
||||
token_response = await self.http_client.post(
|
||||
f"{self.supabase_url}/auth/v1/token",
|
||||
json={
|
||||
"grant_type": "refresh_token",
|
||||
"refresh_token": refresh_token.supabase_refresh_token,
|
||||
},
|
||||
headers={
|
||||
"apikey": self.supabase_anon_key,
|
||||
"Authorization": f"Bearer {self.supabase_anon_key}",
|
||||
},
|
||||
)
|
||||
|
||||
if not token_response.is_success:
|
||||
raise TokenError(
|
||||
error="invalid_grant",
|
||||
error_description="Failed to refresh with Supabase",
|
||||
)
|
||||
|
||||
supabase_tokens = token_response.json()
|
||||
|
||||
# Get updated user info
|
||||
user_response = await self.http_client.get(
|
||||
f"{self.supabase_url}/auth/v1/user",
|
||||
headers={
|
||||
"apikey": self.supabase_anon_key,
|
||||
"Authorization": f"Bearer {supabase_tokens['access_token']}",
|
||||
},
|
||||
)
|
||||
|
||||
user_data = user_response.json() if user_response.is_success else {}
|
||||
|
||||
# Generate new MCP tokens
|
||||
new_access_token = self._generate_mcp_token(
|
||||
client_id=client.client_id,
|
||||
user_id=user_data.get("id", ""),
|
||||
email=user_data.get("email", ""),
|
||||
scopes=scopes or refresh_token.scopes,
|
||||
supabase_access_token=supabase_tokens["access_token"],
|
||||
)
|
||||
|
||||
new_refresh_token = secrets.token_urlsafe(32)
|
||||
|
||||
# Update token mappings
|
||||
self.mcp_to_supabase_tokens[new_access_token] = {
|
||||
"client_id": client.client_id,
|
||||
"user_id": user_data.get("id"),
|
||||
"email": user_data.get("email"),
|
||||
"supabase_access_token": supabase_tokens["access_token"],
|
||||
"supabase_refresh_token": supabase_tokens["refresh_token"],
|
||||
"scopes": scopes or refresh_token.scopes,
|
||||
}
|
||||
|
||||
self.mcp_to_supabase_tokens[new_refresh_token] = {
|
||||
"client_id": client.client_id,
|
||||
"user_id": user_data.get("id"),
|
||||
"supabase_refresh_token": supabase_tokens["refresh_token"],
|
||||
"scopes": scopes or refresh_token.scopes,
|
||||
}
|
||||
|
||||
# Clean up old tokens
|
||||
del self.mcp_to_supabase_tokens[refresh_token.token]
|
||||
|
||||
return OAuthToken(
|
||||
access_token=new_access_token,
|
||||
token_type="bearer",
|
||||
expires_in=supabase_tokens.get("expires_in", 3600),
|
||||
refresh_token=new_refresh_token,
|
||||
scope=" ".join(scopes or refresh_token.scopes),
|
||||
)
|
||||
|
||||
async def load_access_token(self, token: str) -> Optional[SupabaseAccessToken]:
|
||||
"""Load and validate an access token."""
|
||||
# First check our mapping
|
||||
token_data = self.mcp_to_supabase_tokens.get(token)
|
||||
if token_data:
|
||||
return SupabaseAccessToken(
|
||||
token=token,
|
||||
client_id=token_data["client_id"],
|
||||
scopes=token_data["scopes"],
|
||||
supabase_access_token=token_data.get("supabase_access_token"),
|
||||
user_id=token_data.get("user_id"),
|
||||
email=token_data.get("email"),
|
||||
)
|
||||
|
||||
# Try to decode as JWT
|
||||
try:
|
||||
# Verify with Supabase's JWT secret
|
||||
payload = jwt.decode(
|
||||
token,
|
||||
os.getenv("SUPABASE_JWT_SECRET", ""),
|
||||
algorithms=["HS256"],
|
||||
audience="authenticated",
|
||||
)
|
||||
|
||||
return SupabaseAccessToken(
|
||||
token=token,
|
||||
client_id=payload.get("client_id", ""),
|
||||
scopes=payload.get("scopes", []),
|
||||
user_id=payload.get("sub"),
|
||||
email=payload.get("email"),
|
||||
)
|
||||
except jwt.InvalidTokenError:
|
||||
pass
|
||||
|
||||
# Validate with Supabase
|
||||
user_response = await self.http_client.get(
|
||||
f"{self.supabase_url}/auth/v1/user",
|
||||
headers={
|
||||
"apikey": self.supabase_anon_key,
|
||||
"Authorization": f"Bearer {token}",
|
||||
},
|
||||
)
|
||||
|
||||
if user_response.is_success:
|
||||
user_data = user_response.json()
|
||||
return SupabaseAccessToken(
|
||||
token=token,
|
||||
client_id="", # Unknown client for direct Supabase tokens
|
||||
scopes=[],
|
||||
supabase_access_token=token,
|
||||
user_id=user_data.get("id"),
|
||||
email=user_data.get("email"),
|
||||
)
|
||||
|
||||
return None
|
||||
|
||||
async def revoke_token(self, token: SupabaseAccessToken | SupabaseRefreshToken) -> None:
|
||||
"""Revoke a token."""
|
||||
# Remove from our mapping
|
||||
self.mcp_to_supabase_tokens.pop(token.token, None)
|
||||
|
||||
# In production, also revoke in Supabase:
|
||||
# await self.supabase.auth.admin.sign_out(token.user_id)
|
||||
|
||||
def _generate_mcp_token(
|
||||
self,
|
||||
client_id: str,
|
||||
user_id: str,
|
||||
email: str,
|
||||
scopes: list[str],
|
||||
supabase_access_token: str,
|
||||
) -> str:
|
||||
"""Generate an MCP token that wraps Supabase authentication."""
|
||||
payload = {
|
||||
"iss": self.issuer_url,
|
||||
"sub": user_id,
|
||||
"client_id": client_id,
|
||||
"email": email,
|
||||
"scopes": scopes,
|
||||
"supabase_token": supabase_access_token[:10] + "...", # Reference only
|
||||
"exp": datetime.utcnow() + timedelta(hours=1),
|
||||
"iat": datetime.utcnow(),
|
||||
}
|
||||
|
||||
# Use Supabase JWT secret if available
|
||||
secret = os.getenv("SUPABASE_JWT_SECRET", secrets.token_urlsafe(32))
|
||||
|
||||
return jwt.encode(payload, secret, algorithm="HS256")
|
||||
@@ -11,17 +11,41 @@ from basic_memory.mcp.tools.read_content import read_content
|
||||
from basic_memory.mcp.tools.build_context import build_context
|
||||
from basic_memory.mcp.tools.recent_activity import recent_activity
|
||||
from basic_memory.mcp.tools.read_note import read_note
|
||||
from basic_memory.mcp.tools.view_note import view_note
|
||||
from basic_memory.mcp.tools.write_note import write_note
|
||||
from basic_memory.mcp.tools.search import search_notes
|
||||
from basic_memory.mcp.tools.canvas import canvas
|
||||
from basic_memory.mcp.tools.list_directory import list_directory
|
||||
from basic_memory.mcp.tools.edit_note import edit_note
|
||||
from basic_memory.mcp.tools.move_note import move_note
|
||||
from basic_memory.mcp.tools.sync_status import sync_status
|
||||
from basic_memory.mcp.tools.project_management import (
|
||||
list_projects,
|
||||
switch_project,
|
||||
get_current_project,
|
||||
set_default_project,
|
||||
create_project,
|
||||
delete_project,
|
||||
)
|
||||
|
||||
__all__ = [
|
||||
"build_context",
|
||||
"canvas",
|
||||
"create_project",
|
||||
"delete_note",
|
||||
"delete_project",
|
||||
"edit_note",
|
||||
"get_current_project",
|
||||
"list_directory",
|
||||
"list_projects",
|
||||
"move_note",
|
||||
"read_content",
|
||||
"read_note",
|
||||
"recent_activity",
|
||||
"search_notes",
|
||||
"set_default_project",
|
||||
"switch_project",
|
||||
"sync_status",
|
||||
"view_note",
|
||||
"write_note",
|
||||
]
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user