mirror of
https://github.com/basicmachines-co/basic-memory
synced 2026-06-21 13:47:35 +00:00
Merge branch 'main' of github.com:basicmachines-co/basic-memory
This commit is contained in:
phernandez
@@ -60,10 +60,6 @@ jobs:
|
||||
run: |
|
||||
just typecheck
|
||||
|
||||
- name: Run type checks
|
||||
run: |
|
||||
just typecheck
|
||||
|
||||
- name: Run linting
|
||||
run: |
|
||||
just lint
|
||||
|
||||
@@ -1,80 +0,0 @@
|
||||
# Bug Fix: Project Deletion Failure
|
||||
|
||||
## Problem Description
|
||||
|
||||
The `delete_project` MCP tool was failing with "Project 'test-verify' not found" even though the project clearly existed and showed up in `list_memory_projects`.
|
||||
|
||||
## Root Cause
|
||||
|
||||
The bug was in `/Users/drew/code/basic-memory/src/basic_memory/config.py` in the `ConfigManager.remove_project()` method (line 311):
|
||||
|
||||
```python
|
||||
def remove_project(self, name: str) -> None:
|
||||
"""Remove a project from the configuration."""
|
||||
|
||||
project_name, path = self.get_project(name)
|
||||
if not project_name:
|
||||
raise ValueError(f"Project '{name}' not found")
|
||||
|
||||
config = self.load_config()
|
||||
if project_name == config.default_project:
|
||||
raise ValueError(f"Cannot remove the default project '{name}'")
|
||||
|
||||
del config.projects[name] # ← BUG: Using input name instead of found project_name
|
||||
self.save_config(config)
|
||||
```
|
||||
|
||||
**The Issue:**
|
||||
1. Line 305: `get_project(name)` does a permalink-based lookup and returns the **actual** project name from the config (e.g., "test-verify")
|
||||
2. Line 311: `del config.projects[name]` tries to delete using the **input** name parameter instead of the `project_name` that was just found
|
||||
3. Since `get_project()` uses permalink matching, it can find a project even if the input name doesn't match the exact dictionary key
|
||||
|
||||
**Example Scenario:**
|
||||
- Config has project key: `"test-verify"`
|
||||
- User calls: `delete_project("test-verify")`
|
||||
- `get_project("test-verify")` finds it via permalink matching and returns `("test-verify", "/path")`
|
||||
- `del config.projects["test-verify"]` tries to delete using input, which should work...
|
||||
- BUT if there's any normalization mismatch between the stored key and the input, it fails
|
||||
|
||||
## The Fix
|
||||
|
||||
Changed line 311 to use the `project_name` returned by `get_project()`:
|
||||
|
||||
```python
|
||||
# Use the found project_name (which may differ from input name due to permalink matching)
|
||||
del config.projects[project_name]
|
||||
```
|
||||
|
||||
This ensures we're deleting the exact key that exists in the config dictionary, not the potentially non-normalized input name.
|
||||
|
||||
## Testing
|
||||
|
||||
After applying this fix, the delete operation should work correctly:
|
||||
|
||||
```python
|
||||
# This should now succeed
|
||||
await delete_project("test-verify")
|
||||
```
|
||||
|
||||
## Related Code
|
||||
|
||||
The same pattern is correctly used in other methods:
|
||||
- `set_default_project()` correctly uses the found `project_name` when setting default
|
||||
- The API endpoint `remove_project()` in project_router.py correctly passes through to this method
|
||||
|
||||
## Commit Message
|
||||
|
||||
```
|
||||
fix: use found project_name in ConfigManager.remove_project()
|
||||
|
||||
The remove_project() method was using the input name parameter to delete
|
||||
from config.projects instead of the project_name returned by get_project().
|
||||
This caused failures when the input name didn't exactly match the config
|
||||
dictionary key, even though get_project() successfully found the project
|
||||
via permalink matching.
|
||||
|
||||
Now uses the actual project_name returned by get_project() to ensure we're
|
||||
deleting the correct dictionary key.
|
||||
|
||||
Fixes: Project deletion failing with "not found" error despite project existing
|
||||
```
|
||||
@@ -14,22 +14,34 @@ See the [README.md](README.md) file for a project overview.
|
||||
|
||||
### Build and Test Commands
|
||||
|
||||
- Install: `make install` or `pip install -e ".[dev]"`
|
||||
- Run tests: `uv run pytest -p pytest_mock -v` or `make test`
|
||||
- Install: `just install` or `pip install -e ".[dev]"`
|
||||
- Run all tests (with coverage): `just test` - Runs both unit and integration tests with unified coverage
|
||||
- Run unit tests only: `just test-unit` - Fast, no coverage
|
||||
- Run integration tests only: `just test-int` - Fast, no coverage
|
||||
- Generate HTML coverage: `just coverage` - Opens in browser
|
||||
- Single test: `pytest tests/path/to/test_file.py::test_function_name`
|
||||
- Lint: `make lint` or `ruff check . --fix`
|
||||
- Type check: `make type-check` or `uv run pyright`
|
||||
- Format: `make format` or `uv run ruff format .`
|
||||
- Run all code checks: `make check` (runs lint, format, type-check, test)
|
||||
- Create db migration: `make migration m="Your migration message"`
|
||||
- Run development MCP Inspector: `make run-inspector`
|
||||
- Run benchmarks: `pytest test-int/test_sync_performance_benchmark.py -v -m "benchmark and not slow"`
|
||||
- Lint: `just lint` or `ruff check . --fix`
|
||||
- Type check: `just typecheck` or `uv run pyright`
|
||||
- Format: `just format` or `uv run ruff format .`
|
||||
- Run all code checks: `just check` (runs lint, format, typecheck, test)
|
||||
- Create db migration: `just migration "Your migration message"`
|
||||
- Run development MCP Inspector: `just run-inspector`
|
||||
|
||||
**Note:** Project supports Python 3.10+
|
||||
**Note:** Project requires Python 3.12+ (uses type parameter syntax and `type` aliases introduced in 3.12)
|
||||
|
||||
### Test Structure
|
||||
|
||||
- `tests/` - Unit tests for individual components (mocked, fast)
|
||||
- `test-int/` - Integration tests for real-world scenarios (no mocks, realistic)
|
||||
- Both directories are covered by unified coverage reporting
|
||||
- Benchmark tests in `test-int/` are marked with `@pytest.mark.benchmark`
|
||||
- Slow tests are marked with `@pytest.mark.slow`
|
||||
|
||||
### Code Style Guidelines
|
||||
|
||||
- Line length: 100 characters max
|
||||
- Python 3.12+ with full type annotations
|
||||
- Python 3.12+ with full type annotations (uses type parameters and type aliases)
|
||||
- Format with ruff (consistent styling)
|
||||
- Import order: standard lib, third-party, local imports
|
||||
- Naming: snake_case for functions/variables, PascalCase for classes
|
||||
@@ -63,9 +75,11 @@ See the [README.md](README.md) file for a project overview.
|
||||
- Schema changes require Alembic migrations
|
||||
- SQLite is used for indexing and full text search, files are source of truth
|
||||
- Testing uses pytest with asyncio support (strict mode)
|
||||
- Unit tests (`tests/`) use mocks when necessary; integration tests (`test-int/`) use real implementations
|
||||
- Test database uses in-memory SQLite
|
||||
- Avoid creating mocks in tests in most circumstances.
|
||||
- Each test runs in a standalone environment with in memory SQLite and tmp_file directory
|
||||
- Each test runs in a standalone environment with in-memory SQLite and tmp_file directory
|
||||
- Performance benchmarks are in `test-int/test_sync_performance_benchmark.py`
|
||||
- Use pytest markers: `@pytest.mark.benchmark` for benchmarks, `@pytest.mark.slow` for slow tests
|
||||
|
||||
### Async Client Pattern (Important!)
|
||||
|
||||
|
||||
+80
-7
@@ -34,11 +34,18 @@ project and how to get started as a developer.
|
||||
|
||||
4. **Run the Tests**:
|
||||
```bash
|
||||
# Run all tests
|
||||
# Run all tests with unified coverage (unit + integration)
|
||||
just test
|
||||
# or
|
||||
uv run pytest -p pytest_mock -v
|
||||
|
||||
|
||||
# Run unit tests only (fast, no coverage)
|
||||
just test-unit
|
||||
|
||||
# Run integration tests only (fast, no coverage)
|
||||
just test-int
|
||||
|
||||
# Generate HTML coverage report
|
||||
just coverage
|
||||
|
||||
# Run a specific test
|
||||
pytest tests/path/to/test_file.py::test_function_name
|
||||
```
|
||||
@@ -134,7 +141,7 @@ agreement to the DCO.
|
||||
|
||||
## Code Style Guidelines
|
||||
|
||||
- **Python Version**: Python 3.12+ with full type annotations
|
||||
- **Python Version**: Python 3.12+ with full type annotations (3.12+ required for type parameter syntax)
|
||||
- **Line Length**: 100 characters maximum
|
||||
- **Formatting**: Use ruff for consistent styling
|
||||
- **Import Order**: Standard lib, third-party, local imports
|
||||
@@ -144,12 +151,78 @@ agreement to the DCO.
|
||||
|
||||
## Testing Guidelines
|
||||
|
||||
- **Coverage Target**: We aim for 100% test coverage for all code
|
||||
### Test Structure
|
||||
|
||||
Basic Memory uses two test directories with unified coverage reporting:
|
||||
|
||||
- **`tests/`**: Unit tests that test individual components in isolation
|
||||
- Fast execution with extensive mocking
|
||||
- Test individual functions, classes, and modules
|
||||
- Run with: `just test-unit` (no coverage, fast)
|
||||
|
||||
- **`test-int/`**: Integration tests that test real-world scenarios
|
||||
- Test full workflows with real database and file operations
|
||||
- Include performance benchmarks
|
||||
- More realistic but slower than unit tests
|
||||
- Run with: `just test-int` (no coverage, fast)
|
||||
|
||||
### Running Tests
|
||||
|
||||
```bash
|
||||
# Run all tests with unified coverage report
|
||||
just test
|
||||
|
||||
# Run only unit tests (fast iteration)
|
||||
just test-unit
|
||||
|
||||
# Run only integration tests
|
||||
just test-int
|
||||
|
||||
# Generate HTML coverage report
|
||||
just coverage
|
||||
|
||||
# Run specific test
|
||||
pytest tests/path/to/test_file.py::test_function_name
|
||||
|
||||
# Run tests excluding benchmarks
|
||||
pytest -m "not benchmark"
|
||||
|
||||
# Run only benchmark tests
|
||||
pytest -m benchmark test-int/test_sync_performance_benchmark.py
|
||||
```
|
||||
|
||||
### Performance Benchmarks
|
||||
|
||||
The `test-int/test_sync_performance_benchmark.py` file contains performance benchmarks that measure sync and indexing speed:
|
||||
|
||||
- `test_benchmark_sync_100_files` - Small repository performance
|
||||
- `test_benchmark_sync_500_files` - Medium repository performance
|
||||
- `test_benchmark_sync_1000_files` - Large repository performance (marked slow)
|
||||
- `test_benchmark_resync_no_changes` - Re-sync performance baseline
|
||||
|
||||
Run benchmarks with:
|
||||
```bash
|
||||
# Run all benchmarks (excluding slow ones)
|
||||
pytest test-int/test_sync_performance_benchmark.py -v -m "benchmark and not slow"
|
||||
|
||||
# Run all benchmarks including slow ones
|
||||
pytest test-int/test_sync_performance_benchmark.py -v -m benchmark
|
||||
|
||||
# Run specific benchmark
|
||||
pytest test-int/test_sync_performance_benchmark.py::test_benchmark_sync_100_files -v
|
||||
```
|
||||
|
||||
See `test-int/BENCHMARKS.md` for detailed benchmark documentation.
|
||||
|
||||
### Testing Best Practices
|
||||
|
||||
- **Coverage Target**: We aim for high test coverage for all code
|
||||
- **Test Framework**: Use pytest for unit and integration tests
|
||||
- **Mocking**: Use pytest-mock for mocking dependencies only when necessary
|
||||
- **Mocking**: Avoid mocking in integration tests; use sparingly in unit tests
|
||||
- **Edge Cases**: Test both normal operation and edge cases
|
||||
- **Database Testing**: Use in-memory SQLite for testing database operations
|
||||
- **Fixtures**: Use async pytest fixtures for setup and teardown
|
||||
- **Markers**: Use `@pytest.mark.benchmark` for benchmarks, `@pytest.mark.slow` for slow tests
|
||||
|
||||
## Release Process
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
[](https://www.gnu.org/licenses/agpl-3.0)
|
||||
[](https://badge.fury.io/py/basic-memory)
|
||||
[](https://www.python.org/downloads/)
|
||||
[](https://www.python.org/downloads/)
|
||||
[](https://github.com/basicmachines-co/basic-memory/actions)
|
||||
[](https://github.com/astral-sh/ruff)
|
||||
|
||||
|
||||
@@ -7,23 +7,28 @@ install:
|
||||
@echo ""
|
||||
@echo "💡 Remember to activate the virtual environment by running: source .venv/bin/activate"
|
||||
|
||||
# Run unit tests in parallel
|
||||
# Run unit tests only (fast, no coverage)
|
||||
test-unit:
|
||||
uv run pytest -p pytest_mock -v -n auto
|
||||
uv run pytest -p pytest_mock -v --no-cov -n auto tests
|
||||
|
||||
# Run integration tests in parallel
|
||||
# Run integration tests only (fast, no coverage)
|
||||
test-int:
|
||||
uv run pytest -p pytest_mock -v --no-cov -n auto test-int
|
||||
|
||||
# Run all tests
|
||||
# Run all tests with unified coverage report
|
||||
test: test-unit test-int
|
||||
|
||||
# Generate HTML coverage report
|
||||
coverage:
|
||||
uv run pytest -p pytest_mock -v -n auto tests test-int --cov-report=html
|
||||
@echo "Coverage report generated in htmlcov/index.html"
|
||||
|
||||
# Lint and fix code (calls fix)
|
||||
lint: fix
|
||||
|
||||
# Lint and fix code
|
||||
fix:
|
||||
uv run ruff check --fix --unsafe-fixes src tests
|
||||
uv run ruff check --fix --unsafe-fixes src tests test-int
|
||||
|
||||
# Type check code
|
||||
typecheck:
|
||||
|
||||
+8
-4
@@ -3,7 +3,7 @@ name = "basic-memory"
|
||||
dynamic = ["version"]
|
||||
description = "Local-first knowledge management combining Zettelkasten with knowledge graphs"
|
||||
readme = "README.md"
|
||||
requires-python = ">=3.12.1"
|
||||
requires-python = ">=3.12"
|
||||
license = { text = "AGPL-3.0-or-later" }
|
||||
authors = [
|
||||
{ name = "Basic Machines", email = "hello@basic-machines.co" }
|
||||
@@ -54,16 +54,20 @@ build-backend = "hatchling.build"
|
||||
[tool.pytest.ini_options]
|
||||
pythonpath = ["src", "tests"]
|
||||
addopts = "--cov=basic_memory --cov-report term-missing"
|
||||
testpaths = ["tests"]
|
||||
testpaths = ["tests", "test-int"]
|
||||
asyncio_mode = "strict"
|
||||
asyncio_default_fixture_loop_scope = "function"
|
||||
markers = [
|
||||
"benchmark: Performance benchmark tests (deselect with '-m \"not benchmark\"')",
|
||||
"slow: Slow-running tests (deselect with '-m \"not slow\"')",
|
||||
]
|
||||
|
||||
[tool.ruff]
|
||||
line-length = 100
|
||||
target-version = "py312"
|
||||
|
||||
[tool.uv]
|
||||
dev-dependencies = [
|
||||
[dependency-groups]
|
||||
dev = [
|
||||
"gevent>=24.11.1",
|
||||
"icecream>=2.1.3",
|
||||
"pytest>=8.3.4",
|
||||
|
||||
@@ -11,9 +11,9 @@ DEFAULT_IGNORE_PATTERNS = {
|
||||
# Hidden files (files starting with dot)
|
||||
".*",
|
||||
# Basic Memory internal files
|
||||
"memory.db",
|
||||
"memory.db-shm",
|
||||
"memory.db-wal",
|
||||
"*.db",
|
||||
"*.db-shm",
|
||||
"*.db-wal",
|
||||
"config.json",
|
||||
# Version control
|
||||
".git",
|
||||
@@ -84,10 +84,10 @@ def create_default_bmignore() -> None:
|
||||
# Hidden files (files starting with dot)
|
||||
.*
|
||||
|
||||
# Basic Memory internal files
|
||||
memory.db
|
||||
memory.db-shm
|
||||
memory.db-wal
|
||||
# Basic Memory internal files (includes test databases)
|
||||
*.db
|
||||
*.db-shm
|
||||
*.db-wal
|
||||
config.json
|
||||
|
||||
# Version control
|
||||
|
||||
@@ -559,6 +559,48 @@ class SearchRepository:
|
||||
logger.debug(f"indexed row {search_index_row}")
|
||||
await session.commit()
|
||||
|
||||
async def bulk_index_items(self, search_index_rows: List[SearchIndexRow]):
|
||||
"""Index multiple items in a single batch operation.
|
||||
|
||||
Note: This method assumes that any existing records for the entity_id
|
||||
have already been deleted (typically via delete_by_entity_id).
|
||||
|
||||
Args:
|
||||
search_index_rows: List of SearchIndexRow objects to index
|
||||
"""
|
||||
if not search_index_rows:
|
||||
return
|
||||
|
||||
async with db.scoped_session(self.session_maker) as session:
|
||||
# Prepare all insert data with project_id
|
||||
insert_data_list = []
|
||||
for row in search_index_rows:
|
||||
insert_data = row.to_insert()
|
||||
insert_data["project_id"] = self.project_id
|
||||
insert_data_list.append(insert_data)
|
||||
|
||||
# Batch insert all records using executemany
|
||||
await session.execute(
|
||||
text("""
|
||||
INSERT INTO search_index (
|
||||
id, title, content_stems, content_snippet, permalink, file_path, type, metadata,
|
||||
from_id, to_id, relation_type,
|
||||
entity_id, category,
|
||||
created_at, updated_at,
|
||||
project_id
|
||||
) VALUES (
|
||||
:id, :title, :content_stems, :content_snippet, :permalink, :file_path, :type, :metadata,
|
||||
:from_id, :to_id, :relation_type,
|
||||
:entity_id, :category,
|
||||
:created_at, :updated_at,
|
||||
:project_id
|
||||
)
|
||||
"""),
|
||||
insert_data_list,
|
||||
)
|
||||
logger.debug(f"Bulk indexed {len(search_index_rows)} rows")
|
||||
await session.commit()
|
||||
|
||||
async def delete_by_entity_id(self, entity_id: int):
|
||||
"""Delete an item from the search index by entity_id."""
|
||||
async with db.scoped_session(self.session_maker) as session:
|
||||
|
||||
@@ -52,7 +52,9 @@ class EntityService(BaseService[EntityModel]):
|
||||
self.link_resolver = link_resolver
|
||||
self.app_config = app_config
|
||||
|
||||
async def detect_file_path_conflicts(self, file_path: str) -> List[Entity]:
|
||||
async def detect_file_path_conflicts(
|
||||
self, file_path: str, skip_check: bool = False
|
||||
) -> List[Entity]:
|
||||
"""Detect potential file path conflicts for a given file path.
|
||||
|
||||
This checks for entities with similar file paths that might cause conflicts:
|
||||
@@ -63,10 +65,14 @@ class EntityService(BaseService[EntityModel]):
|
||||
|
||||
Args:
|
||||
file_path: The file path to check for conflicts
|
||||
skip_check: If True, skip the check and return empty list (optimization for bulk operations)
|
||||
|
||||
Returns:
|
||||
List of entities that might conflict with the given file path
|
||||
"""
|
||||
if skip_check:
|
||||
return []
|
||||
|
||||
from basic_memory.utils import detect_potential_file_conflicts
|
||||
|
||||
conflicts = []
|
||||
@@ -86,7 +92,10 @@ class EntityService(BaseService[EntityModel]):
|
||||
return conflicts
|
||||
|
||||
async def resolve_permalink(
|
||||
self, file_path: Permalink | Path, markdown: Optional[EntityMarkdown] = None
|
||||
self,
|
||||
file_path: Permalink | Path,
|
||||
markdown: Optional[EntityMarkdown] = None,
|
||||
skip_conflict_check: bool = False,
|
||||
) -> str:
|
||||
"""Get or generate unique permalink for an entity.
|
||||
|
||||
@@ -101,7 +110,9 @@ class EntityService(BaseService[EntityModel]):
|
||||
file_path_str = Path(file_path).as_posix()
|
||||
|
||||
# Check for potential file path conflicts before resolving permalink
|
||||
conflicts = await self.detect_file_path_conflicts(file_path_str)
|
||||
conflicts = await self.detect_file_path_conflicts(
|
||||
file_path_str, skip_check=skip_conflict_check
|
||||
)
|
||||
if conflicts:
|
||||
logger.warning(
|
||||
f"Detected potential file path conflicts for '{file_path_str}': "
|
||||
@@ -445,6 +456,7 @@ class EntityService(BaseService[EntityModel]):
|
||||
resolved_entities = await asyncio.gather(*lookup_tasks, return_exceptions=True)
|
||||
|
||||
# Process results and create relation records
|
||||
relations_to_add = []
|
||||
for rel, resolved in zip(markdown.relations, resolved_entities):
|
||||
# Handle exceptions from gather and None results
|
||||
target_entity: Optional[Entity] = None
|
||||
@@ -465,14 +477,24 @@ class EntityService(BaseService[EntityModel]):
|
||||
relation_type=rel.type,
|
||||
context=rel.context,
|
||||
)
|
||||
relations_to_add.append(relation)
|
||||
|
||||
# Batch insert all relations
|
||||
if relations_to_add:
|
||||
try:
|
||||
await self.relation_repository.add(relation)
|
||||
await self.relation_repository.add_all(relations_to_add)
|
||||
except IntegrityError:
|
||||
# Unique constraint violation - relation already exists
|
||||
logger.debug(
|
||||
f"Skipping duplicate relation {rel.type} from {db_entity.permalink} target: {rel.target}"
|
||||
)
|
||||
continue
|
||||
# Some relations might be duplicates - fall back to individual inserts
|
||||
logger.debug("Batch relation insert failed, trying individual inserts")
|
||||
for relation in relations_to_add:
|
||||
try:
|
||||
await self.relation_repository.add(relation)
|
||||
except IntegrityError:
|
||||
# Unique constraint violation - relation already exists
|
||||
logger.debug(
|
||||
f"Skipping duplicate relation {relation.relation_type} from {db_entity.permalink}"
|
||||
)
|
||||
continue
|
||||
|
||||
return await self.repository.get_by_file_path(path)
|
||||
|
||||
|
||||
@@ -113,8 +113,10 @@ class SearchService:
|
||||
# Add word boundaries
|
||||
variants.update(w.strip() for w in text.lower().split() if w.strip())
|
||||
|
||||
# Add trigrams for fuzzy matching
|
||||
variants.update(text[i : i + 3].lower() for i in range(len(text) - 2))
|
||||
# Trigrams disabled: They create massive search index bloat, increasing DB size significantly
|
||||
# and slowing down indexing performance. FTS5 search works well without them.
|
||||
# See: https://github.com/basicmachines-co/basic-memory/issues/351
|
||||
# variants.update(text[i : i + 3].lower() for i in range(len(text) - 2))
|
||||
|
||||
return variants
|
||||
|
||||
@@ -219,6 +221,9 @@ class SearchService:
|
||||
The project_id is automatically added by the repository when indexing.
|
||||
"""
|
||||
|
||||
# Collect all search index rows to batch insert at the end
|
||||
rows_to_index = []
|
||||
|
||||
content_stems = []
|
||||
content_snippet = ""
|
||||
title_variants = self._generate_variants(entity.title)
|
||||
@@ -241,8 +246,8 @@ class SearchService:
|
||||
|
||||
entity_content_stems = "\n".join(p for p in content_stems if p and p.strip())
|
||||
|
||||
# Index entity
|
||||
await self.repository.index_item(
|
||||
# Add entity row
|
||||
rows_to_index.append(
|
||||
SearchIndexRow(
|
||||
id=entity.id,
|
||||
type=SearchItemType.ENTITY.value,
|
||||
@@ -261,13 +266,13 @@ class SearchService:
|
||||
)
|
||||
)
|
||||
|
||||
# Index each observation with permalink
|
||||
# Add observation rows
|
||||
for obs in entity.observations:
|
||||
# Index with parent entity's file path since that's where it's defined
|
||||
obs_content_stems = "\n".join(
|
||||
p for p in self._generate_variants(obs.content) if p and p.strip()
|
||||
)
|
||||
await self.repository.index_item(
|
||||
rows_to_index.append(
|
||||
SearchIndexRow(
|
||||
id=obs.id,
|
||||
type=SearchItemType.OBSERVATION.value,
|
||||
@@ -287,7 +292,7 @@ class SearchService:
|
||||
)
|
||||
)
|
||||
|
||||
# Only index outgoing relations (ones defined in this file)
|
||||
# Add relation rows (only outgoing relations defined in this file)
|
||||
for rel in entity.outgoing_relations:
|
||||
# Create descriptive title showing the relationship
|
||||
relation_title = (
|
||||
@@ -299,7 +304,7 @@ class SearchService:
|
||||
rel_content_stems = "\n".join(
|
||||
p for p in self._generate_variants(relation_title) if p and p.strip()
|
||||
)
|
||||
await self.repository.index_item(
|
||||
rows_to_index.append(
|
||||
SearchIndexRow(
|
||||
id=rel.id,
|
||||
title=relation_title,
|
||||
@@ -317,6 +322,9 @@ class SearchService:
|
||||
)
|
||||
)
|
||||
|
||||
# Batch insert all rows at once
|
||||
await self.repository.bulk_index_items(rows_to_index)
|
||||
|
||||
async def delete_by_permalink(self, permalink: str):
|
||||
"""Delete an item from the search index."""
|
||||
await self.repository.delete_by_permalink(permalink)
|
||||
|
||||
@@ -10,6 +10,7 @@ from pathlib import Path
|
||||
from typing import Dict, Optional, Set, Tuple
|
||||
|
||||
from loguru import logger
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.exc import IntegrityError
|
||||
|
||||
from basic_memory import db
|
||||
@@ -274,15 +275,25 @@ class SyncService:
|
||||
|
||||
async def get_db_file_state(self) -> Dict[str, str]:
|
||||
"""Get file_path and checksums from database.
|
||||
Args:
|
||||
db_records: database records
|
||||
|
||||
Optimized to query only the columns we need (file_path, checksum) without
|
||||
loading full entities or their relationships. This is 10-100x faster for
|
||||
large projects compared to loading all entities with observations/relations.
|
||||
|
||||
Returns:
|
||||
Dict mapping file paths to FileState
|
||||
:param db_records: the data from the db
|
||||
Dict mapping file paths to checksums
|
||||
"""
|
||||
db_records = await self.entity_repository.find_all()
|
||||
logger.info(f"Found {len(db_records)} db records")
|
||||
return {r.file_path: r.checksum or "" for r in db_records}
|
||||
# Query only the columns we need - no entity objects or relationships
|
||||
query = select(Entity.file_path, Entity.checksum).where(
|
||||
Entity.project_id == self.entity_repository.project_id
|
||||
)
|
||||
|
||||
async with db.scoped_session(self.entity_repository.session_maker) as session:
|
||||
result = await session.execute(query)
|
||||
rows = result.all()
|
||||
|
||||
logger.info(f"Found {len(rows)} db file records")
|
||||
return {row.file_path: row.checksum or "" for row in rows}
|
||||
|
||||
async def sync_file(
|
||||
self, path: str, new: bool = True
|
||||
@@ -340,8 +351,10 @@ class SyncService:
|
||||
|
||||
# if the file contains frontmatter, resolve a permalink (unless disabled)
|
||||
if file_contains_frontmatter and not self.app_config.disable_permalinks:
|
||||
# Resolve permalink - this handles all the cases including conflicts
|
||||
permalink = await self.entity_service.resolve_permalink(path, markdown=entity_markdown)
|
||||
# Resolve permalink - skip conflict checks during bulk sync for performance
|
||||
permalink = await self.entity_service.resolve_permalink(
|
||||
path, markdown=entity_markdown, skip_conflict_check=True
|
||||
)
|
||||
|
||||
# If permalink changed, update the file
|
||||
if permalink != entity_markdown.frontmatter.permalink:
|
||||
@@ -395,8 +408,8 @@ class SyncService:
|
||||
"""
|
||||
checksum = await self._compute_checksum_async(path)
|
||||
if new:
|
||||
# Generate permalink from path
|
||||
await self.entity_service.resolve_permalink(path)
|
||||
# Generate permalink from path - skip conflict checks during bulk sync
|
||||
await self.entity_service.resolve_permalink(path, skip_conflict_check=True)
|
||||
|
||||
# get file timestamps
|
||||
file_stats = self.file_service.file_stats(path)
|
||||
@@ -535,8 +548,10 @@ class SyncService:
|
||||
and not self.app_config.disable_permalinks
|
||||
and self.file_service.is_markdown(new_path)
|
||||
):
|
||||
# generate new permalink value
|
||||
new_permalink = await self.entity_service.resolve_permalink(new_path)
|
||||
# generate new permalink value - skip conflict checks during bulk sync
|
||||
new_permalink = await self.entity_service.resolve_permalink(
|
||||
new_path, skip_conflict_check=True
|
||||
)
|
||||
|
||||
# write to file and get new checksum
|
||||
new_checksum = await self.file_service.update_frontmatter(
|
||||
|
||||
@@ -0,0 +1,139 @@
|
||||
# Performance Benchmarks
|
||||
|
||||
This directory contains performance benchmark tests for Basic Memory's sync/indexing operations.
|
||||
|
||||
## Purpose
|
||||
|
||||
These benchmarks measure baseline performance to track improvements from optimizations. They are particularly important for:
|
||||
- Cloud deployments with ephemeral databases that need fast re-indexing
|
||||
- Large repositories (100s to 1000s of files)
|
||||
- Validating optimization efforts
|
||||
|
||||
## Running Benchmarks
|
||||
|
||||
### Run all benchmarks (excluding slow ones)
|
||||
```bash
|
||||
pytest test-int/test_sync_performance_benchmark.py -v -m "benchmark and not slow"
|
||||
```
|
||||
|
||||
### Run specific benchmark
|
||||
```bash
|
||||
# 100 files (fast, ~10-30 seconds)
|
||||
pytest test-int/test_sync_performance_benchmark.py::test_benchmark_sync_100_files -v
|
||||
|
||||
# 500 files (medium, ~1-3 minutes)
|
||||
pytest test-int/test_sync_performance_benchmark.py::test_benchmark_sync_500_files -v
|
||||
|
||||
# 1000 files (slow, ~3-10 minutes)
|
||||
pytest test-int/test_sync_performance_benchmark.py::test_benchmark_sync_1000_files -v
|
||||
|
||||
# Re-sync with no changes (tests scan performance)
|
||||
pytest test-int/test_sync_performance_benchmark.py::test_benchmark_resync_no_changes -v
|
||||
```
|
||||
|
||||
### Run all benchmarks including slow ones
|
||||
```bash
|
||||
pytest test-int/test_sync_performance_benchmark.py -v -m benchmark
|
||||
```
|
||||
|
||||
### Skip benchmarks in regular test runs
|
||||
```bash
|
||||
pytest -m "not benchmark"
|
||||
```
|
||||
|
||||
## Benchmark Output
|
||||
|
||||
Each benchmark provides detailed metrics including:
|
||||
|
||||
- **Performance Metrics**:
|
||||
- Total sync time
|
||||
- Files processed per second
|
||||
- Milliseconds per file
|
||||
|
||||
- **Database Metrics**:
|
||||
- Initial database size
|
||||
- Final database size
|
||||
- Database growth (total and per file)
|
||||
|
||||
- **Operation Counts**:
|
||||
- New files indexed
|
||||
- Modified files processed
|
||||
- Deleted files handled
|
||||
- Moved files tracked
|
||||
|
||||
## Example Output
|
||||
|
||||
```
|
||||
======================================================================
|
||||
BENCHMARK: Sync 100 files (small repository)
|
||||
======================================================================
|
||||
|
||||
Generating 100 test files...
|
||||
Created files 0-100 (100/100)
|
||||
File generation completed in 0.15s (666.7 files/sec)
|
||||
|
||||
Initial database size: 120.00 KB
|
||||
|
||||
Starting sync of 100 files...
|
||||
|
||||
----------------------------------------------------------------------
|
||||
RESULTS:
|
||||
----------------------------------------------------------------------
|
||||
Files processed: 100
|
||||
New: 100
|
||||
Modified: 0
|
||||
Deleted: 0
|
||||
Moved: 0
|
||||
|
||||
Performance:
|
||||
Total time: 12.34s
|
||||
Files/sec: 8.1
|
||||
ms/file: 123.4
|
||||
|
||||
Database:
|
||||
Initial size: 120.00 KB
|
||||
Final size: 5.23 MB
|
||||
Growth: 5.11 MB
|
||||
Growth per file: 52.31 KB
|
||||
======================================================================
|
||||
```
|
||||
|
||||
## Interpreting Results
|
||||
|
||||
### Good Performance Indicators
|
||||
- **Files/sec > 10**: Good indexing speed for small-medium repos
|
||||
- **Files/sec > 5**: Acceptable for large repos with complex relations
|
||||
- **DB growth < 100KB per file**: Reasonable index size
|
||||
|
||||
### Areas for Improvement
|
||||
- **Files/sec < 5**: May benefit from batch operations
|
||||
- **ms/file > 200**: High latency per file, check for N+1 queries
|
||||
- **DB growth > 200KB per file**: Search index may be bloated (trigrams?)
|
||||
|
||||
## Tracking Improvements
|
||||
|
||||
Before making optimizations:
|
||||
1. Run benchmarks to establish baseline
|
||||
2. Save output for comparison
|
||||
3. Note any particular pain points (e.g., slow search indexing)
|
||||
|
||||
After optimizations:
|
||||
1. Run the same benchmarks
|
||||
2. Compare metrics:
|
||||
- Files/sec should increase
|
||||
- ms/file should decrease
|
||||
- DB growth per file may decrease (with search optimizations)
|
||||
3. Document improvements in PR
|
||||
|
||||
## Related Issues
|
||||
|
||||
- [#351: Performance: Optimize sync/indexing for cloud deployments](https://github.com/basicmachines-co/basic-memory/issues/351)
|
||||
|
||||
## Test File Generation
|
||||
|
||||
Benchmarks generate realistic markdown files with:
|
||||
- YAML frontmatter with tags
|
||||
- 3-10 observations per file with categories
|
||||
- 1-3 relations per file (including forward references)
|
||||
- Varying content to simulate real usage
|
||||
- Files organized in category subdirectories
|
||||
@@ -0,0 +1,369 @@
|
||||
"""
|
||||
Performance benchmark tests for sync operations.
|
||||
|
||||
These tests measure baseline performance for indexing operations to track
|
||||
improvements from optimizations. Tests are marked with @pytest.mark.benchmark
|
||||
and can be run separately.
|
||||
|
||||
Usage:
|
||||
# Run all benchmarks
|
||||
pytest test-int/test_sync_performance_benchmark.py -v
|
||||
|
||||
# Run specific benchmark
|
||||
pytest test-int/test_sync_performance_benchmark.py::test_benchmark_sync_100_files -v
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import time
|
||||
from pathlib import Path
|
||||
from textwrap import dedent
|
||||
|
||||
import pytest
|
||||
|
||||
from basic_memory.config import BasicMemoryConfig, ProjectConfig
|
||||
from basic_memory.sync.sync_service import get_sync_service
|
||||
|
||||
|
||||
async def create_benchmark_file(path: Path, file_num: int, total_files: int) -> None:
|
||||
"""Create a realistic test markdown file with observations and relations.
|
||||
|
||||
Args:
|
||||
path: Path to create the file at
|
||||
file_num: Current file number (for unique content)
|
||||
total_files: Total number of files being created (for relation targets)
|
||||
"""
|
||||
# Create realistic content with varying complexity
|
||||
has_relations = file_num < (total_files - 1) # Most files have relations
|
||||
num_observations = min(3 + (file_num % 5), 10) # 3-10 observations per file
|
||||
|
||||
# Generate relation targets (some will be forward references)
|
||||
relations = []
|
||||
if has_relations:
|
||||
# Reference 1-3 other files
|
||||
num_relations = min(1 + (file_num % 3), 3)
|
||||
for i in range(num_relations):
|
||||
target_num = (file_num + i + 1) % total_files
|
||||
relations.append(f"- relates_to [[test-file-{target_num:04d}]]")
|
||||
|
||||
content = dedent(f"""
|
||||
---
|
||||
type: note
|
||||
tags: [benchmark, test, category-{file_num % 10}]
|
||||
---
|
||||
# Test File {file_num:04d}
|
||||
|
||||
This is benchmark test file {file_num} of {total_files}.
|
||||
It contains realistic markdown content to simulate actual usage.
|
||||
|
||||
## Observations
|
||||
{chr(10).join([f"- [category-{i % 5}] Observation {i} for file {file_num} with some content #tag{i}" for i in range(num_observations)])}
|
||||
|
||||
## Relations
|
||||
{chr(10).join(relations) if relations else "- No relations for this file"}
|
||||
|
||||
## Additional Content
|
||||
|
||||
This section contains additional prose to simulate real documents.
|
||||
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed do eiusmod
|
||||
tempor incididunt ut labore et dolore magna aliqua.
|
||||
|
||||
### Subsection
|
||||
|
||||
More content here to make the file realistic. This helps test the
|
||||
full indexing pipeline including content extraction and search indexing.
|
||||
""").strip()
|
||||
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
path.write_text(content, encoding="utf-8")
|
||||
|
||||
|
||||
async def generate_benchmark_files(project_dir: Path, num_files: int) -> None:
|
||||
"""Generate benchmark test files.
|
||||
|
||||
Args:
|
||||
project_dir: Directory to create files in
|
||||
num_files: Number of files to generate
|
||||
"""
|
||||
print(f"\nGenerating {num_files} test files...")
|
||||
start = time.time()
|
||||
|
||||
# Create files in batches for faster generation
|
||||
batch_size = 100
|
||||
for batch_start in range(0, num_files, batch_size):
|
||||
batch_end = min(batch_start + batch_size, num_files)
|
||||
tasks = [
|
||||
create_benchmark_file(
|
||||
project_dir / f"category-{i % 10}" / f"test-file-{i:04d}.md", i, num_files
|
||||
)
|
||||
for i in range(batch_start, batch_end)
|
||||
]
|
||||
await asyncio.gather(*tasks)
|
||||
print(f" Created files {batch_start}-{batch_end} ({batch_end}/{num_files})")
|
||||
|
||||
duration = time.time() - start
|
||||
print(f" File generation completed in {duration:.2f}s ({num_files / duration:.1f} files/sec)")
|
||||
|
||||
|
||||
def get_db_size(db_path: Path) -> tuple[int, str]:
|
||||
"""Get database file size.
|
||||
|
||||
Returns:
|
||||
Tuple of (size_bytes, formatted_size)
|
||||
"""
|
||||
if not db_path.exists():
|
||||
return 0, "0 B"
|
||||
|
||||
size_bytes = db_path.stat().st_size
|
||||
|
||||
# Format size
|
||||
for unit in ["B", "KB", "MB", "GB"]:
|
||||
if size_bytes < 1024.0:
|
||||
return size_bytes, f"{size_bytes:.2f} {unit}"
|
||||
size_bytes /= 1024.0
|
||||
|
||||
return int(size_bytes * 1024**4), f"{size_bytes:.2f} TB"
|
||||
|
||||
|
||||
async def run_sync_benchmark(
|
||||
project_config: ProjectConfig, app_config: BasicMemoryConfig, num_files: int, test_name: str
|
||||
) -> dict:
|
||||
"""Run a sync benchmark and collect metrics.
|
||||
|
||||
Args:
|
||||
project_config: Project configuration
|
||||
app_config: App configuration
|
||||
num_files: Number of files to benchmark
|
||||
test_name: Name of the test for reporting
|
||||
|
||||
Returns:
|
||||
Dictionary with benchmark results
|
||||
"""
|
||||
project_dir = project_config.home
|
||||
db_path = app_config.database_path
|
||||
|
||||
print(f"\n{'=' * 70}")
|
||||
print(f"BENCHMARK: {test_name}")
|
||||
print(f"{'=' * 70}")
|
||||
|
||||
# Generate test files
|
||||
await generate_benchmark_files(project_dir, num_files)
|
||||
|
||||
# Get initial DB size
|
||||
initial_db_size, initial_db_formatted = get_db_size(db_path)
|
||||
print(f"\nInitial database size: {initial_db_formatted}")
|
||||
|
||||
# Create sync service
|
||||
from basic_memory.repository import ProjectRepository
|
||||
from basic_memory import db
|
||||
|
||||
_, session_maker = await db.get_or_create_db(
|
||||
db_path=app_config.database_path,
|
||||
db_type=db.DatabaseType.FILESYSTEM,
|
||||
)
|
||||
project_repository = ProjectRepository(session_maker)
|
||||
|
||||
# Get or create project
|
||||
projects = await project_repository.find_all()
|
||||
if projects:
|
||||
project = projects[0]
|
||||
else:
|
||||
project = await project_repository.create(
|
||||
{
|
||||
"name": project_config.name,
|
||||
"path": str(project_config.home),
|
||||
"is_active": True,
|
||||
"is_default": True,
|
||||
}
|
||||
)
|
||||
|
||||
sync_service = await get_sync_service(project)
|
||||
|
||||
# Initialize search index (required for FTS5 table)
|
||||
await sync_service.search_service.init_search_index()
|
||||
|
||||
# Run sync and measure time
|
||||
print(f"\nStarting sync of {num_files} files...")
|
||||
sync_start = time.time()
|
||||
|
||||
report = await sync_service.sync(project_dir, project_name=project.name)
|
||||
|
||||
sync_duration = time.time() - sync_start
|
||||
|
||||
# Get final DB size
|
||||
final_db_size, final_db_formatted = get_db_size(db_path)
|
||||
db_growth = final_db_size - initial_db_size
|
||||
db_growth_formatted = f"{db_growth / 1024 / 1024:.2f} MB"
|
||||
|
||||
# Calculate metrics
|
||||
files_per_sec = num_files / sync_duration if sync_duration > 0 else 0
|
||||
ms_per_file = (sync_duration * 1000) / num_files if num_files > 0 else 0
|
||||
|
||||
# Print results
|
||||
print(f"\n{'-' * 70}")
|
||||
print("RESULTS:")
|
||||
print(f"{'-' * 70}")
|
||||
print(f"Files processed: {num_files}")
|
||||
print(f" New: {len(report.new)}")
|
||||
print(f" Modified: {len(report.modified)}")
|
||||
print(f" Deleted: {len(report.deleted)}")
|
||||
print(f" Moved: {len(report.moves)}")
|
||||
print("\nPerformance:")
|
||||
print(f" Total time: {sync_duration:.2f}s")
|
||||
print(f" Files/sec: {files_per_sec:.1f}")
|
||||
print(f" ms/file: {ms_per_file:.1f}")
|
||||
print("\nDatabase:")
|
||||
print(f" Initial size: {initial_db_formatted}")
|
||||
print(f" Final size: {final_db_formatted}")
|
||||
print(f" Growth: {db_growth_formatted}")
|
||||
print(f" Growth per file: {(db_growth / num_files / 1024):.2f} KB")
|
||||
print(f"{'=' * 70}\n")
|
||||
|
||||
return {
|
||||
"test_name": test_name,
|
||||
"num_files": num_files,
|
||||
"sync_duration_sec": sync_duration,
|
||||
"files_per_sec": files_per_sec,
|
||||
"ms_per_file": ms_per_file,
|
||||
"new_files": len(report.new),
|
||||
"modified_files": len(report.modified),
|
||||
"deleted_files": len(report.deleted),
|
||||
"moved_files": len(report.moves),
|
||||
"initial_db_size": initial_db_size,
|
||||
"final_db_size": final_db_size,
|
||||
"db_growth_bytes": db_growth,
|
||||
"db_growth_per_file_bytes": db_growth / num_files if num_files > 0 else 0,
|
||||
}
|
||||
|
||||
|
||||
@pytest.mark.benchmark
|
||||
@pytest.mark.asyncio
|
||||
async def test_benchmark_sync_100_files(app_config, project_config, config_manager):
|
||||
"""Benchmark: Sync 100 files (small repository)."""
|
||||
results = await run_sync_benchmark(
|
||||
project_config, app_config, num_files=100, test_name="Sync 100 files (small repository)"
|
||||
)
|
||||
|
||||
# Basic assertions to ensure sync worked
|
||||
# Note: May be slightly more than 100 due to OS-generated files (.DS_Store, etc.)
|
||||
assert results["new_files"] >= 100
|
||||
assert results["sync_duration_sec"] > 0
|
||||
assert results["files_per_sec"] > 0
|
||||
|
||||
|
||||
@pytest.mark.benchmark
|
||||
@pytest.mark.asyncio
|
||||
async def test_benchmark_sync_500_files(app_config, project_config, config_manager):
|
||||
"""Benchmark: Sync 500 files (medium repository)."""
|
||||
results = await run_sync_benchmark(
|
||||
project_config, app_config, num_files=500, test_name="Sync 500 files (medium repository)"
|
||||
)
|
||||
|
||||
# Basic assertions
|
||||
# Note: May be slightly more than 500 due to OS-generated files
|
||||
assert results["new_files"] >= 500
|
||||
assert results["sync_duration_sec"] > 0
|
||||
assert results["files_per_sec"] > 0
|
||||
|
||||
|
||||
@pytest.mark.benchmark
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.slow
|
||||
async def test_benchmark_sync_1000_files(app_config, project_config, config_manager):
|
||||
"""Benchmark: Sync 1000 files (large repository).
|
||||
|
||||
This test is marked as 'slow' and can be skipped in regular test runs:
|
||||
pytest -m "not slow"
|
||||
"""
|
||||
results = await run_sync_benchmark(
|
||||
project_config, app_config, num_files=1000, test_name="Sync 1000 files (large repository)"
|
||||
)
|
||||
|
||||
# Basic assertions
|
||||
# Note: May be slightly more than 1000 due to OS-generated files
|
||||
assert results["new_files"] >= 1000
|
||||
assert results["sync_duration_sec"] > 0
|
||||
assert results["files_per_sec"] > 0
|
||||
|
||||
|
||||
@pytest.mark.benchmark
|
||||
@pytest.mark.asyncio
|
||||
async def test_benchmark_resync_no_changes(app_config, project_config, config_manager):
|
||||
"""Benchmark: Re-sync with no changes (should be fast).
|
||||
|
||||
This tests the performance of scanning files when nothing has changed,
|
||||
which is important for cloud restarts.
|
||||
"""
|
||||
project_dir = project_config.home
|
||||
num_files = 100
|
||||
|
||||
# First sync
|
||||
print(f"\nFirst sync of {num_files} files...")
|
||||
await generate_benchmark_files(project_dir, num_files)
|
||||
|
||||
from basic_memory.repository import ProjectRepository
|
||||
from basic_memory import db
|
||||
|
||||
_, session_maker = await db.get_or_create_db(
|
||||
db_path=app_config.database_path,
|
||||
db_type=db.DatabaseType.FILESYSTEM,
|
||||
)
|
||||
project_repository = ProjectRepository(session_maker)
|
||||
projects = await project_repository.find_all()
|
||||
if projects:
|
||||
project = projects[0]
|
||||
else:
|
||||
project = await project_repository.create(
|
||||
{
|
||||
"name": project_config.name,
|
||||
"path": str(project_config.home),
|
||||
"is_active": True,
|
||||
"is_default": True,
|
||||
}
|
||||
)
|
||||
|
||||
sync_service = await get_sync_service(project)
|
||||
|
||||
# Initialize search index
|
||||
await sync_service.search_service.init_search_index()
|
||||
|
||||
await sync_service.sync(project_dir, project_name=project.name)
|
||||
|
||||
# Second sync (no changes)
|
||||
print("\nRe-sync with no changes...")
|
||||
resync_start = time.time()
|
||||
report = await sync_service.sync(project_dir, project_name=project.name)
|
||||
resync_duration = time.time() - resync_start
|
||||
|
||||
print(f"\n{'-' * 70}")
|
||||
print("RE-SYNC RESULTS (no changes):")
|
||||
print(f"{'-' * 70}")
|
||||
print(f"Files scanned: {num_files}")
|
||||
print(f"Changes detected: {report.total}")
|
||||
print(f" New: {len(report.new)}")
|
||||
print(f" Modified: {len(report.modified)}")
|
||||
print(f" Deleted: {len(report.deleted)}")
|
||||
print(f" Moved: {len(report.moves)}")
|
||||
print(f"Duration: {resync_duration:.2f}s")
|
||||
print(f"Files/sec: {num_files / resync_duration:.1f}")
|
||||
|
||||
# Debug: Show what changed
|
||||
if report.total > 0:
|
||||
print("\n⚠️ UNEXPECTED CHANGES DETECTED:")
|
||||
if report.new:
|
||||
print(f" New files ({len(report.new)}): {list(report.new)[:5]}")
|
||||
if report.modified:
|
||||
print(f" Modified files ({len(report.modified)}): {list(report.modified)[:5]}")
|
||||
if report.deleted:
|
||||
print(f" Deleted files ({len(report.deleted)}): {list(report.deleted)[:5]}")
|
||||
if report.moves:
|
||||
print(f" Moved files ({len(report.moves)}): {dict(list(report.moves.items())[:5])}")
|
||||
|
||||
print(f"{'=' * 70}\n")
|
||||
|
||||
# Should be no changes
|
||||
assert report.total == 0, (
|
||||
f"Expected no changes but got {report.total}: new={len(report.new)}, modified={len(report.modified)}, deleted={len(report.deleted)}, moves={len(report.moves)}"
|
||||
)
|
||||
assert len(report.new) == 0
|
||||
assert len(report.modified) == 0
|
||||
assert len(report.deleted) == 0
|
||||
Reference in New Issue
Block a user