mirror of
https://github.com/basicmachines-co/basic-memory
synced 2026-06-21 13:47:35 +00:00
Compare commits
30 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 24dc9a2931 | |||
| 85c63e5a7a | |||
| f227ef6a86 | |||
| 897b1edaa4 | |||
| 0c12a39a98 | |||
| efbc758325 | |||
| a0f20eb102 | |||
| 78673d8e51 | |||
| 126c0495c0 | |||
| 4a43d7df4a | |||
| c462faf046 | |||
| 70bb10be1d | |||
| fbf9045d78 | |||
| 1094210c52 | |||
| 391feb639f | |||
| a920a9ff29 | |||
| 05efe8701c | |||
| 0eaf30bb06 | |||
| 0818bda565 | |||
| 6f99d2e551 | |||
| 73d940e064 | |||
| c3678a11d2 | |||
| 203d684c24 | |||
| a872220924 | |||
| 7d763a66ff | |||
| b5d4fb559c | |||
| 830775276d | |||
| ed894fc3ed | |||
| 704338edcf | |||
| 0ca02a7ebe |
@@ -0,0 +1,5 @@
|
||||
{
|
||||
"enabledPlugins": {
|
||||
"basic-memory@basicmachines": true
|
||||
}
|
||||
}
|
||||
@@ -78,21 +78,7 @@ jobs:
|
||||
python-version: [ "3.12", "3.13" ]
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
# Postgres service (only available on Linux runners)
|
||||
services:
|
||||
postgres:
|
||||
image: postgres:17
|
||||
env:
|
||||
POSTGRES_DB: basic_memory_test
|
||||
POSTGRES_USER: basic_memory_user
|
||||
POSTGRES_PASSWORD: dev_password
|
||||
options: >-
|
||||
--health-cmd pg_isready
|
||||
--health-interval 10s
|
||||
--health-timeout 5s
|
||||
--health-retries 5
|
||||
ports:
|
||||
- 5433:5432
|
||||
# Note: No services section needed - testcontainers handles Postgres in Docker
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
@@ -121,7 +107,7 @@ jobs:
|
||||
run: |
|
||||
uv pip install -e .[dev]
|
||||
|
||||
- name: Run tests (Postgres)
|
||||
- name: Run tests (Postgres via testcontainers)
|
||||
run: |
|
||||
uv pip install pytest pytest-cov
|
||||
just test-postgres
|
||||
@@ -1,5 +1,98 @@
|
||||
# CHANGELOG
|
||||
|
||||
## v0.16.3 (2025-12-20)
|
||||
|
||||
### Features
|
||||
|
||||
- **#439**: Add PostgreSQL database backend support
|
||||
([`fb5e9e1`](https://github.com/basicmachines-co/basic-memory/commit/fb5e9e1))
|
||||
- Full PostgreSQL/Neon database support as alternative to SQLite
|
||||
- Async connection pooling with asyncpg
|
||||
- Alembic migrations support for both backends
|
||||
- Configurable via `BASIC_MEMORY_DATABASE_BACKEND` environment variable
|
||||
|
||||
- **#441**: Implement API v2 with ID-based endpoints (Phase 1)
|
||||
([`28cc522`](https://github.com/basicmachines-co/basic-memory/commit/28cc522))
|
||||
- New ID-based API endpoints for improved performance
|
||||
- Foundation for future API enhancements
|
||||
- Backward compatible with existing endpoints
|
||||
|
||||
- Add project_id to Relation and Observation for efficient project-scoped queries
|
||||
([`a920a9f`](https://github.com/basicmachines-co/basic-memory/commit/a920a9f))
|
||||
- Enables faster queries in multi-project environments
|
||||
- Improved database schema for cloud deployments
|
||||
|
||||
- Add bulk insert with ON CONFLICT handling for relations
|
||||
([`0818bda`](https://github.com/basicmachines-co/basic-memory/commit/0818bda))
|
||||
- Faster relation creation during sync operations
|
||||
- Handles duplicate relations gracefully
|
||||
|
||||
### Performance
|
||||
|
||||
- Lightweight permalink resolution to avoid eager loading
|
||||
([`6f99d2e`](https://github.com/basicmachines-co/basic-memory/commit/6f99d2e))
|
||||
- Reduces database queries during entity lookups
|
||||
- Improved response times for read operations
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
- **#464**: Pin FastMCP to 2.12.3 to fix MCP tools visibility
|
||||
([`f227ef6`](https://github.com/basicmachines-co/basic-memory/commit/f227ef6))
|
||||
- Fixes issue where MCP tools were not visible to Claude
|
||||
- Reverts to last known working FastMCP version
|
||||
|
||||
- **#458**: Reduce watch service CPU usage by increasing reload interval
|
||||
([`897b1ed`](https://github.com/basicmachines-co/basic-memory/commit/897b1ed))
|
||||
- Lowers CPU usage during file watching
|
||||
- More efficient resource utilization
|
||||
|
||||
- **#456**: Await background sync task cancellation in lifespan shutdown
|
||||
([`efbc758`](https://github.com/basicmachines-co/basic-memory/commit/efbc758))
|
||||
- Prevents hanging on shutdown
|
||||
- Clean async task cleanup
|
||||
|
||||
- **#434**: Respect --project flag in background sync
|
||||
([`70bb10b`](https://github.com/basicmachines-co/basic-memory/commit/70bb10b))
|
||||
- Background sync now correctly uses specified project
|
||||
- Fixes multi-project sync issues
|
||||
|
||||
- **#446**: Fix observation parsing and permalink limits
|
||||
([`73d940e`](https://github.com/basicmachines-co/basic-memory/commit/73d940e))
|
||||
- Handles edge cases in observation content
|
||||
- Prevents permalink truncation issues
|
||||
|
||||
- **#424**: Handle periods in kebab_filenames mode
|
||||
([`b004565`](https://github.com/basicmachines-co/basic-memory/commit/b004565))
|
||||
- Fixes filename handling for files with multiple periods
|
||||
- Improved kebab-case conversion
|
||||
|
||||
- Fix Postgres/Neon connection settings and search index dedupe
|
||||
([`b5d4fb5`](https://github.com/basicmachines-co/basic-memory/commit/b5d4fb5))
|
||||
- Optimized connection pooling for Postgres
|
||||
- Prevents duplicate search index entries
|
||||
|
||||
### Testing & CI
|
||||
|
||||
- Replace py-pglite with testcontainers for Postgres testing
|
||||
([`c462faf`](https://github.com/basicmachines-co/basic-memory/commit/c462faf))
|
||||
- More reliable Postgres testing infrastructure
|
||||
- Uses Docker-based test containers
|
||||
|
||||
- Add PostgreSQL testing to GitHub Actions workflow
|
||||
([`66b91b2`](https://github.com/basicmachines-co/basic-memory/commit/66b91b2))
|
||||
- CI now tests both SQLite and PostgreSQL backends
|
||||
- Ensures cross-database compatibility
|
||||
|
||||
- **#416**: Add integration test for read_note with underscored folders
|
||||
([`0c12a39`](https://github.com/basicmachines-co/basic-memory/commit/0c12a39))
|
||||
- Verifies folder name handling edge cases
|
||||
|
||||
### Internal
|
||||
|
||||
- Cloud compatibility fixes and performance improvements (#454)
|
||||
- Remove logfire instrumentation for cleaner production deployments
|
||||
- Truncate content_stems to fix Postgres 8KB index row limit
|
||||
|
||||
## v0.16.2 (2025-11-16)
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
@@ -15,10 +15,14 @@ See the [README.md](README.md) file for a project overview.
|
||||
### Build and Test Commands
|
||||
|
||||
- Install: `just install` or `pip install -e ".[dev]"`
|
||||
- Run all tests (with coverage): `just test` - Runs both unit and integration tests with unified coverage
|
||||
- Run unit tests only: `just test-unit` - Fast, no coverage
|
||||
- Run integration tests only: `just test-int` - Fast, no coverage
|
||||
- Generate HTML coverage: `just coverage` - Opens in browser
|
||||
- Run all tests (SQLite + Postgres): `just test`
|
||||
- Run all tests against SQLite: `just test-sqlite`
|
||||
- Run all tests against Postgres: `just test-postgres` (uses testcontainers)
|
||||
- Run unit tests (SQLite): `just test-unit-sqlite`
|
||||
- Run unit tests (Postgres): `just test-unit-postgres`
|
||||
- Run integration tests (SQLite): `just test-int-sqlite`
|
||||
- Run integration tests (Postgres): `just test-int-postgres`
|
||||
- Generate HTML coverage: `just coverage`
|
||||
- Single test: `pytest tests/path/to/test_file.py::test_function_name`
|
||||
- Run benchmarks: `pytest test-int/test_sync_performance_benchmark.py -v -m "benchmark and not slow"`
|
||||
- Lint: `just lint` or `ruff check . --fix`
|
||||
@@ -30,6 +34,8 @@ See the [README.md](README.md) file for a project overview.
|
||||
|
||||
**Note:** Project requires Python 3.12+ (uses type parameter syntax and `type` aliases introduced in 3.12)
|
||||
|
||||
**Postgres Testing:** Uses [testcontainers](https://testcontainers-python.readthedocs.io/) which automatically spins up a Postgres instance in Docker. No manual database setup required - just have Docker running.
|
||||
|
||||
### Test Structure
|
||||
|
||||
- `tests/` - Unit tests for individual components (mocked, fast)
|
||||
@@ -76,8 +82,10 @@ See the [README.md](README.md) file for a project overview.
|
||||
- SQLite is used for indexing and full text search, files are source of truth
|
||||
- Testing uses pytest with asyncio support (strict mode)
|
||||
- Unit tests (`tests/`) use mocks when necessary; integration tests (`test-int/`) use real implementations
|
||||
- Test database uses in-memory SQLite
|
||||
- Each test runs in a standalone environment with in-memory SQLite and tmp_file directory
|
||||
- By default, tests run against SQLite (fast, no Docker needed)
|
||||
- Set `BASIC_MEMORY_TEST_POSTGRES=1` to run against Postgres (uses testcontainers - Docker required)
|
||||
- Each test runs in a standalone environment with isolated database and tmp_path directory
|
||||
- CI runs SQLite and Postgres tests in parallel for faster feedback
|
||||
- Performance benchmarks are in `test-int/test_sync_performance_benchmark.py`
|
||||
- Use pytest markers: `@pytest.mark.benchmark` for benchmarks, `@pytest.mark.slow` for slow tests
|
||||
|
||||
@@ -229,6 +237,11 @@ of using AI just for code generation, we've developed a true collaborative workf
|
||||
This approach has allowed us to tackle more complex challenges and build a more robust system than either humans or AI
|
||||
could achieve independently.
|
||||
|
||||
**Problem-Solving Guidance:**
|
||||
- If a solution isn't working after reasonable effort, suggest alternative approaches
|
||||
- Don't persist with a problematic library or pattern when better alternatives exist
|
||||
- Example: When py-pglite caused cascading test failures, switching to testcontainers-postgres was the right call
|
||||
|
||||
## GitHub Integration
|
||||
|
||||
Basic Memory has taken AI-Human collaboration to the next level by integrating Claude directly into the development workflow through GitHub:
|
||||
|
||||
@@ -433,42 +433,76 @@ See the [Documentation](https://memory.basicmachines.co/) for more info, includi
|
||||
- [Managing multiple Projects](https://docs.basicmemory.com/guides/cli-reference/#project)
|
||||
- [Importing data from OpenAI/Claude Projects](https://docs.basicmemory.com/guides/cli-reference/#import)
|
||||
|
||||
## Logging
|
||||
|
||||
Basic Memory uses [Loguru](https://github.com/Delgan/loguru) for logging. The logging behavior varies by entry point:
|
||||
|
||||
| Entry Point | Default Behavior | Use Case |
|
||||
|-------------|------------------|----------|
|
||||
| CLI commands | File only | Prevents log output from interfering with command output |
|
||||
| MCP server | File only | Stdout would corrupt the JSON-RPC protocol |
|
||||
| API server | File (local) or stdout (cloud) | Docker/cloud deployments use stdout |
|
||||
|
||||
**Log file location:** `~/.basic-memory/basic-memory.log` (10MB rotation, 10 days retention)
|
||||
|
||||
### Environment Variables
|
||||
|
||||
| Variable | Default | Description |
|
||||
|----------|---------|-------------|
|
||||
| `BASIC_MEMORY_LOG_LEVEL` | `INFO` | Log level: DEBUG, INFO, WARNING, ERROR |
|
||||
| `BASIC_MEMORY_CLOUD_MODE` | `false` | When `true`, API logs to stdout with structured context |
|
||||
| `BASIC_MEMORY_ENV` | `dev` | Set to `test` for test mode (stderr only) |
|
||||
|
||||
### Examples
|
||||
|
||||
```bash
|
||||
# Enable debug logging
|
||||
BASIC_MEMORY_LOG_LEVEL=DEBUG basic-memory sync
|
||||
|
||||
# View logs
|
||||
tail -f ~/.basic-memory/basic-memory.log
|
||||
|
||||
# Cloud/Docker mode (stdout logging with structured context)
|
||||
BASIC_MEMORY_CLOUD_MODE=true uvicorn basic_memory.api.app:app
|
||||
```
|
||||
|
||||
## Development
|
||||
|
||||
### Running Tests
|
||||
|
||||
Basic Memory supports dual database backends (SQLite and Postgres). Tests are parametrized to run against both backends automatically.
|
||||
Basic Memory supports dual database backends (SQLite and Postgres). By default, tests run against SQLite. Set `BASIC_MEMORY_TEST_POSTGRES=1` to run against Postgres (uses testcontainers - Docker required).
|
||||
|
||||
**Quick Start:**
|
||||
```bash
|
||||
# Run SQLite tests (default, no Docker needed)
|
||||
# Run all tests against SQLite (default, fast)
|
||||
just test-sqlite
|
||||
|
||||
# Run Postgres tests (requires Docker)
|
||||
# Run all tests against Postgres (uses testcontainers)
|
||||
just test-postgres
|
||||
|
||||
# Run both SQLite and Postgres tests
|
||||
just test
|
||||
```
|
||||
|
||||
**Available Test Commands:**
|
||||
|
||||
- `just test-sqlite` - Run tests against SQLite only (fastest, no Docker needed)
|
||||
- `just test-postgres` - Run tests against Postgres only (requires Docker)
|
||||
- `just test` - Run all tests against both SQLite and Postgres
|
||||
- `just test-sqlite` - Run all tests against SQLite (fast, no Docker needed)
|
||||
- `just test-postgres` - Run all tests against Postgres (uses testcontainers)
|
||||
- `just test-unit-sqlite` - Run unit tests against SQLite
|
||||
- `just test-unit-postgres` - Run unit tests against Postgres
|
||||
- `just test-int-sqlite` - Run integration tests against SQLite
|
||||
- `just test-int-postgres` - Run integration tests against Postgres
|
||||
- `just test-windows` - Run Windows-specific tests (auto-skips on other platforms)
|
||||
- `just test-benchmark` - Run performance benchmark tests
|
||||
- `just test-all` - Run all tests including Windows, Postgres, and benchmarks
|
||||
|
||||
**Postgres Testing Requirements:**
|
||||
**Postgres Testing:**
|
||||
|
||||
To run Postgres tests, you need to start the test database:
|
||||
```bash
|
||||
docker-compose -f docker-compose-postgres.yml up -d
|
||||
```
|
||||
|
||||
Tests will connect to `localhost:5433/basic_memory_test`.
|
||||
Postgres tests use [testcontainers](https://testcontainers-python.readthedocs.io/) which automatically spins up a Postgres instance in Docker. No manual database setup required - just have Docker running.
|
||||
|
||||
**Test Markers:**
|
||||
|
||||
Tests use pytest markers for selective execution:
|
||||
- `postgres` - Tests that run against Postgres backend
|
||||
- `windows` - Windows-specific database optimizations
|
||||
- `benchmark` - Performance tests (excluded from default runs)
|
||||
|
||||
|
||||
@@ -7,44 +7,51 @@ install:
|
||||
@echo ""
|
||||
@echo "💡 Remember to activate the virtual environment by running: source .venv/bin/activate"
|
||||
|
||||
# Run all tests with unified coverage report
|
||||
test: test-unit test-int
|
||||
|
||||
# Run unit tests only (fast, no coverage)
|
||||
test-unit:
|
||||
uv run pytest -p pytest_mock -v --no-cov tests
|
||||
|
||||
# Run integration tests only (fast, no coverage)
|
||||
test-int:
|
||||
uv run pytest -p pytest_mock -v --no-cov test-int
|
||||
|
||||
# ==============================================================================
|
||||
# DATABASE BACKEND TESTING
|
||||
# ==============================================================================
|
||||
# Basic Memory supports dual database backends (SQLite and Postgres).
|
||||
# Tests are parametrized to run against both backends automatically.
|
||||
# By default, tests run against SQLite (fast, no dependencies).
|
||||
# Set BASIC_MEMORY_TEST_POSTGRES=1 to run against Postgres (uses testcontainers).
|
||||
#
|
||||
# Quick Start:
|
||||
# just test-sqlite # Run SQLite tests (default, no Docker needed)
|
||||
# just test-postgres # Run Postgres tests (requires Docker)
|
||||
# just test # Run all tests against SQLite (default)
|
||||
# just test-sqlite # Run all tests against SQLite
|
||||
# just test-postgres # Run all tests against Postgres (testcontainers)
|
||||
# just test-unit-sqlite # Run unit tests against SQLite
|
||||
# just test-unit-postgres # Run unit tests against Postgres
|
||||
# just test-int-sqlite # Run integration tests against SQLite
|
||||
# just test-int-postgres # Run integration tests against Postgres
|
||||
#
|
||||
# For Postgres tests, first start the database:
|
||||
# docker-compose -f docker-compose-postgres.yml up -d
|
||||
# CI runs both in parallel for faster feedback.
|
||||
# ==============================================================================
|
||||
|
||||
# Run tests against SQLite only (default backend, skip Postgres/Benchmark tests)
|
||||
# This is the fastest option and doesn't require any Docker setup.
|
||||
# Use this for local development and quick feedback.
|
||||
# Includes Windows-specific tests which will auto-skip on non-Windows platforms.
|
||||
test-sqlite:
|
||||
uv run pytest -p pytest_mock -v --no-cov -m "not postgres and not benchmark" tests test-int
|
||||
# Run all tests against SQLite and Postgres
|
||||
test: test-sqlite test-postgres
|
||||
|
||||
# Run tests against Postgres only (requires docker-compose-postgres.yml up)
|
||||
# First start Postgres: docker-compose -f docker-compose-postgres.yml up -d
|
||||
# Tests will connect to localhost:5433/basic_memory_test
|
||||
# To reset the database: just postgres-reset
|
||||
test-postgres:
|
||||
uv run pytest -p pytest_mock -v --no-cov -m "postgres and not benchmark" tests test-int
|
||||
# Run all tests against SQLite
|
||||
test-sqlite: test-unit-sqlite test-int-sqlite
|
||||
|
||||
# Run all tests against Postgres (uses testcontainers)
|
||||
test-postgres: test-unit-postgres test-int-postgres
|
||||
|
||||
# Run unit tests against SQLite
|
||||
test-unit-sqlite:
|
||||
uv run pytest -p pytest_mock -v --no-cov tests
|
||||
|
||||
# Run unit tests against Postgres
|
||||
test-unit-postgres:
|
||||
BASIC_MEMORY_TEST_POSTGRES=1 uv run pytest -p pytest_mock -v --no-cov tests
|
||||
|
||||
# Run integration tests against SQLite
|
||||
test-int-sqlite:
|
||||
uv run pytest -p pytest_mock -v --no-cov test-int
|
||||
|
||||
# Run integration tests against Postgres
|
||||
# Note: Uses timeout due to FastMCP Client + asyncpg cleanup hang (tests pass, process hangs on exit)
|
||||
# See: https://github.com/jlowin/fastmcp/issues/1311
|
||||
test-int-postgres:
|
||||
timeout --signal=KILL 600 bash -c 'BASIC_MEMORY_TEST_POSTGRES=1 uv run pytest -p pytest_mock -v --no-cov test-int' || test $? -eq 137
|
||||
|
||||
# Reset Postgres test database (drops and recreates schema)
|
||||
# Useful when Alembic migration state gets out of sync during development
|
||||
@@ -59,7 +66,7 @@ postgres-reset:
|
||||
postgres-migrate:
|
||||
@cd src/basic_memory/alembic && \
|
||||
BASIC_MEMORY_DATABASE_BACKEND=postgres \
|
||||
BASIC_MEMORY_DATABASE_URL=${POSTGRES_TEST_URL:-postgresql://basic_memory_user:dev_password@localhost:5433/basic_memory_test} \
|
||||
BASIC_MEMORY_DATABASE_URL=${POSTGRES_TEST_URL:-postgresql+asyncpg://basic_memory_user:dev_password@localhost:5433/basic_memory_test} \
|
||||
uv run alembic upgrade head
|
||||
@echo "✅ Migrations applied to Postgres test database"
|
||||
|
||||
|
||||
+6
-4
@@ -29,14 +29,15 @@ dependencies = [
|
||||
"alembic>=1.14.1",
|
||||
"pillow>=11.1.0",
|
||||
"pybars3>=0.9.7",
|
||||
"fastmcp>=2.10.2",
|
||||
"fastmcp==2.12.3", # Pinned - 2.14.x breaks MCP tools visibility (issue #463)
|
||||
"pyjwt>=2.10.1",
|
||||
"python-dotenv>=1.1.0",
|
||||
"pytest-aio>=1.9.0",
|
||||
"aiofiles>=24.1.0", # Async file I/O
|
||||
"logfire>=0.73.0", # Optional observability (disabled by default via config)
|
||||
"aiofiles>=24.1.0", # Optional observability (disabled by default via config)
|
||||
"asyncpg>=0.30.0",
|
||||
"nest-asyncio>=1.6.0", # For Alembic migrations with Postgres
|
||||
"pytest-asyncio>=1.2.0",
|
||||
"psycopg==3.3.1",
|
||||
]
|
||||
|
||||
|
||||
@@ -81,7 +82,8 @@ dev = [
|
||||
"pytest-xdist>=3.0.0",
|
||||
"ruff>=0.1.6",
|
||||
"freezegun>=1.5.5",
|
||||
|
||||
"testcontainers[postgres]>=4.0.0",
|
||||
"psycopg>=3.2.0",
|
||||
]
|
||||
|
||||
[tool.hatch.version]
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
"""basic-memory - Local-first knowledge management combining Zettelkasten with knowledge graphs"""
|
||||
|
||||
# Package version - updated by release automation
|
||||
__version__ = "0.16.2"
|
||||
__version__ = "0.16.3"
|
||||
|
||||
# API version for FastAPI - independent of package version
|
||||
__api_version__ = "v0"
|
||||
|
||||
@@ -0,0 +1,56 @@
|
||||
"""Add cascade delete FK from search_index to entity
|
||||
|
||||
Revision ID: a2b3c4d5e6f7
|
||||
Revises: f8a9b2c3d4e5
|
||||
Create Date: 2025-12-02 07:00:00.000000
|
||||
|
||||
"""
|
||||
|
||||
from typing import Sequence, Union
|
||||
|
||||
from alembic import op
|
||||
|
||||
|
||||
# revision identifiers, used by Alembic.
|
||||
revision: str = "a2b3c4d5e6f7"
|
||||
down_revision: Union[str, None] = "f8a9b2c3d4e5"
|
||||
branch_labels: Union[str, Sequence[str], None] = None
|
||||
depends_on: Union[str, Sequence[str], None] = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
"""Add FK with CASCADE delete from search_index.entity_id to entity.id.
|
||||
|
||||
This migration is Postgres-only because:
|
||||
- SQLite uses FTS5 virtual tables which don't support foreign keys
|
||||
- The FK enables automatic cleanup of search_index entries when entities are deleted
|
||||
"""
|
||||
connection = op.get_bind()
|
||||
dialect = connection.dialect.name
|
||||
|
||||
if dialect == "postgresql":
|
||||
# First, clean up any orphaned search_index entries where entity no longer exists
|
||||
op.execute("""
|
||||
DELETE FROM search_index
|
||||
WHERE entity_id IS NOT NULL
|
||||
AND entity_id NOT IN (SELECT id FROM entity)
|
||||
""")
|
||||
|
||||
# Add FK with CASCADE - nullable FK allows search_index entries without entity_id
|
||||
op.create_foreign_key(
|
||||
"fk_search_index_entity_id",
|
||||
"search_index",
|
||||
"entity",
|
||||
["entity_id"],
|
||||
["id"],
|
||||
ondelete="CASCADE",
|
||||
)
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
"""Remove the FK constraint."""
|
||||
connection = op.get_bind()
|
||||
dialect = connection.dialect.name
|
||||
|
||||
if dialect == "postgresql":
|
||||
op.drop_constraint("fk_search_index_entity_id", "search_index", type_="foreignkey")
|
||||
+199
@@ -0,0 +1,199 @@
|
||||
"""Add project_id to relation/observation and pg_trgm for fuzzy link resolution
|
||||
|
||||
Revision ID: f8a9b2c3d4e5
|
||||
Revises: 314f1ea54dc4
|
||||
Create Date: 2025-12-01 12:00:00.000000
|
||||
|
||||
"""
|
||||
|
||||
from typing import Sequence, Union
|
||||
|
||||
import sqlalchemy as sa
|
||||
from alembic import op
|
||||
|
||||
|
||||
# revision identifiers, used by Alembic.
|
||||
revision: str = "f8a9b2c3d4e5"
|
||||
down_revision: Union[str, None] = "314f1ea54dc4"
|
||||
branch_labels: Union[str, Sequence[str], None] = None
|
||||
depends_on: Union[str, Sequence[str], None] = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
"""Add project_id to relation and observation tables, plus pg_trgm indexes.
|
||||
|
||||
This migration:
|
||||
1. Adds project_id column to relation and observation tables (denormalization)
|
||||
2. Backfills project_id from the associated entity
|
||||
3. Enables pg_trgm extension for trigram-based fuzzy matching (Postgres only)
|
||||
4. Creates GIN indexes on entity title and permalink for fast similarity searches
|
||||
5. Creates partial index on unresolved relations for efficient bulk resolution
|
||||
"""
|
||||
connection = op.get_bind()
|
||||
dialect = connection.dialect.name
|
||||
|
||||
# -------------------------------------------------------------------------
|
||||
# Add project_id to relation table
|
||||
# -------------------------------------------------------------------------
|
||||
|
||||
# Step 1: Add project_id column as nullable first
|
||||
op.add_column("relation", sa.Column("project_id", sa.Integer(), nullable=True))
|
||||
|
||||
# Step 2: Backfill project_id from entity.project_id via from_id
|
||||
if dialect == "postgresql":
|
||||
op.execute("""
|
||||
UPDATE relation
|
||||
SET project_id = entity.project_id
|
||||
FROM entity
|
||||
WHERE relation.from_id = entity.id
|
||||
""")
|
||||
else:
|
||||
# SQLite syntax
|
||||
op.execute("""
|
||||
UPDATE relation
|
||||
SET project_id = (
|
||||
SELECT entity.project_id
|
||||
FROM entity
|
||||
WHERE entity.id = relation.from_id
|
||||
)
|
||||
""")
|
||||
|
||||
# Step 3: Make project_id NOT NULL and add foreign key
|
||||
if dialect == "postgresql":
|
||||
op.alter_column("relation", "project_id", nullable=False)
|
||||
op.create_foreign_key(
|
||||
"fk_relation_project_id",
|
||||
"relation",
|
||||
"project",
|
||||
["project_id"],
|
||||
["id"],
|
||||
)
|
||||
else:
|
||||
# SQLite requires batch operations for ALTER COLUMN
|
||||
with op.batch_alter_table("relation") as batch_op:
|
||||
batch_op.alter_column("project_id", nullable=False)
|
||||
batch_op.create_foreign_key(
|
||||
"fk_relation_project_id",
|
||||
"project",
|
||||
["project_id"],
|
||||
["id"],
|
||||
)
|
||||
|
||||
# Step 4: Create index on relation.project_id
|
||||
op.create_index("ix_relation_project_id", "relation", ["project_id"])
|
||||
|
||||
# -------------------------------------------------------------------------
|
||||
# Add project_id to observation table
|
||||
# -------------------------------------------------------------------------
|
||||
|
||||
# Step 1: Add project_id column as nullable first
|
||||
op.add_column("observation", sa.Column("project_id", sa.Integer(), nullable=True))
|
||||
|
||||
# Step 2: Backfill project_id from entity.project_id via entity_id
|
||||
if dialect == "postgresql":
|
||||
op.execute("""
|
||||
UPDATE observation
|
||||
SET project_id = entity.project_id
|
||||
FROM entity
|
||||
WHERE observation.entity_id = entity.id
|
||||
""")
|
||||
else:
|
||||
# SQLite syntax
|
||||
op.execute("""
|
||||
UPDATE observation
|
||||
SET project_id = (
|
||||
SELECT entity.project_id
|
||||
FROM entity
|
||||
WHERE entity.id = observation.entity_id
|
||||
)
|
||||
""")
|
||||
|
||||
# Step 3: Make project_id NOT NULL and add foreign key
|
||||
if dialect == "postgresql":
|
||||
op.alter_column("observation", "project_id", nullable=False)
|
||||
op.create_foreign_key(
|
||||
"fk_observation_project_id",
|
||||
"observation",
|
||||
"project",
|
||||
["project_id"],
|
||||
["id"],
|
||||
)
|
||||
else:
|
||||
# SQLite requires batch operations for ALTER COLUMN
|
||||
with op.batch_alter_table("observation") as batch_op:
|
||||
batch_op.alter_column("project_id", nullable=False)
|
||||
batch_op.create_foreign_key(
|
||||
"fk_observation_project_id",
|
||||
"project",
|
||||
["project_id"],
|
||||
["id"],
|
||||
)
|
||||
|
||||
# Step 4: Create index on observation.project_id
|
||||
op.create_index("ix_observation_project_id", "observation", ["project_id"])
|
||||
|
||||
# Postgres-specific: pg_trgm and GIN indexes
|
||||
if dialect == "postgresql":
|
||||
# Enable pg_trgm extension for fuzzy string matching
|
||||
op.execute("CREATE EXTENSION IF NOT EXISTS pg_trgm")
|
||||
|
||||
# Create trigram indexes on entity table for fuzzy matching
|
||||
# GIN indexes with gin_trgm_ops support similarity searches
|
||||
op.execute("""
|
||||
CREATE INDEX IF NOT EXISTS idx_entity_title_trgm
|
||||
ON entity USING gin (title gin_trgm_ops)
|
||||
""")
|
||||
|
||||
op.execute("""
|
||||
CREATE INDEX IF NOT EXISTS idx_entity_permalink_trgm
|
||||
ON entity USING gin (permalink gin_trgm_ops)
|
||||
""")
|
||||
|
||||
# Create partial index on unresolved relations for efficient bulk resolution
|
||||
# This makes "WHERE to_id IS NULL AND project_id = X" queries very fast
|
||||
op.execute("""
|
||||
CREATE INDEX IF NOT EXISTS idx_relation_unresolved
|
||||
ON relation (project_id, to_name)
|
||||
WHERE to_id IS NULL
|
||||
""")
|
||||
|
||||
# Create index on relation.to_name for join performance in bulk resolution
|
||||
op.execute("""
|
||||
CREATE INDEX IF NOT EXISTS idx_relation_to_name
|
||||
ON relation (to_name)
|
||||
""")
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
"""Remove project_id from relation/observation and pg_trgm indexes."""
|
||||
connection = op.get_bind()
|
||||
dialect = connection.dialect.name
|
||||
|
||||
if dialect == "postgresql":
|
||||
# Drop Postgres-specific indexes
|
||||
op.execute("DROP INDEX IF EXISTS idx_relation_to_name")
|
||||
op.execute("DROP INDEX IF EXISTS idx_relation_unresolved")
|
||||
op.execute("DROP INDEX IF EXISTS idx_entity_permalink_trgm")
|
||||
op.execute("DROP INDEX IF EXISTS idx_entity_title_trgm")
|
||||
# Note: We don't drop the pg_trgm extension as other code may depend on it
|
||||
|
||||
# Drop project_id from observation
|
||||
op.drop_index("ix_observation_project_id", table_name="observation")
|
||||
op.drop_constraint("fk_observation_project_id", "observation", type_="foreignkey")
|
||||
op.drop_column("observation", "project_id")
|
||||
|
||||
# Drop project_id from relation
|
||||
op.drop_index("ix_relation_project_id", table_name="relation")
|
||||
op.drop_constraint("fk_relation_project_id", "relation", type_="foreignkey")
|
||||
op.drop_column("relation", "project_id")
|
||||
else:
|
||||
# SQLite requires batch operations
|
||||
op.drop_index("ix_observation_project_id", table_name="observation")
|
||||
with op.batch_alter_table("observation") as batch_op:
|
||||
batch_op.drop_constraint("fk_observation_project_id", type_="foreignkey")
|
||||
batch_op.drop_column("project_id")
|
||||
|
||||
op.drop_index("ix_relation_project_id", table_name="relation")
|
||||
with op.batch_alter_table("relation") as batch_op:
|
||||
batch_op.drop_constraint("fk_relation_project_id", type_="foreignkey")
|
||||
batch_op.drop_column("project_id")
|
||||
@@ -30,7 +30,7 @@ from basic_memory.api.v2.routers import (
|
||||
prompt_router as v2_prompt,
|
||||
importer_router as v2_importer,
|
||||
)
|
||||
from basic_memory.config import ConfigManager
|
||||
from basic_memory.config import ConfigManager, init_api_logging
|
||||
from basic_memory.services.initialization import initialize_file_sync, initialize_app
|
||||
|
||||
|
||||
@@ -38,6 +38,9 @@ from basic_memory.services.initialization import initialize_file_sync, initializ
|
||||
async def lifespan(app: FastAPI): # pragma: no cover
|
||||
"""Lifecycle manager for the FastAPI app. Not called in stdio mcp mode"""
|
||||
|
||||
# Initialize logging for API (stdout in cloud mode, file otherwise)
|
||||
init_api_logging()
|
||||
|
||||
app_config = ConfigManager().config
|
||||
logger.info("Starting Basic Memory API")
|
||||
|
||||
@@ -56,6 +59,7 @@ async def lifespan(app: FastAPI): # pragma: no cover
|
||||
app.state.sync_task = asyncio.create_task(initialize_file_sync(app_config))
|
||||
else:
|
||||
logger.info("Sync changes disabled. Skipping file sync service.")
|
||||
app.state.sync_task = None
|
||||
|
||||
# proceed with startup
|
||||
yield
|
||||
@@ -64,6 +68,10 @@ async def lifespan(app: FastAPI): # pragma: no cover
|
||||
if app.state.sync_task:
|
||||
logger.info("Stopping sync...")
|
||||
app.state.sync_task.cancel() # pyright: ignore
|
||||
try:
|
||||
await app.state.sync_task
|
||||
except asyncio.CancelledError:
|
||||
logger.info("Sync task cancelled successfully")
|
||||
|
||||
await db.shutdown_db()
|
||||
|
||||
@@ -100,8 +108,6 @@ app.include_router(v2_project, prefix="/v2")
|
||||
app.include_router(project.project_resource_router)
|
||||
app.include_router(management.router)
|
||||
|
||||
# Auth routes are handled by FastMCP automatically when auth is enabled
|
||||
|
||||
|
||||
@app.exception_handler(Exception)
|
||||
async def exception_handler(request, exc): # pragma: no cover
|
||||
|
||||
@@ -2,9 +2,9 @@
|
||||
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
from typing import Annotated
|
||||
from typing import Annotated, Union
|
||||
|
||||
from fastapi import APIRouter, HTTPException, BackgroundTasks, Body
|
||||
from fastapi import APIRouter, HTTPException, BackgroundTasks, Body, Response
|
||||
from fastapi.responses import FileResponse, JSONResponse
|
||||
from loguru import logger
|
||||
|
||||
@@ -25,6 +25,17 @@ from datetime import datetime
|
||||
router = APIRouter(prefix="/resource", tags=["resources"])
|
||||
|
||||
|
||||
def _mtime_to_datetime(entity: EntityModel) -> datetime:
|
||||
"""Convert entity mtime (file modification time) to datetime.
|
||||
|
||||
Returns the file's actual modification time, falling back to updated_at
|
||||
if mtime is not available.
|
||||
"""
|
||||
if entity.mtime:
|
||||
return datetime.fromtimestamp(entity.mtime).astimezone()
|
||||
return entity.updated_at
|
||||
|
||||
|
||||
def get_entity_ids(item: SearchIndexRow) -> set[int]:
|
||||
match item.type:
|
||||
case SearchItemType.ENTITY:
|
||||
@@ -39,7 +50,7 @@ def get_entity_ids(item: SearchIndexRow) -> set[int]:
|
||||
raise ValueError(f"Unexpected type: {item.type}")
|
||||
|
||||
|
||||
@router.get("/{identifier:path}")
|
||||
@router.get("/{identifier:path}", response_model=None)
|
||||
async def get_resource_content(
|
||||
config: ProjectConfigDep,
|
||||
link_resolver: LinkResolverDep,
|
||||
@@ -50,7 +61,7 @@ async def get_resource_content(
|
||||
identifier: str,
|
||||
page: int = 1,
|
||||
page_size: int = 10,
|
||||
) -> FileResponse:
|
||||
) -> Union[Response, FileResponse]:
|
||||
"""Get resource content by identifier: name or permalink."""
|
||||
logger.debug(f"Getting content for: {identifier}")
|
||||
|
||||
@@ -81,13 +92,16 @@ async def get_resource_content(
|
||||
# return single response
|
||||
if len(results) == 1:
|
||||
entity = results[0]
|
||||
file_path = Path(f"{config.home}/{entity.file_path}")
|
||||
if not file_path.exists():
|
||||
# Check file exists via file_service (for cloud compatibility)
|
||||
if not await file_service.exists(entity.file_path):
|
||||
raise HTTPException(
|
||||
status_code=404,
|
||||
detail=f"File not found: {file_path}",
|
||||
detail=f"File not found: {entity.file_path}",
|
||||
)
|
||||
return FileResponse(path=file_path)
|
||||
# Read content via file_service as bytes (works with both local and S3)
|
||||
content = await file_service.read_file_bytes(entity.file_path)
|
||||
content_type = file_service.content_type(entity.file_path)
|
||||
return Response(content=content, media_type=content_type)
|
||||
|
||||
# for multiple files, initialize a temporary file for writing the results
|
||||
with tempfile.NamedTemporaryFile(delete=False, mode="w", suffix=".md") as tmp_file:
|
||||
@@ -97,7 +111,7 @@ async def get_resource_content(
|
||||
# Read content for each entity
|
||||
content = await file_service.read_entity_content(result)
|
||||
memory_url = normalize_memory_url(result.permalink)
|
||||
modified_date = result.updated_at.isoformat()
|
||||
modified_date = _mtime_to_datetime(result).isoformat()
|
||||
checksum = result.checksum[:8] if result.checksum else ""
|
||||
|
||||
# Prepare the delimited content
|
||||
@@ -171,21 +185,17 @@ async def write_resource(
|
||||
else:
|
||||
content_str = str(content)
|
||||
|
||||
# Get full file path
|
||||
full_path = Path(f"{config.home}/{file_path}")
|
||||
|
||||
# Ensure parent directory exists
|
||||
full_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
# Write content to file
|
||||
checksum = await file_service.write_file(full_path, content_str)
|
||||
# Cloud compatibility: do not assume a local filesystem path structure.
|
||||
# Delegate directory creation + writes to the configured FileService (local or S3).
|
||||
await file_service.ensure_directory(Path(file_path).parent)
|
||||
checksum = await file_service.write_file(file_path, content_str)
|
||||
|
||||
# Get file info
|
||||
file_stats = file_service.file_stats(full_path)
|
||||
file_metadata = await file_service.get_file_metadata(file_path)
|
||||
|
||||
# Determine file details
|
||||
file_name = Path(file_path).name
|
||||
content_type = file_service.content_type(full_path)
|
||||
content_type = file_service.content_type(file_path)
|
||||
|
||||
entity_type = "canvas" if file_path.endswith(".canvas") else "file"
|
||||
|
||||
@@ -202,7 +212,7 @@ async def write_resource(
|
||||
"content_type": content_type,
|
||||
"file_path": file_path,
|
||||
"checksum": checksum,
|
||||
"updated_at": datetime.fromtimestamp(file_stats.st_mtime).astimezone(),
|
||||
"updated_at": file_metadata.modified_at,
|
||||
},
|
||||
)
|
||||
status_code = 200
|
||||
@@ -214,8 +224,8 @@ async def write_resource(
|
||||
content_type=content_type,
|
||||
file_path=file_path,
|
||||
checksum=checksum,
|
||||
created_at=datetime.fromtimestamp(file_stats.st_ctime).astimezone(),
|
||||
updated_at=datetime.fromtimestamp(file_stats.st_mtime).astimezone(),
|
||||
created_at=file_metadata.created_at,
|
||||
updated_at=file_metadata.modified_at,
|
||||
)
|
||||
entity = await entity_repository.add(entity)
|
||||
status_code = 201
|
||||
@@ -229,9 +239,9 @@ async def write_resource(
|
||||
content={
|
||||
"file_path": file_path,
|
||||
"checksum": checksum,
|
||||
"size": file_stats.st_size,
|
||||
"created_at": file_stats.st_ctime,
|
||||
"modified_at": file_stats.st_mtime,
|
||||
"size": file_metadata.size,
|
||||
"created_at": file_metadata.created_at.timestamp(),
|
||||
"modified_at": file_metadata.modified_at.timestamp(),
|
||||
},
|
||||
)
|
||||
except Exception as e: # pragma: no cover
|
||||
|
||||
@@ -24,8 +24,26 @@ async def to_graph_context(
|
||||
page: Optional[int] = None,
|
||||
page_size: Optional[int] = None,
|
||||
):
|
||||
# First pass: collect all entity IDs needed for relations
|
||||
entity_ids_needed: set[int] = set()
|
||||
for context_item in context_result.results:
|
||||
for item in (
|
||||
[context_item.primary_result] + context_item.observations + context_item.related_results
|
||||
):
|
||||
if item.type == SearchItemType.RELATION:
|
||||
if item.from_id: # pyright: ignore
|
||||
entity_ids_needed.add(item.from_id) # pyright: ignore
|
||||
if item.to_id:
|
||||
entity_ids_needed.add(item.to_id)
|
||||
|
||||
# Batch fetch all entities at once
|
||||
entity_lookup: dict[int, str] = {}
|
||||
if entity_ids_needed:
|
||||
entities = await entity_repository.find_by_ids(list(entity_ids_needed))
|
||||
entity_lookup = {e.id: e.title for e in entities}
|
||||
|
||||
# Helper function to convert items to summaries
|
||||
async def to_summary(item: SearchIndexRow | ContextResultRow):
|
||||
def to_summary(item: SearchIndexRow | ContextResultRow):
|
||||
match item.type:
|
||||
case SearchItemType.ENTITY:
|
||||
return EntitySummary(
|
||||
@@ -48,8 +66,8 @@ async def to_graph_context(
|
||||
created_at=item.created_at,
|
||||
)
|
||||
case SearchItemType.RELATION:
|
||||
from_entity = await entity_repository.find_by_id(item.from_id) # pyright: ignore
|
||||
to_entity = await entity_repository.find_by_id(item.to_id) if item.to_id else None
|
||||
from_title = entity_lookup.get(item.from_id) if item.from_id else None # pyright: ignore
|
||||
to_title = entity_lookup.get(item.to_id) if item.to_id else None
|
||||
return RelationSummary(
|
||||
relation_id=item.id,
|
||||
entity_id=item.entity_id, # pyright: ignore
|
||||
@@ -57,9 +75,9 @@ async def to_graph_context(
|
||||
file_path=item.file_path,
|
||||
permalink=item.permalink, # pyright: ignore
|
||||
relation_type=item.relation_type, # pyright: ignore
|
||||
from_entity=from_entity.title if from_entity else None,
|
||||
from_entity=from_title,
|
||||
from_entity_id=item.from_id, # pyright: ignore
|
||||
to_entity=to_entity.title if to_entity else None,
|
||||
to_entity=to_title,
|
||||
to_entity_id=item.to_id,
|
||||
created_at=item.created_at,
|
||||
)
|
||||
@@ -70,23 +88,19 @@ async def to_graph_context(
|
||||
hierarchical_results = []
|
||||
for context_item in context_result.results:
|
||||
# Process primary result
|
||||
primary_result = await to_summary(context_item.primary_result)
|
||||
primary_result = to_summary(context_item.primary_result)
|
||||
|
||||
# Process observations
|
||||
observations = []
|
||||
for obs in context_item.observations:
|
||||
observations.append(await to_summary(obs))
|
||||
# Process observations (always ObservationSummary, validated by context_service)
|
||||
observations = [to_summary(obs) for obs in context_item.observations]
|
||||
|
||||
# Process related results
|
||||
related = []
|
||||
for rel in context_item.related_results:
|
||||
related.append(await to_summary(rel))
|
||||
related = [to_summary(rel) for rel in context_item.related_results]
|
||||
|
||||
# Add to hierarchical results
|
||||
hierarchical_results.append(
|
||||
ContextResult(
|
||||
primary_result=primary_result,
|
||||
observations=observations,
|
||||
observations=observations, # pyright: ignore[reportArgumentType]
|
||||
related_results=related,
|
||||
)
|
||||
)
|
||||
|
||||
@@ -11,8 +11,7 @@ Key differences from v1:
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
from fastapi import APIRouter, HTTPException
|
||||
from fastapi.responses import FileResponse
|
||||
from fastapi import APIRouter, HTTPException, Response
|
||||
from loguru import logger
|
||||
|
||||
from basic_memory.deps import (
|
||||
@@ -30,7 +29,6 @@ from basic_memory.schemas.v2.resource import (
|
||||
ResourceResponse,
|
||||
)
|
||||
from basic_memory.utils import validate_project_path
|
||||
from datetime import datetime
|
||||
|
||||
router = APIRouter(prefix="/resource", tags=["resources-v2"])
|
||||
|
||||
@@ -42,7 +40,7 @@ async def get_resource_content(
|
||||
config: ProjectConfigV2Dep,
|
||||
entity_service: EntityServiceV2Dep,
|
||||
file_service: FileServiceV2Dep,
|
||||
) -> FileResponse:
|
||||
) -> Response:
|
||||
"""Get raw resource content by entity ID.
|
||||
|
||||
Args:
|
||||
@@ -53,7 +51,7 @@ async def get_resource_content(
|
||||
file_service: File service for reading file content
|
||||
|
||||
Returns:
|
||||
FileResponse with entity content
|
||||
Response with entity content
|
||||
|
||||
Raises:
|
||||
HTTPException: 404 if entity or file not found
|
||||
@@ -76,14 +74,18 @@ async def get_resource_content(
|
||||
detail="Entity contains invalid file path",
|
||||
)
|
||||
|
||||
file_path = Path(f"{config.home}/{entity.file_path}")
|
||||
if not file_path.exists():
|
||||
# Check file exists via file_service (for cloud compatibility)
|
||||
if not await file_service.exists(entity.file_path):
|
||||
raise HTTPException(
|
||||
status_code=404,
|
||||
detail=f"File not found: {file_path}",
|
||||
detail=f"File not found: {entity.file_path}",
|
||||
)
|
||||
|
||||
return FileResponse(path=file_path)
|
||||
# Read content via file_service as bytes (works with both local and S3)
|
||||
content = await file_service.read_file_bytes(entity.file_path)
|
||||
content_type = file_service.content_type(entity.file_path)
|
||||
|
||||
return Response(content=content, media_type=content_type)
|
||||
|
||||
|
||||
@router.post("", response_model=ResourceResponse)
|
||||
@@ -133,21 +135,17 @@ async def create_resource(
|
||||
f"Use PUT /resource/{existing_entity.id} to update it.",
|
||||
)
|
||||
|
||||
# Get full file path
|
||||
full_path = Path(f"{config.home}/{data.file_path}")
|
||||
|
||||
# Ensure parent directory exists
|
||||
full_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
# Write content to file
|
||||
checksum = await file_service.write_file(full_path, data.content)
|
||||
# Cloud compatibility: avoid assuming a local filesystem path.
|
||||
# Delegate directory creation + writes to FileService (local or S3).
|
||||
await file_service.ensure_directory(Path(data.file_path).parent)
|
||||
checksum = await file_service.write_file(data.file_path, data.content)
|
||||
|
||||
# Get file info
|
||||
file_stats = file_service.file_stats(full_path)
|
||||
file_metadata = await file_service.get_file_metadata(data.file_path)
|
||||
|
||||
# Determine file details
|
||||
file_name = Path(data.file_path).name
|
||||
content_type = file_service.content_type(full_path)
|
||||
content_type = file_service.content_type(data.file_path)
|
||||
entity_type = "canvas" if data.file_path.endswith(".canvas") else "file"
|
||||
|
||||
# Create a new entity model
|
||||
@@ -157,8 +155,8 @@ async def create_resource(
|
||||
content_type=content_type,
|
||||
file_path=data.file_path,
|
||||
checksum=checksum,
|
||||
created_at=datetime.fromtimestamp(file_stats.st_ctime).astimezone(),
|
||||
updated_at=datetime.fromtimestamp(file_stats.st_mtime).astimezone(),
|
||||
created_at=file_metadata.created_at,
|
||||
updated_at=file_metadata.modified_at,
|
||||
)
|
||||
entity = await entity_repository.add(entity)
|
||||
|
||||
@@ -170,9 +168,9 @@ async def create_resource(
|
||||
entity_id=entity.id,
|
||||
file_path=data.file_path,
|
||||
checksum=checksum,
|
||||
size=file_stats.st_size,
|
||||
created_at=file_stats.st_ctime,
|
||||
modified_at=file_stats.st_mtime,
|
||||
size=file_metadata.size,
|
||||
created_at=file_metadata.created_at.timestamp(),
|
||||
modified_at=file_metadata.modified_at.timestamp(),
|
||||
)
|
||||
except HTTPException:
|
||||
# Re-raise HTTP exceptions without wrapping
|
||||
@@ -232,31 +230,27 @@ async def update_resource(
|
||||
"Path must be relative and stay within project boundaries.",
|
||||
)
|
||||
|
||||
# Get full paths
|
||||
old_full_path = Path(f"{config.home}/{entity.file_path}")
|
||||
new_full_path = Path(f"{config.home}/{target_file_path}")
|
||||
|
||||
# If moving file, handle the move
|
||||
if data.file_path and data.file_path != entity.file_path:
|
||||
# Ensure new parent directory exists
|
||||
new_full_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
# Ensure new parent directory exists (no-op for S3)
|
||||
await file_service.ensure_directory(Path(target_file_path).parent)
|
||||
|
||||
# If old file exists, remove it
|
||||
if old_full_path.exists():
|
||||
old_full_path.unlink()
|
||||
# If old file exists, remove it via file_service (for cloud compatibility)
|
||||
if await file_service.exists(entity.file_path):
|
||||
await file_service.delete_file(entity.file_path)
|
||||
else:
|
||||
# Ensure directory exists for in-place update
|
||||
new_full_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
await file_service.ensure_directory(Path(target_file_path).parent)
|
||||
|
||||
# Write content to target file
|
||||
checksum = await file_service.write_file(new_full_path, data.content)
|
||||
checksum = await file_service.write_file(target_file_path, data.content)
|
||||
|
||||
# Get file info
|
||||
file_stats = file_service.file_stats(new_full_path)
|
||||
file_metadata = await file_service.get_file_metadata(target_file_path)
|
||||
|
||||
# Determine file details
|
||||
file_name = Path(target_file_path).name
|
||||
content_type = file_service.content_type(new_full_path)
|
||||
content_type = file_service.content_type(target_file_path)
|
||||
entity_type = "canvas" if target_file_path.endswith(".canvas") else "file"
|
||||
|
||||
# Update entity
|
||||
@@ -268,7 +262,7 @@ async def update_resource(
|
||||
"content_type": content_type,
|
||||
"file_path": target_file_path,
|
||||
"checksum": checksum,
|
||||
"updated_at": datetime.fromtimestamp(file_stats.st_mtime).astimezone(),
|
||||
"updated_at": file_metadata.modified_at,
|
||||
},
|
||||
)
|
||||
|
||||
@@ -280,9 +274,9 @@ async def update_resource(
|
||||
entity_id=entity_id,
|
||||
file_path=target_file_path,
|
||||
checksum=checksum,
|
||||
size=file_stats.st_size,
|
||||
created_at=file_stats.st_ctime,
|
||||
modified_at=file_stats.st_mtime,
|
||||
size=file_metadata.size,
|
||||
created_at=file_metadata.created_at.timestamp(),
|
||||
modified_at=file_metadata.modified_at.timestamp(),
|
||||
)
|
||||
except HTTPException:
|
||||
# Re-raise HTTP exceptions without wrapping
|
||||
|
||||
@@ -2,7 +2,7 @@ from typing import Optional
|
||||
|
||||
import typer
|
||||
|
||||
from basic_memory.config import ConfigManager
|
||||
from basic_memory.config import ConfigManager, init_cli_logging
|
||||
|
||||
|
||||
def version_callback(value: bool) -> None:
|
||||
@@ -31,6 +31,9 @@ def app_callback(
|
||||
) -> None:
|
||||
"""Basic Memory - Local-first personal knowledge management."""
|
||||
|
||||
# Initialize logging for CLI (file only, no stdout)
|
||||
init_cli_logging()
|
||||
|
||||
# Run initialization for every command unless --version was specified
|
||||
if not version and ctx.invoked_subcommand is not None:
|
||||
from basic_memory.services.initialization import ensure_initialization
|
||||
|
||||
@@ -6,7 +6,7 @@ import typer
|
||||
from typing import Optional
|
||||
|
||||
from basic_memory.cli.app import app
|
||||
from basic_memory.config import ConfigManager
|
||||
from basic_memory.config import ConfigManager, init_mcp_logging
|
||||
|
||||
# Import mcp instance
|
||||
from basic_memory.mcp.server import mcp as mcp_server # pragma: no cover
|
||||
@@ -44,6 +44,8 @@ if not config.cloud_mode_enabled:
|
||||
- streamable-http: Recommended for web deployments (default)
|
||||
- sse: Server-Sent Events (for compatibility with existing clients)
|
||||
"""
|
||||
# Initialize logging for MCP (file only, stdout breaks protocol)
|
||||
init_mcp_logging()
|
||||
|
||||
# Validate and set project constraint if specified
|
||||
if project:
|
||||
|
||||
+99
-71
@@ -9,10 +9,9 @@ from typing import Any, Dict, Literal, Optional, List, Tuple
|
||||
from enum import Enum
|
||||
|
||||
from loguru import logger
|
||||
from pydantic import BaseModel, Field, field_validator
|
||||
from pydantic import BaseModel, Field, model_validator
|
||||
from pydantic_settings import BaseSettings, SettingsConfigDict
|
||||
|
||||
import basic_memory
|
||||
from basic_memory.utils import setup_logging, generate_permalink
|
||||
|
||||
|
||||
@@ -100,13 +99,32 @@ class BasicMemoryConfig(BaseSettings):
|
||||
description="Database connection URL. For Postgres, use postgresql+asyncpg://user:pass@host:port/db. If not set, SQLite will use default path.",
|
||||
)
|
||||
|
||||
# Database connection pool configuration (Postgres only)
|
||||
db_pool_size: int = Field(
|
||||
default=20,
|
||||
description="Number of connections to keep in the pool (Postgres only)",
|
||||
gt=0,
|
||||
)
|
||||
db_pool_overflow: int = Field(
|
||||
default=40,
|
||||
description="Max additional connections beyond pool_size under load (Postgres only)",
|
||||
gt=0,
|
||||
)
|
||||
db_pool_recycle: int = Field(
|
||||
default=180,
|
||||
description="Recycle connections after N seconds to prevent stale connections. Default 180s works well with Neon's ~5 minute scale-to-zero (Postgres only)",
|
||||
gt=0,
|
||||
)
|
||||
|
||||
# Watch service configuration
|
||||
sync_delay: int = Field(
|
||||
default=1000, description="Milliseconds to wait after changes before syncing", gt=0
|
||||
)
|
||||
|
||||
watch_project_reload_interval: int = Field(
|
||||
default=30, description="Seconds between reloading project list in watch service", gt=0
|
||||
default=300,
|
||||
description="Seconds between reloading project list in watch service. Higher values reduce CPU usage by minimizing watcher restarts. Default 300s (5 min) balances efficiency with responsiveness to new projects.",
|
||||
gt=0,
|
||||
)
|
||||
|
||||
# update permalinks on move
|
||||
@@ -197,6 +215,36 @@ class BasicMemoryConfig(BaseSettings):
|
||||
# Fall back to config file value
|
||||
return self.cloud_mode
|
||||
|
||||
@classmethod
|
||||
def for_cloud_tenant(
|
||||
cls,
|
||||
database_url: str,
|
||||
projects: Optional[Dict[str, str]] = None,
|
||||
) -> "BasicMemoryConfig":
|
||||
"""Create config for cloud tenant - no config.json, database is source of truth.
|
||||
|
||||
This factory method creates a BasicMemoryConfig suitable for cloud deployments
|
||||
where:
|
||||
- Database is Postgres (Neon), not SQLite
|
||||
- Projects are discovered from the database, not config file
|
||||
- Path validation is skipped (no local filesystem in cloud)
|
||||
- Initialization sync is skipped (stateless deployment)
|
||||
|
||||
Args:
|
||||
database_url: Postgres connection URL for tenant database
|
||||
projects: Optional project mapping (usually empty, discovered from DB)
|
||||
|
||||
Returns:
|
||||
BasicMemoryConfig configured for cloud mode
|
||||
"""
|
||||
return cls(
|
||||
database_backend=DatabaseBackend.POSTGRES,
|
||||
database_url=database_url,
|
||||
projects=projects or {},
|
||||
cloud_mode=True,
|
||||
skip_initialization_sync=True,
|
||||
)
|
||||
|
||||
model_config = SettingsConfigDict(
|
||||
env_prefix="BASIC_MEMORY_",
|
||||
extra="ignore",
|
||||
@@ -213,6 +261,10 @@ class BasicMemoryConfig(BaseSettings):
|
||||
|
||||
def model_post_init(self, __context: Any) -> None:
|
||||
"""Ensure configuration is valid after initialization."""
|
||||
# Skip project initialization in cloud mode - projects are discovered from DB
|
||||
if self.database_backend == DatabaseBackend.POSTGRES:
|
||||
return
|
||||
|
||||
# Ensure at least one project exists; if none exist then create main
|
||||
if not self.projects: # pragma: no cover
|
||||
self.projects["main"] = str(
|
||||
@@ -255,19 +307,26 @@ class BasicMemoryConfig(BaseSettings):
|
||||
"""Get all configured projects as ProjectConfig objects."""
|
||||
return [ProjectConfig(name=name, home=Path(path)) for name, path in self.projects.items()]
|
||||
|
||||
@field_validator("projects")
|
||||
@classmethod
|
||||
def ensure_project_paths_exists(cls, v: Dict[str, str]) -> Dict[str, str]: # pragma: no cover
|
||||
"""Ensure project path exists."""
|
||||
for name, path_value in v.items():
|
||||
@model_validator(mode="after")
|
||||
def ensure_project_paths_exists(self) -> "BasicMemoryConfig": # pragma: no cover
|
||||
"""Ensure project paths exist.
|
||||
|
||||
Skips path creation when using Postgres backend (cloud mode) since
|
||||
cloud tenants don't use local filesystem paths.
|
||||
"""
|
||||
# Skip path creation for cloud mode - no local filesystem
|
||||
if self.database_backend == DatabaseBackend.POSTGRES:
|
||||
return self
|
||||
|
||||
for name, path_value in self.projects.items():
|
||||
path = Path(path_value)
|
||||
if not Path(path).exists():
|
||||
if not path.exists():
|
||||
try:
|
||||
path.mkdir(parents=True)
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to create project path: {e}")
|
||||
raise e
|
||||
return v
|
||||
return self
|
||||
|
||||
@property
|
||||
def data_dir_path(self):
|
||||
@@ -470,69 +529,38 @@ def save_basic_memory_config(file_path: Path, config: BasicMemoryConfig) -> None
|
||||
logger.error(f"Failed to save config: {e}")
|
||||
|
||||
|
||||
# setup logging to a single log file in user home directory
|
||||
user_home = Path.home()
|
||||
log_dir = user_home / DATA_DIR_NAME
|
||||
log_dir.mkdir(parents=True, exist_ok=True)
|
||||
# Logging initialization functions for different entry points
|
||||
|
||||
|
||||
# Process info for logging
|
||||
def get_process_name(): # pragma: no cover
|
||||
def init_cli_logging() -> None: # pragma: no cover
|
||||
"""Initialize logging for CLI commands - file only.
|
||||
|
||||
CLI commands should not log to stdout to avoid interfering with
|
||||
command output and shell integration.
|
||||
"""
|
||||
get the type of process for logging
|
||||
"""
|
||||
import sys
|
||||
log_level = os.getenv("BASIC_MEMORY_LOG_LEVEL", "INFO")
|
||||
setup_logging(log_level=log_level, log_to_file=True)
|
||||
|
||||
if "sync" in sys.argv:
|
||||
return "sync"
|
||||
elif "mcp" in sys.argv:
|
||||
return "mcp"
|
||||
elif "cli" in sys.argv:
|
||||
return "cli"
|
||||
|
||||
def init_mcp_logging() -> None: # pragma: no cover
|
||||
"""Initialize logging for MCP server - file only.
|
||||
|
||||
MCP server must not log to stdout as it would corrupt the
|
||||
JSON-RPC protocol communication.
|
||||
"""
|
||||
log_level = os.getenv("BASIC_MEMORY_LOG_LEVEL", "INFO")
|
||||
setup_logging(log_level=log_level, log_to_file=True)
|
||||
|
||||
|
||||
def init_api_logging() -> None: # pragma: no cover
|
||||
"""Initialize logging for API server.
|
||||
|
||||
Cloud mode (BASIC_MEMORY_CLOUD_MODE=1): stdout with structured context
|
||||
Local mode: file only
|
||||
"""
|
||||
log_level = os.getenv("BASIC_MEMORY_LOG_LEVEL", "INFO")
|
||||
cloud_mode = os.getenv("BASIC_MEMORY_CLOUD_MODE", "").lower() in ("1", "true")
|
||||
if cloud_mode:
|
||||
setup_logging(log_level=log_level, log_to_stdout=True, structured_context=True)
|
||||
else:
|
||||
return "api"
|
||||
|
||||
|
||||
process_name = get_process_name()
|
||||
|
||||
# Global flag to track if logging has been set up
|
||||
_LOGGING_SETUP = False
|
||||
|
||||
|
||||
# Logging
|
||||
|
||||
|
||||
def setup_basic_memory_logging(): # pragma: no cover
|
||||
"""Set up logging for basic-memory, ensuring it only happens once."""
|
||||
global _LOGGING_SETUP
|
||||
if _LOGGING_SETUP:
|
||||
# We can't log before logging is set up
|
||||
# print("Skipping duplicate logging setup")
|
||||
return
|
||||
|
||||
# Check for console logging environment variable - accept more truthy values
|
||||
console_logging_env = os.getenv("BASIC_MEMORY_CONSOLE_LOGGING", "false").lower()
|
||||
console_logging = console_logging_env in ("true", "1", "yes", "on")
|
||||
|
||||
# Check for log level environment variable first, fall back to config
|
||||
log_level = os.getenv("BASIC_MEMORY_LOG_LEVEL")
|
||||
if not log_level:
|
||||
config_manager = ConfigManager()
|
||||
log_level = config_manager.config.log_level
|
||||
|
||||
config_manager = ConfigManager()
|
||||
config = get_project_config()
|
||||
setup_logging(
|
||||
env=config_manager.config.env,
|
||||
home_dir=user_home, # Use user home for logs
|
||||
log_level=log_level,
|
||||
log_file=f"{DATA_DIR_NAME}/basic-memory-{process_name}.log",
|
||||
console=console_logging,
|
||||
)
|
||||
|
||||
logger.info(f"Basic Memory {basic_memory.__version__} (Project: {config.project})")
|
||||
_LOGGING_SETUP = True
|
||||
|
||||
|
||||
# Set up logging
|
||||
setup_basic_memory_logging()
|
||||
setup_logging(log_level=log_level, log_to_file=True)
|
||||
|
||||
+20
-4
@@ -190,21 +190,37 @@ def _create_sqlite_engine(db_url: str, db_type: DatabaseType) -> AsyncEngine:
|
||||
return engine
|
||||
|
||||
|
||||
def _create_postgres_engine(db_url: str) -> AsyncEngine:
|
||||
def _create_postgres_engine(db_url: str, config: BasicMemoryConfig) -> AsyncEngine:
|
||||
"""Create Postgres async engine with appropriate configuration.
|
||||
|
||||
Args:
|
||||
db_url: Postgres connection URL (postgresql+asyncpg://...)
|
||||
config: BasicMemoryConfig with pool settings
|
||||
|
||||
Returns:
|
||||
Configured async engine for Postgres
|
||||
"""
|
||||
# Postgres with asyncpg - use standard async connection
|
||||
# Use NullPool connection issues.
|
||||
# Assume connection pooler like PgBouncer handles connection pooling.
|
||||
engine = create_async_engine(
|
||||
db_url,
|
||||
echo=False,
|
||||
pool_pre_ping=True, # Verify connections before using them
|
||||
poolclass=NullPool, # No pooling - fresh connection per request
|
||||
connect_args={
|
||||
# Disable statement cache to avoid issues with prepared statements on reconnect
|
||||
"statement_cache_size": 0,
|
||||
# Allow 30s for commands (Neon cold start can take 2-5s, sometimes longer)
|
||||
"command_timeout": 30,
|
||||
# Allow 30s for initial connection (Neon wake-up time)
|
||||
"timeout": 30,
|
||||
"server_settings": {
|
||||
"application_name": "basic-memory",
|
||||
# Statement timeout for queries (30s to allow for cold start)
|
||||
"statement_timeout": "30s",
|
||||
},
|
||||
},
|
||||
)
|
||||
logger.debug("Created Postgres engine with NullPool (no connection pooling)")
|
||||
|
||||
return engine
|
||||
|
||||
@@ -228,7 +244,7 @@ def _create_engine_and_session(
|
||||
# Delegate to backend-specific engine creation
|
||||
# Check explicit POSTGRES type first, then config setting
|
||||
if db_type == DatabaseType.POSTGRES or config.database_backend == DatabaseBackend.POSTGRES:
|
||||
engine = _create_postgres_engine(db_url)
|
||||
engine = _create_postgres_engine(db_url, config)
|
||||
else:
|
||||
engine = _create_sqlite_engine(db_url, db_type)
|
||||
|
||||
|
||||
@@ -368,11 +368,10 @@ MarkdownProcessorV2Dep = Annotated[MarkdownProcessor, Depends(get_markdown_proce
|
||||
async def get_file_service(
|
||||
project_config: ProjectConfigDep, markdown_processor: MarkdownProcessorDep
|
||||
) -> FileService:
|
||||
logger.debug(
|
||||
f"Creating FileService for project: {project_config.name}, base_path: {project_config.home}"
|
||||
)
|
||||
file_service = FileService(project_config.home, markdown_processor)
|
||||
logger.debug(f"Created FileService for project: {file_service} ")
|
||||
logger.debug(
|
||||
f"Created FileService for project: {project_config.name}, base_path: {project_config.home} "
|
||||
)
|
||||
return file_service
|
||||
|
||||
|
||||
@@ -382,11 +381,10 @@ FileServiceDep = Annotated[FileService, Depends(get_file_service)]
|
||||
async def get_file_service_v2(
|
||||
project_config: ProjectConfigV2Dep, markdown_processor: MarkdownProcessorV2Dep
|
||||
) -> FileService:
|
||||
logger.debug(
|
||||
f"Creating FileService for project: {project_config.name}, base_path: {project_config.home}"
|
||||
)
|
||||
file_service = FileService(project_config.home, markdown_processor)
|
||||
logger.debug(f"Created FileService for project: {file_service} ")
|
||||
logger.debug(
|
||||
f"Created FileService for project: {project_config.name}, base_path: {project_config.home}"
|
||||
)
|
||||
return file_service
|
||||
|
||||
|
||||
|
||||
@@ -1,6 +1,8 @@
|
||||
"""Utilities for file operations."""
|
||||
|
||||
import hashlib
|
||||
from dataclasses import dataclass
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
import re
|
||||
from typing import Any, Dict, Union
|
||||
@@ -13,6 +15,20 @@ from loguru import logger
|
||||
from basic_memory.utils import FilePath
|
||||
|
||||
|
||||
@dataclass
|
||||
class FileMetadata:
|
||||
"""File metadata for cloud-compatible file operations.
|
||||
|
||||
This dataclass provides a cloud-agnostic way to represent file metadata,
|
||||
enabling S3FileService to return metadata from head_object responses
|
||||
instead of mock stat_result with zeros.
|
||||
"""
|
||||
|
||||
size: int
|
||||
created_at: datetime
|
||||
modified_at: datetime
|
||||
|
||||
|
||||
class FileError(Exception):
|
||||
"""Base exception for file operations."""
|
||||
|
||||
|
||||
@@ -23,6 +23,7 @@ from basic_memory.markdown.schemas import (
|
||||
)
|
||||
from basic_memory.utils import parse_tags
|
||||
|
||||
|
||||
md = MarkdownIt().use(observation_plugin).use(relation_plugin)
|
||||
|
||||
|
||||
|
||||
@@ -5,6 +5,7 @@ from collections import OrderedDict
|
||||
from frontmatter import Post
|
||||
from loguru import logger
|
||||
|
||||
|
||||
from basic_memory import file_utils
|
||||
from basic_memory.file_utils import dump_frontmatter
|
||||
from basic_memory.markdown.entity_parser import EntityParser
|
||||
|
||||
@@ -30,7 +30,9 @@ def is_observation(token: Token) -> bool:
|
||||
|
||||
# Check for proper observation format: [category] content
|
||||
match = re.match(r"^\[([^\[\]()]+)\]\s+(.+)", content)
|
||||
has_tags = "#" in content
|
||||
# Check for standalone hashtags (words starting with #)
|
||||
# This excludes # in HTML attributes like color="#4285F4"
|
||||
has_tags = any(part.startswith("#") for part in content.split())
|
||||
return bool(match) or has_tags
|
||||
|
||||
|
||||
@@ -160,7 +162,7 @@ def parse_inline_relations(content: str) -> List[Dict[str, Any]]:
|
||||
|
||||
target = content[start + 2 : end].strip()
|
||||
if target:
|
||||
relations.append({"type": "links to", "target": target, "context": None})
|
||||
relations.append({"type": "links_to", "target": target, "context": None})
|
||||
|
||||
start = end + 2
|
||||
|
||||
|
||||
@@ -3,6 +3,7 @@
|
||||
from pathlib import Path
|
||||
from typing import Any, Optional
|
||||
|
||||
|
||||
from frontmatter import Post
|
||||
|
||||
from basic_memory.file_utils import has_frontmatter, remove_frontmatter, parse_frontmatter
|
||||
@@ -12,7 +13,10 @@ from basic_memory.models import Observation as ObservationModel
|
||||
|
||||
|
||||
def entity_model_from_markdown(
|
||||
file_path: Path, markdown: EntityMarkdown, entity: Optional[Entity] = None
|
||||
file_path: Path,
|
||||
markdown: EntityMarkdown,
|
||||
entity: Optional[Entity] = None,
|
||||
project_id: Optional[int] = None,
|
||||
) -> Entity:
|
||||
"""
|
||||
Convert markdown entity to model. Does not include relations.
|
||||
@@ -21,6 +25,7 @@ def entity_model_from_markdown(
|
||||
file_path: Path to the markdown file
|
||||
markdown: Parsed markdown entity
|
||||
entity: Optional existing entity to update
|
||||
project_id: Project ID for new observations (uses entity.project_id if not provided)
|
||||
|
||||
Returns:
|
||||
Entity model populated from markdown
|
||||
@@ -50,9 +55,13 @@ def entity_model_from_markdown(
|
||||
metadata = markdown.frontmatter.metadata or {}
|
||||
model.entity_metadata = {k: str(v) for k, v in metadata.items() if v is not None}
|
||||
|
||||
# Get project_id from entity if not provided
|
||||
obs_project_id = project_id or (model.project_id if hasattr(model, "project_id") else None)
|
||||
|
||||
# Convert observations
|
||||
model.observations = [
|
||||
ObservationModel(
|
||||
project_id=obs_project_id,
|
||||
content=obs.content,
|
||||
category=obs.category,
|
||||
context=obs.context,
|
||||
|
||||
@@ -4,7 +4,6 @@ import basic_memory
|
||||
from basic_memory.models.base import Base
|
||||
from basic_memory.models.knowledge import Entity, Observation, Relation
|
||||
from basic_memory.models.project import Project
|
||||
from basic_memory.models.search import SearchIndex
|
||||
|
||||
__all__ = [
|
||||
"Base",
|
||||
@@ -12,6 +11,5 @@ __all__ = [
|
||||
"Observation",
|
||||
"Relation",
|
||||
"Project",
|
||||
"SearchIndex",
|
||||
"basic_memory",
|
||||
]
|
||||
|
||||
@@ -145,6 +145,7 @@ class Observation(Base):
|
||||
)
|
||||
|
||||
id: Mapped[int] = mapped_column(Integer, primary_key=True)
|
||||
project_id: Mapped[int] = mapped_column(Integer, ForeignKey("project.id"), index=True)
|
||||
entity_id: Mapped[int] = mapped_column(Integer, ForeignKey("entity.id", ondelete="CASCADE"))
|
||||
content: Mapped[str] = mapped_column(Text)
|
||||
category: Mapped[str] = mapped_column(String, nullable=False, default="note")
|
||||
@@ -162,9 +163,14 @@ class Observation(Base):
|
||||
|
||||
We can construct these because observations are always defined in
|
||||
and owned by a single entity.
|
||||
|
||||
Content is truncated to 200 chars to stay under PostgreSQL's
|
||||
btree index limit of 2704 bytes.
|
||||
"""
|
||||
# Truncate content to avoid exceeding PostgreSQL's btree index limit
|
||||
content_for_permalink = self.content[:200] if len(self.content) > 200 else self.content
|
||||
return generate_permalink(
|
||||
f"{self.entity.permalink}/observations/{self.category}/{self.content}"
|
||||
f"{self.entity.permalink}/observations/{self.category}/{content_for_permalink}"
|
||||
)
|
||||
|
||||
def __repr__(self) -> str: # pragma: no cover
|
||||
@@ -186,6 +192,7 @@ class Relation(Base):
|
||||
)
|
||||
|
||||
id: Mapped[int] = mapped_column(Integer, primary_key=True)
|
||||
project_id: Mapped[int] = mapped_column(Integer, ForeignKey("project.id"), index=True)
|
||||
from_id: Mapped[int] = mapped_column(Integer, ForeignKey("entity.id", ondelete="CASCADE"))
|
||||
to_id: Mapped[Optional[int]] = mapped_column(
|
||||
Integer, ForeignKey("entity.id", ondelete="CASCADE"), nullable=True
|
||||
|
||||
@@ -1,53 +1,52 @@
|
||||
"""Search models and tables."""
|
||||
"""Search DDL statements for SQLite and Postgres.
|
||||
|
||||
from sqlalchemy import DDL, Column, Integer, String, DateTime, Text
|
||||
from sqlalchemy.dialects.postgresql import JSONB
|
||||
from sqlalchemy.types import JSON
|
||||
The search_index table is created via raw DDL, not ORM models, because:
|
||||
- SQLite uses FTS5 virtual tables (cannot be represented as ORM)
|
||||
- Postgres uses composite primary keys and generated tsvector columns
|
||||
- Both backends use raw SQL for all search operations via SearchIndexRow dataclass
|
||||
"""
|
||||
|
||||
from basic_memory.models.base import Base
|
||||
from sqlalchemy import DDL
|
||||
|
||||
|
||||
class SearchIndex(Base):
|
||||
"""Search index table for Postgres only.
|
||||
# Define Postgres search_index table with composite primary key and tsvector
|
||||
# This DDL matches the Alembic migration schema (314f1ea54dc4)
|
||||
# Used by tests to create the table without running full migrations
|
||||
# NOTE: Split into separate DDL statements because asyncpg doesn't support
|
||||
# multiple statements in a single execute call.
|
||||
CREATE_POSTGRES_SEARCH_INDEX_TABLE = DDL("""
|
||||
CREATE TABLE IF NOT EXISTS search_index (
|
||||
id INTEGER NOT NULL,
|
||||
project_id INTEGER NOT NULL,
|
||||
title TEXT,
|
||||
content_stems TEXT,
|
||||
content_snippet TEXT,
|
||||
permalink VARCHAR,
|
||||
file_path VARCHAR,
|
||||
type VARCHAR,
|
||||
from_id INTEGER,
|
||||
to_id INTEGER,
|
||||
relation_type VARCHAR,
|
||||
entity_id INTEGER,
|
||||
category VARCHAR,
|
||||
metadata JSONB,
|
||||
created_at TIMESTAMP WITH TIME ZONE,
|
||||
updated_at TIMESTAMP WITH TIME ZONE,
|
||||
textsearchable_index_col tsvector GENERATED ALWAYS AS (
|
||||
to_tsvector('english', coalesce(title, '') || ' ' || coalesce(content_stems, ''))
|
||||
) STORED,
|
||||
PRIMARY KEY (id, type, project_id),
|
||||
FOREIGN KEY (project_id) REFERENCES project(id) ON DELETE CASCADE
|
||||
)
|
||||
""")
|
||||
|
||||
For SQLite: This model is skipped; FTS5 virtual table is created via DDL instead.
|
||||
For Postgres: This is the actual table structure with tsvector support.
|
||||
"""
|
||||
|
||||
__tablename__ = "search_index"
|
||||
|
||||
# Primary key (rowid in SQLite FTS5, explicit id in Postgres)
|
||||
id = Column(Integer, primary_key=True, autoincrement=True)
|
||||
|
||||
# Core searchable fields
|
||||
title = Column(Text, nullable=True)
|
||||
content_stems = Column(Text, nullable=True)
|
||||
content_snippet = Column(Text, nullable=True)
|
||||
permalink = Column(String(255), nullable=True, index=True)
|
||||
file_path = Column(Text, nullable=True)
|
||||
type = Column(String(50), nullable=True)
|
||||
|
||||
# Project context
|
||||
project_id = Column(Integer, nullable=True, index=True)
|
||||
|
||||
# Relation fields
|
||||
from_id = Column(Integer, nullable=True)
|
||||
to_id = Column(Integer, nullable=True)
|
||||
relation_type = Column(String(100), nullable=True)
|
||||
|
||||
# Observation fields
|
||||
entity_id = Column(Integer, nullable=True)
|
||||
category = Column(String(100), nullable=True)
|
||||
|
||||
# Common fields
|
||||
# Use JSONB for Postgres, JSON for SQLite
|
||||
# Note: 'metadata' is a reserved name in SQLAlchemy, so we use 'metadata_' and map to 'metadata'
|
||||
metadata_ = Column("metadata", JSON().with_variant(JSONB(), "postgresql"), nullable=True)
|
||||
created_at = Column(DateTime(timezone=True), nullable=True)
|
||||
updated_at = Column(DateTime(timezone=True), nullable=True)
|
||||
|
||||
# Note: textsearchable_index_col (tsvector) will be added by migration for Postgres only
|
||||
CREATE_POSTGRES_SEARCH_INDEX_FTS = DDL("""
|
||||
CREATE INDEX IF NOT EXISTS idx_search_index_fts ON search_index USING gin(textsearchable_index_col)
|
||||
""")
|
||||
|
||||
CREATE_POSTGRES_SEARCH_INDEX_METADATA = DDL("""
|
||||
CREATE INDEX IF NOT EXISTS idx_search_index_metadata_gin ON search_index USING gin(metadata jsonb_path_ops)
|
||||
""")
|
||||
|
||||
# Define FTS5 virtual table creation for SQLite only
|
||||
# This DDL is executed separately for SQLite databases
|
||||
|
||||
@@ -3,6 +3,7 @@
|
||||
from pathlib import Path
|
||||
from typing import List, Optional, Sequence, Union, Any
|
||||
|
||||
|
||||
from loguru import logger
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.exc import IntegrityError
|
||||
@@ -76,6 +77,99 @@ class EntityRepository(Repository[Entity]):
|
||||
)
|
||||
return await self.find_one(query)
|
||||
|
||||
# -------------------------------------------------------------------------
|
||||
# Lightweight methods for permalink resolution (no eager loading)
|
||||
# -------------------------------------------------------------------------
|
||||
|
||||
async def permalink_exists(self, permalink: str) -> bool:
|
||||
"""Check if a permalink exists without loading the full entity.
|
||||
|
||||
This is much faster than get_by_permalink() as it skips eager loading
|
||||
of observations and relations. Use for existence checks in bulk operations.
|
||||
|
||||
Args:
|
||||
permalink: Permalink to check
|
||||
|
||||
Returns:
|
||||
True if permalink exists, False otherwise
|
||||
"""
|
||||
query = select(Entity.id).where(Entity.permalink == permalink).limit(1)
|
||||
query = self._add_project_filter(query)
|
||||
result = await self.execute_query(query, use_query_options=False)
|
||||
return result.scalar_one_or_none() is not None
|
||||
|
||||
async def get_file_path_for_permalink(self, permalink: str) -> Optional[str]:
|
||||
"""Get the file_path for a permalink without loading the full entity.
|
||||
|
||||
Use when you only need the file_path, not the full entity with relations.
|
||||
|
||||
Args:
|
||||
permalink: Permalink to look up
|
||||
|
||||
Returns:
|
||||
file_path string if found, None otherwise
|
||||
"""
|
||||
query = select(Entity.file_path).where(Entity.permalink == permalink)
|
||||
query = self._add_project_filter(query)
|
||||
result = await self.execute_query(query, use_query_options=False)
|
||||
return result.scalar_one_or_none()
|
||||
|
||||
async def get_permalink_for_file_path(self, file_path: Union[Path, str]) -> Optional[str]:
|
||||
"""Get the permalink for a file_path without loading the full entity.
|
||||
|
||||
Use when you only need the permalink, not the full entity with relations.
|
||||
|
||||
Args:
|
||||
file_path: File path to look up
|
||||
|
||||
Returns:
|
||||
permalink string if found, None otherwise
|
||||
"""
|
||||
query = select(Entity.permalink).where(Entity.file_path == Path(file_path).as_posix())
|
||||
query = self._add_project_filter(query)
|
||||
result = await self.execute_query(query, use_query_options=False)
|
||||
return result.scalar_one_or_none()
|
||||
|
||||
async def get_all_permalinks(self) -> List[str]:
|
||||
"""Get all permalinks for this project.
|
||||
|
||||
Optimized for bulk operations - returns only permalink strings
|
||||
without loading entities or relationships.
|
||||
|
||||
Returns:
|
||||
List of all permalinks in the project
|
||||
"""
|
||||
query = select(Entity.permalink)
|
||||
query = self._add_project_filter(query)
|
||||
result = await self.execute_query(query, use_query_options=False)
|
||||
return list(result.scalars().all())
|
||||
|
||||
async def get_permalink_to_file_path_map(self) -> dict[str, str]:
|
||||
"""Get a mapping of permalink -> file_path for all entities.
|
||||
|
||||
Optimized for bulk permalink resolution - loads minimal data in one query.
|
||||
|
||||
Returns:
|
||||
Dict mapping permalink to file_path
|
||||
"""
|
||||
query = select(Entity.permalink, Entity.file_path)
|
||||
query = self._add_project_filter(query)
|
||||
result = await self.execute_query(query, use_query_options=False)
|
||||
return {row.permalink: row.file_path for row in result.all()}
|
||||
|
||||
async def get_file_path_to_permalink_map(self) -> dict[str, str]:
|
||||
"""Get a mapping of file_path -> permalink for all entities.
|
||||
|
||||
Optimized for bulk permalink resolution - loads minimal data in one query.
|
||||
|
||||
Returns:
|
||||
Dict mapping file_path to permalink
|
||||
"""
|
||||
query = select(Entity.file_path, Entity.permalink)
|
||||
query = self._add_project_filter(query)
|
||||
result = await self.execute_query(query, use_query_options=False)
|
||||
return {row.file_path: row.permalink for row in result.all()}
|
||||
|
||||
async def get_by_file_paths(
|
||||
self, session: AsyncSession, file_paths: Sequence[Union[Path, str]]
|
||||
) -> List[Row[Any]]:
|
||||
|
||||
@@ -2,6 +2,7 @@
|
||||
|
||||
from typing import Dict, List, Sequence
|
||||
|
||||
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.ext.asyncio import async_sessionmaker
|
||||
|
||||
|
||||
@@ -5,6 +5,7 @@ import re
|
||||
from datetime import datetime
|
||||
from typing import List, Optional
|
||||
|
||||
|
||||
from loguru import logger
|
||||
from sqlalchemy import text
|
||||
|
||||
@@ -257,7 +258,7 @@ class PostgresSearchRepository(SearchRepositoryBase):
|
||||
{score_expr} as score
|
||||
FROM search_index
|
||||
WHERE {where_clause}
|
||||
ORDER BY score DESC {order_by_clause}
|
||||
ORDER BY score DESC, id ASC {order_by_clause}
|
||||
LIMIT :limit
|
||||
OFFSET :offset
|
||||
"""
|
||||
|
||||
@@ -3,6 +3,7 @@
|
||||
from pathlib import Path
|
||||
from typing import Optional, Sequence, Union
|
||||
|
||||
|
||||
from sqlalchemy import text
|
||||
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
|
||||
|
||||
|
||||
@@ -1,9 +1,11 @@
|
||||
"""Repository for managing Relation objects."""
|
||||
|
||||
from sqlalchemy import and_, delete
|
||||
from typing import Sequence, List, Optional
|
||||
|
||||
from sqlalchemy import select
|
||||
|
||||
from sqlalchemy import and_, delete, select
|
||||
from sqlalchemy.dialects.postgresql import insert as pg_insert
|
||||
from sqlalchemy.dialects.sqlite import insert as sqlite_insert
|
||||
from sqlalchemy.ext.asyncio import async_sessionmaker
|
||||
from sqlalchemy.orm import selectinload, aliased
|
||||
from sqlalchemy.orm.interfaces import LoaderOption
|
||||
@@ -86,5 +88,59 @@ class RelationRepository(Repository[Relation]):
|
||||
result = await self.execute_query(query)
|
||||
return result.scalars().all()
|
||||
|
||||
async def add_all_ignore_duplicates(self, relations: List[Relation]) -> int:
|
||||
"""Bulk insert relations, ignoring duplicates.
|
||||
|
||||
Uses ON CONFLICT DO NOTHING to skip relations that would violate the
|
||||
unique constraint on (from_id, to_name, relation_type). This is useful
|
||||
for bulk operations where the same link may appear multiple times in
|
||||
a document.
|
||||
|
||||
Works with both SQLite and PostgreSQL dialects.
|
||||
|
||||
Args:
|
||||
relations: List of Relation objects to insert
|
||||
|
||||
Returns:
|
||||
Number of relations actually inserted (excludes duplicates)
|
||||
"""
|
||||
if not relations:
|
||||
return 0
|
||||
|
||||
# Convert Relation objects to dicts for insert
|
||||
values = [
|
||||
{
|
||||
"project_id": r.project_id if r.project_id else self.project_id,
|
||||
"from_id": r.from_id,
|
||||
"to_id": r.to_id,
|
||||
"to_name": r.to_name,
|
||||
"relation_type": r.relation_type,
|
||||
"context": r.context,
|
||||
}
|
||||
for r in relations
|
||||
]
|
||||
|
||||
async with db.scoped_session(self.session_maker) as session:
|
||||
# Check dialect to use appropriate insert
|
||||
dialect_name = session.bind.dialect.name if session.bind else "sqlite"
|
||||
|
||||
if dialect_name == "postgresql":
|
||||
# PostgreSQL: use RETURNING to count inserted rows
|
||||
# (rowcount is 0 for ON CONFLICT DO NOTHING)
|
||||
stmt = (
|
||||
pg_insert(Relation)
|
||||
.values(values)
|
||||
.on_conflict_do_nothing()
|
||||
.returning(Relation.id)
|
||||
)
|
||||
result = await session.execute(stmt)
|
||||
return len(result.fetchall())
|
||||
else:
|
||||
# SQLite: rowcount works correctly
|
||||
stmt = sqlite_insert(Relation).values(values)
|
||||
stmt = stmt.on_conflict_do_nothing()
|
||||
result = await session.execute(stmt)
|
||||
return result.rowcount if result.rowcount > 0 else 0
|
||||
|
||||
def get_load_options(self) -> List[LoaderOption]:
|
||||
return [selectinload(Relation.from_entity), selectinload(Relation.to_entity)]
|
||||
|
||||
@@ -2,6 +2,7 @@
|
||||
|
||||
from typing import Type, Optional, Any, Sequence, TypeVar, List, Dict
|
||||
|
||||
|
||||
from loguru import logger
|
||||
from sqlalchemy import (
|
||||
select,
|
||||
|
||||
@@ -4,6 +4,7 @@ from abc import ABC, abstractmethod
|
||||
from datetime import datetime
|
||||
from typing import Any, Dict, List, Optional
|
||||
|
||||
|
||||
from loguru import logger
|
||||
from sqlalchemy import Executable, Result, text
|
||||
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
|
||||
|
||||
@@ -5,6 +5,7 @@ import re
|
||||
from datetime import datetime
|
||||
from typing import List, Optional
|
||||
|
||||
|
||||
from loguru import logger
|
||||
from sqlalchemy import text
|
||||
|
||||
|
||||
@@ -4,6 +4,7 @@ from dataclasses import dataclass, field
|
||||
from datetime import datetime, timezone
|
||||
from typing import List, Optional, Tuple
|
||||
|
||||
|
||||
from loguru import logger
|
||||
from sqlalchemy import text
|
||||
|
||||
|
||||
@@ -3,8 +3,10 @@
|
||||
import fnmatch
|
||||
import logging
|
||||
import os
|
||||
from datetime import datetime
|
||||
from typing import Dict, List, Optional, Sequence
|
||||
|
||||
|
||||
from basic_memory.models import Entity
|
||||
from basic_memory.repository import EntityRepository
|
||||
from basic_memory.schemas.directory import DirectoryNode
|
||||
@@ -12,6 +14,17 @@ from basic_memory.schemas.directory import DirectoryNode
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def _mtime_to_datetime(entity: Entity) -> datetime:
|
||||
"""Convert entity mtime (file modification time) to datetime.
|
||||
|
||||
Returns the file's actual modification time, falling back to updated_at
|
||||
if mtime is not available.
|
||||
"""
|
||||
if entity.mtime:
|
||||
return datetime.fromtimestamp(entity.mtime).astimezone()
|
||||
return entity.updated_at
|
||||
|
||||
|
||||
class DirectoryService:
|
||||
"""Service for working with directory trees."""
|
||||
|
||||
@@ -77,7 +90,7 @@ class DirectoryService:
|
||||
entity_id=file.id,
|
||||
entity_type=file.entity_type,
|
||||
content_type=file.content_type,
|
||||
updated_at=file.updated_at,
|
||||
updated_at=_mtime_to_datetime(file),
|
||||
)
|
||||
|
||||
# Add to parent directory's children
|
||||
@@ -241,7 +254,7 @@ class DirectoryService:
|
||||
entity_id=file.id,
|
||||
entity_type=file.entity_type,
|
||||
content_type=file.content_type,
|
||||
updated_at=file.updated_at,
|
||||
updated_at=_mtime_to_datetime(file),
|
||||
)
|
||||
|
||||
# Add to parent directory's children
|
||||
|
||||
@@ -8,6 +8,7 @@ import yaml
|
||||
from loguru import logger
|
||||
from sqlalchemy.exc import IntegrityError
|
||||
|
||||
|
||||
from basic_memory.config import ProjectConfig, BasicMemoryConfig
|
||||
from basic_memory.file_utils import (
|
||||
has_frontmatter,
|
||||
@@ -106,6 +107,9 @@ class EntityService(BaseService[EntityModel]):
|
||||
4. Generate new unique permalink from file path
|
||||
|
||||
Enhanced to detect and handle character-related conflicts.
|
||||
|
||||
Note: Uses lightweight repository methods that skip eager loading of
|
||||
observations and relations for better performance during bulk operations.
|
||||
"""
|
||||
file_path_str = Path(file_path).as_posix()
|
||||
|
||||
@@ -122,16 +126,20 @@ class EntityService(BaseService[EntityModel]):
|
||||
# If markdown has explicit permalink, try to validate it
|
||||
if markdown and markdown.frontmatter.permalink:
|
||||
desired_permalink = markdown.frontmatter.permalink
|
||||
existing = await self.repository.get_by_permalink(desired_permalink)
|
||||
# Use lightweight method - we only need to check file_path
|
||||
existing_file_path = await self.repository.get_file_path_for_permalink(
|
||||
desired_permalink
|
||||
)
|
||||
|
||||
# If no conflict or it's our own file, use as is
|
||||
if not existing or existing.file_path == file_path_str:
|
||||
if not existing_file_path or existing_file_path == file_path_str:
|
||||
return desired_permalink
|
||||
|
||||
# For existing files, try to find current permalink
|
||||
existing = await self.repository.get_by_file_path(file_path_str)
|
||||
if existing:
|
||||
return existing.permalink
|
||||
# Use lightweight method - we only need the permalink
|
||||
existing_permalink = await self.repository.get_permalink_for_file_path(file_path_str)
|
||||
if existing_permalink:
|
||||
return existing_permalink
|
||||
|
||||
# New file - generate permalink
|
||||
if markdown and markdown.frontmatter.permalink:
|
||||
@@ -140,9 +148,10 @@ class EntityService(BaseService[EntityModel]):
|
||||
desired_permalink = generate_permalink(file_path_str)
|
||||
|
||||
# Make unique if needed - enhanced to handle character conflicts
|
||||
# Use lightweight existence check instead of loading full entity
|
||||
permalink = desired_permalink
|
||||
suffix = 1
|
||||
while await self.repository.get_by_permalink(permalink):
|
||||
while await self.repository.permalink_exists(permalink):
|
||||
permalink = f"{desired_permalink}-{suffix}"
|
||||
suffix += 1
|
||||
logger.debug(f"creating unique permalink: {permalink}")
|
||||
@@ -224,8 +233,11 @@ class EntityService(BaseService[EntityModel]):
|
||||
final_content = dump_frontmatter(post)
|
||||
checksum = await self.file_service.write_file(file_path, final_content)
|
||||
|
||||
# parse entity from file
|
||||
entity_markdown = await self.entity_parser.parse_file(file_path)
|
||||
# parse entity from content we just wrote (avoids re-reading file for cloud compatibility)
|
||||
entity_markdown = await self.entity_parser.parse_markdown_content(
|
||||
file_path=file_path,
|
||||
content=final_content,
|
||||
)
|
||||
|
||||
# create entity
|
||||
created = await self.create_entity_from_markdown(file_path, entity_markdown)
|
||||
@@ -245,8 +257,12 @@ class EntityService(BaseService[EntityModel]):
|
||||
# Convert file path string to Path
|
||||
file_path = Path(entity.file_path)
|
||||
|
||||
# Read existing frontmatter from the file if it exists
|
||||
existing_markdown = await self.entity_parser.parse_file(file_path)
|
||||
# Read existing content via file_service (for cloud compatibility)
|
||||
existing_content = await self.file_service.read_file_content(file_path)
|
||||
existing_markdown = await self.entity_parser.parse_markdown_content(
|
||||
file_path=file_path,
|
||||
content=existing_content,
|
||||
)
|
||||
|
||||
# Parse content frontmatter to check for user-specified permalink and entity_type
|
||||
content_markdown = None
|
||||
@@ -302,8 +318,11 @@ class EntityService(BaseService[EntityModel]):
|
||||
final_content = dump_frontmatter(merged_post)
|
||||
checksum = await self.file_service.write_file(file_path, final_content)
|
||||
|
||||
# parse entity from file
|
||||
entity_markdown = await self.entity_parser.parse_file(file_path)
|
||||
# parse entity from content we just wrote (avoids re-reading file for cloud compatibility)
|
||||
entity_markdown = await self.entity_parser.parse_markdown_content(
|
||||
file_path=file_path,
|
||||
content=final_content,
|
||||
)
|
||||
|
||||
# update entity in db
|
||||
entity = await self.update_entity_and_observations(file_path, entity_markdown)
|
||||
@@ -378,7 +397,9 @@ class EntityService(BaseService[EntityModel]):
|
||||
Uses UPSERT approach to handle permalink/file_path conflicts cleanly.
|
||||
"""
|
||||
logger.debug(f"Creating entity: {markdown.frontmatter.title} file_path: {file_path}")
|
||||
model = entity_model_from_markdown(file_path, markdown)
|
||||
model = entity_model_from_markdown(
|
||||
file_path, markdown, project_id=self.repository.project_id
|
||||
)
|
||||
|
||||
# Mark as incomplete because we still need to add relations
|
||||
model.checksum = None
|
||||
@@ -408,6 +429,7 @@ class EntityService(BaseService[EntityModel]):
|
||||
# add new observations
|
||||
observations = [
|
||||
Observation(
|
||||
project_id=self.observation_repository.project_id,
|
||||
entity_id=db_entity.id,
|
||||
content=obs.content,
|
||||
category=obs.category,
|
||||
@@ -474,6 +496,7 @@ class EntityService(BaseService[EntityModel]):
|
||||
|
||||
# Create the relation
|
||||
relation = Relation(
|
||||
project_id=self.relation_repository.project_id,
|
||||
from_id=db_entity.id,
|
||||
to_id=target_id,
|
||||
to_name=target_name,
|
||||
@@ -546,8 +569,11 @@ class EntityService(BaseService[EntityModel]):
|
||||
# Write the updated content back to the file
|
||||
checksum = await self.file_service.write_file(file_path, new_content)
|
||||
|
||||
# Parse the updated file to get new observations/relations
|
||||
entity_markdown = await self.entity_parser.parse_file(file_path)
|
||||
# Parse the content we just wrote (avoids re-reading file for cloud compatibility)
|
||||
entity_markdown = await self.entity_parser.parse_markdown_content(
|
||||
file_path=file_path,
|
||||
content=new_content,
|
||||
)
|
||||
|
||||
# Update entity and its relationships
|
||||
entity = await self.update_entity_and_observations(file_path, entity_markdown)
|
||||
@@ -766,23 +792,20 @@ class EntityService(BaseService[EntityModel]):
|
||||
raise ValueError(f"Invalid destination path: {destination_path}")
|
||||
|
||||
# 3. Validate paths
|
||||
source_file = project_config.home / current_path
|
||||
destination_file = project_config.home / destination_path
|
||||
|
||||
# Validate source exists
|
||||
if not source_file.exists():
|
||||
# NOTE: In tenantless/cloud mode, we cannot rely on local filesystem paths.
|
||||
# Use FileService for existence checks and moving.
|
||||
if not await self.file_service.exists(current_path):
|
||||
raise ValueError(f"Source file not found: {current_path}")
|
||||
|
||||
# Check if destination already exists
|
||||
if destination_file.exists():
|
||||
if await self.file_service.exists(destination_path):
|
||||
raise ValueError(f"Destination already exists: {destination_path}")
|
||||
|
||||
try:
|
||||
# 4. Create destination directory if needed
|
||||
destination_file.parent.mkdir(parents=True, exist_ok=True)
|
||||
# 4. Ensure destination directory if needed (no-op for S3)
|
||||
await self.file_service.ensure_directory(Path(destination_path).parent)
|
||||
|
||||
# 5. Move physical file
|
||||
source_file.rename(destination_file)
|
||||
# 5. Move physical file via FileService (filesystem rename or cloud move)
|
||||
await self.file_service.move_file(current_path, destination_path)
|
||||
logger.info(f"Moved file: {current_path} -> {destination_path}")
|
||||
|
||||
# 6. Prepare database updates
|
||||
@@ -821,12 +844,14 @@ class EntityService(BaseService[EntityModel]):
|
||||
|
||||
except Exception as e:
|
||||
# Rollback: try to restore original file location if move succeeded
|
||||
if destination_file.exists() and not source_file.exists():
|
||||
try:
|
||||
destination_file.rename(source_file)
|
||||
try:
|
||||
if await self.file_service.exists(
|
||||
destination_path
|
||||
) and not await self.file_service.exists(current_path):
|
||||
await self.file_service.move_file(destination_path, current_path)
|
||||
logger.info(f"Rolled back file move: {destination_path} -> {current_path}")
|
||||
except Exception as rollback_error: # pragma: no cover
|
||||
logger.error(f"Failed to rollback file move: {rollback_error}")
|
||||
except Exception as rollback_error: # pragma: no cover
|
||||
logger.error(f"Failed to rollback file move: {rollback_error}")
|
||||
|
||||
# Re-raise the original error with context
|
||||
raise ValueError(f"Move failed: {str(e)}") from e
|
||||
|
||||
@@ -3,15 +3,16 @@
|
||||
import asyncio
|
||||
import hashlib
|
||||
import mimetypes
|
||||
from os import stat_result
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
from typing import Any, Dict, Tuple, Union
|
||||
|
||||
import aiofiles
|
||||
|
||||
import yaml
|
||||
|
||||
from basic_memory import file_utils
|
||||
from basic_memory.file_utils import FileError, ParseError
|
||||
from basic_memory.file_utils import FileError, FileMetadata, ParseError
|
||||
from basic_memory.markdown.markdown_processor import MarkdownProcessor
|
||||
from basic_memory.models import Entity as EntityModel
|
||||
from basic_memory.schemas import Entity as EntitySchema
|
||||
@@ -220,6 +221,41 @@ class FileService:
|
||||
logger.exception("File read error", path=str(full_path), error=str(e))
|
||||
raise FileOperationError(f"Failed to read file: {e}")
|
||||
|
||||
async def read_file_bytes(self, path: FilePath) -> bytes:
|
||||
"""Read file content as bytes using true async I/O with aiofiles.
|
||||
|
||||
This method reads files in binary mode, suitable for non-text files
|
||||
like images, PDFs, etc. For cloud compatibility with S3FileService.
|
||||
|
||||
Args:
|
||||
path: Path to read (Path or string)
|
||||
|
||||
Returns:
|
||||
File content as bytes
|
||||
|
||||
Raises:
|
||||
FileOperationError: If read fails
|
||||
"""
|
||||
# Convert string to Path if needed
|
||||
path_obj = self.base_path / path if isinstance(path, str) else path
|
||||
full_path = path_obj if path_obj.is_absolute() else self.base_path / path_obj
|
||||
|
||||
try:
|
||||
logger.debug("Reading file bytes", operation="read_file_bytes", path=str(full_path))
|
||||
async with aiofiles.open(full_path, mode="rb") as f:
|
||||
content = await f.read()
|
||||
|
||||
logger.debug(
|
||||
"File read completed",
|
||||
path=str(full_path),
|
||||
content_length=len(content),
|
||||
)
|
||||
return content
|
||||
|
||||
except Exception as e:
|
||||
logger.exception("File read error", path=str(full_path), error=str(e))
|
||||
raise FileOperationError(f"Failed to read file: {e}")
|
||||
|
||||
async def read_file(self, path: FilePath) -> Tuple[str, str]:
|
||||
"""Read file and compute checksum using true async I/O.
|
||||
|
||||
@@ -276,6 +312,43 @@ class FileService:
|
||||
full_path = path_obj if path_obj.is_absolute() else self.base_path / path_obj
|
||||
full_path.unlink(missing_ok=True)
|
||||
|
||||
async def move_file(self, source: FilePath, destination: FilePath) -> None:
|
||||
"""Move/rename a file from source to destination.
|
||||
|
||||
This method abstracts the underlying storage (filesystem vs cloud).
|
||||
Default implementation uses atomic filesystem rename, but cloud-backed
|
||||
implementations (e.g., S3) can override to copy+delete.
|
||||
|
||||
Args:
|
||||
source: Source path (relative to base_path or absolute)
|
||||
destination: Destination path (relative to base_path or absolute)
|
||||
|
||||
Raises:
|
||||
FileOperationError: If the move fails
|
||||
"""
|
||||
# Convert strings to Paths and resolve relative paths against base_path
|
||||
src_obj = self.base_path / source if isinstance(source, str) else source
|
||||
dst_obj = self.base_path / destination if isinstance(destination, str) else destination
|
||||
src_full = src_obj if src_obj.is_absolute() else self.base_path / src_obj
|
||||
dst_full = dst_obj if dst_obj.is_absolute() else self.base_path / dst_obj
|
||||
|
||||
try:
|
||||
# Ensure destination directory exists
|
||||
await self.ensure_directory(dst_full.parent)
|
||||
|
||||
# Use semaphore for concurrency control and run blocking rename in executor
|
||||
async with self._file_semaphore:
|
||||
loop = asyncio.get_event_loop()
|
||||
await loop.run_in_executor(None, lambda: src_full.rename(dst_full))
|
||||
except Exception as e:
|
||||
logger.exception(
|
||||
"File move error",
|
||||
source=str(src_full),
|
||||
destination=str(dst_full),
|
||||
error=str(e),
|
||||
)
|
||||
raise FileOperationError(f"Failed to move file {source} -> {destination}: {e}")
|
||||
|
||||
async def update_frontmatter(self, path: FilePath, updates: Dict[str, Any]) -> str:
|
||||
"""Update frontmatter fields in a file while preserving all content.
|
||||
|
||||
@@ -381,20 +454,31 @@ class FileService:
|
||||
logger.error("Failed to compute checksum", path=str(full_path), error=str(e))
|
||||
raise FileError(f"Failed to compute checksum for {path}: {e}")
|
||||
|
||||
def file_stats(self, path: FilePath) -> stat_result:
|
||||
"""Return file stats for a given path.
|
||||
async def get_file_metadata(self, path: FilePath) -> FileMetadata:
|
||||
"""Return file metadata for a given path.
|
||||
|
||||
This method is async to support cloud implementations (S3FileService)
|
||||
where file metadata requires async operations (head_object).
|
||||
|
||||
Args:
|
||||
path: Path to the file (Path or string)
|
||||
|
||||
Returns:
|
||||
File statistics
|
||||
FileMetadata with size, created_at, and modified_at
|
||||
"""
|
||||
# Convert string to Path if needed
|
||||
path_obj = self.base_path / path if isinstance(path, str) else path
|
||||
full_path = path_obj if path_obj.is_absolute() else self.base_path / path_obj
|
||||
# get file timestamps
|
||||
return full_path.stat()
|
||||
|
||||
# Run blocking stat() in thread pool to maintain async compatibility
|
||||
loop = asyncio.get_event_loop()
|
||||
stat_result = await loop.run_in_executor(None, full_path.stat)
|
||||
|
||||
return FileMetadata(
|
||||
size=stat_result.st_size,
|
||||
created_at=datetime.fromtimestamp(stat_result.st_ctime).astimezone(),
|
||||
modified_at=datetime.fromtimestamp(stat_result.st_mtime).astimezone(),
|
||||
)
|
||||
|
||||
def content_type(self, path: FilePath) -> str:
|
||||
"""Return content_type for a given path.
|
||||
|
||||
@@ -5,8 +5,10 @@ to ensure consistent application startup across all entry points.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import os
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
from loguru import logger
|
||||
|
||||
from basic_memory import db
|
||||
@@ -104,6 +106,12 @@ async def initialize_file_sync(
|
||||
# Get active projects
|
||||
active_projects = await project_repository.get_active_projects()
|
||||
|
||||
# Filter to constrained project if MCP server was started with --project
|
||||
constrained_project = os.environ.get("BASIC_MEMORY_MCP_PROJECT")
|
||||
if constrained_project:
|
||||
active_projects = [p for p in active_projects if p.name == constrained_project]
|
||||
logger.info(f"Background sync constrained to project: {constrained_project}")
|
||||
|
||||
# Start sync for all projects as background tasks (non-blocking)
|
||||
async def sync_project_background(project: Project):
|
||||
"""Sync a single project in the background."""
|
||||
|
||||
@@ -2,6 +2,7 @@
|
||||
|
||||
from typing import Optional, Tuple
|
||||
|
||||
|
||||
from loguru import logger
|
||||
|
||||
from basic_memory.models import Entity
|
||||
|
||||
@@ -8,6 +8,7 @@ from datetime import datetime
|
||||
from pathlib import Path
|
||||
from typing import Dict, Optional, Sequence
|
||||
|
||||
|
||||
from loguru import logger
|
||||
from sqlalchemy import text
|
||||
|
||||
@@ -23,9 +24,6 @@ from basic_memory.config import WATCH_STATUS_JSON, ConfigManager, get_project_co
|
||||
from basic_memory.utils import generate_permalink
|
||||
|
||||
|
||||
config = ConfigManager().config
|
||||
|
||||
|
||||
class ProjectService:
|
||||
"""Service for managing Basic Memory projects."""
|
||||
|
||||
@@ -143,6 +141,7 @@ class ProjectService:
|
||||
"""
|
||||
# If project_root is set, constrain all projects to that directory
|
||||
project_root = self.config_manager.config.project_root
|
||||
sanitized_name = None
|
||||
if project_root:
|
||||
base_path = Path(project_root)
|
||||
|
||||
@@ -199,14 +198,15 @@ class ProjectService:
|
||||
f"Projects cannot share directory trees."
|
||||
)
|
||||
|
||||
# First add to config file (this will validate the project doesn't exist)
|
||||
project_config = self.config_manager.add_project(name, resolved_path)
|
||||
if not self.config_manager.config.cloud_mode:
|
||||
# First add to config file (this will validate the project doesn't exist)
|
||||
self.config_manager.add_project(name, resolved_path)
|
||||
|
||||
# Then add to database
|
||||
project_data = {
|
||||
"name": name,
|
||||
"path": resolved_path,
|
||||
"permalink": generate_permalink(project_config.name),
|
||||
"permalink": sanitized_name,
|
||||
"is_active": True,
|
||||
# Don't set is_default=False to avoid UNIQUE constraint issues
|
||||
# Let it default to NULL, only set to True when explicitly making default
|
||||
|
||||
@@ -4,6 +4,7 @@ import ast
|
||||
from datetime import datetime
|
||||
from typing import List, Optional, Set
|
||||
|
||||
|
||||
from dateparser import parse
|
||||
from fastapi import BackgroundTasks
|
||||
from loguru import logger
|
||||
@@ -15,6 +16,21 @@ from basic_memory.repository.search_repository import SearchRepository, SearchIn
|
||||
from basic_memory.schemas.search import SearchQuery, SearchItemType
|
||||
from basic_memory.services import FileService
|
||||
|
||||
# Maximum size for content_stems field to stay under Postgres's 8KB index row limit.
|
||||
# We use 6000 characters to leave headroom for other indexed columns and overhead.
|
||||
MAX_CONTENT_STEMS_SIZE = 6000
|
||||
|
||||
|
||||
def _mtime_to_datetime(entity: Entity) -> datetime:
|
||||
"""Convert entity mtime (file modification time) to datetime.
|
||||
|
||||
Returns the file's actual modification time, falling back to updated_at
|
||||
if mtime is not available.
|
||||
"""
|
||||
if entity.mtime:
|
||||
return datetime.fromtimestamp(entity.mtime).astimezone()
|
||||
return entity.updated_at
|
||||
|
||||
|
||||
class SearchService:
|
||||
"""Service for search operations.
|
||||
@@ -193,7 +209,7 @@ class SearchService:
|
||||
"entity_type": entity.entity_type,
|
||||
},
|
||||
created_at=entity.created_at,
|
||||
updated_at=entity.updated_at,
|
||||
updated_at=_mtime_to_datetime(entity),
|
||||
project_id=entity.project_id,
|
||||
)
|
||||
)
|
||||
@@ -256,6 +272,10 @@ class SearchService:
|
||||
|
||||
entity_content_stems = "\n".join(p for p in content_stems if p and p.strip())
|
||||
|
||||
# Truncate to stay under Postgres's 8KB index row limit
|
||||
if len(entity_content_stems) > MAX_CONTENT_STEMS_SIZE:
|
||||
entity_content_stems = entity_content_stems[:MAX_CONTENT_STEMS_SIZE]
|
||||
|
||||
# Add entity row
|
||||
rows_to_index.append(
|
||||
SearchIndexRow(
|
||||
@@ -271,17 +291,28 @@ class SearchService:
|
||||
"entity_type": entity.entity_type,
|
||||
},
|
||||
created_at=entity.created_at,
|
||||
updated_at=entity.updated_at,
|
||||
updated_at=_mtime_to_datetime(entity),
|
||||
project_id=entity.project_id,
|
||||
)
|
||||
)
|
||||
|
||||
# Add observation rows
|
||||
# Add observation rows - dedupe by permalink to avoid unique constraint violations
|
||||
# Two observations with same entity/category/content generate identical permalinks
|
||||
seen_permalinks: set[str] = {entity.permalink} if entity.permalink else set()
|
||||
for obs in entity.observations:
|
||||
obs_permalink = obs.permalink
|
||||
if obs_permalink in seen_permalinks:
|
||||
logger.debug(f"Skipping duplicate observation permalink: {obs_permalink}")
|
||||
continue
|
||||
seen_permalinks.add(obs_permalink)
|
||||
|
||||
# Index with parent entity's file path since that's where it's defined
|
||||
obs_content_stems = "\n".join(
|
||||
p for p in self._generate_variants(obs.content) if p and p.strip()
|
||||
)
|
||||
# Truncate to stay under Postgres's 8KB index row limit
|
||||
if len(obs_content_stems) > MAX_CONTENT_STEMS_SIZE:
|
||||
obs_content_stems = obs_content_stems[:MAX_CONTENT_STEMS_SIZE]
|
||||
rows_to_index.append(
|
||||
SearchIndexRow(
|
||||
id=obs.id,
|
||||
@@ -289,7 +320,7 @@ class SearchService:
|
||||
title=f"{obs.category}: {obs.content[:100]}...",
|
||||
content_stems=obs_content_stems,
|
||||
content_snippet=obs.content,
|
||||
permalink=obs.permalink,
|
||||
permalink=obs_permalink,
|
||||
file_path=entity.file_path,
|
||||
category=obs.category,
|
||||
entity_id=entity.id,
|
||||
@@ -297,7 +328,7 @@ class SearchService:
|
||||
"tags": obs.tags,
|
||||
},
|
||||
created_at=entity.created_at,
|
||||
updated_at=entity.updated_at,
|
||||
updated_at=_mtime_to_datetime(entity),
|
||||
project_id=entity.project_id,
|
||||
)
|
||||
)
|
||||
@@ -327,7 +358,7 @@ class SearchService:
|
||||
to_id=rel.to_id,
|
||||
relation_type=rel.relation_type,
|
||||
created_at=entity.created_at,
|
||||
updated_at=entity.updated_at,
|
||||
updated_at=_mtime_to_datetime(entity),
|
||||
project_id=entity.project_id,
|
||||
)
|
||||
)
|
||||
|
||||
@@ -10,7 +10,7 @@ from pathlib import Path
|
||||
from typing import AsyncIterator, Dict, List, Optional, Set, Tuple
|
||||
|
||||
import aiofiles.os
|
||||
import logfire
|
||||
|
||||
from loguru import logger
|
||||
from sqlalchemy.exc import IntegrityError
|
||||
|
||||
@@ -215,17 +215,12 @@ class SyncService:
|
||||
f"path={path}, error={error}"
|
||||
)
|
||||
|
||||
# Record metric for file failure
|
||||
logfire.metric_counter("sync.circuit_breaker.failures").add(1)
|
||||
|
||||
# Log when threshold is reached
|
||||
if failure_info.count >= MAX_CONSECUTIVE_FAILURES:
|
||||
logger.error(
|
||||
f"File {path} has failed {MAX_CONSECUTIVE_FAILURES} times and will be skipped. "
|
||||
f"First failure: {failure_info.first_failure}, Last error: {error}"
|
||||
)
|
||||
# Record metric for file being blocked by circuit breaker
|
||||
logfire.metric_counter("sync.circuit_breaker.blocked_files").add(1)
|
||||
else:
|
||||
# Create new failure record
|
||||
self._file_failures[path] = FileFailureInfo(
|
||||
@@ -255,7 +250,6 @@ class SyncService:
|
||||
logger.info(f"Clearing failure history for {path} after successful sync")
|
||||
del self._file_failures[path]
|
||||
|
||||
@logfire.instrument()
|
||||
async def sync(
|
||||
self, directory: Path, project_name: Optional[str] = None, force_full: bool = False
|
||||
) -> SyncReport:
|
||||
@@ -282,63 +276,58 @@ class SyncService:
|
||||
)
|
||||
|
||||
# sync moves first
|
||||
with logfire.span("process_moves", move_count=len(report.moves)):
|
||||
for old_path, new_path in report.moves.items():
|
||||
# in the case where a file has been deleted and replaced by another file
|
||||
# it will show up in the move and modified lists, so handle it in modified
|
||||
if new_path in report.modified:
|
||||
report.modified.remove(new_path)
|
||||
logger.debug(
|
||||
f"File marked as moved and modified: old_path={old_path}, new_path={new_path}"
|
||||
)
|
||||
else:
|
||||
await self.handle_move(old_path, new_path)
|
||||
for old_path, new_path in report.moves.items():
|
||||
# in the case where a file has been deleted and replaced by another file
|
||||
# it will show up in the move and modified lists, so handle it in modified
|
||||
if new_path in report.modified:
|
||||
report.modified.remove(new_path)
|
||||
logger.debug(
|
||||
f"File marked as moved and modified: old_path={old_path}, new_path={new_path}"
|
||||
)
|
||||
else:
|
||||
await self.handle_move(old_path, new_path)
|
||||
|
||||
# deleted next
|
||||
with logfire.span("process_deletes", delete_count=len(report.deleted)):
|
||||
for path in report.deleted:
|
||||
await self.handle_delete(path)
|
||||
for path in report.deleted:
|
||||
await self.handle_delete(path)
|
||||
|
||||
# then new and modified
|
||||
with logfire.span("process_new_files", new_count=len(report.new)):
|
||||
for path in report.new:
|
||||
entity, _ = await self.sync_file(path, new=True)
|
||||
for path in report.new:
|
||||
entity, _ = await self.sync_file(path, new=True)
|
||||
|
||||
# Track if file was skipped
|
||||
if entity is None and await self._should_skip_file(path):
|
||||
failure_info = self._file_failures[path]
|
||||
report.skipped_files.append(
|
||||
SkippedFile(
|
||||
path=path,
|
||||
reason=failure_info.last_error,
|
||||
failure_count=failure_info.count,
|
||||
first_failed=failure_info.first_failure,
|
||||
)
|
||||
# Track if file was skipped
|
||||
if entity is None and await self._should_skip_file(path):
|
||||
failure_info = self._file_failures[path]
|
||||
report.skipped_files.append(
|
||||
SkippedFile(
|
||||
path=path,
|
||||
reason=failure_info.last_error,
|
||||
failure_count=failure_info.count,
|
||||
first_failed=failure_info.first_failure,
|
||||
)
|
||||
)
|
||||
|
||||
with logfire.span("process_modified_files", modified_count=len(report.modified)):
|
||||
for path in report.modified:
|
||||
entity, _ = await self.sync_file(path, new=False)
|
||||
for path in report.modified:
|
||||
entity, _ = await self.sync_file(path, new=False)
|
||||
|
||||
# Track if file was skipped
|
||||
if entity is None and await self._should_skip_file(path):
|
||||
failure_info = self._file_failures[path]
|
||||
report.skipped_files.append(
|
||||
SkippedFile(
|
||||
path=path,
|
||||
reason=failure_info.last_error,
|
||||
failure_count=failure_info.count,
|
||||
first_failed=failure_info.first_failure,
|
||||
)
|
||||
# Track if file was skipped
|
||||
if entity is None and await self._should_skip_file(path):
|
||||
failure_info = self._file_failures[path]
|
||||
report.skipped_files.append(
|
||||
SkippedFile(
|
||||
path=path,
|
||||
reason=failure_info.last_error,
|
||||
failure_count=failure_info.count,
|
||||
first_failed=failure_info.first_failure,
|
||||
)
|
||||
)
|
||||
|
||||
# Only resolve relations if there were actual changes
|
||||
# If no files changed, no new unresolved relations could have been created
|
||||
with logfire.span("resolve_relations"):
|
||||
if report.total > 0:
|
||||
await self.resolve_relations()
|
||||
else:
|
||||
logger.info("Skipping relation resolution - no file changes detected")
|
||||
if report.total > 0:
|
||||
await self.resolve_relations()
|
||||
else:
|
||||
logger.info("Skipping relation resolution - no file changes detected")
|
||||
|
||||
# Update scan watermark after successful sync
|
||||
# Use the timestamp from sync start (not end) to ensure we catch files
|
||||
@@ -361,15 +350,6 @@ class SyncService:
|
||||
|
||||
duration_ms = int((time.time() - start_time) * 1000)
|
||||
|
||||
# Record metrics for sync operation
|
||||
logfire.metric_histogram("sync.duration", unit="ms").record(duration_ms)
|
||||
logfire.metric_counter("sync.files.new").add(len(report.new))
|
||||
logfire.metric_counter("sync.files.modified").add(len(report.modified))
|
||||
logfire.metric_counter("sync.files.deleted").add(len(report.deleted))
|
||||
logfire.metric_counter("sync.files.moved").add(len(report.moves))
|
||||
if report.skipped_files:
|
||||
logfire.metric_counter("sync.files.skipped").add(len(report.skipped_files))
|
||||
|
||||
# Log summary with skipped files if any
|
||||
if report.skipped_files:
|
||||
logger.warning(
|
||||
@@ -390,7 +370,6 @@ class SyncService:
|
||||
|
||||
return report
|
||||
|
||||
@logfire.instrument()
|
||||
async def scan(self, directory, force_full: bool = False):
|
||||
"""Smart scan using watermark and file count for large project optimization.
|
||||
|
||||
@@ -472,12 +451,6 @@ class SyncService:
|
||||
logger.warning("No scan watermark available, falling back to full scan")
|
||||
file_paths_to_scan = await self._scan_directory_full(directory)
|
||||
|
||||
# Record scan type metric
|
||||
logfire.metric_counter(f"sync.scan.{scan_type}").add(1)
|
||||
logfire.metric_histogram("sync.scan.files_scanned", unit="files").record(
|
||||
len(file_paths_to_scan)
|
||||
)
|
||||
|
||||
# Step 3: Process each file with mtime-based comparison
|
||||
scanned_paths: Set[str] = set()
|
||||
changed_checksums: Dict[str, str] = {}
|
||||
@@ -589,7 +562,6 @@ class SyncService:
|
||||
report.checksums = changed_checksums
|
||||
|
||||
scan_duration_ms = int((time.time() - scan_start_time) * 1000)
|
||||
logfire.metric_histogram("sync.scan.duration", unit="ms").record(scan_duration_ms)
|
||||
|
||||
logger.info(
|
||||
f"Completed {scan_type} scan for directory {directory} in {scan_duration_ms}ms, "
|
||||
@@ -599,7 +571,6 @@ class SyncService:
|
||||
)
|
||||
return report
|
||||
|
||||
@logfire.instrument()
|
||||
async def sync_file(
|
||||
self, path: str, new: bool = True
|
||||
) -> Tuple[Optional[Entity], Optional[str]]:
|
||||
@@ -654,7 +625,6 @@ class SyncService:
|
||||
|
||||
return None, None
|
||||
|
||||
@logfire.instrument()
|
||||
async def sync_markdown_file(self, path: str, new: bool = True) -> Tuple[Optional[Entity], str]:
|
||||
"""Sync a markdown file with full processing.
|
||||
|
||||
@@ -672,12 +642,19 @@ class SyncService:
|
||||
file_contains_frontmatter = has_frontmatter(file_content)
|
||||
|
||||
# Get file timestamps for tracking modification times
|
||||
file_stats = self.file_service.file_stats(path)
|
||||
created = datetime.fromtimestamp(file_stats.st_ctime).astimezone()
|
||||
modified = datetime.fromtimestamp(file_stats.st_mtime).astimezone()
|
||||
file_metadata = await self.file_service.get_file_metadata(path)
|
||||
created = file_metadata.created_at
|
||||
modified = file_metadata.modified_at
|
||||
|
||||
# entity markdown will always contain front matter, so it can be used up create/update the entity
|
||||
entity_markdown = await self.entity_parser.parse_file(path)
|
||||
# Parse markdown content with file metadata (avoids redundant file read/stat)
|
||||
# This enables cloud implementations (S3FileService) to provide metadata from head_object
|
||||
abs_path = self.file_service.base_path / path
|
||||
entity_markdown = await self.entity_parser.parse_markdown_content(
|
||||
file_path=abs_path,
|
||||
content=file_content,
|
||||
mtime=file_metadata.modified_at.timestamp(),
|
||||
ctime=file_metadata.created_at.timestamp(),
|
||||
)
|
||||
|
||||
# if the file contains frontmatter, resolve a permalink (unless disabled)
|
||||
if file_contains_frontmatter and not self.app_config.disable_permalinks:
|
||||
@@ -723,8 +700,8 @@ class SyncService:
|
||||
"checksum": final_checksum,
|
||||
"created_at": created,
|
||||
"updated_at": modified,
|
||||
"mtime": file_stats.st_mtime,
|
||||
"size": file_stats.st_size,
|
||||
"mtime": file_metadata.modified_at.timestamp(),
|
||||
"size": file_metadata.size,
|
||||
},
|
||||
)
|
||||
|
||||
@@ -737,7 +714,6 @@ class SyncService:
|
||||
# Return the final checksum to ensure everything is consistent
|
||||
return entity, final_checksum
|
||||
|
||||
@logfire.instrument()
|
||||
async def sync_regular_file(self, path: str, new: bool = True) -> Tuple[Optional[Entity], str]:
|
||||
"""Sync a non-markdown file with basic tracking.
|
||||
|
||||
@@ -754,9 +730,9 @@ class SyncService:
|
||||
await self.entity_service.resolve_permalink(path, skip_conflict_check=True)
|
||||
|
||||
# get file timestamps
|
||||
file_stats = self.file_service.file_stats(path)
|
||||
created = datetime.fromtimestamp(file_stats.st_ctime).astimezone()
|
||||
modified = datetime.fromtimestamp(file_stats.st_mtime).astimezone()
|
||||
file_metadata = await self.file_service.get_file_metadata(path)
|
||||
created = file_metadata.created_at
|
||||
modified = file_metadata.modified_at
|
||||
|
||||
# get mime type
|
||||
content_type = self.file_service.content_type(path)
|
||||
@@ -772,8 +748,8 @@ class SyncService:
|
||||
created_at=created,
|
||||
updated_at=modified,
|
||||
content_type=content_type,
|
||||
mtime=file_stats.st_mtime,
|
||||
size=file_stats.st_size,
|
||||
mtime=file_metadata.modified_at.timestamp(),
|
||||
size=file_metadata.size,
|
||||
)
|
||||
)
|
||||
return entity, checksum
|
||||
@@ -789,15 +765,15 @@ class SyncService:
|
||||
logger.error(f"Entity not found after constraint violation, path={path}")
|
||||
raise ValueError(f"Entity not found after constraint violation: {path}")
|
||||
|
||||
# Re-get file stats since we're in update path
|
||||
file_stats_for_update = self.file_service.file_stats(path)
|
||||
# Re-get file metadata since we're in update path
|
||||
file_metadata_for_update = await self.file_service.get_file_metadata(path)
|
||||
updated = await self.entity_repository.update(
|
||||
entity.id,
|
||||
{
|
||||
"file_path": path,
|
||||
"checksum": checksum,
|
||||
"mtime": file_stats_for_update.st_mtime,
|
||||
"size": file_stats_for_update.st_size,
|
||||
"mtime": file_metadata_for_update.modified_at.timestamp(),
|
||||
"size": file_metadata_for_update.size,
|
||||
},
|
||||
)
|
||||
|
||||
@@ -811,8 +787,8 @@ class SyncService:
|
||||
raise
|
||||
else:
|
||||
# Get file timestamps for updating modification time
|
||||
file_stats = self.file_service.file_stats(path)
|
||||
modified = datetime.fromtimestamp(file_stats.st_mtime).astimezone()
|
||||
file_metadata = await self.file_service.get_file_metadata(path)
|
||||
modified = file_metadata.modified_at
|
||||
|
||||
entity = await self.entity_repository.get_by_file_path(path)
|
||||
if entity is None: # pragma: no cover
|
||||
@@ -827,8 +803,8 @@ class SyncService:
|
||||
"file_path": path,
|
||||
"checksum": checksum,
|
||||
"updated_at": modified,
|
||||
"mtime": file_stats.st_mtime,
|
||||
"size": file_stats.st_size,
|
||||
"mtime": file_metadata.modified_at.timestamp(),
|
||||
"size": file_metadata.size,
|
||||
},
|
||||
)
|
||||
|
||||
@@ -838,7 +814,6 @@ class SyncService:
|
||||
|
||||
return updated, checksum
|
||||
|
||||
@logfire.instrument()
|
||||
async def handle_delete(self, file_path: str):
|
||||
"""Handle complete entity deletion including search index cleanup."""
|
||||
|
||||
@@ -870,7 +845,6 @@ class SyncService:
|
||||
else:
|
||||
await self.search_service.delete_by_entity_id(entity.id)
|
||||
|
||||
@logfire.instrument()
|
||||
async def handle_move(self, old_path, new_path):
|
||||
logger.debug("Moving entity", old_path=old_path, new_path=new_path)
|
||||
|
||||
@@ -975,7 +949,6 @@ class SyncService:
|
||||
# update search index
|
||||
await self.search_service.index_entity(updated)
|
||||
|
||||
@logfire.instrument()
|
||||
async def resolve_relations(self, entity_id: int | None = None):
|
||||
"""Try to resolve unresolved relations.
|
||||
|
||||
@@ -1026,16 +999,27 @@ class SyncService:
|
||||
"to_name": resolved_entity.title,
|
||||
},
|
||||
)
|
||||
except IntegrityError: # pragma: no cover
|
||||
# update search index only on successful resolution
|
||||
await self.search_service.index_entity(resolved_entity)
|
||||
except IntegrityError:
|
||||
# IntegrityError means a relation with this (from_id, to_id, relation_type)
|
||||
# already exists. The UPDATE was rolled back, so our unresolved relation
|
||||
# (to_id=NULL) still exists in the database. We delete it because:
|
||||
# 1. It's redundant - a resolved relation already captures this relationship
|
||||
# 2. If we don't delete it, future syncs will try to resolve it again
|
||||
# and get the same IntegrityError
|
||||
logger.debug(
|
||||
"Ignoring duplicate relation "
|
||||
"Deleting duplicate unresolved relation "
|
||||
f"relation_id={relation.id} "
|
||||
f"from_id={relation.from_id} "
|
||||
f"to_name={relation.to_name}"
|
||||
f"to_name={relation.to_name} "
|
||||
f"resolved_to_id={resolved_entity.id}"
|
||||
)
|
||||
|
||||
# update search index
|
||||
await self.search_service.index_entity(resolved_entity)
|
||||
try:
|
||||
await self.relation_repository.delete(relation.id)
|
||||
except Exception as e:
|
||||
# Log but don't fail - the relation may have been deleted already
|
||||
logger.debug(f"Could not delete duplicate relation {relation.id}: {e}")
|
||||
|
||||
async def _quick_count_files(self, directory: Path) -> int:
|
||||
"""Fast file count using find command.
|
||||
@@ -1063,8 +1047,6 @@ class SyncService:
|
||||
f"error: {error_msg}. Falling back to manual count. "
|
||||
f"This will slow down watermark detection!"
|
||||
)
|
||||
# Track optimization failures for visibility
|
||||
logfire.metric_counter("sync.scan.file_count_failure").add(1)
|
||||
# Fallback: count using scan_directory
|
||||
count = 0
|
||||
async for _ in self.scan_directory(directory):
|
||||
@@ -1105,8 +1087,6 @@ class SyncService:
|
||||
f"error: {error_msg}. Falling back to full scan. "
|
||||
f"This will cause slow syncs on large projects!"
|
||||
)
|
||||
# Track optimization failures for visibility
|
||||
logfire.metric_counter("sync.scan.optimization_failure").add(1)
|
||||
# Fallback to full scan
|
||||
return await self._scan_directory_full(directory)
|
||||
|
||||
|
||||
@@ -5,7 +5,10 @@ import os
|
||||
from collections import defaultdict
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
from typing import List, Optional, Set, Sequence
|
||||
from typing import List, Optional, Set, Sequence, Callable, Awaitable, TYPE_CHECKING
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from basic_memory.sync.sync_service import SyncService
|
||||
|
||||
from basic_memory.config import BasicMemoryConfig, WATCH_STATUS_JSON
|
||||
from basic_memory.ignore_utils import load_gitignore_patterns, should_ignore_path
|
||||
@@ -71,12 +74,17 @@ class WatchServiceState(BaseModel):
|
||||
self.last_error = datetime.now()
|
||||
|
||||
|
||||
# Type alias for sync service factory function
|
||||
SyncServiceFactory = Callable[[Project], Awaitable["SyncService"]]
|
||||
|
||||
|
||||
class WatchService:
|
||||
def __init__(
|
||||
self,
|
||||
app_config: BasicMemoryConfig,
|
||||
project_repository: ProjectRepository,
|
||||
quiet: bool = False,
|
||||
sync_service_factory: Optional[SyncServiceFactory] = None,
|
||||
):
|
||||
self.app_config = app_config
|
||||
self.project_repository = project_repository
|
||||
@@ -84,10 +92,20 @@ class WatchService:
|
||||
self.status_path = Path.home() / ".basic-memory" / WATCH_STATUS_JSON
|
||||
self.status_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
self._ignore_patterns_cache: dict[Path, Set[str]] = {}
|
||||
self._sync_service_factory = sync_service_factory
|
||||
|
||||
# quiet mode for mcp so it doesn't mess up stdout
|
||||
self.console = Console(quiet=quiet)
|
||||
|
||||
async def _get_sync_service(self, project: Project) -> "SyncService":
|
||||
"""Get sync service for a project, using factory if provided."""
|
||||
if self._sync_service_factory:
|
||||
return await self._sync_service_factory(project)
|
||||
# Fall back to default factory
|
||||
from basic_memory.sync.sync_service import get_sync_service
|
||||
|
||||
return await get_sync_service(project)
|
||||
|
||||
async def _schedule_restart(self, stop_event: asyncio.Event):
|
||||
"""Schedule a restart of the watch service after the configured interval."""
|
||||
await asyncio.sleep(self.app_config.watch_project_reload_interval)
|
||||
@@ -233,9 +251,6 @@ class WatchService:
|
||||
|
||||
async def handle_changes(self, project: Project, changes: Set[FileChange]) -> None:
|
||||
"""Process a batch of file changes"""
|
||||
# avoid circular imports
|
||||
from basic_memory.sync.sync_service import get_sync_service
|
||||
|
||||
# Check if project still exists in configuration before processing
|
||||
# This prevents deleted projects from being recreated by background sync
|
||||
from basic_memory.config import ConfigManager
|
||||
@@ -250,7 +265,7 @@ class WatchService:
|
||||
)
|
||||
return
|
||||
|
||||
sync_service = await get_sync_service(project)
|
||||
sync_service = await self._get_sync_service(project)
|
||||
file_service = sync_service.file_service
|
||||
|
||||
start_time = time.time()
|
||||
|
||||
+65
-64
@@ -5,9 +5,9 @@ import os
|
||||
import logging
|
||||
import re
|
||||
import sys
|
||||
from datetime import datetime
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
from typing import Optional, Protocol, Union, runtime_checkable, List
|
||||
from typing import Protocol, Union, runtime_checkable, List
|
||||
|
||||
from loguru import logger
|
||||
from unidecode import unidecode
|
||||
@@ -67,9 +67,6 @@ class PathLike(Protocol):
|
||||
# This preserves compatibility with existing code while we migrate
|
||||
FilePath = Union[Path, str]
|
||||
|
||||
# Disable the "Queue is full" warning
|
||||
logging.getLogger("opentelemetry.sdk.metrics._internal.instrument").setLevel(logging.ERROR)
|
||||
|
||||
|
||||
def generate_permalink(file_path: Union[Path, str, PathLike], split_extension: bool = True) -> str:
|
||||
"""Generate a stable permalink from a file path.
|
||||
@@ -206,29 +203,35 @@ def generate_permalink(file_path: Union[Path, str, PathLike], split_extension: b
|
||||
|
||||
|
||||
def setup_logging(
|
||||
env: str,
|
||||
home_dir: Path,
|
||||
log_file: Optional[str] = None,
|
||||
log_level: str = "INFO",
|
||||
console: bool = True,
|
||||
log_to_file: bool = False,
|
||||
log_to_stdout: bool = False,
|
||||
structured_context: bool = False,
|
||||
) -> None: # pragma: no cover
|
||||
"""
|
||||
Configure logging for the application.
|
||||
"""Configure logging with explicit settings.
|
||||
|
||||
This function provides a simple, explicit interface for configuring logging.
|
||||
Each entry point (CLI, MCP, API) should call this with appropriate settings.
|
||||
|
||||
Args:
|
||||
env: The environment name (dev, test, prod)
|
||||
home_dir: The root directory for the application
|
||||
log_file: The name of the log file to write to
|
||||
log_level: The logging level to use
|
||||
console: Whether to log to the console
|
||||
log_level: DEBUG, INFO, WARNING, ERROR
|
||||
log_to_file: Write to ~/.basic-memory/basic-memory.log with rotation
|
||||
log_to_stdout: Write to stderr (for Docker/cloud deployments)
|
||||
structured_context: Bind tenant_id, fly_region, etc. for cloud observability
|
||||
"""
|
||||
# Remove default handler and any existing handlers
|
||||
logger.remove()
|
||||
|
||||
# Add file handler if we are not running tests and a log file is specified
|
||||
if log_file and env != "test":
|
||||
# Setup file logger
|
||||
log_path = home_dir / log_file
|
||||
# In test mode, only log to stdout regardless of settings
|
||||
env = os.getenv("BASIC_MEMORY_ENV", "dev")
|
||||
if env == "test":
|
||||
logger.add(sys.stderr, level=log_level, backtrace=True, diagnose=True, colorize=True)
|
||||
return
|
||||
|
||||
# Add file handler with rotation
|
||||
if log_to_file:
|
||||
log_path = Path.home() / ".basic-memory" / "basic-memory.log"
|
||||
log_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
logger.add(
|
||||
str(log_path),
|
||||
level=log_level,
|
||||
@@ -236,42 +239,28 @@ def setup_logging(
|
||||
retention="10 days",
|
||||
backtrace=True,
|
||||
diagnose=True,
|
||||
enqueue=True,
|
||||
enqueue=True, # Thread-safe async logging
|
||||
colorize=False,
|
||||
)
|
||||
|
||||
# Add console logger if requested or in test mode
|
||||
if env == "test" or console:
|
||||
# Add stdout handler (for Docker/cloud)
|
||||
if log_to_stdout:
|
||||
logger.add(sys.stderr, level=log_level, backtrace=True, diagnose=True, colorize=True)
|
||||
|
||||
logger.info(f"ENV: '{env}' Log level: '{log_level}' Logging to {log_file}")
|
||||
|
||||
# Bind environment context for structured logging (works in both local and cloud)
|
||||
tenant_id = os.getenv("BASIC_MEMORY_TENANT_ID", "local")
|
||||
fly_app_name = os.getenv("FLY_APP_NAME", "local")
|
||||
fly_machine_id = os.getenv("FLY_MACHINE_ID", "local")
|
||||
fly_region = os.getenv("FLY_REGION", "local")
|
||||
|
||||
logger.configure(
|
||||
extra={
|
||||
"tenant_id": tenant_id,
|
||||
"fly_app_name": fly_app_name,
|
||||
"fly_machine_id": fly_machine_id,
|
||||
"fly_region": fly_region,
|
||||
}
|
||||
)
|
||||
# Bind structured context for cloud observability
|
||||
if structured_context:
|
||||
logger.configure(
|
||||
extra={
|
||||
"tenant_id": os.getenv("BASIC_MEMORY_TENANT_ID", "local"),
|
||||
"fly_app_name": os.getenv("FLY_APP_NAME", "local"),
|
||||
"fly_machine_id": os.getenv("FLY_MACHINE_ID", "local"),
|
||||
"fly_region": os.getenv("FLY_REGION", "local"),
|
||||
}
|
||||
)
|
||||
|
||||
# Reduce noise from third-party libraries
|
||||
noisy_loggers = {
|
||||
# HTTP client logs
|
||||
"httpx": logging.WARNING,
|
||||
# File watching logs
|
||||
"watchfiles.main": logging.WARNING,
|
||||
}
|
||||
|
||||
# Set log levels for noisy loggers
|
||||
for logger_name, level in noisy_loggers.items():
|
||||
logging.getLogger(logger_name).setLevel(level)
|
||||
logging.getLogger("httpx").setLevel(logging.WARNING)
|
||||
logging.getLogger("watchfiles.main").setLevel(logging.WARNING)
|
||||
|
||||
|
||||
def parse_tags(tags: Union[List[str], str, None]) -> List[str]:
|
||||
@@ -340,7 +329,7 @@ def normalize_file_path_for_comparison(file_path: str) -> str:
|
||||
This function normalizes file paths to help detect potential conflicts:
|
||||
- Converts to lowercase for case-insensitive comparison
|
||||
- Normalizes Unicode characters
|
||||
- Handles path separators consistently
|
||||
- Converts backslashes to forward slashes for cross-platform consistency
|
||||
|
||||
Args:
|
||||
file_path: The file path to normalize
|
||||
@@ -349,19 +338,15 @@ def normalize_file_path_for_comparison(file_path: str) -> str:
|
||||
Normalized file path for comparison purposes
|
||||
"""
|
||||
import unicodedata
|
||||
from pathlib import PureWindowsPath
|
||||
|
||||
# Convert to lowercase for case-insensitive comparison
|
||||
normalized = file_path.lower()
|
||||
# Use PureWindowsPath to ensure backslashes are treated as separators
|
||||
# regardless of current platform, then convert to POSIX-style
|
||||
normalized = PureWindowsPath(file_path).as_posix().lower()
|
||||
|
||||
# Normalize Unicode characters (NFD normalization)
|
||||
normalized = unicodedata.normalize("NFD", normalized)
|
||||
|
||||
# Replace path separators with forward slashes
|
||||
normalized = normalized.replace("\\", "/")
|
||||
|
||||
# Remove multiple slashes
|
||||
normalized = re.sub(r"/+", "/", normalized)
|
||||
|
||||
return normalized
|
||||
|
||||
|
||||
@@ -445,21 +430,37 @@ def validate_project_path(path: str, project_path: Path) -> bool:
|
||||
return False
|
||||
|
||||
|
||||
def ensure_timezone_aware(dt: datetime) -> datetime:
|
||||
"""Ensure a datetime is timezone-aware using system timezone.
|
||||
def ensure_timezone_aware(dt: datetime, cloud_mode: bool | None = None) -> datetime:
|
||||
"""Ensure a datetime is timezone-aware.
|
||||
|
||||
If the datetime is naive, convert it to timezone-aware using the system's local timezone.
|
||||
If it's already timezone-aware, return it unchanged.
|
||||
If the datetime is naive, convert it to timezone-aware. The interpretation
|
||||
depends on cloud_mode:
|
||||
- In cloud mode (PostgreSQL/asyncpg): naive datetimes are interpreted as UTC
|
||||
- In local mode (SQLite): naive datetimes are interpreted as local time
|
||||
|
||||
asyncpg uses binary protocol which returns timestamps in UTC but as naive
|
||||
datetimes. In cloud deployments, cloud_mode=True handles this correctly.
|
||||
|
||||
Args:
|
||||
dt: The datetime to ensure is timezone-aware
|
||||
cloud_mode: Optional explicit cloud_mode setting. If None, loads from config.
|
||||
|
||||
Returns:
|
||||
A timezone-aware datetime
|
||||
"""
|
||||
if dt.tzinfo is None:
|
||||
# Naive datetime - assume it's in local time and add timezone
|
||||
return dt.astimezone()
|
||||
# Determine cloud_mode: use explicit parameter if provided, otherwise load from config
|
||||
if cloud_mode is None:
|
||||
from basic_memory.config import ConfigManager
|
||||
|
||||
cloud_mode = ConfigManager().config.cloud_mode_enabled
|
||||
|
||||
if cloud_mode:
|
||||
# Cloud/PostgreSQL mode: naive datetimes from asyncpg are already UTC
|
||||
return dt.replace(tzinfo=timezone.utc)
|
||||
else:
|
||||
# Local/SQLite mode: naive datetimes are in local time
|
||||
return dt.astimezone()
|
||||
else:
|
||||
# Already timezone-aware
|
||||
return dt
|
||||
|
||||
+78
-49
@@ -50,18 +50,23 @@ The `app` fixture ensures FastAPI dependency overrides are active, and
|
||||
`mcp_server` provides the MCP server with proper project session initialization.
|
||||
"""
|
||||
|
||||
import os
|
||||
from typing import AsyncGenerator, Literal
|
||||
|
||||
import pytest
|
||||
import pytest_asyncio
|
||||
from pathlib import Path
|
||||
from sqlalchemy import text
|
||||
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker, create_async_engine
|
||||
from sqlalchemy.pool import NullPool
|
||||
from testcontainers.postgres import PostgresContainer
|
||||
|
||||
from httpx import AsyncClient, ASGITransport
|
||||
|
||||
from basic_memory.config import BasicMemoryConfig, ProjectConfig, ConfigManager, DatabaseBackend
|
||||
from basic_memory.db import engine_session_factory, DatabaseType
|
||||
from basic_memory.models import Project
|
||||
from basic_memory.models.base import Base
|
||||
from basic_memory.repository.project_repository import ProjectRepository
|
||||
from fastapi import FastAPI
|
||||
|
||||
@@ -72,25 +77,38 @@ from basic_memory.deps import get_project_config, get_engine_factory, get_app_co
|
||||
from basic_memory.mcp import tools # noqa: F401
|
||||
|
||||
|
||||
@pytest.fixture(
|
||||
params=[
|
||||
pytest.param("sqlite", id="sqlite"),
|
||||
pytest.param("postgres", id="postgres", marks=pytest.mark.postgres),
|
||||
]
|
||||
)
|
||||
def db_backend(request) -> Literal["sqlite", "postgres"]:
|
||||
"""Parametrize tests to run against both SQLite and Postgres.
|
||||
# =============================================================================
|
||||
# Database Backend Selection (env var approach)
|
||||
# =============================================================================
|
||||
# By default, integration tests run against SQLite.
|
||||
# Set BASIC_MEMORY_TEST_POSTGRES=1 to run against Postgres (uses testcontainers).
|
||||
|
||||
Usage:
|
||||
pytest # Runs tests against SQLite only (default)
|
||||
pytest -m postgres # Runs tests against Postgres only
|
||||
pytest -m "not postgres" # Runs tests against SQLite only
|
||||
pytest --run-all-backends # Runs tests against both backends
|
||||
|
||||
Note: Only tests that use database fixtures (engine_factory, session_maker, etc.)
|
||||
will be parametrized. Tests that don't use the database won't be affected.
|
||||
@pytest.fixture(scope="session")
|
||||
def db_backend() -> Literal["sqlite", "postgres"]:
|
||||
"""Determine database backend from environment variable.
|
||||
|
||||
Default: sqlite
|
||||
Set BASIC_MEMORY_TEST_POSTGRES=1 to use postgres
|
||||
"""
|
||||
return request.param
|
||||
if os.environ.get("BASIC_MEMORY_TEST_POSTGRES", "").lower() in ("1", "true", "yes"):
|
||||
return "postgres"
|
||||
return "sqlite"
|
||||
|
||||
|
||||
@pytest.fixture(scope="session")
|
||||
def postgres_container(db_backend):
|
||||
"""Session-scoped Postgres container for integration tests.
|
||||
|
||||
Uses testcontainers to spin up a real Postgres instance.
|
||||
Only starts if db_backend is "postgres".
|
||||
"""
|
||||
if db_backend != "postgres":
|
||||
yield None
|
||||
return
|
||||
|
||||
with PostgresContainer("postgres:16-alpine") as postgres:
|
||||
yield postgres
|
||||
|
||||
|
||||
@pytest_asyncio.fixture
|
||||
@@ -98,50 +116,57 @@ async def engine_factory(
|
||||
app_config,
|
||||
config_manager,
|
||||
db_backend: Literal["sqlite", "postgres"],
|
||||
postgres_container,
|
||||
tmp_path,
|
||||
) -> AsyncGenerator[tuple, None]:
|
||||
"""Create engine and session factory for the configured database backend."""
|
||||
from basic_memory.models.search import CREATE_SEARCH_INDEX
|
||||
from basic_memory.models.search import (
|
||||
CREATE_SEARCH_INDEX,
|
||||
CREATE_POSTGRES_SEARCH_INDEX_TABLE,
|
||||
CREATE_POSTGRES_SEARCH_INDEX_FTS,
|
||||
CREATE_POSTGRES_SEARCH_INDEX_METADATA,
|
||||
)
|
||||
from basic_memory import db
|
||||
|
||||
# Determine database type based on backend
|
||||
if db_backend == "postgres":
|
||||
db_type = DatabaseType.FILESYSTEM
|
||||
else:
|
||||
db_type = DatabaseType.FILESYSTEM # Integration tests use file-based SQLite
|
||||
# Postgres mode using testcontainers
|
||||
sync_url = postgres_container.get_connection_url()
|
||||
async_url = sync_url.replace("postgresql+psycopg2", "postgresql+asyncpg")
|
||||
|
||||
# Use tmp_path for SQLite, use config database_path for Postgres
|
||||
if db_backend == "sqlite":
|
||||
db_path = tmp_path / "test.db"
|
||||
else:
|
||||
db_path = app_config.database_path
|
||||
engine = create_async_engine(
|
||||
async_url,
|
||||
echo=False,
|
||||
poolclass=NullPool,
|
||||
)
|
||||
|
||||
if db_backend == "postgres":
|
||||
# Postgres: Create fresh engine for each test with full schema reset
|
||||
config_manager._config = app_config
|
||||
session_maker = async_sessionmaker(
|
||||
bind=engine,
|
||||
class_=AsyncSession,
|
||||
expire_on_commit=False,
|
||||
autoflush=False,
|
||||
)
|
||||
|
||||
# Use context manager to handle engine disposal properly
|
||||
async with engine_session_factory(db_path, db_type) as (engine, session_maker):
|
||||
# Drop and recreate schema for complete isolation
|
||||
async with engine.begin() as conn:
|
||||
await conn.execute(text("DROP SCHEMA IF EXISTS public CASCADE"))
|
||||
await conn.execute(text("CREATE SCHEMA public"))
|
||||
await conn.execute(text("GRANT ALL ON SCHEMA public TO basic_memory_user"))
|
||||
await conn.execute(text("GRANT ALL ON SCHEMA public TO public"))
|
||||
# Drop and recreate all tables for test isolation
|
||||
async with engine.begin() as conn:
|
||||
await conn.execute(text("DROP TABLE IF EXISTS search_index CASCADE"))
|
||||
await conn.run_sync(Base.metadata.drop_all)
|
||||
await conn.run_sync(Base.metadata.create_all)
|
||||
# asyncpg requires separate execute calls for each statement
|
||||
await conn.execute(CREATE_POSTGRES_SEARCH_INDEX_TABLE)
|
||||
await conn.execute(CREATE_POSTGRES_SEARCH_INDEX_FTS)
|
||||
await conn.execute(CREATE_POSTGRES_SEARCH_INDEX_METADATA)
|
||||
|
||||
# Run migrations to create production tables
|
||||
from basic_memory.db import run_migrations
|
||||
yield engine, session_maker
|
||||
|
||||
await run_migrations(app_config, db_type)
|
||||
|
||||
yield engine, session_maker
|
||||
await engine.dispose()
|
||||
|
||||
else:
|
||||
# SQLite: Create fresh database (fast with tmp files)
|
||||
db_path = tmp_path / "test.db"
|
||||
db_type = DatabaseType.FILESYSTEM
|
||||
|
||||
async with engine_session_factory(db_path, db_type) as (engine, session_maker):
|
||||
# Create all tables via ORM
|
||||
from basic_memory.models.base import Base
|
||||
|
||||
async with engine.begin() as conn:
|
||||
await conn.run_sync(Base.metadata.create_all)
|
||||
|
||||
@@ -181,7 +206,11 @@ def config_home(tmp_path, monkeypatch) -> Path:
|
||||
|
||||
@pytest.fixture
|
||||
def app_config(
|
||||
config_home, db_backend: Literal["sqlite", "postgres"], tmp_path, monkeypatch
|
||||
config_home,
|
||||
db_backend: Literal["sqlite", "postgres"],
|
||||
postgres_container,
|
||||
tmp_path,
|
||||
monkeypatch,
|
||||
) -> BasicMemoryConfig:
|
||||
"""Create test app configuration."""
|
||||
# Disable cloud mode for CLI tests
|
||||
@@ -190,12 +219,12 @@ def app_config(
|
||||
# Create a basic config with test-project like unit tests do
|
||||
projects = {"test-project": str(config_home)}
|
||||
|
||||
# Configure database backend based on test parameter
|
||||
# Configure database backend based on env var
|
||||
if db_backend == "postgres":
|
||||
database_backend = DatabaseBackend.POSTGRES
|
||||
database_url = (
|
||||
"postgresql+asyncpg://basic_memory_user:dev_password@localhost:5433/basic_memory_test"
|
||||
)
|
||||
# Get URL from testcontainer and convert to asyncpg driver
|
||||
sync_url = postgres_container.get_connection_url()
|
||||
database_url = sync_url.replace("postgresql+psycopg2", "postgresql+asyncpg")
|
||||
else:
|
||||
database_backend = DatabaseBackend.SQLITE
|
||||
database_url = None
|
||||
|
||||
@@ -0,0 +1,70 @@
|
||||
"""
|
||||
Integration test for FastAPI lifespan shutdown behavior.
|
||||
|
||||
This test verifies the asyncio cancellation pattern used by the API lifespan:
|
||||
when the background sync task is cancelled during shutdown, it must be *awaited*
|
||||
before database shutdown begins. This prevents "hang on exit" scenarios in
|
||||
`asyncio.run(...)` callers (e.g. CLI/MCP clients using httpx ASGITransport).
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
|
||||
from httpx import ASGITransport, AsyncClient
|
||||
|
||||
|
||||
def test_lifespan_shutdown_awaits_sync_task_cancellation(app, monkeypatch):
|
||||
"""
|
||||
Ensure lifespan shutdown awaits the cancelled background sync task.
|
||||
|
||||
Why this is deterministic:
|
||||
- Cancelling a task does not make it "done" immediately; it becomes done only
|
||||
once the event loop schedules it and it processes the CancelledError.
|
||||
- In the buggy version, shutdown proceeded directly to db.shutdown_db()
|
||||
immediately after calling cancel(), so at *entry* to shutdown_db the task
|
||||
is still not done.
|
||||
- In the fixed version, lifespan does `await sync_task` before shutdown_db,
|
||||
so by the time shutdown_db is called, the task is done (cancelled).
|
||||
"""
|
||||
|
||||
# Import the *module* (not the package-level FastAPI `basic_memory.api.app` export)
|
||||
# so monkeypatching affects the exact symbols referenced inside lifespan().
|
||||
#
|
||||
# Note: `basic_memory/api/__init__.py` re-exports `app`, so `import basic_memory.api.app`
|
||||
# can resolve to the FastAPI instance rather than the `basic_memory.api.app` module.
|
||||
import importlib
|
||||
|
||||
api_app_module = importlib.import_module("basic_memory.api.app")
|
||||
|
||||
# Keep startup cheap: we don't need real DB init for this ordering test.
|
||||
async def _noop_initialize_app(_app_config):
|
||||
return None
|
||||
|
||||
async def _fake_get_or_create_db(*_args, **_kwargs):
|
||||
return object(), object()
|
||||
|
||||
monkeypatch.setattr(api_app_module, "initialize_app", _noop_initialize_app)
|
||||
monkeypatch.setattr(api_app_module.db, "get_or_create_db", _fake_get_or_create_db)
|
||||
|
||||
# Make the sync task long-lived so it must be cancelled on shutdown.
|
||||
async def _fake_initialize_file_sync(_app_config):
|
||||
await asyncio.Event().wait()
|
||||
|
||||
monkeypatch.setattr(api_app_module, "initialize_file_sync", _fake_initialize_file_sync)
|
||||
|
||||
# Assert ordering: shutdown_db must be called only after the sync_task is done.
|
||||
async def _assert_sync_task_done_before_db_shutdown():
|
||||
assert api_app_module.app.state.sync_task is not None
|
||||
assert api_app_module.app.state.sync_task.done()
|
||||
|
||||
monkeypatch.setattr(api_app_module.db, "shutdown_db", _assert_sync_task_done_before_db_shutdown)
|
||||
|
||||
async def _run_client_once():
|
||||
async with AsyncClient(transport=ASGITransport(app=app), base_url="http://test") as client:
|
||||
# Any request is sufficient to trigger lifespan startup/shutdown.
|
||||
await client.get("/__nonexistent__")
|
||||
|
||||
# Use asyncio.run to match the CLI/MCP execution model where loop teardown
|
||||
# would hang if a background task is left running.
|
||||
asyncio.run(_run_client_once())
|
||||
|
||||
|
||||
@@ -46,3 +46,57 @@ async def test_read_note_after_write(mcp_server, app, test_project):
|
||||
assert "# Test Note" in result_text
|
||||
assert "This is test content." in result_text
|
||||
assert "test/test-note" in result_text # permalink
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_read_note_underscored_folder_by_permalink(mcp_server, app, test_project):
|
||||
"""Test read_note with permalink from underscored folder.
|
||||
|
||||
Reproduces bug #416: read_note fails to find notes when given permalinks
|
||||
from underscored folder names (e.g., _archive/, _drafts/), even though
|
||||
the permalink is copied directly from the note's YAML frontmatter.
|
||||
"""
|
||||
|
||||
async with Client(mcp_server) as client:
|
||||
# Create a note in an underscored folder
|
||||
write_result = await client.call_tool(
|
||||
"write_note",
|
||||
{
|
||||
"project": test_project.name,
|
||||
"title": "Example Note",
|
||||
"folder": "_archive/articles",
|
||||
"content": "# Example Note\n\nThis is a test note in an underscored folder.",
|
||||
"tags": "test,archive",
|
||||
},
|
||||
)
|
||||
|
||||
assert len(write_result.content) == 1
|
||||
assert write_result.content[0].type == "text"
|
||||
write_text = write_result.content[0].text
|
||||
|
||||
# Verify the file path includes the underscore
|
||||
assert "_archive/articles/Example Note.md" in write_text
|
||||
|
||||
# Verify the permalink has underscores stripped (this is the expected behavior)
|
||||
assert "archive/articles/example-note" in write_text
|
||||
|
||||
# Now try to read the note using the permalink (without underscores)
|
||||
# This is the exact scenario from the bug report - using the permalink
|
||||
# that was generated in the YAML frontmatter
|
||||
read_result = await client.call_tool(
|
||||
"read_note",
|
||||
{
|
||||
"project": test_project.name,
|
||||
"identifier": "archive/articles/example-note", # permalink without underscores
|
||||
},
|
||||
)
|
||||
|
||||
# This should succeed - the note should be found by its permalink
|
||||
assert len(read_result.content) == 1
|
||||
assert read_result.content[0].type == "text"
|
||||
result_text = read_result.content[0].text
|
||||
|
||||
# Should contain the note content
|
||||
assert "# Example Note" in result_text
|
||||
assert "This is a test note in an underscored folder." in result_text
|
||||
assert "archive/articles/example-note" in result_text # permalink
|
||||
|
||||
@@ -5,7 +5,6 @@ and other SQLite configuration settings work correctly in production scenarios.
|
||||
"""
|
||||
|
||||
import pytest
|
||||
from unittest.mock import patch
|
||||
from sqlalchemy import text
|
||||
|
||||
|
||||
@@ -142,23 +141,6 @@ async def test_null_pool_on_windows(tmp_path, monkeypatch):
|
||||
assert isinstance(engine.pool, NullPool)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.skipif(
|
||||
__import__("os").name == "nt", reason="Non-Windows test - cannot mock POSIX paths on Windows"
|
||||
)
|
||||
async def test_regular_pool_on_non_windows(tmp_path):
|
||||
"""Test that regular pooling is used on non-Windows platforms."""
|
||||
from basic_memory.db import engine_session_factory, DatabaseType
|
||||
from sqlalchemy.pool import NullPool
|
||||
|
||||
db_path = tmp_path / "test_posix_pool.db"
|
||||
|
||||
with patch("basic_memory.db.os.name", "posix"):
|
||||
async with engine_session_factory(db_path, DatabaseType.FILESYSTEM) as (engine, _):
|
||||
# Engine should NOT be using NullPool on non-Windows
|
||||
assert not isinstance(engine.pool, NullPool)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.windows
|
||||
@pytest.mark.skipif(
|
||||
|
||||
@@ -1,372 +0,0 @@
|
||||
"""
|
||||
Performance benchmark tests for sync operations.
|
||||
|
||||
These tests measure baseline performance for indexing operations to track
|
||||
improvements from optimizations. Tests are marked with @pytest.mark.benchmark
|
||||
and can be run separately.
|
||||
|
||||
Usage:
|
||||
# Run all benchmarks
|
||||
pytest test-int/test_sync_performance_benchmark.py -v
|
||||
|
||||
# Run specific benchmark
|
||||
pytest test-int/test_sync_performance_benchmark.py::test_benchmark_sync_100_files -v
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import time
|
||||
from pathlib import Path
|
||||
from textwrap import dedent
|
||||
|
||||
import pytest
|
||||
|
||||
from basic_memory.config import BasicMemoryConfig, ProjectConfig
|
||||
from basic_memory.sync.sync_service import get_sync_service
|
||||
|
||||
|
||||
async def create_benchmark_file(path: Path, file_num: int, total_files: int) -> None:
|
||||
"""Create a realistic test markdown file with observations and relations.
|
||||
|
||||
Args:
|
||||
path: Path to create the file at
|
||||
file_num: Current file number (for unique content)
|
||||
total_files: Total number of files being created (for relation targets)
|
||||
"""
|
||||
# Create realistic content with varying complexity
|
||||
has_relations = file_num < (total_files - 1) # Most files have relations
|
||||
num_observations = min(3 + (file_num % 5), 10) # 3-10 observations per file
|
||||
|
||||
# Generate relation targets (some will be forward references)
|
||||
relations = []
|
||||
if has_relations:
|
||||
# Reference 1-3 other files
|
||||
num_relations = min(1 + (file_num % 3), 3)
|
||||
for i in range(num_relations):
|
||||
target_num = (file_num + i + 1) % total_files
|
||||
relations.append(f"- relates_to [[test-file-{target_num:04d}]]")
|
||||
|
||||
content = dedent(f"""
|
||||
---
|
||||
type: note
|
||||
tags: [benchmark, test, category-{file_num % 10}]
|
||||
---
|
||||
# Test File {file_num:04d}
|
||||
|
||||
This is benchmark test file {file_num} of {total_files}.
|
||||
It contains realistic markdown content to simulate actual usage.
|
||||
|
||||
## Observations
|
||||
{chr(10).join([f"- [category-{i % 5}] Observation {i} for file {file_num} with some content #tag{i}" for i in range(num_observations)])}
|
||||
|
||||
## Relations
|
||||
{chr(10).join(relations) if relations else "- No relations for this file"}
|
||||
|
||||
## Additional Content
|
||||
|
||||
This section contains additional prose to simulate real documents.
|
||||
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed do eiusmod
|
||||
tempor incididunt ut labore et dolore magna aliqua.
|
||||
|
||||
### Subsection
|
||||
|
||||
More content here to make the file realistic. This helps test the
|
||||
full indexing pipeline including content extraction and search indexing.
|
||||
""").strip()
|
||||
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
path.write_text(content, encoding="utf-8")
|
||||
|
||||
|
||||
async def generate_benchmark_files(project_dir: Path, num_files: int) -> None:
|
||||
"""Generate benchmark test files.
|
||||
|
||||
Args:
|
||||
project_dir: Directory to create files in
|
||||
num_files: Number of files to generate
|
||||
"""
|
||||
print(f"\nGenerating {num_files} test files...")
|
||||
start = time.time()
|
||||
|
||||
# Create files in batches for faster generation
|
||||
batch_size = 100
|
||||
for batch_start in range(0, num_files, batch_size):
|
||||
batch_end = min(batch_start + batch_size, num_files)
|
||||
tasks = [
|
||||
create_benchmark_file(
|
||||
project_dir / f"category-{i % 10}" / f"test-file-{i:04d}.md", i, num_files
|
||||
)
|
||||
for i in range(batch_start, batch_end)
|
||||
]
|
||||
await asyncio.gather(*tasks)
|
||||
print(f" Created files {batch_start}-{batch_end} ({batch_end}/{num_files})")
|
||||
|
||||
duration = time.time() - start
|
||||
print(f" File generation completed in {duration:.2f}s ({num_files / duration:.1f} files/sec)")
|
||||
|
||||
|
||||
def get_db_size(db_path: Path) -> tuple[int, str]:
|
||||
"""Get database file size.
|
||||
|
||||
Returns:
|
||||
Tuple of (size_bytes, formatted_size)
|
||||
"""
|
||||
if not db_path.exists():
|
||||
return 0, "0 B"
|
||||
|
||||
size_bytes = db_path.stat().st_size
|
||||
|
||||
# Format size
|
||||
for unit in ["B", "KB", "MB", "GB"]:
|
||||
if size_bytes < 1024.0:
|
||||
return size_bytes, f"{size_bytes:.2f} {unit}"
|
||||
size_bytes /= 1024.0
|
||||
|
||||
return int(size_bytes * 1024**4), f"{size_bytes:.2f} TB"
|
||||
|
||||
|
||||
async def run_sync_benchmark(
|
||||
project_config: ProjectConfig, app_config: BasicMemoryConfig, num_files: int, test_name: str
|
||||
) -> dict:
|
||||
"""Run a sync benchmark and collect metrics.
|
||||
|
||||
Args:
|
||||
project_config: Project configuration
|
||||
app_config: App configuration
|
||||
num_files: Number of files to benchmark
|
||||
test_name: Name of the test for reporting
|
||||
|
||||
Returns:
|
||||
Dictionary with benchmark results
|
||||
"""
|
||||
project_dir = project_config.home
|
||||
db_path = app_config.database_path
|
||||
|
||||
print(f"\n{'=' * 70}")
|
||||
print(f"BENCHMARK: {test_name}")
|
||||
print(f"{'=' * 70}")
|
||||
|
||||
# Generate test files
|
||||
await generate_benchmark_files(project_dir, num_files)
|
||||
|
||||
# Get initial DB size
|
||||
initial_db_size, initial_db_formatted = get_db_size(db_path)
|
||||
print(f"\nInitial database size: {initial_db_formatted}")
|
||||
|
||||
# Create sync service
|
||||
from basic_memory.repository import ProjectRepository
|
||||
from basic_memory import db
|
||||
|
||||
_, session_maker = await db.get_or_create_db(
|
||||
db_path=app_config.database_path,
|
||||
db_type=db.DatabaseType.FILESYSTEM,
|
||||
)
|
||||
project_repository = ProjectRepository(session_maker)
|
||||
|
||||
# Get or create project
|
||||
projects = await project_repository.find_all()
|
||||
if projects:
|
||||
project = projects[0]
|
||||
else:
|
||||
project = await project_repository.create(
|
||||
{
|
||||
"name": project_config.name,
|
||||
"path": str(project_config.home),
|
||||
"is_active": True,
|
||||
"is_default": True,
|
||||
}
|
||||
)
|
||||
|
||||
sync_service = await get_sync_service(project)
|
||||
|
||||
# Initialize search index (required for FTS5 table)
|
||||
await sync_service.search_service.init_search_index()
|
||||
|
||||
# Run sync and measure time
|
||||
print(f"\nStarting sync of {num_files} files...")
|
||||
sync_start = time.time()
|
||||
|
||||
report = await sync_service.sync(project_dir, project_name=project.name)
|
||||
|
||||
sync_duration = time.time() - sync_start
|
||||
|
||||
# Get final DB size
|
||||
final_db_size, final_db_formatted = get_db_size(db_path)
|
||||
db_growth = final_db_size - initial_db_size
|
||||
db_growth_formatted = f"{db_growth / 1024 / 1024:.2f} MB"
|
||||
|
||||
# Calculate metrics
|
||||
files_per_sec = num_files / sync_duration if sync_duration > 0 else 0
|
||||
ms_per_file = (sync_duration * 1000) / num_files if num_files > 0 else 0
|
||||
|
||||
# Print results
|
||||
print(f"\n{'-' * 70}")
|
||||
print("RESULTS:")
|
||||
print(f"{'-' * 70}")
|
||||
print(f"Files processed: {num_files}")
|
||||
print(f" New: {len(report.new)}")
|
||||
print(f" Modified: {len(report.modified)}")
|
||||
print(f" Deleted: {len(report.deleted)}")
|
||||
print(f" Moved: {len(report.moves)}")
|
||||
print("\nPerformance:")
|
||||
print(f" Total time: {sync_duration:.2f}s")
|
||||
print(f" Files/sec: {files_per_sec:.1f}")
|
||||
print(f" ms/file: {ms_per_file:.1f}")
|
||||
print("\nDatabase:")
|
||||
print(f" Initial size: {initial_db_formatted}")
|
||||
print(f" Final size: {final_db_formatted}")
|
||||
print(f" Growth: {db_growth_formatted}")
|
||||
print(f" Growth per file: {(db_growth / num_files / 1024):.2f} KB")
|
||||
print(f"{'=' * 70}\n")
|
||||
|
||||
return {
|
||||
"test_name": test_name,
|
||||
"num_files": num_files,
|
||||
"sync_duration_sec": sync_duration,
|
||||
"files_per_sec": files_per_sec,
|
||||
"ms_per_file": ms_per_file,
|
||||
"new_files": len(report.new),
|
||||
"modified_files": len(report.modified),
|
||||
"deleted_files": len(report.deleted),
|
||||
"moved_files": len(report.moves),
|
||||
"initial_db_size": initial_db_size,
|
||||
"final_db_size": final_db_size,
|
||||
"db_growth_bytes": db_growth,
|
||||
"db_growth_per_file_bytes": db_growth / num_files if num_files > 0 else 0,
|
||||
}
|
||||
|
||||
|
||||
@pytest.mark.benchmark
|
||||
@pytest.mark.asyncio
|
||||
async def test_benchmark_sync_100_files(app_config, project_config, config_manager):
|
||||
"""Benchmark: Sync 100 files (small repository)."""
|
||||
results = await run_sync_benchmark(
|
||||
project_config, app_config, num_files=100, test_name="Sync 100 files (small repository)"
|
||||
)
|
||||
|
||||
# Basic assertions to ensure sync worked
|
||||
# Note: May be slightly more than 100 due to OS-generated files (.DS_Store, etc.)
|
||||
assert results["new_files"] >= 100
|
||||
assert results["sync_duration_sec"] > 0
|
||||
assert results["files_per_sec"] > 0
|
||||
|
||||
|
||||
@pytest.mark.benchmark
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.skip
|
||||
async def test_benchmark_sync_500_files(app_config, project_config, config_manager):
|
||||
"""Benchmark: Sync 500 files (medium repository)."""
|
||||
results = await run_sync_benchmark(
|
||||
project_config, app_config, num_files=500, test_name="Sync 500 files (medium repository)"
|
||||
)
|
||||
|
||||
# Basic assertions
|
||||
# Note: May be slightly more than 500 due to OS-generated files
|
||||
assert results["new_files"] >= 500
|
||||
assert results["sync_duration_sec"] > 0
|
||||
assert results["files_per_sec"] > 0
|
||||
|
||||
|
||||
@pytest.mark.benchmark
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.slow
|
||||
@pytest.mark.skip
|
||||
async def test_benchmark_sync_1000_files(app_config, project_config, config_manager):
|
||||
"""Benchmark: Sync 1000 files (large repository).
|
||||
|
||||
This test is marked as 'slow' and can be skipped in regular test runs:
|
||||
pytest -m "not slow"
|
||||
"""
|
||||
results = await run_sync_benchmark(
|
||||
project_config, app_config, num_files=1000, test_name="Sync 1000 files (large repository)"
|
||||
)
|
||||
|
||||
# Basic assertions
|
||||
# Note: May be slightly more than 1000 due to OS-generated files
|
||||
assert results["new_files"] >= 1000
|
||||
assert results["sync_duration_sec"] > 0
|
||||
assert results["files_per_sec"] > 0
|
||||
|
||||
|
||||
@pytest.mark.benchmark
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.skip
|
||||
async def test_benchmark_resync_no_changes(app_config, project_config, config_manager):
|
||||
"""Benchmark: Re-sync with no changes (should be fast).
|
||||
|
||||
This tests the performance of scanning files when nothing has changed,
|
||||
which is important for cloud restarts.
|
||||
"""
|
||||
project_dir = project_config.home
|
||||
num_files = 100
|
||||
|
||||
# First sync
|
||||
print(f"\nFirst sync of {num_files} files...")
|
||||
await generate_benchmark_files(project_dir, num_files)
|
||||
|
||||
from basic_memory.repository import ProjectRepository
|
||||
from basic_memory import db
|
||||
|
||||
_, session_maker = await db.get_or_create_db(
|
||||
db_path=app_config.database_path,
|
||||
db_type=db.DatabaseType.FILESYSTEM,
|
||||
)
|
||||
project_repository = ProjectRepository(session_maker)
|
||||
projects = await project_repository.find_all()
|
||||
if projects:
|
||||
project = projects[0]
|
||||
else:
|
||||
project = await project_repository.create(
|
||||
{
|
||||
"name": project_config.name,
|
||||
"path": str(project_config.home),
|
||||
"is_active": True,
|
||||
"is_default": True,
|
||||
}
|
||||
)
|
||||
|
||||
sync_service = await get_sync_service(project)
|
||||
|
||||
# Initialize search index
|
||||
await sync_service.search_service.init_search_index()
|
||||
|
||||
await sync_service.sync(project_dir, project_name=project.name)
|
||||
|
||||
# Second sync (no changes)
|
||||
print("\nRe-sync with no changes...")
|
||||
resync_start = time.time()
|
||||
report = await sync_service.sync(project_dir, project_name=project.name)
|
||||
resync_duration = time.time() - resync_start
|
||||
|
||||
print(f"\n{'-' * 70}")
|
||||
print("RE-SYNC RESULTS (no changes):")
|
||||
print(f"{'-' * 70}")
|
||||
print(f"Files scanned: {num_files}")
|
||||
print(f"Changes detected: {report.total}")
|
||||
print(f" New: {len(report.new)}")
|
||||
print(f" Modified: {len(report.modified)}")
|
||||
print(f" Deleted: {len(report.deleted)}")
|
||||
print(f" Moved: {len(report.moves)}")
|
||||
print(f"Duration: {resync_duration:.2f}s")
|
||||
print(f"Files/sec: {num_files / resync_duration:.1f}")
|
||||
|
||||
# Debug: Show what changed
|
||||
if report.total > 0:
|
||||
print("\n⚠️ UNEXPECTED CHANGES DETECTED:")
|
||||
if report.new:
|
||||
print(f" New files ({len(report.new)}): {list(report.new)[:5]}")
|
||||
if report.modified:
|
||||
print(f" Modified files ({len(report.modified)}): {list(report.modified)[:5]}")
|
||||
if report.deleted:
|
||||
print(f" Deleted files ({len(report.deleted)}): {list(report.deleted)[:5]}")
|
||||
if report.moves:
|
||||
print(f" Moved files ({len(report.moves)}): {dict(list(report.moves.items())[:5])}")
|
||||
|
||||
print(f"{'=' * 70}\n")
|
||||
|
||||
# Should be no changes
|
||||
assert report.total == 0, (
|
||||
f"Expected no changes but got {report.total}: new={len(report.new)}, modified={len(report.modified)}, deleted={len(report.deleted)}, moves={len(report.moves)}"
|
||||
)
|
||||
assert len(report.new) == 0
|
||||
assert len(report.modified) == 0
|
||||
assert len(report.deleted) == 0
|
||||
@@ -9,17 +9,19 @@ from basic_memory.mcp.async_client import create_client
|
||||
|
||||
|
||||
def test_create_client_uses_asgi_when_no_remote_env():
|
||||
"""Test that create_client uses ASGI transport when BASIC_MEMORY_USE_REMOTE_API is not set."""
|
||||
# Ensure env vars are not set (pop if they exist)
|
||||
"""Test that create_client uses ASGI transport when cloud mode is disabled."""
|
||||
# Ensure env vars are not set and config cloud_mode is False
|
||||
with patch.dict("os.environ", clear=False):
|
||||
os.environ.pop("BASIC_MEMORY_USE_REMOTE_API", None)
|
||||
os.environ.pop("BASIC_MEMORY_CLOUD_MODE", None)
|
||||
|
||||
client = create_client()
|
||||
# Also patch the config's cloud_mode to ensure it's False
|
||||
with patch.object(ConfigManager().config, "cloud_mode", False):
|
||||
client = create_client()
|
||||
|
||||
assert isinstance(client, AsyncClient)
|
||||
assert isinstance(client._transport, ASGITransport)
|
||||
assert str(client.base_url) == "http://test"
|
||||
assert isinstance(client, AsyncClient)
|
||||
assert isinstance(client._transport, ASGITransport)
|
||||
assert str(client.base_url) == "http://test"
|
||||
|
||||
|
||||
def test_create_client_uses_http_when_cloud_mode_env_set():
|
||||
@@ -37,16 +39,18 @@ def test_create_client_uses_http_when_cloud_mode_env_set():
|
||||
|
||||
def test_create_client_configures_extended_timeouts():
|
||||
"""Test that create_client configures 30-second timeouts for long operations."""
|
||||
# Ensure env vars are not set (pop if they exist)
|
||||
# Ensure env vars are not set and config cloud_mode is False
|
||||
with patch.dict("os.environ", clear=False):
|
||||
os.environ.pop("BASIC_MEMORY_USE_REMOTE_API", None)
|
||||
os.environ.pop("BASIC_MEMORY_CLOUD_MODE", None)
|
||||
|
||||
client = create_client()
|
||||
# Also patch the config's cloud_mode to ensure it's False
|
||||
with patch.object(ConfigManager().config, "cloud_mode", False):
|
||||
client = create_client()
|
||||
|
||||
# Verify timeout configuration
|
||||
assert isinstance(client.timeout, Timeout)
|
||||
assert client.timeout.connect == 10.0 # 10 seconds for connection
|
||||
assert client.timeout.read == 30.0 # 30 seconds for reading
|
||||
assert client.timeout.write == 30.0 # 30 seconds for writing
|
||||
assert client.timeout.pool == 30.0 # 30 seconds for pool
|
||||
# Verify timeout configuration
|
||||
assert isinstance(client.timeout, Timeout)
|
||||
assert client.timeout.connect == 10.0 # 10 seconds for connection
|
||||
assert client.timeout.read == 30.0 # 30 seconds for reading
|
||||
assert client.timeout.write == 30.0 # 30 seconds for writing
|
||||
assert client.timeout.pool == 30.0 # 30 seconds for pool
|
||||
|
||||
@@ -218,9 +218,13 @@ async def test_get_resource_entities(client, project_config, entity_repository,
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_get_resource_entities_pagination(
|
||||
client, project_config, entity_repository, project_url
|
||||
client, project_config, entity_repository, project_url, db_backend
|
||||
):
|
||||
"""Test getting content by permalink match."""
|
||||
if db_backend == "postgres":
|
||||
pytest.skip(
|
||||
"Pagination differs: relations expand to multiple entities, ordering is undefined"
|
||||
)
|
||||
# Create entity
|
||||
content1 = "# Test Content\n"
|
||||
data = {
|
||||
|
||||
+28
-1
@@ -1,5 +1,8 @@
|
||||
import os
|
||||
from pathlib import Path
|
||||
from typing import AsyncGenerator
|
||||
|
||||
import pytest
|
||||
import pytest_asyncio
|
||||
from fastapi import FastAPI
|
||||
from httpx import AsyncClient, ASGITransport
|
||||
@@ -8,7 +11,31 @@ from basic_memory.api.app import app as fastapi_app
|
||||
from basic_memory.deps import get_project_config, get_engine_factory, get_app_config
|
||||
|
||||
|
||||
@pytest_asyncio.fixture(autouse=True)
|
||||
@pytest.fixture(autouse=True)
|
||||
def isolated_home(tmp_path, monkeypatch) -> Path:
|
||||
"""Isolate tests from user's HOME directory.
|
||||
|
||||
This prevents tests from reading/writing to ~/.basic-memory/.bmignore
|
||||
or other user-specific configuration.
|
||||
|
||||
Sets BASIC_MEMORY_HOME to tmp_path directly so the default project
|
||||
writes files to tmp_path, which is where tests expect to find them.
|
||||
"""
|
||||
# Clear config cache to ensure fresh config for each test
|
||||
from basic_memory import config as config_module
|
||||
|
||||
config_module._CONFIG_CACHE = None
|
||||
|
||||
monkeypatch.setenv("HOME", str(tmp_path))
|
||||
if os.name == "nt":
|
||||
monkeypatch.setenv("USERPROFILE", str(tmp_path))
|
||||
# Set to tmp_path directly (not tmp_path/basic-memory) so default project
|
||||
# home is tmp_path - tests expect to find imported files there
|
||||
monkeypatch.setenv("BASIC_MEMORY_HOME", str(tmp_path))
|
||||
return tmp_path
|
||||
|
||||
|
||||
@pytest_asyncio.fixture
|
||||
async def app(app_config, project_config, engine_factory, test_config, aiolib) -> FastAPI:
|
||||
"""Create test FastAPI application."""
|
||||
app = fastapi_app
|
||||
|
||||
+113
-93
@@ -1,16 +1,23 @@
|
||||
"""Common test fixtures."""
|
||||
|
||||
import os
|
||||
from dataclasses import dataclass
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
from textwrap import dedent
|
||||
from typing import AsyncGenerator, Literal
|
||||
from typing import AsyncGenerator
|
||||
|
||||
import os
|
||||
import pytest
|
||||
import pytest_asyncio
|
||||
from sqlalchemy import text
|
||||
from sqlalchemy.ext.asyncio import AsyncEngine, AsyncSession, async_sessionmaker
|
||||
from sqlalchemy.ext.asyncio import (
|
||||
AsyncEngine,
|
||||
AsyncSession,
|
||||
async_sessionmaker,
|
||||
create_async_engine,
|
||||
)
|
||||
from sqlalchemy.pool import NullPool
|
||||
from testcontainers.postgres import PostgresContainer
|
||||
|
||||
from basic_memory import db
|
||||
from basic_memory.config import ProjectConfig, BasicMemoryConfig, ConfigManager, DatabaseBackend
|
||||
@@ -37,32 +44,47 @@ from basic_memory.sync.sync_service import SyncService
|
||||
from basic_memory.sync.watch_service import WatchService
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# Database Backend Selection (env var approach)
|
||||
# =============================================================================
|
||||
# By default, tests run against SQLite.
|
||||
# Set BASIC_MEMORY_TEST_POSTGRES=1 to run against Postgres (uses testcontainers).
|
||||
# This allows running sqlite/postgres tests in parallel in CI.
|
||||
|
||||
|
||||
@pytest.fixture(scope="session")
|
||||
def db_backend():
|
||||
"""Determine database backend from environment variable.
|
||||
|
||||
Default: sqlite
|
||||
Set BASIC_MEMORY_TEST_POSTGRES=1 to use postgres
|
||||
"""
|
||||
if os.environ.get("BASIC_MEMORY_TEST_POSTGRES", "").lower() in ("1", "true", "yes"):
|
||||
return "postgres"
|
||||
return "sqlite"
|
||||
|
||||
|
||||
@pytest.fixture(scope="session")
|
||||
def postgres_container(db_backend):
|
||||
"""Session-scoped Postgres container for tests.
|
||||
|
||||
Uses testcontainers to spin up a real Postgres instance in Docker.
|
||||
The container is started once per test session and shared across all tests.
|
||||
Only starts if db_backend is "postgres".
|
||||
"""
|
||||
if db_backend != "postgres":
|
||||
yield None
|
||||
return
|
||||
|
||||
with PostgresContainer("postgres:16-alpine") as postgres:
|
||||
yield postgres
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def anyio_backend():
|
||||
return "asyncio"
|
||||
|
||||
|
||||
@pytest.fixture(
|
||||
params=[
|
||||
pytest.param("sqlite", id="sqlite"),
|
||||
pytest.param("postgres", id="postgres", marks=pytest.mark.postgres),
|
||||
]
|
||||
)
|
||||
def db_backend(request) -> Literal["sqlite", "postgres"]:
|
||||
"""Parametrize tests to run against both SQLite and Postgres.
|
||||
|
||||
Usage:
|
||||
pytest # Runs tests against SQLite only (default)
|
||||
pytest -m postgres # Runs tests against Postgres only
|
||||
pytest -m "not postgres" # Runs tests against SQLite only
|
||||
pytest --run-all-backends # Runs tests against both backends
|
||||
|
||||
Note: Only tests that use database fixtures (engine_factory, session_maker, etc.)
|
||||
will be parametrized. Tests that don't use the database won't be affected.
|
||||
"""
|
||||
return request.param
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def project_root() -> Path:
|
||||
return Path(__file__).parent.parent
|
||||
@@ -81,24 +103,18 @@ def config_home(tmp_path, monkeypatch) -> Path:
|
||||
|
||||
|
||||
@pytest.fixture(scope="function")
|
||||
def app_config(
|
||||
config_home, db_backend: Literal["sqlite", "postgres"], monkeypatch
|
||||
) -> BasicMemoryConfig:
|
||||
"""Create test app configuration."""
|
||||
# Create a basic config without depending on test_project to avoid circular dependency
|
||||
def app_config(config_home, db_backend, postgres_container, monkeypatch) -> BasicMemoryConfig:
|
||||
"""Create test app configuration for the appropriate backend."""
|
||||
projects = {"test-project": str(config_home)}
|
||||
|
||||
# Configure database backend based on test parameter
|
||||
# Set backend based on parameterized db_backend fixture
|
||||
if db_backend == "postgres":
|
||||
database_backend = DatabaseBackend.POSTGRES
|
||||
# Use env var if set, otherwise use default matching docker-compose-postgres.yml
|
||||
# These are local test credentials only - NOT for production
|
||||
database_url = os.getenv(
|
||||
"POSTGRES_TEST_URL",
|
||||
"postgresql+asyncpg://basic_memory_user:dev_password@localhost:5433/basic_memory_test",
|
||||
)
|
||||
backend = DatabaseBackend.POSTGRES
|
||||
# Get URL from testcontainer and convert to asyncpg driver
|
||||
sync_url = postgres_container.get_connection_url()
|
||||
database_url = sync_url.replace("postgresql+psycopg2", "postgresql+asyncpg")
|
||||
else:
|
||||
database_backend = DatabaseBackend.SQLITE
|
||||
backend = DatabaseBackend.SQLITE
|
||||
database_url = None
|
||||
|
||||
app_config = BasicMemoryConfig(
|
||||
@@ -106,7 +122,7 @@ def app_config(
|
||||
projects=projects,
|
||||
default_project="test-project",
|
||||
update_permalinks_on_move=True,
|
||||
database_backend=database_backend,
|
||||
database_backend=backend,
|
||||
database_url=database_url,
|
||||
)
|
||||
|
||||
@@ -162,76 +178,66 @@ def test_config(config_home, project_config, app_config, config_manager) -> Test
|
||||
async def engine_factory(
|
||||
app_config,
|
||||
config_manager,
|
||||
db_backend: Literal["sqlite", "postgres"],
|
||||
db_backend,
|
||||
postgres_container,
|
||||
) -> AsyncGenerator[tuple[AsyncEngine, async_sessionmaker[AsyncSession]], None]:
|
||||
"""Create engine and session factory for the configured database backend."""
|
||||
"""Engine factory for SQLite or Postgres tests.
|
||||
|
||||
Uses parameterized db_backend fixture to run tests against both backends.
|
||||
"""
|
||||
from basic_memory.models.search import CREATE_SEARCH_INDEX
|
||||
|
||||
if db_backend == "postgres":
|
||||
# Postgres: Create fresh engine for each test with full schema reset
|
||||
config_manager._config = app_config
|
||||
db_type = DatabaseType.FILESYSTEM
|
||||
# Postgres mode using testcontainers
|
||||
# Get async connection URL (asyncpg driver - same as production)
|
||||
sync_url = postgres_container.get_connection_url()
|
||||
async_url = sync_url.replace("postgresql+psycopg2", "postgresql+asyncpg")
|
||||
|
||||
# Use context manager to handle engine disposal properly
|
||||
async with db.engine_session_factory(db_path=app_config.database_path, db_type=db_type) as (
|
||||
engine,
|
||||
session_maker,
|
||||
):
|
||||
# Drop and recreate schema for complete isolation
|
||||
async with engine.begin() as conn:
|
||||
await conn.execute(text("DROP SCHEMA IF EXISTS public CASCADE"))
|
||||
await conn.execute(text("CREATE SCHEMA public"))
|
||||
await conn.execute(text("GRANT ALL ON SCHEMA public TO basic_memory_user"))
|
||||
await conn.execute(text("GRANT ALL ON SCHEMA public TO public"))
|
||||
engine = create_async_engine(
|
||||
async_url,
|
||||
echo=False,
|
||||
poolclass=NullPool, # NullPool for better test isolation
|
||||
)
|
||||
|
||||
# Run migrations to create production tables (including search_index with correct schema)
|
||||
# Alembic handles duplicate migration checks, so it's safe to call this for each test
|
||||
from basic_memory.db import run_migrations
|
||||
session_maker = async_sessionmaker(
|
||||
bind=engine,
|
||||
class_=AsyncSession,
|
||||
expire_on_commit=False,
|
||||
autoflush=False,
|
||||
)
|
||||
|
||||
await run_migrations(app_config, db_type)
|
||||
from basic_memory.models.search import (
|
||||
CREATE_POSTGRES_SEARCH_INDEX_TABLE,
|
||||
CREATE_POSTGRES_SEARCH_INDEX_FTS,
|
||||
CREATE_POSTGRES_SEARCH_INDEX_METADATA,
|
||||
)
|
||||
|
||||
# For Postgres, migrations create all production tables with correct schemas
|
||||
# We only need to create test-specific tables (like ModelTest) that aren't in migrations
|
||||
# Don't create search_index via ORM - it's already created by migration with composite PK
|
||||
async with engine.begin() as conn:
|
||||
# List of tables created by migrations - don't recreate them via ORM
|
||||
production_tables = {
|
||||
"entity",
|
||||
"observation",
|
||||
"relation",
|
||||
"project",
|
||||
"search_index",
|
||||
"alembic_version",
|
||||
}
|
||||
# Drop and recreate all tables for test isolation
|
||||
async with engine.begin() as conn:
|
||||
# Must drop search_index first (has FK to project, blocks drop_all)
|
||||
await conn.execute(text("DROP TABLE IF EXISTS search_index CASCADE"))
|
||||
await conn.run_sync(Base.metadata.drop_all)
|
||||
await conn.run_sync(Base.metadata.create_all)
|
||||
# Create search_index via DDL (not ORM - uses composite PK + tsvector)
|
||||
# asyncpg requires separate execute calls for each statement
|
||||
await conn.execute(CREATE_POSTGRES_SEARCH_INDEX_TABLE)
|
||||
await conn.execute(CREATE_POSTGRES_SEARCH_INDEX_FTS)
|
||||
await conn.execute(CREATE_POSTGRES_SEARCH_INDEX_METADATA)
|
||||
|
||||
# Get test-specific tables that aren't created by migrations
|
||||
test_tables = [
|
||||
table
|
||||
for table in Base.metadata.sorted_tables
|
||||
if table.name not in production_tables
|
||||
]
|
||||
if test_tables:
|
||||
await conn.run_sync(
|
||||
lambda sync_conn: Base.metadata.create_all(sync_conn, tables=test_tables)
|
||||
)
|
||||
yield engine, session_maker
|
||||
|
||||
yield engine, session_maker
|
||||
await engine.dispose()
|
||||
else:
|
||||
# SQLite: Create fresh in-memory database for each test
|
||||
# SQLite mode
|
||||
db_type = DatabaseType.MEMORY
|
||||
async with db.engine_session_factory(db_path=app_config.database_path, db_type=db_type) as (
|
||||
engine,
|
||||
session_maker,
|
||||
):
|
||||
# Create all tables via ORM
|
||||
# Create all tables via ORM, then add search_index via FTS5 DDL
|
||||
async with engine.begin() as conn:
|
||||
await conn.run_sync(Base.metadata.create_all)
|
||||
|
||||
# Drop any SearchIndex ORM table, then create FTS5 virtual table
|
||||
async with db.scoped_session(session_maker) as session:
|
||||
await session.execute(text("DROP TABLE IF EXISTS search_index"))
|
||||
await session.execute(CREATE_SEARCH_INDEX)
|
||||
await session.commit()
|
||||
await conn.execute(CREATE_SEARCH_INDEX)
|
||||
|
||||
# Yield after setup is complete
|
||||
yield engine, session_maker
|
||||
@@ -543,8 +549,22 @@ async def test_graph(
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def watch_service(app_config: BasicMemoryConfig, project_repository) -> WatchService:
|
||||
return WatchService(app_config=app_config, project_repository=project_repository)
|
||||
def watch_service(app_config: BasicMemoryConfig, project_repository, sync_service) -> WatchService:
|
||||
"""Create WatchService with injected sync_service factory.
|
||||
|
||||
The sync_service_factory allows tests to use the fixture-provided sync_service
|
||||
instead of the production get_sync_service() which creates its own db connection.
|
||||
"""
|
||||
|
||||
async def sync_service_factory(project):
|
||||
"""Return the test fixture's sync_service regardless of project."""
|
||||
return sync_service
|
||||
|
||||
return WatchService(
|
||||
app_config=app_config,
|
||||
project_repository=project_repository,
|
||||
sync_service_factory=sync_service_factory,
|
||||
)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
|
||||
@@ -85,11 +85,11 @@ async def test_parse_complete_file(project_config, entity_parser, valid_entity_c
|
||||
), "missing [[Auth API Spec]]"
|
||||
|
||||
# inline links in content
|
||||
assert Relation(type="links to", target="Random Link", context=None) in entity.relations, (
|
||||
assert Relation(type="links_to", target="Random Link", context=None) in entity.relations, (
|
||||
"missing [[Random Link]]"
|
||||
)
|
||||
assert (
|
||||
Relation(type="links to", target="Random Link with Title|Titled Link", context=None)
|
||||
Relation(type="links_to", target="Random Link with Title|Titled Link", context=None)
|
||||
in entity.relations
|
||||
), "missing [[Random Link with Title|Titled Link]]"
|
||||
|
||||
@@ -179,7 +179,7 @@ async def test_parse_file_without_section_headers(project_config, entity_parser)
|
||||
assert entity.observations[0].tags == ["test"]
|
||||
|
||||
assert len(entity.relations) == 2
|
||||
assert entity.relations[0].type == "links to"
|
||||
assert entity.relations[0].type == "links_to"
|
||||
assert entity.relations[0].target == "Random Link"
|
||||
|
||||
assert entity.relations[1].type == "references"
|
||||
|
||||
@@ -121,6 +121,44 @@ def test_observation_excludes_markdown_and_wiki_links():
|
||||
assert not is_observation(token), "No space after category should not be valid observation"
|
||||
|
||||
|
||||
def test_observation_excludes_html_color_codes():
|
||||
"""Test that HTML color codes are NOT interpreted as hashtags.
|
||||
|
||||
This test validates the fix for issue #446 where:
|
||||
- HTML color codes like #4285F4 in attributes were incorrectly
|
||||
causing lines to be parsed as observations.
|
||||
"""
|
||||
# HTML color code in font tag should NOT be an observation
|
||||
token = Token("inline", '**<font color="#4285F4">Jane:</font>** Welcome to the show', 0)
|
||||
assert not is_observation(token), "HTML color codes should not trigger hashtag detection"
|
||||
|
||||
# Color code in style attribute
|
||||
token = Token("inline", '<span style="color:#FF5733">Styled text</span>', 0)
|
||||
assert not is_observation(token), "Color codes in style should not be observations"
|
||||
|
||||
# Multiple color codes
|
||||
token = Token(
|
||||
"inline", '<font color="#4285F4">Blue</font> and <font color="#EA4335">Red</font>', 0
|
||||
)
|
||||
assert not is_observation(token), "Multiple color codes should not be observations"
|
||||
|
||||
# Hex color without quotes (edge case)
|
||||
token = Token("inline", "background-color:#FFFFFF is white", 0)
|
||||
assert not is_observation(token), "Inline hex colors should not be observations"
|
||||
|
||||
# But standalone hashtags SHOULD still work
|
||||
token = Token("inline", "This has a #realtag in it", 0)
|
||||
assert is_observation(token), "Standalone hashtags should still work"
|
||||
|
||||
# Multiple real hashtags
|
||||
token = Token("inline", "Tags: #design #feature #important", 0)
|
||||
assert is_observation(token), "Multiple standalone hashtags should work"
|
||||
|
||||
# Mix of color code and real tag - should be observation because of real tag
|
||||
token = Token("inline", '<font color="#4285F4">Text</font> #actualtag', 0)
|
||||
assert is_observation(token), "Real hashtag with color code should still be observation"
|
||||
|
||||
|
||||
def test_relation_plugin():
|
||||
"""Test relation plugin."""
|
||||
md = MarkdownIt().use(relation_plugin)
|
||||
@@ -143,7 +181,7 @@ def test_relation_plugin():
|
||||
token = [t for t in md.parse(content) if t.type == "inline"][0]
|
||||
rels = token.meta["relations"]
|
||||
assert len(rels) == 2
|
||||
assert rels[0]["type"] == "links to"
|
||||
assert rels[0]["type"] == "links_to"
|
||||
assert rels[0]["target"] == "Link"
|
||||
assert rels[1]["target"] == "Another Link"
|
||||
|
||||
@@ -208,4 +246,4 @@ def test_combined_plugins():
|
||||
text_token = inline_tokens[4]
|
||||
assert "relations" in text_token.meta
|
||||
link = text_token.meta["relations"][0]
|
||||
assert link["type"] == "links to"
|
||||
assert link["type"] == "links_to"
|
||||
|
||||
@@ -88,7 +88,7 @@ async def test_missing_sections(tmp_path):
|
||||
entity = await parser.parse_file(test_file)
|
||||
assert len(entity.relations) == 1
|
||||
assert entity.relations[0].target == "links"
|
||||
assert entity.relations[0].type == "links to"
|
||||
assert entity.relations[0].type == "links_to"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
|
||||
@@ -50,6 +50,7 @@ def test_entity_data():
|
||||
}
|
||||
|
||||
|
||||
@pytest_asyncio.fixture(autouse=True)
|
||||
@pytest_asyncio.fixture
|
||||
async def init_search_index(search_service: SearchService):
|
||||
"""Initialize search index. Request this fixture explicitly in tests that need it."""
|
||||
await search_service.init_search_index()
|
||||
|
||||
@@ -1,7 +1,9 @@
|
||||
"""Tests for the move_note MCP tool."""
|
||||
|
||||
import pytest
|
||||
from unittest.mock import patch
|
||||
from pathlib import Path
|
||||
from unittest.mock import patch, MagicMock
|
||||
from contextlib import asynccontextmanager
|
||||
|
||||
from basic_memory.mcp.tools.move_note import move_note, _format_move_error_response
|
||||
from basic_memory.mcp.tools.write_note import write_note
|
||||
@@ -887,36 +889,71 @@ class TestMoveNoteSecurityValidation:
|
||||
|
||||
|
||||
class TestMoveNoteErrorHandling:
|
||||
"""Test move note exception handling."""
|
||||
"""Test move note exception handling.
|
||||
|
||||
These are pure unit tests that mock get_client and other dependencies.
|
||||
They don't need the database or ASGI app.
|
||||
"""
|
||||
|
||||
@pytest.fixture
|
||||
def mock_client(self):
|
||||
"""Create a mock async client context manager."""
|
||||
mock = MagicMock()
|
||||
|
||||
@asynccontextmanager
|
||||
async def mock_get_client():
|
||||
yield mock
|
||||
|
||||
return mock_get_client, mock
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_move_note_exception_handling(self):
|
||||
async def test_move_note_exception_handling(self, mock_client):
|
||||
"""Test exception handling in move_note."""
|
||||
with patch("basic_memory.mcp.tools.move_note.get_active_project") as mock_get_project:
|
||||
mock_get_project.return_value.project_url = "http://test"
|
||||
mock_get_project.return_value.name = "test-project"
|
||||
mock_get_client, _ = mock_client
|
||||
|
||||
with patch(
|
||||
"basic_memory.mcp.tools.move_note.call_post",
|
||||
side_effect=Exception("entity not found"),
|
||||
):
|
||||
result = await move_note.fn("test-note", "target/file.md", project="test-project")
|
||||
with patch("basic_memory.mcp.tools.move_note.get_client", mock_get_client):
|
||||
with patch("basic_memory.mcp.tools.move_note.get_active_project") as mock_get_project:
|
||||
mock_get_project.return_value.project_url = "http://test"
|
||||
mock_get_project.return_value.name = "test-project"
|
||||
mock_get_project.return_value.home = Path("/tmp/test")
|
||||
|
||||
assert isinstance(result, str)
|
||||
assert "# Move Failed - Note Not Found" in result
|
||||
with patch(
|
||||
"basic_memory.mcp.tools.move_note.call_post",
|
||||
side_effect=Exception("entity not found"),
|
||||
):
|
||||
with patch(
|
||||
"basic_memory.mcp.tools.move_note.call_get",
|
||||
side_effect=Exception("not found"),
|
||||
):
|
||||
result = await move_note.fn(
|
||||
"test-note", "target/file.md", project="test-project"
|
||||
)
|
||||
|
||||
assert isinstance(result, str)
|
||||
assert "# Move Failed - Note Not Found" in result
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_move_note_permission_error_handling(self):
|
||||
async def test_move_note_permission_error_handling(self, mock_client):
|
||||
"""Test permission error handling in move_note."""
|
||||
with patch("basic_memory.mcp.tools.move_note.get_active_project") as mock_get_project:
|
||||
mock_get_project.return_value.project_url = "http://test"
|
||||
mock_get_project.return_value.name = "test-project"
|
||||
mock_get_client, _ = mock_client
|
||||
|
||||
with patch(
|
||||
"basic_memory.mcp.tools.move_note.call_post",
|
||||
side_effect=Exception("permission denied"),
|
||||
):
|
||||
result = await move_note.fn("test-note", "target/file.md", project="test-project")
|
||||
with patch("basic_memory.mcp.tools.move_note.get_client", mock_get_client):
|
||||
with patch("basic_memory.mcp.tools.move_note.get_active_project") as mock_get_project:
|
||||
mock_get_project.return_value.project_url = "http://test"
|
||||
mock_get_project.return_value.name = "test-project"
|
||||
mock_get_project.return_value.home = Path("/tmp/test")
|
||||
|
||||
assert isinstance(result, str)
|
||||
assert "# Move Failed - Permission Error" in result
|
||||
with patch(
|
||||
"basic_memory.mcp.tools.move_note.call_post",
|
||||
side_effect=Exception("permission denied"),
|
||||
):
|
||||
with patch(
|
||||
"basic_memory.mcp.tools.move_note.call_get",
|
||||
side_effect=Exception("not found"),
|
||||
):
|
||||
result = await move_note.fn(
|
||||
"test-note", "target/file.md", project="test-project"
|
||||
)
|
||||
|
||||
assert isinstance(result, str)
|
||||
assert "# Move Failed - Permission Error" in result
|
||||
|
||||
@@ -18,10 +18,12 @@ async def entity_with_observations(session_maker, sample_entity):
|
||||
async with db.scoped_session(session_maker) as session:
|
||||
observations = [
|
||||
Observation(
|
||||
project_id=sample_entity.project_id,
|
||||
entity_id=sample_entity.id,
|
||||
content="First observation",
|
||||
),
|
||||
Observation(
|
||||
project_id=sample_entity.project_id,
|
||||
entity_id=sample_entity.id,
|
||||
content="Second observation",
|
||||
),
|
||||
@@ -59,6 +61,7 @@ async def related_results(session_maker, test_project: Project):
|
||||
await session.flush()
|
||||
|
||||
relation = Relation(
|
||||
project_id=test_project.id,
|
||||
from_id=source.id,
|
||||
to_id=target.id,
|
||||
to_name=target.title,
|
||||
@@ -176,6 +179,55 @@ async def test_update_entity(entity_repository: EntityRepository, sample_entity:
|
||||
assert db_entity.title == "Updated title"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_update_entity_returns_with_relations_and_observations(
|
||||
entity_repository: EntityRepository, entity_with_observations, test_project: Project
|
||||
):
|
||||
"""Test that update() returns entity with observations and relations eagerly loaded."""
|
||||
entity = entity_with_observations
|
||||
|
||||
# Create a target entity and relation
|
||||
async with db.scoped_session(entity_repository.session_maker) as session:
|
||||
target = Entity(
|
||||
project_id=test_project.id,
|
||||
title="target",
|
||||
entity_type="test",
|
||||
permalink="target/target",
|
||||
file_path="target/target.md",
|
||||
content_type="text/markdown",
|
||||
created_at=datetime.now(timezone.utc),
|
||||
updated_at=datetime.now(timezone.utc),
|
||||
)
|
||||
session.add(target)
|
||||
await session.flush()
|
||||
|
||||
relation = Relation(
|
||||
project_id=test_project.id,
|
||||
from_id=entity.id,
|
||||
to_id=target.id,
|
||||
to_name=target.title,
|
||||
relation_type="connects_to",
|
||||
)
|
||||
session.add(relation)
|
||||
|
||||
# Now update the entity
|
||||
updated = await entity_repository.update(entity.id, {"title": "Updated with relations"})
|
||||
|
||||
# Verify returned entity has observations and relations accessible
|
||||
# (would raise DetachedInstanceError if not eagerly loaded)
|
||||
assert updated is not None
|
||||
assert updated.title == "Updated with relations"
|
||||
|
||||
# Access observations - should NOT raise DetachedInstanceError
|
||||
assert len(updated.observations) == 2
|
||||
assert updated.observations[0].content in ["First observation", "Second observation"]
|
||||
|
||||
# Access relations - should NOT raise DetachedInstanceError
|
||||
assert len(updated.relations) == 1
|
||||
assert updated.relations[0].relation_type == "connects_to"
|
||||
assert updated.relations[0].to_name == "target"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_delete_entity(entity_repository: EntityRepository, sample_entity):
|
||||
"""Test deleting an entity."""
|
||||
@@ -737,6 +789,7 @@ async def test_get_all_file_paths_performance(entity_repository: EntityRepositor
|
||||
|
||||
# Add observations to entity1
|
||||
observation = Observation(
|
||||
project_id=entity_repository.project_id,
|
||||
entity_id=entity1.id,
|
||||
content="Test observation",
|
||||
category="note",
|
||||
@@ -745,6 +798,7 @@ async def test_get_all_file_paths_performance(entity_repository: EntityRepositor
|
||||
|
||||
# Add relation between entities
|
||||
relation = Relation(
|
||||
project_id=entity_repository.project_id,
|
||||
from_id=entity1.id,
|
||||
to_id=entity2.id,
|
||||
to_name=entity2.title,
|
||||
@@ -810,3 +864,191 @@ async def test_get_all_file_paths_project_isolation(
|
||||
# Should only include files from project 1
|
||||
assert len(file_paths) == 1
|
||||
assert file_paths == ["test/file1.md"]
|
||||
|
||||
|
||||
# -------------------------------------------------------------------------
|
||||
# Tests for lightweight permalink resolution methods
|
||||
# -------------------------------------------------------------------------
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_permalink_exists(entity_repository: EntityRepository, sample_entity: Entity):
|
||||
"""Test checking if a permalink exists without loading full entity."""
|
||||
# Existing permalink should return True
|
||||
assert await entity_repository.permalink_exists(sample_entity.permalink) is True # pyright: ignore [reportArgumentType]
|
||||
|
||||
# Non-existent permalink should return False
|
||||
assert await entity_repository.permalink_exists("nonexistent/permalink") is False
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_permalink_exists_project_isolation(
|
||||
entity_repository: EntityRepository, session_maker
|
||||
):
|
||||
"""Test that permalink_exists respects project isolation."""
|
||||
async with db.scoped_session(session_maker) as session:
|
||||
# Create entity in repository's project
|
||||
entity1 = Entity(
|
||||
project_id=entity_repository.project_id,
|
||||
title="Project 1 Entity",
|
||||
entity_type="test",
|
||||
permalink="test/entity1",
|
||||
file_path="test/entity1.md",
|
||||
content_type="text/markdown",
|
||||
created_at=datetime.now(timezone.utc),
|
||||
updated_at=datetime.now(timezone.utc),
|
||||
)
|
||||
session.add(entity1)
|
||||
|
||||
# Create a second project with same permalink
|
||||
project2 = Project(name="other-project", path="/tmp/other")
|
||||
session.add(project2)
|
||||
await session.flush()
|
||||
|
||||
entity2 = Entity(
|
||||
project_id=project2.id,
|
||||
title="Project 2 Entity",
|
||||
entity_type="test",
|
||||
permalink="test/entity2",
|
||||
file_path="test/entity2.md",
|
||||
content_type="text/markdown",
|
||||
created_at=datetime.now(timezone.utc),
|
||||
updated_at=datetime.now(timezone.utc),
|
||||
)
|
||||
session.add(entity2)
|
||||
|
||||
# Should find entity1's permalink in project 1
|
||||
assert await entity_repository.permalink_exists("test/entity1") is True
|
||||
|
||||
# Should NOT find entity2's permalink (it's in project 2)
|
||||
assert await entity_repository.permalink_exists("test/entity2") is False
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_get_file_path_for_permalink(
|
||||
entity_repository: EntityRepository, sample_entity: Entity
|
||||
):
|
||||
"""Test getting file_path for a permalink without loading full entity."""
|
||||
# Existing permalink should return file_path
|
||||
file_path = await entity_repository.get_file_path_for_permalink(sample_entity.permalink) # pyright: ignore [reportArgumentType]
|
||||
assert file_path == sample_entity.file_path
|
||||
|
||||
# Non-existent permalink should return None
|
||||
result = await entity_repository.get_file_path_for_permalink("nonexistent/permalink")
|
||||
assert result is None
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_get_permalink_for_file_path(
|
||||
entity_repository: EntityRepository, sample_entity: Entity
|
||||
):
|
||||
"""Test getting permalink for a file_path without loading full entity."""
|
||||
# Existing file_path should return permalink
|
||||
permalink = await entity_repository.get_permalink_for_file_path(sample_entity.file_path)
|
||||
assert permalink == sample_entity.permalink
|
||||
|
||||
# Non-existent file_path should return None
|
||||
result = await entity_repository.get_permalink_for_file_path("nonexistent/path.md")
|
||||
assert result is None
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_get_all_permalinks(entity_repository: EntityRepository, session_maker):
|
||||
"""Test getting all permalinks without loading full entities."""
|
||||
async with db.scoped_session(session_maker) as session:
|
||||
entity1 = Entity(
|
||||
project_id=entity_repository.project_id,
|
||||
title="Entity 1",
|
||||
entity_type="test",
|
||||
permalink="test/entity1",
|
||||
file_path="test/entity1.md",
|
||||
content_type="text/markdown",
|
||||
created_at=datetime.now(timezone.utc),
|
||||
updated_at=datetime.now(timezone.utc),
|
||||
)
|
||||
entity2 = Entity(
|
||||
project_id=entity_repository.project_id,
|
||||
title="Entity 2",
|
||||
entity_type="test",
|
||||
permalink="test/entity2",
|
||||
file_path="test/entity2.md",
|
||||
content_type="text/markdown",
|
||||
created_at=datetime.now(timezone.utc),
|
||||
updated_at=datetime.now(timezone.utc),
|
||||
)
|
||||
session.add_all([entity1, entity2])
|
||||
|
||||
permalinks = await entity_repository.get_all_permalinks()
|
||||
|
||||
assert len(permalinks) == 2
|
||||
assert set(permalinks) == {"test/entity1", "test/entity2"}
|
||||
|
||||
# Results should be strings, not entities
|
||||
for permalink in permalinks:
|
||||
assert isinstance(permalink, str)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_get_permalink_to_file_path_map(entity_repository: EntityRepository, session_maker):
|
||||
"""Test getting permalink -> file_path mapping for bulk operations."""
|
||||
async with db.scoped_session(session_maker) as session:
|
||||
entity1 = Entity(
|
||||
project_id=entity_repository.project_id,
|
||||
title="Entity 1",
|
||||
entity_type="test",
|
||||
permalink="test/entity1",
|
||||
file_path="test/entity1.md",
|
||||
content_type="text/markdown",
|
||||
created_at=datetime.now(timezone.utc),
|
||||
updated_at=datetime.now(timezone.utc),
|
||||
)
|
||||
entity2 = Entity(
|
||||
project_id=entity_repository.project_id,
|
||||
title="Entity 2",
|
||||
entity_type="test",
|
||||
permalink="test/entity2",
|
||||
file_path="test/entity2.md",
|
||||
content_type="text/markdown",
|
||||
created_at=datetime.now(timezone.utc),
|
||||
updated_at=datetime.now(timezone.utc),
|
||||
)
|
||||
session.add_all([entity1, entity2])
|
||||
|
||||
mapping = await entity_repository.get_permalink_to_file_path_map()
|
||||
|
||||
assert len(mapping) == 2
|
||||
assert mapping["test/entity1"] == "test/entity1.md"
|
||||
assert mapping["test/entity2"] == "test/entity2.md"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_get_file_path_to_permalink_map(entity_repository: EntityRepository, session_maker):
|
||||
"""Test getting file_path -> permalink mapping for bulk operations."""
|
||||
async with db.scoped_session(session_maker) as session:
|
||||
entity1 = Entity(
|
||||
project_id=entity_repository.project_id,
|
||||
title="Entity 1",
|
||||
entity_type="test",
|
||||
permalink="test/entity1",
|
||||
file_path="test/entity1.md",
|
||||
content_type="text/markdown",
|
||||
created_at=datetime.now(timezone.utc),
|
||||
updated_at=datetime.now(timezone.utc),
|
||||
)
|
||||
entity2 = Entity(
|
||||
project_id=entity_repository.project_id,
|
||||
title="Entity 2",
|
||||
entity_type="test",
|
||||
permalink="test/entity2",
|
||||
file_path="test/entity2.md",
|
||||
content_type="text/markdown",
|
||||
created_at=datetime.now(timezone.utc),
|
||||
updated_at=datetime.now(timezone.utc),
|
||||
)
|
||||
session.add_all([entity1, entity2])
|
||||
|
||||
mapping = await entity_repository.get_file_path_to_permalink_map()
|
||||
|
||||
assert len(mapping) == 2
|
||||
assert mapping["test/entity1.md"] == "test/entity1"
|
||||
assert mapping["test/entity2.md"] == "test/entity2"
|
||||
|
||||
@@ -28,6 +28,7 @@ async def test_upsert_entity_with_observations_conflict(entity_repository: Entit
|
||||
|
||||
# Add observations to the entity
|
||||
obs1 = Observation(
|
||||
project_id=entity_repository.project_id,
|
||||
content="This is a test observation",
|
||||
category="testing",
|
||||
tags=["test"],
|
||||
@@ -56,11 +57,13 @@ async def test_upsert_entity_with_observations_conflict(entity_repository: Entit
|
||||
|
||||
# Add different observations
|
||||
obs2 = Observation(
|
||||
project_id=entity_repository.project_id,
|
||||
content="This is an updated observation",
|
||||
category="updated",
|
||||
tags=["updated"],
|
||||
)
|
||||
obs3 = Observation(
|
||||
project_id=entity_repository.project_id,
|
||||
content="This is a second observation",
|
||||
category="second",
|
||||
tags=["second"],
|
||||
|
||||
@@ -22,6 +22,7 @@ async def repo(observation_repository):
|
||||
async def sample_observation(repo, sample_entity: Entity):
|
||||
"""Create a sample observation for testing"""
|
||||
observation_data = {
|
||||
"project_id": sample_entity.project_id,
|
||||
"entity_id": sample_entity.id,
|
||||
"content": "Test observation",
|
||||
"context": "test-context",
|
||||
@@ -35,6 +36,7 @@ async def test_create_observation(
|
||||
):
|
||||
"""Test creating a new observation"""
|
||||
observation_data = {
|
||||
"project_id": sample_entity.project_id,
|
||||
"entity_id": sample_entity.id,
|
||||
"content": "Test content",
|
||||
"context": "test-context",
|
||||
@@ -52,6 +54,7 @@ async def test_create_observation_entity_does_not_exist(
|
||||
):
|
||||
"""Test creating a new observation"""
|
||||
observation_data = {
|
||||
"project_id": sample_entity.project_id,
|
||||
"entity_id": 99999, # Non-existent entity ID (integer for Postgres compatibility)
|
||||
"content": "Test content",
|
||||
"context": "test-context",
|
||||
@@ -104,10 +107,12 @@ async def test_delete_observations(session_maker: async_sessionmaker, repo, test
|
||||
|
||||
# Create test observations
|
||||
obs1 = Observation(
|
||||
project_id=test_project.id,
|
||||
entity_id=entity.id,
|
||||
content="Test observation 1",
|
||||
)
|
||||
obs2 = Observation(
|
||||
project_id=test_project.id,
|
||||
entity_id=entity.id,
|
||||
content="Test observation 2",
|
||||
)
|
||||
@@ -144,6 +149,7 @@ async def test_delete_observation_by_id(
|
||||
|
||||
# Create test observation
|
||||
obs = Observation(
|
||||
project_id=test_project.id,
|
||||
entity_id=entity.id,
|
||||
content="Test observation",
|
||||
)
|
||||
@@ -180,10 +186,12 @@ async def test_delete_observation_by_content(
|
||||
|
||||
# Create test observations
|
||||
obs1 = Observation(
|
||||
project_id=test_project.id,
|
||||
entity_id=entity.id,
|
||||
content="Delete this observation",
|
||||
)
|
||||
obs2 = Observation(
|
||||
project_id=test_project.id,
|
||||
entity_id=entity.id,
|
||||
content="Keep this observation",
|
||||
)
|
||||
@@ -220,16 +228,19 @@ async def test_find_by_category(session_maker: async_sessionmaker, repo, test_pr
|
||||
# Create test observations with different categories
|
||||
observations = [
|
||||
Observation(
|
||||
project_id=test_project.id,
|
||||
entity_id=entity.id,
|
||||
content="Tech observation",
|
||||
category="tech",
|
||||
),
|
||||
Observation(
|
||||
project_id=test_project.id,
|
||||
entity_id=entity.id,
|
||||
content="Design observation",
|
||||
category="design",
|
||||
),
|
||||
Observation(
|
||||
project_id=test_project.id,
|
||||
entity_id=entity.id,
|
||||
content="Another tech observation",
|
||||
category="tech",
|
||||
@@ -278,21 +289,25 @@ async def test_observation_categories(
|
||||
# Create observations with various categories
|
||||
observations = [
|
||||
Observation(
|
||||
project_id=test_project.id,
|
||||
entity_id=entity.id,
|
||||
content="First tech note",
|
||||
category="tech",
|
||||
),
|
||||
Observation(
|
||||
project_id=test_project.id,
|
||||
entity_id=entity.id,
|
||||
content="Second tech note",
|
||||
category="tech", # Duplicate category
|
||||
),
|
||||
Observation(
|
||||
project_id=test_project.id,
|
||||
entity_id=entity.id,
|
||||
content="Design note",
|
||||
category="design",
|
||||
),
|
||||
Observation(
|
||||
project_id=test_project.id,
|
||||
entity_id=entity.id,
|
||||
content="Feature note",
|
||||
category="feature",
|
||||
@@ -341,6 +356,7 @@ async def test_find_by_category_case_sensitivity(
|
||||
|
||||
# Create a test observation
|
||||
obs = Observation(
|
||||
project_id=test_project.id,
|
||||
entity_id=entity.id,
|
||||
content="Tech note",
|
||||
category="tech", # lowercase in database
|
||||
@@ -356,3 +372,97 @@ async def test_find_by_category_case_sensitivity(
|
||||
|
||||
upper_case = await repo.find_by_category("TECH")
|
||||
assert len(upper_case) == 0 # Currently case-sensitive
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_observation_permalink_truncates_long_content(
|
||||
session_maker: async_sessionmaker, repo, test_project: Project
|
||||
):
|
||||
"""Test that observation permalinks truncate long content.
|
||||
|
||||
This test validates the fix for issue #446 where:
|
||||
- Long observation content (like transcript dialogue) created permalinks
|
||||
exceeding PostgreSQL's btree index limit of 2704 bytes.
|
||||
- Content is now truncated to 200 chars in the permalink property.
|
||||
"""
|
||||
async with db.scoped_session(session_maker) as session:
|
||||
entity = Entity(
|
||||
project_id=test_project.id,
|
||||
title="test_entity",
|
||||
entity_type="test",
|
||||
permalink="test/test-entity",
|
||||
file_path="test/test_entity.md",
|
||||
content_type="text/markdown",
|
||||
created_at=datetime.now(timezone.utc),
|
||||
updated_at=datetime.now(timezone.utc),
|
||||
)
|
||||
session.add(entity)
|
||||
await session.flush()
|
||||
|
||||
# Create observation with very long content (5000+ chars to simulate transcript)
|
||||
long_content = "A" * 5000 # Well over the 200 char limit
|
||||
obs = Observation(
|
||||
project_id=test_project.id,
|
||||
entity_id=entity.id,
|
||||
content=long_content,
|
||||
category="transcript",
|
||||
)
|
||||
session.add(obs)
|
||||
await session.flush()
|
||||
|
||||
# Access the permalink property
|
||||
permalink = obs.permalink
|
||||
|
||||
# The full content would create a permalink like:
|
||||
# test/test-entity/observations/transcript/AAAA...5000 chars
|
||||
# With truncation, it should be much shorter
|
||||
|
||||
# Content portion should be truncated to 200 chars
|
||||
# Permalink format: entity_permalink/observations/category/content
|
||||
assert len(permalink) < 300 # Should be well under 300 chars total
|
||||
assert len(long_content[:200]) == 200 # Verify truncation length
|
||||
|
||||
# Verify the permalink contains expected parts
|
||||
assert "test/test-entity" in permalink or "test-entity" in permalink
|
||||
assert "observations" in permalink
|
||||
assert "transcript" in permalink
|
||||
|
||||
# Full 5000-char content should NOT be in permalink
|
||||
assert long_content not in permalink
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_observation_permalink_short_content_unchanged(
|
||||
session_maker: async_sessionmaker, repo, test_project: Project
|
||||
):
|
||||
"""Test that short observation content is not unnecessarily truncated."""
|
||||
async with db.scoped_session(session_maker) as session:
|
||||
entity = Entity(
|
||||
project_id=test_project.id,
|
||||
title="test_entity",
|
||||
entity_type="test",
|
||||
permalink="test/test-entity",
|
||||
file_path="test/test_entity.md",
|
||||
content_type="text/markdown",
|
||||
created_at=datetime.now(timezone.utc),
|
||||
updated_at=datetime.now(timezone.utc),
|
||||
)
|
||||
session.add(entity)
|
||||
await session.flush()
|
||||
|
||||
# Create observation with short content
|
||||
short_content = "Short observation content"
|
||||
obs = Observation(
|
||||
project_id=test_project.id,
|
||||
entity_id=entity.id,
|
||||
content=short_content,
|
||||
category="note",
|
||||
)
|
||||
session.add(obs)
|
||||
await session.flush()
|
||||
|
||||
permalink = obs.permalink
|
||||
|
||||
# Short content should be fully included (after permalink normalization)
|
||||
# The generate_permalink function normalizes the content
|
||||
assert "short-observation-content" in permalink.lower()
|
||||
|
||||
@@ -50,16 +50,18 @@ async def target_entity(session_maker, test_project: Project):
|
||||
|
||||
|
||||
@pytest_asyncio.fixture
|
||||
async def test_relations(session_maker, source_entity, target_entity):
|
||||
async def test_relations(session_maker, source_entity, target_entity, test_project: Project):
|
||||
"""Create test relations."""
|
||||
relations = [
|
||||
Relation(
|
||||
project_id=test_project.id,
|
||||
from_id=source_entity.id,
|
||||
to_id=target_entity.id,
|
||||
to_name=target_entity.title,
|
||||
relation_type="connects_to",
|
||||
),
|
||||
Relation(
|
||||
project_id=test_project.id,
|
||||
from_id=source_entity.id,
|
||||
to_id=target_entity.id,
|
||||
to_name=target_entity.title,
|
||||
@@ -349,3 +351,156 @@ async def test_delete_nonexistent_relation(relation_repository):
|
||||
"""Test deleting a relation that doesn't exist."""
|
||||
result = await relation_repository.delete_by_fields(relation_type="nonexistent")
|
||||
assert result is False
|
||||
|
||||
|
||||
# -------------------------------------------------------------------------
|
||||
# Tests for add_all_ignore_duplicates
|
||||
# -------------------------------------------------------------------------
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_add_all_ignore_duplicates_basic(
|
||||
relation_repository: RelationRepository, sample_entity: Entity, related_entity: Entity
|
||||
):
|
||||
"""Test bulk inserting relations with ON CONFLICT DO NOTHING."""
|
||||
relations = [
|
||||
Relation(
|
||||
from_id=sample_entity.id,
|
||||
to_id=related_entity.id,
|
||||
to_name=related_entity.title,
|
||||
relation_type="links_to",
|
||||
),
|
||||
Relation(
|
||||
from_id=sample_entity.id,
|
||||
to_id=related_entity.id,
|
||||
to_name=related_entity.title,
|
||||
relation_type="references",
|
||||
),
|
||||
]
|
||||
|
||||
inserted = await relation_repository.add_all_ignore_duplicates(relations)
|
||||
|
||||
# Both should be inserted
|
||||
assert inserted == 2
|
||||
|
||||
# Verify they exist
|
||||
found = await relation_repository.find_by_entities(sample_entity.id, related_entity.id)
|
||||
assert len(found) == 2
|
||||
relation_types = {r.relation_type for r in found}
|
||||
assert relation_types == {"links_to", "references"}
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_add_all_ignore_duplicates_skips_duplicates(
|
||||
relation_repository: RelationRepository, sample_entity: Entity, related_entity: Entity
|
||||
):
|
||||
"""Test that duplicate relations are silently ignored."""
|
||||
# Same relation appearing multiple times (common when same [[link]] appears twice in doc)
|
||||
relations = [
|
||||
Relation(
|
||||
from_id=sample_entity.id,
|
||||
to_id=None, # Unresolved
|
||||
to_name="Some Target",
|
||||
relation_type="links_to",
|
||||
),
|
||||
Relation(
|
||||
from_id=sample_entity.id,
|
||||
to_id=None,
|
||||
to_name="Some Target", # Duplicate!
|
||||
relation_type="links_to",
|
||||
),
|
||||
Relation(
|
||||
from_id=sample_entity.id,
|
||||
to_id=None,
|
||||
to_name="Some Target", # Triple duplicate!
|
||||
relation_type="links_to",
|
||||
),
|
||||
]
|
||||
|
||||
inserted = await relation_repository.add_all_ignore_duplicates(relations)
|
||||
|
||||
# Only 1 should be inserted (duplicates ignored)
|
||||
assert inserted == 1
|
||||
|
||||
# Verify only one exists
|
||||
all_relations = await relation_repository.find_all()
|
||||
matching = [r for r in all_relations if r.to_name == "Some Target"]
|
||||
assert len(matching) == 1
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_add_all_ignore_duplicates_empty_list(relation_repository: RelationRepository):
|
||||
"""Test with empty list returns 0."""
|
||||
inserted = await relation_repository.add_all_ignore_duplicates([])
|
||||
assert inserted == 0
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_add_all_ignore_duplicates_mixed(
|
||||
relation_repository: RelationRepository, sample_entity: Entity, related_entity: Entity
|
||||
):
|
||||
"""Test with mix of new and duplicate relations."""
|
||||
# First, insert one relation
|
||||
first_relation = Relation(
|
||||
from_id=sample_entity.id,
|
||||
to_id=None,
|
||||
to_name="Existing Target",
|
||||
relation_type="links_to",
|
||||
)
|
||||
await relation_repository.add_all_ignore_duplicates([first_relation])
|
||||
|
||||
# Now try to insert a mix of new and duplicate
|
||||
relations = [
|
||||
Relation(
|
||||
from_id=sample_entity.id,
|
||||
to_id=None,
|
||||
to_name="Existing Target", # Duplicate of first_relation
|
||||
relation_type="links_to",
|
||||
),
|
||||
Relation(
|
||||
from_id=sample_entity.id,
|
||||
to_id=None,
|
||||
to_name="New Target 1", # New
|
||||
relation_type="links_to",
|
||||
),
|
||||
Relation(
|
||||
from_id=sample_entity.id,
|
||||
to_id=None,
|
||||
to_name="New Target 2", # New
|
||||
relation_type="references",
|
||||
),
|
||||
]
|
||||
|
||||
inserted = await relation_repository.add_all_ignore_duplicates(relations)
|
||||
|
||||
# Only 2 new ones should be inserted
|
||||
assert inserted == 2
|
||||
|
||||
# Verify total count
|
||||
all_relations = await relation_repository.find_all()
|
||||
from_sample = [r for r in all_relations if r.from_id == sample_entity.id]
|
||||
assert len(from_sample) == 3 # 1 existing + 2 new
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_add_all_ignore_duplicates_with_context(
|
||||
relation_repository: RelationRepository, sample_entity: Entity, related_entity: Entity
|
||||
):
|
||||
"""Test that context field is properly inserted."""
|
||||
relations = [
|
||||
Relation(
|
||||
from_id=sample_entity.id,
|
||||
to_id=related_entity.id,
|
||||
to_name=related_entity.title,
|
||||
relation_type="links_to",
|
||||
context="some context here",
|
||||
),
|
||||
]
|
||||
|
||||
inserted = await relation_repository.add_all_ignore_duplicates(relations)
|
||||
assert inserted == 1
|
||||
|
||||
# Verify context was saved
|
||||
found = await relation_repository.find_by_entities(sample_entity.id, related_entity.id)
|
||||
assert len(found) == 1
|
||||
assert found[0].context == "some context here"
|
||||
|
||||
@@ -286,6 +286,7 @@ async def test_project_isolation_in_find_related(session_maker, app_config):
|
||||
|
||||
# Create relation in project1 (between entities of project1)
|
||||
relation_p1 = Relation(
|
||||
project_id=project1.id,
|
||||
from_id=entity1_p1.id,
|
||||
to_id=entity2_p1.id,
|
||||
to_name="Entity2_P1",
|
||||
|
||||
@@ -432,14 +432,14 @@ async def test_create_with_content(entity_service: EntityService, file_service:
|
||||
assert entity.observations[0].context == "Reduces merge conflicts"
|
||||
|
||||
assert len(entity.relations) == 4
|
||||
assert entity.relations[0].relation_type == "links to"
|
||||
assert entity.relations[0].relation_type == "links_to"
|
||||
assert entity.relations[0].to_name == "Git"
|
||||
assert entity.relations[1].relation_type == "links to"
|
||||
assert entity.relations[1].relation_type == "links_to"
|
||||
assert entity.relations[1].to_name == "Trunk Based Development"
|
||||
assert entity.relations[2].relation_type == "implements"
|
||||
assert entity.relations[2].to_name == "Branch Strategy"
|
||||
assert entity.relations[2].context == "Our standard workflow"
|
||||
assert entity.relations[3].relation_type == "links to"
|
||||
assert entity.relations[3].relation_type == "links_to"
|
||||
assert entity.relations[3].to_name == "Git Cheat Sheet"
|
||||
|
||||
# Verify file has new content but preserved metadata
|
||||
@@ -557,14 +557,14 @@ async def test_update_with_content(entity_service: EntityService, file_service:
|
||||
assert entity.observations[0].context == "Reduces merge conflicts"
|
||||
|
||||
assert len(entity.relations) == 4
|
||||
assert entity.relations[0].relation_type == "links to"
|
||||
assert entity.relations[0].relation_type == "links_to"
|
||||
assert entity.relations[0].to_name == "Git"
|
||||
assert entity.relations[1].relation_type == "links to"
|
||||
assert entity.relations[1].relation_type == "links_to"
|
||||
assert entity.relations[1].to_name == "Trunk Based Development"
|
||||
assert entity.relations[2].relation_type == "implements"
|
||||
assert entity.relations[2].to_name == "Branch Strategy"
|
||||
assert entity.relations[2].context == "Our standard workflow"
|
||||
assert entity.relations[3].relation_type == "links to"
|
||||
assert entity.relations[3].relation_type == "links_to"
|
||||
assert entity.relations[3].to_name == "Git Cheat Sheet"
|
||||
|
||||
# Verify file has new content but preserved metadata
|
||||
@@ -1772,7 +1772,7 @@ async def test_move_entity_with_complex_observations(
|
||||
# Check relations
|
||||
relation_types = {rel.relation_type for rel in moved_entity.relations}
|
||||
assert "implements" in relation_types
|
||||
assert "links to" in relation_types
|
||||
assert "links_to" in relation_types
|
||||
|
||||
relation_targets = {rel.to_name for rel in moved_entity.relations}
|
||||
assert "Branch Strategy" in relation_targets
|
||||
|
||||
@@ -162,3 +162,77 @@ async def test_write_unicode_content(tmp_path: Path, file_service: FileService):
|
||||
content, _ = await file_service.read_file(test_path)
|
||||
|
||||
assert content == test_content
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_read_file_content(tmp_path: Path, file_service: FileService):
|
||||
"""Test read_file_content returns just the content without checksum."""
|
||||
test_path = tmp_path / "test.md"
|
||||
test_content = "test content\nwith multiple lines"
|
||||
|
||||
# Write file
|
||||
await file_service.write_file(test_path, test_content)
|
||||
|
||||
# Read content only
|
||||
content = await file_service.read_file_content(test_path)
|
||||
assert content == test_content
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_read_file_content_missing_file(tmp_path: Path, file_service: FileService):
|
||||
"""Test read_file_content raises error for missing files."""
|
||||
test_path = tmp_path / "missing.md"
|
||||
|
||||
with pytest.raises(FileOperationError):
|
||||
await file_service.read_file_content(test_path)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_read_file_bytes(tmp_path: Path, file_service: FileService):
|
||||
"""Test read_file_bytes for binary file reading."""
|
||||
test_path = tmp_path / "test.bin"
|
||||
# Create binary content with non-UTF8 bytes
|
||||
binary_content = b"\x00\x01\x02\x03\xff\xfe\xfd"
|
||||
|
||||
# Write binary file directly
|
||||
test_path.write_bytes(binary_content)
|
||||
|
||||
# Read back using read_file_bytes
|
||||
content = await file_service.read_file_bytes(test_path)
|
||||
assert content == binary_content
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_read_file_bytes_image(tmp_path: Path, file_service: FileService):
|
||||
"""Test read_file_bytes with image-like binary content."""
|
||||
test_path = tmp_path / "test.png"
|
||||
# PNG header signature
|
||||
png_header = b"\x89PNG\r\n\x1a\n"
|
||||
fake_image_content = png_header + b"\x00" * 100
|
||||
|
||||
test_path.write_bytes(fake_image_content)
|
||||
|
||||
content = await file_service.read_file_bytes(test_path)
|
||||
assert content == fake_image_content
|
||||
assert content.startswith(png_header)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_read_file_bytes_missing_file(tmp_path: Path, file_service: FileService):
|
||||
"""Test read_file_bytes raises error for missing files."""
|
||||
test_path = tmp_path / "missing.bin"
|
||||
|
||||
with pytest.raises(FileOperationError):
|
||||
await file_service.read_file_bytes(test_path)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_read_file_bytes_text_file(tmp_path: Path, file_service: FileService):
|
||||
"""Test read_file_bytes can read text files as bytes."""
|
||||
test_path = tmp_path / "test.txt"
|
||||
text_content = "Hello, World!"
|
||||
|
||||
test_path.write_text(text_content)
|
||||
|
||||
content = await file_service.read_file_bytes(test_path)
|
||||
assert content == text_content.encode("utf-8")
|
||||
|
||||
@@ -178,3 +178,71 @@ async def test_initialize_file_sync_background_tasks(
|
||||
|
||||
# Watch service should still be started
|
||||
mock_watch_service.run.assert_called_once()
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@patch("basic_memory.services.initialization.db.get_or_create_db")
|
||||
@patch("basic_memory.sync.sync_service.get_sync_service")
|
||||
@patch("basic_memory.sync.WatchService")
|
||||
@patch("basic_memory.services.initialization.asyncio.create_task")
|
||||
@patch.dict("os.environ", {"BASIC_MEMORY_MCP_PROJECT": "project1"})
|
||||
async def test_initialize_file_sync_respects_project_constraint(
|
||||
mock_create_task, mock_watch_service_class, mock_get_sync_service, mock_get_db, app_config
|
||||
):
|
||||
"""Test that file sync only syncs the constrained project when BASIC_MEMORY_MCP_PROJECT is set."""
|
||||
# Setup mocks
|
||||
mock_session_maker = AsyncMock()
|
||||
mock_get_db.return_value = (None, mock_session_maker)
|
||||
|
||||
mock_watch_service = AsyncMock()
|
||||
mock_watch_service.run = AsyncMock()
|
||||
mock_watch_service_class.return_value = mock_watch_service
|
||||
|
||||
mock_repository = AsyncMock()
|
||||
mock_project1 = MagicMock()
|
||||
mock_project1.name = "project1"
|
||||
mock_project1.path = "/path/to/project1"
|
||||
mock_project1.id = 1
|
||||
|
||||
mock_project2 = MagicMock()
|
||||
mock_project2.name = "project2"
|
||||
mock_project2.path = "/path/to/project2"
|
||||
mock_project2.id = 2
|
||||
|
||||
mock_project3 = MagicMock()
|
||||
mock_project3.name = "project3"
|
||||
mock_project3.path = "/path/to/project3"
|
||||
mock_project3.id = 3
|
||||
|
||||
mock_sync_service = AsyncMock()
|
||||
mock_sync_service.sync = AsyncMock()
|
||||
mock_get_sync_service.return_value = mock_sync_service
|
||||
|
||||
# Mock background tasks
|
||||
mock_task = MagicMock()
|
||||
mock_create_task.return_value = mock_task
|
||||
|
||||
# Mock the repository
|
||||
with patch("basic_memory.services.initialization.ProjectRepository") as mock_repo_class:
|
||||
mock_repo_class.return_value = mock_repository
|
||||
# Return all 3 projects from get_active_projects
|
||||
mock_repository.get_active_projects.return_value = [
|
||||
mock_project1,
|
||||
mock_project2,
|
||||
mock_project3,
|
||||
]
|
||||
|
||||
# Run the function
|
||||
result = await initialize_file_sync(app_config)
|
||||
|
||||
# Assertions
|
||||
mock_repository.get_active_projects.assert_called_once()
|
||||
|
||||
# Should only create 1 background task for project1 (the constrained project)
|
||||
assert mock_create_task.call_count == 1
|
||||
|
||||
# Verify the function returns None
|
||||
assert result is None
|
||||
|
||||
# Watch service should still be started
|
||||
mock_watch_service.run.assert_called_once()
|
||||
|
||||
@@ -890,11 +890,6 @@ async def test_add_project_without_project_root_allows_arbitrary_paths(
|
||||
if "BASIC_MEMORY_PROJECT_ROOT" in os.environ:
|
||||
monkeypatch.delenv("BASIC_MEMORY_PROJECT_ROOT")
|
||||
|
||||
# Force reload config without project_root
|
||||
from basic_memory.services import project_service as ps_module
|
||||
|
||||
monkeypatch.setattr(ps_module, "config", config_manager.load_config())
|
||||
|
||||
# Create a test directory
|
||||
test_dir = Path(temp_dir) / "arbitrary-location"
|
||||
test_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
@@ -755,3 +755,182 @@ async def test_search_title_via_repository_direct(search_service, session_maker,
|
||||
# Should find the entity without throwing FTS5 syntax errors
|
||||
assert len(results) >= 1
|
||||
assert any(result.title == "Note (with parentheses)" for result in results)
|
||||
|
||||
|
||||
# Tests for duplicate observation permalink deduplication
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_index_entity_with_duplicate_observations(
|
||||
search_service, session_maker, test_project
|
||||
):
|
||||
"""Test that indexing an entity with duplicate observations doesn't cause unique constraint violations.
|
||||
|
||||
Two observations with the same category and content generate identical permalinks,
|
||||
which would violate the unique constraint on the search_index table.
|
||||
"""
|
||||
from basic_memory.repository import EntityRepository, ObservationRepository
|
||||
from unittest.mock import AsyncMock
|
||||
from datetime import datetime
|
||||
|
||||
entity_repo = EntityRepository(session_maker, project_id=test_project.id)
|
||||
obs_repo = ObservationRepository(session_maker, project_id=test_project.id)
|
||||
|
||||
# Create entity
|
||||
entity_data = {
|
||||
"title": "Entity With Duplicate Observations",
|
||||
"entity_type": "note",
|
||||
"entity_metadata": {},
|
||||
"content_type": "text/markdown",
|
||||
"file_path": "test/duplicate-obs.md",
|
||||
"permalink": "test/duplicate-obs",
|
||||
"project_id": test_project.id,
|
||||
"created_at": datetime.now(),
|
||||
"updated_at": datetime.now(),
|
||||
}
|
||||
|
||||
entity = await entity_repo.create(entity_data)
|
||||
|
||||
# Create duplicate observations - same category and content
|
||||
duplicate_content = "This is a duplicated observation"
|
||||
await obs_repo.create(
|
||||
{"entity_id": entity.id, "category": "note", "content": duplicate_content}
|
||||
)
|
||||
await obs_repo.create(
|
||||
{"entity_id": entity.id, "category": "note", "content": duplicate_content}
|
||||
)
|
||||
|
||||
# Reload entity with observations (get_by_permalink eagerly loads observations)
|
||||
entity = await entity_repo.get_by_permalink("test/duplicate-obs")
|
||||
|
||||
# Verify we have duplicate observations
|
||||
assert len(entity.observations) == 2
|
||||
assert entity.observations[0].permalink == entity.observations[1].permalink
|
||||
|
||||
# Mock file service to avoid file I/O
|
||||
search_service.file_service.read_entity_content = AsyncMock(return_value="")
|
||||
|
||||
# This should not raise a unique constraint violation
|
||||
await search_service.index_entity(entity)
|
||||
|
||||
# Verify entity is searchable
|
||||
results = await search_service.search(SearchQuery(text="Duplicate Observations"))
|
||||
assert len(results) >= 1
|
||||
assert any(r.title == "Entity With Duplicate Observations" for r in results)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_index_entity_dedupes_observations_by_permalink(
|
||||
search_service, session_maker, test_project
|
||||
):
|
||||
"""Test that only unique observation permalinks are indexed.
|
||||
|
||||
When an entity has observations with identical permalinks, only the first one
|
||||
should be indexed to avoid unique constraint violations.
|
||||
"""
|
||||
from basic_memory.repository import EntityRepository, ObservationRepository
|
||||
from unittest.mock import AsyncMock
|
||||
from datetime import datetime
|
||||
|
||||
entity_repo = EntityRepository(session_maker, project_id=test_project.id)
|
||||
obs_repo = ObservationRepository(session_maker, project_id=test_project.id)
|
||||
|
||||
# Create entity
|
||||
entity_data = {
|
||||
"title": "Dedupe Test Entity",
|
||||
"entity_type": "note",
|
||||
"entity_metadata": {},
|
||||
"content_type": "text/markdown",
|
||||
"file_path": "test/dedupe-test.md",
|
||||
"permalink": "test/dedupe-test",
|
||||
"project_id": test_project.id,
|
||||
"created_at": datetime.now(),
|
||||
"updated_at": datetime.now(),
|
||||
}
|
||||
|
||||
entity = await entity_repo.create(entity_data)
|
||||
|
||||
# Create three observations: two duplicates and one unique
|
||||
duplicate_content = "Duplicate observation content"
|
||||
unique_content = "Unique observation content"
|
||||
|
||||
await obs_repo.create(
|
||||
{"entity_id": entity.id, "category": "note", "content": duplicate_content}
|
||||
)
|
||||
await obs_repo.create(
|
||||
{"entity_id": entity.id, "category": "note", "content": duplicate_content}
|
||||
)
|
||||
await obs_repo.create({"entity_id": entity.id, "category": "note", "content": unique_content})
|
||||
|
||||
# Reload entity with observations (get_by_permalink eagerly loads observations)
|
||||
entity = await entity_repo.get_by_permalink("test/dedupe-test")
|
||||
assert len(entity.observations) == 3
|
||||
|
||||
# Mock file service to avoid file I/O
|
||||
search_service.file_service.read_entity_content = AsyncMock(return_value="")
|
||||
|
||||
# Index the entity
|
||||
await search_service.index_entity(entity)
|
||||
|
||||
# Search for the unique observation - should find it
|
||||
results = await search_service.search(SearchQuery(text="Unique observation"))
|
||||
assert len(results) >= 1
|
||||
|
||||
# Search for duplicate observation - should find it (only one indexed)
|
||||
results = await search_service.search(SearchQuery(text="Duplicate observation"))
|
||||
assert len(results) >= 1
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_index_entity_multiple_categories_same_content(
|
||||
search_service, session_maker, test_project
|
||||
):
|
||||
"""Test that observations with same content but different categories are not deduped.
|
||||
|
||||
The permalink includes the category, so observations with different categories
|
||||
but same content should have different permalinks and both be indexed.
|
||||
"""
|
||||
from basic_memory.repository import EntityRepository, ObservationRepository
|
||||
from unittest.mock import AsyncMock
|
||||
from datetime import datetime
|
||||
|
||||
entity_repo = EntityRepository(session_maker, project_id=test_project.id)
|
||||
obs_repo = ObservationRepository(session_maker, project_id=test_project.id)
|
||||
|
||||
# Create entity
|
||||
entity_data = {
|
||||
"title": "Multi Category Entity",
|
||||
"entity_type": "note",
|
||||
"entity_metadata": {},
|
||||
"content_type": "text/markdown",
|
||||
"file_path": "test/multi-category.md",
|
||||
"permalink": "test/multi-category",
|
||||
"project_id": test_project.id,
|
||||
"created_at": datetime.now(),
|
||||
"updated_at": datetime.now(),
|
||||
}
|
||||
|
||||
entity = await entity_repo.create(entity_data)
|
||||
|
||||
# Create observations with same content but different categories
|
||||
shared_content = "Shared content across categories"
|
||||
await obs_repo.create({"entity_id": entity.id, "category": "tech", "content": shared_content})
|
||||
await obs_repo.create({"entity_id": entity.id, "category": "design", "content": shared_content})
|
||||
|
||||
# Reload entity with observations (get_by_permalink eagerly loads observations)
|
||||
entity = await entity_repo.get_by_permalink("test/multi-category")
|
||||
assert len(entity.observations) == 2
|
||||
|
||||
# Verify permalinks are different due to different categories
|
||||
permalinks = {obs.permalink for obs in entity.observations}
|
||||
assert len(permalinks) == 2 # Should be 2 unique permalinks
|
||||
|
||||
# Mock file service to avoid file I/O
|
||||
search_service.file_service.read_entity_content = AsyncMock(return_value="")
|
||||
|
||||
# Index the entity - both should be indexed since permalinks differ
|
||||
await search_service.index_entity(entity)
|
||||
|
||||
# Search for the shared content - should find both observations
|
||||
results = await search_service.search(SearchQuery(text="Shared content"))
|
||||
assert len(results) >= 2
|
||||
|
||||
@@ -106,6 +106,89 @@ Target content
|
||||
assert source.relations[0].to_name == target.title
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_resolve_relations_deletes_duplicate_unresolved_relation(
|
||||
sync_service: SyncService,
|
||||
project_config: ProjectConfig,
|
||||
entity_service: EntityService,
|
||||
):
|
||||
"""Test that resolve_relations deletes duplicate unresolved relations on IntegrityError.
|
||||
|
||||
When resolving a forward reference would create a duplicate (from_id, to_id, relation_type),
|
||||
the unresolved relation should be deleted since a resolved version already exists.
|
||||
"""
|
||||
from unittest.mock import patch
|
||||
from sqlalchemy.exc import IntegrityError
|
||||
from basic_memory.models import Relation
|
||||
|
||||
project_dir = project_config.home
|
||||
|
||||
# Create source entity
|
||||
source_content = """
|
||||
---
|
||||
type: knowledge
|
||||
---
|
||||
# Source Entity
|
||||
Content
|
||||
"""
|
||||
await create_test_file(project_dir / "source.md", source_content)
|
||||
|
||||
# Create target entity
|
||||
target_content = """
|
||||
---
|
||||
type: knowledge
|
||||
---
|
||||
# Target Entity
|
||||
Content
|
||||
"""
|
||||
await create_test_file(project_dir / "target.md", target_content)
|
||||
|
||||
# Sync to create both entities
|
||||
await sync_service.sync(project_config.home)
|
||||
|
||||
source = await entity_service.get_by_permalink("source")
|
||||
await entity_service.get_by_permalink("target")
|
||||
|
||||
# Create an unresolved relation that will resolve to target
|
||||
unresolved_relation = Relation(
|
||||
from_id=source.id,
|
||||
to_id=None, # Unresolved
|
||||
to_name="target", # Will resolve to target entity
|
||||
relation_type="relates_to",
|
||||
)
|
||||
await sync_service.relation_repository.add(unresolved_relation)
|
||||
unresolved_id = unresolved_relation.id
|
||||
|
||||
# Verify we have the unresolved relation
|
||||
source = await entity_service.get_by_permalink("source")
|
||||
assert len(source.outgoing_relations) == 1
|
||||
assert source.outgoing_relations[0].to_id is None
|
||||
|
||||
# Mock the repository update to raise IntegrityError (simulating existing duplicate)
|
||||
|
||||
async def mock_update_raises_integrity_error(entity_id, data):
|
||||
# Simulate: a resolved relation with same (from_id, to_id, relation_type) already exists
|
||||
raise IntegrityError(
|
||||
"UNIQUE constraint failed: relation.from_id, relation.to_id, relation.relation_type",
|
||||
None,
|
||||
None, # pyright: ignore [reportArgumentType]
|
||||
)
|
||||
|
||||
with patch.object(
|
||||
sync_service.relation_repository, "update", side_effect=mock_update_raises_integrity_error
|
||||
):
|
||||
# Call resolve_relations - should hit IntegrityError and delete the duplicate
|
||||
await sync_service.resolve_relations()
|
||||
|
||||
# Verify the unresolved relation was deleted
|
||||
deleted = await sync_service.relation_repository.find_by_id(unresolved_id)
|
||||
assert deleted is None
|
||||
|
||||
# Verify no unresolved relations remain
|
||||
unresolved = await sync_service.relation_repository.find_unresolved_relations()
|
||||
assert len(unresolved) == 0
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_sync(
|
||||
sync_service: SyncService, project_config: ProjectConfig, entity_service: EntityService
|
||||
@@ -629,11 +712,8 @@ async def test_sync_preserves_timestamps(
|
||||
sync_service: SyncService,
|
||||
project_config: ProjectConfig,
|
||||
entity_service: EntityService,
|
||||
db_backend,
|
||||
):
|
||||
"""Test that sync preserves file timestamps and frontmatter dates."""
|
||||
if db_backend == "postgres":
|
||||
pytest.skip("Postgres timestamp handling differs from SQLite")
|
||||
project_dir = project_config.home
|
||||
|
||||
# Create a file with explicit frontmatter dates
|
||||
@@ -685,7 +765,6 @@ async def test_sync_updates_timestamps_on_file_modification(
|
||||
sync_service: SyncService,
|
||||
project_config: ProjectConfig,
|
||||
entity_service: EntityService,
|
||||
db_backend,
|
||||
):
|
||||
"""Test that sync updates entity timestamps when files are modified.
|
||||
|
||||
@@ -694,9 +773,6 @@ async def test_sync_updates_timestamps_on_file_modification(
|
||||
not the database operation time. This is critical for accurate temporal ordering in
|
||||
search and recent_activity queries.
|
||||
"""
|
||||
if db_backend == "postgres":
|
||||
pytest.skip("Postgres timestamp handling differs from SQLite")
|
||||
|
||||
project_dir = project_config.home
|
||||
|
||||
# Create initial file
|
||||
|
||||
@@ -8,7 +8,7 @@ import pytest
|
||||
from watchfiles import Change
|
||||
|
||||
from basic_memory.models.project import Project
|
||||
from basic_memory.sync.watch_service import WatchService, WatchServiceState
|
||||
from basic_memory.sync.watch_service import WatchServiceState
|
||||
|
||||
|
||||
async def create_test_file(path: Path, content: str = "test content") -> None:
|
||||
@@ -17,10 +17,7 @@ async def create_test_file(path: Path, content: str = "test content") -> None:
|
||||
path.write_text(content)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def watch_service(sync_service, file_service, project_config):
|
||||
"""Create watch service instance."""
|
||||
return WatchService(sync_service, file_service, project_config)
|
||||
# Note: watch_service fixture is defined in conftest.py with sync_service_factory
|
||||
|
||||
|
||||
def test_watch_service_init(watch_service, project_config):
|
||||
|
||||
@@ -85,8 +85,8 @@ async def test_run_handles_no_projects():
|
||||
with patch.object(watch_service, "write_status", return_value=None):
|
||||
await watch_service.run()
|
||||
|
||||
# Should have slept for 30 seconds when no projects found
|
||||
mock_sleep.assert_called_with(30)
|
||||
# Should have slept for the configured reload interval when no projects found
|
||||
mock_sleep.assert_called_with(config.watch_project_reload_interval)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
|
||||
@@ -0,0 +1,96 @@
|
||||
"""Tests for timezone utilities."""
|
||||
|
||||
from datetime import datetime, timezone
|
||||
|
||||
|
||||
from basic_memory.utils import ensure_timezone_aware
|
||||
|
||||
|
||||
class TestEnsureTimezoneAware:
|
||||
"""Tests for ensure_timezone_aware function."""
|
||||
|
||||
def test_already_timezone_aware_returns_unchanged(self):
|
||||
"""Timezone-aware datetime should be returned unchanged."""
|
||||
dt = datetime(2024, 1, 15, 12, 30, 0, tzinfo=timezone.utc)
|
||||
result = ensure_timezone_aware(dt)
|
||||
assert result == dt
|
||||
assert result.tzinfo == timezone.utc
|
||||
|
||||
def test_naive_datetime_cloud_mode_true_interprets_as_utc(self):
|
||||
"""In cloud mode, naive datetimes should be interpreted as UTC."""
|
||||
naive_dt = datetime(2024, 1, 15, 12, 30, 0)
|
||||
result = ensure_timezone_aware(naive_dt, cloud_mode=True)
|
||||
|
||||
# Should have UTC timezone
|
||||
assert result.tzinfo == timezone.utc
|
||||
# Time values should be unchanged (just tagged as UTC)
|
||||
assert result.year == 2024
|
||||
assert result.month == 1
|
||||
assert result.day == 15
|
||||
assert result.hour == 12
|
||||
assert result.minute == 30
|
||||
|
||||
def test_naive_datetime_cloud_mode_false_interprets_as_local(self):
|
||||
"""In local mode, naive datetimes should be interpreted as local time."""
|
||||
naive_dt = datetime(2024, 1, 15, 12, 30, 0)
|
||||
result = ensure_timezone_aware(naive_dt, cloud_mode=False)
|
||||
|
||||
# Should have some timezone info (local)
|
||||
assert result.tzinfo is not None
|
||||
# The datetime should be converted to local timezone
|
||||
# We can't assert exact timezone as it depends on system
|
||||
|
||||
def test_cloud_mode_true_does_not_shift_time(self):
|
||||
"""Cloud mode should use replace() not astimezone() - time values unchanged."""
|
||||
naive_dt = datetime(2024, 6, 15, 18, 0, 0) # Summer time
|
||||
result = ensure_timezone_aware(naive_dt, cloud_mode=True)
|
||||
|
||||
# Hour should remain 18, not be shifted by timezone offset
|
||||
assert result.hour == 18
|
||||
assert result.tzinfo == timezone.utc
|
||||
|
||||
def test_explicit_cloud_mode_skips_config_loading(self):
|
||||
"""When cloud_mode is explicitly passed, config should not be loaded."""
|
||||
# This test verifies we can call ensure_timezone_aware without
|
||||
# triggering ConfigManager import when cloud_mode is explicit
|
||||
naive_dt = datetime(2024, 1, 15, 12, 30, 0)
|
||||
|
||||
# Should work without any config setup
|
||||
result_cloud = ensure_timezone_aware(naive_dt, cloud_mode=True)
|
||||
assert result_cloud.tzinfo == timezone.utc
|
||||
|
||||
result_local = ensure_timezone_aware(naive_dt, cloud_mode=False)
|
||||
assert result_local.tzinfo is not None
|
||||
|
||||
def test_none_cloud_mode_falls_back_to_config(self):
|
||||
"""When cloud_mode is None, should load from config."""
|
||||
from unittest.mock import patch, MagicMock
|
||||
|
||||
naive_dt = datetime(2024, 1, 15, 12, 30, 0)
|
||||
|
||||
# Mock ConfigManager to return cloud_mode_enabled=True
|
||||
mock_config = MagicMock()
|
||||
mock_config.cloud_mode_enabled = True
|
||||
|
||||
with patch("basic_memory.config.ConfigManager") as mock_manager:
|
||||
mock_manager.return_value.config = mock_config
|
||||
result = ensure_timezone_aware(naive_dt, cloud_mode=None)
|
||||
|
||||
# Should have used cloud mode (UTC)
|
||||
assert result.tzinfo == timezone.utc
|
||||
|
||||
def test_asyncpg_naive_utc_scenario(self):
|
||||
"""Simulate asyncpg returning naive datetime that's actually UTC.
|
||||
|
||||
asyncpg binary protocol returns timestamps in UTC but as naive datetimes.
|
||||
In cloud mode, we interpret these as UTC rather than local time.
|
||||
"""
|
||||
# Simulate what asyncpg returns: a naive datetime that's actually UTC
|
||||
asyncpg_result = datetime(2024, 1, 15, 18, 30, 0) # 6:30 PM UTC
|
||||
|
||||
# In cloud mode, interpret as UTC
|
||||
cloud_result = ensure_timezone_aware(asyncpg_result, cloud_mode=True)
|
||||
assert cloud_result == datetime(2024, 1, 15, 18, 30, 0, tzinfo=timezone.utc)
|
||||
|
||||
# The hour should remain 18, not shifted
|
||||
assert cloud_result.hour == 18
|
||||
@@ -126,18 +126,19 @@ dependencies = [
|
||||
{ name = "fastapi", extra = ["standard"] },
|
||||
{ name = "fastmcp" },
|
||||
{ name = "greenlet" },
|
||||
{ name = "logfire" },
|
||||
{ name = "loguru" },
|
||||
{ name = "markdown-it-py" },
|
||||
{ name = "mcp" },
|
||||
{ name = "nest-asyncio" },
|
||||
{ name = "pillow" },
|
||||
{ name = "psycopg" },
|
||||
{ name = "pybars3" },
|
||||
{ name = "pydantic", extra = ["email", "timezone"] },
|
||||
{ name = "pydantic-settings" },
|
||||
{ name = "pyjwt" },
|
||||
{ name = "pyright" },
|
||||
{ name = "pytest-aio" },
|
||||
{ name = "pytest-asyncio" },
|
||||
{ name = "python-dotenv" },
|
||||
{ name = "python-frontmatter" },
|
||||
{ name = "pyyaml" },
|
||||
@@ -153,12 +154,14 @@ dev = [
|
||||
{ name = "freezegun" },
|
||||
{ name = "gevent" },
|
||||
{ name = "icecream" },
|
||||
{ name = "psycopg" },
|
||||
{ name = "pytest" },
|
||||
{ name = "pytest-asyncio" },
|
||||
{ name = "pytest-cov" },
|
||||
{ name = "pytest-mock" },
|
||||
{ name = "pytest-xdist" },
|
||||
{ name = "ruff" },
|
||||
{ name = "testcontainers" },
|
||||
]
|
||||
|
||||
[package.metadata]
|
||||
@@ -169,20 +172,21 @@ requires-dist = [
|
||||
{ name = "asyncpg", specifier = ">=0.30.0" },
|
||||
{ name = "dateparser", specifier = ">=1.2.0" },
|
||||
{ name = "fastapi", extras = ["standard"], specifier = ">=0.115.8" },
|
||||
{ name = "fastmcp", specifier = ">=2.10.2" },
|
||||
{ name = "fastmcp", specifier = "==2.12.3" },
|
||||
{ name = "greenlet", specifier = ">=3.1.1" },
|
||||
{ name = "logfire", specifier = ">=0.73.0" },
|
||||
{ name = "loguru", specifier = ">=0.7.3" },
|
||||
{ name = "markdown-it-py", specifier = ">=3.0.0" },
|
||||
{ name = "mcp", specifier = ">=1.2.0" },
|
||||
{ name = "nest-asyncio", specifier = ">=1.6.0" },
|
||||
{ name = "pillow", specifier = ">=11.1.0" },
|
||||
{ name = "psycopg", specifier = "==3.3.1" },
|
||||
{ name = "pybars3", specifier = ">=0.9.7" },
|
||||
{ name = "pydantic", extras = ["email", "timezone"], specifier = ">=2.10.3" },
|
||||
{ name = "pydantic-settings", specifier = ">=2.6.1" },
|
||||
{ name = "pyjwt", specifier = ">=2.10.1" },
|
||||
{ name = "pyright", specifier = ">=1.1.390" },
|
||||
{ name = "pytest-aio", specifier = ">=1.9.0" },
|
||||
{ name = "pytest-asyncio", specifier = ">=1.2.0" },
|
||||
{ name = "python-dotenv", specifier = ">=1.1.0" },
|
||||
{ name = "python-frontmatter", specifier = ">=1.1.0" },
|
||||
{ name = "pyyaml", specifier = ">=6.0.1" },
|
||||
@@ -198,12 +202,14 @@ dev = [
|
||||
{ name = "freezegun", specifier = ">=1.5.5" },
|
||||
{ name = "gevent", specifier = ">=24.11.1" },
|
||||
{ name = "icecream", specifier = ">=2.1.3" },
|
||||
{ name = "psycopg", specifier = ">=3.2.0" },
|
||||
{ name = "pytest", specifier = ">=8.3.4" },
|
||||
{ name = "pytest-asyncio", specifier = ">=0.24.0" },
|
||||
{ name = "pytest-cov", specifier = ">=4.1.0" },
|
||||
{ name = "pytest-mock", specifier = ">=3.12.0" },
|
||||
{ name = "pytest-xdist", specifier = ">=3.0.0" },
|
||||
{ name = "ruff", specifier = ">=0.1.6" },
|
||||
{ name = "testcontainers", extras = ["postgres"], specifier = ">=4.0.0" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -449,6 +455,20 @@ wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/68/1b/e0a87d256e40e8c888847551b20a017a6b98139178505dc7ffb96f04e954/dnspython-2.7.0-py3-none-any.whl", hash = "sha256:b4c34b7d10b51bcc3a5071e7b8dee77939f1e878477eeecc965e9835f63c6c86", size = 313632, upload-time = "2024-10-05T20:14:57.687Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "docker"
|
||||
version = "7.1.0"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "pywin32", marker = "sys_platform == 'win32'" },
|
||||
{ name = "requests" },
|
||||
{ name = "urllib3" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/91/9b/4a2ea29aeba62471211598dac5d96825bb49348fa07e906ea930394a83ce/docker-7.1.0.tar.gz", hash = "sha256:ad8c70e6e3f8926cb8a92619b832b4ea5299e2831c14284663184e200546fa6c", size = 117834, upload-time = "2024-05-23T11:13:57.216Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/e3/26/57c6fb270950d476074c087527a558ccb6f4436657314bfb6cdf484114c4/docker-7.1.0-py3-none-any.whl", hash = "sha256:c96b93b7f0a746f9e77d325bcfb87422a3d8bd4f03136ae8a85b37f1898d5fc0", size = 147774, upload-time = "2024-05-23T11:13:55.01Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "docstring-parser"
|
||||
version = "0.17.0"
|
||||
@@ -574,7 +594,7 @@ wheels = [
|
||||
|
||||
[[package]]
|
||||
name = "fastmcp"
|
||||
version = "2.11.3"
|
||||
version = "2.12.3"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "authlib" },
|
||||
@@ -589,9 +609,9 @@ dependencies = [
|
||||
{ name = "python-dotenv" },
|
||||
{ name = "rich" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/06/80/13aec687ec21727b0fe6d26c6fe2febb33ae24e24c980929a706db3a8bc2/fastmcp-2.11.3.tar.gz", hash = "sha256:e8e3834a3e0b513712b8e63a6f0d4cbe19093459a1da3f7fbf8ef2810cfd34e3", size = 2692092, upload-time = "2025-08-11T21:38:46.493Z" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/99/5e/035fdfa23646de8811776cd62d93440e334e8a4557b35c63c1bff125c08c/fastmcp-2.12.3.tar.gz", hash = "sha256:541dd569d5b6c083140b04d997ba3dc47f7c10695cee700d0a733ce63b20bb65", size = 5246812, upload-time = "2025-09-12T12:28:07.136Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/61/05/63f63ad5b6789a730d94b8cb3910679c5da1ed5b4e38c957140ac9edcf0e/fastmcp-2.11.3-py3-none-any.whl", hash = "sha256:28f22126c90fd36e5de9cc68b9c271b6d832dcf322256f23d220b68afb3352cc", size = 260231, upload-time = "2025-08-11T21:38:44.746Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/96/79/0fd386e61819e205563d4eb15da76564b80dc2edd3c64b46f2706235daec/fastmcp-2.12.3-py3-none-any.whl", hash = "sha256:aee50872923a9cba731861fc0120e7dbe4642a2685ba251b2b202b82fb6c25a9", size = 314031, upload-time = "2025-09-12T12:28:05.024Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -637,18 +657,6 @@ wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/60/16/b71171e97ec7b4ded8669542f4369d88d5a289e2704efbbde51e858e062a/gevent-25.5.1-cp314-cp314-macosx_11_0_universal2.whl", hash = "sha256:0bacf89a65489d26c7087669af89938d5bfd9f7afb12a07b57855b9fad6ccbd0", size = 2937113, upload-time = "2025-05-12T11:12:03.191Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "googleapis-common-protos"
|
||||
version = "1.70.0"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "protobuf" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/39/24/33db22342cf4a2ea27c9955e6713140fedd51e8b141b5ce5260897020f1a/googleapis_common_protos-1.70.0.tar.gz", hash = "sha256:0e1b44e0ea153e6594f9f394fef15193a68aaaea2d843f83e2742717ca753257", size = 145903, upload-time = "2025-04-14T10:17:02.924Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/86/f1/62a193f0227cf15a920390abe675f386dec35f7ae3ffe6da582d3ade42c7/googleapis_common_protos-1.70.0-py3-none-any.whl", hash = "sha256:b8bfcca8c25a2bb253e0e0b0adaf8c00773e5e6af6fd92397576680b807e0fd8", size = 294530, upload-time = "2025-04-14T10:17:01.271Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "greenlet"
|
||||
version = "3.2.4"
|
||||
@@ -780,18 +788,6 @@ wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/76/c6/c88e154df9c4e1a2a66ccf0005a88dfb2650c1dffb6f5ce603dfbd452ce3/idna-3.10-py3-none-any.whl", hash = "sha256:946d195a0d259cbba61165e88e65941f16e9b36ea6ddb97f00452bae8b1287d3", size = 70442, upload-time = "2024-09-15T18:07:37.964Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "importlib-metadata"
|
||||
version = "8.7.0"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "zipp" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/76/66/650a33bd90f786193e4de4b3ad86ea60b53c89b669a5c7be931fac31cdb0/importlib_metadata-8.7.0.tar.gz", hash = "sha256:d13b81ad223b890aa16c5471f2ac3056cf76c5f10f82d6f9292f0b415f389000", size = 56641, upload-time = "2025-04-27T15:29:01.736Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/20/b0/36bd937216ec521246249be3bf9855081de4c5e06a0c9b4219dbeda50373/importlib_metadata-8.7.0-py3-none-any.whl", hash = "sha256:e5dd1551894c77868a30651cef00984d50e1002d06942a7101d34870c5f02afd", size = 27656, upload-time = "2025-04-27T15:29:00.214Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "iniconfig"
|
||||
version = "2.1.0"
|
||||
@@ -879,24 +875,6 @@ wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/e7/1e/fb441c07b6662ec1fc92b249225ba6e6e5221b05623cb0131d082f782edc/lazy_object_proxy-1.11.0-py3-none-any.whl", hash = "sha256:a56a5093d433341ff7da0e89f9b486031ccd222ec8e52ec84d0ec1cdc819674b", size = 16635, upload-time = "2025-04-16T16:53:47.198Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "logfire"
|
||||
version = "4.13.2"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "executing" },
|
||||
{ name = "opentelemetry-exporter-otlp-proto-http" },
|
||||
{ name = "opentelemetry-instrumentation" },
|
||||
{ name = "opentelemetry-sdk" },
|
||||
{ name = "protobuf" },
|
||||
{ name = "rich" },
|
||||
{ name = "typing-extensions" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/11/51/004bbe0276fcecfea8d4a4c76ad33426c9b34428e0723f22c6bb801dbc22/logfire-4.13.2.tar.gz", hash = "sha256:4e756e140c3b8fd25653d20437ebcb75734975f5382de6ae28be775c75575d95", size = 547796, upload-time = "2025-10-13T16:17:53.392Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/4b/5f/0803848cc5ce524ff830e5f2a2f1400fd6ee72be705d87d0432cec42b1e4/logfire-4.13.2-py3-none-any.whl", hash = "sha256:887e99897a1818864aa5bfc595b02c93264ce23d1860866369eff6b6e2dde1c6", size = 228152, upload-time = "2025-10-13T16:17:50.641Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "loguru"
|
||||
version = "0.7.3"
|
||||
@@ -1091,103 +1069,6 @@ wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/27/dd/b3fd642260cb17532f66cc1e8250f3507d1e580483e209dc1e9d13bd980d/openapi_spec_validator-0.7.2-py3-none-any.whl", hash = "sha256:4bbdc0894ec85f1d1bea1d6d9c8b2c3c8d7ccaa13577ef40da9c006c9fd0eb60", size = 39713, upload-time = "2025-06-07T14:48:54.077Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "opentelemetry-api"
|
||||
version = "1.37.0"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "importlib-metadata" },
|
||||
{ name = "typing-extensions" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/63/04/05040d7ce33a907a2a02257e601992f0cdf11c73b33f13c4492bf6c3d6d5/opentelemetry_api-1.37.0.tar.gz", hash = "sha256:540735b120355bd5112738ea53621f8d5edb35ebcd6fe21ada3ab1c61d1cd9a7", size = 64923, upload-time = "2025-09-11T10:29:01.662Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/91/48/28ed9e55dcf2f453128df738210a980e09f4e468a456fa3c763dbc8be70a/opentelemetry_api-1.37.0-py3-none-any.whl", hash = "sha256:accf2024d3e89faec14302213bc39550ec0f4095d1cf5ca688e1bfb1c8612f47", size = 65732, upload-time = "2025-09-11T10:28:41.826Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "opentelemetry-exporter-otlp-proto-common"
|
||||
version = "1.37.0"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "opentelemetry-proto" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/dc/6c/10018cbcc1e6fff23aac67d7fd977c3d692dbe5f9ef9bb4db5c1268726cc/opentelemetry_exporter_otlp_proto_common-1.37.0.tar.gz", hash = "sha256:c87a1bdd9f41fdc408d9cc9367bb53f8d2602829659f2b90be9f9d79d0bfe62c", size = 20430, upload-time = "2025-09-11T10:29:03.605Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/08/13/b4ef09837409a777f3c0af2a5b4ba9b7af34872bc43609dda0c209e4060d/opentelemetry_exporter_otlp_proto_common-1.37.0-py3-none-any.whl", hash = "sha256:53038428449c559b0c564b8d718df3314da387109c4d36bd1b94c9a641b0292e", size = 18359, upload-time = "2025-09-11T10:28:44.939Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "opentelemetry-exporter-otlp-proto-http"
|
||||
version = "1.37.0"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "googleapis-common-protos" },
|
||||
{ name = "opentelemetry-api" },
|
||||
{ name = "opentelemetry-exporter-otlp-proto-common" },
|
||||
{ name = "opentelemetry-proto" },
|
||||
{ name = "opentelemetry-sdk" },
|
||||
{ name = "requests" },
|
||||
{ name = "typing-extensions" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/5d/e3/6e320aeb24f951449e73867e53c55542bebbaf24faeee7623ef677d66736/opentelemetry_exporter_otlp_proto_http-1.37.0.tar.gz", hash = "sha256:e52e8600f1720d6de298419a802108a8f5afa63c96809ff83becb03f874e44ac", size = 17281, upload-time = "2025-09-11T10:29:04.844Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/e9/e9/70d74a664d83976556cec395d6bfedd9b85ec1498b778367d5f93e373397/opentelemetry_exporter_otlp_proto_http-1.37.0-py3-none-any.whl", hash = "sha256:54c42b39945a6cc9d9a2a33decb876eabb9547e0dcb49df090122773447f1aef", size = 19576, upload-time = "2025-09-11T10:28:46.726Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "opentelemetry-instrumentation"
|
||||
version = "0.58b0"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "opentelemetry-api" },
|
||||
{ name = "opentelemetry-semantic-conventions" },
|
||||
{ name = "packaging" },
|
||||
{ name = "wrapt" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/f6/36/7c307d9be8ce4ee7beb86d7f1d31027f2a6a89228240405a858d6e4d64f9/opentelemetry_instrumentation-0.58b0.tar.gz", hash = "sha256:df640f3ac715a3e05af145c18f527f4422c6ab6c467e40bd24d2ad75a00cb705", size = 31549, upload-time = "2025-09-11T11:42:14.084Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/d4/db/5ff1cd6c5ca1d12ecf1b73be16fbb2a8af2114ee46d4b0e6d4b23f4f4db7/opentelemetry_instrumentation-0.58b0-py3-none-any.whl", hash = "sha256:50f97ac03100676c9f7fc28197f8240c7290ca1baa12da8bfbb9a1de4f34cc45", size = 33019, upload-time = "2025-09-11T11:41:00.624Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "opentelemetry-proto"
|
||||
version = "1.37.0"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "protobuf" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/dd/ea/a75f36b463a36f3c5a10c0b5292c58b31dbdde74f6f905d3d0ab2313987b/opentelemetry_proto-1.37.0.tar.gz", hash = "sha256:30f5c494faf66f77faeaefa35ed4443c5edb3b0aa46dad073ed7210e1a789538", size = 46151, upload-time = "2025-09-11T10:29:11.04Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/c4/25/f89ea66c59bd7687e218361826c969443c4fa15dfe89733f3bf1e2a9e971/opentelemetry_proto-1.37.0-py3-none-any.whl", hash = "sha256:8ed8c066ae8828bbf0c39229979bdf583a126981142378a9cbe9d6fd5701c6e2", size = 72534, upload-time = "2025-09-11T10:28:56.831Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "opentelemetry-sdk"
|
||||
version = "1.37.0"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "opentelemetry-api" },
|
||||
{ name = "opentelemetry-semantic-conventions" },
|
||||
{ name = "typing-extensions" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/f4/62/2e0ca80d7fe94f0b193135375da92c640d15fe81f636658d2acf373086bc/opentelemetry_sdk-1.37.0.tar.gz", hash = "sha256:cc8e089c10953ded765b5ab5669b198bbe0af1b3f89f1007d19acd32dc46dda5", size = 170404, upload-time = "2025-09-11T10:29:11.779Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/9f/62/9f4ad6a54126fb00f7ed4bb5034964c6e4f00fcd5a905e115bd22707e20d/opentelemetry_sdk-1.37.0-py3-none-any.whl", hash = "sha256:8f3c3c22063e52475c5dbced7209495c2c16723d016d39287dfc215d1771257c", size = 131941, upload-time = "2025-09-11T10:28:57.83Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "opentelemetry-semantic-conventions"
|
||||
version = "0.58b0"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "opentelemetry-api" },
|
||||
{ name = "typing-extensions" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/aa/1b/90701d91e6300d9f2fb352153fb1721ed99ed1f6ea14fa992c756016e63a/opentelemetry_semantic_conventions-0.58b0.tar.gz", hash = "sha256:6bd46f51264279c433755767bb44ad00f1c9e2367e1b42af563372c5a6fa0c25", size = 129867, upload-time = "2025-09-11T10:29:12.597Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/07/90/68152b7465f50285d3ce2481b3aec2f82822e3f52e5152eeeaf516bab841/opentelemetry_semantic_conventions-0.58b0-py3-none-any.whl", hash = "sha256:5564905ab1458b96684db1340232729fce3b5375a06e140e8904c78e4f815b28", size = 207954, upload-time = "2025-09-11T10:28:59.218Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "packaging"
|
||||
version = "25.0"
|
||||
@@ -1291,18 +1172,16 @@ wheels = [
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "protobuf"
|
||||
version = "6.33.0"
|
||||
name = "psycopg"
|
||||
version = "3.3.1"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/19/ff/64a6c8f420818bb873713988ca5492cba3a7946be57e027ac63495157d97/protobuf-6.33.0.tar.gz", hash = "sha256:140303d5c8d2037730c548f8c7b93b20bb1dc301be280c378b82b8894589c954", size = 443463, upload-time = "2025-10-15T20:39:52.159Z" }
|
||||
dependencies = [
|
||||
{ name = "typing-extensions", marker = "python_full_version < '3.13'" },
|
||||
{ name = "tzdata", marker = "sys_platform == 'win32'" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/18/ed/3a30e8ef82d4128c76aa9bd6b2a7fe6c16c283811e6655997f5047801b47/psycopg-3.3.1.tar.gz", hash = "sha256:ccfa30b75874eef809c0fbbb176554a2640cc1735a612accc2e2396a92442fc6", size = 165596, upload-time = "2025-12-02T21:09:55.545Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/7e/ee/52b3fa8feb6db4a833dfea4943e175ce645144532e8a90f72571ad85df4e/protobuf-6.33.0-cp310-abi3-win32.whl", hash = "sha256:d6101ded078042a8f17959eccd9236fb7a9ca20d3b0098bbcb91533a5680d035", size = 425593, upload-time = "2025-10-15T20:39:40.29Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/7b/c6/7a465f1825872c55e0341ff4a80198743f73b69ce5d43ab18043699d1d81/protobuf-6.33.0-cp310-abi3-win_amd64.whl", hash = "sha256:9a031d10f703f03768f2743a1c403af050b6ae1f3480e9c140f39c45f81b13ee", size = 436882, upload-time = "2025-10-15T20:39:42.841Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/e1/a9/b6eee662a6951b9c3640e8e452ab3e09f117d99fc10baa32d1581a0d4099/protobuf-6.33.0-cp39-abi3-macosx_10_9_universal2.whl", hash = "sha256:905b07a65f1a4b72412314082c7dbfae91a9e8b68a0cc1577515f8df58ecf455", size = 427521, upload-time = "2025-10-15T20:39:43.803Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/10/35/16d31e0f92c6d2f0e77c2a3ba93185130ea13053dd16200a57434c882f2b/protobuf-6.33.0-cp39-abi3-manylinux2014_aarch64.whl", hash = "sha256:e0697ece353e6239b90ee43a9231318302ad8353c70e6e45499fa52396debf90", size = 324445, upload-time = "2025-10-15T20:39:44.932Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/e6/eb/2a981a13e35cda8b75b5585aaffae2eb904f8f351bdd3870769692acbd8a/protobuf-6.33.0-cp39-abi3-manylinux2014_s390x.whl", hash = "sha256:e0a1715e4f27355afd9570f3ea369735afc853a6c3951a6afe1f80d8569ad298", size = 339159, upload-time = "2025-10-15T20:39:46.186Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/21/51/0b1cbad62074439b867b4e04cc09b93f6699d78fd191bed2bbb44562e077/protobuf-6.33.0-cp39-abi3-manylinux2014_x86_64.whl", hash = "sha256:35be49fd3f4fefa4e6e2aacc35e8b837d6703c37a2168a55ac21e9b1bc7559ef", size = 323172, upload-time = "2025-10-15T20:39:47.465Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/07/d1/0a28c21707807c6aacd5dc9c3704b2aa1effbf37adebd8caeaf68b17a636/protobuf-6.33.0-py3-none-any.whl", hash = "sha256:25c9e1963c6734448ea2d308cfa610e692b801304ba0908d7bfa564ac5132995", size = 170477, upload-time = "2025-10-15T20:39:51.311Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/b6/f3/0b4a4c25a47c2d907afa97674287dab61bc9941c9ac3972a67100e33894d/psycopg-3.3.1-py3-none-any.whl", hash = "sha256:e44d8eae209752efe46318f36dd0fdf5863e928009338d736843bb1084f6435c", size = 212760, upload-time = "2025-12-02T21:02:36.029Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -1475,14 +1354,15 @@ wheels = [
|
||||
|
||||
[[package]]
|
||||
name = "pytest-asyncio"
|
||||
version = "1.1.0"
|
||||
version = "1.3.0"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "pytest" },
|
||||
{ name = "typing-extensions", marker = "python_full_version < '3.13'" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/4e/51/f8794af39eeb870e87a8c8068642fc07bce0c854d6865d7dd0f2a9d338c2/pytest_asyncio-1.1.0.tar.gz", hash = "sha256:796aa822981e01b68c12e4827b8697108f7205020f24b5793b3c41555dab68ea", size = 46652, upload-time = "2025-07-16T04:29:26.393Z" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/90/2c/8af215c0f776415f3590cac4f9086ccefd6fd463befeae41cd4d3f193e5a/pytest_asyncio-1.3.0.tar.gz", hash = "sha256:d7f52f36d231b80ee124cd216ffb19369aa168fc10095013c6b014a34d3ee9e5", size = 50087, upload-time = "2025-11-10T16:07:47.256Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/c7/9d/bf86eddabf8c6c9cb1ea9a869d6873b46f105a5d292d3a6f7071f5b07935/pytest_asyncio-1.1.0-py3-none-any.whl", hash = "sha256:5fe2d69607b0bd75c656d1211f969cadba035030156745ee09e7d71740e58ecf", size = 15157, upload-time = "2025-07-16T04:29:24.929Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/e5/35/f8b19922b6a25bc0880171a2f1a003eaeb93657475193ab516fd87cac9da/pytest_asyncio-1.3.0-py3-none-any.whl", hash = "sha256:611e26147c7f77640e6d0a92a38ed17c3e9848063698d5c93d5aa7aa11cebff5", size = 15075, upload-time = "2025-11-10T16:07:45.537Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -2004,6 +1884,22 @@ wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/f7/1f/b876b1f83aef204198a42dc101613fefccb32258e5428b5f9259677864b4/starlette-0.47.2-py3-none-any.whl", hash = "sha256:c5847e96134e5c5371ee9fac6fdf1a67336d5815e09eb2a01fdb57a351ef915b", size = 72984, upload-time = "2025-07-20T17:31:56.738Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "testcontainers"
|
||||
version = "4.13.3"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "docker" },
|
||||
{ name = "python-dotenv" },
|
||||
{ name = "typing-extensions" },
|
||||
{ name = "urllib3" },
|
||||
{ name = "wrapt" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/fc/b3/c272537f3ea2f312555efeb86398cc382cd07b740d5f3c730918c36e64e1/testcontainers-4.13.3.tar.gz", hash = "sha256:9d82a7052c9a53c58b69e1dc31da8e7a715e8b3ec1c4df5027561b47e2efe646", size = 79064, upload-time = "2025-11-14T05:08:47.584Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/73/27/c2f24b19dafa197c514abe70eda69bc031c5152c6b1f1e5b20099e2ceedd/testcontainers-4.13.3-py3-none-any.whl", hash = "sha256:063278c4805ffa6dd85e56648a9da3036939e6c0ac1001e851c9276b19b05970", size = 124784, upload-time = "2025-11-14T05:08:46.053Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "typer"
|
||||
version = "0.16.0"
|
||||
@@ -2291,15 +2187,6 @@ wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/1f/f6/a933bd70f98e9cf3e08167fc5cd7aaaca49147e48411c0bd5ae701bb2194/wrapt-1.17.3-py3-none-any.whl", hash = "sha256:7171ae35d2c33d326ac19dd8facb1e82e5fd04ef8c6c0e394d7af55a55051c22", size = 23591, upload-time = "2025-08-12T05:53:20.674Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "zipp"
|
||||
version = "3.23.0"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/e3/02/0f2892c661036d50ede074e376733dca2ae7c6eb617489437771209d4180/zipp-3.23.0.tar.gz", hash = "sha256:a07157588a12518c9d4034df3fbbee09c814741a33ff63c05fa29d26a2404166", size = 25547, upload-time = "2025-06-08T17:06:39.4Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/2e/54/647ade08bf0db230bfea292f893923872fd20be6ac6f53b2b936ba839d75/zipp-3.23.0-py3-none-any.whl", hash = "sha256:071652d6115ed432f5ce1d34c336c0adfd6a884660d1e9712a256d3d3bd4b14e", size = 10276, upload-time = "2025-06-08T17:06:38.034Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "zope-event"
|
||||
version = "5.1.1"
|
||||
|
||||
Reference in New Issue
Block a user