Commit Graph

68 Commits

Author SHA1 Message Date
Paul Hernandez 4791e19685 feat: add Logfire phased instrumentation (#692)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24 20:39:42 -05:00
Paul Hernandez c8b00449d2 fix(core): exclude stale entity rows from embedding coverage stats (#675)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-15 18:45:36 -05:00
phernandez d15f6a8427 Add batched vector sync orchestration across repositories 2026-03-03 16:36:06 -06:00
phernandez 54b968b93c fix: reduce excessive log volume by demoting per-request noise to DEBUG (#613)
Demote high-frequency per-request/per-item logs from INFO to DEBUG:
- 🔇 Client routing decisions (async_client.py) — logged every MCP tool call
- 🔇 DB migration checks (db.py) — logged every ASGI client creation
- 🔇 Vector table ensure/ready (sqlite + postgres search repos) — logged every search
- 🔇 Per-entity search index start/complete (search_service.py) — logged every file sync
- 🔇 Incremental scan details + per-file permalink updates (sync_service.py)
- 🔇 MCP search tool params and no-results (search.py)
- 📉 Log retention: "10 days" → 5 files (~50MB cap)

API v2 request/response logs remain at INFO for observability.

Closes #613

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: phernandez <paul@basicmachines.co>
2026-02-26 19:48:24 -06:00
Paul Hernandez c44291830c chore: rename entity_type to note_type (#600)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-22 20:28:57 -06:00
Paul Hernandez edb7991ccf fix: strip NUL bytes from content before PostgreSQL search indexing (#592)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-20 18:11:04 -06:00
phernandez 8c05a9ec80 fix: stabilize semantic search defaults, FTS fallback, and postgres project sync
Signed-off-by: phernandez <paul@basicmachines.co>
2026-02-17 15:36:46 -06:00
Paul Hernandez 55d675e278 feat: min_similarity override, cloud promo improvements (#570)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-16 16:44:02 -06:00
Paul Hernandez 07778790d3 feat: add semantic vector search for SQLite and Postgres (#550)
Signed-off-by: phernandez <paul@basicmachines.co>
Signed-off-by: bm-clawd <clawd@basicmemory.com>
Co-authored-by: bm-clawd <clawd@basicmemory.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-12 15:44:47 -06:00
Paul Hernandez f1d50c2ba7 feat: Support tag: query shorthand in search (#535)
Signed-off-by: phernandez <paul@basicmachines.co>
2026-02-02 19:22:16 -06:00
Paul Hernandez 530cbac73f feat: fast edit entities, refactors for webui, enhance search (#532)
Signed-off-by: phernandez <paul@basicmachines.co>
2026-01-31 15:16:52 -06:00
Paul Hernandez b4486d20bd test: remove stdlib mocks, strengthen integration coverage (#489)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-02 14:22:01 -06:00
phernandez 2744c4b6a5 add info logging to index_entity_data background task
Signed-off-by: phernandez <paul@basicmachines.co>
2025-12-30 12:04:49 -06:00
Paul Hernandez 4a43d7df4a remove logfire instrumentation
Signed-off-by: phernandez <paul@basicmachines.co>
2025-12-13 15:22:14 -06:00
phernandez c3678a11d2 truncate content_stems to fix Postgres 8KB index row limit
Large documents (like ~1MB conversation imports) exceed Postgres's 8KB
index row limit, causing ProgramLimitExceededError. Truncate content_stems
to 6000 characters (with headroom for other columns) before indexing.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: phernandez <paul@basicmachines.co>
2025-11-29 20:23:09 -06:00
phernandez 7d763a66ff use entity.mtime for updated at in api
Signed-off-by: phernandez <paul@basicmachines.co>
2025-11-29 13:21:58 -06:00
phernandez b5d4fb559c fix: postgres/neon connection settings and search index dedupe
- Reduce db_pool_recycle from 3600s to 180s for Neon scale-to-zero
- Add connect_args for Neon serverless (statement cache, timeouts, app name)
- Dedupe observation permalinks in search indexing to avoid unique constraint violations
- Add tests for duplicate observation permalink handling

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: phernandez <paul@basicmachines.co>
2025-11-29 11:24:08 -06:00
phernandez 830775276d remove record_return=True from logfire spans
Signed-off-by: phernandez <paul@basicmachines.co>
2025-11-28 18:36:09 -06:00
phernandez 0ca02a7ebe add logfire instrumentation to services and repository code
Signed-off-by: phernandez <paul@basicmachines.co>
2025-11-28 12:47:29 -06:00
phernandez 7a8954c37e add extra logic for cloud-indexing improvements 2025-11-25 13:52:58 -06:00
Paul Hernandez fb5e9e1d77 feat: Add PostgreSQL database backend support (#439)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude <noreply@anthropic.com>
2025-11-20 11:20:29 -06:00
Paul Hernandez c0538ad2dd perf: Optimize sync/indexing for 43% faster performance (#352)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude <noreply@anthropic.com>
2025-10-12 14:41:44 -05:00
phernandez ac08a8d024 fix: update FastMCP initialization for API changes
- Remove deprecated auth_server_provider parameter
- Use auth parameter correctly with OAuthProvider instead of AuthSettings
- Fixes type error after dependency updates

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-06-03 09:08:17 -05:00
Paul Hernandez 634486107b feat: v0.13.0 pre (#122)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude <noreply@anthropic.com>
2025-06-03 01:00:40 -05:00
bm-claudeai 020957cd76 feat: Multi-project support, OAuth authentication, and major improvements (#119)
Signed-off-by: phernandez <paul@basicmachines.co>
Co-authored-by: phernandez <paul@basicmachines.co>
Co-authored-by: Claude <noreply@anthropic.com>
2025-05-25 10:07:34 -05:00
Paul Hernandez 617e60bda4 feat: permalink enhancements (#82)
- Avoiding "useless permalink values" for files without metadata
- Enable permalinks to be updated on move via config setting
2025-04-06 14:54:59 -05:00
Paul Hernandez 6a4bd54646 chore: Pre beta prep (#20)
fix: drop search_index table on db reindex 
fix: ai_resource_guide.md path
chore: remove logfire
Signed-off-by: phernandez <paul@basicmachines.co>
2025-03-07 14:31:35 -06:00
Paul Hernandez e6496df595 feat: Beta work (#17)
feat: Add multiple projects support 
feat: enhanced read_note for when initial result is not found
fix: merge frontmatter when updating note
fix: handle directory removed on sync watch
2025-03-05 18:46:04 -06:00
Paul Hernandez f8078cdd46 Fix: Beta testing fixes (#16)
Co-authored-by: phernandez <phernandez@basicmachines.co>
2025-02-28 20:23:53 -06:00
Paul Hernandez 093dab5f03 feat: Add enhanced prompts and resources (#15)
## Summary
- Add comprehensive documentation to all MCP prompt modules
- Enhance search prompt with detailed contextual output formatting
- Implement consistent logging and docstring patterns across prompt
utilities
- Fix type checking in prompt modules

## Prompts Added/Enhanced
- `search.py`: New formatted output with relevance scores, excerpts, and
next steps
- `recent_activity.py`: Enhanced with better metadata handling and
documentation
- `continue_conversation.py`: Improved context management

## Resources Added/Enhanced
- `ai_assistant_guide`: Resource with description to give to LLM to
understand how to use the tools

## Technical improvements
- Added detailed docstrings to all prompt modules explaining their
purpose and usage
- Enhanced the search prompt with rich contextual output that helps LLMs
understand results
- Created a consistent pattern for formatting output across prompts
- Improved error handling in metadata extraction
- Standardized import organization and naming conventions
- Fixed various type checking issues across the codebase

This PR is part of our ongoing effort to improve the MCP's interaction
quality with LLMs, making the system more helpful and intuitive for AI
assistants to navigate knowledge bases.

🤖 Generated with [Claude Code](https://claude.ai/code)

---------

Co-authored-by: phernandez <phernandez@basicmachines.co>
2025-02-27 20:48:56 -06:00
Paul Hernandez 37a01b806d feat: Incremental sync on watch (#13)
- incremental sync on watch
- sync non-markdown files in knowledge base
- experimental `read_resource` tool for reading non-markdown files in raw form (pdf, image)
2025-02-24 22:13:15 -06:00
phernandez 00d23a5ee1 fix: add logfire spans to cli 2025-02-18 19:56:20 -06:00
Paul Hernandez 39bd5ca08f fix: Re do enhanced read note format (#10)
Co-authored-by: phernandez <phernandez@basicmachines.co>
2025-02-17 18:47:59 -06:00
phernandez a87ac58646 Merge branch 'installer' 2025-02-12 23:23:30 -06:00
Paul Hernandez 5b4f0eafcc Pre release fixups (#5)
* configure logging

* set mcp output logging also

* fix type check errors

* fix type check

* rename Permalink schema type

* fix type errors

* add typechecks to ci workflow

* pytest coverage setup

* add tests for status cli

* sync tests coverage

* watch_service test coverage

* tests for tool_utils.py

* clean up imports

* file_utils coverage

* markdown plugins coverage

* 99% test coverage

* more test coverage, remove ObservationCategory

* more tool coverage

* fix type-check

* format, upgrade deps

---------

Co-authored-by: phernandez <phernandez@basicmachines.co>
2025-02-12 08:23:10 -06:00
phernandez f0fd1ee942 fix fuzzy search on note create 2025-01-31 16:38:06 -06:00
phernandez c2aa947be9 remove observation, relation services 2025-01-27 17:01:34 -06:00
phernandez 18bd850019 fix updates and save content. remove entity.summary 2025-01-26 00:00:37 -06:00
phernandez 9f35f8b8d2 fix unresolved links on sync 2025-01-22 22:37:34 -06:00
phernandez 220914879b delete relations/observations from search_index on delete entity during sync 2025-01-22 15:30:57 -06:00
phernandez f61c5970df add resource api route 2025-01-19 20:07:20 -06:00
phernandez 324025f87d dry up search indexing 2025-01-19 14:44:44 -06:00
phernandez e2a4fa63e5 fix context_service tests 2025-01-16 22:38:19 -06:00
phernandez 3a8a900032 search refactor 2025-01-16 18:36:47 -06:00
phernandez fe252a8578 context builder mostly working 2025-01-16 14:27:22 -06:00
phernandez a593f50785 simplify search logic 2025-01-15 17:44:38 -06:00
phernandez c4a8b85b03 refactor relation indexing 2025-01-14 19:28:05 -06:00
phernandez 73e8c5bee7 add relations and observations to search index 2025-01-14 19:06:00 -06:00
phernandez 9dff24746d index relations and observations 2025-01-14 18:14:24 -06:00
phernandez ef13b3b30a refactor sync to use link_resolver 2025-01-13 22:31:32 -06:00