Files
basicmachines-co-basic-memory/tests
claude[bot] bb02091aa3 feat: Optimize memory usage for large file syncs with streaming checksums
Implements streaming checksum computation for files larger than 1MB to
significantly reduce memory usage during sync operations. This addresses
high memory consumption (1.5-1.7GB) when syncing projects with large
binary files (PDFs, images, videos).

Changes:
- Add compute_checksum_streaming() function that reads files in 64KB chunks
- Update sync_service._compute_checksum_async() to use streaming for files >1MB
- Add streaming_checksum_threshold_mb config option (default: 1MB)
- Add comprehensive tests for streaming checksum functionality

Impact:
- Reduces memory usage for large file syncs from ~80MB to <1MB per file
- Maintains backward compatibility with existing checksum behavior
- Configurable threshold allows tuning for different deployment scenarios

Fixes #382

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Paul Hernandez <phernandez@users.noreply.github.com>
2025-10-17 02:00:14 +00:00
..