Files
SpecterOps-Nemesis/projects
Lee Chagolla-Christensen 562a4e71ca Optimize observability stack and workflow throughput
Performance and configuration improvements:
- Tune OpenTelemetry Collector with batching, queuing, and memory limits
- Optimize Jaeger for write-heavy workloads with consistency disabled
- Add Prometheus scraping for OTel Collector internal metrics
- Increase document conversion prefetch count from 1 to 5
- Add CPU limit (2 cores) for document-conversion service
- Disable Tesseract OCR in default Tika config for faster processing

Workflow tracking enhancements:
- Add WorkflowTrackingService for centralized workflow state management
- Implement finalize_workflow activities for both services
- Track enrichment success/failure per workflow instance
- Support updating workflow status by object_id for subscriptions

Code improvements:
- Refactor PDF analyzer with modular extraction functions
- Add Unicode surrogate sanitization to prevent JSONB errors
- Update PII analyzer with improved error handling
- Add documentation links to Tika and Tesseract configuration

Frontend:
- Update StatsOverview component for workflow metrics display
2025-11-07 00:58:58 -08:00
..
2025-08-29 15:24:32 -07:00
2025-06-13 11:33:07 +02:00

Naming Conventions

Projects and libraries should follow Python's PEP8 naming conventions:

Modules should have short, all-lowercase names. Underscores can be used in the module name if it improves readability. Python packages should also have short, all-lowercase names, although the use of underscores is discouraged.

When an extension module written in C or C++ has an accompanying Python module that provides a higher level (e.g. more object oriented) interface, the C/C++ module has a leading underscore (e.g. _socket).