* Conversion of NoseyParker scanner to Titus
- Old `noseyparker_scanner` project removed, new `titus_scaner` Golang project built
- Replaced `noseyparker` references with `titus` as appropriate
- Bumped various built-in docker image versions (Dapr, Gotenberg, etc.)
* Replace ZIP-only extraction with Titus library archive support
Replace custom ZIP extraction code with Titus library's enum.ExtractText()
to support scanning secrets inside zip, jar, war, ear, apk, ipa, xpi, crx,
tar, tar.gz/tgz, and 7z archives. Document formats (xlsx, docx, pdf, etc.)
are intentionally excluded since Nemesis handles those via document_conversion.
- Add original_path to TitusInput so Titus can dispatch by file extension
(MinIO stores files by UUID without extensions)
- Add magic-byte fallback detection for ZIP, 7z, gzip, and tar
- Preserve git-repo-in-ZIP detection as a special case before archive scan
- Rename env vars: DECOMPRESS_ZIPS -> EXTRACT_ARCHIVES,
MAX_EXTRACT_SIZE_MB -> EXTRACT_MAX_TOTAL_SIZE_MB, and add
EXTRACT_MAX_FILE_SIZE_MB and EXTRACT_MAX_DEPTH
- Add unit tests for archive detection and config loading
* Integrated (optional) Titus secret validation
- Integrated (optional) Titus secret validation that can be enabled
* Added Titus rule ID display and ability to disable specific rules
- Added Titus rule ID display and ability to disable specific built-in rules (by ID)
* `titus_scanner` fixes
- One scanner per concurrent file to prevent timeouts
- Fix custom rule regexes to prevent timeouts
- Exclude a few default run-away rules causing timeouts
- Bumped cores for titus scanner to 2
* Enable Vectorscan/Hyperscan acceleration for titus_scanner
Switch from the pure-Go regexp2 engine to the Vectorscan (Hyperscan)
SIMD-accelerated engine, using a fork with parallelized regexp2
confirmation and content-based deduplication
Key changes:
- Dockerfile: install libvectorscan-dev/libvectorscan5, build with
CGO_ENABLED=1 and -tags vectorscan
- go.mod: replace praetorian-inc/titus with HarmJ0y/titus fork that
parallelizes the regexp2 confirmation loop and uses content-based
dedup (matching portable engine behavior)
- Add bounded-concurrency semaphore in handler to prevent OOM from
unbounded goroutines while still returning 200 to Dapr immediately
- Add content-based deduplication, match count cap, snippet truncation,
and payload size logging to prevent oversized publishes
- Replace O(N*M) line/column computation with precomputed line index
using binary search
- Add GOMEMLIMIT and memory limit to container resources
* Update README.md
* remove `noseyparker_scanner` container
* Add EVTX (Windows Event Log) enrichment module
Parses .evtx files and extracts security-relevant data:
- Markdown summary transform with event counts, timeline, unique accounts/IPs
- Downloadable CSV transforms for account changes, group membership changes,
explicit credential use (4648), process creation (4688), and task changes
- Findings for high-signal events: audit log cleared, new services, admin group
changes, account changes, explicit creds, scheduled tasks, PS script blocks
- PowerShell 4104 script blocks reassembled and resubmitted to enrichment
pipeline as child .ps1 files for full Titus/YARA scanning
- Added de-duping for duplicate PS scripts carved from EVTXs (hash-based)
* EVTX mods
- Power timelining done for SYSTEM
- Inbound/outbound auth for SECURITY
- Filtering for default system accounts
- CSV viewer now can download files, transforms for EVTX transformed to the viewer
* Convert titus scanner to Dapr bulk subscribe for batch processing
- Add bulk subscribe types to models (BulkMessagePayload, BulkResponse, etc.)
- Replace async single-event handler with synchronous HandleBulkEvent
- Process batches with bounded concurrency via semaphore + WaitGroup
- Handle edge cases: duplicate entryId, empty objectID/entryId, panic recovery
- Add BulkMaxMessages and BulkMaxAwaitDurationMs config with clamping
- Set app-max-concurrency to 1 (parallelism within work batch via scanner pool)
---------
Co-authored-by: Lee Chagolla-Christensen <lee@localhost>
- Raise requires-python to >=3.13 across all projects/libs; set ruff target to py313
- Fix bare excepts, unused variables/imports, Optional→union syntax, and import ordering
- Refactor chatbot MCP server to async subprocess with health checks and graceful shutdown
- Simplify file-enrichment prod Dockerfile to use uv sync with UV_PROJECT_ENVIRONMENT
- Fix module_loader to use fully qualified analyzer module name
- Make YARA _analyze_yara async and fix rule_manager access in workflow
- Enable LiteLLM healthcheck and add agents→litellm dependency
- Add PE analyzer tests with PyInstaller fixture;
- Update PDF tests for nested metadata
- Suppress third-party DeprecationWarnings, use async-lru for LLM status cache
- Add PHOENIX_ENABLED validation to nemesis-ctl.sh; remove debug pprint in agents
- Add Claude Code managing-packages skill, update CLAUDE.md guidance
* Added Cobalt Strike Connector
* Updated cli readme
* Add Cobalt Strike to C2 connectors list in README
Updated README to include Cobalt Strike in C2 connectors.
* Update usage guide to include Cobalt Strike
* Document Cobalt Strike Connector setup and usage
Added Cobalt Strike Connector section with configuration and usage instructions.
* Updated documentation on REST API startup and modified Path to include the file name
* Update Cobalt Strike URL to include port 50443
* adding code 403 to force reauthentication to mathc with the CS response
- PII analyzer disabled by default, can be enabled via ENV variable
- Created `enrichment_configuration.md` documentation documenting all enrichment module configurations
* upgrade to dapr postgresv2 statestore
* actually make it v2
* dapr state table name, cleanup subscriptions/globals
* proper exceptions
* formatting/lint
* Refactor workflow tracking and improve activity input handling
- Extract workflow tracking logic into dedicated WorkflowTrackingService
- Simplify activity signatures to accept specific parameters instead of generic dicts
- Remove unused asyncio event loop references from enrichment modules
- Update YaraRuleManager initialization and method names
* re-added workflow tracking in the DB
* update uvicorn prod options
* enrichment work parallelism, convert queues from broadcast to task queues
* Refactor pubsub and improve workflow parallelism
- Split Dapr pubsub Dapr yaml components into topic-specific queues (alerting, dotnet, dpapi, files, noseyparker, workflow_monitor)
- Update all Dapr volume mounts to reference new topic-specific pubsub components
- Converted queues to task queues
- Use YAML anchors to reduce duplication for file-enrichment replicas
- Pass asyncpg pool to enrichment modules instead of creating connections
- Add asyncpg_pool parameter throughout chromium and enrichment module analyzers
- Update VSCode workspace (removed InspectAssembly, renamed dotnet_api to dotnet_service)
- Added curl commands for Jaeger API to performance docs to help with perf troubleshooting
- Created common.queues module to centralize pubsub/topic names (eases
future refactoring)
* worker mods
* Workflow performance tuning, fix pubsub config, CLI arg changes
- Fix pubsub deleteWhenUnused typo (deletedWhenUnused)
- Add LOG_LEVEL environment variable support across services
- CLI: Rename --repeat to --times, add --max-files option
- Increase files pubsub prefetchCount from 25 to 50
- Add MAX_PARALLEL_WORKFLOWS configuration
- Fix DotNetAssemblyAnalysis null handling with field validators
- Update dashboard to show cumulative files/findings over time
- Add RUST_LOG environment variable support to noseyparker
- Update CHANGELOG for 2.1.4 release notes
* Dapr 1.16.2 and use db transactions
- Upgrade all Dapr containers from 1.16.1 to 1.16.2
- Reduce enrichment parallelism default from 25 to 5 workflows
- Reduce healthcheck intervals from 10s to 5s for alerting and document conversion
- Fixed DPAPI eventing to use new pubsubs
- Refactor file_linking database operations to use atomic upserts and avoid deadlocks
- Add WriteOnceViolationError handling in DPAPI masterkey analyzer
- Wrap database operations in transactions for enrichment storage and plaintext indexing
- Fix postgres notification handler closure variable capture
* remove unused start_time
* Scheduler persistence, workflow concurrency tuning, and config cleanup
- Add volume for Dapr scheduler and init service
- Add scheduler dependency to file enrichment service
- Add async workflow client libraries
- Format and cleanup compose.yaml (spacing, indentation, empty lines)
* Migrate file_enrichment to async Dapr client and optimize Dockerfile
- Use async DaprClient where possible in file_enrichment
- Improve Dockerfile caching
- Add asyncpg connection pool helper and fix typo in secret store name
- Include VS Code debug configuration for document_conversion
- Remove unused dapr_client from DpapiBlobAnalyzer
- Clean up activity return types and better handle exceptions
* Enrichment tracking for NoseyParker and logging cleanup
- Add workflow_id to NoseyParkerInput and NoseyParkerOutput models
- Remove workflow lookup query in noseyparker subscription handler
- Adjust jaeger_perf_stats.sh output formatting and precision
- Add type hints for async functions
* noseyparker scanner perf, tracing for update_enrichment_results
---------
Co-authored-by: Lee Chagolla-Christensen <lee@localhost>
- Switch from using POSTGRES_CONNECTION_STRING to
POSTGRES_USER/POSTGRES_PASSWORD/POSTGRES_DATABASE/etc.
- When no apprise URL is configured, use the test echo'ing endpoint
Will Schroeder