10 Commits

Author SHA1 Message Date
Lee Chagolla-Christensen 1b5997ffe6 chore: bump dependencies and format code 2026-05-20 14:30:16 -07:00
Lee Chagolla-Christensen e5b14901a4 Upgrade pillow, vite, pytest, cryptography for Dependabot alerts
- pillow 12.2.0 (HIGH: FITS GZIP decompression bomb)
- vite 6.4.2 (HIGH: arbitrary file read via WebSocket)
- pytest 9.0.3 (MEDIUM: vulnerable tmpdir handling)
- cryptography 46.0.7 (MEDIUM: buffer overflow)
- Dismiss 3 stale alerts for deleted poetry.lock files
2026-04-14 07:40:47 -07:00
harmj0y d5942b3a62 dependabot fixes 2026-04-02 12:47:37 -07:00
harmj0y db7ebc0b32 More dependabot fixes 2026-04-02 12:42:53 -07:00
harmj0y f17f232ae9 Dependabot fixes
- `aiohttp` and `fastmcp` package fixes
2026-04-02 12:21:27 -07:00
Lee Chagolla-Christensen 11a67703d1 Fix dependabot security vulnerabilities across all projects
- cryptography 46.0.3 → 46.0.5 (high: subgroup attack on SECT curves)
- pillow 11.3.0 → 12.1.1 (high: OOB write loading PSD images)
- pydantic-ai 0.7.2 → 1.64.0 (high: SSRF in URL download handling)
- rollup 4.53.3 → 4.59.0 (high: arbitrary file write via path traversal)
- werkzeug 3.1.5 → 3.1.6 (medium: safe_join Windows device names)
- flask 3.1.2 → 3.1.3 (low: session Vary Cookie header)
- go-git 5.16.4 → 5.16.5 (medium: .idx/.pack integrity check bypass)
- circl 1.6.1 → 1.6.3 (low: secp384r1 CombinedMult miscalculation)
- xz 0.5.12 → 0.5.15 (medium: memory leak decoding corrupted LZMA)
2026-03-03 16:01:14 -08:00
Lee Chagolla-Christensen 9d62493623 Type checker, reg parsing improvements, UI improvements (#101)
* fix: upgrade python-multipart to 0.0.22 to address CVE-2026-24486 (Dependabot #683)

* update deps+skill, pyright

* pyright + tests in CLI

* Add pyright type checking and enhance registry hive analysis

Registry Hive Analyzer Enhancements:
- Extract machine SID from SAM domain V value with binary SID decoding
- Extract per-user metadata via regipy (ACB flags, timestamps, full
  name, comment via USER_ACCOUNT_V)
- Compute password expiration from domain max password age policy
- Detect empty LM/NT hashes via well-known constants
- Parse DCC cached domain credentials into structured entries
- Store DPAPI system machine_key/user_key as separate fields
- Add structured secret_type field to LSA secrets (dcc, dpapi_system,
  hex_blob, generic)
- Add detailed markdown and plain-text formatters for SAM accounts
  and LSA secrets, replacing _get_lsa_secret_output_string
- Fix SYSTEM hive attribute names: computer_name -> machinename,
  current_control_set -> currentcontrol
- Switch SAM user iteration from sam.users to sam.secrets (pypykatz API)
- Add FILETIME-to-UTC and regipy value-to-bytes helpers

New Tests:
- Add test_registry_hive.py with SAM, SECURITY, and SYSTEM hive fixtures
- Add test_container.py for container analyzer
- Add SAM/SECURITY/SYSTEM binary test fixtures

Pyright Setup:
- Add pyrightconfig.json (basic mode, Python 3.13) to all libs and projects
- Add pyright>=1.1 as dev dependency to all pyproject.toml files
- Update all uv.lock files accordingly
- Update lint.sh to deactivate active venvs and verify pyright availability

Type Annotation Fixes:
- Fix globals initialized as None without Optional type across
  file_enrichment, document_conversion, and agents global_vars
- Fix StorageMinio return types: upload/upload_file/upload_uploadfile
  return str, not uuid.UUID
- Fix MockStorageMinio to match updated StorageMinio return types
- Add explicit type annotations to dict literals in chromekey.py,
  pdf/analyzer.py, registry_hive/analyzer.py, and publish_findings.py
- Fix kubeconfig current_context parameter to accept str | None
- Fix container_contents allowed_extensions: set = None -> set | None = None
- Fix file subscription file_queue: asyncio.Queue = None -> asyncio.Queue | None = None
- Fix web_api upload_file to wrap object_id in uuid.UUID() for response

None-safety Assertions:
- Add assert statements for asyncpg_pool, tracking_service,
  workflow_client, workflow_manager, file_linking_engine, file_queue,
  gotenberg_url, and process.stdout across all activity, subscription,
  route, and workflow files in file_enrichment and document_conversion
- Add assertions for asyncpg_pool in all chromium processors
- Add assertions for File.from_metadata timestamp/expiration fields
- Add assertion for alerting GQL client session type

Pyright Ignore Annotations:
- Suppress third-party type issues in Dapr workflow/activity APIs,
  gRPC subscription imports, ccache/lnk/office_doc attribute access,
  and nemesis_dpapi FlagMixin operators
- Add file-level suppression for office2john.py, pdf2john.py,
  pe/analyzer.py, and test harness files

Bug Fixes:
- Fix office2john.py format string: bare % filename -> % (filename, stream)
- Fix container analyzer 7z iteration: iterate sz.files list instead
  of calling .items()
- Fix file_linking rules_engine: store match result to avoid double call
- Fix logger.exception calls: remove exception object as first arg in
  storage.py, cookies.py, enrichments.py, housekeeping/main.py
- Fix document_conversion lifespan: use stack.callback() for sync shutdown
- Fix NoseyParkerOutput fallback: add missing workflow_id field
- Fix regipy hive_type: handle None return from RegistryHive.hive_type

DPAPI Manager:
- Remove unused guid parameter from get_system_credentials across
  DpapiManager, NullDpapiManager, and DpapiManagerProtocol

Added Missing Dependencies (file_enrichment_modules):
- pypykatz>=0.6.11, pyarrow>=19.0.1, msoffcrypto-tool>=5.4.2,
  oletools>=0.60.2, regipy>=5.2.0, pillow>=11.3.0

* quiet console logs

* feat: lazy file loading with backend range request support

Add offset/length query params to the download endpoint so the frontend
can request partial file content.  FileViewer now fetches data on demand
— hex, transform (Strings, etc.), ZIP, SQLite, and image tabs only load
when activated.  Text-based content is capped at 10 MB previews.

* fix: transform tabs stuck on "Loading content..."

- Prevent stale WS subscription from overwriting fetched content
- Show retry button when transform content fails to load
- Reset fetch guard on non-OK HTTP responses

* fix: hex tab deferred loading with full file content

* feat: truncation dropdown, spinner overlay, and tab render refactor

- Add truncation dropdown to MonacoContentViewer for files > 10MB,
  replacing the old banner alert and hex "Load Hex View" button
- Add spinner overlay on Monaco editor while loading full content
- Wire truncation support to transform tabs (monaco/json types),
  enrichment tabs, and text tabs
- Change hex tab to auto-load first 10MB preview with dropdown for
  full file instead of requiring manual load
- Default word wrap to off
- Remove "File is too large" warning message
- Refactor ~150-line ternary chain into explicit per-tab render
  functions (renderPreviewTab, renderZipTab, renderSqliteTab,
  renderTextTab, renderHexTab, renderTabContent) with shared helpers
  (renderFullFileContent, getTextContent)

* feat: improve Yara Rules UI

- Compact table rows with tighter padding
- Add X button and Esc key to close editor dialog
- Disable Save button when rule content is unchanged
- Disable Create button with inline warning when rule name already exists
- Default source to "Created manually by <user>" for new rules
- Update source placeholder to "e.g. /yara_rules/custom.yara"

* update gitignore

---------

Co-authored-by: Lee Chagolla-Christensen <lee@localhost>
2026-02-05 14:34:25 -08:00
Lee Chagolla-Christensen 05719c4e58 bump deps 2026-02-03 23:08:11 -08:00
Lee Chagolla-Christensen 3d8336977d Python 3.13, linting/quality, and agent reliability
- Raise requires-python to >=3.13 across all projects/libs; set ruff target to py313
- Fix bare excepts, unused variables/imports, Optional→union syntax, and import ordering
- Refactor chatbot MCP server to async subprocess with health checks and graceful shutdown
- Simplify file-enrichment prod Dockerfile to use uv sync with UV_PROJECT_ENVIRONMENT
- Fix module_loader to use fully qualified analyzer module name
- Make YARA _analyze_yara async and fix rule_manager access in workflow
- Enable LiteLLM healthcheck and add agents→litellm dependency
- Add PE analyzer tests with PyInstaller fixture;
- Update PDF tests for nested metadata
- Suppress third-party DeprecationWarnings, use async-lru for LLM status cache
- Add PHOENIX_ENABLED validation to nemesis-ctl.sh; remove debug pprint in agents
- Add Claude Code managing-packages skill, update CLAUDE.md guidance
2026-02-03 20:36:12 -08:00
Lee Chagolla-Christensen b5b00ca72a Migrate to uv from poetry (#99)
* initial uv migration

* update github workflows

* use python module not system commands

* remove poetry references

* fix uv in prod images

---------

Co-authored-by: Lee Chagolla-Christensen <lee@localhost>
2026-01-14 15:20:31 -08:00