Falls back to http.ProxyFromEnvironment for the HTTP transport and
reads the same vars manually for the headless launcher (chromedp
does not read env).
Closes#2492
* fix(db): persist CPE in postgres and mysql (#2487)
Adds cpe column to both SQL schemas, idempotent migration for
existing tables, and JSON binding in InsertBatch.
* adding comment note
* fixing lint
---------
Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
- Handle error from format.Parse in streamInput
- Add InputMode validation to ValidateOptions for early error detection
- Extract duplicate format validation to getInputFormat helper
- Fix shadow variable err in loadFromFormat callback
- Add bounds check in test before index access
- Add test for malformed XML input
- Run go mod tidy to fix dependency marking
Add support for storing httpx scan results directly to databases with
both CLI flags and YAML config file options.
Supported databases:
- MongoDB
- PostgreSQL
- MySQL
Features:
- Batched writes for performance (configurable batch size)
- Auto-flush with configurable interval
- Individual columns for each Result field (not JSON blob)
- Support for environment variable HTTPX_DB_CONNECTION_STRING
- Option to omit raw request/response data (-rdbor)
New CLI flags under OUTPUT group:
- -rdb, -result-db: enable database storage
- -rdbc, -result-db-config: path to YAML config file
- -rdbt, -result-db-type: database type (mongodb, postgres, mysql)
- -rdbcs, -result-db-conn: connection string
- -rdbn, -result-db-name: database name (default: httpx)
- -rdbtb, -result-db-table: table/collection name (default: results)
- -rdbbs, -result-db-batch-size: batch size (default: 100)
- -rdbor, -result-db-omit-raw: omit raw data
Closes#1973Closes#2360Closes#2361Closes#2362
Add support for passive detection of CPE (Common Platform Enumeration)
identifiers and WordPress plugins/themes using awesome-search-queries.
CPE Detection (-cpe flag):
- Matches response title, body, and favicon hash against patterns
- Extracts product, vendor, and generates CPE 2.3 identifiers
- Uses patterns from Shodan, FOFA, Google dorks
WordPress Detection (-wp flag):
- Detects plugins via /wp-content/plugins/[name]/ patterns
- Detects themes via /wp-content/themes/[name]/ patterns
- Validates against known plugins/themes list
New CLI flags in PROBES group:
- -cpe: display CPE based on awesome-search-queries
- -wp, -wordpress: display WordPress plugins and themes
Both are automatically included in JSON/CSV output.
Closes#1975
- Introduce `MarkDownOutput` flag and CLI option (`-markdown` / `-md`).
- Implement `Result.MarkdownOutput` to generate a formatted Markdown table with optional title and CDN columns, and a response body preview.
- Add markdown escaping helper.
- Update runner to create `.md` output file, handle markdown generation per result, and write to file or stdout.
- Adjust option parsing and related logic to include markdown handling.
* more additions and enhancements to httpx
* feat: added enhancements to favicon + made API public
* update deps
* misc additions
* fix hasChain with 1 status code not working
* Revert "fix hasChain with 1 status code not working"
This reverts commit fabadcddbc.
---------
Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
Dogan Can Bakir