- Handle error from format.Parse in streamInput
- Add InputMode validation to ValidateOptions for early error detection
- Extract duplicate format validation to getInputFormat helper
- Fix shadow variable err in loadFromFormat callback
- Add bounds check in test before index access
- Add test for malformed XML input
- Run go mod tidy to fix dependency marking
Add support for storing httpx scan results directly to databases with
both CLI flags and YAML config file options.
Supported databases:
- MongoDB
- PostgreSQL
- MySQL
Features:
- Batched writes for performance (configurable batch size)
- Auto-flush with configurable interval
- Individual columns for each Result field (not JSON blob)
- Support for environment variable HTTPX_DB_CONNECTION_STRING
- Option to omit raw request/response data (-rdbor)
New CLI flags under OUTPUT group:
- -rdb, -result-db: enable database storage
- -rdbc, -result-db-config: path to YAML config file
- -rdbt, -result-db-type: database type (mongodb, postgres, mysql)
- -rdbcs, -result-db-conn: connection string
- -rdbn, -result-db-name: database name (default: httpx)
- -rdbtb, -result-db-table: table/collection name (default: results)
- -rdbbs, -result-db-batch-size: batch size (default: 100)
- -rdbor, -result-db-omit-raw: omit raw data
Closes#1973Closes#2360Closes#2361Closes#2362
When FollowRedirects is enabled and the target redirects to a different host
(e.g., hackerone.com -> www.hackerone.com), the favicon URL is correctly
resolved to the final host, but the cloned request's Host header still
contains the original host. This causes the server to return 404 because
the Host header doesn't match the URL.
This fix updates the Host header to match the resolved URL host before
making the favicon request.
Add support for passive detection of CPE (Common Platform Enumeration)
identifiers and WordPress plugins/themes using awesome-search-queries.
CPE Detection (-cpe flag):
- Matches response title, body, and favicon hash against patterns
- Extracts product, vendor, and generates CPE 2.3 identifiers
- Uses patterns from Shodan, FOFA, Google dorks
WordPress Detection (-wp flag):
- Detects plugins via /wp-content/plugins/[name]/ patterns
- Detects themes via /wp-content/themes/[name]/ patterns
- Validates against known plugins/themes list
New CLI flags in PROBES group:
- -cpe: display CPE based on awesome-search-queries
- -wp, -wordpress: display WordPress plugins and themes
Both are automatically included in JSON/CSV output.
Closes#1975
Use URL.Hostname() instead of URL.Host to extract the hostname for DNS
resolution. This ensures the port number is not included in the DNS
query when probing all IPs for a URL like http://example.com:8080.
Fixes#2346
Signed-off-by: majiayu000 <1835304752@qq.com>
* Consolidate output‑type checks into a single `jsonOrCsvOrMD` variable.
* Adjust file‑opening logic to add the “.md” suffix only when needed.
* Update related conditionals to respect the new combined flag handling.
* Ensure markdown output is correctly written without creating “.md.md” files.
- Split Markdown generation into `MarkdownHeader` and `MarkdownRow` for clearer separation.
- Remove body‑preview block from Markdown rows.
- Add guard in runner to write the header only once per run.
- Introduce `MarkDownOutput` flag and CLI option (`-markdown` / `-md`).
- Implement `Result.MarkdownOutput` to generate a formatted Markdown table with optional title and CDN columns, and a response body preview.
- Add markdown escaping helper.
- Update runner to create `.md` output file, handle markdown generation per result, and write to file or stdout.
- Adjust option parsing and related logic to include markdown handling.
String concatenation doesn't properly handle IPv6 addresses which need brackets.
Example: [::1]:443 would become ::1:80 instead of [::1]:80
Uses net.JoinHostPort() which correctly formats IPv6 addresses.
- When falling back from HTTPS to HTTP on port 443, now switches to port 80
- When falling back from HTTP to HTTPS on port 80, now switches to port 443
- Prevents 'plain HTTP request sent to HTTPS port' errors during fallback
- Added test cases for determineMostLikelySchemeOrder and port fallback logic
- Changed 'host' field to return actual hostname (e.g., example.com)
- Added new 'host_ip' field for the resolved/dialed IP address
- Fixes semantic issue where 'host' was incorrectly returning IP
feat: update -ldp option to show default ports in CLI output
- Modified URL formatting in runner.go to respect LeaveDefaultPorts option
- Fixed AddURLDefaultPort function to actually add default ports (80/443)
- When -ldp is used, CLI output now shows https://example.com:443 instead of https://example.com
- Maintains backward compatibility - default behavior unchanged
Fixes CLI output inconsistency where -ldp flag only affected Host headers
but not the displayed URL in console output.
Dogan Can Bakir