Features • Installation • Usage • Running httpx • Notes • Join Discord

httpx is a fast and multi-purpose HTTP toolkit allow to run multiple probers using retryablehttp library, it is designed to maintain the result reliability with increased threads.

Features

• Simple and modular code base making it easy to contribute.
• Fast And fully configurable flags to probe multiple elements.
• Supports multiple HTTP based probings.
• Smart auto fallback from https to http as default.
• Supports hosts, URLs and CIDR as input.
• Handles edge cases doing retries, backoffs etc for handling WAFs.

Supported probes:-

Probes	Default check	Probes	Default check
URL	true	IP	true
Title	true	CNAME	true
Status Code	true	Raw HTTP	false
Content Length	true	HTTP2	false
TLS Certificate	true	HTTP 1.1 Pipeline	false
CSP Header	true	Virtual host	false
Location Header	true	CDN	false
Web Server	true	Path	false
Web Socket	true	Ports	false
Response Time	true	Request method	false

Installation Instructions

httpx requires go1.14+ to install successfully. Run the following command to get the repo -

GO111MODULE=on go get -v github.com/projectdiscovery/httpx/cmd/httpx

Usage

httpx -h

This will display help for the tool. Here are all the switches it supports.

👉 httpx help menu 👈

Usage:
  httpx [flags]
  Flags:
TARGET:
   -vhost-input        Get a list of vhosts as input
   -H string[]         Custom Header to send with request
   -ports string[]     Port ranges to scan (nmap syntax: eg 1,2-10,11)
   -http-proxy string  HTTP Proxy, eg http://127.0.0.1:8080
   -l string           Input file containing list of hosts to process
   -x string           Request Methods to use, use 'all' to probe all HTTP methods
   -path string        Request path/file (example '/api')
   -paths string       Command separated paths or file containing one path per line (example '/api/v1,/apiv2')
   -body string        Content to send in body with HTTP request

TEMPLATE:
   -tls-grab                   Perform TLS(SSL) data grabbing
   -tech-detect                Perform wappalyzer based technology detection
   -threads int                Number of threads (default 50)
   -retries int                Number of retries
   -timeout int                Timeout in seconds (default 5)
   -vhost                      Check for VHOSTs
   -follow-redirects           Follow HTTP Redirects
   -follow-host-redirects      Only Follow redirects on the same host
   -max-redirects int          Max number of redirects to follow per host (default 10)
   -tls-probe                  Send HTTP probes on the extracted TLS domains
   -csp-probe                  Send HTTP probes on the extracted CSP domains
   -unsafe                     Send raw requests skipping golang normalization
   -pipeline                   HTTP1.1 Pipeline probe
   -http2                      HTTP2 probe
   -no-fallback                Probe both protocol (HTTPS and HTTP)
   -no-fallback-scheme         Probe with input protocol scheme
   -random-agent               Use randomly selected HTTP User-Agent header value (default true)
   -allow string[]             Allow list of IP/CIDR's to process (file or comma separated)
   -deny string[]              Deny list of IP/CIDR's to process (file or comma separated)
   -response-size-to-save int  Max response size to save in bytes (default - unlimited) (default 2147483647)
   -response-size-to-read int  Max response size to read in bytes (default - unlimited) (default 2147483647)
   -resume                     Resume scan using resume.cfg
   -exclude-cdn                Skip full port scans for CDNs (only checks for 80,443)
   -max-host-error int         Max error count per host before skipping remaining path/s (default 30)

FILTERING:
   -mc string             Match response with specific status code (-mc 200,302)
   -ml string             Match response with specific content length (-ml 102)
   -fc string             Filter response with specific status code (-fc 403,401)
   -fl string             Filter response with specific content length (-fl 23)
   -filter-string string  Filter response with specific string
   -match-string string   Match response with specific string
   -filter-regex string   Filter response with specific regex
   -match-regex string    Match response with specific regex
   -extract-regex string  Display response content with matched regex

RATE-LIMIT:
   -rate-limit int  Maximum requests to send per second (default 150)

OUTPUT:
   -o string          File to write output to (optional)
   -status-code       Display HTTP response status code
   -title             Display page title
   -location          Display location header
   -content-length    Display HTTP response content length
   -sr                Store HTTP response to directory (default 'output')
   -srd string        Custom directory to store HTTP responses (default "output")
   -json              Display output in JSON format
   -csv               Display output in CSV format
   -method            Display request method
   -silent            Silent mode
   -version           Show version of httpx
   -verbose           Verbose Mode
   -no-color          Disable colored output
   -web-server        Display server header
   -websocket         Display server using websocket
   -response-in-json  Show Raw HTTP response In Output (-json only) (deprecated)
   -include-response  Show Raw HTTP response In Output (-json only)
   -include-chain     Show Raw HTTP Chain In Output (-json only)
   -content-type      Display content-type header
   -ip                Display Host IP
   -request string    File containing raw request
   -debug             Debug mode
   -cname             Display Host cname
   -cdn               Display CDN
   -response-time     Display the response time
   -stats             Enable statistic on keypress (terminal may become unresponsive till the end)
   -store-chain       Save chain to file (default 'output')
   -probe             Display probe status

Running httpX

Running httpx with stdin

This will run the tool against all the hosts and subdomains in hosts.txt and returns URLs running HTTP webserver.

▶ cat hosts.txt | httpx 

    __    __  __       _  __
   / /_  / /_/ /_____ | |/ /
  / __ \/ __/ __/ __ \|   / 
 / / / / /_/ /_/ /_/ /   |  
/_/ /_/\__/\__/ .___/_/|_|   v1.0  
             /_/            

    projectdiscovery.io

[WRN] Use with caution. You are responsible for your actions
[WRN] Developers assume no liability and are not responsible for any misuse or damage.

https://mta-sts.managed.hackerone.com
https://mta-sts.hackerone.com
https://mta-sts.forwarding.hackerone.com
https://docs.hackerone.com
https://www.hackerone.com
https://resources.hackerone.com
https://api.hackerone.com
https://support.hackerone.com

Running httpx with file input

This will run the tool with the probe flag against all of the hosts in hosts.txt and return URLs with probed status.

▶ httpx -l hosts.txt -silent -probe

http://ns.hackerone.com [FAILED]
https://docs.hackerone.com [SUCCESS]
https://mta-sts.hackerone.com [SUCCESS]
https://mta-sts.managed.hackerone.com [SUCCESS]
http://email.hackerone.com [FAILED]
https://mta-sts.forwarding.hackerone.com [SUCCESS]
http://links.hackerone.com [FAILED]
https://api.hackerone.com [SUCCESS]
https://www.hackerone.com [SUCCESS]
http://events.hackerone.com [FAILED]
https://support.hackerone.com [SUCCESS]
https://gslink.hackerone.com [SUCCESS]
http://o1.email.hackerone.com [FAILED]
http://info.hackerone.com [FAILED]
https://resources.hackerone.com [SUCCESS]
http://o2.email.hackerone.com [FAILED]
http://o3.email.hackerone.com [FAILED]
http://go.hackerone.com [FAILED]
http://a.ns.hackerone.com [FAILED]
http://b.ns.hackerone.com [FAILED]

Running httpx with CIDR input

▶ echo 173.0.84.0/24 | httpx -silent

https://173.0.84.29
https://173.0.84.43
https://173.0.84.31
https://173.0.84.44
https://173.0.84.12
https://173.0.84.4
https://173.0.84.36
https://173.0.84.45
https://173.0.84.14
https://173.0.84.25
https://173.0.84.46
https://173.0.84.24
https://173.0.84.32
https://173.0.84.9
https://173.0.84.13
https://173.0.84.6
https://173.0.84.16
https://173.0.84.34

Running httpx with subfinder

subfinder -d hackerone.com | httpx -title -tech-detect -status-code

    __    __  __       _  __
   / /_  / /_/ /_____ | |/ /
  / __ \/ __/ __/ __ \|   /
 / / / / /_/ /_/ /_/ /   |
/_/ /_/\__/\__/ .___/_/|_|
             /_/              v1.0.6

    projectdiscovery.io

Use with caution. You are responsible for your actions
Developers assume no liability and are not responsible for any misuse or damage.
https://mta-sts.managed.hackerone.com [404] [Page not found · GitHub Pages] [Varnish,GitHub Pages,Ruby on Rails]
https://mta-sts.hackerone.com [404] [Page not found · GitHub Pages] [Varnish,GitHub Pages,Ruby on Rails]
https://mta-sts.forwarding.hackerone.com [404] [Page not found · GitHub Pages] [GitHub Pages,Ruby on Rails,Varnish]
https://docs.hackerone.com [200] [HackerOne Platform Documentation] [Ruby on Rails,jsDelivr,Gatsby,React,webpack,Varnish,GitHub Pages]
https://support.hackerone.com [301,302,301,200] [HackerOne] [Cloudflare,Ruby on Rails,Ruby]
https://resources.hackerone.com [301,301,404] [Sorry, no Folders found.]

📋 Notes

• As default, httpx checks for HTTPS probe and fall-back to HTTP only if HTTPS is not reachable.
• For printing both HTTP/HTTPS results, no-fallback flag can be used.
• Custom scheme for ports can be defined, for example -ports http:443,http:80,https:8443
• vhost, http2, pipeline, ports, csp-probe, tls-probe and path are unique flag with different probes.
• Unique flags should be used for specific use cases instead of running them as default with other flags.
• When using json flag, all the information (default probes) included in the JSON output.

Thanks

httpx is made with 🖤 by the projectdiscovery team. Community contributions have made the project what it is. See the Thanks.md file for more details. Do also check out these similar awesome projects that may fit in your workflow:

Probing feature is inspired by @tomnomnom/httprobe work ❤️