Features • Installation • Usage • Running httpx • Notes • Join Discord

httpx is a fast and multi-purpose HTTP toolkit allow to run multiple probers using retryablehttp library, it is designed to maintain the result reliability with increased threads.

Features

• Simple and modular code base making it easy to contribute.
• Fast And fully configurable flags to probe multiple elements.
• Supports multiple HTTP based probings.
• Smart auto fallback from https to http as default.
• Supports hosts, URLs and CIDR as input.
• Handles edge cases doing retries, backoffs etc for handling WAFs.

Supported probes:-

Probes	Default check	Probes	Default check
URL	true	IP	true
Title	true	CNAME	true
Status Code	true	Raw HTTP	false
Content Length	true	HTTP2	false
TLS Certificate	true	HTTP 1.1 Pipeline	false
CSP Header	true	Virtual host	false
Location Header	true	CDN	false
Web Server	true	Path	false
Web Socket	true	Ports	false
Response Time	true	Request method	false

Installation Instructions

httpx requires go1.14+ to install successfully. Run the following command to get the repo -

GO111MODULE=on go get -v github.com/projectdiscovery/httpx/cmd/httpx

Usage

httpx -h

This will display help for the tool. Here are all the switches it supports.

👉 httpx help menu 👈

  -H value
        Custom Header to send with request
  -allow value
        Allow list of IP/CIDR's to process (file or comma separated)
  -body string
        Content to send in body with HTTP request
  -cdn
        Diplay CDN
  -cname
        Display Host cname
  -content-length
        Display HTTP response content length
  -content-type
        Display content-type header
  -csp-probe
        Send HTTP probes on the extracted CSP domains
  -csv
        Display output in CSV format
  -debug
        Debug mode
  -deny value
        Deny list of IP/CIDR's to process (file or comma separated)
  -exclude-cdn
        Skip full port scans for CDNs (only checks for 80,443)
  -extract-regex string
        Display response content with matched regex
  -fc string
        Filter response with specific status code (-fc 403,401)
  -filter-regex string
        Filter response with specific regex
  -filter-string string
        Filter response with specific string
  -fl string
        Filter response with specific content length (-fl 23)
  -follow-host-redirects
        Only Follow redirects on the same host
  -follow-redirects
        Follow HTTP Redirects
  -http-proxy string
        HTTP Proxy, eg http://127.0.0.1:8080
  -http2
        HTTP2 probe
  -include-chain
        Show Raw HTTP Chain In Output (-json only)
  -include-response
        Show Raw HTTP response In Output (-json only)
  -ip
        Display Host IP
  -json
        Display output in JSON format
  -l string
        Input file containing list of hosts to process
  -location
        Display location header
  -match-regex string
        Match response with specific regex
  -match-string string
        Match response with specific string
  -max-host-error int
        Max error count per host before skipping remaining path/s (default 30)
  -max-redirects int
        Max number of redirects to follow per host (default 10)
  -mc string
        Match response with specific status code (-mc 200,302)
  -method
        Display request method
  -ml string
        Match response with specific content length (-ml 102)
  -no-color
        Disable colored output
  -no-fallback
        Probe both protocol (HTTPS and HTTP)
  -no-fallback-scheme
        Probe with input protocol scheme
  -o string
        File to write output to (optional)
  -path string
        Request path/file (example '/api')
  -paths string
        Command separated paths or file containing one path per line (example '/api/v1,/apiv2')
  -pipeline
        HTTP1.1 Pipeline probe
  -ports value
        Port ranges to scan (nmap syntax: eg 1,2-10,11)
  -probe
        Display probe status
  -random-agent
        Use randomly selected HTTP User-Agent header value (default true)
  -rate-limit int
        Maximum requests to send per second (default 150)
  -request string
        File containing raw request
  -response-in-json
        Show Raw HTTP response In Output (-json only) (deprecated)
  -response-size-to-read int
        Max response size to read in bytes (default - unlimited) (default 2147483647)
  -response-size-to-save int
        Max response size to save in bytes (default - unlimited) (default 2147483647)
  -response-time
        Display the response time
  -resume
        Resume scan using resume.cfg
  -retries int
        Number of retries
  -silent
        Silent mode
  -sr
        Store HTTP response to directoy (default 'output')
  -srd string
        Custom directory to store HTTP responses (default "output")
  -stats
        Enable statistic on keypress (terminal may become unresponsive till the end)
  -status-code
        Display HTTP response status code
  -store-chain
        Save chain to file (default 'output')
  -tech-detect
        Perform wappalyzer based technology detection
  -threads int
        Number of threads (default 50)
  -timeout int
        Timeout in seconds (default 5)
  -title
        Display page title
  -tls-grab
        Perform TLS(SSL) data grabbing
  -tls-probe
        Send HTTP probes on the extracted TLS domains
  -unsafe
        Send raw requests skipping golang normalization
  -verbose
        Verbose Mode
  -version
        Show version of httpx
  -vhost
        Check for VHOSTs
  -vhost-input
        Get a list of vhosts as input
  -web-server
        Display server header
  -websocket
        Display server using websocket
  -x string
        Request Methods to use, use 'all' to probe all HTTP methods

Running httpX

Running httpx with stdin

This will run the tool against all the hosts and subdomains in hosts.txt and returns URLs running HTTP webserver.

▶ cat hosts.txt | httpx 

    __    __  __       _  __
   / /_  / /_/ /_____ | |/ /
  / __ \/ __/ __/ __ \|   / 
 / / / / /_/ /_/ /_/ /   |  
/_/ /_/\__/\__/ .___/_/|_|   v1.0  
             /_/            

    projectdiscovery.io

[WRN] Use with caution. You are responsible for your actions
[WRN] Developers assume no liability and are not responsible for any misuse or damage.

https://mta-sts.managed.hackerone.com
https://mta-sts.hackerone.com
https://mta-sts.forwarding.hackerone.com
https://docs.hackerone.com
https://www.hackerone.com
https://resources.hackerone.com
https://api.hackerone.com
https://support.hackerone.com

Running httpx with file input

This will run the tool with the probe flag against all of the hosts in hosts.txt and return URLs with probed status.

▶ httpx -l hosts.txt -silent -probe

http://ns.hackerone.com [FAILED]
https://docs.hackerone.com [SUCCESS]
https://mta-sts.hackerone.com [SUCCESS]
https://mta-sts.managed.hackerone.com [SUCCESS]
http://email.hackerone.com [FAILED]
https://mta-sts.forwarding.hackerone.com [SUCCESS]
http://links.hackerone.com [FAILED]
https://api.hackerone.com [SUCCESS]
https://www.hackerone.com [SUCCESS]
http://events.hackerone.com [FAILED]
https://support.hackerone.com [SUCCESS]
https://gslink.hackerone.com [SUCCESS]
http://o1.email.hackerone.com [FAILED]
http://info.hackerone.com [FAILED]
https://resources.hackerone.com [SUCCESS]
http://o2.email.hackerone.com [FAILED]
http://o3.email.hackerone.com [FAILED]
http://go.hackerone.com [FAILED]
http://a.ns.hackerone.com [FAILED]
http://b.ns.hackerone.com [FAILED]

Running httpx with CIDR input

▶ echo 173.0.84.0/24 | httpx -silent

https://173.0.84.29
https://173.0.84.43
https://173.0.84.31
https://173.0.84.44
https://173.0.84.12
https://173.0.84.4
https://173.0.84.36
https://173.0.84.45
https://173.0.84.14
https://173.0.84.25
https://173.0.84.46
https://173.0.84.24
https://173.0.84.32
https://173.0.84.9
https://173.0.84.13
https://173.0.84.6
https://173.0.84.16
https://173.0.84.34

Running httpx with subfinder

subfinder -d hackerone.com | httpx -title -tech-detect -status-code

    __    __  __       _  __
   / /_  / /_/ /_____ | |/ /
  / __ \/ __/ __/ __ \|   /
 / / / / /_/ /_/ /_/ /   |
/_/ /_/\__/\__/ .___/_/|_|
             /_/              v1.0.6

    projectdiscovery.io

Use with caution. You are responsible for your actions
Developers assume no liability and are not responsible for any misuse or damage.
https://mta-sts.managed.hackerone.com [404] [Page not found · GitHub Pages] [Varnish,GitHub Pages,Ruby on Rails]
https://mta-sts.hackerone.com [404] [Page not found · GitHub Pages] [Varnish,GitHub Pages,Ruby on Rails]
https://mta-sts.forwarding.hackerone.com [404] [Page not found · GitHub Pages] [GitHub Pages,Ruby on Rails,Varnish]
https://docs.hackerone.com [200] [HackerOne Platform Documentation] [Ruby on Rails,jsDelivr,Gatsby,React,webpack,Varnish,GitHub Pages]
https://support.hackerone.com [301,302,301,200] [HackerOne] [Cloudflare,Ruby on Rails,Ruby]
https://resources.hackerone.com [301,301,404] [Sorry, no Folders found.]

📋 Notes

• As default, httpx checks for HTTPS probe and fall-back to HTTP only if HTTPS is not reachable.
• For printing both HTTP/HTTPS results, no-fallback flag can be used.
• Custom scheme for ports can be defined, for example -ports http:443,http:80,https:8443
• vhost, http2, pipeline, ports, csp-probe, tls-probe and path are unique flag with different probes.
• Unique flags should be used for specific use cases instead of running them as default with other flags.
• When using json flag, all the information (default probes) included in the JSON output.

Thanks

httpx is made with 🖤 by the projectdiscovery team. Community contributions have made the project what it is. See the Thanks.md file for more details. Do also check out these similar awesome projects that may fit in your workflow:

Probing feature is inspired by @tomnomnom/httprobe work ❤️